Limited memory access window for refining motion vectors

Abstract

This disclosure relates to the refinement of motion vectors. [Solution] As the first step, the initial motion vector is obtained. Next, the initial motion The refinement of the vector is determined by bilateral matching in the search space. The search space is at a position given by the initial motion vector, and contains one or more fractional samples. Each fractional sample position belonging to the search space, including its location, is an integer sample within the window. Interpolation filtering using a predefined tap-size filter that evaluates only those taps And thus the window is obtained for bilateral matching in the search space It is formed by accessible integer samples.

Description

Technical Field

[0001] The present invention relates to the determination and refinement of motion vectors that may be used during video encoding and decoding.

Background Art

[0002] Current hybrid video coders use predictive encoding. Pictures in a video sequence are sub-divided into blocks of pixels, which are then encoded. Instead of encoding the block pixel-by-pixel, the entire block is predicted using spatially or temporally nearby already-encoded pixels of the block. The encoder further processes only the difference between the block and its prediction. Further processing generally includes the conversion of the block pixels to coefficients in the transform domain. Then, to form a bitstream, the coefficients may be further compressed by quantization and further compacted by entropy encoding. The bitstream further includes any signaling information that enables a decoder to decode the encoded video. For example, the signaling may include settings regarding encoder settings such as the size of the input picture, the frame rate, the quantization step indication, the prediction applied to the blocks of the picture, etc.

[0003] Temporal prediction utilizes the temporal correlation between pictures, also called frames of a video. Temporal prediction is also called inter prediction because it is a prediction that uses the dependency between different video frames (inter). Therefore, the current block, also called ​​​​​​​​​The encoded block is one or more already encoded pictures called reference pictures. Predicted from the chat. The reference picture is in the current block in the display order of the video sequence. The encoder does not necessarily mean that it is a picture that comes before the current picture. The picture may be encoded in a different encoding order. As a prediction for the current block, Blocks at the same position within the illuminated picture may be determined. Blocks at the same position are referenced. It is the block that is in the same position as the current block within the current picture. Such predictions are based on areas of motionless pictures, that is, motionless pictures for each picture. The area is accurate.

[0004] To obtain a predictor that takes motion into account, that is, a motion-compensated predictor Generally, motion estimation is used when determining the prediction for the current block. Therefore, The current block is located a distance from the position of the block at the same location given by the motion vector. Predicted by the block in the reference picture located at [location]. The decoder is the same as the current block. To enable the determination of the prediction, the motion vector is used as a signal within the bitstream. It may be ringed. By signaling the motion vectors for each block. To further reduce the signaling overhead caused by this, the motion vector The motion vector itself may be estimated. The estimation of the motion vector is performed in the spatial and / or temporal domains. This may be performed based on the motion vectors of adjacent blocks.

[0005] The current block's prediction is weighted by predictions obtained from two or more reference pictures. It may be calculated using one reference picture. The reference picture is adjacent to The picture, that is, the picture immediately before and / or immediately after the current picture in the display order. Since Kucha is most likely to be similar to the current picture, it is best to use an adjacent picture. It is also possible. However, generally, the reference picture should appear before or after the current picture in the display order. and any other picture that precedes the current picture in the bitstream (decoded order) This may be the case, for example, with occlusion and / or nonlinear motion in video content. This may be advantageous in certain cases. Therefore, the identification information of the reference picture is also bit Signaling may be performed within the trem.

[0006] A special mode of interpretation is when two reference pictures generate the prediction for the current block. This is a so-called bidirectional prediction used when creating a picture. In particular, it is used when creating a picture of each of the two reference pictures. The two predictions determined are combined into the prediction signal for the current block. Measurement is a unidirectional prediction, meaning it uses only a single reference picture, rather than a prediction that uses only a single reference picture. This may lead to a more accurate prediction of the block. A more accurate prediction is the current block and prediction. This leads to smaller differences between pixels (also called "residuals"), and those differences are more It may be encoded efficiently, that is, compressed into a shorter bitstream. Generally speaking, And, more than two reference pictures, each more than two to predict the current block It may also be used to discover many reference blocks, that is, multi-reference intersection prediction , can be applied. Therefore, the term multi-reference prediction is applicable to bidirectional prediction and more than two. Includes predictions that use the reference picture.

[0007] To provide more accurate motion estimation, the resolution of the reference picture may be increased by interpolating the samples between pixels. Fractional pixel interpolation can be performed by the weighted average of the nearest pixels. In the case of half-pixel resolution, for example, bilinear interpolation is generally used. Other fractional pixels are calculated as the average of the nearest pixels weighted by the reciprocal of the distance between the pixel and the pixel predicted to be the nearest one. The estimation of motion vectors is a computationally complex task in which the similarity is calculated between the current block and the corresponding prediction block indicated by the candidate motion vectors within the reference picture. Generally, the search area includes M×M samples of the image, and each of the sample positions of the M×M candidate positions is tested. The test includes the calculation of a measure of similarity between the reference block C of N×N and the block R at the candidate position to be tested in the search area. Due to its simplicity, the sum of absolute differences (SAD) is a measure frequently used for this purpose. In the above formula, x and y define the candidate position within the search area, while the indices i and j represent the samples within the reference block C and the candidate block R. The candidate position is, in many cases, called the displacement or offset of the block, which reflects the fact that block matching can be expressed as shifting the reference block within the search area and calculating the similarity between the overlapping part of the reference block C and the search area. To reduce the complexity, usually

[0008]

Number

[0009] ​​​​​​​​​​​​​​ The number of candidate motion vectors can be reduced by restricting the candidate motion vectors to a specific search space. It is reduced. The search space is, for example, a reference corresponding to the position of the current block in the current image. It may be defined by the number of pixels around a position in the picture and / or by the position itself. After calculating the SAD for all M candidate positions x and y, the most matching block R is: The block at the position that yields the lowest SAD corresponding to the highest similarity with reference block C. On the other hand, the candidate motion vector is formed by the motion vectors of adjacent blocks. It may also be defined by a list of complementary motion vectors.

[0010] Typically, the motion vector is determined at least partially on the encoder side and encoded as a vector. The signal is transmitted to the decoder within the output stream. However, the motion vector is transmitted to the decoder. It may also be derived in the following case. In such a case, the current block is useful in the decoder. Not available, calculate the similarity between the candidate motion vector and the block pointed to within the reference picture. It cannot be used for that purpose. Therefore, instead of the current block, it can be used to recover A template is used that is constructed from the pixels of the specified block. For example, currently Already decoded pixels adjacent to the block may be used. This has the advantage of reducing signaling, meaning that the motion vector is encoded. The signaling is derived in the same way in both the decoder and the signaler, and therefore the signaling is not necessary. It is not required. On the other hand, the accuracy of such motion estimation can be lower.

[0011] To strike a balance between accuracy and signaling overhead, the motion vector Lu's estimation involves two steps: derivation of the motion vector and refinement of the motion vector. It may be divided into: For example, the derivation of the motion vector is the motion vector from a list of candidates. The selection of a vector may include, for example, within the search space. The search may be further refined by the search. The search within the search area is performed for each candidate motion vector. Regarding this, that is, with respect to each candidate position of the block pointed to by the candidate motion vector, the cost function It is based on calculating [something].

[0012] Document JVET-D0029: Decoder-Side Motion Vector Refinement Based on Bilateral Templ ate Matching, X. Chen, J. An, J. Zheng (References: http: / / phenix.it-sudparis.eu / jvet) (Can be found on the site of / ) The first motion vector of integer pixel resolution is It is observed, and the search is performed with a resolution of half a pixel in the search space around the first motion vector. This further refines the motion vectors, demonstrating their refinement.

[0013] In order to perform motion vector refinement, at a minimum, the current block must be refined. The samples needed to perform the action, that is, the samples corresponding to the search space and the search space When template matching is performed, the sample data that may be accessed is stored in memory. You need to remember.

[0014] External memory access depends on the current hardware architecture and / or software This is an important design parameter in the implementation of the software. This is because external memory access is not utilized by internal memory. The reason lies in the fact that it slows down processing compared to its intended use. On the other hand, the internal memory on the chip is For example, it is limited by the implementation size of the chip. [Overview of the Initiative] [Means for solving the problem]

[0015] This disclosure describes how the refinement of motion vectors is on-clockwise when performed in combination with fractional interpolation. This may require an increase in the size of the internal memory or even further increases in external memory access. This is based on the observation that both options may be undesirable.

[0016] In light of the above-mentioned issues, this disclosure relates to the number of accesses to external memory and the encoded blocks. It is necessary to access the motion vector for refining the motion vector related to the q. It provides motion vector predictions that allow taking into account the number of samples.

[0017] This is the number of samples needed for template matching of integer samples. By limiting the number of samples and using a predetermined interpolation filter without requiring additional integer samples... This is achieved by enabling only the fractional positions that can be obtained.

[0018] According to an aspect of the present invention, the apparatus for determining motion vectors related to a prediction block was Then, obtain the initial motion vector and template for the prediction block and place it in the search space. The initial motion vector is refined through template matching with the aforementioned template. An apparatus is provided that includes a processing circuit configured to make a determination. The search space is an initial movement The location is given by the vector, and includes one or more fractional sample locations, and the search space is empty. Each fractional sample position within the interval evaluates only integer samples within the window. Obtained by interpolation filtering using a defined tap size filter, and the above Window is an integer accessible for template matching in the search space. It is formed by samples.

[0019] One of the advantages of determining such motion vectors is that they can be accessed from or widely referenced to external memory. Limit the number of times you access the memory / storage / cache that stores the entire picture. While doing so, it is accessible to perform motion vector refinement on the prediction block. This is the limited number of samples that need to be included.

[0020] In the example, the window is N integer samples relative to the initial motion vector of the prediction block. Defined as a column and M integer sample row, where N and M are non-zero integer values. The definition of which sample is taken for the purpose of determining and / or refining the motion vector. A simple means of specifying what should be done may be provided. It could also, for example, It may be easily configurable within a bitstream or standard.

[0021] In one embodiment, the processing circuit finds more of the search space in the most recent iteration. In a search space that is iteratively expanded in the direction given by one of the most matching positions Refinement of the initial motion vector through template matching with the aforementioned template. The window is configured to determine the number of iterations, and is defined by a predefined maximum number of iterations. It will be done.

[0022] The search space is accessed for interpolation filtering of each fractional sample within the subwindow. All integer samples being sampled are pre-defined with respect to an interpolation filter of a predefined tap size. This window may include rectangular subwindows of windows, such as those located within the window itself.

[0023] The search space may include a rectangular search subwindow of the window, and the initial movement vector The refinement of the algorithm is for interpolation filtering of each fractional sample within the search subwindow. The integer samples accessed are related to an interpolation filter with a predefined tap size. The template in the rectangular search subwindow, such as within the window It is determined by rate matching.

[0024] In one implementation, the processing circuit finds more of the search space in the most recent iteration. In a search space that is iteratively expanded in the direction given by one of the matching positions The initial motion vector is refined by template matching with the aforementioned template. It may be configured to define at least one in the search space of the most recent iteration The process terminates when the sample is outside the search subwindow.

[0025] In particular, as a specific example, the interpolation filter is used when the fractional position is horizontally relative to each integer sample. When on a horizontal line or a vertical line, K horizontal or vertical lines This is a one-dimensional filter that also evaluates integer samples.

[0026] Furthermore, for example, the search space is, - Located adjacent to the top or bottom of the subwindow, on the horizontal line of integer samples or - Adjacent to the left or right side of the subwindow, on the vertical line of integer samples Is it located there? This further includes any fractional positions outside of the subwindow.

[0027] According to another aspect of the present invention, a video image divided into prediction blocks is bitstream An encoding device is provided for encoding the above-mentioned prediction block. A device for determining motion vectors, and based on the prediction block and the determined motion vector. To encode the difference between the position and the predictor given by the prediction block, A code for generating a bitstream containing the encoded difference and initial motion vector. Includes a numbering circuit.

[0028] According to another aspect of the present invention, a video image divided into prediction blocks is bitstream A decoding device is provided for decoding from the initial motion vector and predict The position specified by the block and refined motion vector is given by the prediction block. For analyzing the encoded difference between the obtained predictor and the bitstream Analysis unit and for determining the refined motion vectors for the prediction block mentioned above The device, the analyzed difference, and the predicted block at the position specified by the refined motion vector. Reconstructing the prediction block as a function of the predictor given by the block Includes a number circuit. The function may be a sum or include a sum. The function is a cutout, circle. This may further include scaling or additional calculations.

[0029] According to another aspect of the present invention, a method for determining motion vectors with respect to a prediction block The steps include obtaining initial motion vectors and templates for the prediction blocks. In the search space, the initial movement vector is determined by template matching with the template. The step includes determining the refinement of the tor, wherein the search space is given by the initial motion vector. Fractional samples that are located in a position where they can be obtained, include one or more fractional sample positions, and belong to the search space Each position evaluates only integer samples within the window using a predefined tap size. The window is obtained by interpolation filtering using the filter, and the search Formed by integer samples accessible for template matching in space. A method is provided to do so.

[0030] For example, the window is a sequence of N integer samples relative to the initial motion vector of the prediction block. N and M are defined as integer sample rows, where N and M are non-zero integer values.

[0031] In one embodiment, the refinement of the initial motion vector is performed in the most recent iteration of the search space Iteratively expands in the direction given by one of the most matching positions. Determined by template matching with the aforementioned template in the search space, The end is defined by a predefined maximum number of iterations.

[0032] In the example implementation, the search space is interpolated fill for each fractional sample in the subwindow. All integer samples accessed for tarring are of a predefined tap size Regarding the interpolation filter, a rectangular subwindow of the window, such as the one located within the aforementioned window. Includes.

[0033] The search space may include a rectangular search subwindow of the window, and the initial movement vector The refinement of the algorithm is for interpolation filtering of each fractional sample within the search subwindow. The integer samples accessed are related to an interpolation filter with a predefined tap size. The template in the rectangular search subwindow, such as within the window It is determined by rate matching.

[0034] In one implementation, the refinement of the initial motion vector is applied to the search space in the most recent iteration. Iteratively extended in the direction given by one of the most matching positions Even if determined by template matching with the aforementioned template in the search space Often, iterations are performed when at least one sample in the search space of the most recent iteration is searched for in the sub-space. It terminates when it is outside of the ndou.

[0035] Furthermore, for example, the interpolation filter considers the fractional position to be the horizontal position of each integer sample. When on a line or a vertical line, K horizontal or vertical alignments This is a one-dimensional filter that evaluates even a few samples.

[0036] Advantageously, the search space is adjacent to the subwindow, either above or below, and consists of integer samples. Either on a horizontal line, or adjacent to the left or right side of a subwindow, integer Any fractional position outside the subwindow that lies on the vertical line of the sample It also includes.

[0037] According to another aspect of the present invention, a video image divided into prediction blocks is bitstream An encoding method for encoding is provided, and the encoding method relates to any of the methods described above. The steps involve determining the motion vector for the predicted block and the predicted block and The difference between the position based on the motion vector and the predictor given by the prediction block. The system encodes and generates a bitstream containing the encoded difference and initial motion vector. Includes steps.

[0038] According to another aspect of the present invention, a video image divided into prediction blocks is bitstream A decoding method is provided for decoding from an initial motion vector and a prediction. The position specified by the block and refined motion vector is given by the prediction block. Steps to analyze the encoded difference between the obtained predictor and the bitstream. The refined motion vector for the prediction block related to one of the methods described above is determined. The position is specified by the defined step, the analyzed difference, and the refined motion vector. Reconstruct the prediction block as a sum with the predictor given by the prediction block. Includes steps.

[0039] According to aspects of the present invention, when executed by a processor / processing circuit, the above aspects or store instructions for performing a step relating to any embodiment or combination thereof. A non-temporary computer-readable storage medium is provided.

[0040] Exemplary embodiments are described in more detail below with reference to the attached figures and drawings. [Brief explanation of the drawing]

[0041] [Figure 1] This is a block diagram illustrating an exemplary encoder structure in which motion vector derivation and refinement may be used. [Figure 2] A block diagram illustrating an exemplary decoder structure in which motion vector derivation and refinement may be used. [Figure 3] This is a schematic diagram illustrating an example of template matching suitable for bidirectional prediction. [Figure 4] This is a schematic diagram illustrating exemplary template matching suitable for unidirectional and bidirectional prediction. [Figure 5] This is a block diagram showing the steps of deriving a motion vector that operates without providing an initial motion vector refined within the bitstream. [Figure 6] A block diagram showing exemplary hardware for implementing embodiments of the present invention. [Figure 7] This is a schematic diagram showing an exemplary window with samples that should be available for access regarding the encoded block. [Figure 8] This is a schematic diagram showing an iterative search space. [Figure 9] This is a schematic diagram illustrating the horizontal expansion of the memory access window through interpolation filtering. [Figure 10] This is a schematic diagram showing the definition of the subwindow related to the fractional sample position. [Figure 11] This is a schematic diagram illustrating an example definition of a memory access window. [Figure 12] This is a schematic diagram showing exemplary positions, including fractional positions, that are allowed to form positions in the search space for refining motion vectors. [Figure 13] This is a schematic diagram illustrating an exemplary fractional position that is not permitted to form a position in the search space for refining the motion vector. [Figure 14] This flowchart illustrates a method for determining which positions in the search space are allowed to be tested by template matching for refinement of the movement vector. [Figure 15] This is a schematic diagram showing exemplary positions, including fractional positions, that are allowed to form positions in the search space for refining motion vectors. [Figure 16] This flowchart illustrates the iterative refinement process in the memory access window. [Figure 17] This flowchart illustrates the iterative refinement process in the search subwindow. [Modes for carrying out the invention]

[0042] This disclosure provides a refinement and supplementation of motion vectors to obtain fractional positions within a reference picture. This concerns adjusting the number of samples that should be accessible in order to perform the operation.

[0043] As mentioned above, external memory access is an advanced technology in today's hardware and software. It is one of the most important design considerations in architecture. For example, motion vector In the case of Torr's refinement, the estimation of motion vectors, especially when template matching is involved, It may also be used in conjunction with interpolation filtering to obtain a fractional position in the search space. The use of interpolation filtering is related to the number of samples that need to be accessed in memory. An increase may be necessary. However, this would require an increase in expensive on-chip memory or external memory. This could lead to an increase in the number of accesses to Mori, which on the one hand would delay implementation. These issues, especially on the decoder side, result in undesirable, more expensive, or slower applications. This could lead to a situation.

[0044] To prevent such situations, this disclosure imposes restrictions on external memory access. According to the embodiment, the sample that should be accessible for refining the motion vector The window is based on an unrefined motion vector, that is, on the initial motion vector. Therefore, it is defined around the indicated position. The window implements the refinement of motion vectors. Defines the maximum number of samples that need to be accessed in memory to execute a row. Therefore, the samples that should be accessible are the ones that template matching should be performed on. A sample of locations in the search space and a template and matching for all locations in the search space. It includes samples that should be searched. The latter generally exceeds the search space. The principle of simplicity Therefore, the memory access window is the encoded block (where motion vectors should be found). It may also be defined as an extension around the current block. For example, to the left of the current block And R samples from the right boundary, as well as R samples from the top and bottom boundaries, are in the window. It may be defined as follows: In other words, when the current block has a size of N × N samples, The access window has a sample size of (R+N+R)×(R+N+R), that is, (N+2R)×(N+2R). It may have. For example, R may be equal to 4. However, the current block is vertical The size N in the direction may differ from the size N in the horizontal direction, and this applies to the upward, downward, left, and right directions. The number of directional extension samples may also vary.

[0045] According to this disclosure, in order to limit the memory access window, the necessary for interpolation Memory access for refining motion vectors defined for integer samples. Fractional pixel coordinates are only applied when they are within the window, due to the refinement of the motion vector. Accessed.

[0046] Figure 1 shows the input image sample of a video stream frame or picture. Encoding includes an input and an output for generating an encoded video bitstream. -Da 100 is shown. The term “frame” in this disclosure is used as a synonym for “picture.” However, this disclosure is also applicable to fields where interlacing is applied. It should be noted that, generally speaking, a picture contains m x n pixels. This is because the image is a sun It corresponds to pull and may contain one or more color components. For brevity, the following description is for shine. This refers to pixels, which represent a sample of degrees. However, the search for motion vectors in this invention is This can be applied to any color component, including chrominance or components of the search space such as RGB. It should be noted that, on the other hand, if only the motion vector estimation for one component is performed and determined, Applying the motion vector to further (or all) components may be beneficial. .

[0047] The input blocks being encoded are not necessarily the same size. One picture may be different. It may include blocks of different sizes, and block rasters of different pictures may also be different. stomach.

[0048] In the embodiment for the explanation, the encoder 100 predicts, converts, and adds volume to the video stream. It is configured to apply subscripting and entropy coding. The coded video bit To generate a stream as output, transformation, quantization, and entropy coding are performed. , by the conversion unit 106, the quantization unit 108, and the entropy coding unit 170 Each of these will be executed.

[0049] A video stream may contain multiple frames, each frame being intra-encoded. It is divided into blocks of a certain size, which are either encoded or intercoded. For example, the first block of frames in the video stream is intraprediction unit 1 It is intra-encoded by 54. An intra-frame uses only the information within the same frame. Encoded using [a specific method], and therefore, intraframes can be decoded independently. Yes, it provides an entry point within the bitstream for random access. This is possible. Other blocks of frames in the video stream are interpretation units. It may also be inter-encoded by T144, that is, an already encoded frame (see reference F Information from the frame is used to reduce temporal redundancy, and therefore, the intercode Each block in the numbered frame is predicted from the blocks in the reference frame. The selection unit 160 determines that the frame blocks are processed by the intra-prediction unit 154. Configured to select whether to do so or to be processed by the interpretation unit 144. This mode selection unit 160 also controls the parameters of intra or interprediction. Controls the image information refresh. The 'k' may be provided within the inter-encoded frame. Furthermore, intra-encoded An intraframe containing only the selected blocks is the entry point for decryption, i.e. The decoder can start decoding without having information from the already encoded frame. They may be periodically inserted into the video sequence to provide points of interest.

[0050] The intra estimation unit 152 and the intra prediction unit 154 perform intra predictions. This is a unit. In particular, the intra-estimation unit 152 predicts based on knowledge of the original image. The mode may be derived, while the intra prediction unit 154 is responsible for differential coding. A predictor that provides predicted samples using the selected prediction mode. To perform spatial or temporal predictions, the encoded blocks are inversely quantized. Further processing may be performed by the knit 110 and the inverse conversion unit 112. Reconstruction of the block Later, the loop filtering unit 120 further improves the quality of the decoded image. It is then applied. The filtered blocks then form a reference frame, and then Then, the reference frame is stored in the decoded picture buffer 130. A decoding loop (decoder) like this is the same as the reference picture that is reconstructed on the decoder side. It offers the advantage of generating a reference frame. Therefore, on the encoder side and the decode The D side operates in the corresponding manner. The term "reconstruction" as used herein refers to the decoded residuals This refers to obtaining a reconstructed block by adding a predicted block to an existing block.

[0051] The interestimation unit 142 interencodes the current frame or picture. The input is a block and one or more reference frames from the decoded picture buffer 130. The motion estimation is performed by the interestimation unit 142, while motion compensation is performed. This is applied by the interpretation unit 144. Motion estimation is, for example, encoded The original image is also used to obtain motion vectors and reference frames based on a specific cost function. It is used to provide an initial motion vector estimate. For example, the motion estimation unit 142 provides an initial motion vector estimate. This may be done. Then, the initial motion vector is directly in vector form, or encoded by an encoder. And in the decoder, the list of candidates is constructed in the same way based on predetermined rules. Even if it is signaled within the bitstream as an index pointing to a candidate vector Good. Next, motion compensation is performed by referencing the block at the same position as the current block in the reference frame. As a transformation to a reference block within the frame, that is, by the motion vector, the current block The predictor of the block is derived. The interpretation unit 144 predicts the current block. A prediction block is output, and the prediction block minimizes the cost function. For example, cost The function is the difference between the current block being encoded and the predicted block for that current block. This may also be the case, meaning the cost function minimizes the residual block. Miniaturization, for example, applies to all candidate blocks in the current block and candidate reference picture. It is based on calculating the sum of absolute differences (SAD) between pixels (samples). However, generally speaking... any other similarity metric such as mean squared error (MSE) or structural similarity metric (SSIM) The term "standard" may be used.

[0052] However, the cost function also requires the bit necessary to encode such interblocks. The number of bits and / or the distortion resulting from such encoding may be the same. Therefore, the rate-distortion optimization procedure involves selecting motion vectors, as well as / or generally blocking Regarding the question of whether to use interpretation or intraprediction, and which It may be used to determine encoding parameters, such as whether they should be used in the settings.

[0053] The intra estimation unit 152 and the intra prediction unit 154 are currently being intra-encoded. The current frame or block of the picture, and the already reconstructed area of ​​the current frame. It takes one or more reference samples from as input. Then, the intra prediction is current The frame reference sample function records the pixels of the current block in the current frame. The intra prediction unit 154 outputs a predicted block for the current block, The prediction block is advantageous because it contains the current block to be encoded and the current block Minimize the difference between the predicted block and the residual block, that is, the predicted block minimizes the difference between the predicted block and the residual block. Minimize the residual block. Minimizing the residual block can be based, for example, on a rate-distortion optimization procedure. It is possible. In particular, the prediction block is obtained as directional interpolation of the reference sample. The direction is, Similar to the above, relating to rate-distortion optimization and / or interpretation. The scale may be determined by calculating it.

[0054] The interestimation unit 142 interencodes the current frame or picture. Block or more universal form image samples and two or more already decoded pictures It takes ya231 as input. Then, the interpretation predicts the reference image sample of the reference picture. The motion vector to the current frame describes the current image sample. The prediction unit 142 outputs one or more motion vectors for the current image sample, and the motion The reference image sample indicated by the vector is advantageously encoded Minimize the difference between the current image sample and its reference image sample, that is, The aforementioned reference image sample minimizes the residual image sample. Then, in the current block The predictor for this is provided by the interpretation unit 144 for differential coding. ru.

[0055] Next, the difference between the current block and the prediction for the current block, i.e., the residual block. k105 is converted by the conversion unit 106. The conversion coefficient 107 is converted by the quantization unit 108. It is then quantized and entropy-coded by the entropy coding unit 170. The encoded picture data 171 thus generated, that is, the encoded video video The stream consists of intra-encoded blocks and inter-encoded blocks. (and also (mode indication, motion vector indication, and / or Includes corresponding signaling (such as the direction of the prediction). The transformation unit 106 also performs Fourier analysis. A linear transformation such as the discrete cosine transform (DFT / FFT or DCT) may be applied. Such a conversion to the region results in a coefficient 107 that is generally lower in frequency. This brings the advantage of having a higher value. Therefore, the effective coefficient (such as zigzag) After scanning and quantization, the resulting sequence of values ​​is generally a sequence of zeros. It has some larger values ​​at the beginning and end. This is a more efficient encoding. This makes it possible. The quantization unit 108 reduces the resolution of the coefficient values, thereby reducing the actual complexity. The reverse compression is then performed. Next, the entropy coding unit 170 converts the coefficient value into a binary codeword. The dots are assigned and a bitstream is generated. The entropy coding unit 170 also, The signaling information (not shown in Figure 1) is encoded.

[0056] Figure 2 shows the video decoder 200. The video decoder 200 is particularly useful for decoded picture data. Intraprediction is a block prediction unit consisting of a 230 refractor, an interprediction unit 244, and a block prediction unit. Includes unit 254. The decoded picture buffer 230 contains the encoded video bitstream. Reconstructed from the frame, at least one (for unidirectional prediction) or at least two (bidirectional prediction) It is configured to store a reference frame (for direction prediction), and the reference frame is coded The current frame of the decoded video bitstream (the frame currently being decoded) is different from this. Yes. The intra-prediction unit 254 generates a predicted block, which is an estimate of the block to be decoded. It is configured to do so. The intra prediction unit 254 is the reconstructed block 215 or This is configured to generate the prediction based on the reference sample obtained from buffer 216. ru.

[0057] The decoder 200 receives the encoded video bits generated by the video encoder 100. Both the decoder 200 and the encoder 100 are configured to decode the stream, and the encoded stream It is preferable to generate the same prediction for each block that is being decrypted. The numbered picture buffer 230, the reconstructed block 215, buffer 216, and intra The features of prediction unit 254 are the decoded picture buffer 130 in Figure 1 and the reconstructed block 1 15, the buffer 116, and the intra-prediction unit 154 are similar in features.

[0058] The video decoder 200 is, for example, the inverse quantization unit 110 of the video encoder 100, inverse Inverse quantization units corresponding to conversion unit 112 and loop filtering unit 120, respectively Like the knit 210, the inverse conversion unit 212, and the loop filtering unit 220, This includes additional units also present in the DeoEncoder 100.

[0059] The entropy decoding unit 204 processes the received encoded video bitstream. The data is decoded, and accordingly, the quantized residual transformation coefficients 209 and signaling information are obtained. It is configured as follows. The quantized residual transformation coefficient 209 is inverse to generate the residual block. The residual block is supplied to the quantization unit 210 and the inverse transform unit 212. It is added to 265, and the added value is loop filtering to obtain the decoded video. The decoded video frames are supplied to the KIT 220. It is stored and can function as a decoded picture 231 for interpretation.

[0060] In general, the intra-prediction units 154 and 254 in Figures 1 and 2 need to be encoded. To generate a prediction signal about the block that needs to be encoded or decoded, Reference samples from the specified region can be used.

[0061] The entropy decoding unit 204 receives the encoded bitstream 171 as its input. It receives. Generally, the bitstream is first parsed, that is, the signaling parameters The data and residuals are extracted from the bitstream. Typically, the bitstream's syntax The codes and semantics enable encoders and decoders to be interoperable. It is defined by standards so that it may be used. Furthermore, the encoded bitstream not only contains the prediction residual, but also the motion-compensated prediction field. Additionally, motion vector indications are encoded into the bitstream and then sent to the decoder. It is then analyzed from the bitstream. Motion vector indication is the motion vector The coordinates may be given by the provided reference picture and the coordinates of the motion vector. Up to this point, encoding the complete motion vector has been considered. However, The only difference between the current motion vector and the previous motion vector in the bitstream is the sign It may be modified. This technique utilizes redundancy between the motion vectors of adjacent blocks. This makes it possible.

[0062] To efficiently encode the reference picture, use the H.265 codec (ITU-T, H265, Series H: Audiovisual and multimedia systems (High Efficient Video Coding) are each Provides a list of reference pictures to which reference frames are assigned to list indices. Next Then, the reference frame puts the corresponding assigned list index into the bitstream. By including it, it is signaled within the bitstream. Such a list is... Defined by a frame, or signaled at the beginning of a video or a set of frames. It may be done. In H.265, two defined reference pictures are called L0 and L1. It should be noted that there are two lists. The reference picture then shows the list (L0 or L1). And by indicating the index in that list related to the desired reference picture, Signaled within the stream. Providing two or more lists is better. It may have the advantage of compression. For example, L0 is bidirectionally interpreted with predicted slices. It may be used for both interpreted slices in the direction, while L1 is bi It may be used only for slices predicted in the direction. However, generally speaking, Disclosures are not limited to anything in the L0 and L1 lists.

[0063] Lists L0 and L1 may be defined and fixed in the standard. However, those lists By signaling at the beginning of the video sequence, greater flexibility in encoding / decoding is achieved. The property may be realized. Therefore, the encoder indexes a specific reference picture. Lists L0 and L1 may be constructed so that they are ordered according to the following. The lists may have the same fixed size. Generally, there may be more than two lists. The motion vector may be directly signaled by the coordinates in the reference picture. Alternatively, as specified in H.265, even if a list of candidate motion vectors is constructed Often, indices related to specific motion vectors within a list may be sent.

[0064] Typically, the motion vector of the current block is derived from the current picture or a previously encoded picture. It is correlated with the movement vector of adjacent blocks within the Kucha. This is because adjacent blocks are It is highly likely that this corresponds to the same moving object that performs similar movements, and the movement of the object changes over time. This is because the possibility of a sudden change is low. Therefore, the motion vectors within adjacent blocks are not considered. Using it as a predictor means that the size of the difference in the signaled motion vectors is Reduce. Typically, motion vector predictors (MVPs) are spatially adjacent within the reference picture. Already encoded from a block or from a block that is temporally adjacent or at the same location It is derived from the decoded motion vector. In H.264 / AVC, this is derived from three spatial factors. This is done by taking the median of each component of the adjacent motion vectors. Then, predictor signaling is not required. The temporal MVP from lock is only considered in the so-called time-direct mode of H.264 / AVC. It should be considered. The direct mode of H.264 / AVC derives motion data other than motion vectors. They are also used for this purpose. Therefore, those modes are based on the HEVC concept of block merging. Related. In HEVC, the method for implicitly deriving the MVP is to select which MVP from the list of MVPs. Explicitly signaling whether the motion vector is used for the derivation of the motion vector. It was replaced by a technology known as a competitor. HEVC's variable coding quadtree block structure 1 has several adjacent blocks with motion vectors as potential MVP candidates It can result in two blocks. Taking the adjacent block on the left as an example, in the worst case... The 64x64 Luma coding tree block is not further subdivided, and the leftmost block reaches the maximum depth. When divided, the 64x64 Luma prediction block has 16 4x4 Luma prediction blocks to the left. It is possible.

[0065] Advanced Motion Vector Prediction (AMVP) takes into account such flexible block structures. It was introduced to correct vector races. During the development of HEVC, the initial AMVP design was coded It was significantly simplified to strike a good balance between efficiency and ease of implementation. The initial design of VP was five MVPs from three different classes of predictors, namely spatial Three motion vectors from adjacent blocks, the median of three spatial predictors, This includes scaled motion vectors from temporally adjacent blocks at the same location. Furthermore, the list of predictors places the most promising movement predictor in the first position. By changing the order to place them, and minimizing signaling overhead. This was corrected by removing redundant candidates to ensure accuracy. The AMVP candidate list structure The final design of the building is based on the following two MVP candidates: a) Five spatially adjacent blocks a) Up to two spatial candidate MVPs derived from, b) neither spatial candidate MVP is available or when they are the same, one temporally derived from two blocks of the same temporal position c) No candidate MVP available, and no spatial candidate, temporal candidate, or both candidate. This includes zero motion vectors. For details on determining motion vectors, please refer to this specification. The book incorporated into this work is High Efficiency Video Coding (HEVC): Alg.com, edited by V. Sze et al. This can be found in *Orthoms and Architectures*, Springer, 2014, particularly in Chapter 5.

[0066] Further improvement in motion vector estimation without further increase in signaling overhead To do this, the motion vector derived on the encoder side and provided in the bitstream is used Further refinement may be beneficial. Refinement of motion vectors can be done with encoders or It may be performed in the decoder without their assistance. The encoder is in its decoder loop. Within, the same refinement may be used to obtain the corresponding motion vector. Refinement involves a search space that includes integer and fractional pixel positions of the reference picture. It is executed as follows. For example, a fractional pixel position is a half-pixel position or a quarter-pixel position. Or it may be a further fractional position. The fractional pixel position is interpolated by bilinear interpolation or the like. This can also be obtained from integer (full pixel) positions.

[0067] In the current block's bidirectional prediction, each of the first motion vectors in list L0 The two prediction blocks obtained using the second motion vector of list L1 are a single prediction. This is combined into the signal, resulting in a better adaptation to the original signal than unidirectional prediction. This allows for less residual information and, presumably, more efficient compression. To drip.

[0068] In the decoder, the current block is available because it has been decoded. Since it is not possible, the current block is estimated for the purpose of refining the motion vector, and has already been processed. The portion of the image that has been encoded (i.e., encoded on the encoder side and decoded on the decoder side) A template built on this will be used.

[0069] First, the estimated MV0 of the first motion vector and the estimated MV1 of the second motion vector are used by decoder 20 It is received as input at 0. On the encoder side 100, the motion vector is estimated MV0 And MV1 is determined by block matching, and / or (within the same picture or (In adjacent pictures) The current block is formed by the motion vector of the block adjacent to it. This may be obtained by searching through a list of candidates (such as a merge list). Then MV0 And MV1 is signaled in the bitstream in a way that is advantageous to the decoder. However, Generally speaking, the first decision-making stage of an encoder also reduces signaling overhead. It should be noted that this can be done through template matching, which offers the advantage of [this].

[0070] On the decoder side 200, the motion vectors MV0 and MV1 are advantageously bitstream This is obtained based on information within the system. MV0 and MV1 are either directly signaled or It is either differentially signaled and / or a list of motion vectors (mar The index of the Gillist is signaled. However, this disclosure is Bitstreet It is not limited to signaling motion vectors within the system. Rather, motion vectors are, In response to the encoder's operation, the first stage is already determined by template matching. It may be determined. The template matching in the first stage (derivation of motion vectors) is performed in the second stage. The refinement may be performed based on a different search space than the one used in the refinement stage. In particular, refinement may be performed based on a different search space. , run in a search space with higher resolution (i.e., shorter distances between search locations) It may also be used.

[0071] The indicators for the two reference pictures RefPic0 and RefPic1 that MV0 and MV1 point to. The reference picture is also provided to the decoder. The reference picture is used in the previous processing, that is, it As a result of the encoding and decoding, the decoded picture bar on the encoder and decoder side It is stored in the file. One of these reference pictures is searched for, and the motion vector Selected for refinement. Reference picture selection unit for device for determining motion vectors. The program selects the first reference picture pointed to by MV0 and the second reference picture pointed to by MV1. It is configured to do so. Following the selection, the reference picture selection unit precisely determines the motion vector. Is the first reference picture used or the second reference picture used to perform the refinement? Determine whether it will be done. In order to perform motion vector refinement, within the first reference picture The search region is defined around the candidate position indicated by the motion vector MV0. The candidate search space position is determined by performing template matching within the search space and the sum of absolute differences (SAD). By determining similarity metrics such as ), the most similar block to the template block is determined. It is analyzed to find the lock. The position in the search space is the top left corner of the template. This indicates the position where the image is placed. As mentioned above, the upper left corner is merely a convention, and generally, the center is the preferred position. Any point in the search space, such as a point in the map, can be used to represent a matching position.

[0072] According to the aforementioned document JVET-D0029, decoder-side motion vector refinement (DMVR) involves two of these. Initial motion vectors MV0 and MV1 pointing to their respective reference pictures RefPict0 and RefPict1 These initial motion vectors are taken as input. These are used to determine the search space. Furthermore, motion vectors MV0 and MV1 are used. Then, as follows, each block (of the sample) indicated by MV0 and MV1 The template is built based on A and B. Template = function ((Block A, Block B))

[0073] The function may be a sample clipping operation combined with a weighted sum for each sample. Next, the template is applied to MV0 and MV1 in each of the reference pictures 0 and 1. It is used to perform template matching within a search space determined based on the above. The cost function for determining the best template match in each search space is SAD( Template, Block candA') and block candA' is located at the position given by MV0 This is a candidate coded block pointed to by a candidate MV in the search space. Figure 3 shows the most... This shows the determination of block A' and the resulting refined motion vector MV0'. Correspondingly, as shown in Figure 3, the same template best matches Block B. This is used to find the corresponding motion vector MV1' that points to 'and block B'. In other words, the template is a block pointed to by the initial motion vectors MV0 and MV1. After being constructed based on blocks A and B, the refined motion vectors MV0' and MV1' are then used. Discovered by searching RefPic0 and RefPic1 using the template.

[0074] Motion vector derivation technology is also known as frame rate upconversion (FRUC). There are such things. Generally, the initial motion vectors MV0 and MV1 are used to describe the motion of the encoder and decoder. Bitst It may be shown within the ream. Alternatively, the initial motion vector may include one or more initial candidates. This may be obtained by providing a list of initial candidates. Then, the refined motion vector is determined, and finally, the refinement that minimizes the cost function is The selected motion vector is chosen.

[0075] The present invention is not limited to the template matching described above with reference to Figure 3. Further consideration should be given to this. Figure 4 shows an alternative template that can also be applied to unidirectional prediction. This shows the tingling. For details, see reference JVET-A1001, in particular, http: / / phenix.it-sudparis.eu / jvet / "Algorithm Description of Joint Explorer" by Jianle Chen et al., which is accessible at [location]. Section 2.4.6. Pattern matched motion in the document JVET-A1001 titled "Pattern Matched Motion Test Model 1" This can be found in "vector derivation". This template matching method is a template. The sample is determined as the sample adjacent to the current block within the current frame. Figure 1 As shown, the already rebuilt sun adjacent to the top and left border of the current block The pull can also be retrieved, and is called an "L-shaped template".

[0076] Figure 5 shows the derivation of another type of motion vector that may be used further. Motion vector The input to the derivation process is a flag indicating whether or not the derivation of the motion vector is applied. Implicitly, another input to the derivation process is an already signed (temporal or spatial) adjacent sign. This is the motion vector of the transformed / reconstructed block. It represents the movement of multiple adjacent blocks. The vector is used as a candidate for the initial search step in deriving the motion vector. The outputs of the process are, respectively, MV0' (and probably MV1' if bidirectional prediction is used). The corresponding reference picture indices are refPict0 and possibly refPict1. Next The refinement stage of the motion vector, as described above, includes template matching. Convert one (unidirectional prediction) or multiple (bidirectional prediction / multi-frame prediction) motion vectors After discovery, the predictor of the current block is weighted (for bidirectional / multi-prediction). The sample prediction, otherwise indicated by the refined MV It is constructed (by referring to the sample).

[0077] The present invention is not limited to the two template matching methods described above. For example, (References) (Also described in JVET-A1001) A third template matching called bilateral matching The tuning method can also be used to refine the motion vector, and the present invention The same applies. According to bilateral matching, within two different reference pictures The best match between two blocks along the trajectory of the current block's movement is searched for. Under the assumption of a continuous motion trajectory, motion vectors MV0 and point to two reference blocks. MV1 is the temporal distance between the current picture and the two reference pictures, i.e., TD0 and T It is proportional to D1. In bilateral matching, SAD(Block cand0', Block cand1') Cost functions such as the following may be used, and block cand0' is pointed to by MV0, Block cand1' is pointed to by MV1.

[0078] According to embodiments of the present invention, the apparatus for determining motion vectors relating to a prediction block is The device is provided and includes a processing circuit. The processing circuit generates an initial motion vector with respect to the prediction block. And obtain a template and perform a template match with the said template in the search space. The system is configured to determine the refinement of the initial motion vector by the search. The search space is Located at a position given by the initial motion vector, and including one or more fractional sample positions, Each fractional sample position in the search space evaluates only integer samples within the window. This is obtained by interpolation filtering using a predefined tap size filter, The aforementioned window is accessible for template matching in the search space. It is formed by integer samples.

[0079] The processing circuit 600 is shown in Figure 6. The processing circuit may include any hardware, and The result is achieved through any kind of programming or hardware design of both combinations. It may be implemented as follows. For example, the processing circuit may be the corresponding software that implements the steps described above. It may be formed by a single processor, such as a general-purpose processor with software. On the other hand, the processing circuit is an ASIC (Application-Specific Integrated Circuit) or a DSP (Digital Signal Processor) F Implemented by special hardware such as PGA (Field Programmable Gate Array). It's okay.

[0080] The processing circuit is connected to the above-mentioned hardware in order to perform the derivation of the motion vector described above. It may include one or more of the hardware components. The processing circuit 600 performs two functions, that is, and one initial motion vector (or multiple initial motions if bidirectional / multi-prediction is used) The process involves obtaining a vector and template (610) and refining the motion vector (620). This includes the computational logic to be implemented. These two functions may be implemented on the same hardware. , or initial motion vector and template determination unit 610 and motion vector precision The process may be performed by separate hardware units, such as the refinement unit 620. Circuit 600 is capable of communicating with external memory 650 where the reconstructed reference picture samples are stored. They may be connected in this way. Furthermore, the processing circuit 600 is transferred from the external memory and simultaneously processes Samples within a window used to determine the motion vectors for the block being analyzed. It may further include an internal memory 640 for buffering the data. The processing circuit is an integrated circuit and This may be implemented on a single chip.

[0081] The processing circuit is an encoder and / or decoder as described with reference to Figures 1 and 2. It should be noted that further functionality may be implemented. Internal memory is used for cache or load. On-chip memory such as chip memory can also be used. Chip memory improves the speed of calculations. To achieve this, it is advantageous to implement it on the encoder / decoder chip. Because the size is limited, on-chip memory is usually small. On the other hand, external memory is external Mori can be very large in size, but access to external memory is more difficult. It consumes a lot of energy, and access is much slower. Usually, all the necessary information is available. The data is retrieved from external memory to on-chip memory before the calculation is performed. In the worst case... External memory access (or the need to be provisioned when designing the memory bus) (A certain bandwidth) is used between external memory and the chip during decoding of a frame or encoding unit. This represents the maximum possible amount of memory transfer between them. Memory (especially external memory) is usually predefined. It can only be accessed in block units. In other words, generally speaking, a single pixel can only be accessed in block units. It cannot be accessed, and instead, the smallest unit (for example, 8x8) is not accessed. It must be. Large on-chip memory increases costs, so the on-chip memory size Size is also an important design consideration.

[0082] In other words, the above-mentioned device consists of an internal memory incorporated into an integrated circuit and the window Memory access to fetch an integer sample from external memory to internal memory. The integrated circuit may further include units (interfaces).

[0083] The term "predicted block" used above refers to the current block that is to be predicted. Blocks divide an image into smaller units (for example, coding tree units CTUs hierarchically). (By dividing) it into smaller blocks of equal or different sizes. A block in the image that may be obtained by the following: A square or more broadly rectangular is the current Since these are typical shapes used in decoders, the blocks are It may be in the shape of the block. However, this disclosure does not apply to any size / shape of the block. Not limited.

[0084] The device including the processing circuit is an encoder, or decoder, or even such an encoder A device including a coder or decoder, for example, a recording device and / or playback device A chair would also be acceptable.

[0085] The fractional sample position is determined by reconstructing the encoded reference picture as shown in Figure 1. This is the position between the actual picture sample positions obtained by and . Therefore, fractional The position must be obtained by interpolation based on the nearest integer position, according to H.265. For details on the exemplary interpolation filtering used, see the book High Efficiency by V. Sze et al. In the section "5.3 Fractional Sample Interpolation" of Video Coding (HEVC), Springer, 2014: It can be found.

[0086] Typically, interpolation filtering generates different fractional Pell (sample) positions. Apply a filter. For example, in the H.265 video compression standard, the following 1D separable A filter is applied to generate 1 / 4 PEL and half PEL positions. [Table 1]

[0087] As can be seen from the table above, interpolation filtering is applied to the taps of the filter (the number of coefficients in the table). We need several samples around the corresponding fractional Pell position. To generate the half-Pell position... Using the exemplary filter above, four integer samples are taken from the left / top and right / bottom. However, this is required. The length of the interpolation filter is (7 taps) with respect to the 1 / 4 Pel sample position. Please note that this differs from the length of the half-per sample position (which is 8 taps).

[0088] In some embodiments of the present invention, an interpolation filter with a predefined tap size is used before Only integer samples accessible for template matching in the registration search space are evaluated, by the integer samples given within the window. The window may contain many more samples than those actually used in the calculation of a particular prediction block. This is usually because the refinement search operation is carried out using a fast search method (in contrast to an exhaustive search method), according to which part of the samples is not evaluated as the progressive progress of the search operation proceeds. As a result, the number of iterations of template matching and the samples used for the calculation of the refinement search operation may vary for each prediction block. This disclosure sets an upper limit (boundary of the region) for integer samples that can be used during the refinement search operation, considering that interpolation filtering needs to be applied to fractional sample positions. This corresponds to the term "integer samples accessible for template matching". Which samples are actually accessed depends on the method of forming the search space, as illustrated below. Figure 7 shows corresponding samples of an encoded block (prediction block) and a window. The samples shown in Figure 7 are reference picture samples, where it is noted that the encoded block is actually a block in the current frame within the reference picture where the motion vector should be derived, corresponding in size and position to the current block. Therefore, the encoded block in Figure 7 is actually the block at the same position as the block where the predictor is searched. However, for simplicity, this block is hereinafter referred to as the "encoded block".

[0089]

[0090] ​ It is called [name].

[0091] In this example, the unrefined motion vector MV0 points to an integer sample position. (The initial motion vector can point to the fractional sample position, making it easy to illustrate.) (To achieve this, integer sample positions are selected). The granularity of the search for refining motion vectors is 1 Since it is an integer sample, that is, the starting point is an integer sample, only integer sample points are included. The search is performed. In this example, the search is carried out in a gradually evolving search space. This is because the search space aligns with the best direction in terms of the cost function for already tested positions. This means that the process can be improved with each iteration by adding new search locations.

[0092] Such a method is shown in a simplified form in Figure 8. In Figure 8, the initial motion vector is, It was pointing to the center point 810. The search space was gradually constructed around the position of the initial motion vector. In the first step, above and below the position 810 indicated by the initial motion vector. , pointed to by the four positions directly adjacent to the left and right, as well as the initial motion vector. Position 810 is tested. It yields the lowest cost function among the five points tested. Based on the direction, further locations to be tested are added to the search space. In this example... The lowest cost function can be found at the point on the right, and therefore, the search The space was extended horizontally to the right by three additional points in the second step. In step , the lowest cost function is (relative to the lowest cost point in step 1) (and) it is possible to see at the point on the right, and as a result, only three points can be found in the horizontal direction to the right. Further expand the search space. In the third step, the lowest cost function is again, Observed at the point to the right of the lowest cost point of pp 2, and three more horizontally to the right This results in expanding the search space by only one point. As shown in the example in Figure 8, three further steps are taken. The search is performed in the following order: in the direction, upward, and to the right. In the example, (from the 5 search points) A diamond-shaped pattern is used for each iteration, and the search space is changed at each step. It is extended to fill in the missing search points.

[0093] In each iteration of determining the search space, the search space is increased by one or more integer sample positions. It is acceptable if it becomes 4. Now, let's return to Figure 7 in the example where the maximum number of search iterations is 4. Since iterations are possible, if the gradual development of the search space is directed to the left, the left-facing side All of the samples need to be retrieved from memory in order to perform the search operation. An extension of 4 samples above is required. Therefore, the search space is refined to M Since V can move in any direction, it can be extended in both directions (left and right and up and down), and The software implementation is such that all samples that may be needed before applying refinement exploration are outside It requires fetching from memory. In addition, the template matches the size of the coded block (prediction block) with the template. Since the checking needs access to some of those samples, four more samples Extension by pull is necessary. Furthermore, hardware implementations generally involve irregular shapes. Since it cannot be fetched (it is easier to access the rectangle), the corner sample ( For example, (upper right) must also be fetched from memory.

[0094] The above-described iterative development of the search space is merely exemplary, and the rules and the number of points for expanding the search space in each iteration may be different, that is, they may be specified in different ways. This is noted.

[0095] Also, FIG. 8 shows a scenario that may occur due to the above-described rules for external memory access. The number of samples fetched from the external memory is much larger than the samples actually used in the calculation step. Assuming that the template here is of the size of 1 sample (for simplicity), the white circles represent the samples fetched from the external memory and the filled samples actually used. However, when the current block starts to be processed, since the samples actually required are not yet known, such redundancy is necessary if the number of accesses to the external memory should be kept low.

[0096] It is noted that the search space may also be defined in different ways, for example, as a stable shape at a position indicated by an initial motion vector. The shape may be any shape such as a square, rectangle, diamond, etc.

[0097] FIG. 9 shows an example where the search space may also include fractional samples. In FIGS. 7 and 8, the search for the motion vector is performed for integer samples, resulting in positions indicated by the larger solid dots included in the access window. At this time, if the search is performed for samples having a half-pel resolution (smaller solid dots), interpolation filtering Assuming that Ruta is symmetrical and has 8 taps, the fractional sample shown on the right produces To achieve this, three additional columns of the sample also need to be retrieved from memory. Furthermore, the search operation is symmetrical so that fractional pixels may be on the left side of the window. The same applies to the left side, due to the fact that it can move repeatedly to the left and right. It must be (an expansion of 3 rows of pixels).

[0098] As a result, interpolation filtering causes samples that need to be retrieved from memory. The number of circles has been further increased, and now also includes dotted circles representing positions added due to fractional interpolation. This is indicated by a dashed line. Similarly, if we allow the vertical semi-perpendicular position to be explored, The sample window accessed from Mori is also expanded vertically, both upwards and downwards. It is necessary to do so (not shown in the example in Figure 9).

[0099] The memory access window is a motion vector related to the prediction block (encoded block). This includes all the samples that need to be taken out of memory in order to perform the search. Defined as a rectangle. The memory access window is the actual sample required. In addition to including the remaining data, which may be accessed during the motion vector search operation, This includes all of the examples. In the example in Figure 9, the search for the motion vector moved to the right. However, Furthermore, it can also be moved to the left, which was not previously known. Therefore, in order to avoid repeatedly accessing external memory, a memory access window (also The access window is all samples that can be accessed by each process. Includes ru.

[0100] Figure 10 shows the memory access window for refining the motion vector. Center point 10 10 is an unrefined input motion vector (from a bitstream or as described above). This is obtained either by template matching or candidate testing that has already been performed. This is the position indicated by the initial motion vector. Add fractional positions to the search space. To avoid further increases in window size, which would result from this, the precision of the motion vector Elaboration is performed according to the following rules. A) The window for memory access for refinement is the initial motion vector that has not been refined. It is defined around the coordinates of Torr (i.e., the position indicated by the initial motion vector). The window refines the motion vector through template matching in the search space. Determine the maximum number of pixel samples that need to be accessed from memory to perform the operation. do. 1. In this example, for simplicity, the current block size (encoded block) is used. The size is 1x1 for the sample, but the current block size can be larger. Yes, and generally larger. 2. The memory access window is 4 samples from the left / right and top / bottom as shown in the diagram. It is defined as an extension around coding blocks, such as 4 samples. B) The fractional pixel coordinates are the window of memory access where the samples needed for interpolation are located. Accessed only when it is present for MV refinement.

[0101] Requirement B is the number of samples required for refining the motion vector for integer samples. This rule ensures that the defined access window is not further expanded. The actual fractional samples that are accessible are given by the size and shape of the interpolation filter. Therefore, assuming a 6-tap interpolation filter in Figure 10, the dotted line represents, This shows the region where fractional samples may be present. However, as shown in Figure 12, It should be noted that fractional pixel positions may be acceptable. In particular, access Vertical filtering only or horizontal filtering that does not require extension beyond the nd Fractional positions that require a fractional position may continue to be used. Therefore, the fractional positions are shown in Figure 10. Restricting the sample window to a given fraction is an overrestriction for some applications. It may be limited.

[0102] In other words, according to the embodiment, the memory access window is for integer samples. Includes all samples accessible by refining the motion vectors performed. , the motion vector refinement performed on integer samples does not make it accessible It does not contain a sample. Therefore, fractional samples are used for refining the motion vector. In this case, those fractional samples are obtained in a way that does not require additional samples.

[0103] In the example in Figure 10, this relates to the shape and size of the predefined interpolation filter. By allowing only fractional samples that do not require samples outside the access window, It is realized. The dotted fraction sample window expands within the access window. T is If it is the number of taps in the interpolation filter, then the window boundary of the fractional sample is, From the access window boundary sample, an integer sample within a distance of 1020 (floor(T / 2)-1) is used. It is defined as follows. In particular, in this example, T=6 and T / 2=3, and the access window The distance from the boundary sample to the fractional window sample is T / 2-1=2 integer samples. be.

[0104] However, it should be noted that this determination of the fractional window is merely an example. Generally speaking, The windows may have different shapes and sizes. Vertical and horizontal interpolation is This may be done by filters of different sizes. Furthermore, some fractional positions are generally Both vertical and horizontal filters are required, which may be separable or inseparable. That is also acceptable.

[0105] Alternatively, the interpolation filter is modified with respect to fractional positions outside the dotted window in Figure 10. (For example, the number of taps in at least one direction is reduced.) However, the purpose of the implementation For reasons related to the quality of interpolation, such solutions may not be particularly attractive.

[0106] The window for memory access may be defined in various ways. Figure 11 shows the note. The reaccess window is at the position of the coding block given by the initial motion vector. Examples are shown where extensions are defined as left / right or top / bottom extensions of the corresponding (encoded) block. The amount may depend on the size and shape of the encoded or predicted block. In Figure 11, The extension is the length of 4 samples in each direction (up, down, left, right). However, EXT is (vertical direction) Block size and / or search space (may have different sizes in the directional and horizontal directions) It should be noted that the values ​​may vary in different directions depending on the shape and size of the space between them. It is intended.

[0107] For example, according to the example, the window is N-shaped relative to the initial motion vector of the prediction block. Defined as a number of sample columns and M integer sample rows, where at least one of N and M is non It is an integer value of zero (both are integers, but one of N and M can be zero). See Figure 11. In this case, N and M are shown, but they have the same size. As mentioned above, N and M are different. It may have a size such that N and M are integers, and at least one of them is , non-zero. Retrieve parameters N and M, as well as the template's form and size. Then, the size of the access window may be determined. In particular, if the template is T1 row and T2 If there are columns, the size of the memory access window is (N+T2+N) rows and (M+T1+M) columns. This can be calculated as follows: the search moves N samples to the left or right, and 2N samples horizontally. This is because it can cause a pull, resulting in M ​​samples advancing upwards or downwards, and 2M samples being generated vertically. .

[0108] On the other hand, certain methods for constructing the search space, such as the methods described with reference to Figures 7 and 8, Regarding this, the memory access window is later converted to the maximum amount of left, right, up, and down displacement. The maximum number of iterations of refinement (iterations of building the search space) that can be performed and (that can be achieved in each iteration) It can be defined by the step size of the iteration (in terms of the distance of the largest possible sample). Therefore, the memory access window is defined as the maximum displacement in each direction. For example, four iterations in which each iteration may advance the search space by up to one integer sample position are called EXT=4. It will be the result.

[0109] In other words, according to this example, the processing circuit is more efficient in the search space in the most recent iteration. Iteratively expand in the direction given by one (or more) of the most matching positions. Initial movement by template matching with the template in the search space being stretched The window is configured to determine the refinement of the vector, and the window is a predefined minimum of the iterations. Defined by the number of times

[0110] In general, this disclosure is not limited to any particular shape, form, or type of determination of the exploration space. It should be noted that this is not fixed. In another example, the search space is each fraction in the subwindow All integer samples accessed for sample interpolation filtering are predefined. With respect to the interpolation filter of the tap size, the window is located within the window. This is a rectangular subwindow. A similar example has already been discussed above, with reference to Figure 10. In this case, the search space consists of 9x9 integer samples, and the position of the initial motion vector is the center of the sample. Given by fractional samples within a region formed by 5x5 integer samples. It is possible.

[0111] The definition of the memory access window is probably based on the corresponding signaling parameters. To include in the bitstream (for example, parameter EXT or parameters N and M) It may be reasonable. However, the size of the memory access window is defined by the standard. or other (such as template size, prediction block size, image resolution, etc.) It may be possible to derive it based on the coding parameters.

[0112] Figure 12 shows examples with different fractional pixel positions. For this example, the template The size is (for simplicity) 1x1 sample, and a 6-tap interpolation filter is applied to each half-pill position. It is assumed that this is used in relation to. In the example, the search coordinates being searched are those search coordinates. A number indicating the order in which targets are examined, i.e., the order in which template matching searches are performed. Represented by letters 1-6. Positions 1 and 2 are semi-perfect positions (i.e., their positions are , is located midway between two integer sample positions, and 'per' is an abbreviation for pixel, and pixel and The term "sample" is used in this application as interchangeable with the term "sample". Places 1 and 2 are necessary extensions for interpolation filtering within the memory access window. It can be investigated as it is located there (3 integer samples diagonally to the upper left and 3 integer samples to the lower right from position 1). (3 integer samples to the right and 3 integer samples to the left of position 2). Fractional sample point 1 is horizontal and vertical. It is noted that both perpendicular and perpendicular extensions are required, and that both must be within the window. Position 2 requires only extension to the right and left.

[0113] Positions 3, 4, and 5 are integer sample (integer per) positions. These positions are interpolated. It can be explored because no extension is needed for filtering. The fractional (half-per) sample 6 is also vertical. Only perpendicular extensions (3 integer positions up and down, respectively) are needed, and that is still win It's accessible because it's within the DOUBLE. Horizontal expansion is not necessary. Therefore, the above implementation is sufficient. In this case, fractional sample points are obtained when the required interpolation extension is within the memory access window. Access is restricted to only those users.

[0114] In other words, as shown in the example, the interpolation filter determines the fractional position of each sample of water. When on a horizontal or vertical line, K horizontal or vertical This is a one-dimensional filter that evaluates any integer sample.

[0115] The line between two adjacent integer positions in the horizontal or vertical directions. One-dimensional fractional positions (for example, positions 2 and 6 in Figure 12) are either horizontal or vertical. It requires extension only for interpolation, that is, each horizontal or vertical interpolation Filtering should be done only by inter-filters. As many fractional positions as possible In order to be usable, in addition to the fractional positions allowed in the example of Figure 10, the ones shown in Figure 12 It may be advantageous to add further one-dimensional positions, such as position 6.

[0116] In other words, the search space is - Located adjacent to the top or bottom of the subwindow, on the horizontal line of integer samples or - Adjacent to the left or right side of the subwindow, on the vertical line of integer samples Is it located there? Any fraction position outside the fraction subwindow (see the dotted window in Figure 10) It also includes.

[0117] Some fractional samples are further integer samples in a given horizontal or vertical direction. It should be noted that this may be necessary. This is because the size of the predefined filter is The conditions for generating that position in each direction may differ.

[0118] Figure 13 shows examples of fractional half-pel positions 1 and 2 that cannot be accessed. The position is outside the subwindow shown in Figure 10. In this example, 6 taps interpolation is used. It is assumed that a filter is used for half-pel positions. Half-pel search points 1 and 2 are water Horizontal or vertical interpolation filtering requires samples that are outside the window. Therefore, it is not allowed to be searched. To filter position 1, horizontal f Required by the filter, a vertical filter is used to filter position 2 The required integer sample positions are shown by dashed lines in Figure 13. As shown, the dashed circles correspond to integer positions that are not within the memory access window.

[0119] In the example above, the memory access window is during the motion vector refinement process. (Even for interpolation filtering) to prevent samples outside the window from being accessed. It was defined as follows. In other words, the memory access window is the refinement of the motion vector and The smallest win containing the sample that may need to be accessed for interpolation. It is. Furthermore, the memory access window moves based on the integer sample position. Designed according to the samples needed for the refinement of the clef. Then, further integer positions Only fractional positions that do not require further expansion of such access windows are permitted. It will be done.

[0120] It should be noted that the above example is given with respect to half-Pell interpolation. However, this disclosure is not It is not limited to these. Generally, any fractional position such as 1 / 4, 1 / 8, etc., can be used, that is, opposite Interpolation may be performed using a corresponding interpolation filter.

[0121] The processing circuit described in relation to Figure 6 is an encoder as shown in Figures 1 and 2. It may be used in a bil / decoder.

[0122] In particular, a code for encoding a video image divided into prediction blocks into a bitstream A coding device may be provided, and the coding device includes a processing circuit in the above-mentioned prediction block. A device for determining the motion vector, and a prediction block and the determined motion vector Encode the difference between the predicted block and the position specified by the prediction block. To do this, and to generate a bitstream containing encoded difference and initial motion vectors It includes an encoding circuit for achieving this.

[0123] Refer to Figure 1 for further units and functions of the encoder described above, as well as the processing circuit It may be provided or implemented in [location].

[0124] In response, the video image, which has been divided into prediction blocks, is decoded from the bitstream. A decoding device may be provided for this purpose, and the decoding device uses an initial motion vector and a predicted vector. The position is given by the prediction block, specified by the lock and refined motion vector. The encoded difference between the predictor and the bitstream is the solution for analyzing the bitstream. The refined motion vector for the prediction block described above, including the analysis unit and processing circuit. The device for determining the position, the analyzed difference, and the prediction based on the refined motion vector. To reconstruct the predicted block as a sum with the predictor given by the block It includes a decoding circuit. For example, the predictor is based on the position of the refined motion vector. It may be given directly. However, (filtering, cropping, further refinement, etc.) You can get the motion vector of the current prediction block and further modify the motion vector. There may be a processing step that results in the following.

[0125] Refer to Figure 2; further units and functions of the decoder described above are also included in the processing circuit. It may be provided or implemented.

[0126] Furthermore, embodiments of the present invention include a processing circuit for performing motion vector refinement. The device was described in terms of the apparatus. However, this disclosure is not limited thereto, and the above-mentioned processing circuit may be implemented in practice. A corresponding method is also provided, which includes processing steps corresponding to the processing steps configured for a row. ru.

[0127] In particular, the step of obtaining the initial motion vector and template for the prediction block. In the search space, the initial movement vector is determined by template matching with the template. The step includes determining the refinement of the tor, wherein the search space is given by the initial motion vector. Fractional samples that are located in a position where they can be obtained, include one or more fractional sample positions, and belong to the search space Each position evaluates only integer samples within the window using a predefined tap size. The window is obtained by interpolation filtering using the filter, and the search Formed by integer samples accessible for template matching in space. A method is provided for determining the motion vectors related to the predicted blocks.

[0128] The taps correspond to the filter coefficients. The tap size corresponds to the filter order. Here And the filter is assumed to be a linear filter. In some examples the filter is It may be symmetric, that is, it may have symmetric coefficients. However, this disclosure does not provide a symmetric fill A filter, or a linear filter, or not limited to any type of filter. Generally speaking, The numerical position may be obtained in any way based on adjacent samples.

[0129] Furthermore, for encoding the video image, which has been divided into prediction blocks, into a bitstream An encoding method is provided, and the encoding method relates to a prediction block relating to any of the above methods. The steps involve determining the motion vector and then, based on the predicted block and the determined motion vector... The difference between the predictor given by the prediction block at a given position is encoded, and the encoded The process includes the step of generating a bitstream containing the difference and initial motion vectors.

[0130] The encoding method may further include the steps described with reference to the function of the block in Figure 1. good.

[0131] Furthermore, to decode the video image, which has been divided into prediction blocks, from the bitstream... A decoding method is provided, and the decoding method is refined with an initial motion vector and a predicted block. The predictor block gives the position specified by the motion vector The steps involve analyzing the encoded difference between the two from the bitstream, and the method described above. A step of determining a refined motion vector for any of the prediction blocks, The predicted block is placed at the position specified by the analyzed difference and the refined motion vector. This includes the step of reconstructing the prediction block as a sum with the given predictor.

[0132] The decryption method may further include the steps described with reference to the function of the block in Figure 2. stomach.

[0133] However, it should be noted that Figures 1 and 2 should not limit this disclosure. To give non-limiting examples of implementations of the present invention in existing encoders and / or decoders It's not enough.

[0134] Figure 14 shows an exemplary implementation of the method according to the embodiment. Function InterpolationFilterLength( C) The necessary additional horizontal and vertical saturation to apply interpolation filtering. Returns the number of samples. The required number of samples is: - Whether the search coordinates are integer Pell positions, half Pell positions, or quarter Pell positions Is there any? - A horizontal interpolation filter needs to be applied to generate the search coordinate samples. Should a vertical interpolation filter be applied, or both interpolation filters be applied? Is it okay for the "ta" to be applied? It varies depending on the situation.

[0135] The method begins in step 1430. In particular, the position of the initial motion vector is tested. This is the position C(x,y) in the first search space. The function InterpolationFilterLength(C) is calculated at this position. Regarding this, the number of horizontal and vertical samples to apply interpolation filtering. It returns the sum of C(x,y) and InterpolationFilterLength(C(x,y)) by MAX(max_x, max_y). If the access window size is exceeded, then the position is part of the search space. It is not used as such. Instead, the next search coordinates C(x,y) are used to test the step In step 1440, the selected (for example, x or y or so depending on the order in which the search is performed) (Both are incremented). The location tested in step 1430 is accessed. If it is not necessary to cross the window, in step 1410, the precision of the motion vector As part of the refinement process, template matching is performed with respect to its position C(x,y). Next And, whether there are still positions in the search space left for template matching, Tested in step 1420. If none remain, refinement is terminated. If present, the following coordinates are selected in step 1440, and the condition in step 1430 is met. The position is evaluated. These steps are repeated.

[0136] As already explained above, (perhaps as shown in Figure 12, further fractional samples) Alternative possibilities for defining allowed fractional positions (such as the window in Figure 10, which is extended by) A property exists. Based on that, we determine whether position C(x,y) belongs to a window where it is allowed. A simpler condition for evaluation may be formulated in step 1430. For example. The search is performed using a memory access window, such as the memory access window shown in Figure 10. It is limited to integer and fractional search points within a fractional search window encapsulated within it.

[0137] The search space may include a rectangular search subwindow of the window, and the initial movement vector The refinement of the algorithm is for interpolation filtering of each fractional sample within the search subwindow. The integer samples accessed are related to an interpolation filter with a predefined tap size. The template in the rectangular search subwindow, such as within the window It may be determined by rate matching.

[0138] More precisely, the allowed integer and fractional positions are the unrefined initial motion vectors. It may be determined based on the distance of the search point from a point related to it. In particular, if it is not refined The initial motion vector is at a distance P in the x direction from the initial motion vector and the unrefined initial motion vector A search point located at a distance R in the y-direction from is allowed. In particular, the memory access window is precise. Defined for a point in a reference picture pointed to by an unprocessed initial motion vector. This is illustrated in Figure 15, where the center of the memory access window is , is a half-point between two integer points along the horizontal axis. In the figure, memory access win The doh is defined as an extension in the x-direction (N) and an extension in the y-direction (M), where N and M are integers. , at least one of them is non-zero. According to a particular implementation, subwindows, In other words, the subwindow (or search window) for the search point is a memory access window. Everything defined within the ndou that is allowed to be searched by template matching Defined as x-direction extensions (P) and y-direction extensions (R) that include integer Pell and fractional Pell search points. In Figure 15, numbers P and R are equal to 1 for illustrative purposes. Therefore, the center If the distance of the search point to the target exceeds P or R in the x and y directions respectively, the search point It is not included in the memory access window. In this particular example, the fractional search point is Since it requires access to additive samples, the memory access defined by N and M The Seth window encapsulates or contains the secondary window. P and R are horizontal. It is a real number that represents the distance between two integer sample points in the directional and vertical directions. For example, if P and R are defined as P=1.5 and R=1.5, then the initial center Search points (integer or fraction) located 1.5 in the x-direction from the point are subject to the motion vector refinement process. It is permitted to explore by [this method]. Furthermore, in Figure 15, the left-right expansion and up-down expansion are permitted. Directional extensions are defined to be equal, and those extensions are generally not required to be equal. Generally speaking, all four extensions—left, up, right, and down—can be defined independently.

[0139] Figure 16 shows a possible implementation of an iterative refinement process in a memory access window. As shown in Figure 16, the refinement search is applied iteratively, meaning that only a maximum of K search points are explored. However, these are explored in each iteration. Firstly, K search points are explored around the initial starting point or the previous point. The best search point selected as a result of the iteration is determined around (1610). Secondly, the K searches If all points are within the memory access window (1620), the refinement search operation is K This applies to the number of search points. However, if any one of the K points is within the memory access window If it is outside, the search iteration is then terminated. Thirdly, the condition for the maximum number of search iterations. However, it was checked (1630), and if the current iteration exceeds the maximum number of iterations allowed for the search, the iteration is , and terminate. Finally, refinement search is applied to K search points (1640), and the K search points The best point among them is selected according to the matching cost function (1650). The application increases the number of iterations by 1. After 1650, the process is best at 161, which is the output of 1650. It is repeated using 0 as input. According to Figure 16, the search points explored during one iteration. The number K that determines the number defines the maximum number of search points in each iteration. The number of points K and the pattern may vary depending on the search strategy used. For example, According to the exemplary search strategy, K=5 search points (center, left, right, down, and up) are found in the first iteration. The search may be conducted in this manner, while K = 3 points (center, bottom right, bottom left) may be searched in the second iteration. It may be searched. The number K is a number smaller than the total number of search points in the memory access window. be.

[0140] According to a specific implementation shown in Figure 16, a single search point among the K search points of an iteration is If the memory access window is closed, the iteration of the search is terminated. There may be other points among the K search points in the window, but the iteration will end. Therefore, those points within the memory access window are also not explored. The goal is to reduce the number of iterations of the search, while simultaneously avoiding searches that fall outside the memory access window. The goal is to ensure that no points are explored.

[0141] 1620 allows us to check whether the search point is within the memory access window. This can be done by checking the distance of the search points to the initial starting point. Therefore, the distance If the x-component of the distance is greater than N, or if the y-component of the distance is greater than M, the search point is memo. It is determined to be outside the reaccess window. Generally, N and M are integers, and their At least one of our values ​​is greater than 0.

[0142] Refining the initial motion vector leads to more and more matches in the search space in the most recent iteration. The previous in the search space which is iteratively extended in the direction given by one of the positions This may be determined by template matching with the specified template, and the iteration is the most recent When at least one sample in the search space of a new iteration is outside the search subwindow It will be terminated.

[0143] Figure 17 shows possible iteration methods that can be applied to the search subwindow as shown in Figure 15. This is the flowchart shown. The flowchart in Figure 17 is the same as the flowchart in Figure 16, except for step 1720. That is correct. According to step 1720 in Figure 17, perform an elaboration search for K search points. The decision involves checking whether all K points are within the subwindow for the search points. This is determined by the following. In other words, as shown in Figure 15, the initial, unrefined version The distance to any one of the K search points to the center point indicated by the motion vector. If is greater than P or R, then the condition shown in 1720 evaluates to false, and the iteration ends. It is completed.

[0144] As mentioned above, the determination of the motion vector with memory window limitations is based on the video signal (moving image). This can be done as part of the encoding and / or decoding. However, the determination of the motion vector is Furthermore, its use is not limited to encoding / decoding, but also includes motion detection and motion analysis. It may also be used for other purposes in image processing, such as those mentioned above.

[0145] The determination of motion vectors may be implemented as a device. Such a device may use software. A may be a combination of A and hardware. For example, the determination of motion vectors is a general Processor for use, or digital signal processor (DSP), or field programmable It may be implemented by a chip such as a gate array (FPGA). However, the present invention is prog The present invention is not limited to implementations on ramming hardware. The present invention is an application-specific integrated circuit. It may be implemented in an ASIC, or by a combination of the hardware components described above.

[0146] The determination of motion vectors is also done by program instructions stored in a computer-readable medium. It may be implemented in the manner described above. When the program is executed, the computer will perform the steps described above. Run the program. Computer-readable media include DVDs, CDs, USB (flash) drives, and hard drives. The program is written to disks, server storage accessible via the network, etc. It can be any medium that can be imagined.

[0147] Encoders and / or decoders are used in television receivers, set-top boxes, PCs, and other devices. Various devices including tablets and smartphones, that is, any recording, encoding, code It may be implemented in a number conversion, decoding, or playback device. Encoder and / or Deco -da implements the steps of the method and stores in an electronic device such as the electronic device described above. / Software or app that runs on a processor included in an electronic device That's fine.

[0148] In summary, this disclosure relates to the refinement of motion vectors. As the first step, prediction Initial motion vectors and templates for the block are obtained. Then, the initial motion The refinement of the vector is followed by template matching with the template in the search space. Determined by . The search space is at the position given by the initial motion vector, 1 The search space contains more than one fractional sample location, and each fractional sample location belonging to the search space is a win A predefined tap-size filter is used to evaluate only integer samples within the dow. The window is obtained by filtering and is a template in the search space. It is formed by integer samples that are accessible for matching. [Explanation of Symbols]

[0149] 100 encoders 105 Residual Block 106 Conversion Unit 107 Conversion coefficients 108 Quantization Units 110 Inverse Quantization Unit 112 Inverse Conversion Unit 115 Reconstructed Blocks 116 buffers 120 Loop Filtering Unit 130 Decoded picture buffers 142 Interest Units 144 Interpretation Units 152 Intra Estimation Units 154 Intra Prediction Units 160 Mode Selection Unit 170 Entropy Coding Units 171 Encoded picture data, encoded bitstream 200 Decoders 204 Entropy Decoding Unit 209 Quantized residual transformation coefficients 210 Inverse Quantization Unit 212 Inverse Conversion Unit 215 Reconstructed Blocks 216 buffers 220 Loop Filtering Unit 230 Decoded picture buffers 231 Decrypted picture 244 Interpretation Units 254 Intra Prediction Units 265 Prediction Blocks 600 processing circuits 610 Initial motion vector and template determination unit 620 Motion Vector Refinement Unit 640 internal memory 650 External Memory 810 center point, position 1010 center point 1020 distance

Claims

1. A device for determining motion vectors related to prediction blocks, The initial motion vector and template for the aforementioned prediction block are obtained, In the search space, the initial movement is determined by template matching with the template. Includes a processing circuit configured to determine the refinement of the vector, The search space is located at a position given by the initial motion vector, and one or more minutes The search space includes several sample positions, and each of the fractional sample positions belonging to the search space is a window Interpolation using a predefined tap-size filter that evaluates only integer samples within the given range. The window obtained by filtering is the template in the search space A device formed by integer samples accessible for matching.

2. The window is a sequence of N integer samples relative to the initial motion vector of the prediction block. Defined as M integer sample rows, where at least one of N and M is a non-zero integer value. The apparatus according to claim 1.

3. The processing circuit finds more of the best matching positions in the search space in the most recent iteration. The temp in the search space is iteratively extended in the direction given by one of the The refinement of the initial motion vector is determined by template matching with the rate. It is configured to do so, Claim 1, where the window is defined by a predefined maximum number of iterations. The apparatus described in 2.

4. The search space is used for interpolation filtering of each fractional sample within the subwindow. All integer samples accessed are interpolated fills of the predefined tap size. Regarding the subwindow, the rectangular subwindow of the window is located within the aforementioned window. Apparatus according to any one of claims 1 to 3.

5. The search space includes a rectangular search subwindow of the window, and the initial movement The refinement of the cult is an interpolation filter of each fractional sample in the search subwindow. The integer samples accessed for the purpose of the tap are the complement of the predefined tap size. Regarding the inter-filter, in the rectangular search subwindow located within the aforementioned window The method described in claim 1 or 2, which is determined by template matching with the aforementioned template. A mounted device.

6. The processing circuit finds more of the best matching positions in the search space in the most recent iteration. The temp in the search space is iteratively extended in the direction given by one of the The refinement of the initial motion vector is determined by template matching with the rate. It is configured such that the iteration is at least one in the search space of the most recent iteration The apparatus according to claim 5, which is terminated when the sample is outside the search subwindow. 。

7. The interpolation filter determines that the fractional position corresponds to each horizontal line of the integer sample. When it lies on a vertical line, K integer samples in either the horizontal or vertical direction. The apparatus according to claim 4, which is a one-dimensional filter that also evaluates.

8. The aforementioned search space is - Adjacent to the above or below the aforementioned subwindow, on the horizontal line of integer samples It is located in, or - Adjacent to the left or right side of the aforementioned subwindow, the vertical line of integer samples Is it on top of The apparatus according to claim 5, further comprising any fractional position outside the subwindow.

9. Encoding software for encoding video images divided into prediction blocks into a bitstream It is placed, For determining motion vectors relating to prediction blocks according to any one of claims 1 to 8 The device and The prediction block gives the position based on the determined motion vector. To encode the difference between the predictor and the encoded difference and the initial An encoding circuit for generating a bitstream containing a motion vector, Place.

10. Decoder for decoding video images divided into prediction blocks from a bitstream And, The initial motion vector, and the predicted block and refined motion vector are specified. The encoded difference between the predictor given by the prediction block at the position is the same as above. An analysis unit for analyzing bitstreams, The refined movement relating to the prediction block according to any one of claims 1 to 8. A device for determining vectors, The analyzed difference and the position specified by the refined motion vector The prediction block is reconstructed as the sum of the prediction block and the predictor given by the measurement block. A decoding device including a decoding circuit for the purpose of decoding.

11. A method for determining motion vectors related to prediction blocks, A step of obtaining an initial motion vector and template for the prediction block, In the search space, the initial movement is determined by template matching with the template. This includes a step of determining the refinement of the vector, The search space is located at a position given by the initial motion vector, and one or more minutes The search space includes several sample positions, and each of the fractional sample positions belonging to the search space is a window Interpolation using a predefined tap-size filter that evaluates only integer samples within the given range. The window obtained by filtering is the template in the search space A method formed by an accessible set of integer samples for matching.

12. The window is a sequence of N integer samples relative to the initial motion vector of the prediction block. The following is defined as an integer sample row, where N and M are non-zero integer values, as described in claim 11. Method of loading.

13. The refinement of the initial motion vector in the most recent iteration results in a larger search space in the search space. The search space is iteratively expanded in the direction given by one of the most matching positions. Determined by template matching with the aforementioned template in between, Claim 11, where the window is defined by a predefined maximum number of iterations. Or the method described in 12.

14. The search space is used for interpolation filtering of each fractional sample within the subwindow. All integer samples accessed are interpolated fills of the predefined tap size. Regarding the subwindow, the rectangular subwindow of the window is located within the aforementioned window. The method according to any one of claims 11 to 13.

15. The search space includes a rectangular search subwindow of the window, and the initial movement The refinement of the cult is an interpolation filter of each fractional sample in the search subwindow. The integer samples accessed for the purpose of the tap are the complement of the predefined tap size. Regarding the inter-filter, in the rectangular search subwindow located within the aforementioned window The following is determined by template matching with the aforementioned template: Method of description.

16. The refinement of the initial motion vector in the most recent iteration results in a larger search space in the search space. The search space is iteratively expanded in the direction given by one of the most matching positions. The iteration is determined by template matching with the template in between, and the iteration is , at least one sample in the search space of the most recent iteration is the search subw The method of claim 15, which is terminated when outside the ndow.

17. The interpolation filter determines that the fractional position corresponds to each horizontal line of the integer sample. When it lies on a vertical line, K integer samples in either the horizontal or vertical direction. The method according to claim 14, which is a one-dimensional filter for evaluating.

18. The aforementioned search space is - Adjacent to the above or below the aforementioned subwindow, on the horizontal line of integer samples It is located in, or - Adjacent to the left or right side of the aforementioned subwindow, the vertical line of integer samples Is it on top of The method according to claim 17, further comprising any fractional position outside the subwindow.

19. Encoding method for encoding video images divided into prediction blocks into a bitstream It is a law, Determining the motion vector for the prediction block according to any one of claims 11 to 18 Steps and The prediction block gives the position based on the determined motion vector. Encode the difference between the predictor and the initial motion vector, and the encoded difference and the initial motion vector An encoding method comprising the step of generating a bitstream containing .

20. Decryption method for decoding video images divided into prediction blocks from a bitstream And, The initial motion vector, and the predicted block and refined motion vector are specified. The encoded difference between the predictor given by the prediction block at the position is the same as above. Steps include analyzing from a bitstream, The refined motion relating to the prediction block according to any one of claims 11 to 18 The step of determining the vector, The analyzed difference and the position specified by the refined motion vector The prediction block is reconstructed as the sum of the prediction block and the predictor given by the measurement block. A decryption method that includes the steps of doing so.

21. When executed on the processor, the processor is described in any one of claims 11 to 20. A computer-readable medium that stores instructions for executing the specified method.

Images