Motion estimation method, chip, electronic device and storage medium

Abstract

A motion estimation method, chip, electronic device and storage medium, the motion estimation method comprising: providing an original frame; using a full search method to obtain a matching block corresponding to a current block of a current frame in a previous frame, and obtaining a matching block corresponding to a current block of a previous frame in a current frame; obtaining a first candidate vector set corresponding to a corresponding block of a current frame according to a current block of a current frame and a corresponding matching block, and obtaining a second candidate vector set corresponding to a corresponding block of a previous frame according to a current block of a previous frame and a corresponding matching block; setting a frame to be interpolated between the previous frame and the current frame; obtaining a bidirectional motion vector corresponding to the current block of a frame to be interpolated according to the first candidate vector set and the second candidate vector set, and using the bidirectional motion vector corresponding to the current block of a frame to be interpolated as a candidate motion vector of a corresponding block of a frame to be interpolated. The embodiment of the present invention is conducive to improving the accuracy of motion estimation.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a motion estimation method, a chip, an electronic device and a storage medium. Background Art

[0002] The video frame rate is a measure of the number of frames played by a display device in one second. Generally, when the frame rate is 60 frames per second, the human eye will feel that the video is realistic and smooth. In actual applications, when low-frame-rate video resources are played on a display device, the human eye can feel jitter and blur. In order to solve the video jitter and blur effect caused by low frame rate, the subjective visual effect of the video image can be improved by increasing the video frame rate technology. If we only use simple frame repetition or adjacent frame averaging methods to increase the frame rate, it still cannot solve the problem of video jitter and blur.

[0003] The main technologies for video frame rate conversion are motion estimation (ME) and motion compensation (MC). The basic idea of ​​motion estimation is to divide each frame of the image sequence into many non-overlapping macroblocks, and assume that the displacement of all pixels in the macroblock is the same. Then, for each macroblock, the most similar block to the current block is found according to a certain matching criterion within a given specific search range of the reference frame, i.e., the matching block. The relative displacement between the matching block and the current block is the motion vector, and the process of obtaining the motion vector is called motion estimation. Motion compensation refers to the use of the motion vector calculated by the motion estimation method to predict and compensate the local image of the current interpolated frame.

[0004] A widely used motion estimation algorithm is the 3D recursive search algorithm, which uses a 3D recursive search block matching method to estimate the motion of an image. In the 3D recursive search block matching method, the motion vector of the current block is a small-scale correction of the temporal and spatial predicted motion vector. The motion vector of the current block is also the spatial and temporal candidate vectors of the next block and the next row of adjacent blocks. Summary of the invention

[0005] The problem solved by the embodiments of the present invention is to provide a motion estimation method, a chip, an electronic device and a storage medium to improve the accuracy of motion estimation.

[0006] To solve the above problem, an embodiment of the present invention provides a motion estimation method, including: providing an original frame, including an adjacent previous frame and a current frame; using a full search method to obtain a matching block corresponding to a current block of the current frame in the previous frame, and obtaining a matching block corresponding to the current block of the previous frame in the current frame; obtaining a first candidate vector set corresponding to a corresponding block of the current frame according to the current block of the current frame and the corresponding matching block, and obtaining a second candidate vector set corresponding to a corresponding block of the previous frame according to the current block of the previous frame and the corresponding matching block; setting a frame to be interpolated between the previous frame and the current frame; and obtaining a bidirectional motion vector corresponding to the current block of the frame to be interpolated according to the first candidate vector set and the second candidate vector set, for use as a candidate motion vector of the corresponding block of the frame to be interpolated.

[0007] Correspondingly, an embodiment of the present invention further provides a chip, which includes a motion estimation module; the motion estimation module includes: a providing unit, which is used to provide an original frame, including an adjacent previous frame and a current frame; a searching unit, which is used to obtain a matching block corresponding to a current block of a current frame in a previous frame by using a full search method, and obtain a matching block corresponding to a current block of a previous frame in a current frame; a calculating unit, which is used to obtain a first candidate vector set corresponding to a corresponding block of a current frame according to a current block of a current frame and a corresponding matching block, and is also used to obtain a second candidate vector set corresponding to a corresponding block of a previous frame according to a current block of a previous frame and a corresponding matching block; a setting unit, which is used to set a frame to be inserted between the previous frame and the current frame; and an acquiring unit, which is used to obtain a bidirectional motion vector corresponding to the current block of the frame to be inserted according to the first candidate vector set and the second candidate vector set output by the calculating unit, and output it as a candidate motion vector of the corresponding block of the frame to be inserted.

[0008] Correspondingly, an embodiment of the present invention further provides an electronic device, including the chip provided by the embodiment of the present invention.

[0009] Correspondingly, an embodiment of the present invention further provides a storage medium, wherein the storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the motion estimation method provided by the embodiment of the present invention.

[0010] Compared with the prior art, the technical solution of the embodiment of the present invention has the following advantages:

[0011] In the motion estimation method provided by the embodiment of the present invention, by using the full search method, the current block of the current frame is block matched with all blocks within the search range of the corresponding position of the previous frame, and the current block of the previous frame is block matched with all blocks within the search range of the corresponding position of the current frame, and the search step size of the full search method is small, thereby providing a more accurate motion direction of the original frame for estimating the motion direction of the frame to be interpolated, which is conducive to increasing the accuracy of motion estimation, especially increasing the accuracy of small object motion estimation. Moreover, the embodiment of the present invention also obtains a bidirectional motion vector corresponding to the current block of the frame to be interpolated based on the first candidate vector set and the second candidate vector set obtained by using the full search method, which is conducive to ensuring the motion continuity between the corresponding block of the frame to be interpolated and the previous frame and the current frame, especially conducive to ensuring the continuity of small object motion, thereby improving the accuracy of motion estimation, and also conducive to reducing the probability of problems such as "overlap" and "holes". BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a schematic diagram of a motion estimation method;

[0013] Figure 2 is a flow chart of an embodiment of a motion estimation method provided by the present invention;

[0014] Figure 3 yes Figure 2 A schematic diagram of a block matching process of a full search in a motion estimation method shown;

[0015] Figure 4 yes Figure 2 A schematic diagram of the process of obtaining a bidirectional motion vector in the motion estimation method shown;

[0016] Figure 5 yes Figure 2 Flow chart of step S5 in an embodiment;

[0017] Figure 6 is a functional block diagram of a chip according to an embodiment of the present invention;

[0018] Figure 7 yes Figure 6 Functional block diagram of an embodiment of a motion estimation module in FIG.

[0019] Figure 8 yes Figure 7 A functional block diagram of an embodiment of a computing unit in FIG.

[0020] Fig. 9 yes Figure 7 A functional block diagram of an embodiment of an acquisition unit in FIG. DETAILED DESCRIPTION

[0021] As can be seen from the background technology, the three-dimensional recursive search algorithm is currently widely used in frame rate conversion technology. However, this motion estimation method has the problem of poor accuracy of motion vector estimation, especially poor accuracy of motion estimation for small foreground moving objects.

[0022] Specifically, the 3D recursive search algorithm uses the continuity of the motion of objects in the video to perform motion estimation through time and space recursion. Time recursion uses the correlation between the motion of the block of the current frame and the adjacent block at the corresponding position of the previous frame to transfer the motion vector of the adjacent block of the previous frame to the block of the current frame. Similarly, space recursion uses the correlation between the motion of the block of the current frame and the adjacent block of the current frame to transfer the motion vector of the adjacent block of the current frame to the current block.

[0023] However, in some special scenarios, the motion vector of the current block is not completely the motion vector of the temporal and spatial adjacent blocks. For example, in a scene where there is a small object moving in the image, if only the temporal and spatial recursive motion vector is used as the candidate motion vector, the small object is likely to be submerged in the background, and the correct motion vector pointing to the small object cannot be found.

[0024] like Figure 1 , which shows a schematic diagram of a motion estimation method, wherein the gray area represents the background in the image, the block B is a small object moving in the foreground, and the small object refers to a block whose area is approximately equal to the area of ​​a block, the P frame represents the previous frame, and the C frame represents the current frame; Figure 1 In the example, the motion vector of each background block is relatively small, most of them are stationary, and the motion vector is approximately zero. When calculating the motion vector of block B, since most of the blocks around the moving foreground small object are stationary and the motion vector is approximately zero, the adjacent motion vectors transferred to the moving small area cannot correctly express the motion vector of the small object.

[0025] That is to say, in the three-dimensional recursive search algorithm, the image is divided into several blocks of the same size, and the candidate motion vectors of the current block are obtained by the three-dimensional recursive search algorithm in time recursion and space recursion. However, the candidate motion vectors obtained by the three-dimensional recursive search algorithm are only from the adjacent motion vectors of the current frame and the previous frame. Small moving objects in the image may not be able to find the correct motion vector, and the image of the small moving objects may be easily lost.

[0026] In order to solve the technical problem, an embodiment of the present invention provides a motion estimation method, including: providing an original frame, including an adjacent previous frame and a current frame; using a full search method to obtain a matching block corresponding to a current block of the current frame in the previous frame, and obtaining a matching block corresponding to the current block of the previous frame in the current frame; obtaining a first candidate vector set corresponding to a corresponding block of the current frame according to the current block of the current frame and the corresponding matching block, and obtaining a second candidate vector set corresponding to a corresponding block of the previous frame according to the current block of the previous frame and the corresponding matching block; setting a frame to be interpolated between the previous frame and the current frame; and obtaining a bidirectional motion vector corresponding to the current block of the frame to be interpolated according to the first candidate vector set and the second candidate vector set, for use as a candidate motion vector of the corresponding block of the frame to be interpolated.

[0027] In the motion estimation method provided by the embodiment of the present invention, by using the full search method, the current block of the current frame is block matched with all blocks within the search range of the corresponding position of the previous frame, and the current block of the previous frame is block matched with all blocks within the search range of the corresponding position of the current frame, and the search step size of the full search method is small, thereby providing a more accurate motion direction of the original frame for estimating the motion direction of the frame to be interpolated, which is conducive to increasing the accuracy of motion estimation, especially increasing the accuracy of small object motion estimation. Moreover, the embodiment of the present invention also obtains a bidirectional motion vector corresponding to the current block of the frame to be interpolated based on the first candidate vector set and the second candidate vector set obtained by using the full search method, which is conducive to ensuring the motion continuity between the corresponding block of the frame to be interpolated and the previous frame and the current frame, especially conducive to ensuring the continuity of small object motion, thereby improving the accuracy of motion estimation, and also conducive to reducing the probability of problems such as "overlap" and "holes".

[0028] Figure 2 1 is a flow chart of an embodiment of a motion estimation method provided by the present invention. As an example, the motion estimation method described in this embodiment includes the following basic steps:

[0029] Step S1: providing an original frame, including an adjacent previous frame and a current frame;

[0030] Step S2: using a full search method, obtaining a matching block corresponding to a current block of a current frame in a previous frame, and obtaining a matching block corresponding to a current block of a previous frame in a current frame;

[0031] Step S3: obtaining a first candidate vector set corresponding to the corresponding block of the current frame according to the current block of the current frame and the corresponding matching block, and obtaining a second candidate vector set corresponding to the corresponding block of the previous frame according to the current block of the previous frame and the corresponding matching block;

[0032] Step S4: setting a frame to be inserted between the previous frame and the current frame;

[0033] Step S5: According to the first candidate vector set and the second candidate vector set, a bidirectional motion vector corresponding to the current block of the frame to be interpolated is obtained, and used as a candidate motion vector of the corresponding block of the frame to be interpolated.

[0034] In order to make the above-mentioned purposes, features and advantages of the embodiments of the present invention more obvious and understandable, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0035] refer to Figure 2 , execute step S1: provide an original frame, including an adjacent previous frame and a current frame.

[0036] The original frame is a video frame sequence corresponding to the video image that has not been processed by the video frame rate conversion technology.

[0037] Subsequently, motion estimation is performed on the previous frame and the current frame to obtain a unidirectional motion vector from the previous frame to the current frame, or from the current frame to the previous frame, as a candidate motion vector of the original frame.

[0038] The previous frame and the current frame may be any two adjacent frames in the original frame.

[0039] Continue to refer Figure 2 In this embodiment, the motion estimation method further includes: before obtaining the matching block corresponding to the current block of the current frame in the previous frame by using the full search method and obtaining the matching block corresponding to the current block of the previous frame in the current frame by using the full search method, identifying the blocks corresponding to the foreground moving small objects in the previous frame and the current frame. Specifically, the image segmentation technology can be used to identify the blocks corresponding to the foreground moving small objects in the previous frame and the current frame.

[0040] In this embodiment, the small object refers to a block having an area approximately equal to that of a block.

[0041] In a video image, the blocks around a small moving object in the foreground are the background area, and the blocks in the background area are usually stationary blocks, that is, the motion vectors of the blocks in the background area are approximately zero, while the area of ​​the small moving object in the foreground is small and the motion difference with the background area is large. If the adjacent blocks around the small object are directly used to estimate the motion of the small object, inaccurate estimation is likely to occur. Therefore, by first identifying the blocks corresponding to the moving small object in the current frame and the previous frame, and then only using the full search method to obtain the corresponding matching blocks for the moving small object, it is beneficial to reduce the computational complexity and amount while improving the accuracy of motion estimation, thereby improving response speed and saving costs.

[0042] Continue to refer Figure 2, execute step S2: use the full search method to obtain the matching block corresponding to the current block of the current frame in the previous frame, and obtain the matching block corresponding to the current block of the previous frame in the current frame.

[0043] The current block is a block in the current frame or the previous frame that needs motion estimation.

[0044] The matching block corresponding to the current block of the current frame in the previous frame is obtained, so that the unidirectional motion vector of the block in the current frame pointing to the matching block in the previous frame can be obtained according to the current block of the current frame and the corresponding matching block, as the backward motion vector of the corresponding block of the current frame.

[0045] The matching block corresponding to the current block of the previous frame is obtained in the current frame, so that the unidirectional motion vector of the block in the previous frame pointing to the matching block in the current frame can be obtained based on the current block and the corresponding matching block of the previous frame as the forward motion vector of the corresponding block of the previous frame.

[0046] The full search algorithm is the simplest and most reliable search and matching method. By using the full search method, the current block of the current frame is block matched with all blocks in the search range of the corresponding position of the previous frame, and the current block of the previous frame is block matched with all blocks in the search range of the corresponding position of the current frame. In other words, the full search algorithm calculates the matching error of two blocks for all possible positions in the search window. The motion vector (Motion Vector, MV) corresponding to the minimum matching error is the global optimal motion vector. In addition, the search step size of the full search method is small, thereby providing a more accurate motion direction of the original frame for estimating the motion direction of the frame to be inserted, which is conducive to increasing the accuracy of motion estimation, especially increasing the accuracy of motion estimation of small objects.

[0047] In this embodiment, before using the full search method to obtain the matching block corresponding to the current block of the current frame in the previous frame, and obtaining the matching block corresponding to the current block of the previous frame in the current frame, the blocks corresponding to the foreground moving small objects in the previous frame and the current frame are also identified. Therefore, using the full search method to obtain the matching block corresponding to the corresponding block of the moving small object is beneficial to searching for the block in the previous frame that best matches the corresponding block of the small object in the current frame, and searching for the block in the current frame that best matches the corresponding block of the small object in the previous frame, thereby providing a more accurate original frame motion direction for the moving small object, and further facilitating finding a more accurate motion vector for the moving small object, and also facilitating reducing the amount of calculation and improving the calculation efficiency.

[0048] Specifically, in this embodiment, the full search method searches all pixels within the search range in order. As an example, the search step length of the full search method is one pixel. The search and matching process is a carpet search with one pixel as the step length, and the similarity of blocks is compared point by point. It is of great significance for the search and matching of small objects in the video, and is conducive to improving the accuracy of obtaining the corresponding matching blocks of the blocks of the moving small objects. Accordingly, when the backward motion vector is obtained according to the current block and the corresponding matching block in the subsequent process, it is conducive to increasing the accuracy of the estimation of the motion direction of the moving small object.

[0049] It should be noted that, in other embodiments, in order to reduce the amount of calculation, improve the response speed, and according to the actual calculation accuracy requirements, the search step length of the full search method can also be greater than one pixel, for example: the search step length of the full search method is an integer multiple of one pixel and is less than the side length of the block. In other words, the search step length of the full search method is greater than or equal to one pixel and less than the side length of the block. The search step length of the full search method is always less than the side length of the block, that is, the search step length of the full search method is less than the side length of the moving small object. Compared with using the surrounding adjacent blocks of the block of the small object to estimate the motion of the small object, the search step length of the full search method in the embodiment of the present invention is smaller, and the search is more refined, which is conducive to estimating a more accurate motion direction for the moving small object.

[0050] In this embodiment, the step of obtaining the matching block includes: using a full search method to obtain a matching block corresponding to the current block based on a matching error and a distance from the current block.

[0051] Specifically, the full search method is used to obtain the matching block corresponding to the current block of the current frame in the previous frame based on the matching error and the distance between the current block and the current block; the full search method is used to obtain the matching block corresponding to the current block of the previous frame in the current frame based on the matching error and the distance between the current block and the current block.

[0052] In the embodiment of the present invention, in the process of searching for matching blocks, not only the influencing factors of the matching error are taken into account, but also the distance from the current block is taken into account. This is because in the actual search and matching process, when the full search method is used to directly obtain the block with the smallest matching error (or within a preset threshold range) with the current block, it is affected by interference factors such as noise and boundaries in the image, and there is a certain probability that the found block is not the block that actually corresponds to the current block. For example, although the matching error between the found block and the current block is the smallest, the distance from the current block is very far, and the probability that the found block is noise is high. By taking the distance as a factor in the process of searching for matching blocks, it is helpful to find the block that actually matches and is most similar to the current block, so that when estimating the motion vector of the current block later, it is helpful to estimate the real motion direction of the current block, thereby improving the accuracy of motion estimation.

[0053] Specifically, in the actual search and matching process, the matching error and distance can be assigned corresponding weight factors respectively, so that the importance and priority of the matching error and distance can be adjusted by adjusting the corresponding weight factors according to the actual situation and needs, so as to improve the search accuracy of the matching block and reduce the influence of interference factors such as noise and boundaries in the image on the search and matching process.

[0054] In this embodiment, the step of obtaining a matching block corresponding to the current block of the current frame in the previous frame includes: using a full search method to obtain multiple candidate blocks in the previous frame whose matching errors with the current block of the current frame are within a preset threshold (Threshold); and using a candidate block with the smallest distance to the current block of the current frame as the matching block.

[0055] In this embodiment, the step of obtaining a matching block in the current frame corresponding to the current block of the previous frame includes: using a full search method to obtain multiple candidate blocks in the current frame whose matching errors with the current block of the previous frame are within a preset threshold range; and using a candidate block with the smallest distance to the current block of the previous frame as the matching block.

[0056] Combined with reference Figure 3 , taking the use of the full search method to obtain the matching block corresponding to the current block of the current frame in the previous frame as an example, a schematic diagram of the block matching process of the full search in an embodiment of the present invention is illustrated.

[0057] In this embodiment, based on formula (I), the current block B in the previous frame and the current frame is obtained. c The candidate blocks whose matching error is within the preset threshold range.

[0058]

[0059] Among them, B p and B c Respectively represent the blocks of the previous frame and the current frame, and the size of the block is N*N, (r, c) represents the position of the current block in the previous frame, (r+Δr, c+Δc) represents the position of the blocks around the current block, Δr and Δc represent the vertical and horizontal offsets of the surrounding blocks relative to the current block, w represents the search range of the full search method in the previous frame, and error represents the matching error. In this embodiment, min(error) represents the process of finding the minimum matching error of all matching blocks.

[0060] As an example, the matching error error can be obtained based on matching criteria such as SAD (Sum of Absolute Differences), MAD (Mean Absolute Differences), MSD (Mean Square Differences), SSD (Sum of Squared Differences), and NCCF (Normalized Cross-Correlation Function).

[0061] Among them, w represents the search range of the full search method in the previous frame, that is, w defines the size of the search window of the full search method. The larger the search window, the easier it is to find the global optimal matching block, but it is also easy to increase the probability of finding blocks with interference factors such as noise or boundaries. Moreover, a search window that is too large is likely to increase the operation cost and amount of calculation; the smaller the search window, the easier it is to find a matching block with the smallest matching error with the current block. However, due to the small search window, the distance between the blocks in the search range and the current block is relatively close, and the probability of finding blocks with interference factors such as noise and boundaries is also low. In addition, a small search window is conducive to reducing the amount of calculation. Therefore, in the actual search and matching process, the size of w can be reasonably set according to the actual situation and needs.

[0062] Specifically, in the actual search and matching process, the matching errors between multiple candidate blocks and the current block may all be minimum values, or the matching errors between multiple candidate blocks and the current block may all be within a preset threshold range.

[0063] The preset threshold range is used to ensure that the matching error between the searched candidate block and the current block is not too large, so as to ensure that the candidate block and the current block have a high degree of similarity and matching. Moreover, in the actual search and matching process, the matching error between the current block and the corresponding block in the previous frame may not be zero, but close to zero. By setting a preset threshold range, it is helpful to improve the rationality of the search and matching.

[0064] In a specific implementation, the preset threshold may be a value given in advance, such as a minimum possible SAD value being increased by a certain range, and this range may be set empirically, or an adaptive value may be given based on the complexity analysis of the current image block.

[0065] When there are multiple candidate blocks, a candidate block with the smallest distance from the current block is used as the matching block. In a video, the time interval between two adjacent video frames is short. In the process of motion estimation, it is generally believed that a moving object approximately moves in a straight line within the time interval between two adjacent frames. Since the time interval between two adjacent frames is short, the displacement of the moving object between two adjacent frames is usually small. That is, the distance between the current block of the current frame and the corresponding block in the previous frame is relatively close, or the distance between the current block of the previous frame and the corresponding block in the current frame is relatively close. By using a candidate block with the smallest distance from the current block as a matching block, it is not only helpful to exclude blocks corresponding to interference factors such as noise and boundaries in the candidate blocks, but also helpful to search for a matching block that actually corresponds to the current block of the current frame in the previous frame, and to search for a matching block that actually corresponds to the current block of the previous frame in the current frame, thereby improving the accuracy of search matching.

[0066] Continue to refer Figure 2 , execute step S3: obtain a first candidate vector set corresponding to the corresponding block of the current frame according to the current block of the current frame and the corresponding matching block, and obtain a second candidate vector set corresponding to the corresponding block of the previous frame according to the current block of the previous frame and the corresponding matching block.

[0067] By utilizing the full search method, the matching block corresponding to the current block of the current frame in the previous frame is obtained, which is helpful for searching the block that best matches the current block in the previous frame, especially for small moving objects, searching for the block that best matches the block of the small object, thereby providing a more accurate original frame motion direction for the small moving object, and further helping to find a more accurate motion vector for the small moving object.

[0068] The first candidate vector set is used to provide a candidate vector set for obtaining a bidirectional motion vector corresponding to a corresponding block of the frame to be interpolated, which is beneficial to ensuring the motion continuity between the corresponding block of the frame to be interpolated and the previous frame and the current frame, especially to ensuring the continuity of the motion of small objects, thereby improving the accuracy of motion estimation.

[0069] By utilizing the full search method, the matching block corresponding to the current block of the previous frame in the current frame is obtained, which is helpful for searching the block that best matches the current block of the previous frame in the current frame, especially for small moving objects, searching for the block that best matches the corresponding block of the small object, thereby providing a more accurate original frame motion direction for the small moving object, and further helping to find a more accurate motion vector for the small moving object.

[0070] The second candidate vector set is used to provide a candidate vector set for obtaining a bidirectional motion vector corresponding to a corresponding block of the frame to be interpolated, which is beneficial to ensuring the motion continuity between the corresponding block of the frame to be interpolated and the previous frame and the current frame, especially to ensuring the continuity of the motion of small objects, thereby improving the accuracy of motion estimation.

[0071] In this embodiment, the step of obtaining the first candidate vector set and the second candidate vector set includes: obtaining a backward motion vector of a corresponding block of the current frame according to a current block of the current frame and a corresponding matching block, and the backward motion vectors corresponding to multiple blocks of the current frame constitute the first candidate vector set; and obtaining a forward motion vector of a corresponding block of a previous frame according to a current block of a previous frame and a corresponding matching block, and the forward motion vectors corresponding to multiple blocks of the previous frame constitute the second candidate vector set.

[0072] Specifically, after obtaining the matching block corresponding to the current block of the current frame in the previous frame, the position offset of the current block of the current frame relative to the matching block is obtained according to the relative position relationship between the current block of the current frame and the corresponding matching block, which is used as the backward motion vector.

[0073] Specifically, after obtaining the matching block corresponding to the current block of the previous frame in the current frame, the position offset of the current block of the previous frame relative to the matching block is obtained according to the relative position relationship between the current block of the previous frame and the corresponding matching block, which is used as the forward motion vector.

[0074] Combined with reference Figure 2 and Figure 4 , execute step S4: set the previous frame (such as Figure 4 P frame) and the current frame (as shown in Figure 4 The interpolated frames are shown in the C frame. Figure 4 A schematic diagram of a process of obtaining a bidirectional motion vector in a motion estimation method according to an embodiment of the present invention is illustrated.

[0075] By setting the interpolated frame between the previous frame and the current frame, the video frame rate is increased, which is beneficial to improving the fluency and smoothness of the video, thereby improving the user experience.

[0076] Specifically, in the time sequence, a frame to be inserted is set between the previous frame and the current frame.

[0077] In the time sequence, the frame to be inserted can be located at any position between the previous frame and the current frame.

[0078] The frame to be inserted has a plurality of blocks to be inserted which are arranged in sequence.

[0079] The frame to be inserted is set between the previous frame and the current frame, and subsequently the bidirectional motion vector corresponding to the block of the frame to be inserted is obtained according to the first candidate vector set corresponding to the current frame and the second candidate vector set corresponding to the previous frame.

[0080] Continue to refer Figure 2 and Figure 4, executing step S5: obtaining a bidirectional motion vector corresponding to the current block of the frame to be interpolated according to the first candidate vector set and the second candidate vector set, and using the bidirectional motion vector as a candidate motion vector of the corresponding block of the frame to be interpolated.

[0081] In the embodiment of the present invention, by using the full search method, the current block of the current frame is block matched with all blocks within the search range of the corresponding position of the previous frame, and the current block of the previous frame is block matched with all blocks within the search range of the corresponding position of the current frame, and the search step size of the full search method is small, thereby providing a more accurate motion direction of the original frame for estimating the motion direction of the frame to be interpolated, which is conducive to increasing the accuracy of motion estimation, especially increasing the accuracy of small object motion estimation. Moreover, according to the first candidate vector set and the second candidate vector set obtained by using the full search method, a bidirectional motion vector corresponding to the current block of the frame to be interpolated is obtained, which is conducive to ensuring the motion continuity between the corresponding block of the frame to be interpolated and the previous frame and the current frame, especially conducive to ensuring the continuity of small object motion, thereby improving the accuracy of motion estimation, and also conducive to reducing the probability of problems such as "overlap" and "holes".

[0082] Combined with reference Figure 5 , indicating Figure 2 In this embodiment, the step of obtaining the bidirectional motion vector corresponding to the current block to be interpolated according to the first candidate vector set and the second candidate vector set includes:

[0083] Step S51 is executed to obtain backward motion vectors corresponding to all blocks within a preset search range at corresponding positions in the current frame based on the first candidate vector set, taking the current block of the frame to be interpolated as a reference, as the first motion vector.

[0084] As an example, Figure 4 As shown, taking the current block M in the frame to be inserted as a reference, based on the first candidate vector set, the current frame (such as Figure 4 The backward motion vectors corresponding to blocks D, E and F in the search range of the corresponding positions in the frame C are used as the first motion vectors and correspond to V d 、V e and V f .

[0085] Step S52 is executed to obtain forward motion vectors corresponding to all blocks within a preset search range at corresponding positions in the previous frame based on the second candidate vector set, taking the current block of the frame to be inserted as a reference, as the second motion vector.

[0086] As an example, Figure 4 As shown, taking the current block M in the frame to be inserted as a reference, based on the second candidate vector set, the previous frame (such as Figure 4The backward motion vectors corresponding to blocks A, B, and C in the search range of the corresponding positions in the P frame are used as the second motion vectors, respectively corresponding to V a 、V b and V c .

[0087] Step S53 is executed to obtain a first motion vector and a second motion vector that pass through the current block and have the same direction as each other as the bidirectional motion vector.

[0088] The first motion vector and the second motion vector corresponding to the bidirectional motion vector have the same motion direction and pass through the current block of the frame to be interpolated, that is, the motion directions of the two blocks mapped to the previous frame and the current frame by the bidirectional motion vector are consistent, indicating that the motion trajectory of the block corresponding to the bidirectional motion vector in the previous frame and the current frame passes through the current block of the frame to be interpolated, and is consistent with the motion of the current block of the frame to be interpolated, which is conducive to enabling the obtained candidate motion vector to accurately reflect the motion state of the moving object, especially the real motion state of the small moving object.

[0089] When the obtained bidirectional motion vector has only one corresponding first motion vector and second motion vector, the bidirectional motion vector is used as a candidate motion vector of the corresponding block to be interpolated.

[0090] It should be noted that in the actual motion estimation process, the moving object does not necessarily move in a strictly standard straight line between adjacent frames. Therefore, in the actual motion estimation process, the first motion vector and the second motion vector that pass through the current block and whose motion directions have an angle of approximately 180° can be obtained as bidirectional motion vectors. The angle between the first motion vector and the second motion vector is approximately 180°, and it can be considered that the continuity of the moving object along the motion directions of the first motion vector and the second motion vector is relatively high.

[0091] In a specific embodiment, a direction reference value may be set, and the absolute value of the difference between the angle between the first motion vector and the second motion vector of the current block to be interpolated and 180° is compared with the direction reference value, and if the absolute value of the difference is less than the direction reference value, the corresponding first motion vector and the second motion vector are used as bidirectional motion vectors. The reference value may be set to a value close to zero.

[0092] It should also be noted that, when there are multiple pairs of bidirectional motion vectors, the step of obtaining candidate motion vectors for the corresponding block of the frame to be interpolated also includes: comparing the matching errors of the blocks corresponding to the bidirectional motion vectors in the previous frame and the current frame; and obtaining the bidirectional motion vector corresponding to the minimum matching error as the candidate motion vector for the corresponding block of the frame to be interpolated.

[0093] The matching errors of the blocks corresponding to the bidirectional motion vectors in the previous frame and the current frame are compared. The smaller the matching error, the higher the similarity and matching degree of the blocks corresponding to the bidirectional motion vectors in the previous frame and the current frame. Therefore, the accuracy of motion estimation is further improved by taking the bidirectional motion vector corresponding to the minimum matching error as the candidate motion vector of the frame to be inserted.

[0094] As an example, Figure 4 There are two pairs of first motion vectors and second motion vectors that pass through the current block M to be inserted and have the same direction, namely: V e and V c , and V a and V f . Figure 4 The second motion vector V corresponding to block B b , the first motion vector V corresponding to block D d Although it passes through the frame to be inserted, it does not pass through the current block M, but passes through the block N of the frame to be inserted, and V b and V d The directions of the two motion vectors are inconsistent, so these two motion vectors are excluded.

[0095] Next, the matching errors between block A and block F, and the matching errors between block C and block E are calculated respectively, and the bidirectional motion vectors corresponding to the two blocks with smaller matching errors are used as candidate motion vectors of block M to be interpolated.

[0096] In other embodiments, the first motion vector and the second motion vector may not pass through the current block of the frame to be interpolated and have the same direction. In this embodiment, the motion vectors of the neighboring blocks around the block to be interpolated can be used as candidate motion vectors of the corresponding block of the frame to be interpolated. Alternatively, other methods can be used to obtain the candidate motion vectors of the corresponding block of the frame to be interpolated.

[0097] Accordingly, the present invention further provides a chip, wherein the chip includes a motion estimation module. Figure 6 , shows a functional block diagram of an embodiment of a chip of the present invention, and combined with reference Figure 7 , showing Figure 6 A functional block diagram of an embodiment of a motion module in FIG.

[0098] The motion estimation module 100 is used to estimate the corresponding motion vector for the block to be inserted into the frame.

[0099] In this embodiment, the motion estimation module 100 includes: a providing unit 10, which is used to provide an original frame, including an adjacent previous frame and a current frame; a searching unit 30, which is used to obtain a matching block corresponding to a current block of a current frame in a previous frame and a matching block corresponding to a current block of a previous frame in a current frame by using a full search method; a calculating unit 40, which is used to obtain a first candidate vector set corresponding to a corresponding block of a current frame according to a current block of a current frame and a corresponding matching block, and is also used to obtain a second candidate vector set corresponding to a corresponding block of a previous frame according to a current block of a previous frame and a corresponding matching block; a setting unit 50, which is used to set a frame to be interpolated between the previous frame and the current frame; and an acquiring unit 60, which is used to obtain a bidirectional motion vector corresponding to the current block of a frame to be interpolated according to the first candidate vector set and the second candidate vector set output by the calculating unit 40, and output the bidirectional motion vector as a candidate motion vector of a corresponding block of a frame to be interpolated.

[0100] The chip 300 provided by the embodiment of the present invention includes a motion estimation module 100, and the motion estimation module 100 includes a search unit 30. The search unit 30 uses a full search method to block match the current block of the current frame with all blocks in the search range of the corresponding position of the previous frame, and block match the current block of the previous frame with all blocks in the search range of the corresponding position of the current frame. The search step size of the full search method is small, so that a more accurate motion direction of the original frame is provided for estimating the motion direction of the frame to be inserted, which is conducive to increasing the accuracy of motion estimation, especially increasing the accuracy of motion estimation of small objects. Moreover, the acquisition unit 60 also obtains a bidirectional motion vector corresponding to the current block of the frame to be inserted based on the first candidate vector set and the second candidate vector set obtained by using the full search method, so as to ensure the motion continuity of the corresponding block of the frame to be inserted and the previous frame and the current frame, especially to ensure the continuity of the motion of small objects, thereby improving the accuracy of motion estimation and correspondingly improving the performance of the chip 300, for example: improving the audio and video processing capability of the chip 300.

[0101] The original frames provided by the providing unit 10 are video frame sequences corresponding to video images that have not been processed by the video frame rate conversion technology. The providing unit 10 outputs the previous frame and the current frame in the original frames to the searching unit 30 .

[0102] The previous frame and the current frame may be any two adjacent frames in the original frame.

[0103] In this embodiment, the motion module 100 further includes: an identification unit 20 for identifying blocks corresponding to the moving small object in the previous frame and the current frame. Specifically, the identification unit 20 is used to identify blocks corresponding to the moving small object in the previous frame and the current frame using image segmentation technology.

[0104] In this embodiment, the small object refers to a block having an area approximately equal to the size of a block.

[0105] In a video image, the blocks around a small moving object in the foreground are the background area, and the blocks in the background area are usually stationary blocks, that is, the motion vectors of the blocks in the background area are approximately zero, while the area of ​​the small moving object in the foreground is small and the motion difference with the background area is large. If the adjacent blocks around the small object are directly used to estimate the motion of the small object, there is a problem of inaccurate estimation. Therefore, the recognition unit 20 is used to identify the blocks corresponding to the moving small object in the previous frame and the current frame, so that the search unit 30 only uses the full search method to search for the corresponding matching blocks for the moving small object, which is conducive to reducing the computational complexity and the amount of computation, thereby improving the response speed and saving costs.

[0106] The search unit 30 is used to obtain a matching block corresponding to a current block of a current frame in a previous frame, and to obtain a matching block corresponding to a current block of a previous frame in a current frame, by using a full search method.

[0107] The search unit 30 obtains the matching block corresponding to the current block of the current frame in the previous frame, so that the calculation unit 40 obtains the unidirectional motion vector of the block in the current frame pointing to the matching block in the previous frame according to the current block of the current frame and the corresponding matching block, as the backward motion vector of the corresponding block of the current frame.

[0108] The search unit 30 obtains the matching block corresponding to the current block of the previous frame in the current frame, so that the calculation unit 40 obtains the unidirectional motion vector of the block in the previous frame pointing to the matching block in the current frame based on the current block of the previous frame and the corresponding matching block, as the forward motion vector of the corresponding block of the previous frame.

[0109] The full search algorithm is the simplest and most reliable search and matching method. By using the full search method, the blocks of the current frame are block matched with all blocks in the search range of the corresponding position of the previous frame, and the blocks of the previous frame are block matched with all blocks in the search range of the corresponding position of the current frame. In other words, the full search algorithm calculates the matching error of two blocks for all possible positions in the search window. The motion vector corresponding to the minimum matching error is the globally optimal motion vector. In addition, the search step size of the full search method is small, thereby providing a more accurate motion direction of the original frame for estimating the motion direction of the frame to be inserted, which is beneficial to increase the accuracy of motion estimation, especially the accuracy of motion estimation of small objects.

[0110] In this embodiment, the motion module 100 also includes an identification unit 20 for identifying whether the current block is a small moving object. Therefore, the search unit 30 is used to use a full search method to obtain a matching block corresponding to the block of the small moving object in the current frame in the previous frame, and to obtain a matching block corresponding to the block of the small object in the previous frame in the current frame. By obtaining the matching block corresponding to the corresponding block of the small moving object, it is helpful to search for the block that best matches the corresponding block of the small object in the previous frame for the current frame, and to search for the block that best matches the corresponding block of the small object in the current frame for the previous frame, thereby providing a more accurate original frame motion direction for the small moving object, finding a more accurate motion vector for the small moving object, while reducing the amount of calculation and improving the response speed of the chip 300.

[0111] Specifically, in this embodiment, the full search method searches all points within the search range in order. As an example, the search step length of the full search method is one pixel. The search matching process is a carpet search with a step length of one pixel, and the similarity of blocks is compared point by point. It is of great significance for the search and matching of small objects in the video, and is conducive to improving the accuracy of obtaining the block corresponding to the matching block of the small object. Accordingly, when the calculation unit 40 obtains the backward motion vector according to the current block of the current frame and the corresponding matching block, and obtains the forward motion vector according to the current block of the previous frame and the corresponding matching block, it is conducive to increasing the accuracy of the estimation of the motion direction of the moving small object.

[0112] It should be noted that, in other embodiments, in order to reduce the amount of calculation, improve the response speed, and according to the actual calculation accuracy requirements, the search step length of the full search method can also be greater than one pixel, for example: the search step length of the full search method is an integer multiple of one pixel and is less than the side length of the block. In other words, the search step length of the full search method is greater than or equal to one pixel and less than the side length of the block. The search step length of the full search method is always less than the side length of the block, that is, the search step length of the full search method is less than the side length of the small object. Compared with using the surrounding adjacent blocks of the block of the small object to estimate the motion of the small object, the search step length of the full search method in the embodiment of the present invention is smaller, and the search is more refined, which is conducive to estimating a more accurate motion direction for the moving small object.

[0113] As an example, the full search method can compare the similarity and matching degree between blocks based on matching criteria such as SAD (Sum of Absolute Differences), MAD (Mean Absolute Differences), MSD (Mean Square Differences), SSD (Sum of Squared Differences), and NCCF (Normalized Cross-Correlation Function).

[0114] In this embodiment, the search unit 30 uses a full search method to obtain a matching block corresponding to the current block based on a matching error and a distance from the current block.

[0115] In this embodiment, the search unit 30 not only considers the influencing factors of the matching error, but also considers the distance from the current block. This is because in the actual search and matching process, when using the full search method to directly obtain the block with the smallest matching error with the current block (or within a preset threshold range), it is affected by interference factors such as noise and boundaries in the image. There is a certain probability that the block found is not the block that actually corresponds to the current block. For example, although the matching error between the found block and the current block is the smallest, the distance between the found block and the current block is very far, and the probability that the found block is noise is high. By taking the distance as a factor in the process of searching for matching blocks, it is helpful to find the block that actually matches the current block the most, so when estimating the motion vector of the current block, it is helpful to estimate the true motion direction of the current block, thereby improving the accuracy of motion estimation.

[0116] Specifically, in the actual search and matching process, the matching error and distance can be assigned corresponding weight factors respectively, so that according to the actual situation and needs, the importance and priority of the matching error and distance can be adjusted by adjusting the weight factors corresponding to the matching error and distance to improve the search accuracy of the matching block, thereby reducing the influence of interference factors such as noise and boundaries in the image on the search and matching process.

[0117] Specifically, the search unit 30 uses a full search method to obtain a matching block corresponding to the current block of the current frame in the previous frame and a matching block corresponding to the current block of the previous frame in the current frame based on a matching error and a distance from the current block.

[0118] In this embodiment, the search unit 30 uses a full search method to obtain multiple candidate blocks in the previous frame whose matching errors with the current block of the current frame are within a preset threshold range; and uses the candidate block with the smallest distance to the current block as the matching block corresponding to the current block of the current frame.

[0119] In this embodiment, the search unit 30 uses a full search method to obtain multiple candidate blocks in the current frame whose matching errors with the current block of the previous frame are within a preset threshold range; and uses a candidate block with the smallest distance to the current block as the matching block corresponding to the current block of the previous frame.

[0120] Specifically, in the actual search and matching process, the matching errors between multiple candidate blocks and the current block may all be minimum values, or the matching errors between multiple candidate blocks and the current block may all be within a preset threshold range.

[0121] The preset threshold range is used to ensure that the matching error between the searched candidate block and the current block is not too large, so as to ensure that the candidate block and the current block have a high degree of similarity and matching. Moreover, in the actual search and matching process, the matching error between the current block of the current frame and the corresponding block in the previous frame, and the matching error between the current block of the previous frame and the corresponding block in the current frame may not be zero, but close to zero. By setting a preset threshold range, it is helpful to improve the rationality of the search and matching.

[0122] Specifically, in a specific implementation, the preset threshold may be a value given in advance, such as a minimum possible SAD value being increased by a certain range, and this range may be set empirically, or an adaptive value may be given based on the complexity analysis of the current image block.

[0123] When there are multiple candidate blocks, the candidate block with the smallest distance from the current block is used as the matching block. In a video, the time interval between two adjacent frames is short. In the process of motion estimation, it is usually believed that a moving object moves approximately in a straight line within the time interval between two adjacent frames. Since the time interval between two adjacent frames is short, the amount of motion of the moving object between two adjacent frames is usually small, that is, the distance between the current block and the corresponding block in the previous frame is small. By using the candidate block with the smallest distance from the current block as the matching block, it is not only helpful to exclude blocks with interference factors such as noise and boundaries in the candidate blocks, but also helpful to search for a matching block that actually corresponds to the current block of the current frame in the previous frame, and to search for a matching block that actually corresponds to the current block of the previous frame in the current frame, thereby improving the accuracy of search matching.

[0124] The search unit 30 uses a full search method, which is beneficial for searching for a block in the previous frame that best matches the current block of the current frame, and for searching for a block in the current frame that best matches the current block of the previous frame, especially for searching for a block that best matches the corresponding block of a small moving object, thereby facilitating the calculation unit 40 to estimate a more accurate unidirectional motion vector between two adjacent frames in the original frame, especially to find a more accurate motion vector for a small moving object, thereby providing a more accurate original frame motion direction for the small moving object.

[0125] Combined with reference Figure 8 ,yes Figure 7 The functional block diagram of an embodiment of a calculation unit in the embodiment, in this embodiment, the calculation unit includes 40: a first sub-calculation unit 401, used to obtain the backward motion vector of the corresponding block of the current frame according to the current block of the current frame and the corresponding matching block, and output the backward motion vectors corresponding to multiple blocks of the current frame as the first candidate vector set; a second sub-calculation unit 402, used to obtain the forward motion vector of the corresponding block of the previous frame according to the current block of the previous frame and the corresponding matching block, and output the forward motion vectors corresponding to multiple blocks of the previous frame as the second candidate vector set.

[0126] The first candidate vector set is used to provide a candidate vector set for obtaining a bidirectional motion vector corresponding to a corresponding block of the frame to be interpolated, which is beneficial to ensuring the motion continuity between the corresponding block of the frame to be interpolated and the previous frame and the current frame, especially to ensuring the continuity of the motion of small objects, thereby improving the accuracy of motion estimation.

[0127] Specifically, the calculation unit 40 obtains the position offset of the current block of the current frame relative to the matching block according to the relative position relationship between the current block of the current frame and the corresponding matching block, which is used as the backward motion vector.

[0128] The second candidate vector set is used to provide a candidate vector set for obtaining a bidirectional motion vector corresponding to a corresponding block of the frame to be interpolated, which is beneficial to ensuring the motion continuity between the corresponding block of the frame to be interpolated and the previous frame and the current frame, especially to ensuring the continuity of the motion of small objects, thereby improving the accuracy of motion estimation.

[0129] Specifically, the calculation unit 40 obtains the position offset of the current block of the previous frame relative to the matching block according to the relative position relationship between the current block of the previous frame and the corresponding matching block, which is used as the forward motion vector.

[0130] The setting unit 50 is used to set the frame to be inserted between the previous frame and the current frame, so as to improve the video frame rate, which is beneficial to improve the fluency and smoothness of the video. Specifically, the setting unit 50 is used to set the frame to be inserted between the previous frame and the current frame along the time sequence. Along the time sequence, the frame to be inserted can be set at any position between the previous frame and the current frame.

[0131] The frame to be inserted has a plurality of blocks to be inserted which are arranged in sequence.

[0132] The setting unit 50 sets the frame to be inserted between the previous frame and the current frame, so that the acquisition unit 60 obtains the bidirectional motion vector corresponding to the block of the frame to be inserted according to the first candidate vector set corresponding to the current frame and the second candidate vector set corresponding to the previous frame.

[0133] The acquisition unit 60 is used to obtain a bidirectional motion vector corresponding to the current block of the frame to be interpolated according to the first candidate vector set and the second candidate vector set obtained by the search unit 30 using the full search method, so as to help ensure the motion continuity between the corresponding block of the frame to be interpolated and the previous frame and the current frame, especially to ensure the continuity of the motion of small objects, thereby improving the accuracy of motion estimation and reducing the probability of problems such as "overlap" and "holes".

[0134] Combined with reference Fig. 9 ,yes Figure 7 , in this embodiment, the acquisition unit 60 includes: an extraction unit 601, which is used to obtain, based on the current block of the frame to be interpolated and the first candidate vector set, the backward motion vectors corresponding to all blocks in the preset search range of the corresponding position in the current frame as the first motion vector, and to obtain, based on the current block of the frame to be interpolated and the second candidate vector set, the forward motion vectors corresponding to all blocks in the preset search range of the corresponding position in the previous frame as the second motion vector; a screening unit 602, which is used to obtain the first motion vector and the second motion vector that pass through the current block and have the same direction as the bidirectional motion vector.

[0135] The first motion vector and the second motion vector corresponding to the bidirectional motion vector have the same motion direction and pass through the current block of the frame to be interpolated, that is, the motion directions of the two blocks mapped to the previous frame and the current frame by the bidirectional motion vector are consistent, indicating that the motion trajectory of the block corresponding to the bidirectional motion vector in the previous frame and the current frame passes through the current block of the frame to be interpolated, and is consistent with the motion of the current block of the frame to be interpolated, which is conducive to enabling the obtained candidate motion vector to accurately reflect the motion state of the moving object, especially the real motion state of the small moving object.

[0136] When the obtained bidirectional motion vector has only one corresponding first motion vector and second motion vector, the bidirectional motion vector is used as a candidate motion vector of the corresponding block to be interpolated.

[0137] It should be noted that in the actual motion estimation process, the moving object does not necessarily move in a strictly standard straight line between adjacent frames. Therefore, in the actual motion estimation process, the first motion vector and the second motion vector that pass through the current block and whose motion directions have an angle of approximately 180° can be obtained as bidirectional motion vectors. The angle between the first motion vector and the second motion vector is approximately 180°, and it can be considered that the continuity of the moving object along the motion directions of the first motion vector and the second motion vector is relatively high.

[0138] In a specific implementation, a direction reference value may be set, and the absolute value of the difference between the angle between the first motion vector and the second motion vector of the current block to be interpolated and 180° is compared with the direction reference value, and if the absolute value of the difference is less than the direction reference value, the corresponding first motion vector and the second motion vector are used as bidirectional motion vectors. The reference value may be set to a value close to zero.

[0139] In this embodiment, the acquisition unit 60 also includes: a comparison unit 603, which is used to compare the matching errors of the blocks corresponding to the bidirectional motion vectors in the previous frame and the current frame when the bidirectional motion vectors output by the screening unit 602 are multiple pairs, and output the bidirectional motion vector corresponding to the minimum matching error as a candidate motion vector of the corresponding block of the frame to be inserted.

[0140] The comparison unit 603 is used to compare the matching errors of the blocks corresponding to the bidirectional motion vectors in the previous frame and the current frame. The smaller the matching error, the higher the similarity and matching degree of the blocks corresponding to the bidirectional motion vectors in the previous frame and the current frame. Therefore, by taking the bidirectional motion vector corresponding to the minimum matching error as the candidate motion vector with a fast response to the interpolated frame, it is helpful to further improve the accuracy of motion estimation.

[0141] In other embodiments, the first motion vector and the second motion vector may not pass through the current block of the frame to be interpolated and have the same direction. In this embodiment, the motion vectors of the neighboring blocks around the current block of the frame to be interpolated can be used as candidate motion vectors corresponding to the frame to be interpolated. Alternatively, other methods can be used to obtain candidate motion vectors of the corresponding block of the frame to be interpolated.

[0142] In this embodiment, the chip 300 further includes: a motion compensation module 200, configured to perform motion compensation on the block to be inserted according to the candidate motion vector output by the motion estimation module 100.

[0143] In this embodiment, the chip 300 may be a multimedia chip, an audio and video chip, a video processing chip, a MEMC (Motion Estimation and Motion Compensation) chip, etc.

[0144] The chip 300 in the embodiment of the present invention includes the motion estimation module, which is beneficial to improving the audio and video processing capability of the chip, and further beneficial to optimizing the user's visual experience and improving the user's perception.

[0145] The chip 300 in the embodiment of the present invention can be implemented by a digital logic circuit, each of which is used to perform the method mentioned in the previous corresponding embodiment through logical operations. Alternatively, the chip 300 can be implemented by a general-purpose processor, and the general-purpose processor runs the relevant processing method by executing a computer software program.

[0146] Correspondingly, an embodiment of the present invention further provides an electronic device, including the chip 300 provided by an embodiment of the present invention.

[0147] In this embodiment, the electronic device is a device capable of performing video image processing, such as a set-top box or a display device, wherein the display device may be a television, a projector, or a mobile phone.

[0148] It can be seen from the foregoing embodiments that the chip provided by the embodiments of the present invention has a high audio and video processing capability, and therefore is conducive to improving the video fluency and smoothness of electronic devices, thereby helping to improve user perception and providing users with a higher visual experience.

[0149] Correspondingly, an embodiment of the present invention further provides a storage medium, wherein the storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the motion estimation method provided by the embodiment of the present invention.

[0150] The storage medium is a computer-readable storage medium, and the storage medium can be a read-only memory (ROM), a random access memory (RAM), a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk, or other medium that can store program codes.

[0151] The above-mentioned embodiments of the present invention are combinations of elements and features of the present invention. Unless otherwise mentioned, the elements or features may be considered as optional. Each element or feature may be put into practice without being combined with other elements or features. In addition, the embodiments of the present invention may be constructed by combining some elements and / or features. The order of operations described in the embodiments of the present invention may be rearranged. Some constructions of any one embodiment may be included in another embodiment, and may be replaced by the corresponding construction of another embodiment. It is obvious to those skilled in the art that claims that do not have a clear reference relationship to each other in the attached claims may be combined into embodiments of the present invention, or may be included as new claims in the amendment after submitting this application.

[0152] The embodiments of the present invention can be implemented by various means such as hardware, firmware, software or a combination thereof. In a hardware configuration, the method according to the exemplary embodiment of the present invention can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0153] In a firmware or software configuration, the embodiments of the present invention may be implemented in the form of modules, procedures, functions, etc. The software code may be stored in a memory unit and executed by a processor. The memory unit is located inside or outside the processor and may send data to and receive data from the processor via various known means.

[0154] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A motion estimation method, characterized in that: include: Providing original frames, including adjacent previous frames and current frames; Using the full search method, obtain the matching block corresponding to the current block of the current frame in the previous frame, and obtain the matching block corresponding to the current block of the previous frame in the current frame; According to the current block of the current frame and the corresponding matching block, a first candidate vector set corresponding to the corresponding block of the current frame is obtained, and according to the current block of the previous frame and the corresponding matching block, a second candidate vector set corresponding to the corresponding block of the previous frame is obtained; Setting a frame to be inserted between the previous frame and the current frame; According to the first candidate vector set and the second candidate vector set, a bidirectional motion vector corresponding to the current block of the frame to be interpolated is obtained, which is used as a candidate motion vector of the corresponding block of the frame to be interpolated. According to the first candidate vector set and the second candidate vector set, the step of obtaining the bidirectional motion vector corresponding to the current block of the frame to be interpolated includes: taking the current block of the frame to be interpolated as a reference, based on the first candidate vector set, obtaining backward motion vectors corresponding to all blocks in a preset search range of a corresponding position in the current frame as the first motion vector; Taking the current block of the frame to be inserted as a reference and based on the second candidate vector set, obtaining the forward motion vectors corresponding to all blocks within a preset search range at the corresponding position in the previous frame as the second motion vector; obtaining the first motion vector and the second motion vector that pass through the current block and have the same direction as the bidirectional motion vector.

2. The motion estimation method according to claim 1, characterized in that: The step of obtaining the matching block includes: using a full search method to obtain a matching block corresponding to the current block based on a matching error and a distance from the current block.

3. The motion estimation method according to claim 2, characterized in that: The step of obtaining a matching block corresponding to the current block of the previous frame in the current frame includes: using a full search method to obtain a plurality of candidate blocks in the current frame whose matching errors with the current block are within a preset threshold range; and taking a candidate block with the smallest distance to the current block as the matching block; The step of obtaining a matching block corresponding to a current block of a current frame in a previous frame includes: using a full search method to obtain a plurality of candidate blocks in the previous frame whose matching errors with the current block are within a preset threshold range; and taking a candidate block with the smallest distance to the current block as the matching block.

4. The motion estimation method according to claim 1, characterized in that: The search step length of the full search method is greater than or equal to one pixel and less than the side length of the block.

5. The motion estimation method according to claim 1, characterized in that: The step of obtaining the first candidate vector set and the second candidate vector set includes: obtaining a backward motion vector of a corresponding block of the current frame according to a current block of the current frame and a corresponding matching block, wherein the backward motion vectors corresponding to a plurality of blocks of the current frame constitute the first candidate vector set; According to the current block of the previous frame and the corresponding matching block, a forward motion vector of the corresponding block of the previous frame is obtained, and the forward motion vectors corresponding to multiple blocks of the previous frame constitute the second candidate vector set.

6. The motion estimation method according to claim 1, characterized in that: When there are multiple pairs of bidirectional motion vectors, the step of obtaining candidate motion vectors of corresponding blocks of the to-be-interpolated frame further includes: comparing matching errors of blocks corresponding to the bidirectional motion vectors in the previous frame and the current frame; A bidirectional motion vector corresponding to the minimum matching error is obtained as a candidate motion vector of the corresponding block of the to-be-interpolated frame.

7. The motion estimation method according to claim 1, characterized in that: The motion estimation method further includes: before obtaining a matching block corresponding to a current block of a current frame in a previous frame and obtaining a matching block corresponding to a current block of a previous frame in a current frame by using a full search method, identifying blocks corresponding to a small moving object in a previous frame and a current frame; By using the full search method, the matching block corresponding to the block of the small moving object in the current frame in the previous frame is obtained, and the matching block corresponding to the block of the small moving object in the previous frame in the current frame is obtained.

8. The motion estimation method according to claim 7, characterized in that: Using image segmentation techniques, blocks corresponding to small moving objects in the previous and current frames are identified.

9. A chip, characterized in that: A motion estimation module is included, wherein the motion estimation module includes: A providing unit, used for providing an original frame, including an adjacent previous frame and a current frame; A search unit, used to obtain a matching block corresponding to a current block of a current frame in a previous frame, and to obtain a matching block corresponding to a current block of a previous frame in a current frame, by using a full search method; A calculation unit, configured to obtain a first candidate vector set corresponding to a corresponding block of a current frame according to a current block of a current frame and a corresponding matching block, and to obtain a second candidate vector set corresponding to a corresponding block of a previous frame according to a current block of a previous frame and a corresponding matching block; A setting unit, used for setting a frame to be inserted between the previous frame and the current frame; an acquiring unit, configured to acquire a bidirectional motion vector corresponding to a current block of the frame to be interpolated according to the first candidate vector set and the second candidate vector set output by the calculating unit, and output the bidirectional motion vector as a candidate motion vector of a corresponding block of the frame to be interpolated; The acquisition unit includes: an extraction unit, which is used to obtain, based on the first candidate vector set, backward motion vectors corresponding to all blocks in a preset search range at a corresponding position in the current frame by taking the current block of the frame to be interpolated as a reference, as a first motion vector, and to obtain, based on the second candidate vector set, forward motion vectors corresponding to all blocks in a preset search range at a corresponding position in a previous frame by taking the current block of the frame to be interpolated as a reference, as a second motion vector; and a screening unit, which is used to obtain, as the bidirectional motion vector, the first motion vector and the second motion vector that pass through the current block and have the same direction.

10. The chip according to claim 9, characterized in that The calculation unit includes: a first sub-calculation unit, configured to obtain a backward motion vector of a corresponding block of the current frame according to a current block of the current frame and a corresponding matching block, and output backward motion vectors corresponding to a plurality of blocks of the current frame as the first candidate vector set; The second sub-calculation unit is used to obtain the forward motion vector of the corresponding block of the previous frame according to the current block and the corresponding matching block of the previous frame, and output the forward motion vectors corresponding to multiple blocks of the previous frame as the second candidate vector set.

11. The chip according to claim 9, characterized in that: The acquisition unit also includes: a comparison unit, which is used to compare the matching errors of the blocks corresponding to the bidirectional motion vectors in the previous frame and the current frame when the bidirectional motion vectors output by the screening unit are multiple pairs, and output the bidirectional motion vector corresponding to the minimum matching error as a candidate motion vector of the corresponding block of the frame to be inserted.

12. The chip according to claim 9, characterized in that: The motion estimation module further includes: an identification unit for identifying blocks corresponding to the moving small object in the previous frame and the current frame; The search unit is used to obtain the matching block corresponding to the block of the small moving object in the current frame in the previous frame and obtain the matching block corresponding to the block of the small moving object in the previous frame in the current frame by using the full search method.

13. The chip according to claim 9, characterized in that: The chip further comprises: a motion compensation module, configured to perform motion compensation on the block to be inserted according to the candidate motion vector output by the motion estimation module.

14. An electronic device, characterized in that: Comprising a chip as described in any one of claims 9 to 13.

15. The electronic device according to claim 14, characterized in that: The electronic device is a set-top box or a display device.

16. A storage medium, characterized in that: The storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the motion estimation method according to any one of claims 1-8.

Images