Image motion estimation method and related device

Abstract

The present application discloses an image motion estimation method and a related device, which relate to the field of information processing, including: for each coding block divided in a first manner to obtain a first prediction unit, based on each motion vector of an object in two adjacent frames of images, for each coding block, a motion search is performed with each pixel in the first prediction unit to obtain a first difference parameter set, and each first difference parameter corresponds to a motion vector difference of a pixel in the first prediction unit; for each coding block divided in a second manner to obtain each second prediction unit, based on the first difference parameter set, for each coding block, a motion search is performed with each pixel in the second prediction unit to obtain a second difference parameter set, and each second motion vector parameter corresponds to a motion vector difference of a pixel in the second prediction unit; based on the first difference parameter set and the second difference parameter set of each coding block, a target prediction unit is determined, and the difference parameter of the target prediction unit is superior to the difference parameter of the non-target prediction unit.

Description

Technical Field

[0001] The present application relates to the field of information processing, and in particular to an image motion estimation method and related devices. Background Art

[0002] Video compression can improve compression efficiency by eliminating redundant video information in the time domain through inter-frame prediction, predict the current image macroblock by searching the motion vector (mv) of the object in the image, find the most suitable prediction unit by traversing and dividing the image macroblock, and match the best block through precise subpixel motion search to reduce residuals and save the bit-rate to be transmitted.

[0003] In the existing scheme, the image macroblock is traversed and divided into four prediction units, such as Figure 1 It is a schematic diagram of prediction units divided into image macroblocks, and the prediction units are 2N×2N, 2N×N, N×2N and N×N, where N represents N pixels and N is an integer.

[0004] Among them, first, for each motion vector in each prediction unit, the optimal sub-pixel mv (refinedmv) is searched based on the SATD (Sum of Absolute Transformed Difference) cost (loss) minimum principle; then, for all motion vectors in each prediction unit, the optimal first N sub-pixel MVs are selected based on the SATD cost minimum principle; the optimal first M prediction units are selected from all prediction units based on the SATD cost minimum principle, and finally, the optimal first M prediction units and the corresponding sub-pixel MVs are used as the output of motion estimation.

[0005] However, in order to meet the encoding performance requirements, the time for processing each macroblock (16×16) is less than 200 cycles. The searches of 2N×2N, 2N×N and N×2N require the SATD calculation of 9 points, which becomes a performance bottleneck due to limited hardware resources. Therefore, each prediction unit is usually limited to processing sub-pixel searches of two MVs at most. However, the current sub-pixel point (pu, mv) may be one of the optimal MVs. Due to the limitation on the number of MVs, the optimal MV is missed and the search accuracy is lost, which affects the compression efficiency. Summary of the invention

[0006] The first aspect of the present application provides an image motion estimation method, comprising:

[0007] Obtain the motion vectors of objects in any two adjacent frames of the video;

[0008] Obtaining a number of coding blocks corresponding to the object motion vectors in any two adjacent frames of images;

[0009] For each coding block, a first prediction unit is obtained by dividing the coding block, and based on each motion vector of the object in two adjacent frames of images, a motion search is performed on each pixel in the first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit, each first difference parameter corresponds to a motion vector difference of a pixel in the first prediction unit of the two adjacent frames of images, and the first prediction unit is obtained by dividing the coding block in a first manner;

[0010] At least two second prediction units are obtained by dividing each coding block, and based on the first difference parameter set, motion search is performed on each pixel in the second prediction unit for each coding block to obtain a second difference parameter set for each motion vector in each second prediction unit, each second motion vector parameter corresponds to a motion vector difference of a pixel in the second prediction unit of the two adjacent frames of image, and the second prediction unit is obtained by dividing the coding block in a second manner, and the second manner is different from the first manner;

[0011] Saving a first difference parameter set and a second difference parameter set for each coding block;

[0012] Based on a first difference parameter set and a second difference parameter set for each coding block, a target prediction unit is determined, where the target prediction unit is a prediction unit in a first image that motion matches a second image, the first image is adjacent to the second image and is generated later than the second image, and the difference parameter of the target prediction unit is better than the difference parameter of a non-target prediction unit.

[0013] In a possible implementation, based on each motion vector of an object in two adjacent frames of images, performing motion search with each pixel in a first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit includes:

[0014] Based on each motion vector of the object in two adjacent frames of images, searching in the first prediction unit with at least two search accuracies to obtain a difference parameter of each pixel in the first prediction unit at the corresponding at least two search accuracies;

[0015] Based on the difference parameters of the pixels in the first prediction unit at the corresponding at least two search accuracies, a target search accuracy pixel of the corresponding accuracy is obtained, wherein the difference parameters of the target search accuracy pixel in the first prediction unit are better than the difference parameters of the non-target search accuracy pixel;

[0016] The difference parameters of each pixel in the first prediction unit at corresponding at least two search accuracies form a first difference parameter set;

[0017] Among them, the first target search accuracy pixel is determined based on the first search accuracy, and the second target search accuracy pixel is determined based on the second search accuracy searched in the first target search accuracy pixel, the second search accuracy and the first search accuracy are two adjacent search accuracy, and the second search accuracy is greater than the first search accuracy.

[0018] In a possible implementation, the dividing each coding block to obtain at least two second prediction units, performing motion search on each pixel in the second prediction unit for each coding block based on the first difference parameter set, to obtain a second difference parameter set for each motion vector in each second prediction unit, includes:

[0019] Based on the position of each pixel included in the second prediction unit, determining the pixel at the corresponding position in the first prediction unit;

[0020] Based on the target difference parameters of each pixel at the corresponding position in the first prediction unit, a second difference parameter set of each motion vector in each second prediction unit is accumulated, and the target difference parameter is a difference parameter searched for the search accuracy corresponding to the second method.

[0021] In a possible implementation, the number of pixels included in the first prediction unit is greater than the number of pixels included in any second prediction unit.

[0022] In a possible implementation, before saving the first difference parameter set and the second difference parameter set of each coding block, the method includes:

[0023] Based on the second difference parameter set of each coding block, determine a target second prediction unit in the second prediction unit of each coding block, wherein the difference parameter of the motion vector in the target second prediction unit is better than the difference parameter of the motion vector in the non-target second prediction unit in each coding block;

[0024] The motion vector of the target second prediction unit is used as the motion vector of the object in the two adjacent frames of images.

[0025] In a possible implementation, the storing the first difference parameter set and the second difference parameter set of each coding block includes:

[0026] Determine a coding block division method corresponding to the first difference parameter set and the second difference parameter set;

[0027] Based on the corresponding coding block division method, the first difference parameter set and the second difference parameter set are respectively stored in the storage space of the corresponding division method.

[0028] In a possible implementation, determining a target prediction unit in the plurality of coding blocks based on the first difference parameter set and the second difference parameter set of each coding block includes:

[0029] For the first prediction unit and the second prediction unit of each coding block, respectively obtain a difference parameter set of each prediction unit;

[0030] Determine, according to the difference parameter set of each prediction unit, a third prediction unit in the corresponding coding block, wherein the difference parameter of the third prediction unit is better than the difference parameter of the non-third prediction unit in the corresponding coding block;

[0031] A target prediction unit is determined according to the third prediction unit in each coding block, wherein a difference parameter of the target prediction unit is better than a difference parameter of a non-target prediction unit.

[0032] A second aspect of the present application provides an electronic device, including an image motion estimation device, wherein the image motion estimation device includes:

[0033] The first acquisition module is used to obtain each motion vector of an object in any two adjacent frames of images in a video;

[0034] A second obtaining module is used to obtain a number of coding blocks corresponding to the object motion vectors in any two adjacent frames of images;

[0035] A first search module is used to obtain a first prediction unit by dividing each coding block, and based on each motion vector of an object in two adjacent frames of images, perform motion search with each pixel in the first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit, each first difference parameter corresponding to a motion vector difference of a pixel in the first prediction unit of the two adjacent frames of images, wherein the first prediction unit is obtained by dividing the coding block in a first manner;

[0036] a second search module, configured to obtain at least two second prediction units by dividing each coding block, and based on the first difference parameter set, perform motion search on each pixel in the second prediction unit for each coding block to obtain a second difference parameter set for each motion vector in each second prediction unit, wherein each second motion vector parameter corresponds to a motion vector difference of a pixel in the second prediction unit of the two adjacent frames of images, and the second prediction unit is obtained by dividing the coding block in a second manner, and the second manner is different from the first manner;

[0037] A data storage module, used for storing a first difference parameter set and a second difference parameter set for each coding block;

[0038] A determination module is used to determine a target prediction unit based on a first difference parameter set and a second difference parameter set for each coding block, wherein the target prediction unit is a prediction unit in a first image that motion matches a second image, the first image is adjacent to the second image and is generated later than the second image, and the difference parameter of the target prediction unit is better than the difference parameter of the non-target prediction unit.

[0039] In a possible implementation, the data storage module includes:

[0040] At least two storage spaces, each storage space is used to store difference parameters corresponding to the division method.

[0041] In a possible implementation, the data storage module further includes:

[0042] at least two accumulators, one accumulator corresponding to a coding block division mode of the second prediction unit;

[0043] Among them, the accumulator accumulates the target difference parameters of the pixels at the corresponding positions in the first prediction unit for the positions of each pixel in the corresponding second prediction unit, and obtains a second difference parameter set of each motion vector in the corresponding second prediction unit, and the target difference parameter is the difference parameter obtained by searching for the search accuracy corresponding to the second method.

[0044] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the image motion estimation method of the first aspect or any implementation of the first aspect.

[0045] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0046] The memory is used to store computer programs;

[0047] The processor is used to execute the computer program so that the electronic device can implement the image motion estimation method of the first aspect or any implementation manner of the first aspect.

[0048] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the image motion estimation method of the first aspect or any implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0050] Figure 1 It is a schematic diagram of the prediction unit divided into image macroblocks;

[0051] Figure 2 It is a flowchart of an image motion estimation method provided by an embodiment of the present application;

[0052] Figure 3 It is a flowchart of performing motion search with each pixel in the first prediction unit for each coding block based on each motion vector of an object in two adjacent frames of images, and obtaining a first difference parameter set corresponding to each motion vector in the first prediction unit, provided by an embodiment of the present application;

[0053] Figure 4 is a schematic diagram of a process of searching for a first prediction unit provided in an embodiment of the present application;

[0054] Figure 5 The present invention provides a flowchart of dividing each coding block into at least two second prediction units, performing motion search on each pixel in the second prediction unit for each coding block based on the first difference parameter set, and obtaining a second difference parameter set for each motion vector in each second prediction unit;

[0055] Figure 6 It is a schematic diagram of a process of determining a prediction unit corresponding to an optimal motion vector in a motion search process provided by an embodiment of the present application;

[0056] Figure 7 It is a schematic diagram of a process of saving a first difference parameter set and a second difference parameter set for each coding block provided in an embodiment of the present application;

[0057] Figure 8 is a schematic diagram of a process of determining a target prediction unit in a plurality of coding blocks based on a first difference parameter set and a second difference parameter set for each coding block provided by an embodiment of the present application;

[0058] Fig. 9 is a structural schematic diagram of an image motion estimation device provided in an embodiment of the present application;

[0059] Fig.10 This is a structural schematic diagram of a data storage module provided by an embodiment of the present application;

[0060] Fig.11is another structural schematic diagram of a data storage module provided in an embodiment of the present application;

[0061] Fig.12 This is a schematic diagram of an application scenario of an image motion estimation method provided by the present application;

[0062] Fig.13 It is a structural schematic diagram of an electronic device provided by this application. DETAILED DESCRIPTION

[0063] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0064] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0065] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0066] Reference Figure 2 , Figure 2 is a flow chart of an image motion estimation method provided by an embodiment of the present application, such as Figure 2 As shown, an image motion estimation method provided by an embodiment of the present application may include steps 201 to 206, and these steps are described in detail below.

[0067] 201. Obtain motion vectors of objects in any two adjacent frames of images in a video;

[0068] The motion vector may be an inter-frame motion vector hint mv.

[0069] The motion vector may be any one of five parameters, namely, temporal information, spatial information, global information, constant information, and external information.

[0070] The motion vectors of the objects in the two frames of images obtained may be the same parameters or different parameters, which is not limited in the present application.

[0071] 202. Obtain a number of coding blocks corresponding to the object motion vectors in any two adjacent frames of images;

[0072] The coding block may be a macroblock, which is divided into a plurality of macroblocks for two adjacent frames of images. The macroblock is used as a coding block to provide a basis for subsequent motion search of a motion vector.

[0073] 203. For each coding block, a first prediction unit is obtained by dividing the coding blocks. Based on each motion vector of an object in two adjacent frames of images, a motion search is performed on each pixel in the first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit, where each first difference parameter corresponds to a motion vector difference of a pixel in the first prediction unit of the two adjacent frames of images. The first prediction unit is obtained by dividing the coding blocks in the first manner.

[0074] The coding block may be divided into various prediction units PU in different ways.

[0075] There are multiple motion vectors in the same coding block. The motion vectors can be used to perform motion search on each pixel in the first prediction module for the coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit.

[0076] The first difference parameter set includes the results of motion search at multiple search accuracies for each pixel in the first prediction unit, specifically, difference parameters at multiple search accuracies.

[0077] Among them, the difference parameters involved in the embodiments of the present application can be represented by SATD (Sum of Absolute Transformed Difference, sum of absolute values ​​after Hammond transformation) value.

[0078] When performing motion search on each pixel in the first prediction unit for the coding block, multiple search accuracies are required to obtain corresponding difference parameters. Figure 3 The process of performing motion search in the first prediction unit is described in detail.

[0079] In a specific implementation, the SATD value corresponding to each motion vector is calculated for all the motion vectors in the first prediction unit.

[0080] The difference parameter is the SATD value of the 4×4 block in each prediction unit, and correspondingly, the difference parameter set is a set of SATD values ​​of each 4×4 block in the corresponding prediction unit.

[0081] Among them, for the same motion vector, for different prediction units, its difference parameters can be obtained by accumulating SATD4×4, and the 4x4 is the basic unit of Hadamard transform in SATD. Therefore, in the present application, when performing motion search for each pixel in the first prediction unit, multiple search precisions are used, and the difference parameters corresponding to various search precisions are recorded to provide a difference parameter basis for the subsequent motion search of the prediction units obtained by other division methods.

[0082] 204. Divide each coding block to obtain at least two second prediction units. Based on the first difference parameter set, perform motion search on each pixel in the second prediction unit for each coding block to obtain a second difference parameter set for each motion vector in each second prediction unit, where each second motion vector parameter corresponds to a motion vector difference of a pixel in the second prediction unit of two adjacent frames of images. The second prediction unit is obtained by dividing the coding block in a second manner, where the second manner is different from the first manner.

[0083] The coding block is divided in a second manner to obtain a plurality of second prediction units, where the second prediction units generally refer to other prediction units except the first prediction units divided in the first manner.

[0084] In a possible implementation, the number of pixels included in the first prediction unit is greater than the number of pixels included in any second prediction unit.

[0085] Refer to the above Figure 1 , the prediction units that can be divided for the coding block are 2N×2N, 2N×N, N×2N and N×N, where N represents N pixels and N is an integer.

[0086] Among them, the first method can be divided according to 2N×2N, and the second method can be divided according to N×2N, 2N×N or N×N; accordingly, the first prediction unit is a 2N×2N PU, and the second prediction unit can be an N×2N, 2N×N and N×N PU.

[0087] When performing motion search on each pixel in the second prediction unit, the difference parameters in the first difference parameter set obtained by performing motion search on the first prediction unit are used to obtain the second difference parameter set of the second prediction unit.

[0088] Among them, since a variety of search accuracies are used to search and obtain corresponding difference parameters when performing motion search for each pixel in the first prediction unit, correspondingly, when performing motion search for each pixel in the second prediction unit, the difference parameters obtained by searching with various search accuracies in the first preset unit can be directly used without further search calculation.

[0089] In a possible implementation, based on the correspondence between each pixel in the second prediction unit and each pixel in the first prediction unit, a second difference parameter set is accumulated based on the first difference parameter set, and then Figure 5 The process of obtaining the second difference parameter set is described in detail.

[0090] For the second prediction units obtained by different second division methods, motion search is performed on pixels therein respectively to obtain a corresponding second difference parameter set.

[0091] Among them, for the second prediction unit obtained by N×2N division, motion search is performed on each pixel therein to obtain a corresponding N×2N second difference parameter set; for the second prediction unit obtained by 2N×N division, motion search is performed on each pixel therein to obtain a corresponding 2N×N second difference parameter set; for the second prediction unit obtained by N×N division, motion search is performed on each pixel therein to obtain a corresponding N×N second difference parameter set.

[0092] Among them, since the SATD of different prediction units for the same motion vector can be accumulated by SATD4×4, therefore, when performing motion search for the first prediction unit, SATD4×4 has been calculated for various search accuracy pixels. Accordingly, when performing motion search for the second prediction unit subsequently, it can be directly calculated based on the SATD4×4 obtained from the aforementioned search for the first prediction unit.

[0093] When performing motion search on each pixel in the second prediction unit, motion search with multiple search precisions may also be performed. The search process is all accumulated based on the first difference parameter set of the first prediction unit, without the need for an actual search calculation process.

[0094] In a possible implementation, when performing motion search for each pixel in each prediction unit, the optimal motion vector in the coding block can be determined based on the obtained difference parameter set, and then the prediction unit with the optimal motion vector in the coding block is determined. Figure 6 The process of determining the prediction unit of the optimal motion vector during the motion search process is described in detail.

[0095] 205. Save a first difference parameter set and a second difference parameter set for each coding block;

[0096] The first difference parameter set and the second difference parameter set obtained by performing motion search on each coding block are stored respectively.

[0097] In a possible implementation, for different partitioning methods, the difference parameter sets of each prediction unit are stored separately. Figure 7 The stored procedure is described in detail in.

[0098] 206. Based on the first difference parameter set and the second difference parameter set of each coding block, determine a target prediction unit, where the target prediction unit is a prediction unit in the first image that motion matches the second image, the first image is adjacent to the second image and is generated later than the second image, and the difference parameter of the target prediction unit is better than the difference parameter of the non-target prediction unit.

[0099] A plurality of prediction units are obtained based on a plurality of division modes in each coding block, and motion searches with a plurality of search precisions are performed for each prediction unit, so that difference parameter sets with a plurality of precisions can be obtained.

[0100] Correspondingly, among the difference parameter sets obtained by motion search for the multiple prediction units, the motion vector corresponding to the optimal difference parameter set in each prediction unit can be determined, and a target prediction unit is determined based on the various difference parameter sets. The target prediction unit is a prediction unit in the first image that motion matches the second image, and the difference parameters in the target prediction unit are better than the difference parameters of other prediction units.

[0101] In a possible implementation, the target coding block is a target coding block including a prediction unit having an optimal difference parameter.

[0102] In a possible implementation, for each coding block, a first difference parameter set and a second difference parameter set may be used to determine a target prediction unit with the best difference parameter in the first image. Figure 8 The process of determining the target prediction unit is described in detail.

[0103] After the target prediction unit is determined based on the above steps, image compression can be performed based on the target prediction unit.

[0104] Accordingly, the vector parameters of the target prediction unit are used as the optimal vector parameters in the frame image, and the image is pixel compressed according to the optimal vector parameters to reduce the amount of compressed data and improve the compression efficiency.

[0105] In this embodiment, after a number of coding blocks corresponding to the motion vectors of the objects in any two adjacent images are targeted, the coding blocks are divided in a first manner to obtain a first prediction unit, and based on the motion vectors of the objects in the two adjacent frames of images, a motion search is performed on each pixel in the first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit; then, when a motion search is performed on each pixel in the second prediction unit obtained by dividing the coding blocks in a second manner, the motion search is performed based on the first difference parameter set to obtain a second difference parameter set corresponding to each motion vector in the second prediction unit, the first difference parameter set and the second difference parameter set are saved, and a target prediction unit is determined based on the first difference parameter set and the second difference parameter set of each coding block. The target prediction unit is a prediction unit in the first image that matches the motion of the second image, and its difference parameter is better than the difference parameter of the non-target coding block in the first image, thereby reducing the amount of data processing when performing motion search on each pixel in the second prediction unit, and only substantial search calculation needs to be performed when performing motion search on each pixel in the first prediction unit, thereby reducing the data processing burden of the entire image motion estimation process.

[0106] Figure 3 The present application provides a flowchart of performing motion search on each pixel in the first prediction unit for each coding block based on each motion vector of an object in two adjacent frames of images to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit. The flowchart may include steps 301 to 303, and these steps are described in detail below.

[0107] 301. Based on each motion vector of an object in two adjacent frames of images, search in the first prediction unit with at least two search accuracies to obtain a difference parameter of each pixel in the first prediction unit at the corresponding at least two search accuracies;

[0108] In this embodiment, a specific implementation process of performing motion search on each pixel in the first prediction unit for the coding block is described.

[0109] When performing motion search on each pixel in any prediction unit, multiple search precisions are used, and the types of search precisions can be set according to the situation.

[0110] In this application, full pixel search, 1 / 2 pixel search and 1 / 4 pixel search are used as examples for explanation. In the specific implementation, a higher search accuracy can be set, such as 1 / 8, etc. This application does not limit the specific type of search accuracy.

[0111] For example, when the first prediction unit is 2N×2N, a full pixel search, a 1 / 2 pixel search, and a 1 / 4 pixel search are performed on each pixel in the first prediction unit, and accordingly, a difference parameter corresponding to each search accuracy is obtained.

[0112] Among them, when searching for 1 / 2 pixels, interpolation is performed on the full pixels to obtain a motion vector of 1 / 2 pixels, so as to calculate the difference parameters based on the motion vector of 1 / 2 pixels; when searching for 1 / 4 pixels, interpolation is performed on the 1 / 2 pixels to obtain a motion vector of 1 / 4 pixels, so as to calculate the difference parameters based on the motion vector of 1 / 4 pixels.

[0113] 302. Obtain target search precision pixels of corresponding precision based on difference parameters of at least two corresponding search precisions of each pixel in the first prediction unit, wherein the difference parameters of the target search precision pixels in the first prediction unit are better than the difference parameters of the non-target search precision pixels;

[0114] Among them, the first target search accuracy pixel is determined based on the first search accuracy, and the second target search accuracy pixel is determined based on the second search accuracy searched in the first target search accuracy pixel, the second search accuracy and the first search accuracy are two adjacent search accuracy, and the second search accuracy is greater than the first search accuracy.

[0115] The pixels in the first prediction unit are searched sequentially with the set search precision to obtain corresponding difference parameters.

[0116] After obtaining the difference parameter of the first prediction unit at a certain search accuracy, a target search accuracy pixel is determined among several pixels of the first prediction unit according to the difference parameter, and the difference parameter of the target search accuracy pixel is better than the difference parameters of other pixels in the first prediction unit.

[0117] Among them, the number of pixels determined for target search accuracy can be multiple, and the number can be set according to actual conditions, and is not limited in this application.

[0118] Among them, when a search of a certain search accuracy is performed on the pixels in the first prediction unit and the target search accuracy pixels of the corresponding accuracy are determined, a search of a higher search accuracy is performed on the target search accuracy pixels based on the target search accuracy pixels to determine the target search accuracy pixels of the corresponding accuracy, and so on, until the set maximum search accuracy is reached, and the target search accuracy pixels corresponding to the maximum search accuracy are obtained.

[0119] As an example, the maximum search accuracy is set to 1 / 4 pixel search. For all pixels of the first prediction unit, a full pixel search is first used to determine the optimal full pixels for the set number SATD. Then, for the determined full pixels, a 1 / 2 pixel search is used to determine the optimal 1 / 2 pixel for the set number SATD. Then, for the determined 1 / 2 pixels, a 1 / 4 pixel search is used to determine the optimal 1 / 4 pixel for the set number SATD.

[0120] Among them, full pixel is the basic pixel unit in the image, which is used to describe the basic resolution of the image; 1 / 2 pixel is a smaller unit between full pixels, and 1 / 4 pixel is a smaller unit between half pixels, which is generally used for fine operations in image processing, such as motion estimation and difference calculation.

[0121] 303. Difference parameters of pixels in the first prediction unit corresponding to at least two search accuracies form a first difference parameter set.

[0122] A search with various search accuracies is performed on each pixel in the first prediction unit to obtain difference parameters with corresponding accuracies, and the difference parameters with various search accuracies are combined to obtain a first difference parameter set.

[0123] In a possible implementation, difference parameters are obtained for pixels of various search precisions respectively. Accordingly, the difference parameters obtained for various search precisions can be grouped into subsets, and the subsets are combined to obtain a first difference parameter set.

[0124] Figure 4 It is a schematic diagram of the process of searching for the first prediction unit provided in an embodiment of the present application, in which the first prediction unit is obtained by dividing the coding block into 8×8.

[0125] (a) shows a full pixel search for the first prediction unit, where I represents a full pixel, and a motion search is performed on each full pixel in the first prediction unit to obtain a corresponding SATD value. The optimal full pixel determined in (a) is marked in gray.

[0126] (b) shows a 1 / 2 pixel search for the first prediction unit. In this figure, H represents a 1 / 2 pixel. A 1 / 2 pixel is inserted between full pixels, and a motion search is performed on the 1 / 2 pixel to obtain the corresponding SATD value. In this figure, after multiple full pixels with a small SATD cost are determined, a 1 / 2 pixel search is performed on the multiple full pixels. The optimal 1 / 2 pixel determined in (b) is marked in gray.

[0127] (c) shows a 1 / 4 pixel search for the first prediction unit. In the figure, Q represents a 1 / 4 pixel. A 1 / 4 pixel is inserted between 1 / 2 pixels, and a motion search is performed on the 1 / 4 pixel to obtain a corresponding SATD value. In this case, after determining multiple 1 / 2 pixels with a small SATD cost, a 1 / 4 pixel search is performed on the multiple 1 / 2 pixels.

[0128] Among them, (d) shows the motion vector of the first prediction unit, specifically the vector difference of the corresponding pixel points in the two images, and the dn (n is 0-7) marked in each small grid represents the motion vector information of the corresponding pixel (full pixel, 1 / 2 pixel and 1 / 4 pixel). In (d), each SATD4×4 is framed with a bold solid line.

[0129] For the first prediction unit of 8×8, its SATD value is obtained by accumulating each SATD 4×4 therein.

[0130] Among them, since each prediction unit performs searches with multiple search accuracies and adopts the same search method, the difference parameters of various search accuracies generated during the search of the first prediction unit in this application will be directly applied to the search process of the second prediction unit, and only the same search accuracies need to be applied.

[0131] In a possible implementation, in order to reduce the data storage burden, the calculated SATD4×4 value and the corresponding pixel position in the first prediction unit can be recorded. When performing a motion search for the second prediction unit later, the difference parameter set can be directly obtained by accumulating the SATD4×4 value and the corresponding pixel.

[0132] In this embodiment, based on each motion vector of the object in two adjacent frames of images, the first prediction unit is searched with at least two search accuracies to obtain difference parameters of each pixel in the first prediction unit at the corresponding at least two search accuracies; based on the difference parameters of each pixel in the first prediction unit at the corresponding at least two search accuracies, a target search accuracy pixel of the corresponding accuracy is obtained, and the difference parameters of the target search accuracy pixel in the first prediction unit are better than the difference parameters of the non-target search accuracy pixel; the difference parameters of each pixel in the first prediction unit at the corresponding at least two search accuracies are combined into a first difference parameter set; wherein the first target search accuracy pixel is determined based on the first search accuracy, the second target search accuracy pixel is determined by searching the first target search accuracy pixel based on the second search accuracy, the second search accuracy and the first search accuracy are two adjacent search accuracies, and the second search accuracy is greater than the first search accuracy, a motion vector search with multiple search accuracies is provided for each pixel in the first prediction unit, and the difference parameters determined by the search are stored, providing a data basis for the subsequent motion vector search of the second prediction unit.

[0133] Figure 5The embodiment of the present application provides a flowchart for obtaining at least two second prediction units for each coding block, performing motion search on each pixel in the second prediction unit for each coding block based on the first difference parameter set, and obtaining a second difference parameter set for each motion vector in each second prediction unit. The flowchart may include steps 501 to 502, and these steps are described in detail below.

[0134] 501. Determine pixels at corresponding positions in the first prediction unit based on positions of pixels contained in the second prediction unit.

[0135] In the above steps, a motion search is performed on each pixel in the first prediction unit to obtain its difference parameter.

[0136] Since the SATD of different prediction units for the same motion vector can be accumulated by SATD4×4, when the motion search is performed on the first prediction unit, SATD4×4 has been calculated for pixels of various search precisions. Accordingly, when the motion search is performed on the second prediction unit subsequently, it can be directly calculated based on the SATD4×4 obtained from the aforementioned search for the first prediction unit.

[0137] The second prediction unit and the first prediction unit belong to the same coding block, and both can correspond to the same pixel.

[0138] Wherein, based on the position of each pixel in the second prediction unit, the pixel at the corresponding position is determined in the first prediction unit.

[0139] refer to Figure 4 The second prediction unit includes part of the pixels in the first prediction unit. For example, when the second prediction unit is N×2N, it may be the left half or the right half of the first prediction unit. The pixels in the second prediction unit overlap with the pixels in the first prediction unit.

[0140] 502. Based on target difference parameters of each pixel at a corresponding position in the first prediction unit, accumulate and obtain a second difference parameter set of each motion vector in each second prediction unit, where the target difference parameter is a difference parameter obtained by searching for a search accuracy corresponding to the second method.

[0141] Wherein, based on the position correspondence between the second prediction unit and the first prediction unit, and the difference parameter has been determined in the first prediction unit, the target difference parameter corresponding to the second prediction unit can be determined.

[0142] Correspondingly, the target difference parameters corresponding to the second prediction unit are accumulated to obtain the second difference parameters of each motion vector therein.

[0143] Combination Figure 4(d) in the step may determine the corresponding SATD4×4 value for each pixel at the corresponding position in the second prediction unit. When searching for each pixel in the second prediction unit, the values ​​of each SATD4×4 area included in the second prediction unit may be accumulated to obtain its second difference parameter set.

[0144] In a possible implementation, a 1 / 2 pixel search, a 1 / 4 pixel search, etc. may be performed for the second prediction unit, which is consistent with the search accuracy of the first prediction unit. The present application does not limit the search accuracy of the second prediction unit.

[0145] In this embodiment, based on the position of each pixel included in the second prediction unit, the pixel at the corresponding position in the first prediction unit is determined; based on the target difference parameter of each pixel at the corresponding position in the first prediction unit, a second difference parameter set of each motion vector in each second prediction unit is accumulated, and the target difference parameter is a difference parameter obtained by searching for the search accuracy corresponding to the second method. Since the first prediction unit has obtained the corresponding difference parameter from multiple search accuracy during the motion search process, when performing motion search on the second prediction unit, the difference parameter obtained by the first prediction unit is directly used for calculation, and there is no need to perform actual motion vector search calculation, thereby reducing the data processing burden.

[0146] Figure 6 This is a flow chart of determining a prediction unit corresponding to an optimal motion vector during a motion search process provided by an embodiment of the present application, which may include steps 601 to 602. These steps are described in detail below.

[0147] 601. Based on the second difference parameter set of each coding block, determine a target second prediction unit in the second prediction unit of each coding block, wherein the difference parameter of the motion vector in the target second prediction unit is better than the difference parameter of the motion vector in the non-target second prediction unit in each coding block;

[0148] Among them, since the first prediction unit includes the largest number of pixels, the difference parameter of the motion vector thereof is poorer than that of each second prediction unit. Therefore, the optimal prediction unit may be determined only for each second prediction unit.

[0149] Wherein, for each coding block, a target second prediction unit is determined, and the target second prediction unit may be one or more.

[0150] Among them, for all motion vectors in each second prediction unit, based on the SATD cost minimum principle, the second difference parameter set corresponding to the second prediction unit can be used to determine the optimal specific number of sub-pixel motion vectors in each second prediction unit; then, among all the second prediction units, based on the SATD cost minimum principle, the optimal multiple target second prediction units are determined, and the SATD cost of the target second prediction unit is the optimal multiple among all the second prediction units.

[0151] 602. Use the motion vector of the target second prediction unit as the motion vector of the object in the two adjacent image frames.

[0152] After the target second prediction unit is determined, the SATD cost of the target second prediction unit is the smallest, so the target second prediction unit in the corresponding two frames of images is most similar, and accordingly, the motion vector of the target second prediction unit is optimal.

[0153] The motion vector of the target second prediction unit is used as the motion vector of the corresponding object in the two adjacent needle images.

[0154] In this embodiment, after performing motion search on each pixel of each prediction unit, a prediction unit with the best difference parameter among the second prediction units is selected according to the second difference parameter set of each coding block, thereby implementing a process of performing motion search on each prediction unit and determining the optimal multiple prediction units.

[0155] Figure 7 It is a flowchart of saving the first difference parameter set and the second difference parameter set of each coding block provided by an embodiment of the present application, which may include steps 701 to 702, and these steps are described in detail below.

[0156] 701. Determine a coding block division method corresponding to a first difference parameter set and a second difference parameter set;

[0157] Among them, various prediction units with different division methods are adopted for the coding block, and when searching for the motion vector of each pixel in the prediction unit, a corresponding difference parameter set of each prediction unit is obtained.

[0158] The coding block may be divided into 2N×2N, 2N×N, N×2N and N×N. Accordingly, a motion search is performed for each division method to obtain a corresponding difference parameter set.

[0159] For the first prediction unit 2N×2N, a motion search is performed on each pixel in the prediction unit to obtain a corresponding first difference parameter set, and the first difference parameter set is stored.

[0160] Among them, for the second prediction unit 2N×N, a motion search is performed on each pixel in the prediction unit to obtain its corresponding second difference parameter set, and the second difference parameter set is stored; for the second prediction unit N×2N, a motion search is performed on each pixel in the prediction unit to obtain its corresponding second difference parameter set, and the second difference parameter set is stored; for the second prediction unit N×N, a motion search is performed on each pixel in the prediction unit to obtain its corresponding second difference parameter set, and the second difference parameter set is stored.

[0161] Since different division modes can obtain different second prediction units, in this embodiment, the coding block division modes corresponding to the difference parameter sets are first determined to correspond the difference parameter sets to their prediction units.

[0162] 702. Based on a corresponding coding block division method, store the first difference parameter set and the second difference parameter set in storage spaces of the corresponding division methods respectively.

[0163] In the storage area, separate storage spaces are set for different coding block division methods.

[0164] Correspondingly, after performing motion search on any prediction unit and obtaining its difference parameter set, the difference parameter set can be stored in a corresponding storage space.

[0165] In one possible implementation, when searching for the first prediction unit 2N×2N, it is generally determined that the first prediction unit cannot be used as the target prediction unit, but one of the second prediction units is used as the target prediction unit. Accordingly, the first difference parameter set and the second difference parameter set may not be stored in the same storage area, but before the first difference parameter set is output to the storage space, the second difference parameter set of each second prediction unit is directly calculated and stored in the storage area.

[0166] Accordingly, in step 104, the process of performing motion search for each pixel in the second prediction unit for each coding block based on the first difference parameter set can be simplified to a process of calculating based on the first parameter set without actually performing motion search.

[0167] In this embodiment, in the storage area, a separate storage space is set for different coding block division methods. When a difference parameter set is received, each difference parameter set is stored in a storage space corresponding to the division method for the coding block division method corresponding to each difference parameter set, thereby realizing separate storage of difference parameter sets of different prediction units for easy query.

[0168] Figure 8It is a flowchart of determining a target prediction unit in several coding blocks based on a first difference parameter set and a second difference parameter set for each coding block provided by an embodiment of the present application, which may include steps 801 to 803, and these steps are described in detail below.

[0169] 801. For each first prediction unit and second prediction unit of each coding block, obtain a difference parameter set for each prediction unit respectively;

[0170] For various prediction units in each coding block, a difference parameter set for each prediction unit is determined respectively.

[0171] The difference parameter is the SATD value of the 4×4 block in each prediction unit, and correspondingly, the difference parameter set is a set of SATD values ​​of each 4×4 block in the corresponding prediction unit.

[0172] For the four prediction units of 2N×2N, 2N×N, N×2N and N×N, the corresponding difference parameter sets are obtained respectively.

[0173] 802. Determine, according to the difference parameter set of each prediction unit, a third prediction unit in the corresponding coding block, where the difference parameter of the third prediction unit is better than the difference parameter of the non-third prediction unit in the corresponding coding block;

[0174] In which, within the coding block, a third prediction unit is determined for the multiple prediction units divided therein.

[0175] The difference parameter in the prediction unit is obtained by searching the pixels therein with various search accuracies.

[0176] In a possible implementation, the optimal prediction unit is searched based on the SATD cost minimum principle, and the third prediction unit is a prediction unit corresponding to the optimal sub-pixel.

[0177] The smaller the difference parameter is, the more similar the prediction unit is to the corresponding prediction unit in the previous frame image.

[0178] 803. Determine a target prediction unit according to the third prediction unit in each coding block, wherein a difference parameter of the target prediction unit is better than a difference parameter of a non-target prediction unit.

[0179] Among them, for a frame image, it can be divided into several coding blocks, and for the third prediction unit determined in each coding block, a target prediction unit is determined, and the target prediction unit is a prediction unit whose difference parameter in the entire frame image is better than other non-target prediction units.

[0180] The number of target prediction units may be preset to m, where m is a positive integer. Accordingly, m target prediction units may be determined in one frame of image.

[0181] Among them, one or more prediction units may be determined for each coding block as the third prediction unit, so as to finally determine multiple optimal target prediction units.

[0182] The vector parameters of the target prediction unit are taken as the optimal vector parameters.

[0183] In this embodiment, for the first prediction unit and the second prediction unit of each coding block, a difference parameter set of each prediction unit is obtained respectively; according to the difference parameter set of each prediction unit, a third prediction unit in the corresponding coding block is determined, and the difference parameter of the third prediction unit is better than the difference parameter of the non-third prediction unit in the corresponding coding block; according to the third prediction unit in each coding block, a target prediction unit is determined, and the difference parameter of the target prediction unit is better than the difference parameter of the non-target prediction unit, so that a comparison is made within each coding block to determine the third prediction unit with better difference parameters, and then a comparison is made between the coding blocks included in the entire image to determine the final target prediction unit, so as to achieve the purpose of obtaining the target prediction unit in the image.

[0184] An image motion estimation method provided by an embodiment of the present application is introduced above. An electronic device for executing the above image motion estimation method will be introduced below. The electronic device includes an image motion estimation device.

[0185] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of an image motion estimation device provided in an embodiment of the present application. Fig. 9 As shown, the image motion estimation device 900 includes:

[0186] The first obtaining module 901 is used to obtain each motion vector of an object in any two adjacent frames of images in a video;

[0187] The second obtaining module 902 is used to obtain a number of coding blocks corresponding to the object motion vector in any two adjacent frames of images;

[0188] A first search module 903 is used to obtain a first prediction unit by dividing each coding block, and based on each motion vector of an object in two adjacent frames of images, perform motion search on each pixel in the first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit, each first difference parameter corresponding to a motion vector difference of a pixel in the first prediction unit of the two adjacent frames of images, and the first prediction unit is obtained by dividing the coding block in a first manner;

[0189] A second search module 904 is configured to obtain at least two second prediction units by dividing each coding block, and based on the first difference parameter set, perform motion search on each pixel in the second prediction unit for each coding block to obtain a second difference parameter set for each motion vector in each second prediction unit, wherein each second motion vector parameter corresponds to a motion vector difference of a pixel in the second prediction unit of two adjacent frames of images, and the second prediction unit is obtained by dividing the coding block in a second manner, and the second manner is different from the first manner;

[0190] A data storage module 905, configured to store a first difference parameter set and a second difference parameter set for each coding block;

[0191] A determination module 906 is used to determine a target coding block based on a first difference parameter set and a second difference parameter set for each coding block, where the target coding block is a coding block in a first image that motion matches the second image, the first image is adjacent to the second image and is generated later than the second image, and the difference parameter of the target coding block is better than the difference parameter of the non-target coding block.

[0192] In a possible implementation, the first search module is specifically configured to:

[0193] Based on each motion vector of the object in two adjacent frames of images, searching in the first prediction unit with at least two search accuracies to obtain a difference parameter of each pixel in the first prediction unit at the corresponding at least two search accuracies;

[0194] Based on the difference parameters of the pixels in the first prediction unit at the corresponding at least two search accuracies, a target search accuracy pixel of the corresponding accuracy is obtained, and the difference parameters of the target search accuracy pixel in the first prediction unit are better than the difference parameters of the non-target search accuracy pixel;

[0195] The difference parameters of each pixel in the first prediction unit at corresponding at least two search accuracies form a first difference parameter set;

[0196] Among them, the first target search accuracy pixel is determined based on the first search accuracy, and the second target search accuracy pixel is determined based on the second search accuracy searched in the first target search accuracy pixel, the second search accuracy and the first search accuracy are two adjacent search accuracy, and the second search accuracy is greater than the first search accuracy.

[0197] In a possible implementation, the number of pixels included in the first prediction unit is greater than the number of pixels included in any second prediction unit.

[0198] In a possible implementation, the method further includes:

[0199] A prediction unit determination module is used to determine a target second prediction unit in the second prediction unit of each coding block based on the second difference parameter set of each coding block before saving the first difference parameter set and the second difference parameter set of each coding block, wherein the difference parameter of the motion vector in the target second prediction unit is better than the difference parameter of the motion vector in the non-target second prediction unit in each coding block; and use the motion vector of the target second prediction unit as the motion vector of the object in the two adjacent frame images.

[0200] In a possible implementation, a module is determined, specifically for:

[0201] For the first prediction unit and the second prediction unit of each coding block, respectively obtain a difference parameter set of each prediction unit;

[0202] Determine, according to the difference parameter set of each prediction unit, a third prediction unit in the corresponding coding block, wherein the difference parameter of the third prediction unit is better than the difference parameter of a non-third prediction unit in the corresponding coding block;

[0203] A target prediction unit is determined according to the third prediction unit in each coding block, and the difference parameter of the target prediction unit is better than the difference parameter of the non-target prediction unit.

[0204] Fig.10 : is a structural schematic diagram of a data storage module provided in an embodiment of the present application, the data storage module comprises:

[0205] At least two storage spaces 1001, each storage space is used to store difference parameters corresponding to the division method.

[0206] The storage space corresponds to the prediction units corresponding to the division modes, and an independent storage space is set for the prediction units corresponding to each division mode, so as to realize the separate storage of the difference parameter sets of the prediction units corresponding to different division modes.

[0207] As an example, corresponding storage spaces are set for 2N×2N, 2N×N, and N×2N, respectively.

[0208] In a possible implementation, since 2N×2N is generally not selected as a target prediction unit, the storage space for storing the difference parameter set of the first prediction unit may be omitted, and only the storage space is set for the second prediction unit.

[0209] Fig.11 is another structural schematic diagram of a data storage module provided in an embodiment of the present application, the data storage module comprising:

[0210] At least two storage spaces 1101 and at least two accumulators 1102 .

[0211] Wherein, one accumulator corresponds to one coding block division mode of the second prediction unit;

[0212] Among them, the accumulator accumulates the target difference parameters of the pixels at the corresponding positions in the first prediction unit for the positions of each pixel in the corresponding second prediction unit, and obtains a second difference parameter set of each motion vector in the corresponding second prediction unit, and the target difference parameter is the difference parameter obtained by searching for the search accuracy corresponding to the second method.

[0213] In a possible implementation, the search result of the second search module can be directly calculated by an accumulator and used as the search result of the second search module.

[0214] In a possible implementation, the accumulator may be ACC (accumulator).

[0215] The accumulator may be a 16-bit SATD4×4 accumulator, and the storage space may be three 24-bit×18×2 memories.

[0216] It should be noted that for the functional explanation of each structure in the image motion estimation device provided in this embodiment, please refer to the explanation in the aforementioned method embodiment, which will not be repeated in this embodiment.

[0217] The image motion estimation device may correspond to or be integrated into a video codec chip of an electronic device.

[0218] Among them, an electronic device using the image motion estimation device, when transmitting a video, uses the image motion estimation device to determine the target prediction unit. Compared with performing search calculations on various prediction units, there is no need to perform a motion search calculation process on the second prediction unit, thereby reducing the data processing burden of the chip, reducing the power consumption of the device, and being able to accurately determine the prediction unit with the best difference parameter. In the process of compressing video frames, it can reduce residuals, save transmission bit rate, and improve transmission efficiency.

[0219] In this embodiment, after a number of coding blocks corresponding to the motion vectors of the objects in any two adjacent images are targeted, the coding blocks are divided in a first manner to obtain a first prediction unit, and based on the motion vectors of the objects in the two adjacent frames of images, a motion search is performed on each pixel in the first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit; then, when a motion search is performed on each pixel in the second prediction unit obtained by dividing each coding block in a second manner, the motion search is performed based on the first difference parameter set to obtain a second difference parameter set corresponding to each motion vector in the second prediction unit, the first difference parameter set and the second difference parameter set are saved, and a target prediction unit is determined based on the first difference parameter set and the second difference parameter set of each coding block. The target prediction unit is a prediction unit in the first image that matches the motion of the second image, and its difference parameter is better than the difference parameter of the non-target prediction unit in the first image, thereby reducing the amount of data processing when performing motion search on each pixel in the second prediction unit, and only substantial search calculation needs to be performed when performing motion search on each pixel in the first prediction unit, thereby reducing the data processing burden of the entire image motion estimation process.

[0220] Fig.12 This is a schematic diagram of an application scenario of an image motion estimation method provided in the present application, and the application scenario includes the following structures: a cache 1201, an image unit 1202, a 1 / 2 pixel search unit 1203, a 1 / 4 pixel search unit 1204, an accumulator 1205, a storage module 1206 and a packing module 1207.

[0221] The cache 1201 is used to store a plurality of prediction units obtained by division; the motion vector unit 1202 is used to store the motion vector mv of each full pixel in any two adjacent frames of images.

[0222] Among them, the 1 / 2 pixel search unit 1203 receives the input full pixel motion vector, performs a 3×3 half pixel search for each full pixel, calculates the 4×4 SATD value included in the current prediction unit, and selects the optimal 1 / 2 pixel motion vector mv according to the SATD minimum principle.

[0223] Among them, the processed prediction unit is 2N×2N, and the 4×4 SATD value obtained by interpolation calculation of the 1 / 2 pixel search unit is sent to the accumulator 1205. The accumulator 1205 first accumulates the SATD value corresponding to the 1 / 2 pixel to obtain the corresponding N×2N, 2N×N and N×N SATD values, and stores them in the corresponding storage space in the storage module 1206.

[0224] The 1 / 4 pixel search unit 1204 performs a 1 / 4 pixel search on the motion vector optimized by the 1 / 2 pixel search, and selects the optimal 1 / 4 pixel motion vector mv according to the SATD minimum principle.

[0225] Among them, the processed prediction unit is 2N×2N, and the 4×4 SATD value obtained by interpolation calculation of the 1 / 4 pixel search unit is sent to the accumulator 1205. The accumulator 1205 first accumulates the SATD value corresponding to the 1 / 4 pixel to obtain the corresponding N×2N, 2N×N and N×N SATD values, and stores them in the corresponding storage space in the storage module 1206.

[0226] Among them, after the search for each coding block in the image is completed and the MVPACK (compression) stage is performed, when the motion vector of each prediction unit is processed in sequence, the newly added motion vector and the corresponding SATD are read from the corresponding storage space, inserted between the motion vectors that pass the normal search, and sent together to the subsequent selection strategy, which can adopt RDO (rate-distortion optimization).

[0227] The present application also provides an electronic device in an embodiment. Fig.13 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig.13 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0228] like Fig.13 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage device 1308 to a random access memory (RAM) 1303. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1303. The processing device 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0229] Typically, the following devices may be connected to the I / O interface 1305: an input device 1306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1308 including, for example, a memory card, a hard disk, etc.; and a communication device 1309. The communication device 1309 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.13 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0230] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the image motion estimation methods provided in the embodiments of the present application.

[0231] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any image motion estimation method provided in the embodiment of the present application.

[0232] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0233] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0234] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0235] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for estimating image motion, comprising: Obtain the motion vectors of objects in any two adjacent frames of the video; Obtaining a number of coding blocks corresponding to the object motion vectors in any two adjacent frames of images; For each coding block, a first prediction unit is obtained by dividing the coding block, and based on each motion vector of the object in two adjacent frames of images, a motion search is performed on each pixel in the first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit, each first difference parameter corresponds to a motion vector difference of a pixel in the first prediction unit of the two adjacent frames of images, and the first prediction unit is obtained by dividing the coding block in a first manner; At least two second prediction units are obtained by dividing each coding block, and based on the first difference parameter set, motion search is performed on each pixel in the second prediction unit for each coding block to obtain a second difference parameter set for each motion vector in each second prediction unit, each second motion vector parameter corresponds to a motion vector difference of a pixel in the second prediction unit of the two adjacent frames of image, and the second prediction unit is obtained by dividing the coding block in a second manner, and the second manner is different from the first manner; Saving a first difference parameter set and a second difference parameter set for each coding block; Based on a first difference parameter set and a second difference parameter set for each coding block, a target prediction unit is determined, where the target prediction unit is a prediction unit in a first image that motion matches a second image, the first image is adjacent to the second image and is generated later than the second image, and the difference parameter of the target prediction unit is better than the difference parameter of a non-target prediction unit.

2. The image motion estimation method according to claim 1, wherein the step of performing motion search on each pixel in the first prediction unit for each coding block based on each motion vector of an object in two adjacent frames of images to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit comprises: Based on each motion vector of the object in two adjacent frames of images, searching in the first prediction unit with at least two search accuracies to obtain a difference parameter of each pixel in the first prediction unit at the corresponding at least two search accuracies; Based on the difference parameters of the pixels in the first prediction unit at the corresponding at least two search accuracies, a target search accuracy pixel of the corresponding accuracy is obtained, wherein the difference parameters of the target search accuracy pixel in the first prediction unit are better than the difference parameters of the non-target search accuracy pixel; The difference parameters of each pixel in the first prediction unit at corresponding at least two search accuracies form a first difference parameter set; Among them, the first target search accuracy pixel is determined based on the first search accuracy, and the second target search accuracy pixel is determined based on the second search accuracy searched in the first target search accuracy pixel, the second search accuracy and the first search accuracy are two adjacent search accuracy, and the second search accuracy is greater than the first search accuracy.

3. The image motion estimation method according to claim 1, wherein the step of dividing each coding block into at least two second prediction units, performing motion search on each pixel in the second prediction unit for each coding block based on the first difference parameter set, and obtaining a second difference parameter set for each motion vector in each second prediction unit comprises: Based on the position of each pixel included in the second prediction unit, determining the pixel at the corresponding position in the first prediction unit; Based on the target difference parameters of each pixel at the corresponding position in the first prediction unit, a second difference parameter set of each motion vector in each second prediction unit is accumulated, and the target difference parameter is a difference parameter searched for the search accuracy corresponding to the second method. 4 . The image motion estimation method according to claim 1 , wherein the number of pixels included in the first prediction unit is greater than the number of pixels included in any second prediction unit.

5. The image motion estimation method according to claim 1, before saving the first difference parameter set and the second difference parameter set of each coding block, comprising: Based on the second difference parameter set of each coding block, determine a target second prediction unit in the second prediction unit of each coding block, wherein the difference parameter of the motion vector in the target second prediction unit is better than the difference parameter of the motion vector in the non-target second prediction unit in each coding block; The motion vector of the target second prediction unit is used as the motion vector of the object in the two adjacent frames of images.

6. The image motion estimation method according to claim 1, wherein the storing the first difference parameter set and the second difference parameter set of each coding block comprises: Determine a coding block division method corresponding to the first difference parameter set and the second difference parameter set; Based on the corresponding coding block division method, the first difference parameter set and the second difference parameter set are respectively stored in the storage space of the corresponding division method.

7. The image motion estimation method according to claim 6, wherein the step of determining the target prediction unit based on the first difference parameter set and the second difference parameter set of each coding block comprises: For the first prediction unit and the second prediction unit of each coding block, respectively obtain a difference parameter set of each prediction unit; Determine, according to the difference parameter set of each prediction unit, a third prediction unit in the corresponding coding block, wherein the difference parameter of the third prediction unit is better than the difference parameter of the non-third prediction unit in the corresponding coding block; A target prediction unit is determined according to the third prediction unit in each coding block, wherein a difference parameter of the target prediction unit is better than a difference parameter of a non-target prediction unit.

8. An electronic device, comprising an image motion estimation device, the image motion estimation device comprising: The first acquisition module is used to obtain each motion vector of an object in any two adjacent frames of images in a video; A second obtaining module is used to obtain a number of coding blocks corresponding to the object motion vectors in any two adjacent frames of images; A first search module is used to obtain a first prediction unit by dividing each coding block, and based on each motion vector of an object in two adjacent frames of images, perform motion search with each pixel in the first prediction unit for each coding block to obtain a first difference parameter set corresponding to each motion vector in the first prediction unit, each first difference parameter corresponding to a motion vector difference of a pixel in the first prediction unit of the two adjacent frames of images, wherein the first prediction unit is obtained by dividing the coding block in a first manner; a second search module, configured to obtain at least two second prediction units by dividing each coding block, and based on the first difference parameter set, perform motion search on each pixel in the second prediction unit for each coding block to obtain a second difference parameter set for each motion vector in each second prediction unit, wherein each second motion vector parameter corresponds to a motion vector difference of a pixel in the second prediction unit of the two adjacent frames of images, and the second prediction unit is obtained by dividing the coding block in a second manner, and the second manner is different from the first manner; A data storage module, used for storing a first difference parameter set and a second difference parameter set for each coding block; A determination module is used to determine a target prediction unit based on a first difference parameter set and a second difference parameter set for each coding block, wherein the target prediction unit is a prediction unit in a first image that motion matches a second image, the first image is adjacent to the second image and is generated later than the second image, and the difference parameter of the target prediction unit is better than the difference parameter of the non-target prediction unit.

9. The electronic device according to claim 8, wherein the data storage module comprises: At least two storage spaces, each storage space is used to store difference parameters corresponding to the division method.

10. The electronic device according to claim 9, the data storage module further comprising: at least two accumulators, one accumulator corresponding to a coding block division mode of the second prediction unit; Among them, the accumulator accumulates the target difference parameters of the pixels at the corresponding positions in the first prediction unit for the positions of each pixel in the corresponding second prediction unit, and obtains a second difference parameter set of each motion vector in the corresponding second prediction unit, and the target difference parameter is the difference parameter obtained by searching for the search accuracy corresponding to the second method.

Images