Method and system for calculating score precision motion estimation based on error curved surface

Abstract

The invention discloses a method and system for calculating fractional motion estimation based on an error curved surface, and relates to the technical field of image coding compression. According to the method, the last stage of integer motion vector search is searched by using a small square with the step of 1, so that the minimum rate distortion cost value of a fractional motion vector is ensured to fall between [-1, 1] of x and y; fractional multiple motion estimation of sub-pixels without interpolation calculation is carried out in an error curved surface fitting mode, multiplication and division operation can be omitted in the process, each step of calculation can be expanded into pipeline operation, calculation efficiency is improved, hardware resource occupation is reduced, and the method can be well suitable for hardware calculation; in order to further simplify the operand, when the score coordinates are enumerated, only 1 / 2, 1 / 4 and 1 multiple score coordinates are enumerated, and a large number of computing resources can be saved while the computing precision is guaranteed. Compared with the prior art, the method has the advantages that the operation speed is high, the motion compensation quality is high, and sub-pixels do not need to be calculated through interpolation.

Description

Technical Field

[0001] The present invention relates to the technical field of image coding compression, and in particular to a method and system for fractional precision motion estimation based on error surface calculation. Background Art

[0002] In video image compression technology, motion estimation in inter-frame prediction can effectively eliminate the temporal correlation between frames in a video sequence, and has been widely used in various video coding standards based on block matching. Motion estimation is the most time-consuming and computationally complex step in the entire video compression coding process.

[0003] With the gradual popularization of ultra-high-definition video, the new generation of image coding standards has introduced larger computing blocks and more complex block divisions to continuously improve coding efficiency, making the amount of data processed by video coding very large. In order to meet the needs of real-time or fast compression, higher requirements are placed on the computing speed. For video encoders, the inter-frame prediction part accounts for more than 60% of the total encoder computing volume, and motion estimation is the most important part of inter-frame prediction.

[0004] Fractional precision motion estimation includes motion estimation at the integer pixel level and motion estimation at the sub-pixel level. Because the optimal motion vector of the current block does not necessarily fall on an integer multiple of pixels, motion estimation of the fractional precision level motion vector is required in inter-frame prediction, which usually includes two steps: integer precision motion estimation corresponding to integer pixels and fractional precision motion estimation corresponding to sub-pixels. The purpose of integer precision motion estimation is to quickly converge to an optimal integer pixel precision motion vector within a specific search range, and then further search for fractional precision motion vectors corresponding to sub-pixels with this optimal integer pixel matching position as the center.

[0005] The computational complexity of fractional precision motion estimation is relatively large. Fractional precision motion estimation involves multiple 1 / 2 and 1 / 4 level fractional sub-pixel interpolation calculations. A single integer pixel can derive dozens of sub-pixel-level calculations. Each sub-pixel requires additional storage space, and each sub-pixel may also involve the calculation process of the sum of absolute difference (SAD, Sum of Absolute Difference) or the sum of absolute transformed difference (SATD, Sum of Absolute Transformed Difference). Then, the rate-distortion cost value of each candidate fractional motion vector of the sub-pixel is compared, and finally the integer precision motion vector with the minimum rate-distortion cost and the fractional precision motion vector with the minimum rate-distortion cost are selected as the final optimal motion vector. Fractional precision motion estimation can help the encoder select the encoding mode and parameters with the minimum rate-distortion cost. By minimizing the rate-distortion cost of each coding unit, the encoder can maximize the overall video quality under limited bit rate conditions. This is a very important optimization goal and technology in video encoding, so a fractional motion estimation method that can improve the operation speed and accuracy is needed. Summary of the invention

[0006] In view of the above problems, the present invention proposes a method and system for calculating fractional precision motion estimation based on error surface, which uses error surface for function fitting and is well suitable for hardware calculation. It has the advantages of fast computing speed, high motion compensation quality, and no need for additional interpolation calculation of sub-pixels.

[0007] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0008] A method for calculating fractional precision motion estimation based on an error surface, the method comprising:

[0009] S1: Perform a coarse search on the integer motion vector until the integer motion vector position where the minimum rate distortion cost value is located is located in the exact center of the search shape, then switch to a small square to continue fine search;

[0010] S2: The last stage of integer motion estimation uses a small square with a step size of 1 to search for the minimum rate distortion cost value. Each search point is the center position and 8 adjacent integer pixels. When the integer motion vector position where the minimum rate distortion cost value is located is found to be at the center position of the small square, the integer motion vector search is stopped.

[0011] S3: After stopping the integer motion vector search, calculate the SATD values ​​of the integer pixel corresponding to the center position of the small square and the 8 adjacent integer pixels, and obtain the integer motion vector corresponding to the center position of the small square at this time as the optimal integer motion vector MV_X int and MV_Y int, complete integer motion estimation;

[0012] S4: Taking the optimal integer motion vector as the center position of the starting point of the fractional motion vector search, constructing a single-peak error surface function model whose center position is the lowest point of the rate-distortion cost value in the integer pixel plane, bringing the SATD values ​​and corresponding integer coordinate values ​​of the 9 integer pixel points calculated in step S3 into the error surface function model, solving the polynomial parameters of the error surface function model, and completing the fitting of the fractional level error surface function model;

[0013] S5: Use the enumeration method to enumerate the candidate fractional coordinates in the range of [-1,1] and bring them into the error surface function model for solution to obtain the SATD value of each candidate fractional coordinate;

[0014] S6: Calculate the rate-distortion cost value of the candidate fractional motion vector corresponding to each candidate fractional coordinate and compare them, output the minimum value of the rate-distortion cost value corresponding to the candidate fractional motion vector, and the candidate fractional motion vector corresponding to the minimum rate-distortion cost value as the optimal fractional motion vector MV_X at the 1 / 4 sub-pixel accuracy level. frac and MV_Y frac , complete the fractional motion vector calculation;

[0015] S7: The optimal integer motion vector MV_X int and MV_Y int and the optimal fractional motion vector MV_X frac and MV_Y frac Add them together to get the final optimal fractional precision level motion vectors MV_X and MV_Y, and complete the fractional precision level motion estimation.

[0016] As a preferred solution of the present invention, in step S3, the SATD values ​​of the integer pixel corresponding to the center position of the small square and the eight adjacent integer pixels are calculated, specifically by using a two-level series one-dimensional Hadamard change matrix for calculation.

[0017] As a preferred solution of the present invention, the error surface function model includes a 5-parameter model, a 6-parameter model and a 9-parameter model, and the function expressions are respectively:

[0018]

[0019] Where x and y represent the pixel coordinates, f(x, y) is the SATD value corresponding to the x and y coordinates; AI is the polynomial parameter of the error surface function model.

[0020] As a preferred solution of the present invention, when the error surface function model is a 9-parameter model, the polynomial parameters of the error surface function model are solved by the following method:

[0021] The optimal integer motion vector is used as the center position of the starting point of the fractional motion vector search, recorded as the origin (0,0), and the coordinates of the 8 adjacent integer pixels are (-1,-1), (0,-1), (1,-1), (-1,0), (1,0), (-1,1), (0,1), (1,1). The SATD values ​​corresponding to these 9 integer pixels are recorded as f(-1,-1)-f(1,1), and are substituted into the function expression of the 9-parameter model to obtain:

[0022]

[0023] Simplify the above formula into a matrix expression, expressed as:

[0024] f(x, y) = O*P;

[0025] Where f(x,y) is the SATD value corresponding to the x and y coordinates, Q represents the constant matrix corresponding to the product of the pixel point coordinate polynomial x or y, and P represents the polynomial parameter AI;

[0026] The 9 integer pixel coordinates are independent of each other, so the determinant of Q is not equal to zero, and there is a reversible matrix. By solving the least squares method, we get the following expression:

[0027] P=(Q T Q) -1 Q T *f(x,y);

[0028] Where T represents matrix transpose; (Q T Q) -1 Q T is a constant matrix;

[0029] Solve the polynomial parameter AI:

[0030]

[0031] As a preferred solution of the present invention, the enumerating fractional coordinates in the range of [-1, 1] specifically enumerates fractional coordinates of multiples of 1 / 2, 1 / 4 and 1.

[0032] As a preferred solution of the present invention, in step S6, the rate-distortion cost value of the candidate fractional motion vector corresponding to each fractional coordinate is calculated by the following formula:

[0033] cost = S + λ * R;

[0034] Where cost represents the rate-distortion cost; S represents the difference measure between the reference macroblock and the motion prediction macroblock, and the SATD value is used as the measure for calculation; R represents the number of bits required to measure the motion information itself, which is obtained according to the preset lookup table corresponding to the coordinate value; λ is the weight coefficient.

[0035] A system for calculating fractional precision motion estimation based on an error surface, the system comprising:

[0036] The integer motion estimation module is used to perform integer motion estimation on the integer motion vector to obtain the optimal integer motion vector MV_X int and MV_Y int ;

[0037] The fractional motion estimation module is used to perform fractional motion estimation centered on the optimal integer motion vector to obtain the optimal fractional motion vector MV_X frac and MV_Y frac , and the optimal integer motion vector MV_X int and MV_Y int and the optimal fractional motion vector MV_X frac and MV_Y frac Add them together to get the final optimal fractional accuracy level motion vectors MV_X and MV_Y.

[0038] As a preferred solution of the present invention, the integer motion estimation module and the fractional motion estimation module both include a rate-distortion cost calculation unit and a rate-distortion cost comparison unit;

[0039] The rate-distortion cost calculation unit is used to calculate the rate-distortion cost value of the integer motion vector or the fractional motion vector;

[0040] The rate-distortion cost comparison unit is used to compare the rate-distortion cost values ​​of integer motion vectors or fractional motion vectors.

[0041] As a preferred solution of the present invention, the integer motion estimation module further includes:

[0042] An integer motion vector search unit is used to perform a coarse search on the integer motion vector until the integer motion vector position where the minimum rate distortion cost value is located is located at the exact center of the search shape, then switch to a small square to continue fine search, each time the search point is the center position and 8 adjacent integer pixels, until the integer motion vector position where the minimum rate distortion cost value is located is located at the center position of the small square, then stop the integer motion vector search;

[0043] The SAD and SATD calculation unit is used to calculate the SAD value of the integer motion vector, or use a two-level series one-dimensional Hadamard change matrix to calculate the SATD value of the integer motion vector. The SAD or SATD value is one of the distortion metrics for calculating the rate-distortion cost. Generally, in the integer motion vector search, the rate-distortion cost uses the SAD value as a metric. In the fine search or fractional motion vector search, since the SAD value difference at each point is relatively small, the use of the SATD value can better match the position with a smaller bitstream and prevent falling into the local optimum.

[0044] As a preferred solution of the present invention, the fractional motion estimation module further includes:

[0045] The error surface calculation unit is used to use the optimal integer motion vector as the center position of the starting point of the fractional motion vector search, construct an error surface function model with a single peak value whose center position is the lowest point of the rate-distortion cost in the integer pixel plane (the minimum value of the rate-distortion cost in the integer motion vector search is located at the center of the small square), bring the coordinate values ​​and SATD values ​​of the integer pixel and 8 adjacent integer pixels corresponding to the optimal integer motion vector into the error surface function model, solve the polynomial parameters of the error surface function model, and complete the fitting of the fractional level error surface function model;

[0046] An enumeration fractional motion vector unit is used to enumerate candidate fractional coordinates in the range of [-1,1];

[0047] The SATD calculation unit is used to bring the enumerated candidate fractional coordinates into the error surface function model for solution to obtain the SATD value of each fractional motion vector.

[0048] Compared with the prior art, the beneficial effects of the present invention are: no additional interpolation calculation of sub-pixels is required, and the SATD value corresponding to the candidate fractional coordinates is directly obtained by substituting the candidate fractional coordinates in the form of an error surface fitting function, and the rate-distortion cost of the candidate motion vector is further obtained, and the optimal motion vector with the smallest rate-distortion cost as the best match is selected. The process is optimized according to the characteristics of hardware calculation. The calculation process can convert multiplication operations into only shift operations and addition and subtraction operations, omitting multiplication and division operations, and each step of the calculation can be expanded into a pipeline operation to improve calculation efficiency and reduce hardware resource occupation. It has the advantages of fast calculation speed, high motion compensation quality, and no need for additional interpolation calculation of sub-pixels. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0050] in:

[0051] Figure 1 Schematic diagram of fractional motion estimation for hierarchical search;

[0052] Figure 2 A schematic diagram of fitting a sub-pixel three-dimensional surface using an interpolation function method;

[0053] Figure 3 is a flow chart of a method in an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of a curved surface of 9 integer pixels in an embodiment of the present invention;

[0055] Figure 5 It is a schematic diagram of the error function fitting surface in an embodiment of the present invention;

[0056] Figure 6 Schematic diagram of sub-pixel coordinates in an embodiment of the present invention;

[0057] Figure 7 It is a system modular structure diagram of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0059] like Figure 1 As shown, the motion estimation methods with fractional precision levels commonly used in the prior art are interpolation sub-pixels and hierarchical search systems. Interpolation sub-pixels are obtained by interpolating existing integer pixels, and the sub-pixel levels usually include 1 / 2 precision and 1 / 4 precision; the hierarchical search system is a step-by-step refinement search method, which first searches at a coarser precision (such as the integer pixel level), and then gradually searches at a finer precision (such as the 1 / 2 pixel, 1 / 4 pixel level), and improves the encoding precision by gradually refining the motion vector.

[0060] Prior art step 1 (s1): Assuming that an optimal integer pixel precision motion vector has been found through a block matching algorithm or other methods, the optimal integer pixel precision motion vector is used as the center (0,0), and a 6-tap interpolation filter is used to interpolate the integer pixel point and the 6 integer pixels in the same row or column nearby to obtain a 1 / 2 sub-pixel at a central position, and then the 1 / 2 precision sub-pixels in 8 directions around the center of the integer pixel are obtained in turn, and then the difference between the motion vector at each 1 / 2 precision sub-pixel position and the matching block and the rate-distortion cost are calculated, and the 1 / 2 precision sub-pixel motion vector with the smallest rate-distortion cost is selected as the optimal 1 / 2 precision fractional motion vector;

[0061] In the second step (s2) of the prior art, a linear interpolation filter is used to calculate eight sub-pixels of 1 / 4 precision level around the optimal 1 / 2 precision sub-pixel position such as (1 / 2, -1 / 2) as the center, and the rate-distortion costs of the motion vectors of these eight 1 / 4 precision positions are compared. The 1 / 4 precision sub-pixel motion vector with the smallest rate-distortion cost is selected as the optimal 1 / 4 precision fractional motion vector.

[0062] In this fractional motion estimation process, the computational complexity comes from the calculation process of multiple sub-pixel interpolation processes and the rate-distortion cost of deriving multiple sub-pixel fractional motion vectors. At the same time, because the 1 / 4 sub-pixel precision motion vector must rely on the 1 / 2 sub-pixel precision motion vector result of the previous step, the time efficiency is slow, and the buffer data generated by the use of multi-tap filters during the interpolation process also occupies more SRAM resources.

[0063] The surface composed of 1 / 2 sub-pixel and 1 / 4 sub-pixel obtained by traditional six-tap filtering interpolation and linear interpolation is as follows Figure 2As shown, it can be seen that the sub-pixel surface obtained by interpolation is relatively rough and uneven, and the relative fitting error is large. Therefore, the present invention proposes a method and system for fractional precision motion estimation based on error surface calculation, by selecting a suitable surface function model, and then solving the modeling parameter AI for the surface function model to complete the fitting of the error surface function f(x, y). When calculating the fractional motion vector, the enumeration method is used to bring each candidate fractional coordinate into the error surface function to directly obtain the SATD value of the 1 / 4 fractional precision level (the traditional method must first calculate the 1 / 2 precision level sub-pixel, and then filter and interpolate the 1 / 2 precision level sub-pixel to obtain the 1 / 4 fractional precision sub-pixel. After obtaining the corresponding sub-pixel, it is necessary to calculate the residual of the sub-pixel of the reference macroblock and the current macroblock, and then perform Hadamard transform on the residual macroblock, and then calculate the sum of the absolute values ​​of the residual block after the Hadamard transform to obtain the SATD value. The SATD value of the fractional precision level cannot be obtained directly), and then the rate-distortion cost value of the corresponding candidate fractional coordinates is obtained, and the integer and fractional motion vector with the smallest rate-distortion cost is selected as the optimal fractional precision level motion vector.

[0064] Compared with the conventional interpolation method and the hierarchical search system, which need to calculate the corresponding 1 / 2 sub-pixel and then the 1 / 4 sub-pixel through the interpolation filter in turn, and then obtain the optimal fractional motion vector of the 1 / 4 precision fractional coordinate according to the residual and rate-distortion cost with the matching block, the transition of the error surface fitting used in the present invention is smoother and more accurate. The process does not need to calculate the specific sub-pixel and multiplication and division operations separately, nor does it require additional integer pixel points for interpolation filtering. The SATD value of the corresponding fractional motion vector can be directly substituted into the error surface function according to the candidate fractional coordinates, and then the rate-distortion cost value is obtained.

[0065] Example 1: Figure 3-Figure 6 As shown, this embodiment provides a method for calculating fractional motion estimation based on an error surface, characterized in that the rate-distortion cost adopts the SAD value during the fine search of the small square, and after the fine search converges, the SATD value is additionally calculated for the optimal integer motion vector of the small square and the motion vectors of 8 nearby integer pixels, and the SATD values ​​of the 9 integer motion vectors are substituted into the error surface function to complete the fitting of the error surface function f(x, y), and finally, in the process of solving the optimal fractional motion vector, the candidate fractional coordinates are enumerated and the size of the rate-distortion cost is compared.

[0066] The specific steps include:

[0067] S1: Perform a coarse search on the integer motion vector until the integer motion vector position where the minimum rate distortion cost value is located is located in the exact center of the search shape, then switch to a small square to continue fine search;

[0068] In integer motion estimation, the current frame is divided into several macroblocks (each macroblock is at most 16×16, including sub-macroblocks such as 16×8, 8×8, etc.); for each macroblock, a matching macroblock is searched in the reference frame, and the search range is usually a fixed window (for example, ±112 pixels); a matching metric (such as the sum of absolute differences (SAD) and squared difference (SSD)) is used to evaluate the similarity between the current block and the reference block; and the motion vector that optimizes the matching criterion (minimum rate distortion cost) is selected as the optimal integer pixel precision motion vector.

[0069] In this step, a coarse search for the optimal integer motion vector is performed on integer pixels using a search method with a larger step and a wider range, such as a hexagonal search, a diamond search, or other applicable integer search methods, which are not limited here. Optionally, the hexagonal search is used in this embodiment.

[0070] When the integer motion vector position where the minimum rate-distortion cost value is found is located at the exact center of the search shape, it means that the optimal integer motion vector of the coarse search has converged to the center position, and it can be regarded as completing the initial convergence. That is to say, after the coarse search in step S1, we can get a relatively optimal integer motion vector, and then switch to the small square search to further refine the search area. It should be noted that the rate-distortion cost metric here uses the SAD value.

[0071] S2: The last stage of integer motion estimation uses a small square with a step size of 1 to search for the minimum rate distortion cost value. Each search point is the center position and 8 adjacent integer pixels until the integer motion vector position where the minimum rate distortion cost value is found is located at the center position of the small square (such as Figure 4 ), then the integer motion vector search is stopped; the rate distortion cost value here still uses the SAD value;

[0072] The last level must use a small square search with a step size of 1 to further refine the search area. When the small square converges to the center point of 9 integer pixels, which is the location of the minimum rate-distortion cost value, the motion vector corresponding to the center point is the optimal integer motion vector. The optimal integer motion vector is used as the search origin to construct a single-peak error surface function model whose center position is the lowest or lower point of the rate-distortion cost value in the integer pixel plane (the minimum rate-distortion cost in the integer motion vector search is located at the center of the small square).

[0073] S3: After stopping the integer motion vector search, calculate the SATD values ​​of the integer pixel corresponding to the center position of the small square and the 8 adjacent integer pixels, and obtain the integer motion vector corresponding to the center position of the small square at this time as the optimal integer motion vector MV_X int and MV_Y int , complete integer motion estimation;

[0074] In one embodiment, the SATD values ​​of 9 integer pixels are calculated by using a two-level series-connected one-dimensional Hadamard transformation matrix for operation;

[0075] The one-dimensional Hadamard change matrix operation with two-stage series connection is a simple time-frequency transform. In the sub-pixel search, the difference of SAD value at each point is relatively small, so the use of SATD can better match the position with smaller code stream and prevent falling into the local optimum.

[0076] S4: Taking the optimal integer motion vector as the center position of the starting point of the fractional motion vector search, construct an error surface function model with a single peak whose center position is the low point or lower point of the rate-distortion cost in the integer pixel plane (the small square fine search in the previous step has confirmed that the integer motion vector with the smallest rate-distortion cost is located at the center position of the integer pixel plane, such as Figure 4 , where the center position is at the lowest point or the lower point relative to the whole pixel plane, excluding sub-pixel points), the SATD values ​​and corresponding integer coordinate values ​​of the 9 whole pixel points calculated in step S3 are brought into the error surface function model, the polynomial parameters of the error surface function model are solved, and the fitting of the fractional level error surface function model is completed;

[0077] After the integer motion vector search process, the best match value for fractional precision motion estimation should fall within the entire [-1,1] search area. At the same time, since the SATD value of each integer pixel in the search plane is close to a single peak function, the least squares error surface model can be used for fitting. Common error surface models include 5-parameter model, 6-parameter model and 9-parameter model. The 9-parameter model has the highest accuracy, but is also the most complex, followed by the 6-parameter model.

[0078] The function expressions are:

[0079]

[0080] In the formula, x and y represent the pixel coordinates, f(x,y) is the SATD value corresponding to the x and y coordinates; AI is the polynomial parameter of the error surface function model. Because the motion estimation in the video encoding process is generally only 1 / 4 sub-pixel accuracy, x and y are generally multiples of 0.25.

[0081] This embodiment adopts a 9-parameter model, which has a good fitting effect and is suitable for hardware calculation.

[0082] The optimal integer motion vector is used as the center position of the starting point of the fractional motion vector search, recorded as the origin (0,0), and the coordinates of the 8 adjacent integer pixels are (-1,-1), (0,-1), (1,-1), (-1,0), (1,0), (-1,1), (0,1), (1,1). The SATD values ​​corresponding to these 9 integer pixels are recorded as f(-1,-1)-f(1,1), and are substituted into the function expression of the 9-parameter model to obtain:

[0083]

[0084] Simplify the above formula into a matrix expression, expressed as:

[0085] f(x, y) = Q*P;

[0086] Where f(x,y) is the SATD value corresponding to the x and y coordinates, Q represents the constant matrix corresponding to the product of the pixel point coordinate polynomial x or y, and P represents the polynomial parameter AI;

[0087] The 9 integer pixel coordinates are independent of each other, so the determinant of Q is not equal to zero, and there is a reversible matrix. By solving the least squares method, we get the following expression:

[0088] P=(Q T Q) -1 Q T *f(x,y);

[0089] Because the values ​​of the 9 integer pixel points searched using the small square have been determined (that is, the values ​​of x and y are -1, 0 or 1), (Q T Q) -1 Q T is a constant matrix, which can be used as constants -1, 0, 1 in the operation, and after sorting, we get:

[0090]

[0091] That is, solve the polynomial parameter AI:

[0092]

[0093] It can be seen that the matrix is ​​calculated by 1 / 4, 1 / 2 or 1 times, which is equivalent to the calculation of the constant matrix. The parameter AI can be solved by bit operation and addition and subtraction of f(-1,-1)-f(1,1). There is no multiplication and division, which saves a lot of computing units and is suitable for fast hardware calculation.

[0094] S5: Use the enumeration method to enumerate the candidate fractional coordinates in the range of [-1,1] and bring them into the error surface function model for solution to obtain the SATD value of each candidate fractional coordinate;

[0095] After obtaining the polynomial parameters AI, the error surface is fitted with the binary function f(x, y). Simply substitute the coordinate value corresponding to the 1 / 4 sub-pixel accuracy into the error surface function model f(x, y) to obtain the SATD value of the corresponding fractional coordinate on the error surface.

[0096] In practical applications, the partial derivative of the binary function can be used to obtain the minimum SATD value, but a saddle point may appear for the binary function, so the derivative results must be classified and discussed, and the derivative involves multiplication and division calculations. Therefore, in the method of the present invention, because the small square search of 9 points and the convergence of integer motion vectors are completed during the integer pixel search, it can be ensured that the minimum SATD value of the fractional sub-pixel position should fall between [-1,1] of the optimal integer motion vectors x and y, and the optimal fractional motion vector can be obtained within this range, so the candidate fractional coordinate method is directly adopted to obtain the corresponding SATD value, and further calculate the rate-distortion cost value of the candidate fractional coordinate.

[0097] In one of the embodiments, the values ​​of 9×9 fractional points (such as -1, -3 / 4, -1 / 2, -1 / 4, 0, 1 / 4, 1 / 2, 3 / 4, 1 of x or y, a total of 9 points) are substituted into the function expression of the 9-parameter model, all SATD values ​​are calculated, and then the rate-distortion cost values ​​are compared, that is, all 9 integer pixels and 72 fractional pixels are enumerated. Since the 9 integer pixels have been calculated, there is no need to repeat the calculation.

[0098] Optionally, to further simplify the computation, only Figure 6 The points that are multiples of 1 / 2, 1 / 4, and 1 (i.e. Figure 6 The fractional coordinates marked in the figure are 40 sub-pixel points in total. These points are shift operations and addition and subtraction operations on the calculated parameters AI in the calculation of the function f(x,y), which can save a lot of computing resources while ensuring the calculation accuracy.

[0099] Therefore, in a specific embodiment, fractional coordinates in the range of [-1, 1] are enumerated, specifically, fractional coordinates of 1 / 2, 1 / 4 and 1 multiples are enumerated (i.e. Figure 6 (the points of the circle and cross in the middle), omitting fractional coordinates that are multiples of 3 / 4.

[0100] For example: When the calculated candidate score coordinates are When , the candidate fractional coordinate value is substituted into the function expression of the error surface function model, and the result is:

[0101]

[0102] The polynomial parameter AI has been obtained in step S4, and the calculation result of the above formula is the SATD value of f(1 / 2,1 / 4).

[0103] S6: Calculate the rate-distortion cost value of the candidate fractional motion vector corresponding to each candidate fractional coordinate and compare them, output the minimum value of the rate-distortion cost value corresponding to the candidate fractional motion vector, and the candidate fractional motion vector corresponding to the minimum rate-distortion cost value as the optimal fractional motion vector MV_X at the 1 / 4 sub-pixel accuracy level. frac and MV_Y frac , complete the fractional motion vector calculation;

[0104] According to the SATD value of each fractional coordinate, the rate-distortion cost value of the candidate fractional motion vector corresponding to each fractional coordinate can be calculated. The calculation formula is:

[0105] cost = S + λ * R;

[0106] Where cost represents the rate-distortion cost; S represents the difference measure between the reference macroblock and the motion prediction macroblock, and the SATD value is used as the measure for calculation; R represents the number of bits required to measure the motion information itself, which is obtained according to the preset lookup table corresponding to the coordinate value; λ is the weight coefficient.

[0107] S7: The optimal integer motion vector MV_X int (such as MV_X int =3) and MV_Y int and the optimal fractional motion vector MV_X frac (such as MV_X frac =0.25) and MV_Y frac Add them together to obtain the final optimal fractional precision level motion vector MV_X (such as MV_X=3+0.25=3.25) and MV_Y, completing the fractional precision level motion estimation;

[0108] In the "New Generation Video Compression Coding Standard H.264-AVC", 3.1.4.3.2, the fractional motion vector is described as follows: "Due to the continuity of natural object motion, the motion vector of the block between two adjacent frames is not based on integer pixels. The actual motion displacement may be based on sub-pixels such as 1 / 4 pixel or even 1 / 8 pixel." Therefore, fractional precision motion estimation includes integer pixel level motion estimation and sub-pixel level motion estimation: The purpose of integer precision motion estimation is to converge to an optimal integer pixel precision motion vector MV_int within a specific search range, and then further search for the sub-pixel corresponding fractional precision motion vector MV_frac with this optimal integer pixel matching position as the center. Therefore, the fractional precision level motion vector is composed of the optimal integer motion vector MV_X int(such as MV_X int =3) and MV_Y int and the optimal fractional motion vector MV_X centered around the optimal integer motion vector frac (such as MV_X frac =0.25) and MV_Y frac Add together to obtain the optimal fractional precision level motion vector MV_X (such as MV_X=3+0.25=3.25) and MV_Y.

[0109] Embodiment 2: Different from Embodiment 1, in this embodiment, the distortion metric in the rate-distortion cost of the small square directly uses the SATD value to avoid repeated calculation of the SATD value, and the process of finally solving the optimal fractional motion vector adopts a hierarchical search method similar to the prior art.

[0110] The specific steps include:

[0111] Step 1: In integer motion estimation, a search method with a larger step size and a wider search range (such as hexagonal search, diamond search or other applicable search methods) is first used to perform a rough search for the optimal integer motion vector of integer pixels. When the convergence to the minimum rate-distortion cost value is located at the center of the search shape (indicating that the optimal motion vector of the integer coarse search has converged to the center position), a small square search with a step size of 1 integer pixel is used to further refine the search area, that is, each time the search point becomes the center position and 8 adjacent integer pixels. It should be noted that the rate-distortion cost metric used in the coarse search process here is the SAD value.

[0112] Step 2: Continue fine search using small squares with a step size of 1. Here, the rate-distortion cost value is switched to use the SATD value as the metric to avoid recalculating the SATD value. Finally, the SATD value of the corresponding center integer pixel and 8 adjacent integer pixels is obtained. When the small square converges to the center point of 9 integer pixels, which is the minimum value of the rate-distortion cost value, the integer motion vector search with a minimum step size of 1 integer pixel is completed, and an optimal integer motion vector MV_X is obtained. int and MV_Y int , completing integer motion estimation.

[0113] Step 3: Because the metric of the rate-distortion cost value of the small square in the previous step is the SATD value, there is no need to recalculate the SATD value. Instead, directly substitute the SATD and corresponding coordinates corresponding to the 9 integer pixel motion vectors of the small square in the last step into the error surface function model.

[0114] Step 4: Use the coordinate position of the optimal integer motion vector as the center position of the starting point of the fractional motion vector search, and construct a single-peak surface function model whose center position is the lowest point or lower point of the integer pixel planar rate distortion cost. Substitute the values ​​from f(-1,-1) to f(1,1) (i.e., the SATD values ​​in step 3) into the error surface function to obtain the parameters AI, and complete the construction of the error surface function f(x,y).

[0115] Step 5: After obtaining the function coefficient AI and the error surface function f(x, y), first substitute the 8 1 / 2 sub-pixel fractional motion vectors into f(x, y) in a hierarchical search manner, as follows: Figure 1 The SATD values ​​and f(0,0) are calculated for f(-1 / 2,-1 / 2), f(0,-1 / 2), f(1 / 2,-1 / 2), f(1 / 2,0), f(-1 / 2,-1 / 2), f(-1 / 2,-1 / 2), f(-1 / 2,-1 / 2), and f(-1 / 2,-1 / 2) respectively, and the rate-distortion cost values ​​of these 9 1 / 2 level sub-pixels are further calculated. The minimum value of the rate-distortion cost is the optimal fractional motion vector at the 1 / 2 precision level. Figure 1 The motion vector s1 in .

[0116] Step 6: With the optimal fractional motion vector of the 1 / 2 precision level in the previous step as the center, continue to search for the optimal motion vector of 1 / 4 precision; first substitute the 1 / 4 sub-pixel fractional motion vector into f(x, y), such as Figure 1 Calculate the SATD value and the obtained f(1 / 2,1 / 2) for f(1 / 4,-3 / 4), f(1 / 2,-3 / 4), f(3 / 4,-3 / 4), f(1 / 4,-1 / 2), f(3 / 4,-1 / 2), f(1 / 4,-1 / 4), f(1 / 2,-1 / 4), and f(3 / 4,-1 / 4) respectively, and further calculate the rate-distortion cost value of these 9 points. The minimum value of the rate-distortion cost is the optimal fractional motion vector at the 1 / 4 accuracy level. Figure 1 The motion vector s2 in .

[0117] Step 7: The optimal integer motion vector (such as MV_X) obtained in step 2 is int =3), the optimal fractional motion vector (such as MV_X) of the 1 / 2 accuracy level obtained in step 5 1 / 2 =1 / 2) and the optimal fractional motion vector (such as MV_X) with 1 / 4 accuracy obtained in step 6. 1 / 4 =-1 / 4) are added to obtain the optimal fractional precision level motion vector MV_X (such as MV_X=3+1 / 2-1 / 4=3.25) and MV_Y based on the hierarchical search to achieve the 1 / 4 precision level, thus completing the motion estimation at the fractional precision level.

[0118] Embodiment 3: A system for calculating fractional precision motion estimation based on an error surface, comprising:

[0119] The integer motion estimation module is used to perform integer motion estimation on the integer motion vector to obtain the optimal integer motion vector MV_X int and MV_Y int ;

[0120] The fractional motion estimation module is used to perform fractional motion estimation with the optimal integer motion vector as the center to obtain the optimal fractional motion vector MV_X frac and MV_Y frac , and the optimal integer motion vector MV_X int (such as MV_X int =3) and MV_Y int and the optimal fractional motion vector MV_X frac (such as MV_X frac =0.25) and MV_Y frac Add together to obtain the final optimal fractional precision level motion vector MV_X (such as MV_X=3.25) and MV_Y.

[0121] The integer motion estimation module and the fractional motion estimation module both include a rate-distortion cost calculation unit and a rate-distortion cost comparison unit;

[0122] The rate-distortion cost calculation unit is used to calculate the rate-distortion cost value of the integer motion vector or the fractional motion vector;

[0123] The rate-distortion cost comparison unit is used to compare the rate-distortion cost values ​​of integer motion vectors or fractional motion vectors.

[0124] The integer motion estimation module also includes:

[0125] An integer motion vector search unit is used to perform a coarse search on the integer motion vector until the integer motion vector position where the minimum rate distortion cost value is located is located at the exact center of the search shape, then switch to a small square to continue fine search, each time the search point is the center position and 8 adjacent integer pixels, until the integer motion vector position where the minimum rate distortion cost value is located is located at the center position of the small square, then stop the integer motion vector search;

[0126] The SAD and SATD calculation unit is used to calculate the SAD value of the integer motion vector, or use the two-level series one-dimensional Hadamard change matrix to calculate the SATD value of the integer motion vector. The SAD or SATD value is used as one of the distortion metrics for calculating the rate-distortion cost. Generally, the rate-distortion cost is measured by the SAD value in the integer motion vector search. In the fine search or fractional motion vector search, since the SAD value difference at each point is relatively small, the use of the SATD value can better match the position with a smaller bitstream and prevent falling into the local optimum.

[0127] The fractional motion estimation module also includes:

[0128] The error surface calculation unit is used to use the optimal integer motion vector as the center position of the fractional motion vector search starting point, construct an error surface function model with a single peak value whose center position is the lowest point of the rate-distortion cost value in the integer pixel plane, bring the coordinate values ​​and SATD values ​​of the integer pixel and 8 adjacent integer pixels corresponding to the optimal integer motion vector into the error surface function model, solve the polynomial parameters of the error surface function model, and complete the fitting of the fractional level error surface function model;

[0129] An enumeration fractional motion vector unit is used to enumerate candidate fractional coordinates of fractional motion vectors in the range of [-1,1];

[0130] The SATD calculation unit is used to bring the enumerated candidate fractional coordinates into the error surface function model for solution to obtain the SATD value of each fractional motion vector.

[0131] In summary, the present invention does not require additional interpolation calculation of sub-pixels. After substituting the candidate fractional coordinates in the form of an error surface fitting function, the SATD value corresponding to the candidate fractional coordinates is directly obtained, and the rate-distortion cost of the candidate motion vector is further obtained. The optimal motion vector with the smallest rate-distortion cost is selected as the best match. The process is optimized according to the characteristics of hardware calculation. The calculation process can convert multiplication operations into only shift operations and addition and subtraction operations, omitting multiplication and division operations, and each step of the calculation can be expanded into a pipeline operation to improve calculation efficiency and reduce hardware resource occupation. It has the advantages of fast calculation speed, high motion compensation quality, and no need for additional interpolation calculation of sub-pixels.

[0132] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0133] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0134] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for calculating fractional precision motion estimation based on an error surface, characterized in that: The method comprises: S1: Perform a coarse search on the integer motion vector until the integer motion vector position where the minimum rate distortion cost value is located is located in the exact center of the search shape, then switch to a small square to continue fine search; S2: The last stage of integer motion estimation uses a small square with a step size of 1 to search for the minimum rate distortion cost value. Each search point is the center position and 8 adjacent integer pixels. When the integer motion vector position where the minimum rate distortion cost value is located is found to be at the center position of the small square, the integer motion vector search is stopped. S3: After stopping the integer motion vector search, calculate the SATD values ​​of the integer pixel corresponding to the center position of the small square and the 8 adjacent integer pixels, and obtain the integer motion vector corresponding to the center position of the small square at this time as the optimal integer motion vector MV_X int and MV_Y int , complete integer motion estimation; S4: Taking the optimal integer motion vector as the center position of the starting point of the fractional motion vector search, constructing a single-peak error surface function model whose center position is the lowest point of the rate-distortion cost value in the integer pixel plane, bringing the SATD values ​​and corresponding integer coordinate values ​​of the 9 integer pixel points calculated in step S3 into the error surface function model, solving the polynomial parameters of the error surface function model, and completing the fitting of the fractional level error surface function model; S5: Use the enumeration method to enumerate the candidate fractional coordinates in the range of [-1,1] and bring them into the error surface function model for solution to obtain the SATD value of each candidate fractional coordinate; S6: Calculate the rate-distortion cost value of the candidate fractional motion vector corresponding to each candidate fractional coordinate and compare them, output the minimum value of the rate-distortion cost value corresponding to the candidate fractional motion vector, and the candidate fractional motion vector corresponding to the minimum rate-distortion cost value as the optimal fractional motion vector MV_X at the 1 / 4 sub-pixel accuracy level. frac and MV_Y frac , complete the fractional motion vector calculation; S7: The optimal integer motion vector MV_X int and MV_Y int and the optimal fractional motion vector MV_X frac and MV_Y frac Add them together to get the final optimal fractional precision level motion vectors MV_X and MV_Y, and complete the fractional precision level motion estimation.

2. The method for fractional precision motion estimation based on error surface calculation according to claim 1, characterized in that: In step S3, the SATD values ​​of the integer pixel corresponding to the center position of the small square and the eight adjacent integer pixels are calculated, specifically by using a two-level series one-dimensional Hadamard change matrix for calculation.

3. The method for fractional precision motion estimation based on error surface calculation according to claim 1, characterized in that: The error surface function model includes a 5-parameter model, a 6-parameter model and a 9-parameter model, and the function expressions are: Where x and y represent the pixel coordinates, f(x, y) is the SATD value corresponding to the x and y coordinates; AI is the polynomial parameter of the error surface function model.

4. The method for fractional precision motion estimation based on error surface calculation according to claim 3, characterized in that: When the error surface function model is a 9-parameter model, the polynomial parameters of the error surface function model are solved by the following method: The optimal integer motion vector is used as the center position of the starting point of the fractional motion vector search, denoted as the origin (0,0). The coordinates of the eight adjacent integer pixels are (-1,-1), (0,-1), (1,-1), (-1,0), (1,0), (-1,1), (0,1), (1,1). The SATD values ​​corresponding to these nine integer pixels are denoted as f(-1,-1) to f(1,1). They are substituted into the function expression of the nine-parameter model to obtain: Simplify the above formula into a matrix expression, expressed as: f(x,y)=Q*P; Where f(x,y) is the SATD value corresponding to the x and y coordinates, Q represents the constant matrix corresponding to the product of the pixel point coordinate polynomial x or y, and P represents the polynomial parameter AI; The 9 integer pixel coordinates are independent of each other, so the determinant of Q is not equal to zero, and there is a reversible matrix. By solving the least squares method, we get the following expression: P=(Q T Q) -1 Q T *f(x,y); Where T represents matrix transpose; (Q T Q) -1 Q T is a constant matrix; Solve the polynomial parameter AI:

5. The method for fractional precision motion estimation based on error surface calculation according to claim 1, characterized in that: The enumeration of fractional coordinates in the range of [-1, 1] specifically enumerates fractional coordinates of multiples of 1 / 2, 1 / 4 and 1.

6. The method for fractional precision motion estimation based on error surface calculation according to claim 1, characterized in that: In step S6, the rate-distortion cost value of the candidate fractional motion vector corresponding to each fractional coordinate is calculated using the following formula: cost = S + λ * R; Where cost represents the rate-distortion cost; S represents the difference measure between the reference macroblock and the motion prediction macroblock, and the SATD value is used as the measure for calculation; R represents the number of bits required to measure the motion information itself, which is obtained according to the preset lookup table corresponding to the coordinate value; λ is the weight coefficient.

7. A system based on the method for calculating fractional precision motion estimation based on error surface according to any one of claims 1 to 6, characterized in that: The system comprises: The integer motion estimation module is used to perform integer motion estimation on the integer motion vector to obtain the optimal integer motion vector MV_X int and MV_Y int ; The fractional motion estimation module is used to perform fractional motion estimation centered on the optimal integer motion vector to obtain the optimal fractional motion vector MV_X frac and MV_Y frac , and the optimal integer motion vector MV_X int and MV_Y int and the optimal fractional motion vector MV_X frac and MV_Y frac Add them together to get the final optimal fractional accuracy level motion vectors MV_X and MV_Y.

8. The system according to claim 7, characterized in that The integer motion estimation module and the fractional motion estimation module each include a rate-distortion cost calculation unit and a rate-distortion cost comparison unit; The rate-distortion cost calculation unit is used to calculate the rate-distortion cost value of the integer motion vector or the fractional motion vector; The rate-distortion cost comparison unit is used to compare the rate-distortion cost values ​​of integer motion vectors or fractional motion vectors.

9. The system according to claim 8, characterized in that The integer motion estimation module further includes: an integer motion vector search unit, configured to perform a coarse search on the integer motion vector until the integer motion vector position where the minimum rate distortion cost value is located is located at the exact center of the search shape, then switch to a small square to continue a fine search, each time the search point is the center position and 8 adjacent integer pixels, until the integer motion vector position where the minimum rate distortion cost value is located is located at the center position of the small square, then stop the integer motion vector search; The SAD and SATD calculation unit is used to calculate the SAD value of the integer motion vector, or to calculate the SATD value of the integer motion vector by using a two-level series-connected one-dimensional Hadamard change matrix.

10. The system according to claim 9, characterized in that The fractional motion estimation module further comprises: The error surface calculation unit is used to use the optimal integer motion vector as the center position of the fractional motion vector search starting point, construct an error surface function model with a single peak value whose center position is the lowest point of the rate-distortion cost value in the integer pixel plane, bring the coordinate values ​​and SATD values ​​of the integer pixel and 8 adjacent integer pixels corresponding to the optimal integer motion vector into the error surface function model, solve the polynomial parameters of the error surface function model, and complete the fitting of the fractional level error surface function model; An enumeration fractional motion vector unit is used to enumerate candidate fractional coordinates in the range of [-1,1]; The SATD calculation unit is used to bring the enumerated candidate fractional coordinates into the error surface function model for solution to obtain the SATD value of each fractional motion vector.

Images