A fractional pixel search method, apparatus, device, storage medium and product

Abstract

Embodiments of the present application provide a fractional pixel search method, device, equipment, storage medium and product. The technical solution provided by the embodiments of the present application determines an initial motion vector of first fractional pixel search according to integer pixel search of a to-be-encoded image, determines an acceleration mode of the first fractional pixel search according to a first calculation cost of the initial motion vector and a second calculation cost of a neighboring motion vector corresponding to the initial motion vector in a set search direction, performs first fractional pixel search on the to-be-encoded image according to the initial motion vector and the acceleration mode to obtain a first fractional pixel search result, determines a search direction to be skipped by second fractional pixel search according to the acceleration mode, performs second fractional pixel search on the to-be-encoded image based on the reserved search direction to obtain a second fractional pixel search result, effectively reduces the number of times and the amount of calculation of fractional pixel search, and improves the image coding efficiency on the premise of ensuring the image coding quality.

Description

Technical Field

[0001] This application relates to the field of coding technology, and in particular to a fractional pixel search method, apparatus, device, storage medium, and product. Background Technology

[0002] In the pattern analysis of HEVC (High Efficiency Video Coding) coding, the motion search part of the inter-frame coding unit is an important part of the pattern analysis. The motion search of HEVC coding includes integer pixel search and fractional pixel search, among which fractional pixel search includes half-pixel search and quarter-pixel search.

[0003] Integer search typically relies on the computational cost of summing the absolute values ​​of residuals. A pre-defined search algorithm is used to find the optimal integer motion vector that minimizes this computational cost. Then, based on this optimal integer motion vector, the optimal motion vector for one-half a pixel is found in the surrounding area with a certain offset. After finding the optimal motion vector for one-half a pixel, the optimal motion vector for one-quarter a pixel is found, using this one-half-pixel optimal motion vector as the center. However, fractional pixel searches require interpolation of reference frames, resulting in a high computational cost for inter-frame motion search and consequently, low image coding efficiency. Summary of the Invention

[0004] This application provides a fractional pixel search method, apparatus, device, storage medium, and product to solve the technical problem that the computational load of fractional pixel search in inter-frame motion search is large, resulting in low image coding efficiency, thereby effectively reducing the computational load of fractional pixel search and improving image coding efficiency.

[0005] In a first aspect, embodiments of this application provide a fractional pixel search method, comprising:

[0006] The initial motion vector for the first fractional pixel search of the image to be encoded is determined based on the integer pixel search results of the image to be encoded;

[0007] Based on the first computational cost of the initial motion vector and the second computational cost corresponding to the adjacent motion vectors of the initial motion vector in the set search direction, an acceleration method for the first fractional pixel search is determined. The first computational cost and the second computational cost are obtained based on the integer pixel search of the image to be encoded.

[0008] Based on the initial motion vector and the acceleration method, a first fractional pixel search is performed on the image to be encoded to obtain a first fractional pixel search result;

[0009] Based on the acceleration method, the search direction skipped during the second fractional pixel search of the image to be encoded is determined, and the second fractional pixel search result is obtained by performing the second fractional pixel search on the image to be encoded based on the retained search direction.

[0010] In a second aspect, embodiments of this application provide a fractional pixel search device, including a vector determination module, an acceleration determination module, a first search module, and a second search module, wherein:

[0011] The vector determination module is configured to determine the initial motion vector for the first fractional pixel search of the image to be encoded based on the integer pixel search results of the image to be encoded.

[0012] The acceleration determination module is configured to determine the acceleration method for the first fractional pixel search based on a first computational cost of the initial motion vector and a second computational cost corresponding to adjacent motion vectors of the initial motion vector in a set search direction. The first computational cost and the second computational cost are obtained based on an integer pixel search of the image to be encoded.

[0013] The first search module is configured to perform a first fractional pixel search on the image to be encoded based on the initial motion vector and the acceleration method to obtain a first fractional pixel search result;

[0014] The second search module is configured to determine the search direction skipped during the second fractional pixel search of the image to be encoded based on the acceleration method, and to perform a second fractional pixel search on the image to be encoded based on the retained search direction to obtain the second fractional pixel search result.

[0015] In a third aspect, embodiments of this application provide a fractional pixel search device, including: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the fractional pixel search method as described in the first aspect.

[0018] In a fourth aspect, embodiments of this application provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the fractional pixel search method as described in the first aspect.

[0019] In a fifth aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the fractional pixel search method as described in the first aspect.

[0020] This embodiment of the application determines the initial motion vector for the first fractional pixel search based on the integer pixel search of the image to be encoded, and determines the acceleration method for the first fractional pixel search based on the first computational cost of the initial motion vector and the second computational cost corresponding to the adjacent motion vectors of the initial motion vector in the set search direction. The first fractional pixel search is performed on the image to be encoded based on the initial motion vector and the acceleration method to obtain the first fractional pixel search result. The search direction to be skipped in the second fractional pixel search is determined based on the acceleration method. The second fractional pixel search is performed on the image to be encoded based on the retained search direction to obtain the second fractional pixel search result. The acceleration method for the first fractional pixel search is determined based on the integer pixel search result, and the search direction to be skipped in the second fractional pixel search is determined based on the first fractional pixel search result. This effectively reduces the number of fractional pixel searches and the computational load, improving image encoding efficiency while ensuring image encoding quality. Attached Figure Description

[0021] Figure 1 This is a flowchart of a fractional pixel search method provided in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram illustrating a process for determining an acceleration method for searching a first fractional pixel, provided in an embodiment of this application.

[0023] Figure 3 This is a schematic diagram of the structure of a fractional pixel search device provided in an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the structure of a fractional pixel search device provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.

[0026] The fractional pixel search method provided in this application can be applied to the pattern analysis part of image coding (e.g., image coding based on HEVC encoders). It aims to reduce the number of fractional pixel searches and computational load by determining the acceleration method for the first fractional pixel search based on the integer pixel search results, and determining the search direction to skip for the second fractional pixel search based on the first fractional pixel search results. This improves image coding efficiency while ensuring image coding quality. Traditional fractional pixel search schemes are similar to integer pixel searches in that they calculate the cost of each motion vector (mv, motion vector) in the search space and use the motion vector with the lowest computational cost as the corresponding fractional pixel search result. Since interpolation of reference frames is required when performing fractional pixel searches on the image to be coded, the computational load of fractional pixel searches for inter-frame motion search is large, resulting in low image coding efficiency. Therefore, this application provides a fractional pixel search method according to an embodiment to solve the technical problem of high computational load and low image coding efficiency in existing fractional pixel search schemes.

[0027] Figure 1 A flowchart of a fractional pixel search method provided in an embodiment of this application is given. The fractional pixel search method provided in this embodiment of the application can be executed by a fractional pixel search device, which can be implemented by hardware and / or software and integrated into a fractional pixel search device.

[0028] The following description uses a fractional-pixel search device to perform a fractional-pixel search method as an example. (Reference) Figure 1 The fractional pixel search method includes:

[0029] S101: Determine the initial motion vector for the first fractional pixel search of the image to be encoded based on the integer pixel search results of the image to be encoded.

[0030] When performing pattern analysis on the image to be encoded, the pixel search results of the image need to be referenced. Generally, pixel search includes integer pixel search and fractional pixel search (FME, Fractional Motion Estimation). Fractional pixel search typically includes a first fractional pixel search (1 / 2 pixel search) and a second fractional pixel search (1 / 4 pixel search). The search precision of the second fractional pixel search is higher than that of the first fractional pixel search. For example, the first fractional pixel search can be a half-pixel search (1 / 2 pixel search), and the second fractional pixel search can be a quarter-pixel search (1 / 4 pixel search). Pixel search of the image to be encoded is generally performed sequentially as integer pixel search, first fractional pixel search, and second fractional pixel search. After determining the integer pixel search results, first fractional pixel search results, and second fractional pixel search results, pattern analysis of the image to be encoded can continue.

[0031] In one embodiment, when performing an integer pixel search on the image to be encoded, based on a set search algorithm, the adjacent motion vectors of adjacent pixels in each direction corresponding to the set search algorithm can be calculated based on the current motion vector (center motion vector) corresponding to the current pixel. The computational cost of the current motion vector and each adjacent motion vector is also calculated. If the computational cost of the current motion vector is less than the computational cost of the adjacent motion vectors, or if the number of iterations of the current integer pixel search has reached a set iteration threshold, the integer pixel search can be terminated, and the current motion vector is determined as the optimal motion vector corresponding to the integer pixel search result. If there exists an adjacent motion vector whose computational cost is less than or equal to the computational cost of the current motion vector, and the number of iterations of the current integer pixel search has not reached the set iteration threshold, the adjacent pixel with the lowest computational cost is taken as the center motion vector for the next iteration of the current integer pixel search, and the iteration continues to find the optimal motion vector according to the above integer pixel search method. The pixel corresponding to the initial center motion vector in the integer pixel search can be the center point of the image to be encoded, or a pixel predicted by the encoder.

[0032] Optionally, the computational cost provided by this scheme can be calculated based on the sum of absolute differences (SAD) or the sum of absolute differences after transformation (SATD). For example, integer pixel search can use the cost of the sum of absolute differences (SAD) as the computational cost, while fractional pixel search can use the cost of the sum of absolute differences after transformation (SATD) as the computational cost.

[0033] For example, after completing the integer pixel search of the image to be encoded, an initial motion vector for performing a first fractional pixel search on the image to be encoded is determined based on the integer pixel search results. Subsequently, when performing the first fractional pixel search on the image to be encoded, this initial motion vector is used as the center vector to find the optimal motion vector corresponding to the first fractional pixel search. In one embodiment, the optimal motion vector corresponding to the integer pixel search results can be used as the initial motion vector for the binary search.

[0034] Optionally, the search algorithm provided in this scheme can be a diamond search (DIA), a hexagonal search (HEX), or a full search (FULL). The diamond search detects the motion vectors in the four directions (up, down, left, and right) of the initial central motion vector, calculates the computational cost of each of these four vectors, and selects the direction with the lowest computational cost as the initial direction for the next diamond search. If the central motion vector's computational cost is lower than that of all four directions, the central motion vector is directly identified as the optimal motion vector, and the search is terminated prematurely. The hexagonal search is similar to the diamond search, detecting the motion vectors in the six directions surrounding the initial central motion vector, calculating the corresponding computational cost, and selecting the direction with the lowest computational cost as the initial direction for the next hexagonal search. If the central motion vector's computational cost is lower than that of all six directions, the central motion vector is directly identified as the optimal motion vector, and the search is terminated prematurely. The full search requires calculating the computational cost of the motion vectors in every direction of the search space and selecting the motion vector with the lowest computational cost as the optimal motion vector.

[0035] S102: Based on the first computational cost of the initial motion vector and the second computational cost corresponding to the adjacent motion vectors of the initial motion vector in the set search direction, determine the acceleration method for the first fractional pixel search. The first computational cost and the second computational cost are obtained based on the integer pixel search of the image to be encoded.

[0036] For example, the adjacent motion vectors of the initial motion vector in the set search directions are determined. For instance, when the set search algorithm is a rhombus search, the corresponding search directions are the four directions (up, down, left, and right) of the central motion vector (initial motion vector). When the set search algorithm is a hexagonal search, the corresponding search directions are the six directions surrounding the central motion vector (initial motion vector). When the set search algorithm is a full search, the corresponding search directions are the eight directions surrounding the central motion vector (initial motion vector).

[0037] A first computational cost for the initial motion vector and a second computational cost corresponding to each adjacent motion vector are determined. An acceleration method for the subsequent first fractional pixel search of the image to be encoded is then determined based on the first and second computational costs. Optionally, the acceleration method for the first fractional pixel search can be to determine the reliability of the integer pixel search result based on the first and second computational costs. When the reliability meets a set requirement, the integer pixel search result can be directly used as the first fractional pixel search result, and the first fractional pixel search can be skipped to perform the second fractional pixel search. The reliability meeting the set requirement can be defined as the minimum second computational cost being greater than a threshold determined based on the first computational cost (greater than the first computational cost). When the reliability does not meet the set requirement, the search for the direction least likely to find the optimal motion vector (e.g., the direction of the adjacent motion vector corresponding to the maximum second computational cost) can be skipped in the first iteration.

[0038] S103: Perform a first fractional pixel search on the image to be encoded based on the initial motion vector and acceleration method to obtain the first fractional pixel search result.

[0039] For example, based on the acceleration method for the first fractional pixel search determined above, the first fractional pixel search is performed on the image to be encoded, starting from the determined initial motion vector as the center motion vector. The optimal motion vector for the first fractional pixel search is found, thereby obtaining the first fractional pixel search result.

[0040] In one embodiment, when performing a first fractional pixel search on the image to be encoded, the computational cost of the corresponding fractional precision of the adjacent motion vectors of the initial motion vector in the set search direction is calculated. If the computational cost of the corresponding fractional precision of the initial motion vector is less than the computational cost of the corresponding fractional precision of each adjacent motion vector, then the initial motion vector is taken as the optimal motion vector of the first fractional pixel search. Otherwise, the adjacent motion vector with the smallest computational cost is taken as the center motion vector of the next iteration of the first fractional pixel search, and the process is iterated in the above manner until the optimal motion vector is found, or until the set iteration number threshold is reached.

[0041] S104: Determine the search direction skipped during the second fractional pixel search of the image to be encoded based on the acceleration method, and perform the second fractional pixel search of the image to be encoded based on the retained search direction to obtain the second fractional pixel search result.

[0042] For example, the search directions skipped during the second fractional pixel search of the image to be encoded are determined based on the acceleration method of the first fractional pixel search. For instance, when the acceleration method for the first fractional pixel search is to skip the second fractional pixel search, search directions that are not skipped during the second fractional pixel search can be determined, i.e., all search directions in the set search directions are retained, reducing the possibility of excessive calculations for skipped fractional pixel searches affecting encoding quality. When the acceleration method for the first fractional pixel search is to skip the search in the direction least likely to find the optimal motion vector, the search directions skipped during the second fractional pixel search can be the direction corresponding to the motion vector with the highest computational cost for the fractional precision among the adjacent motion vectors of the optimal motion vector corresponding to the first fractional pixel search result. The skipped search directions are excluded from the set search directions, and the remaining search directions are the retained search directions.

[0043] Optionally, there can be one or more skipped search directions. When there are multiple skipped search directions, the directions corresponding to multiple adjacent motion vectors are determined sequentially in descending order of computational cost to determine the skipped search directions. Optionally, the skipped search directions during the second fractional pixel search can be applied only during the first iteration of the second fractional pixel search, that is, in subsequent iterations of the second fractional pixel search, the second fractional pixel search is performed based on the complete set search directions.

[0044] In one embodiment, the optimal motion vector corresponding to the first fractional pixel search result is used as the initial center motion vector. This initial center motion vector is then used as the iterative starting point for the second fractional pixel search, and the second fractional pixel search is performed on the image to be encoded. The optimal motion vector for the second fractional pixel search is found, thus obtaining the second fractional pixel search result.

[0045] For example, when performing a second fractional pixel search on the image to be encoded, the computational cost of the corresponding fractional precision of the adjacent motion vectors in the retained search direction of the initial center motion vector is calculated. If the computational cost of the corresponding fractional precision of the initial center motion vector is less than the computational cost of the corresponding fractional precision of each adjacent motion vector, then the initial motion vector is taken as the optimal motion vector for the second fractional pixel search. Otherwise, the adjacent motion vector with the smallest computational cost is taken as the center motion vector for the next iteration of the second fractional pixel search, and the process is iterated in the above manner until the optimal motion vector is found (subsequent iterations will not skip the search direction, that is, all the set search directions are retained), or until the set iteration number threshold is reached.

[0046] The above describes a method for determining the initial motion vector for the first fractional pixel search based on integer pixel searches of the image to be encoded. It then determines the acceleration method for the first fractional pixel search based on the first computational cost of the initial motion vector and the second computational cost corresponding to adjacent motion vectors along a set search direction. The method performs a first fractional pixel search on the image to be encoded based on the initial motion vector and the acceleration method to obtain the first fractional pixel search result. Based on the acceleration method, it determines the search direction to be skipped in the second fractional pixel search. Based on the retained search direction, it performs a second fractional pixel search on the image to be encoded to obtain the second fractional pixel search result. By determining the acceleration method for the first fractional pixel search based on the integer pixel search result and the search direction to be skipped in the second fractional pixel search based on the first fractional pixel search result, the number of fractional pixel searches and the computational load can be effectively reduced, improving image encoding efficiency while ensuring image encoding quality.

[0047] In one possible embodiment, the fractional pixel search method provided by this solution, when determining the acceleration method for the first fractional pixel search based on the first computational cost of the initial motion vector and the second computational cost corresponding to the adjacent motion vectors of the initial motion vector in the set search direction, includes:

[0048] If the first computational cost of the initial motion vector is less than or equal to the second computational cost corresponding to the adjacent motion vectors in each of the set search directions of the initial motion vector, it is determined whether to skip the first fractional pixel search of the image to be encoded or to skip some search directions during the first iteration of the first fractional pixel search, based on the first computational cost and the second computational cost; if the first computational cost of the initial motion vector is greater than the second computational cost corresponding to the adjacent motion vectors in one of the set search directions of the initial motion vector, it is determined whether to skip some search directions during the first iteration of the first fractional pixel search.

[0049] For example, after searching for the initial motion vector of the first fractional pixel of the image to be encoded, a first computational cost of the initial motion vector at integer pixel precision and a second computational cost at integer pixel precision corresponding to the adjacent motion vectors in each search direction of the initial motion vector are determined, and the first computational cost and each second computational cost are compared.

[0050] When the first computational cost is less than or equal to the second computational cost corresponding to each adjacent motion vector, the acceleration method for the first fractional pixel search is determined based on whether the first computational cost and the second computational cost should skip the first fractional pixel search of the image to be encoded, or skip part of the search direction during the first iteration of the first fractional pixel search. When the first computational cost is greater than the second computational cost corresponding to at least one of the adjacent motion vectors, the acceleration method for the first fractional pixel search is determined to be skipping part of the search direction during the first iteration of the first fractional pixel search.

[0051] This scheme uses the computational cost of the optimal motion vector obtained by integer pixel search in the set search direction to determine whether all or part of the search direction can be ignored when performing the first fractional pixel search. It accurately determines the acceleration method for the first fractional pixel search, reduces the amount of computation in unnecessary search directions when performing the second fractional pixel search, and improves image coding efficiency.

[0052] It should be explained that the first computational cost of the initial motion vector and the second computational cost of each adjacent motion vector have already been obtained during the integer pixel search (during the last iteration of the integer pixel search). This scheme does not need to recalculate the first and second computational costs; it can directly call the computational costs calculated during the integer pixel search.

[0053] In one possible embodiment, such as Figure 2 The provided flowchart illustrates a process for determining an acceleration method for the first fractional pixel search. The fractional pixel search method in this solution determines, based on a first computational cost and a second computational cost, whether to skip the first fractional pixel search of the image to be encoded, or to skip part of the search direction during the first iteration of the first fractional pixel search. This includes:

[0054] S1021: Determine the cost threshold based on the first calculation cost, where the cost threshold is greater than the first calculation cost.

[0055] S1022: If the second computational cost corresponding to the adjacent motion vectors of the initial motion vector in the set search direction is greater than the cost threshold, determine to skip the first fractional pixel search of the image to be encoded.

[0056] S1023: If there is an adjacent motion vector in the initial motion vector in the set search direction whose second computational cost is less than or equal to the cost threshold, determine to skip part of the search direction during the first iteration of the first fractional pixel search.

[0057] For example, when the first computational cost of the initial motion vector is less than or equal to the second computational costs of each adjacent motion vector, a cost threshold is determined based on the first computational cost. Optionally, the cost threshold can be determined based on the first computational cost and a set adjustment range, adjustment ratio, or adjustment method. For example, when determining the cost threshold based on the first computational cost and a set adjustment range, the sum of the first computational cost and the adjustment range can be used as the cost threshold. When determining the cost threshold based on the first computational cost and a set adjustment ratio, the product of the first computational cost and the adjustment ratio (e.g., 1.1 to 1.8) can be used as the cost threshold. When determining the cost threshold based on the first computational cost and a set adjustment method, the first computational cost can be processed based on the adjustment method to obtain the cost threshold, for example, by adding the first computational cost to the value obtained by right-shifting the first computational cost by one bit (i.e., dividing the first computational cost by 2).

[0058] Furthermore, the second calculation generation corresponding to each adjacent motion vector in the set search direction of the initial motion vector is compared with the cost threshold determined above. When the second calculation generation corresponding to each adjacent motion vector is greater than the cost threshold, the acceleration method for the first fractional pixel search is determined to be to skip the first fractional pixel search of the image to be encoded, directly determine the second fractional pixel search result and perform the second fractional pixel search.

[0059] When there are adjacent motion vectors whose second computational cost is less than or equal to the cost threshold, the acceleration method for the first fractional pixel search is determined to be skipping some search directions during the first iteration of the first fractional pixel search. The number of search directions skipped is set according to different search algorithms. For example, rhombus search, hexagonal search, and full search skip 1, 2, and 3 search directions, respectively. Of course, the number of search directions skipped for different search algorithms can also be the same, for example, skipping one search direction for each algorithm.

[0060] This solution sets a cost threshold based on the first computational cost, accurately determines whether to skip the entire first fractional pixel search or a portion of the first fractional pixel search, and accurately determines the conditions for skipping the first fractional pixel search based on the integer pixel search results, effectively reducing the computational load of the first fractional pixel search and improving its search efficiency.

[0061] In one possible embodiment, the fractional pixel search method provided by this solution, when performing a first fractional pixel search on the image to be encoded based on the initial motion vector and acceleration method to obtain a first fractional pixel search result, includes:

[0062] When the acceleration method for the first fractional pixel search is to skip the first fractional pixel search of the image to be encoded, the initial motion vector is determined as the first fractional pixel search result; when the acceleration method for the first fractional pixel search is to skip some search directions in the first iteration of the first fractional pixel search, the search directions skipped in the first iteration of the first fractional pixel search are determined based on the sorting of the second computational cost and the set first skip number, and the first fractional pixel search of the image to be encoded is performed based on the retained search directions to obtain the first fractional pixel search result.

[0063] For example, after determining the acceleration method for the first fractional pixel search of the image to be encoded, the first fractional pixel search is performed on the image to be encoded based on the determined acceleration method. When the acceleration method for the first fractional pixel search is to skip the first fractional pixel search of the image to be encoded, all searches for motion vectors at the first fractional pixel precision can be skipped, and the aforementioned determined initial motion vector can be determined as the optimal motion vector corresponding to the first fractional pixel search result, allowing direct search for motion vectors at the subsequent second fractional pixel precision.

[0064] When accelerating the first fractional pixel search by skipping some search directions during the initial iteration, the search directions skipped during the initial fractional pixel search are determined based on the sorting of the second computational cost from largest to smallest and the set number of first skips. After determining the skipped search directions, the first fractional pixel search is performed on the image to be encoded based on the retained search directions, and the first fractional pixel search results are obtained. For example, assuming the set number of first skips is 1, the direction corresponding to the adjacent motion vector with the largest second computational cost is determined as the search direction skipped during the initial iteration of the first fractional pixel search.

[0065] For example, in the first iteration of the first fractional pixel search, the initial motion vector is used as the center motion vector. The computational cost of the initial motion vector at the first fractional pixel precision is calculated, as well as the computational cost of the adjacent motion vectors at the first fractional pixel precision along the retained search direction. The adjacent motion vector with the lowest computational cost is then used as the center motion vector for the next iteration of the first fractional pixel search. This process continues until a center motion vector with a computational cost less than that of its adjacent motion vectors is found, or the number of iterations reaches a set threshold. The finally determined center motion vector is then identified as the optimal motion vector corresponding to the first fractional pixel search result. This scheme accelerates the search efficiency of the first fractional pixel search using different acceleration methods, effectively reducing unnecessary searches for the optimal motion vector along the search direction and improving image coding efficiency.

[0066] In one possible embodiment, the fractional pixel search method provided by this solution, when determining the search direction skipped during the second fractional pixel search of the image to be encoded based on the acceleration method, includes:

[0067] When the acceleration method for the first fractional pixel search is to skip the first fractional pixel search of the image to be encoded, all search directions for the second fractional pixel search of the image to be encoded are determined to be retained; when the acceleration method for the first fractional pixel search is to skip some search directions during the first iteration of the first fractional pixel search, the search directions skipped during the second fractional pixel search of the image to be encoded are determined based on the adjacent motion vectors corresponding to the last iteration of the first fractional pixel search.

[0068] For example, the acceleration method for the first fractional pixel search is to skip the first fractional pixel search of the image to be encoded, while retaining all search directions for the second fractional pixel search of the image to be encoded, so as to reduce the number of calculations of the skipped fractional pixel search and thus affect the encoding quality.

[0069] When the acceleration method for the first fractional pixel search involves skipping some search directions during the initial iteration of the first fractional pixel search, the search directions to be skipped during the second fractional pixel search of the image to be encoded are determined based on the computational cost of the adjacent motion vectors with the first fractional pixel precision of the center motion vector in each set search direction during the last iteration of the first fractional pixel search. For example, the direction corresponding to the adjacent motion vector with the highest computational cost among all adjacent motion vectors is taken as the search direction to be skipped in the second fractional pixel search. This scheme effectively reduces the computational load of the second fractional pixel search and improves its search efficiency by determining the search directions to be skipped in the second fractional pixel search based on the acceleration method for the first fractional pixel search.

[0070] In one possible embodiment, the fractional pixel search method provided by this solution, when determining the search direction skipped during the second fractional pixel search of the image to be encoded based on the neighboring motion vectors corresponding to the last iteration of the first fractional pixel search, includes:

[0071] Determine the third computational cost corresponding to the adjacent motion vectors in the set search direction of the iteration center motion vector during the last iteration of the first fractional pixel search; based on the sorting of the third computational cost and the set second skip number, determine the search direction skipped during the first iteration of the second fractional pixel search of the image to be encoded.

[0072] For example, if the first fractional pixel search is not skipped, then during the last iteration of the first fractional pixel search, the third computational cost corresponding to the adjacent motion vectors in each search direction of the center motion vector in the last iteration of the first fractional pixel search is recorded. Here, the third computational cost is the computational cost of the first fractional pixel precision corresponding to the adjacent motion vectors in the search direction corresponding to the center motion vector during the first fractional pixel search. It should be explained that the computational cost of the center motion vector is less than the third computational cost corresponding to each adjacent motion vector, or the number of iterations of the second fractional pixel search reaches a set iteration threshold, and the current first fractional pixel search is determined to be the last iteration of the first fractional pixel search.

[0073] Furthermore, based on the sorting of the third computational cost from largest to smallest and the set second skip number, the search direction skipped during the first iteration of the second fractional pixel search is determined. After determining the skipped search direction, the second fractional pixel search is performed on the image to be encoded based on the retained search direction, and the second fractional pixel search result is obtained. For example, assuming the set second skip number is 1, the direction corresponding to the adjacent motion vector with the largest third computational cost is determined as the search direction skipped during the first iteration of the second fractional pixel search.

[0074] For example, in the first iteration of the second fractional pixel search, the optimal motion vector corresponding to the first fractional pixel search result is used as the center motion vector. The computational cost of the center motion vector at the second fractional pixel precision is calculated, as well as the computational cost of the adjacent motion vectors at the second fractional pixel precision in the retained search direction. The adjacent motion vector with the lowest computational cost is then used as the center motion vector for the next iteration of the second fractional pixel search. This process continues until a center motion vector with a computational cost less than that of its adjacent motion vectors is found, or the number of iterations reaches a set threshold. The finally determined center motion vector is then identified as the optimal motion vector corresponding to the second fractional pixel search result. This scheme determines the skipped search directions in the second fractional pixel search based on the first fractional pixel search result, effectively reducing the computational load of the second fractional pixel search and improving its search efficiency.

[0075] In one possible embodiment, after determining the initial motion vector of the first fractional pixel search of the image to be encoded based on the integer pixel search results of the image to be encoded, the fractional pixel search method provided by this solution further includes: if the number of searches corresponding to the integer pixel search results of the image to be encoded reaches a set integer search number threshold, determining to skip part of the search direction during the first iteration of the first fractional pixel search, and performing the first fractional pixel search on the image to be encoded based on the retained search direction to obtain the first fractional pixel search result.

[0076] For example, after determining the initial motion vector for the first fractional pixel search of the image to be encoded, if the optimal motion vector corresponding to the integer pixel search result is the center motion vector whose computational cost is less than that of the adjacent motion vectors, then an acceleration method for the first fractional pixel search is determined based on the first computational cost and the second computational cost. If the optimal motion vector corresponding to the integer pixel search result is the center motion vector determined when the number of searches for the integer pixel search reaches the integer search number threshold, then it is determined that some search directions are skipped during the first iteration of the first fractional pixel search, and the first fractional pixel search is performed on the image to be encoded based on the retained search directions to obtain the first fractional pixel search result. The possibility of skipping the first fractional pixel search is small, and there is no need to determine whether to skip the first fractional pixel search.

[0077] This scheme skips the decision on whether to skip the first fractional pixel search when the number of searches for integer pixels reaches a set threshold. Instead, it directly performs the first fractional pixel search based on the determined skipped search direction, reducing unnecessary decision-making processes and improving image encoding efficiency.

[0078] For example, assuming that the search algorithms used for integer pixel search, first fractional pixel search, and second fractional pixel search are all diamond search algorithms, and the number of skipped search directions is set to 1, the optimal motion vector corresponding to the integer pixel search result is used as the initial motion vector for the first fractional pixel search, and the first computational cost corresponding to the initial motion vector is Cost. cen The second computational cost corresponding to the adjacent motion vectors in the four directions (up, down, left, and right) of the initial motion vector is Cost. up Cost down Cost left and Cost right At Cost cen ≤min(Cost up Cost down Cost left Cost right When determining the first computational cost of the initial motion vector, it is determined that it is less than or equal to each of the second computational costs. Then, a cost threshold is set and it is determined whether to skip the first fractional pixel search.

[0079] Let the minimum second computational cost among the adjacent motion vectors of the initial motion vector be Cost. min =min(Cost) up Cost down Cost left Cost right The cost threshold can be set to: Cost thre =Cost cen +Cost cen>>1, that is, the sum of the first computational cost and the first computational cost shifted one bit to the right (that is, the first computational cost divided by 2) is used as the cost threshold. When the Cost is satisfied... min Cost thre When the first fractional pixel search of the image to be encoded is skipped, the initial motion vector is used as the optimal motion vector corresponding to the first fractional pixel search result.

[0080] If Cost is not met min Cost thre Then, determine the search direction corresponding to the maximum second computational cost among the adjacent motion vectors of the initial motion vector: dir = argmax(Cost) up Cost down Cost left Cost right ), and the search

[0081] dir

[0082] The direction is determined by skipping some search directions during the first iteration of the first fractional pixel search, and using the initial motion vector as the center motion vector for the first iteration. Based on the retained search directions, the first fractional pixel search is performed on the image to be encoded to obtain the first fractional pixel search result. Furthermore, during the last iteration of the first fractional pixel search, the third computational cost corresponding to the adjacent motion vectors in the four search directions (up, down, left, and right) of the iteration center motion vector of the first fractional pixel search is recorded. and At Cost cen >min(Cost up Cost down Cost left Cost right When determining the first computational cost of the initial motion vector, it is determined that it is at least greater than one of the second computational costs, and the search direction corresponding to the maximum second computational cost is skipped during the first iteration of the first fractional pixel search. Similarly, during the last iteration of the first fractional pixel search, the third computational cost corresponding to the adjacent motion vectors in the four search directions (up, down, left, and right) of the iteration center motion vector of the first fractional pixel search is recorded.

[0083] If, during the first fractional pixel search, the search direction corresponding to the maximum second computational cost is skipped only during the first iteration of the first fractional pixel search, then the direction corresponding to the maximum third computational cost is selected as the search direction skipped during the first iteration of the second fractional pixel search of the image to be encoded: The second fractional pixel search is performed on the image to be encoded based on the retained search directions to obtain the second fractional pixel search result. If the first fractional pixel search is skipped and the first fractional pixel search result is determined directly, then all search directions for the second fractional pixel search are retained, and the second fractional pixel search is performed to obtain the second fractional pixel search result. This scheme determines whether to skip the entire first fractional pixel search or skip the vector search in some search directions based on the adjacent motion vectors corresponding to the integer pixel search results. Then, based on the first fractional pixel search results, it determines whether to skip the upper vector search in some search directions for the second fractional pixel search. This effectively reduces the computational load of the fractional pixel search and improves image encoding efficiency while ensuring encoding quality.

[0084] Figure 3 This is a schematic diagram of the structure of a fractional pixel search device provided in an embodiment of this application. (Reference) Figure 3 The fractional pixel search device includes a vector determination module 31, an acceleration determination module 32, a first search module 33, and a second search module 34.

[0085] The vector determination module 31 is configured to determine the initial motion vector for the first fractional pixel search of the image to be encoded based on the integer pixel search results of the image to be encoded; the acceleration determination module 32 is configured to determine the acceleration method for the first fractional pixel search based on the first computational cost of the initial motion vector and the second computational cost corresponding to the adjacent motion vectors of the initial motion vector in the set search direction, wherein the first computational cost and the second computational cost are obtained based on the integer pixel search of the image to be encoded; the first search module 33 is configured to perform the first fractional pixel search of the image to be encoded according to the initial motion vector and the acceleration method to obtain the first fractional pixel search result; and the second search module 34 is configured to determine the search direction skipped during the second fractional pixel search of the image to be encoded based on the acceleration method, and perform the second fractional pixel search of the image to be encoded based on the retained search direction to obtain the second fractional pixel search result.

[0086] In one possible embodiment, the acceleration determination module 32 includes a first acceleration determination unit and a second acceleration determination unit.

[0087] The first acceleration determination unit is configured to determine, based on the first and second computational costs, whether to skip the first fractional pixel search of the image to be encoded or to skip some search directions during the first iteration of the first fractional pixel search when the first computational cost of the initial motion vector is less than or equal to the second computational cost of the adjacent motion vectors in each set search direction of the initial motion vector.

[0088] The second acceleration determination unit is configured to determine to skip a portion of the search directions during the first iteration of the first fractional pixel search if the first computational cost of the initial motion vector is greater than the second computational cost corresponding to the adjacent motion vector in one of the set search directions of the initial motion vector.

[0089] In one possible embodiment, when the first acceleration determination unit determines whether to skip the first fractional pixel search of the image to be encoded, or to skip part of the search direction during the first iteration of the first fractional pixel search, based on the first computational cost and the second computational cost, it is configured as follows:

[0090] A cost threshold is determined based on the first computational cost, and the cost threshold is greater than the first computational cost.

[0091] If the second computational cost corresponding to the adjacent motion vectors of the initial motion vector in the set search direction is greater than the cost threshold, it is determined to skip the first fractional pixel search of the image to be encoded.

[0092] If there are adjacent motion vectors in the initial motion vector along the set search direction whose second computational cost is less than or equal to the cost threshold, it is determined that a portion of the search direction should be skipped during the first iteration of the first fractional pixel search.

[0093] In one possible embodiment, the first search module 33 includes a first search unit and a second search unit.

[0094] The first search unit is configured to determine the initial motion vector as the first fractional pixel search result when the acceleration method for the first fractional pixel search is to skip the first fractional pixel search of the image to be encoded.

[0095] The second search unit is configured to, when the acceleration method for the first fractional pixel search is to skip some search directions during the first iteration of the first fractional pixel search, determine the search directions skipped during the first iteration of the first fractional pixel search based on the sorting of the second computational cost and the set first skip number, and perform the first fractional pixel search on the image to be encoded based on the retained search directions to obtain the first fractional pixel search result.

[0096] In one possible embodiment, the second search module 34 includes a first direction determination unit and a second direction determination unit.

[0097] The first direction determination unit is configured to determine all search directions for performing a second fractional pixel search on the image to be encoded when the acceleration method for the first fractional pixel search is to skip the first fractional pixel search of the image to be encoded.

[0098] The second direction determination unit is configured to determine the skipped search directions when performing a second fractional pixel search on the image to be encoded, based on the adjacent motion vectors corresponding to the first fractional pixel search in the last iteration, when the acceleration method for the first fractional pixel search is to skip some search directions during the first iteration of the first fractional pixel search.

[0099] In one possible embodiment, when the second direction determination unit determines the search direction skipped during the second fractional pixel search of the image to be encoded based on the adjacent motion vectors corresponding to the last iteration of the first fractional pixel search, it is configured as follows:

[0100] Determine the third computational cost corresponding to the adjacent motion vectors in the set search direction of the iteration center motion vector during the last iteration of the first fractional pixel search;

[0101] Based on the sorting of the third computational cost and the set second skip number, the search direction to be skipped during the first iteration of the second fractional pixel search of the image to be encoded is determined.

[0102] In one possible embodiment, the fractional pixel search device further includes a third search module, which is configured to determine, when the number of searches corresponding to the integer pixel search results of the image to be encoded reaches a set integer search count threshold, to skip some search directions during the first iteration of the first fractional pixel search, and to perform a first fractional pixel search on the image to be encoded based on the retained search directions to obtain the first fractional pixel search results.

[0103] It is worth noting that in the above embodiments of the fractional pixel search device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0104] This application also provides a fractional pixel search device, which can integrate the fractional pixel search apparatus provided in this application. Figure 4 This is a schematic diagram of the structure of a fractional pixel search device provided in an embodiment of this application. (Reference) Figure 4 The fractional pixel search device includes: an input device 43, an output device 44, a memory 42, and one or more processors 41; the memory 42 is used to store one or more programs; when one or more programs are executed by one or more processors 41, the one or more processors 41 implement the fractional pixel search method provided in the above embodiments. The fractional pixel search device, apparatus, and computer provided above can be used to execute the fractional pixel search method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0105] This application also provides a non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to perform the fractional pixel search method provided in the above embodiments. Of course, the computer-executable instructions provided in this application are not limited to the fractional pixel search method provided above; they can also perform related operations in the fractional pixel search method provided in any embodiment of this application. The fractional pixel search apparatus, device, and storage medium provided in the above embodiments can execute the fractional pixel search method provided in any embodiment of this application. Technical details not described in detail in the above embodiments can be found in the fractional pixel search method provided in any embodiment of this application.

[0106] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause a computer device, mobile terminal, or processor therein to execute all or part of the steps of the fractional pixel search method provided in the various embodiments of this application.

Claims

1. A fractional pel search method characterized by, The method comprises: determining an initial motion vector for a first fractional pixel search of a to-be-encoded image based on an integer pixel search result of the to-be-encoded image; determining an acceleration mode for the first fractional pixel search of the to-be-encoded image based on a first calculation cost of the initial motion vector and a second calculation cost of a neighboring motion vector of the initial motion vector in a set search direction, wherein the first calculation cost and the second calculation cost are based on the integer pixel search of the to-be-encoded image, and wherein the acceleration mode for the first fractional pixel search comprises skipping the first fractional pixel search of the to-be-encoded image and skipping part of the search direction in a first iteration of the first fractional pixel search; performing the first fractional pixel search of the to-be-encoded image based on the initial motion vector and the acceleration mode to obtain a first fractional pixel search result; determining a search direction skipped by a second fractional pixel search of the to-be-encoded image based on the acceleration mode, and performing the second fractional pixel search of the to-be-encoded image based on the remaining search direction to obtain a second fractional pixel search result.

2. The fractional pel search method of claim 1, wherein, The method further comprises: determining whether to skip the first fractional pixel search of the to-be-encoded image or skip part of the search direction in the first iteration of the first fractional pixel search based on the first calculation cost and the second calculation cost in a case that the first calculation cost of the initial motion vector is less than or equal to the second calculation cost of the neighboring motion vector of the initial motion vector in each set search direction; determining to skip part of the search direction in the first iteration of the first fractional pixel search in a case that the first calculation cost of the initial motion vector is greater than the second calculation cost of the neighboring motion vector of the initial motion vector in one set search direction.

3. The fractional pel search method of claim 2, wherein, The method further comprises: determining a cost threshold greater than the first calculation cost based on the first calculation cost; determining to skip the first fractional pixel search of the to-be-encoded image in a case that the second calculation cost of the neighboring motion vector of the initial motion vector in each set search direction is greater than the cost threshold; determining to skip part of the search direction in the first iteration of the first fractional pixel search in a case that there is a neighboring motion vector of the initial motion vector in the set search direction with a second calculation cost less than or equal to the cost threshold.

4. The fractional pel search method of claim 2, wherein, The method further comprises: determining the initial motion vector as the first fractional pixel search result in a case that the acceleration mode for the first fractional pixel search is to skip the first fractional pixel search of the to-be-encoded image. In a case that the acceleration mode of the first fractional pixel search is skipping part of the search directions in the first iteration of the first fractional pixel search, the search directions skipped in the first iteration of the first fractional pixel search are determined based on the ranking of the second calculation costs and a set first skipping number, and the first fractional pixel search is performed on the image to be encoded based on the reserved search directions to obtain a first fractional pixel search result.

5. The fractional pel search method of claim 2, wherein, The determining the search directions skipped in the second fractional pixel search on the image to be encoded based on the acceleration mode comprises: In a case that the acceleration mode of the first fractional pixel search is skipping the first fractional pixel search on the image to be encoded, all the search directions in the second fractional pixel search on the image to be encoded are determined to be reserved; In a case that the acceleration mode of the first fractional pixel search is skipping part of the search directions in the first iteration of the first fractional pixel search, the search directions skipped in the second fractional pixel search on the image to be encoded are determined based on the neighboring motion vectors corresponding to the last iteration of the first fractional pixel search.

6. The fractional pel search method of claim 5, wherein, The determining the search directions skipped in the second fractional pixel search on the image to be encoded based on the neighboring motion vectors corresponding to the last iteration of the first fractional pixel search comprises: determining third calculation costs of the neighboring motion vectors of the iteration center motion vector in the last iteration of the first fractional pixel search in the set search directions; determining the search directions skipped in the first iteration of the second fractional pixel search on the image to be encoded based on the ranking of the third calculation costs and a set second skipping number.

7. The fractional pel search method of claim 1, wherein, After the determining the initial motion vector of the first fractional pixel search on the image to be encoded based on the integer pixel search result on the image to be encoded, the method further comprises: In a case that the search times corresponding to the integer pixel search result on the image to be encoded reach a set integer search times threshold, the search directions skipped in the first iteration of the first fractional pixel search are determined, and the first fractional pixel search is performed on the image to be encoded based on the reserved search directions to obtain a first fractional pixel search result.

8. An apparatus for fractional pixel search, characterized by The device comprises a vector determining module, an acceleration determining module, a first search module and a second search module, wherein: The vector determining module is configured to determine the initial motion vector of the first fractional pixel search on the image to be encoded based on the integer pixel search result on the image to be encoded; The acceleration determining module is configured to determine the acceleration mode of the first fractional pixel search based on a first calculation cost of the initial motion vector and a second calculation cost of the neighboring motion vectors of the initial motion vector in the set search directions, the first calculation cost and the second calculation cost being based on the integer pixel search on the image to be encoded, wherein the acceleration mode of the first fractional pixel search comprises skipping the first fractional pixel search on the image to be encoded and skipping part of the search directions in the first iteration of the first fractional pixel search; The first search module is configured to perform a first fractional pixel search on the image to be encoded according to the initial motion vector and the acceleration mode to obtain a first fractional pixel search result. The second search module is configured to determine a search direction skipped by a second fractional pixel search on the image to be encoded based on the acceleration mode, and perform the second fractional pixel search on the image to be encoded based on the reserved search direction to obtain a second fractional pixel search result.

9. An apparatus for fractional pel search, the apparatus comprising: The method comprises: a memory and one or more processors; the memory is configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the fractional pixel search method according to any one of claims 1-7.

10. A non-volatile storage medium for storing computer-executable instructions, characterized in that, The computer executable instructions, when executed by a computer processor, are configured to perform the fractional pixel search method according to any one of claims 1-7.

11. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the fractional pixel search method according to any one of claims 1-7.

Images