Motion estimation method, device, equipment, medium and product

Abstract

The present disclosure provides a motion estimation method, apparatus, device, medium, and product, relating to the fields of artificial intelligence technology, specifically cloud storage, cloud computing, video coding, and other technical fields. The motion estimation method includes: determining a candidate search space and a candidate search starting point based on a forward motion vector and a predicted search starting point of a current block; determining a target search starting point from the candidate search starting points, and determining a target search space from the candidate search space; performing a search based on the target search starting point and the target search space to obtain an initial motion estimation result for the current block; and obtaining a target motion estimation result for the current block based on the initial motion estimation result.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically cloud storage, cloud computing, video coding and other technical fields, and more particularly to a motion estimation method, apparatus, device, medium and product. Background Art

[0002] High Efficiency Video Coding (HEVC) is a video coding standard that can effectively improve coding efficiency while maintaining the same video quality, thereby encoding the video at the lowest possible bit rate while maintaining a certain level of video quality.

[0003] In HEVC, inter-frame prediction coding is an important video compression technology, which obtains the difference between the current frame and the reference frame through motion estimation, and encodes the difference to achieve efficient compression.

[0004] Motion estimation uses pixel blocks as units, searches for a block that best matches the current block in the current frame within the search space of the reference frame, and obtains a motion estimation result based on the matching result. Summary of the Invention

[0005] The present disclosure provides a motion estimation method, apparatus, device, medium, and product.

[0006] According to one aspect of the present disclosure, a motion estimation method is provided, comprising: determining a candidate search space and a candidate search starting point based on a forward motion vector and a predicted search starting point of a current block; determining a target search starting point among the candidate search starting points, and determining a target search space within the candidate search space; searching based on the target search starting point and the target search space to obtain an initial motion estimation result for the current block; and obtaining a target motion estimation result for the current block based on the initial motion estimation result.

[0007] According to another aspect of the present disclosure, a motion estimation device is provided, comprising: determining a candidate search space and a candidate search starting point based on a forward motion vector and a predicted search starting point of a current block; determining a target search starting point among the candidate search starting points, and determining a target search space among the candidate search spaces; searching based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block; and obtaining a target motion estimation result of the current block based on the initial motion estimation result.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of the methods described in any one of the above aspects.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to any one of the above aspects.

[0010] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of the above aspects.

[0011] According to the embodiments of the present disclosure, the motion estimation effect can be improved.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0014] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0015] Figure 2 is a schematic diagram of an application scenario for implementing an embodiment of the present disclosure;

[0016] Figure 3 is a schematic diagram of a candidate search space provided according to an embodiment of the present disclosure;

[0017] Figure 4 is a schematic diagram according to a second embodiment of the present disclosure;

[0018] Figure 5 is a schematic diagram of a motion estimation process in a look-ahead phase according to an embodiment of the present disclosure;

[0019] Figure 6 is a schematic diagram of a motion estimation process in an encoding stage according to an embodiment of the present disclosure;

[0020] Figure 7 is a schematic diagram according to a third embodiment of the present disclosure;

[0021] Figure 8is a schematic diagram of an electronic device used to implement the motion estimation method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] The motion estimation process primarily involves dividing the current frame and reference frame images into several equal-sized pixel blocks. The pixel blocks for motion estimation are selected based on specific rules, such as those containing more texture or motion information. For each selected pixel block in the current frame (referred to as the current block), the search space in the reference frame is used to find the pixel block that best matches it (referred to as the matching block). The relative position coordinates between the matching block and the current block are calculated, known as the motion vector (MV). Motion estimation yields a motion vector, which reflects the direction and distance of motion of the matching block relative to the current block.

[0024] In addition, the cost corresponding to the MV can also be calculated, and the MV and its cost can be used as the motion estimation result.

[0025] The aforementioned cost can specifically be the rate-distortion cost. The rate-distortion cost comprehensively considers the number of coding bits and the degree of distortion. The number of coding bits refers to the number of bits required to encode motion vectors and residual information. The degree of distortion is usually calculated using an error metric such as mean square error or peak signal-to-noise ratio, which represents the difference between the reconstructed image and the original image.

[0026] The size of the search space is determined by the search range value (searchRange). The search space range value can be represented by the number of pixels from the center point to the boundary of the search space. For example, the search range value is represented by d (a positive integer). The search space is a square area with a side length of 2d pixels and a center point corresponding to the mapping point of the current block to the reference frame.

[0027] In related technologies, the search range value is usually a fixed value. However, if the search range value is set to a small value, the corresponding search space is small, and it may be impossible to find the matching block that best matches the current block. If the search range value is set to a large value, the search time for the matching block will be too long, reducing the coding efficiency.

[0028] In the video coding scenario, it can be divided into a lookahead stage and a coding stage. The lookahead stage mainly determines coding parameters, and the coding stage performs specific coding operations according to the coding parameters, such as the above-mentioned inter-frame prediction coding.

[0029] During the lookahead phase, the current frame's encoding parameters, such as frame type, block division strategy, and encoding mode, can be determined by analyzing the current frame and reference frames. For example, if the reference frame differs significantly from the current frame, an I-frame can be selected for encoding to provide better random access points and higher image quality. Alternatively, if the motion of a region in the reference frame is complex, a smaller block division and a more complex encoding mode can be used for that region in the current frame to improve encoding efficiency and accuracy.

[0030] In video encoding, there are three common frame types:

[0031] I-frame (Intra-coded frame): also known as a key frame, is an independently encoded frame that does not rely on other frames for encoding.

[0032] P frame (Predictive-coded frame): relies on the previous I frame or P frame for encoding, and uses information in the previous frame to predict the current frame through motion estimation and motion compensation technology.

[0033] B frame (Bidirectionally predictive-coded frame): relies on the previous and next frames for encoding, and uses bidirectional motion estimation and motion compensation technology to use reference frames in both the previous and next directions to predict the current frame.

[0034] In the lookahead stage, the current frame and the reference frame can also be spatially downsampled. Spatial downsampling is to reduce the resolution or size of the image, that is, to reduce the number of pixels in the image; motion estimation is performed based on the downsampled image to obtain a preliminary motion estimation result, which can be expressed as lmv.

[0035] In the related art, the search range value of the search space in the encoding stage is usually a set fixed value, which is not accurate enough, thereby affecting the accuracy of the motion estimation result.

[0036] In the disclosed embodiment, the search range value in the encoding stage is determined based on the above-mentioned lmv, thereby improving the accuracy of the search space and enhancing the motion estimation effect.

[0037] In order to improve the motion estimation effect, the present disclosure provides the following embodiments.

[0038] Figure 1 This embodiment provides a motion estimation method, such as Figure 1 As shown, the method includes:

[0039] 101. Determine a candidate search space and a candidate search starting point based on the forward motion vector and the predicted search starting point of the current block.

[0040] 102. Determine a target search starting point from the candidate search starting points, and determine a target search space from the candidate search space.

[0041] 103. Perform a search based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block.

[0042] 104. Obtain a target motion estimation result of the current block based on the initial motion estimation result.

[0043] In the video coding scenario, the current block refers to the pixel block currently being coded in the video.

[0044] The video coding process can be divided into a lookahead stage and a coding stage. Both the lookahead stage and the coding stage can perform a motion estimation process. The motion vector obtained by performing the motion estimation process in the lookahead stage is called a lookahead motion vector, which can be represented by lmv.

[0045] For the current block, a candidate motion vector (candidate MV) of the current block can be obtained. The position point obtained based on the candidate MV is called the prediction search starting point and can be represented by MVP.

[0046] After obtaining the lmv and mvp of the current block, the candidate search space and the candidate search starting point can be determined based on the lmv and mvp.

[0047] There are multiple candidate search spaces and candidate search starting points. Afterwards, one candidate search space can be determined as a target search space from the multiple candidate search spaces, and one candidate search starting point can be determined as a target search starting point from the multiple candidate search starting points.

[0048] After obtaining the target search space and the target search starting point, a search is performed based on the target search space and the target search starting point to obtain the initial motion estimation result. Specifically, starting from the target search starting point, the target search space is searched for a matching block corresponding to the current block, and the relative position coordinates between the matching block and the current block are calculated. This relative position coordinate can be called the initial motion vector (initial MV). Afterwards, the cost of the initial MV can also be obtained, and the initial MV and its cost are used as the initial motion estimation result.

[0049] After obtaining the initial motion estimation result, the initial motion estimation result can be used as the target motion estimation result; alternatively, the initial motion estimation result can be further processed to obtain the target motion estimation result; for example, the search starting point and search space can be re-determined based on the initial motion estimation result, and the search can be re-performed based on the new search starting point and search space to obtain a new matching block, and the target motion estimation result can be obtained based on the relative position coordinates of the new matching block and the current block.

[0050] In this embodiment, the candidate search space and the candidate search starting point are determined based on the forward motion vector and the predicted search starting point, the target search starting point is determined in the candidate search starting point, and the target search space is determined in the candidate search space. In this way, the forward motion vector is referenced in the motion estimation process, which improves the accuracy of the target search space and the target search starting point, thereby improving the motion estimation effect.

[0051] In order to better understand the present disclosure, the application scenarios involved in the present disclosure are described as follows:

[0052] Figure 2 It is a schematic diagram of an application scenario for implementing the embodiment of the present disclosure.

[0053] In video coding scenarios, the current frame is the image frame to be encoded in the video, and the reference frame is the frame used as a reference for encoding. Depending on the encoding mode, the reference frame can be the previous frame and / or the next frame to the current frame. Motion estimation is typically performed in pixel blocks. To this end, the current frame and the reference frame can be divided into multiple pixel blocks. The pixel block currently being encoded in the current frame is called the current block, and the pixel block in the reference frame that best matches the current block is called the matching block.

[0054] like Figure 2 As shown in the figure, for the current block, the motion estimation process can obtain a motion vector (MV), which is the relative position coordinates between the matching block and the current block. In addition, the cost corresponding to the MV can also be obtained. The MV and its cost are used as the motion estimation result for the current block.

[0055] A matching block is a pixel block in the reference frame, determined after searching in the search space.

[0056] The size of the search space can be represented by the search range value (searchRange). The larger the search range value, the larger the search space. Therefore, the search can be performed in a larger search space, which improves the accuracy of the matching block. However, due to the large search space, it takes longer. The smaller the search range value, the smaller the search space and the shorter the search time. However, due to the limited search range, the accuracy of the matching block is affected.

[0057] In order to improve the accuracy of the search space, in the embodiment of the present disclosure, a candidate search space may be first obtained, and then a target search space may be determined in the candidate search space, and then a search may be performed in the target search space.

[0058] Figure 3 Schematic diagram of a candidate search space provided according to an embodiment of the present disclosure.

[0059] like Figure 3 As shown, in the embodiment of the present disclosure, the candidate search space includes a first search space and a second search space. The search range value of the first search space is represented by d1, and the search range value of the second search space is represented by d2. Therefore, the first search space is a square area with a side length of 2*d1, and the second search space is a square area with a side length of 2*d2.

[0060] The two candidate search spaces (the first search space and the second search space) are determined based on the forward motion vector and the prediction search starting point.

[0061] In the video coding scenario, it can be divided into a lookahead stage and a coding stage. The motion vector obtained in the lookahead stage is called a lookahead motion vector, which can be represented by lmv.

[0062] Specifically, in the look-ahead phase, spatial downsampling may be performed on the current frame and the reference frame, and motion estimation may be performed on the downsampled current frame and the reference frame to obtain LMV.

[0063] For the current block, the motion vector of the encoded neighboring block of the current block can be used as the candidate motion vector of the current block. The neighboring blocks can include spatial neighboring blocks and / or temporal neighboring blocks. Taking the spatial neighboring blocks as an example, they can specifically include the upper block, left block, upper right block, etc. of the current block.

[0064] After obtaining the candidate motion vectors for the current block, the prediction search starting point can be determined based on the candidate motion vectors, represented by MVP. For example, one of these candidate motion vectors is selected, such as the candidate motion vector with the highest frequency or the best encoding effect, and the position corresponding to the selected candidate motion vector is used as the prediction search starting point; or the mean of these candidate motion vectors is calculated, and the position corresponding to the mean is used as the prediction search starting point. For example, if the selected candidate motion vector is (4, -3), and the positive direction of the x-axis is rightward and the positive direction of the y-axis is downward, then in the reference frame, the mapping position of the current block is horizontally shifted 4 pixels to the right and 3 pixels upward as the prediction search starting point.

[0065] After obtaining the forward motion vector lmv and the predicted search starting point mvp, the candidate search space is determined based on these two pieces of information.

[0066] The first search space is determined based on the forward motion vector and the prediction search starting point, and the second search space is determined based on the forward motion vector and the first search space.

[0067] The search range value of the first search space is called a first search range value, denoted by d1, and the search range value of the second search space is called a second search range value, denoted by d2.

[0068] In some cases, d1 = d, and in other cases, d1 = d0. d0 is a preset initial search range value, and d is the distance between the forward motion vector lmv and the vector corresponding to the prediction search starting point mvp. The specific situation is determined based on d.

[0069] In some cases, d2 = d1 * k, where k is a preset multiple greater than 1, and in other cases, d2 = d1. The specific case is determined based on the corresponding length of the forward motion vector lmv.

[0070] In this embodiment, the first search space is determined based on the forward motion vector and the predicted search starting point, and the second search space is determined based on the first search space and the forward motion vector. Multiple candidate search spaces can be obtained, and an accurate target search space can be determined based on the candidate search spaces, thereby improving the motion estimation effect.

[0071] Specifically, if the distance between the forward motion vector and the vector corresponding to the predicted search starting point is greater than a first length threshold and less than or equal to a maximum range threshold, the distance is used as the first search range value; the first length threshold is determined based on a preset initial search range value; if the distance is less than or equal to the first length threshold, the initial search range value is used as the first search range value; and the search space corresponding to the first search range value is used as the first search space.

[0072] The initial search range value is represented by d0, which is a preset value.

[0073] For the first search range value, the formula is as follows:

[0074]

[0075] Wherein, d1 is the first search range value;

[0076] d0 is the preset initial range value;

[0077] L1 is the first length threshold; it is determined according to the initial search range value, such as,

[0078] L1=k1*d0

[0079] k1 is a preset ratio value greater than 1.

[0080] L max Is the maximum range threshold. In video encoding, assuming that the image size in the video is expressed as L*W (length*width), then L max is the minimum value of L and W.

[0081] d is the distance between the forward motion vector lmv and the vector corresponding to the prediction search starting point mvp. This distance is the distance between vectors. For example, lmv = (3, 2), and the vector corresponding to mvp is (4, -3). The distance between the two can be expressed as:

[0082]

[0083] Based on the above formula, the first search range value d1 can be obtained, and then the first search space can be determined.

[0084] For the second search space:

[0085] If the length corresponding to the forward motion vector is greater than a second length threshold, a preset multiple of the first search range value of the first search space is used as the second search range value; the second length threshold is determined based on the first search range value; if the length corresponding to the forward motion vector is less than or equal to the second length threshold, the first search range value is used as the second search range value; and the search space corresponding to the second search range value is used as the second search space.

[0086] The formula is as follows:

[0087]

[0088] Wherein, d2 is the second search range value;

[0089] d1 is the first search range value;

[0090] k is a preset multiple, which is a value greater than 1, and the maximum value is determined based on the video resolution so that the second search range value does not exceed the video resolution;

[0091] L2 is a second length threshold, which is determined according to the first search range value d1, for example,

[0092] L2=k2*d1

[0093] k2 is a preset ratio value greater than 1.

[0094] |lmv| is the length of the forward motion vector; it can be represented by the distance between the position point corresponding to the forward motion vector and the coordinate origin, or it can be represented by the absolute value of the position coordinate of the forward motion vector itself. For example, the forward motion vector is represented as:

[0095] lmv=(x,y)

[0096] Alternatively, |lmv|=max(|x|,|y|), where || is the absolute value and max() is the maximum value.

[0097] Based on the above formula, the second search range value d2 can be obtained, and then the second search space can be determined.

[0098] In addition, a candidate search starting point may be determined based on the forward motion vector and the predicted search starting point.

[0099] The candidate search starting points may include: a predicted search starting point and an optimal search starting point. The optimal search starting point is a position point corresponding to an optimal motion vector. The optimal motion vector is determined based on the forward motion vector and the candidate motion vectors.

[0100] For example, the candidate motion vectors are represented by mv1~mvn, and the forward motion vector is represented by lmv. After obtaining these motion vectors, the cost of each motion vector is calculated, and the motion vector with the smallest cost is taken as the optimal motion vector, represented by mv*. The position point corresponding to the optimal mv* is then taken as the optimal search starting point, represented by mvq. The candidate search starting points include: mvp and mvq.

[0101] In this embodiment, the position point corresponding to the predicted search starting point and the optimal motion vector is used as a candidate search starting point. In the subsequent process, a suitable starting point can be selected as the target search starting point according to the specific situation, thereby improving the accuracy of the target search starting point and thus improving the accuracy of motion estimation.

[0102] After obtaining the candidate search space and the candidate search starting point, the target search space can be determined from the candidate search space, and the target search starting point can be determined from the candidate search starting point, thereby obtaining the target motion estimation result.

[0103] In some embodiments, if the cost of the optimal motion vector is greater than a preset first cost threshold, the predicted search starting point is used as the target search starting point, and the second search space is used as the target search space. Accordingly, an initial motion estimation result obtained based on the target search starting point and the target search space is used as the target motion estimation result for the current block.

[0104] Specifically, assuming that the optimal motion vector is represented by mv*, its cost is represented by mv*_cost, and the first cost threshold is represented by cost1, then if mv*_cost>cost1, the target search starting point = mvp, the target search space = the second search space, and the corresponding search range value is d2. At this time, starting from mvp, the search is performed in the search space with a search range value of d2.

[0105] After searching, the matching block of the current block is obtained. Assuming that the relative position coordinates of the matching block and the current block are represented by mv1 and the corresponding cost is represented by mv1_cost, the initial motion estimation result and the target motion estimation result both include: mv1 and mv1_cost.

[0106] In some embodiments, if the cost of the optimal motion vector is less than or equal to a preset first cost threshold, the optimal search starting point is used as the target search starting point, and the first search space is used as the target search space.

[0107] Furthermore, the initial motion estimation result includes: the cost of the initial motion vector;

[0108] If the cost of the initial motion vector is less than or equal to a preset second cost threshold, the initial motion estimation result is used as the target motion estimation result.

[0109] Specifically, assuming that the optimal motion vector is represented by mv*, its cost is represented by mv*_cost, and the first cost threshold is represented by cost1, then if mv*_cost<=cost1, the target search starting point = mvq, the target search space = the first search space, and the corresponding search range value is d1. At this time, starting from mvq, the search is performed in the search space with a search range value of d1.

[0110] After searching, a matching block of the current block is obtained. Assuming that the relative position coordinates of the matching block and the current block are represented by mv2 and the corresponding cost is represented by mv2_cost, the initial motion estimation result includes: mv2 and mv2_cost.

[0111] Assume that the second cost threshold is represented by cost2. If mv2_cost<=cost2, the target motion estimation result is the same as the initial motion estimation result, that is, the target motion estimation result includes: mv2 and mv2_cost.

[0112] In some embodiments, if the cost of the initial motion vector is greater than a preset second cost threshold, a search is performed based on the predicted search starting point and the second search space to obtain an updated motion estimation result; and based on the initial motion estimation result and the updated motion estimation result, the target motion estimation result is obtained.

[0113] Specifically, assuming mv2_cost>cost2, then based on mvp, re-search in the search space of size d2 to obtain a new matching block, calculate the relative position coordinates of the new matching block and the current block, denoted by mv3, and the corresponding cost is denoted by mv3_cost. Afterwards, compare mv2_cost and mv3_cost, and take the smaller cost and its corresponding mv as the target motion estimation result. For example, if mv3_cost is less than mv2_cost, the target motion estimation result includes: mv3 and mv3_cost.

[0114] In this way, appropriate target motion estimation results can be obtained in different situations.

[0115] In combination with the above application scenarios, the present disclosure also provides the following embodiments.

[0116] Figure 4 is a schematic diagram according to a second embodiment of the present disclosure. This embodiment provides a motion estimation method, the method comprising:

[0117] 401. In a look-ahead phase of video encoding, determine a target search method from among multiple candidate search methods based on a frame distance between a current frame and a reference frame; and obtain a look-ahead motion vector of a current block based on the target search method.

[0118] 402. In a coding stage of video coding, obtain a target motion estimation result of the current block based on the forward motion vector.

[0119] During the look-ahead phase, two candidate search methods are assumed: the star (STAR) search algorithm and the hexagonal (HEX) search algorithm. The STAR search algorithm is a motion estimation algorithm, a search method used in video coding technology. The STAR search algorithm combines the advantages of full search and diamond search, reducing computational effort and improving search efficiency by selectively searching specific areas. The HEX search algorithm searches pixel blocks within the search area in a hexagonal pattern to find the best match, reducing computational effort and improving search efficiency.

[0120] Figure 5 2 is a schematic diagram of a motion estimation process in a forward-looking phase according to an embodiment of the present disclosure.

[0121] like Figure 5 As shown, the method includes:

[0122] 501. Perform spatial downsampling on the current frame and the reference frame.

[0123] 502. Determine whether the frame distance between the current frame and the reference frame is greater than the preset frame distance. If so, execute; otherwise, execute

[0124] The frame distance can be represented by the difference in sequence numbers between the current frame and the reference frame. For example, if the current frame is the t-th frame and the reference frame is the (t-2)-th frame, then the frame distance = 2.

[0125] 503. Based on the STAR search algorithm, motion estimation is performed on the current frame and the reference frame after spatial downsampling to obtain a forward motion vector lmv.

[0126] 504. Based on the HEX search algorithm, motion estimation is performed on the current frame and the reference frame after spatial downsampling to obtain a forward motion vector lmv.

[0127] Since the STAR search algorithm has a larger search range, it can search in a wider range when the frame distance is large, thereby improving reliability. When the frame distance is small, the HEX search algorithm can improve search efficiency and thus improve coding efficiency.

[0128] In this embodiment, in the forward-looking stage, by determining the target search method from multiple candidate search methods and obtaining the forward-looking motion vector based on the target search method, the accuracy of the forward-looking motion vector can be improved, thereby improving the accuracy of motion estimation in the encoding stage and improving the video encoding effect.

[0129] Figure 6 It is a schematic diagram of the motion estimation process in the encoding stage provided according to an embodiment of the present disclosure.

[0130] like Figure 6 As shown, the method includes:

[0131] 601. Obtain the forward motion vector lmv and the prediction search starting point mvp of the current block.

[0132] The forward motion vector lmv is obtained in the forward phase, and the prediction search starting point mvp is obtained based on the candidate mv. The specific acquisition process can be found in the above description.

[0133] 602. Determine whether the distance d between the vectors corresponding to lmv and mvp is greater than the first length threshold L1 and less than or equal to the maximum range threshold L max If yes, execute 603; otherwise, execute 604.

[0134] 603. Use the distance as a first search range value, expressed as d1=d.

[0135] 604. Use the preset initial search range value d0 as the first search range value, expressed as d1 = d0.

[0136] The specific calculation process of the first search range value d1 can refer to the above related description.

[0137] In this way, after obtaining the first search range value d1 in 603 or 604, the search space corresponding to the first search range value is used as the first search space.

[0138] In this embodiment, the distance between the forward motion vector and the vector corresponding to the predicted search starting point or the initial search range value is used as the first search range value, so that adaptive adjustment of the first search space can be achieved, thereby obtaining accurate target motion estimation results with higher efficiency and improving the motion estimation effect.

[0139] 605. Determine whether the length L corresponding to lmv is greater than the second length threshold L2. If so, execute 606; otherwise, execute 607.

[0140] 606. Use the preset multiple k of the first search range value as the second search range value, expressed as d2 = d1 * k.

[0141] 607. Use the first search range value as the second search range value, expressed as d2 = d1.

[0142] The specific calculation process of the second search range value d2 can refer to the above related description.

[0143] In this way, after obtaining the second search range value d2 in 607 or 608, the search space corresponding to the second search range value is used as the second search space.

[0144] In this embodiment, the preset multiple of the first search range value or the first search range value is used as the second search range value, which can realize adaptive adjustment of the second search space, thereby obtaining accurate target motion estimation results with higher efficiency and improving motion estimation effect.

[0145] 608. Obtain the optimal mv based on lmv and the candidate mvs, and obtain the cost of the optimal mv, represented by mv*_cost. Then, determine whether mv*_cost is greater than a preset first cost threshold cost1. If so, execute 609; otherwise, execute 610.

[0146] 609. Search based on mvp and d2 to obtain target motion estimation results.

[0147] Specifically, MVP is used as the target search starting point, the second search space is used as the target search space, a search is performed based on the target search starting point and the target search space, an initial motion estimation result of the current block is obtained, and the initial motion estimation result is used as the target motion estimation result of the current block.

[0148] In this embodiment, when the cost of the optimal motion vector is greater than the first cost threshold, searching in a larger second search space based on MVP can improve the accuracy of the target motion estimation result.

[0149] 610. Search based on mvq and d1 to obtain mv_out and mv_out_cost.

[0150] Specifically, the position point mvq corresponding to the optimal mv is used as the target search starting point, the first search space is used as the target search space, and a search is performed based on the target search starting point and the target search space to obtain the initial motion estimation result of the current block, which is represented by mv_out and mv_out_cost.

[0151] 611. Determine whether mv_out_cost is greater than a preset second cost threshold cost2. If so, execute 612; otherwise, execute 614.

[0152] 612. Search based on mvp and d2 to obtain an updated motion estimation result.

[0153] Specifically, mvp is used as a new search starting point, the second search space is used as a new search space, and a search is performed based on the new search starting point and the new search space to obtain a new matching block of the current block. An updated motion estimation result is obtained based on the new matching block, which is represented by mv_out' and mv_out_cost'.

[0154] 613. Select the better one between the initial motion estimation result and the updated motion estimation result as the target motion estimation result.

[0155] Specifically, the result with the smaller cost is taken as the better one. For example, if mv_out_cost' is smaller than mv_out_cost', the target motion estimation result includes mv_out' and mv_out_cost'.

[0156] In this embodiment, when the cost of the initial motion vector is greater than the second cost threshold, a new search is performed to obtain an updated motion estimation result, and a target motion estimation result is obtained based on the initial motion estimation result and the updated motion estimation result, which can improve the accuracy of the target motion estimation result.

[0157] 614. Use the initial motion estimation result as the target motion estimation result.

[0158] Specifically, the target motion estimation result includes: mv_out and mv_out_cost.

[0159] In this embodiment, when the cost of the initial motion vector is less than or equal to the second cost threshold, the initial motion estimation result obtained by searching based on mvq and d1 is used as the target motion estimation result, which can improve the motion estimation efficiency.

[0160] Figure 73 is a schematic diagram according to the third embodiment of the present disclosure. This embodiment provides a motion estimation device. The device 700 includes: a first determination module 701 , a second determination module 702 , a third determination module 703 and a fourth determination module 704 .

[0161] The first determination module 701 is used to determine the candidate search space and the candidate search starting point based on the forward motion vector and the predicted search starting point of the current block; the second determination module 702 is used to determine the target search starting point in the candidate search starting point, and to determine the target search space in the candidate search space; the third determination module 703 is used to search according to the target search starting point and the target search space to obtain the initial motion estimation result of the current block; the fourth determination module 704 is used to obtain the target motion estimation result of the current block based on the initial motion estimation result.

[0162] In this embodiment, the candidate search space and the candidate search starting point are determined based on the forward motion vector and the predicted search starting point, the target search starting point is determined in the candidate search starting point, and the target search space is determined in the candidate search space. In this way, the forward motion vector is referenced in the motion estimation process, which improves the accuracy of the target search space and the target search starting point, thereby improving the motion estimation effect.

[0163] In some embodiments, the first determining module 701 is further configured to:

[0164] Based on the forward motion vector and the predicted search starting point, a first search space is determined; based on the forward motion vector and the first search space, a second search space is determined; and the first search space and the second search space are used as the candidate search spaces.

[0165] In this embodiment, the first search space is determined based on the forward motion vector and the predicted search starting point, and the second search space is determined based on the first search space and the forward motion vector. Multiple candidate search spaces can be obtained, and an accurate target search space can be determined based on the candidate search spaces, thereby improving the motion estimation effect.

[0166] In some embodiments, the first determining module 701 is further configured to:

[0167] If the distance between the forward motion vector and the vector corresponding to the predicted search starting point is greater than a first length threshold and less than or equal to a maximum range threshold, the distance is used as a first search range value; the first length threshold is determined based on a preset initial search range value; if the distance is less than or equal to the first length threshold, the initial search range value is used as the first search range value; and the search space corresponding to the first search range value is used as the first search space.

[0168] In this embodiment, the distance between the forward motion vector and the vector corresponding to the predicted search starting point or the initial search range value is used as the first search range value, so that adaptive adjustment of the first search space can be achieved, thereby obtaining accurate target motion estimation results with higher efficiency and improving the motion estimation effect.

[0169] In some embodiments, the first determining module 701 is further configured to:

[0170] If the length corresponding to the forward motion vector is greater than a second length threshold, a preset multiple of the first search range value of the first search space is used as the second search range value; the second length threshold is determined based on the first search range value; if the length corresponding to the forward motion vector is less than or equal to the second length threshold, the first search range value is used as the second search range value; and the search space corresponding to the second search range value is used as the second search space.

[0171] In this embodiment, the preset multiple of the first search range value or the first search range value is used as the second search range value, which can realize adaptive adjustment of the second search space, thereby obtaining accurate target motion estimation results with higher efficiency and improving motion estimation effect.

[0172] In some embodiments, the first determining module 701 is further configured to:

[0173] Based on the forward motion vector and the candidate motion vector corresponding to the prediction search starting point, an optimal motion vector is determined; and the position point corresponding to the prediction search starting point and the optimal motion vector is used as the candidate search starting point.

[0174] In this embodiment, the position point corresponding to the predicted search starting point and the optimal motion vector is used as a candidate search starting point. In the subsequent process, a suitable starting point can be selected as the target search starting point according to the specific situation, thereby improving the accuracy of the target search starting point and thus improving the accuracy of motion estimation.

[0175] In some embodiments, the third determining module 703 is further configured to:

[0176] If the cost of the optimal motion vector is greater than a preset first cost threshold, using the predicted search starting point as the target search starting point, and using the second search space as the target search space;

[0177] The fourth determining module 704 is further configured to:

[0178] The initial motion estimation result is used as the target motion estimation result.

[0179] In this embodiment, when the cost of the optimal motion vector is greater than the first cost threshold, searching in a larger second search space based on MVP can improve the accuracy of the target motion estimation result.

[0180] In some embodiments, the third determining module 703 is further configured to:

[0181] If the cost of the optimal motion vector is less than or equal to a preset first cost threshold, the position point corresponding to the optimal motion vector is used as the target search starting point, and the first search space is used as the target search space.

[0182] In some embodiments, the initial motion estimation result includes: a cost of an initial motion vector;

[0183] The fourth determining module 704 is further configured to:

[0184] If the cost of the initial motion vector is greater than a preset second cost threshold, searching based on the prediction search starting point and the second search space to obtain an updated motion estimation result; obtaining the target motion estimation result based on the initial motion estimation result and the updated motion estimation result; or

[0185] If the cost of the initial motion vector is less than or equal to a preset second cost threshold, the initial motion estimation result is used as the target motion estimation result.

[0186] In this embodiment, when the cost of the initial motion vector is greater than the second cost threshold, a new search is performed to obtain an updated motion estimation result, and a target motion estimation result is obtained based on the initial motion estimation result and the updated motion estimation result, which can improve the accuracy of the target motion estimation result.

[0187] In this embodiment, when the cost of the initial motion vector is less than or equal to the second cost threshold, the initial motion estimation result obtained by searching based on mvq and d1 is used as the target motion estimation result, which can improve the motion estimation efficiency.

[0188] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.

[0189] It can be understood that the terms “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.

[0190] It is understandable that unless there is any special limitation on the order of the steps in the process, it means that the temporal relationship between these steps is not limited.

[0191] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0192] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0193] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device 800 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0194] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0195] Multiple components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0196] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the motion estimation method. For example, in some embodiments, the motion estimation method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 809. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the motion estimation method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the motion estimation method by any other appropriate means (e.g., by means of firmware).

[0197] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0198] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable task processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0199] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0200] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0201] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0202] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0203] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0204] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A motion estimation method, comprising: Determine a candidate search space and a candidate search starting point based on the forward motion vector and the predicted search starting point of the current block; Determining a target search starting point from among the candidate search starting points, and determining a target search space from among the candidate search spaces; Searching based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block; Obtaining a target motion estimation result of the current block based on the initial motion estimation result; The determining of the candidate search space based on the forward motion vector of the current block and the prediction search starting point includes: Determining a first search space based on the forward motion vector and the predicted search starting point includes: if a distance between the forward motion vector and a vector corresponding to the predicted search starting point is greater than a first length threshold and less than or equal to a maximum range threshold, using the distance as a first search range value; the first length threshold is determined based on a preset initial search range value; if the distance is less than or equal to the first length threshold, using the initial search range value as the first search range value; and using a search space corresponding to the first search range value as the first search space; Determining a second search space based on the forward motion vector and the first search space includes: if the length corresponding to the forward motion vector is greater than a second length threshold, using a preset multiple of a first search range value of the first search space as a second search range value; the second length threshold is determined based on the first search range value; if the length corresponding to the forward motion vector is less than or equal to the second length threshold, using the first search range value as the second search range value; and using a search space corresponding to the second search range value as the second search space; The first search space and the second search space are used as the candidate search spaces.

2. The method according to claim 1, wherein The step of determining a candidate search starting point based on the forward motion vector and the predicted search starting point includes: Determining an optimal motion vector based on the forward motion vector and the candidate motion vector corresponding to the prediction search starting point; The position point corresponding to the predicted search starting point and the optimal motion vector is used as the candidate search starting point.

3. The method according to claim 2, wherein: The determining of a target search starting point from among the candidate search starting points, and determining a target search space from among the candidate search spaces, comprises: If the cost of the optimal motion vector is greater than a preset first cost threshold, using the predicted search starting point as the target search starting point, and using the second search space as the target search space; The obtaining a target motion estimation result of the current block based on the initial motion estimation result includes: The initial motion estimation result is used as the target motion estimation result.

4. The method according to claim 2, wherein: The determining of a target search starting point from among the candidate search starting points, and determining a target search space from among the candidate search spaces, comprises: If the cost of the optimal motion vector is less than or equal to a preset first cost threshold, using the position point corresponding to the optimal motion vector as the target search starting point, and using the first search space as the target search space; The initial motion estimation result includes: the cost of the initial motion vector; The obtaining a target motion estimation result of the current block based on the initial motion estimation result includes: If the cost of the initial motion vector is greater than a preset second cost threshold, searching based on the prediction search starting point and the second search space to obtain an updated motion estimation result; obtaining the target motion estimation result based on the initial motion estimation result and the updated motion estimation result; or If the cost of the initial motion vector is less than or equal to a preset second cost threshold, the initial motion estimation result is used as the target motion estimation result.

5. The method according to claim 1, further comprising: In the look-ahead phase of video coding, a target search method is determined from multiple candidate search methods based on the frame distance between the current frame and the reference frame; The forward motion vector is obtained based on the target search method.

6. A motion estimation apparatus, comprising: A first determining module is configured to determine a candidate search space and a candidate search starting point according to a forward motion vector of a current block and a predicted search starting point; a second determining module, configured to determine a target search starting point from among the candidate search starting points, and to determine a target search space from among the candidate search spaces; a third determining module, configured to search according to the target search starting point and the target search space to obtain an initial motion estimation result of the current block; a fourth determining module, configured to obtain a target motion estimation result of the current block according to the initial motion estimation result; The first determining module is further configured to: Based on the forward motion vector and the predicted search starting point, determining a first search space, including: if the distance between the forward motion vector and the vector corresponding to the predicted search starting point is greater than a first length threshold and less than or equal to a maximum range threshold, using the distance as a first search range value; the first length threshold is determined based on a preset initial search range value; if the distance is less than or equal to the first length threshold, using the initial search range value as the first search range value; using the search space corresponding to the first search range value as the first search space; based on the forward motion vector and the first search space, determining a second search space, including: if the length corresponding to the forward motion vector is greater than a second length threshold, using a preset multiple of the first search range value of the first search space as a second search range value; the second length threshold is determined based on the first search range value; if the length corresponding to the forward motion vector is less than or equal to the second length threshold, using the first search range value as the second search range value; using the search space corresponding to the second search range value as the second search space; and using the first search space and the second search space as the candidate search spaces; and / or, Based on the forward motion vector and the candidate motion vector corresponding to the prediction search starting point, an optimal motion vector is determined; and the position point corresponding to the prediction search starting point and the optimal motion vector is used as the candidate search starting point.

7. The device according to claim 6, wherein The third determining module is further configured to: If the cost of the optimal motion vector is greater than a preset first cost threshold, using the predicted search starting point as the target search starting point, and using the second search space as the target search space; The fourth determining module is further configured to: The initial motion estimation result is used as the target motion estimation result.

8. The device according to claim 6, wherein The third determining module is further configured to: If the cost of the optimal motion vector is less than or equal to a preset first cost threshold, using the position point corresponding to the optimal motion vector as the target search starting point, and using the first search space as the target search space; The initial motion estimation result includes: the cost of the initial motion vector; The fourth determining module is further configured to: If the cost of the initial motion vector is greater than a preset second cost threshold, searching based on the prediction search starting point and the second search space to obtain an updated motion estimation result; obtaining the target motion estimation result based on the initial motion estimation result and the updated motion estimation result; or If the cost of the initial motion vector is less than or equal to a preset second cost threshold, the initial motion estimation result is used as the target motion estimation result.

9. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

11. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images