Motion estimation method, device, equipment, medium and product

Abstract

The invention provides a motion estimation method, device and equipment, a medium and a product, and relates to the technical field of artificial intelligence, in particular to the technical fields of cloud storage, cloud computing, video coding and the like. The motion estimation method comprises the following steps: determining a candidate search space and a candidate search starting point based on a look-ahead motion vector and a predicted search starting point of a current block; determining a target search starting point in the candidate search starting points, and determining a target search space in the candidate search space; searching based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block; and obtaining a target motion estimation result of the current block based on the initial motion estimation result.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to technical fields such as cloud storage, cloud computing, and video coding, and in particular to a motion estimation method, device, equipment, medium, and product. Background Art

[0002] High Efficiency Video Coding (HEVC) is a video coding standard that can effectively improve coding efficiency under the same video quality, thereby encoding the video at the lowest possible bit rate while maintaining a certain level of video quality.

[0003] In HEVC, inter-frame prediction coding is an important video compression technology, which obtains the difference between the current frame and the reference frame through motion estimation, and encodes the difference to achieve efficient compression.

[0004] Motion estimation uses pixel blocks as units, searches for a block that best matches the current block in the current frame in the search space of the reference frame, and obtains a motion estimation result based on the matching result. Summary of the invention

[0005] The present disclosure provides a motion estimation method, apparatus, device, medium and product.

[0006] According to one aspect of the present disclosure, a motion estimation method is provided, comprising: determining a candidate search space and a candidate search starting point based on a forward motion vector and a predicted search starting point of a current block; determining a target search starting point among the candidate search starting points, and determining a target search space among the candidate search space; searching based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block; and obtaining a target motion estimation result of the current block based on the initial motion estimation result.

[0007] According to another aspect of the present disclosure, a motion estimation device is provided, comprising: determining a candidate search space and a candidate search starting point based on a forward motion vector and a predicted search starting point of a current block; determining a target search starting point in the candidate search starting points, and determining a target search space in the candidate search space; searching based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block; and obtaining a target motion estimation result of the current block based on the initial motion estimation result.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method as described in any of the above aspects.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to any one of the above aspects.

[0010] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the methods described in any one of the above aspects.

[0011] According to the embodiments of the present disclosure, the motion estimation effect can be improved.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0014] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0015] Figure 2 is a schematic diagram of an application scenario for implementing an embodiment of the present disclosure;

[0016] Figure 3 is a schematic diagram of a candidate search space provided according to an embodiment of the present disclosure;

[0017] Figure 4 is a schematic diagram according to a second embodiment of the present disclosure;

[0018] Figure 5 is a schematic diagram of a motion estimation process in a forward-looking phase provided according to an embodiment of the present disclosure;

[0019] Figure 6 is a schematic diagram of a motion estimation process in an encoding stage provided according to an embodiment of the present disclosure;

[0020] Figure 7 is a schematic diagram according to a third embodiment of the present disclosure;

[0021] Figure 8It is a schematic diagram of an electronic device used to implement the motion estimation method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] The motion estimation process mainly includes: dividing the images of the current frame and the reference frame into several pixel blocks of the same size; selecting the pixel blocks that need to be motion estimated according to certain rules, such as selecting blocks containing more texture information or motion information; for each selected pixel block in the current frame (which can be called the current block), find the pixel block that best matches it in the search space of the reference frame (which can be called the matching block); calculate the relative position coordinates between the matching block and the current block, which is called the motion vector (MV). To this end, the motion vector can be obtained through motion estimation, which reflects the motion direction and distance of the matching block relative to the current block.

[0024] In addition, the cost corresponding to the MV may also be calculated, and the MV and its cost may be used as the motion estimation result.

[0025] The above cost can be specifically the rate-distortion cost. The rate-distortion cost comprehensively considers the number of coding bits and the degree of distortion. The number of coding bits refers to the number of bits required for encoding motion vectors and residual information. The degree of distortion is usually calculated using some error metric, such as mean square error or peak signal-to-noise ratio, which indicates the difference between the reconstructed image and the original image.

[0026] The size of the search space is determined according to the search range value (searchRange). The search space range value can be represented by the number of pixels from the center point to the boundary of the search space. For example, the search range value is represented by d (a positive integer). The search space is a square area with the mapping point corresponding to the current block to the reference frame as the center point and a side length of 2d pixels.

[0027] In the related art, the search range value is usually a fixed value. However, if the search range value is set to a small value, the corresponding search space is small, and it may be impossible to find the matching block that best matches the current block. If the search range value is set to a large value, the search time for the matching block will be too long, reducing the encoding efficiency.

[0028] In the video coding scenario, it can be divided into a lookahead stage and a coding stage. The lookahead stage is mainly to determine coding parameters, and the coding stage performs specific coding operations according to the coding parameters, such as the above-mentioned inter-frame prediction coding.

[0029] For the lookahead stage: the encoding parameters of the current frame, such as frame type, block division strategy, and encoding mode, can be obtained by analyzing the current frame and the reference frame. For example, if the reference frame is significantly different from the current frame, the I frame can be selected for encoding to provide better random access points and higher image quality. For another example, if the motion of a certain area in the reference frame is more complex, the corresponding area in the current frame can be divided into smaller blocks and a more complex encoding mode to improve encoding efficiency and accuracy.

[0030] In video encoding, there are three common frame types:

[0031] I frame (Intra-coded frame): also known as a key frame, is an independently encoded frame that does not rely on other frames for encoding.

[0032] P frame (Predictive-coded frame): relies on the previous I frame or P frame for encoding, and uses motion estimation and motion compensation technology to use information in the previous frame to predict the current frame.

[0033] B frame (Bidirectionally predictive-coded frame): relies on the previous and next frames for encoding. It uses bidirectional motion estimation and motion compensation technology to use reference frames in both the previous and next directions to predict the current frame.

[0034] In the lookahead stage, the current frame and the reference frame can also be spatially downsampled. Spatial downsampling is to reduce the resolution or size of the image, that is, to reduce the number of pixels in the image. Motion estimation is performed based on the downsampled image to obtain a preliminary motion estimation result, which can be expressed as lmv.

[0035] In the related art, the search range value of the search space in the encoding stage is usually a set fixed value, which is not accurate enough, thereby affecting the accuracy of the motion estimation result.

[0036] In the disclosed embodiment, the search range value in the encoding stage is determined based on the above-mentioned lmv, thereby improving the accuracy of the search space and enhancing the motion estimation effect.

[0037] In order to improve the motion estimation effect, the present disclosure provides the following embodiments.

[0038] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides a motion estimation method, such as Figure 1 As shown, the method includes:

[0039] 101. Determine a candidate search space and a candidate search starting point based on the forward motion vector and the predicted search starting point of the current block.

[0040] 102. Determine a target search starting point from among the candidate search starting points, and determine a target search space from among the candidate search spaces.

[0041] 103. Search based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block.

[0042] 104. Obtain a target motion estimation result of the current block based on the initial motion estimation result.

[0043] In the video encoding scenario, the current block refers to the pixel block currently being encoded in the video.

[0044] The video encoding process can be divided into a lookahead stage and an encoding stage. Both the lookahead stage and the encoding stage can perform a motion estimation process. The motion vector obtained by performing the motion estimation process in the lookahead stage is called a lookahead motion vector, which can be represented by lmv.

[0045] For the current block, a candidate motion vector (candidate MV) of the current block can be obtained. The position point obtained based on the candidate MV is called the prediction search starting point and can be represented by MVP.

[0046] After obtaining the lmv and mvp of the current block, the candidate search space and the candidate search starting point can be determined based on the lmv and mvp.

[0047] There are multiple candidate search spaces and candidate search starting points. Afterwards, one candidate search space can be determined from the multiple candidate search spaces as the target search space, and one candidate search starting point can be determined from the multiple candidate search starting points as the target search starting point.

[0048] After obtaining the target search space and the target search starting point, a search is performed based on the target search space and the target search starting point to obtain an initial motion estimation result. Specifically, starting from the target search starting point, a matching block corresponding to the current block is searched in the target search space, and the relative position coordinates between the matching block and the current block are calculated. The relative position coordinates can be called an initial motion vector (initial mv). After that, the cost of the initial mv can also be obtained, and the initial mv and its cost are used as the initial motion estimation result.

[0049] After obtaining the initial motion estimation result, the initial motion estimation result can be used as the target motion estimation result; or, the initial motion estimation result can be further processed to obtain the target motion estimation result; for example, the search starting point and search space are redetermined based on the initial motion estimation result, the search is re-performed based on the new search starting point and search space to obtain a new matching block, and the target motion estimation result is obtained based on the relative position coordinates of the new matching block and the current block.

[0050] In this embodiment, the candidate search space and the candidate search starting point are determined based on the forward motion vector and the predicted search starting point, the target search starting point is determined in the candidate search starting point, and the target search space is determined in the candidate search space. In this way, the forward motion vector is referenced in the motion estimation process, which improves the accuracy of the target search space and the target search starting point, thereby improving the motion estimation effect.

[0051] In order to better understand the present disclosure, the application scenarios involved in the present disclosure are described as follows:

[0052] Figure 2 It is a schematic diagram of an application scenario for implementing the embodiment of the present disclosure.

[0053] In the video coding scenario, the current frame is the image frame to be coded in the video, and the reference frame is the frame referenced by the coding. Based on different coding modes, the reference frame can be the forward frame and / or backward frame of the current frame. When motion estimation is performed, encoding is usually performed in units of pixel blocks. For this purpose, the current frame and the reference frame can be divided into multiple pixel blocks. The pixel block currently coded in the current frame is called the current block, and the pixel block in the reference frame that best matches the current block is called the matching block.

[0054] like Figure 2 As shown, for the current block, through the motion estimation process, a motion vector (MV) can be obtained, and the MV is the relative position coordinate between the matching block and the current block. In addition, the cost corresponding to the MV can also be obtained, and the MV and its cost are used as the motion estimation result of the current block.

[0055] A matching block is a block of pixels in the reference frame, determined after searching in the search space.

[0056] The size of the search space can be represented by the search range value (searchRange). The larger the search range value, the larger the search space, and the search can be performed in a larger search space, thereby improving the accuracy of the matching block. However, due to the large search space, it takes a longer time. The smaller the search range value, the smaller the search space, and the time is shorter. However, due to the limited search range, the accuracy of the matching block is affected.

[0057] In order to improve the accuracy of the search space, in the embodiment of the present disclosure, a candidate search space may be first acquired, and then a target search space may be determined in the candidate search space, and then a search may be performed in the target search space.

[0058] Figure 3 It is a schematic diagram of a candidate search space provided according to an embodiment of the present disclosure.

[0059] like Figure 3 As shown, in the embodiment of the present disclosure, the candidate search space includes a first search space and a second search space. The search range value of the first search space is represented by d1, and the search range value of the second search space is represented by d2. For this reason, the first search space is a square area with a side length of 2*d1, and the second search space is a square area with a side length of 2*d2.

[0060] The above two candidate search spaces (the first search space and the second search space) are determined based on the forward motion vector and the prediction search starting point.

[0061] In the video coding scenario, it can be divided into a lookahead stage and a coding stage. The motion vector obtained in the lookahead stage is called a lookahead motion vector, which can be represented by lmv.

[0062] Specifically, in the look-ahead stage, the current frame and the reference frame may be spatially downsampled, and motion estimation may be performed on the downsampled current frame and the reference frame to obtain the lmv.

[0063] For the current block, the motion vector of the encoded neighboring block of the current block can be used as the candidate motion vector of the current block. The neighboring blocks may include spatial neighboring blocks and / or temporal neighboring blocks. Taking spatial neighboring blocks as an example, they may specifically include the upper block, left block, upper right block, etc. of the current block.

[0064] After obtaining the candidate motion vectors of the current block, the prediction search starting point can be determined based on the candidate motion vectors, represented by MVP. For example, one of these candidate motion vectors is selected, such as the candidate motion vector with the highest frequency of occurrence or the best encoding effect, and the position point corresponding to the selected candidate motion vector is used as the prediction search starting point; or, the mean of these candidate motion vectors is calculated, and the position point corresponding to the mean is used as the prediction search starting point. For example, a selected candidate motion vector is (4, -3), and the positive direction of the x-axis is to the right and the positive direction of the y-axis is downward, then in the reference frame, the mapping position of the current block is horizontally moved 4 pixels to the right and 3 pixels upward as the prediction search starting point.

[0065] After obtaining the forward motion vector lmv and the predicted search starting point mvp, the candidate search space is determined based on these two pieces of information.

[0066] The first search space is determined based on the forward motion vector and the prediction search starting point, and the second search space is determined based on the forward motion vector and the first search space.

[0067] The search range value of the first search space is called a first search range value, denoted by d1, and the search range value of the second search space is called a second search range value, denoted by d2.

[0068] In some cases, d1=d, and in other cases, d1=d0. d0 is a preset initial search range value, and d is the distance between the forward motion vector lmv and the vector corresponding to the prediction search starting point mvp. The specific situation is determined based on d.

[0069] In some cases, d2=d1*k, where k is a preset multiple greater than 1, and in other cases, d2=d1. The specific case is determined based on the corresponding length of the forward motion vector lmv.

[0070] In this embodiment, the first search space is determined based on the forward motion vector and the predicted search starting point, and the second search space is determined based on the first search space and the forward motion vector. Multiple candidate search spaces can be obtained, and an accurate target search space can be determined based on the candidate search spaces, thereby improving the motion estimation effect.

[0071] Specifically, if the distance between the forward motion vector and the vector corresponding to the predicted search starting point is greater than a first length threshold and less than or equal to a maximum range threshold, the distance is used as a first search range value; the first length threshold is determined based on a preset initial search range value; if the distance is less than or equal to the first length threshold, the initial search range value is used as the first search range value; and the search space corresponding to the first search range value is used as the first search space.

[0072] The initial search range value is represented by d0, which is a preset value.

[0073] For the first search range value, the formula is as follows:

[0074]

[0075] Wherein, d1 is the first search range value;

[0076] d0 is the preset initial range value;

[0077] L1 is the first length threshold; it is determined according to the initial search range value, such as,

[0078] L1=k1*d0

[0079] k1 is a preset ratio value greater than 1.

[0080] L max is the maximum range threshold. In video encoding, assuming that the image size in the video is expressed as L*W (length*width), then L max is the minimum value of L and W.

[0081] d is the distance between the forward motion vector lmv and the vector corresponding to the prediction search starting point mvp, which is the distance between vectors. For example, lmv = (3, 2), the vector corresponding to mvp is (4, -3), and the distance between the two can be expressed as:

[0082]

[0083] Based on the above formula, the first search range value d1 can be obtained, and then the first search space can be determined.

[0084] For the second search space:

[0085] If the length corresponding to the forward motion vector is greater than a second length threshold, a preset multiple of the first search range value of the first search space is used as the second search range value; the second length threshold is determined based on the first search range value; if the length corresponding to the forward motion vector is less than or equal to the second length threshold, the first search range value is used as the second search range value; and the search space corresponding to the second search range value is used as the second search space.

[0086] The formula is as follows:

[0087]

[0088] Wherein, d2 is the second search range value;

[0089] d1 is the first search range value;

[0090] k is a preset multiple, which is a value greater than 1, and the maximum value is determined based on the video resolution so that the second search range value does not exceed the video resolution;

[0091] L2 is a second length threshold, which is determined according to the first search range value d1, for example,

[0092] L2=k2*d1

[0093] k2 is a preset ratio value greater than 1.

[0094] |lmv| is the length corresponding to the forward motion vector; it can be represented by the distance between the position point corresponding to the forward motion vector and the coordinate origin, or it can be represented by the absolute value of the position coordinate of the forward motion vector itself. For example, the forward motion vector is represented as:

[0095] lmv=(x,y)

[0096] Or, |lmv|=max(|x|,|y|), where || is the absolute value and max() is the maximum value.

[0097] Based on the above formula, the second search range value d2 can be obtained, and then the second search space can be determined.

[0098] In addition, a candidate search starting point may be determined based on the forward motion vector and the predicted search starting point.

[0099] The candidate search starting points may include: a predicted search starting point and an optimal search starting point, the optimal search starting point is a position point corresponding to an optimal motion vector, and the optimal motion vector is determined based on the forward motion vector and the candidate motion vector.

[0100] For example, the candidate motion vectors are represented by mv1~mvn, and the forward motion vector is represented by lmv. After obtaining these motion vectors, the cost of each motion vector is calculated, and the motion vector with the smallest cost is taken as the optimal motion vector, represented by mv*. The position point corresponding to the optimal mv* is taken as the optimal search starting point, represented by mvq. The candidate search starting points include: mvp and mvq.

[0101] In this embodiment, the position point corresponding to the predicted search starting point and the optimal motion vector is used as a candidate search starting point. In the subsequent process, a suitable starting point can be selected as the target search starting point according to the specific situation, thereby improving the accuracy of the target search starting point and further improving the accuracy of motion estimation.

[0102] After obtaining the candidate search space and the candidate search starting point, the target search space can be determined from the candidate search space, and the target search starting point can be determined from the candidate search starting point, thereby obtaining the target motion estimation result.

[0103] In some embodiments, if the cost of the optimal motion vector is greater than a preset first cost threshold, the predicted search starting point is used as the target search starting point, and the second search space is used as the target search space. Accordingly, the initial motion estimation result obtained based on the target search starting point and the target search space is used as the target motion estimation result of the current block.

[0104] Specifically, assuming that the optimal motion vector is represented by mv*, its cost is represented by mv*_cost, and the first cost threshold is represented by cost1, then, if mv*_cost>cost1, the target search starting point = mvp, the target search space = the second search space, and the corresponding search range value is d2. At this time, taking mvp as the starting point, the search is performed in the search space with a search range value of d2.

[0105] After searching, a matching block of the current block is obtained. Assuming that the relative position coordinates of the matching block and the current block are represented by mv1, and the corresponding cost is represented by mv1_cost, the initial motion estimation result and the target motion estimation result both include: mv1 and mv1_cost.

[0106] In some embodiments, if the cost of the optimal motion vector is less than or equal to a preset first cost threshold, the optimal search starting point is used as the target search starting point, and the first search space is used as the target search space.

[0107] Further, the initial motion estimation result includes: the cost of the initial motion vector;

[0108] If the cost of the initial motion vector is less than or equal to a preset second cost threshold, the initial motion estimation result is used as the target motion estimation result.

[0109] Specifically, assuming that the optimal motion vector is represented by mv*, its cost is represented by mv*_cost, and the first cost threshold is represented by cost1, then, if mv*_cost<=cost1, the target search starting point = mvq, the target search space = the first search space, and the corresponding search range value is d1. At this time, taking mvq as the starting point, the search is performed in the search space with a search range value of d1.

[0110] After searching, a matching block of the current block is obtained. Assuming that the relative position coordinates of the matching block and the current block are represented by mv2, and the corresponding cost is represented by mv2_cost, the initial motion estimation result includes: mv2 and mv2_cost.

[0111] Assuming that the second cost threshold is represented by cost2, if mv2_cost<=cost2, the target motion estimation result is the same as the initial motion estimation result, that is, the target motion estimation result includes: mv2 and mv2_cost.

[0112] In some embodiments, if the cost of the initial motion vector is greater than a preset second cost threshold, a search is performed based on the predicted search starting point and the second search space to obtain an updated motion estimation result; based on the initial motion estimation result and the updated motion estimation result, the target motion estimation result is obtained.

[0113] Specifically, assuming that mv2_cost>cost2, a new search is performed based on mvp in the search space of size d2 to obtain a new matching block, and the relative position coordinates of the new matching block and the current block are calculated and represented by mv3. The corresponding cost is represented by mv3_cost. After that, mv2_cost and mv3_cost are compared, and the smaller cost and its corresponding mv are taken as the target motion estimation result. For example, if mv3_cost is less than mv2_cost, the target motion estimation result includes: mv3 and mv3_cost.

[0114] In this way, appropriate target motion estimation results can be obtained in different situations.

[0115] In combination with the above application scenarios, the present disclosure also provides the following embodiments.

[0116] Figure 4 is a schematic diagram according to a second embodiment of the present disclosure. This embodiment provides a motion estimation method, the method comprising:

[0117] 401. In a look-ahead phase of video coding, a target search method is determined from a plurality of candidate search methods based on a frame distance between a current frame and a reference frame; and a look-ahead motion vector of a current block is obtained based on the target search method.

[0118] 402. In a coding stage of video coding, obtain a target motion estimation result of the current block based on the forward motion vector.

[0119] In the forward-looking phase, it is assumed that there are two candidate search methods, namely the star (STAR) search algorithm and the hexagonal (HEX) search algorithm. The STAR search algorithm is an algorithm for motion estimation, which belongs to a search method in video coding technology. The STAR search algorithm combines the advantages of full search and diamond search, and reduces the amount of calculation and improves the search efficiency by selectively searching specific areas. The HEX search algorithm searches the pixel blocks in the search area in a hexagonal pattern to find the best matching block, thereby reducing the amount of calculation and improving the search efficiency.

[0120] Figure 5 It is a schematic diagram of the motion estimation process in the forward-looking stage provided according to an embodiment of the present disclosure.

[0121] like Figure 5 As shown, the method includes:

[0122] 501. Perform spatial domain downsampling on the current frame and the reference frame.

[0123] 502. Determine whether the frame distance between the current frame and the reference frame is greater than the preset frame distance. If so, execute. Otherwise, execute

[0124] The frame distance can be represented by the difference in sequence numbers between the current frame and the reference frame. For example, if the current frame is the t-th frame and the reference frame is the (t-2)-th frame, then the frame distance=2.

[0125] 503. Based on the STAR search algorithm, motion estimation is performed on the current frame and the reference frame after spatial downsampling to obtain a forward motion vector lmv.

[0126] 504. Based on the HEX search algorithm, motion estimation is performed on the current frame and the reference frame after spatial downsampling to obtain a forward motion vector lmv.

[0127] Since the STAR search algorithm has a larger search range, it can search in a wider range when the frame distance is large, thereby improving reliability. When the frame distance is small, the HEX search algorithm can improve the search efficiency and thus improve the coding efficiency.

[0128] In this embodiment, in the forward-looking stage, by determining a target search method from a plurality of candidate search methods and obtaining a forward-looking motion vector based on the target search method, the accuracy of the forward-looking motion vector can be improved, thereby improving the accuracy of motion estimation in the encoding stage and improving the video encoding effect.

[0129] Figure 6 It is a schematic diagram of the motion estimation process in the encoding stage provided according to an embodiment of the present disclosure.

[0130] like Figure 6 As shown, the method includes:

[0131] 601. Obtain the forward motion vector lmv and the prediction search starting point mvp of the current block.

[0132] The forward motion vector lmv is obtained in the forward phase, and the prediction search starting point mvp is obtained based on the candidate mv. The specific acquisition process can refer to the above related description.

[0133] 602. Determine whether the distance d between the vectors corresponding to lmv and mvp is greater than the first length threshold L1 and less than or equal to the maximum range threshold L max If yes, execute 603; otherwise, execute 604.

[0134] 603. Use the distance as a first search range value, expressed as d1=d.

[0135] 604. Use the preset initial search range value d0 as the first search range value, expressed as d1 = d0.

[0136] The specific calculation process of the first search range value d1 can refer to the above related description.

[0137] In this way, after the first search range value d1 of 603 or 604 is obtained, the search space corresponding to the first search range value is used as the first search space.

[0138] In this embodiment, the distance between the forward motion vector and the vector corresponding to the prediction search starting point or the initial search range value is used as the first search range value, so as to realize adaptive adjustment of the first search space, thereby obtaining accurate target motion estimation results with higher efficiency and improving the motion estimation effect.

[0139] 605. Determine whether the length L corresponding to lmv is greater than the second length threshold L2. If so, execute 606; otherwise, execute 607.

[0140] 606. Use a preset multiple k of the first search range value as a second search range value, expressed as d2 = d1*k.

[0141] 607. Use the first search range value as the second search range value, expressed as d2=d1.

[0142] The specific calculation process of the second search range value d2 can refer to the above related description.

[0143] In this way, after the second search range value d2 of 607 or 608 is obtained, the search space corresponding to the second search range value is used as the second search space.

[0144] In this embodiment, the preset multiple of the first search range value or the first search range value is used as the second search range value, which can realize adaptive adjustment of the second search space, thereby obtaining accurate target motion estimation results with higher efficiency and improving motion estimation effect.

[0145] 608. Obtain the optimal mv based on lmv and candidate mv, and obtain the cost of the optimal mv, represented by mv*_cost. Then, determine whether mv*_cost is greater than a preset first cost threshold cost1. If so, execute 609, otherwise execute 610.

[0146] 609. Search based on mvp and d2 to obtain the target motion estimation result.

[0147] Specifically, MVP is used as the target search starting point, and the second search space is used as the target search space. A search is performed based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block, and the initial motion estimation result is used as the target motion estimation result of the current block.

[0148] In this embodiment, when the cost of the optimal motion vector is greater than the first cost threshold, searching in a larger second search space based on MVP can improve the accuracy of the target motion estimation result.

[0149] 610. Search based on mvq and d1 to obtain mv_out and mv_out_cost.

[0150] Specifically, the position point mvq corresponding to the optimal mv is used as the target search starting point, the first search space is used as the target search space, and a search is performed based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block, which is represented by mv_out and mv_out_cost.

[0151] 611. Determine whether mv_out_cost is greater than a preset second cost threshold value cost2. If so, execute 612; otherwise, execute 614.

[0152] 612. Search based on mvp and d2 to obtain an updated motion estimation result.

[0153] Specifically, mvp is used as a new search starting point, and the second search space is used as a new search space. A search is performed based on the new search starting point and the new search space to obtain a new matching block of the current block. An updated motion estimation result is obtained based on the new matching block, which is represented by mv_out' and mv_out_cost'.

[0154] 613. Select the better one between the initial motion estimation result and the updated motion estimation result as the target motion estimation result.

[0155] Specifically, the result with a smaller cost is taken as the better one, for example, if mv_out_cost' is less than mv_out_cost, the target motion estimation result includes mv_out' and mv_out_cost'.

[0156] In this embodiment, when the cost of the initial motion vector is greater than the second cost threshold, a new search is performed to obtain an updated motion estimation result, and a target motion estimation result is obtained based on the initial motion estimation result and the updated motion estimation result, so as to improve the accuracy of the target motion estimation result.

[0157] 614. Use the initial motion estimation result as the target motion estimation result.

[0158] Specifically, the target motion estimation result includes: mv_out and mv_out_cost.

[0159] In this embodiment, when the cost of the initial motion vector is less than or equal to the second cost threshold, the initial motion estimation result obtained by searching based on mvq and d1 is used as the target motion estimation result, which can improve the motion estimation efficiency.

[0160] Figure 7is a schematic diagram according to the third embodiment of the present disclosure. This embodiment provides a motion estimation device. The device 700 includes: a first determination module 701 , a second determination module 702 , a third determination module 703 and a fourth determination module 704 .

[0161] The first determination module 701 is used to determine the candidate search space and the candidate search starting point according to the forward motion vector and the predicted search starting point of the current block; the second determination module 702 is used to determine the target search starting point in the candidate search starting point, and to determine the target search space in the candidate search space; the third determination module 703 is used to search according to the target search starting point and the target search space to obtain the initial motion estimation result of the current block; the fourth determination module 704 is used to obtain the target motion estimation result of the current block according to the initial motion estimation result.

[0162] In this embodiment, the candidate search space and the candidate search starting point are determined based on the forward motion vector and the predicted search starting point, the target search starting point is determined in the candidate search starting point, and the target search space is determined in the candidate search space. In this way, the forward motion vector is referenced in the motion estimation process, which improves the accuracy of the target search space and the target search starting point, thereby improving the motion estimation effect.

[0163] In some embodiments, the first determining module 701 is further configured to:

[0164] Based on the forward motion vector and the predicted search starting point, a first search space is determined; based on the forward motion vector and the first search space, a second search space is determined; and the first search space and the second search space are used as the candidate search spaces.

[0165] In this embodiment, the first search space is determined based on the forward motion vector and the predicted search starting point, and the second search space is determined based on the first search space and the forward motion vector. Multiple candidate search spaces can be obtained, and an accurate target search space can be determined based on the candidate search spaces, thereby improving the motion estimation effect.

[0166] In some embodiments, the first determining module 701 is further configured to:

[0167] If the distance between the forward motion vector and the vector corresponding to the predicted search starting point is greater than a first length threshold and less than or equal to a maximum range threshold, the distance is used as a first search range value; the first length threshold is determined based on a preset initial search range value; if the distance is less than or equal to the first length threshold, the initial search range value is used as the first search range value; and the search space corresponding to the first search range value is used as the first search space.

[0168] In this embodiment, the distance between the forward motion vector and the vector corresponding to the prediction search starting point or the initial search range value is used as the first search range value, so as to realize adaptive adjustment of the first search space, thereby obtaining accurate target motion estimation results with higher efficiency and improving the motion estimation effect.

[0169] In some embodiments, the first determining module 701 is further configured to:

[0170] If the length corresponding to the forward motion vector is greater than a second length threshold, a preset multiple of the first search range value of the first search space is used as the second search range value; the second length threshold is determined based on the first search range value; if the length corresponding to the forward motion vector is less than or equal to the second length threshold, the first search range value is used as the second search range value; and the search space corresponding to the second search range value is used as the second search space.

[0171] In this embodiment, the preset multiple of the first search range value or the first search range value is used as the second search range value, which can realize adaptive adjustment of the second search space, thereby obtaining accurate target motion estimation results with higher efficiency and improving motion estimation effect.

[0172] In some embodiments, the first determining module 701 is further configured to:

[0173] Based on the forward motion vector and the candidate motion vector corresponding to the prediction search starting point, an optimal motion vector is determined; and the position point corresponding to the prediction search starting point and the optimal motion vector is used as the candidate search starting point.

[0174] In this embodiment, the position point corresponding to the predicted search starting point and the optimal motion vector is used as a candidate search starting point. In the subsequent process, a suitable starting point can be selected as the target search starting point according to the specific situation, thereby improving the accuracy of the target search starting point and further improving the accuracy of motion estimation.

[0175] In some embodiments, the third determining module 703 is further configured to:

[0176] If the cost of the optimal motion vector is greater than a preset first cost threshold, using the predicted search starting point as the target search starting point, and using the second search space as the target search space;

[0177] The fourth determining module 704 is further configured to:

[0178] The initial motion estimation result is used as the target motion estimation result.

[0179] In this embodiment, when the cost of the optimal motion vector is greater than the first cost threshold, searching in a larger second search space based on MVP can improve the accuracy of the target motion estimation result.

[0180] In some embodiments, the third determining module 703 is further configured to:

[0181] If the cost of the optimal motion vector is less than or equal to a preset first cost threshold, the position point corresponding to the optimal motion vector is used as the target search starting point, and the first search space is used as the target search space.

[0182] In some embodiments, the initial motion estimation result includes: a cost of an initial motion vector;

[0183] The fourth determining module 704 is further configured to:

[0184] If the cost of the initial motion vector is greater than a preset second cost threshold, searching based on the prediction search starting point and the second search space to obtain an updated motion estimation result; obtaining the target motion estimation result based on the initial motion estimation result and the updated motion estimation result; or

[0185] If the cost of the initial motion vector is less than or equal to a preset second cost threshold, the initial motion estimation result is used as the target motion estimation result.

[0186] In this embodiment, when the cost of the initial motion vector is greater than the second cost threshold, a new search is performed to obtain an updated motion estimation result, and a target motion estimation result is obtained based on the initial motion estimation result and the updated motion estimation result, so as to improve the accuracy of the target motion estimation result.

[0187] In this embodiment, when the cost of the initial motion vector is less than or equal to the second cost threshold, the initial motion estimation result obtained by searching based on mvq and d1 is used as the target motion estimation result, which can improve the motion estimation efficiency.

[0188] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.

[0189] It can be understood that the “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.

[0190] It is understandable that unless there is any special limitation on the sequence of steps in the process, it means that the timing relationship between these steps is not limited.

[0191] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0192] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0193] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 800 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0194] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0195] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0196] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as motion estimation methods. For example, in some embodiments, the motion estimation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 809. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the motion estimation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the motion estimation method in any other appropriate manner (e.g., by means of firmware).

[0197] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0198] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable task processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0199] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0200] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0201] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0202] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0203] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0204] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A motion estimation method, comprising: Determine a candidate search space and a candidate search starting point based on the forward motion vector and the predicted search starting point of the current block; Determining a target search starting point among the candidate search starting points, and determining a target search space among the candidate search spaces; Searching based on the target search starting point and the target search space to obtain an initial motion estimation result of the current block; A target motion estimation result of the current block is obtained based on the initial motion estimation result.

2. The method according to claim 1, wherein: The determining of a candidate search space based on the forward motion vector and the candidate motion vector comprises: Determining a first search space based on the forward motion vector and the prediction search starting point; Determine a second search space based on the forward motion vector and the first search space; The first search space and the second search space are used as the candidate search spaces.

3. The method according to claim 2, wherein: The determining of the first search space based on the forward motion vector and the prediction search starting point includes: If the distance between the forward motion vector and the vector corresponding to the prediction search starting point is greater than a first length threshold and less than or equal to a maximum range threshold, the distance is used as a first search range value; the first length threshold is determined based on a preset initial search range value; If the distance is less than or equal to the first length threshold, taking the initial search range value as the first search range value; The search space corresponding to the first search range value is used as the first search space.

4. The method according to claim 2, wherein: The determining of the second search space based on the forward motion vector and the first search space includes: If the length corresponding to the forward motion vector is greater than a second length threshold, taking a preset multiple of the first search range value of the first search space as a second search range value; the second length threshold is determined based on the first search range value; If the length corresponding to the forward motion vector is less than or equal to the second length threshold, using the first search range value as the second search range value; The search space corresponding to the second search range value is used as the second search space.

5. The method according to claim 1, wherein: The step of determining a candidate search starting point based on the forward motion vector and the predicted search starting point includes: Determining an optimal motion vector based on the forward motion vector and the candidate motion vector corresponding to the prediction search starting point; The position point corresponding to the predicted search starting point and the optimal motion vector is used as the candidate search starting point.

6. The method according to claim 5, wherein: The step of determining a target search starting point from among the candidate search starting points and determining a target search space from among the candidate search spaces comprises: If the cost of the optimal motion vector is greater than a preset first cost threshold, using the predicted search starting point as the target search starting point, and using the second search space as the target search space; The obtaining the target motion estimation result of the current block based on the initial motion estimation result includes: The initial motion estimation result is used as the target motion estimation result.

7. The method according to claim 5, wherein: The step of determining a target search starting point from among the candidate search starting points and determining a target search space from among the candidate search spaces comprises: If the cost of the optimal motion vector is less than or equal to a preset first cost threshold, taking the position point corresponding to the optimal motion vector as the target search starting point, and taking the first search space as the target search space; The initial motion estimation result includes: the cost of the initial motion vector; The obtaining the target motion estimation result of the current block based on the initial motion estimation result includes: If the cost of the initial motion vector is greater than a preset second cost threshold, searching based on the prediction search starting point and the second search space to obtain an updated motion estimation result; obtaining the target motion estimation result based on the initial motion estimation result and the updated motion estimation result; or If the cost of the initial motion vector is less than or equal to a preset second cost threshold, the initial motion estimation result is used as the target motion estimation result.

8. The method according to claim 1, further comprising: In the look-ahead phase of the video encoding, based on the frame distance between the current frame and the reference frame, determining a target search mode from among a plurality of candidate search modes; The forward motion vector is obtained based on the target search method.

9. A motion estimation device, comprising: A first determination module is used to determine a candidate search space and a candidate search starting point according to a forward motion vector and a predicted search starting point of a current block; A second determination module, configured to determine a target search starting point from among the candidate search starting points, and to determine a target search space from among the candidate search spaces; A third determination module, configured to search according to the target search starting point and the target search space to obtain an initial motion estimation result of the current block; The fourth determination module is used to obtain a target motion estimation result of the current block according to the initial motion estimation result.

10. The device according to claim 9, wherein: The first determining module is further configured to: Determine a first search space based on the forward motion vector and the predicted search starting point; determine a second search space based on the forward motion vector and the first search space; Using the first search space and the second search space as the candidate search spaces; and / or, Based on the forward motion vector and the candidate motion vector corresponding to the prediction search starting point, an optimal motion vector is determined; and the position point corresponding to the prediction search starting point and the optimal motion vector is used as the candidate search starting point.

11. The device according to claim 10, wherein: The first determining module is further configured to: If the distance between the forward motion vector and the vector corresponding to the prediction search starting point is greater than a first length threshold and less than or equal to a maximum range threshold, the distance is used as a first search range value; the first length threshold is determined based on a preset initial search range value; if the distance is less than or equal to the first length threshold, the initial search range value is used as the first search range value; and a search space corresponding to the first search range value is used as the first search space; and / or, If the length corresponding to the forward motion vector is greater than a second length threshold, a preset multiple of the first search range value of the first search space is used as the second search range value; the second length threshold is determined based on the first search range value; if the length corresponding to the forward motion vector is less than or equal to the second length threshold, the first search range value is used as the second search range value; and the search space corresponding to the second search range value is used as the second search space.

12. The device according to claim 10, wherein: The third determination module is further used for: If the cost of the optimal motion vector is greater than a preset first cost threshold, using the predicted search starting point as the target search starting point, and using the second search space as the target search space; The fourth determination module is further configured to: The initial motion estimation result is used as the target motion estimation result.

13. The device according to claim 10, wherein: The third determination module is further used for: If the cost of the optimal motion vector is less than or equal to a preset first cost threshold, taking the position point corresponding to the optimal motion vector as the target search starting point, and taking the first search space as the target search space; The initial motion estimation result includes: the cost of the initial motion vector; The fourth determination module is further configured to: If the cost of the initial motion vector is greater than a preset second cost threshold, searching based on the prediction search starting point and the second search space to obtain an updated motion estimation result; obtaining the target motion estimation result based on the initial motion estimation result and the updated motion estimation result; or If the cost of the initial motion vector is less than or equal to a preset second cost threshold, the initial motion estimation result is used as the target motion estimation result.

14. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

16. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images