Motion estimation method and device
By using the motion vector calculation of the forward prediction module and adjacent macroblocks in video encoding, it is possible to determine whether the macroblock is a skipped block and determine the target motion vector, which solves the problem of large amount of motion estimation calculation in the prior art, and improves the Efficiency and accuracy.
Patent Information
- Application Number
- CN202411325871.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-12-27
AI Technical Summary
The existing video encoding methods have a large amount of calculation during the motion estimation process, resulting in an increase in time overhead and affecting efficiency.
The forward prediction module obtains the motion vector, intra prediction cost and inter prediction cost, determine whether the macroblock is a skipped block, and calculates the target motion vector based on the motion vector of adjacent macroblocks, reducing the complex cost of each macroblock Calculate and improve efficiency.
The calculation amount of motion estimation is reduced, efficiency is improved, and the accuracy of the target motion vector is improved, and unnecessary coding operations are reduced.
Smart Images
Figure CN119211575A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of video encoding processing, and more specifically, to a motion estimation method and device. Background Art
[0002] Inter-frame prediction is a technology used in video coding to remove redundant information in the temporal domain by using the temporal correlation between video frames. It predicts the pixel value of the current frame through the encoded reference frame, thereby achieving the purpose of compressing video data. Motion estimation is the core link of inter-frame prediction, and its purpose is to find a best matching block for the current macroblock in the reference frame, so as to obtain an accurate motion vector. Related studies have shown that the time consumption of motion estimation accounts for 60%-80% of the total video encoding time. Therefore, the efficiency of motion estimation is crucial to video encoding.
[0003] Existing methods usually calculate the inter-frame prediction cost of the current macroblock through complex motion search and determine the target motion vector based on the cost.
[0004] However, this method performs complex cost calculations on each macroblock, resulting in a large amount of calculations, thereby increasing the time overhead of motion estimation. Summary of the invention
[0005] The present application is provided to solve the above problems existing in the prior art. According to a motion estimation method and device of an embodiment of the present application, the amount of calculation can be reduced and the efficiency of motion estimation can be improved.
[0006] In a first aspect, the present application provides a motion estimation method, comprising:
[0007] Obtaining the first motion vector, intra-frame prediction cost and inter-frame prediction cost of the current macroblock from the forward prediction module;
[0008] Determining whether the current macroblock is a skip block based on the first motion vector, the intra prediction cost, and the inter prediction cost;
[0009] In response to the current macroblock not being a skip block, calculating a second motion vector of the current macroblock based on target motion vectors of a plurality of first macroblocks adjacent to the current macroblock;
[0010] determining whether a difference between the second motion vector and the first motion vector is within a preset range;
[0011] In response to the difference being within a preset range, determining a target motion vector of the current macroblock based on the first motion vector;
[0012] In response to the difference being not within a preset range, a target motion vector of the current macroblock is determined based on the second motion vector.
[0013] In a second aspect, the present application provides a motion estimation device, comprising:
[0014] An acquisition module, configured to acquire a first motion vector, an intra-frame prediction cost, and an inter-frame prediction cost of a current macroblock from a forward prediction module;
[0015] a calculation module configured to determine whether the current macroblock is a skip block based on the first motion vector, the intra prediction cost and the inter prediction cost; in response to the current macroblock not being a skip block, calculate a second motion vector of the current macroblock based on target motion vectors of a plurality of first macroblocks adjacent to the current macroblock;
[0016] A determination module is configured to determine whether the difference between the second motion vector and the first motion vector is within a preset range; in response to the difference being within the preset range, determine the target motion vector of the current macroblock based on the first motion vector; in response to the difference not being within the preset range, determine the target motion vector of the current macroblock based on the second motion vector.
[0017] In a third aspect, the present application provides an electronic device, including:
[0018] one or more processors;
[0019] a storage device for storing one or more programs,
[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.
[0021] In a fourth aspect, the present application provides a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the above embodiments.
[0022] The beneficial effect of the embodiment of the present application is that: based on the first motion vector provided by the forward prediction module, the intra-frame prediction cost and the inter-frame prediction cost, motion estimation is performed, and there is no need to calculate the intra-frame prediction cost, the inter-frame prediction cost and the target motion vector for each macroblock, which can reduce the amount of calculation and improve the efficiency of motion estimation. Specifically, the embodiment of the present application determines whether the first motion vector provided by the forward prediction module is available based on the difference between the second motion vector and the first motion vector. If the first motion vector is significantly different from the second motion vector calculated by the adjacent macroblock, indicating that the error of the first motion vector is large, the target motion vector is calculated based on the second motion vector, otherwise, the target motion vector is calculated based on the first motion vector. This process can improve the accuracy of the calculated target motion vector. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with letter suffixes or different letter suffixes may represent different instances of similar parts. The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the specification and claims, are used to illustrate the disclosed embodiments. When appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.
[0024] Figure 1 is a flow chart of a motion estimation method provided by an embodiment of the present application;
[0025] Figure 2 is a flow chart of a motion estimation method provided by another embodiment of the present application;
[0026] Figure 3 It is a schematic diagram of a motion estimation device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific embodiments, but are not intended to limit the present application. The terms "first", "second" and "third" used in the present application are only intended to distinguish the corresponding features, do not represent the need for such an order, and do not necessarily represent only the singular form.
[0028] like Figure 1 As shown, the embodiment of the present application provides a motion estimation method, including:
[0029] Step 101: Obtain the first motion vector, intra-frame prediction cost and inter-frame prediction cost of the current macroblock from the forward prediction module.
[0030] The forward prediction module is the lookahead module, which corresponds to the lookahead stage, such as the lookahead stage of the H.264 encoder.
[0031] In the encoding process, the lookahead stage is a step that occurs after the raw data is input and before the actual encoding process. It is responsible for analyzing future frames in order to make better decisions during the encoding process. This pre-analysis allows the encoder to optimize bitrate allocation, determine frame types (such as I-frames, P-frames, or B-frames), predict motion vectors, and make other encoding optimization decisions.
[0032] The intra prediction cost represents the cost incurred for encoding the current macroblock during the intra prediction process, while the inter prediction cost represents the cost incurred for encoding the macroblock during the inter prediction process.
[0033] Step 102: Determine whether the current macroblock is a skip block based on the first motion vector, the intra prediction cost and the inter prediction cost.
[0034] Step 103: In response to the current macroblock not being a skip block, a second motion vector of the current macroblock is calculated based on target motion vectors of a plurality of first macroblocks adjacent to the current macroblock.
[0035] In response to the current macroblock belonging to a skip block, the operation on the current macroblock is terminated.
[0036] Step 104: Determine whether the difference between the second motion vector and the first motion vector is within a preset range. In response to the difference being within the preset range, execute step 105. In response to the difference not being within the preset range, execute step 106.
[0037] The difference between the two vectors may be characterized by the projection length of the vector on the X-axis and / or the projection length of the vector on the Y-axis. The preset range may be set according to the actual application scenario.
[0038] Step 105: Based on the first motion vector, determine the target motion vector of the current macroblock.
[0039] Step 106: Determine a target motion vector of the current macroblock based on the second motion vector.
[0040] The embodiment of the present application performs motion estimation based on the first motion vector, intra-frame prediction cost and inter-frame prediction cost provided by the forward prediction module. It is not necessary to calculate the intra-frame prediction cost, inter-frame prediction cost and target motion vector for each macroblock, which can improve the efficiency of motion estimation. Specifically, the embodiment of the present application determines whether the first motion vector provided by the forward prediction module is available based on the difference between the second motion vector and the first motion vector. If the first motion vector is significantly different from the second motion vector calculated by the adjacent macroblock, indicating that the error of the first motion vector is relatively large, the target motion vector is calculated based on the second motion vector. Otherwise, the target motion vector is calculated based on the first motion vector. This process can improve the accuracy of the calculated target motion vector.
[0041] In one embodiment of the present application, determining whether the current macroblock is a skip block based on the first motion vector, the intra prediction cost, and the inter prediction cost includes:
[0042] Calculating a third motion vector of the current macroblock based on target motion vectors of a plurality of second macroblocks adjacent to the current macroblock;
[0043] Based on the difference between the first motion vector and the third motion vector, the intra prediction cost, and the inter prediction cost, it is determined whether the current macroblock is a skip block.
[0044] A skip block refers to a macroblock that does not need to be encoded during the encoding process.
[0045] Take the example that the second macroblock is the adjacent macroblock on the left and above the current macroblock. Specifically, the reference frame index and target motion vector of each second macroblock can be obtained. If there is a second macroblock whose reference frame index is invalid or whose target motion vector is a zero vector, the third motion vector of the current macroblock is a zero vector. Otherwise, each second macroblock is compared with the reference frame index of the current macroblock. If the reference frames of more than two second macroblocks are the same as the reference frame of the current macroblock, the third motion vector of the current macroblock is determined to be the average of the target motion vectors of the second macroblocks with the same reference frame. If only one second macroblock has the same reference frame as the reference frame of the current macroblock, the third motion vector of the current macroblock is determined to be the target motion vector of the second macroblock. If the reference frames of each second macroblock are different from the reference frame of the current macroblock, the third motion vector of the current macroblock is determined to be the average of the target motion vectors of the second macroblocks with the same reference frame.
[0046] Of course, in actual application scenarios, adjacent macroblocks in other directions may be used as second macroblocks, and the third motion vector may also be calculated in other ways, such as the third motion vector being the target motion vector of any second macroblock.
[0047] Through the embodiments of the present application, it is possible to accurately determine whether the current macroblock is a non-skipped block.
[0048] In one embodiment of the present application, determining whether the current macroblock is a skip block based on the difference between the first motion vector and the third motion vector, the intra prediction cost, and the inter prediction cost includes:
[0049] Determine whether an absolute value of a difference between a projection length of the first motion vector on the X-axis and a projection length of the third motion vector on the X-axis, and an absolute value of a difference between a projection length of the first motion vector on the Y-axis and a projection length of the third motion vector on the Y-axis are both less than a preset first credibility threshold;
[0050] Based on the determination result, the intra prediction cost and the inter prediction cost, it is determined whether the current macroblock is a skip block.
[0051] Specifically, if both are less than the preset first credibility threshold, it means that the first motion vector provided by the lookahead module is credible and can be used for subsequent motion estimation operations. In the case where both are less than the preset first credibility threshold, the embodiment of the present application can further determine whether the current macroblock is a skip block based on the intra-frame prediction cost and the inter-frame prediction cost. If the inter-frame prediction cost is less than the intra-frame prediction cost, the current macroblock is a skip block, otherwise, it is a non-skip block.
[0052] The first credibility threshold can be adjusted according to the actual application scenario. For example, in a video conference scenario, since the background of the picture is fixed and the characters do not move frequently, a higher first credibility threshold can be set for this scenario, thereby increasing the probability that the first motion vector participates in subsequent motion estimation, thereby reducing the amount of subsequent calculations and improving coding efficiency. In complex motion scenarios, a smaller first credibility threshold can be set to appropriately reduce the number of skipped blocks to ensure that subtle movements can also be captured and ensure picture quality.
[0053] In actual application scenarios, it is possible to determine whether only the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the third motion vector on the X-axis is less than a preset first credibility threshold, or to determine whether only the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the third motion vector on the Y-axis is less than a preset first credibility threshold, and it is not limited to one implementation method.
[0054] In one embodiment of the present application, determining whether the current macroblock is a skip block based on the determination result, the intra-frame prediction cost, and the inter-frame prediction cost includes:
[0055] In response to the absolute values of the differences being less than a first credibility threshold, calculating a prediction factor based on a ratio of an inter-frame prediction cost to an intra-frame prediction cost;
[0056] The prediction factor is compared with a preset type threshold, and whether the current macroblock is a skip block is determined based on the comparison result.
[0057] When both are less than the preset first credibility threshold, directly comparing the size of the intra-frame prediction cost and the inter-frame prediction cost, the result is relatively simple, and it is not possible to dynamically adjust the number of skipped blocks and non-skipped blocks according to the business scenario. Therefore, the embodiment of the present application calculates the prediction factor based on the inter-frame prediction cost and the intra-frame prediction cost, and identifies the skipped block by comparing the prediction factor with the type threshold. Similar to the first credibility threshold, the type threshold can be adjusted according to the actual application scenario.
[0058] The prediction factor may be a ratio of the inter-frame prediction cost to the intra-frame prediction cost, or may be as shown in formula (1).
[0059] S=(1-a / b)×1024 (1)
[0060] Where a is the inter-frame prediction cost, b is the intra-frame prediction cost. Taking formula (1) as an example, the type threshold is 500. If S is greater than 500, the current macroblock is a skip block, otherwise, the current macroblock is a non-skip block.
[0061] In one embodiment of the present application, comparing the prediction factor with a preset type threshold, and determining whether the current macroblock is a skip block based on the comparison result includes:
[0062] In response to the prediction factor being greater than the type threshold, determining the current macroblock to be a skip block;
[0063] In response to the prediction factor being not greater than the type threshold, motion compensation is performed on the current macroblock based on the third motion vector to generate a prediction block of the luminance plane and a prediction block of the chrominance plane; a luminance residual and a chrominance residual are calculated; discrete cosine transform is performed on the luminance residual and the chrominance residual, respectively, to obtain corresponding luminance discrete cosine transform coefficients and chrominance discrete cosine transform coefficients; luminance residual energy is calculated based on the luminance discrete cosine transform coefficient; chrominance residual energy is calculated based on the chrominance discrete cosine transform coefficient; based on the luminance residual energy and a preset luminance energy threshold, and based on the chrominance residual energy and a preset chrominance energy threshold, it is determined whether the current macroblock is a skip block; wherein the luminance residual is used to characterize the difference between the prediction block of the luminance plane and the current macroblock, and the chrominance residual is used to characterize the difference between the prediction block of the chrominance plane and the current macroblock.
[0064] Discrete cosine transform is a mathematical transform that converts macroblocks from the spatial domain to the frequency domain. It can concentrate most of the energy in the image into fewer frequency components, making subsequent compression more effective.
[0065] After obtaining the discrete cosine transform coefficients, these coefficients will be quantized. Quantization is to reduce the frequency domain data after discrete cosine transform according to the quantization parameter, erase the subtle high-frequency information, and further compress the data volume. The quantized result is used to calculate the residual energy. The luminance energy threshold and the chrominance energy threshold are pre-set values, which are used to determine whether the luminance residual energy and the chrominance residual energy are small enough. If the residual energy is lower than the corresponding threshold, it means that the prediction of the block is very close to the actual value and the encoding can be skipped; conversely, if the residual energy exceeds the corresponding threshold, the macroblock needs to be encoded.
[0066] The embodiment of the present application further confirms the type of the current macroblock through motion compensation, thereby improving the accuracy of macroblock classification and improving the encoding quality.
[0067] In actual application scenarios, only the luminance residual or the chrominance residual may be calculated, and the type of the current macroblock may be determined based on the luminance residual or the chrominance residual. Of course, when the prediction factor is not greater than the type threshold, the current macroblock may be directly determined to be a non-skipped block.
[0068] In one embodiment of the present application, determining whether the difference between the second motion vector and the first motion vector is within a preset range includes:
[0069] Determine whether the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the second motion vector on the X-axis, and the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the second motion vector on the Y-axis are both less than a preset second credibility threshold.
[0070] The second credibility threshold is similar to the first credibility threshold and can be adjusted according to the actual application scenario. The calculation method of the second motion vector is similar to the calculation method of the third motion vector. The difference from the third motion vector is that the second motion vector is generally calculated using adjacent macroblocks located to the left, above, and to the right of the current macroblock.
[0071] Take the example that the first macroblock is the adjacent macroblock to the left, above, and upper right of the current macroblock. Specifically, each first macroblock is compared with the reference frame index of the current macroblock. If the reference frames of more than two first macroblocks are the same as the reference frame of the current macroblock, the second motion vector of the current macroblock is determined to be the average of the target motion vectors of the first macroblocks with the same reference frame. If only one first macroblock has the same reference frame as the reference frame of the current macroblock, the second motion vector of the current macroblock is determined to be the target motion vector of the first macroblock. If the reference frames of each first macroblock are different from the reference frame of the current macroblock, the second motion vector of the current macroblock is determined to be the average of the target motion vectors of the first macroblocks with the same reference frame.
[0072] Of course, in actual application scenarios, adjacent macroblocks in other directions may be used as first macroblocks, and the second motion vector may also be calculated in other ways, such as the second motion vector being the target motion vector of any first macroblock.
[0073] In actual application scenarios, it is possible to determine whether only the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the second motion vector on the X-axis is less than a preset second credibility threshold, or to determine whether only the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the second motion vector on the Y-axis is less than a preset second credibility threshold, and it is not limited to one implementation method.
[0074] Through the embodiments of the present application, it is possible to accurately determine whether the first motion vector can be used for subsequent motion estimation, thereby improving the accuracy of motion estimation.
[0075] In one embodiment of the present application, determining a target motion vector of a current macroblock based on the first motion vector includes:
[0076] Determine the range of motion based on the position of the current macroblock and the size of the image where the current macroblock is located;
[0077] determining whether the first motion vector is within the motion range;
[0078] In response to the first motion vector being within the motion range, determining a plurality of candidate motion vectors based on the first motion vector; wherein the candidate motion vectors have the same starting point as the first motion vector and an end point that is a preset number of pixels apart;
[0079] Calculating the cost of the first motion vector and the costs of each candidate motion vector;
[0080] The target motion vector of the current macroblock is determined to be the first motion vector or the candidate motion vector having the minimum cost.
[0081] In actual application scenarios, the motion range may also be adjusted, such as by setting a buffer area, and the adjusted motion range is the original motion range minus the buffer area.
[0082] For example, the end point of the candidate motion vector is two pixels or three pixels away from the end point of the first motion vector. The cost of the motion vector includes the cost of the first motion vector and the cost of the candidate motion vector. The cost of the motion vector includes the distortion between the current macroblock and the reference block and the coding cost. The distortion can be calculated by the mean square error, the absolute error and the coding cost. The coding cost can be calculated by the number of required coding bits.
[0083] If the first motion vector has the minimum cost, the first motion vector is used as the target motion vector; if the candidate motion vector has the minimum cost, the candidate motion vector with the minimum cost is used as the target motion vector.
[0084] The embodiment of the present application can obtain the target motion vector based on the data provided by the forward prediction module, without calculating the motion vector, intra-frame prediction cost and inter-frame prediction cost of each macroblock, thereby improving subsequent encoding efficiency.
[0085] In one embodiment of the present application, determining a target motion vector of the current macroblock based on the second motion vector includes:
[0086] Determine a plurality of candidate motion vectors based on the second motion vector and a preset search algorithm;
[0087] Calculating the cost of the second motion vector and the costs of each candidate motion vector;
[0088] Determine the target motion vector of the current macroblock as the second motion vector or the candidate motion vector having the minimum cost;
[0089] The search algorithm includes any one of diamond search, hexagon search, asymmetric hexagon search and global search.
[0090] If the difference between the second motion vector and the first motion vector is large, it means that the error of the first motion vector is large and cannot be used for subsequent motion estimation. At this time, subsequent motion estimation will be performed based on the second motion vector to improve the encoding quality.
[0091] The cost of the second motion vector and the cost of the candidate motion vector in the embodiment of the present application are calculated in the same manner as the cost of the motion vector in the aforementioned embodiment, and will not be repeated here.
[0092] like Figure 2 As shown, the embodiment of the present application provides a motion estimation method, including:
[0093] Step 201: Obtain the first motion vector, intra-frame prediction cost and inter-frame prediction cost of the current macroblock from the forward prediction module.
[0094] Step 202: Calculate a third motion vector of the current macroblock based on target motion vectors of a plurality of second macroblocks adjacent to the current macroblock.
[0095] Step 203: In response to the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the third motion vector on the X-axis, and the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the third motion vector on the Y-axis being less than a preset first credibility threshold, a prediction factor is calculated based on a ratio of the inter-frame prediction cost to the intra-frame prediction cost. In response to the prediction factor being greater than the type threshold, step 204 is executed; otherwise, step 205 is executed.
[0096] Step 204: Determine whether the current macroblock is a skip block.
[0097] The skipped block does not participate in subsequent operations.
[0098] Step 205: Perform motion compensation on the current macroblock based on the third motion vector, generate a prediction block of the luminance plane and a prediction block of the chrominance plane, and calculate the luminance residual and the chrominance residual; perform discrete cosine transform on the luminance residual and the chrominance residual, respectively, to obtain corresponding luminance discrete cosine transform coefficients and chrominance discrete cosine transform coefficients, calculate the luminance residual energy based on the luminance discrete cosine transform coefficients, and calculate the chrominance residual energy based on the chrominance discrete cosine transform coefficients.
[0099] Step 206: Determine whether the current macroblock is a skip block based on the luminance residual energy and the preset luminance energy threshold, as well as the chrominance residual energy and the preset chrominance energy threshold; if so, execute step 204; otherwise, execute step 207.
[0100] Step 207: Calculate a second motion vector of the current macroblock based on target motion vectors of a plurality of first macroblocks adjacent to the current macroblock.
[0101] Step 208: In response to the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the second motion vector on the X-axis, and the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the second motion vector on the Y-axis being less than a preset second credibility threshold, based on the position of the current macroblock and the size of the image where the current macroblock is located, determine the motion range, in response to the first motion vector being within the motion range, determine multiple candidate motion vectors based on the first motion vector, calculate the cost of the first motion vector and the costs of each candidate motion vector, and determine that the target motion vector of the current macroblock is the first motion vector or the candidate motion vector with the minimum cost.
[0102] Step 209: In response to the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the second motion vector on the X-axis, and the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the second motion vector on the Y-axis being less than a preset second credibility threshold, based on the second motion vector and a preset search algorithm, multiple candidate motion vectors are determined, the cost of the second motion vector and the costs of each candidate motion vector are calculated, and the target motion vector of the current macroblock is determined to be the second motion vector or the candidate motion vector with the minimum cost.
[0103] The embodiments of the present application can effectively utilize the data provided by the forward prediction module and improve the efficiency of motion estimation.
[0104] like Figure 3 As shown, the present application provides a motion estimation device, comprising:
[0105] An acquisition module 301 is configured to acquire a first motion vector, an intra-frame prediction cost, and an inter-frame prediction cost of a current macroblock from a forward prediction module;
[0106] The calculation module 302 is configured to determine whether the current macroblock is a skip block based on the first motion vector, the intra prediction cost and the inter prediction cost; in response to the current macroblock not being a skip block, calculate a second motion vector of the current macroblock based on target motion vectors of a plurality of first macroblocks adjacent to the current macroblock;
[0107] The determination module 303 is configured to determine whether the difference between the second motion vector and the first motion vector is within a preset range; in response to the difference being within the preset range, determine the target motion vector of the current macroblock based on the first motion vector; in response to the difference not being within the preset range, determine the target motion vector of the current macroblock based on the second motion vector.
[0108] In one embodiment of the present application, the calculation module 302 is configured to calculate the third motion vector of the current macroblock based on the target motion vectors of multiple second macroblocks adjacent to the current macroblock; and determine whether the current macroblock is a skip block based on the difference between the first motion vector and the third motion vector, the intra-frame prediction cost, and the inter-frame prediction cost.
[0109] In one embodiment of the present application, the calculation module 302 is configured to determine whether the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the third motion vector on the X-axis, and the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the third motion vector on the Y-axis are both less than a preset first credibility threshold; in response to the absolute values of the differences being less than the first credibility threshold, calculate a prediction factor based on a ratio of an inter-frame prediction cost to an intra-frame prediction cost; compare the prediction factor with a preset type threshold, and determine whether the current macroblock is a skip block based on the comparison result.
[0110] In one embodiment of the present application, the calculation module 302 is configured to determine that the current macroblock is a skip block in response to the prediction factor being greater than the type threshold; in response to the prediction factor being not greater than the type threshold, perform motion compensation on the current macroblock based on the third motion vector to generate a prediction block of the luminance plane and a prediction block of the chrominance plane; calculate the luminance residual and the chrominance residual; perform discrete cosine transform on the luminance residual and the chrominance residual, respectively, to obtain corresponding luminance discrete cosine transform coefficients and chrominance discrete cosine transform coefficients; calculate the luminance residual energy based on the luminance discrete cosine transform coefficient; calculate the chrominance residual energy based on the chrominance discrete cosine transform coefficient; determine whether the current macroblock is a skip block based on the luminance residual energy and a preset luminance energy threshold, and based on the chrominance residual energy and a preset chrominance energy threshold; wherein the luminance residual is used to characterize the difference between the prediction block of the luminance plane and the current macroblock, and the chrominance residual is used to characterize the difference between the prediction block of the chrominance plane and the current macroblock.
[0111] In one embodiment of the present application, the determination module 303 is configured to determine whether the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the second motion vector on the X-axis, and the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the second motion vector on the Y-axis are both less than a preset second credibility threshold.
[0112] In one embodiment of the present application, the determination module 303 is configured to determine the motion range based on the position of the current macroblock and the size of the image where the current macroblock is located; determine whether the first motion vector is within the motion range; in response to the first motion vector being within the motion range, determine multiple candidate motion vectors based on the first motion vector; wherein the candidate motion vector has the same starting point as the first motion vector and an end point is a preset number of pixels apart; calculate the cost of the first motion vector and the cost of each candidate motion vector; and determine that the target motion vector of the current macroblock is the first motion vector or the candidate motion vector with the smallest cost.
[0113] In one embodiment of the present application, the determination module 303 is configured to determine multiple candidate motion vectors based on the second motion vector and a preset search algorithm; calculate the cost of the second motion vector and the cost of each candidate motion vector; determine that the target motion vector of the current macroblock is the second motion vector or the candidate motion vector with the minimum cost; wherein the search algorithm includes any one of a diamond search, a hexagonal search, an asymmetric hexagonal search, and a global search.
[0114] An embodiment of the present application provides an electronic device, including:
[0115] one or more processors;
[0116] a storage device for storing one or more programs,
[0117] When one or more programs are executed by one or more processors, the one or more processors implement the method of any of the above embodiments.
[0118] An embodiment of the present application provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, a method as described in any of the above embodiments is implemented.
[0119] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. Although several embodiments are listed and described for the wireless communication method and the wireless communication component, it should be noted that the method details involved in the description of the wireless communication component can also be incorporated into the various embodiments of the wireless communication method, and vice versa.
[0120] The elements in the claims are to be interpreted broadly based on the language employed in the claims and are not limited to the examples described in this specification or during the prosecution of the application, which examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, with the true scope and spirit being indicated by the claims and the full scope of their equivalents.
[0121] The order of each step in the present application is merely exemplary and not restrictive. Without affecting the implementation of the present application (without destroying the required logical relationship between the steps), the execution order of the steps can be adjusted, and the various embodiments obtained after the adjustment still fall within the scope of the present application.
[0122] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that does not require protection is necessary for any claim. On the contrary, the subject matter of the present invention may be less than all the features of a specific disclosed embodiment. Thus, the claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.
Claims
1. A motion estimation method, characterized in that: include: Obtaining the first motion vector, intra-frame prediction cost and inter-frame prediction cost of the current macroblock from the forward prediction module; Determining whether the current macroblock is a skip block based on the first motion vector, the intra prediction cost, and the inter prediction cost; In response to the current macroblock not being a skip block, calculating a second motion vector of the current macroblock based on target motion vectors of a plurality of first macroblocks adjacent to the current macroblock; determining whether a difference between the second motion vector and the first motion vector is within a preset range; In response to the difference being within a preset range, determining a target motion vector of the current macroblock based on the first motion vector; In response to the difference being not within a preset range, a target motion vector of the current macroblock is determined based on the second motion vector.
2. The method according to claim 1, characterized in that Determining whether the current macroblock is a skip block based on the first motion vector, the intra prediction cost, and the inter prediction cost includes: Calculating a third motion vector of the current macroblock based on target motion vectors of a plurality of second macroblocks adjacent to the current macroblock; Based on the difference between the first motion vector and the third motion vector, the intra prediction cost, and the inter prediction cost, it is determined whether the current macroblock is a skip block.
3. The method according to claim 2, characterized in that Determining whether the current macroblock is a skip block based on a difference between the first motion vector and the third motion vector, the intra prediction cost, and the inter prediction cost includes: Determine whether an absolute value of a difference between a projection length of the first motion vector on the X-axis and a projection length of the third motion vector on the X-axis, and an absolute value of a difference between a projection length of the first motion vector on the Y-axis and a projection length of the third motion vector on the Y-axis are both less than a preset first credibility threshold; In response to the absolute values of the differences being less than the first credibility threshold, calculating a prediction factor based on a ratio of the inter-frame prediction cost to the intra-frame prediction cost; The prediction factor is compared with a preset type threshold, and whether the current macroblock is a skip block is determined based on the comparison result.
4. The method according to claim 3, characterized in that Comparing the prediction factor with a preset type threshold, and determining whether the current macroblock is a skip block based on the comparison result, includes: In response to the prediction factor being greater than the type threshold, determining that the current macroblock is a skip block; In response to the prediction factor being not greater than the type threshold, motion compensation is performed on the current macroblock based on the third motion vector to generate a prediction block of a luminance plane and a prediction block of a chrominance plane; a luminance residual and a chrominance residual are calculated; discrete cosine transform is performed on the luminance residual and the chrominance residual, respectively, to obtain corresponding luminance discrete cosine transform coefficients and chrominance discrete cosine transform coefficients; luminance residual energy is calculated based on the luminance discrete cosine transform coefficient; chrominance residual energy is calculated based on the chrominance discrete cosine transform coefficient; based on the luminance residual energy and a preset luminance energy threshold, and based on the chrominance residual energy and a preset chrominance energy threshold, it is determined whether the current macroblock is a skip block; wherein the luminance residual is used to characterize the difference between the prediction block of the luminance plane and the current macroblock, and the chrominance residual is used to characterize the difference between the prediction block of the chrominance plane and the current macroblock.
5. The method according to claim 1, characterized in that Determining whether a difference between the second motion vector and the first motion vector is within a preset range includes: Determine whether the absolute value of the difference between the projection length of the first motion vector on the X-axis and the projection length of the second motion vector on the X-axis, and the absolute value of the difference between the projection length of the first motion vector on the Y-axis and the projection length of the second motion vector on the Y-axis are both less than a preset second credibility threshold.
6. The method according to claim 1, characterized in that Determining a target motion vector of the current macroblock based on the first motion vector includes: Determine a motion range based on a position of the current macroblock and a size of an image where the current macroblock is located; determining whether the first motion vector is within the motion range; In response to the first motion vector being within the motion range, determining a plurality of candidate motion vectors based on the first motion vector; wherein the candidate motion vectors have the same starting point as the first motion vector and have an end point that is a preset number of pixels apart; Calculating the cost of the first motion vector and the costs of each of the candidate motion vectors; The target motion vector of the current macroblock is determined to be a first motion vector or a candidate motion vector having a minimum cost.
7. The method according to claim 1, characterized in that Determining a target motion vector of the current macroblock based on the second motion vector includes: Determine a plurality of candidate motion vectors based on the second motion vector and a preset search algorithm; Calculating the cost of the second motion vector and the costs of each of the candidate motion vectors; Determine the target motion vector of the current macroblock as a second motion vector or a candidate motion vector having a minimum cost; The search algorithm includes any one of diamond search, hexagon search, asymmetric hexagon search and global search.
8. A motion estimation device, characterized in that: include: An acquisition module, configured to acquire a first motion vector, an intra-frame prediction cost, and an inter-frame prediction cost of a current macroblock from a forward prediction module; A calculation module, configured to determine whether the current macroblock is a skip block based on the first motion vector, the intra prediction cost and the inter prediction cost; In response to the current macroblock not being a skip block, calculating a second motion vector of the current macroblock based on target motion vectors of a plurality of first macroblocks adjacent to the current macroblock; a determination module, configured to determine whether a difference between the second motion vector and the first motion vector is within a preset range; In response to the difference being within a preset range, determining a target motion vector of the current macroblock based on the first motion vector; In response to the difference being not within a preset range, a target motion vector of the current macroblock is determined based on the second motion vector.
9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.