A low-cost multi-reference frame motion estimation method and device

Abstract

The application discloses a low-cost multi-reference frame motion estimation method. The to-be-encoded image block is respectively subjected to CIME with each reference frame to obtain the optimal CIMV of the to-be-encoded image block on each reference frame and the corresponding prediction cost; the minimum prediction cost is selected as the optimal CIMV of the to-be-encoded image block, and the reference frame corresponding to the optimal CIMV is selected as the optimal reference frame of the to-be-encoded image block. The to-be-encoded image block is subjected to RIME with the optimal reference frame with the optimal CIMV as the search center to obtain the optimal RIMV of the to-be-encoded image block. The to-be-encoded image block is subjected to FME with the optimal reference frame with the optimal RIMV as the search center to obtain the optimal FMV of the to-be-encoded image block. The inter-frame prediction mode of the to-be-encoded image block is selected in combination with the motion vectors of the spatially and temporally adjacent coded image blocks to obtain the optimal inter-frame prediction mode and the optimal motion vector of the to-be-encoded image block. The application greatly reduces the operation amount of motion estimation and is suitable for hardware design.

Description

Technical Field

[0001] This invention relates to a digital video coding technique, and more particularly to a low-cost, multi-reference frame motion estimation method suitable for hardware implementation. Background Technology

[0002] Video coding is a technique that compresses redundant components in video images and represents video information using as little data as possible. Common video coding standards include HEVC (High Efficiency Video Coding, also known as H.265) and AVC (Advanced Video Coding, also known as H.264).

[0003] Video coding technology uses image blocks as the most basic coding unit. For example, in HEVC, the basic unit of video coding is the CU (Coding Unit). A CU can be an image block of 64 pixels × 64 pixels, 32 pixels × 32 pixels, 16 pixels × 16 pixels, or 8 pixels × 8 pixels. The 64-pixel × 64-pixel image block is also called the LCU (Largest Coding Unit).

[0004] To maximize video compression, common video coding algorithms often employ lossy compression, resulting in differences between the reconstructed video and the original (distortion). HEVC and AVC are both lossy compression algorithms. The coding performance of lossy compression algorithms is jointly measured by the coding bit rate (Rate) and the distortion introduced by the coding. In video coding, the coding bit rate R and coding distortion D are mutually restrictive and contradictory; reducing the coding bit rate R often increases coding distortion D, and reducing coding distortion D often increases coding bit rate R. Rate-Distortion Optimization (RDO) aims to minimize coding distortion D at a given coding bit rate R, or minimize coding bit rate R at a given coding distortion D. Common video coding algorithms employ rate-distortion optimization methods based on Lagrange multipliers λ, which aim to achieve the minimum coding rate-distortion cost (RD Cost) J, where J = D + λ × R.

[0005] Motion estimation, in the context of already encoded video frames (called reference frames), searches for the optimal matching block for the image block to be encoded in the current video frame, minimizing rate-distortion costs. Motion estimation is one of the core technologies of video coding algorithms; its role is to eliminate temporal redundancy in the video signal, thereby improving coding efficiency. Please refer to [link to relevant documentation]. Figure 1 The relative offset between the optimal matching block in the reference frame and the image block to be encoded in the current encoded video frame at the same position in the reference frame is the optimal motion vector (MV) of the image block to be encoded. During motion estimation, the image block to be encoded in the current encoded video frame is matched one by one with all image blocks within the search window (i.e., the search range of motion estimation) in the reference frame (which may or may not include image blocks at the same position as the image block to be encoded, depending on the location of the search center and the search range) to find the optimal matching block.

[0006] With the development of video coding standards, video coding efficiency has increased significantly, but so has its complexity and computational load. This has made it difficult for purely software-based video encoders to meet real-time coding requirements. To improve video coding speed, using hardware such as Application-Specific Integrated Circuits (ASICs) to accelerate the video coding process has become a common industry practice. This type of hardware that accelerates the video coding process is called a hardware video encoder.

[0007] Among the various algorithm modules of a video encoder, the motion estimation module has a very high computational load, especially when there are multiple reference frames for the image block to be encoded. The computational burden of motion estimation becomes considerable, leading to a sharp increase in the area and power consumption of the hardware video encoder. Therefore, designing a multi-reference frame motion estimation method suitable for hardware implementation and minimizing the cost of motion estimation becomes a necessary requirement for hardware video encoders.

[0008] In video coding technology, the motion estimation process generally includes integer motion estimation (IME) and fractional motion estimation (FME). In existing hardware video encoders, to reduce the computational load of motion estimation, the integer motion estimation process is generally divided into coarse integer motion estimation (CIME) and refined integer motion estimation (RIME). Thus, in a hardware video encoder, the motion estimation process consists of three stages: CIME, RIME, and FME. CIME is a downsampling search with a preset point as the search center over a large area to obtain the optimal coarse pixel-precision motion vector (optimal CIMV) for the image block to be encoded. RIME is a local full search with the optimal coarse pixel-precision motion vector as the search center over a smaller area to obtain the optimal integer pixel-precision motion vector (optimal RIMV) for the image block to be encoded. FME is a subpixel search with the optimal integer pixel-precision motion vector as the search center over a smaller area to obtain the optimal subpixel-precision motion vector (optimal FMV) for the image block to be encoded. After obtaining the optimal FMV of the image block to be encoded, the inter-frame prediction mode of the image block to be encoded is selected by combining the motion vectors of neighboring encoded image blocks in the spatial and temporal domains, so as to obtain the optimal inter-frame prediction mode and optimal motion vector of the image block to be encoded.

[0009] This hardware video encoder, employing CIME, RIME, and FME three-stage motion estimation techniques, performs motion estimation on a single reference frame as follows: Figure 2 As shown, the process of motion estimation on two reference frames is as follows: Figure 3 As shown. Generally, hardware video encoders support motion estimation operations on a maximum of two reference frames, labeled reference frame 0 and reference frame 1. The optimal CIMV, optimal RIMV, and optimal FMV obtained on reference frame 0 are labeled as optimal CIMV_0, optimal RIMV_0, and optimal FMV_0, respectively. The optimal CIMV, optimal RIMV, and optimal FMV obtained on reference frame 1 are labeled as optimal CIMV_1, optimal RIMV_1, and optimal FMV_1, respectively. Comparison Figure 2 , Figure 3It can be observed that for this type of hardware video encoder, when the number of reference frames for the image block to be encoded increases from one to two, the computational load of the motion estimation process doubles because CIME, RIME, and FME motion estimation operations need to be performed on both reference frames. Simultaneously, due to the increased number of reference frames, the optimal FMV of the image block also increases accordingly, leading to a significant increase in the computational load of the inter-frame prediction mode selection process. Therefore, supporting motion estimation operations with multiple reference frames is extremely costly in traditional hardware video encoders. Summary of the Invention

[0010] The technical problem to be solved by this invention is: how to make the motion estimation method of multiple reference frames suitable for hardware implementation and reduce costs.

[0011] To address the aforementioned technical problems, this invention proposes a low-cost multi-reference frame motion estimation method, comprising the following steps: Step S1: Perform coarse integer-pixel motion estimation (CIME) on the image block to be encoded and each reference frame to obtain the optimal coarse pixel-precision motion vector (CIMV) and corresponding prediction cost of the image block to be encoded on each reference frame. Step S2: Compare the prediction costs corresponding to the optimal CIMV of the image block to be encoded on each reference frame, and select the one with the lowest prediction cost as the optimal CIMV of the image block to be encoded. The reference frame corresponding to the optimal CIMV of the image block to be encoded is selected as the optimal reference frame of the image block to be encoded. Step S3: Using the optimal CIMV as the search center, perform fine integer-pixel motion estimation (RIME) on the image block to be encoded and the optimal reference frame to obtain the optimal integer-pixel-precision motion vector (RIMV) of the image block to be encoded. Step S4: Using the optimal RIMV as the search center, perform sub-pixel motion estimation (FME) on the image block to be encoded and the optimal reference frame to obtain the optimal sub-pixel-precision motion vector (FMV) of the image block to be encoded. Step S5: After obtaining the optimal FMV of the image block to be encoded, the inter-frame prediction mode of the image block to be encoded is selected by combining the motion vectors of the spatially and temporally adjacent encoded image blocks, so as to obtain the optimal inter-frame prediction mode and the optimal motion vector of the image block to be encoded.

[0012] Further, in step S1, the image block to be encoded and each reference frame are first downsampled, and then the downsampled image block to be encoded is searched on each downsampled reference frame with a preset point as the search center; each downsampled pixel position corresponds to a CIMV; the prediction cost corresponding to each downsampled pixel position of the image block to be encoded in each reference frame is calculated, and the CIMV corresponding to the downsampled pixel position with the smallest prediction cost is selected as the optimal CIMV of the image block to be encoded on this reference frame.

[0013] Furthermore, after step S2 is completed, subsequent steps are based solely on the optimal CIMV and optimal reference frame of the image block to be encoded.

[0014] Further, in step S3, the search center is set to the optimal CIMV, the reference frame is set to the optimal reference frame, and each integer pixel position corresponds to an RIMV; the prediction cost corresponding to each integer pixel position of the image block to be encoded within the search window of the optimal reference frame is calculated, and the RIMV corresponding to the integer pixel position with the smallest prediction cost is selected as the optimal RIMV of the image block to be encoded.

[0015] Further, in step S4, the search center is set to the optimal RIMV, the reference frame is set to the optimal reference frame, and each sub-pixel position corresponds to an FMV; the prediction cost corresponding to each sub-pixel position of the image block to be encoded within the search window of the optimal reference frame is calculated, and the FMV corresponding to the sub-pixel position with the smallest prediction cost is selected as the optimal FMV of the image block to be encoded.

[0016] Furthermore, in steps S1, S3, and S4, the distortion cost is calculated using the absolute error and SAD algorithm or the absolute transform error and SATD algorithm, and the coding bit rate cost is calculated using the exponential Columbus algorithm.

[0017] Further, in step S5, the distortion cost is calculated using the SAD algorithm or the SATD algorithm, and the coding bit rate cost is calculated using the exponential Columbus algorithm. Alternatively, the distortion cost can be calculated using the squared error and SSD algorithm, and the coding bit rate cost can be calculated using the context-adaptive binary arithmetic coding (CABAC) algorithm or the context-adaptive variable-length coding (CAVLC) algorithm.

[0018] Further, in step S5, the rate-distortion cost corresponding to the optimal FMV of the image block to be encoded is calculated, and the rate-distortion cost of the motion vectors of the spatially and temporally neighboring encoded image blocks when applied to the current image block to be encoded is calculated. When the image block to be encoded is performing inter-frame prediction, each rate-distortion cost corresponds to a motion vector, and each inter-frame prediction mode corresponds to a motion vector. The motion vector corresponding to the smallest rate-distortion cost is selected as the optimal motion vector of the image block to be encoded, and the inter-frame prediction mode corresponding to the optimal motion vector is the optimal inter-frame prediction mode of the image block to be encoded.

[0019] This invention also proposes a low-cost multi-reference frame motion estimation device, comprising multiple CIME modules, a CIMV selection module, a RIME module, an FME module, and an inter-frame prediction mode selection module; wherein the number of CIME modules is equal to the number of reference frames, and each of the other modules has only one. The multiple CIME modules are used to perform coarse integer-pixel motion estimation (CIME) on the image block to be encoded and each reference frame, respectively, to obtain the optimal coarse-pixel-precision motion vector (CIMV) of the image block to be encoded on each reference frame and the corresponding prediction cost. The CIMV selection module is used to compare the prediction cost corresponding to the optimal CIMV of the image block to be encoded on each reference frame, selecting the one with the lowest prediction cost as the optimal CIMV of the image block to be encoded, and the reference frame corresponding to the optimal CIMV of the image block to be encoded is designated as the optimal reference frame of the image block to be encoded. The RIME module is used to perform fine integer-pixel motion estimation (RIME) on the image block to be encoded and the optimal reference frame, using the optimal CIMV as the search center, to obtain the optimal integer-pixel-precision motion vector (RIMV) of the image block to be encoded. The FME module is used to perform sub-pixel motion estimation (FME) between the image block to be encoded and the optimal reference frame, using the optimal RIMV as the search center, to obtain the optimal sub-pixel precision motion vector (FMV) of the image block to be encoded. The inter-frame prediction mode selection module is used to select the inter-frame prediction mode of the image block to be encoded by combining the motion vectors of spatially and temporally neighboring encoded image blocks, thereby obtaining the optimal inter-frame prediction mode and optimal motion vector of the image block to be encoded.

[0020] The technical effect achieved by this invention is that the optimal CIMV and corresponding optimal reference frame of the image block to be encoded are determined in advance, and subsequent calculations are performed only on the optimal CIMV and only on the optimal reference frame, which greatly reduces the amount of computation for motion estimation, is suitable for hardware design, and can significantly reduce the number and area of ​​logic operation units and storage units of hardware video encoders. Attached Figure Description

[0021] Figure 1 This is a simplified diagram of the motion estimation process.

[0022] Figure 2 This is a schematic diagram of the motion estimation process performed by a hardware video encoder on a single reference frame.

[0023] Figure 3 This is a schematic diagram of the motion estimation process performed by a hardware video encoder on two reference frames.

[0024] Figure 4 This is a flowchart illustrating the low-cost multi-reference frame motion estimation method proposed in this invention.

[0025] Figure 5This is a schematic diagram of the structure of the low-cost multi-reference frame motion estimation device proposed in this invention.

[0026] The following are the annotations in the figure: 1a and 1b are both CIME modules, 2 is the CIMV selection module, 3 is the RIME module, 4 is the FME module, and 5 is the inter-frame prediction mode selection module. Detailed Implementation

[0027] Please see Figure 4 The low-cost multi-reference frame motion estimation method proposed in this invention includes the following steps. The following example uses a two-reference-frame motion estimation method, but it is also applicable to motion estimation methods with a larger number of reference frames.

[0028] Step S1: Perform coarse integer pixel motion estimation (CIME) on the image block to be encoded and the two reference frames respectively to obtain the optimal coarse pixel precision motion vector (CIMV) of the image block to be encoded on each reference frame and the corresponding prediction cost.

[0029] In this step, the image block to be encoded and two reference frames are first downsampled. The downsampling stride in the horizontal and vertical directions is generally set to 2:1 or 4:1, and each downsampled pixel position corresponds to a CIMV. For example, if the downsampling stride in the horizontal and vertical directions is set to 4:1, then each pixel position after downsampling corresponds to a motion vector with 4-pixel precision, and the change step of the horizontal and vertical components of this CIMV is 4 pixels. If the downsampling stride in the horizontal and vertical directions is set to 2:1, then each pixel position after downsampling corresponds to a motion vector with 2-pixel precision, and the change step of the horizontal and vertical components of this CIMV is 2 pixels. Then, the downsampled image block to be encoded is searched over a large range on the two downsampled reference frames, with a preset point as the search center. The preset search center is generally set to the (0,0) point, or to the motion vector of a neighboring image block, or calculated from the motion vector of a neighboring image block. During CIME search operations, a cost calculation method with lower computational complexity and lower precision is generally used. The distortion cost (i.e., coding distortion D) is typically calculated using the SAD algorithm (Sum of Absolute Difference), and the coding bit rate cost (i.e., coding bit rate R) is typically calculated using a simplified bit rate estimation algorithm, such as the exponential Golomb algorithm. Then, the prediction cost corresponding to each downsampled pixel position within the search window of each reference frame—i.e., the CIME search range—is calculated using the rate-distortion cost calculation formula J = D + λ × R. The CIMV corresponding to the downsampled pixel position with the minimum prediction cost is selected as the optimal CIMV of the image block to be encoded in that reference frame. After this step, the optimal CIMV and its corresponding prediction cost for the image block to be encoded in each reference frame are obtained. For ease of description, CIMV_0 and CIMV_1 are used to represent the optimal CIMV of the image block to be encoded in reference frame 0 and reference frame 1, respectively.

[0030] Step S2: Compare the prediction costs corresponding to the two optimal CIMVs of the image block to be encoded on two reference frames, select the one with the smaller prediction cost as the optimal CIMV of the image block to be encoded, and use the reference frame corresponding to the optimal CIMV of the image block to be encoded as the optimal reference frame of the image block to be encoded.

[0031] For example, if the prediction cost corresponding to CIMV_0 is relatively low, then CIMV_0 is selected as the optimal CIMV for the image block to be encoded, and reference frame 0 is selected as the optimal reference frame. Subsequent motion estimation and inter-frame prediction mode selection operations are performed solely based on CIMV_0 and only on reference frame 0; CIMV_1 and reference frame 1 are no longer involved in subsequent operations. If the prediction costs corresponding to the two optimal CIMVs are exactly equal, then auxiliary information such as the selection of reference frames for surrounding encoded image blocks and the sampling time distance between the two reference frames and the current video frame to be encoded can be used to determine the optimal CIMV and optimal reference frame for the image block to be encoded.

[0032] This step determines the optimal CIMV and optimal reference frame for the image block to be encoded in advance. Subsequent RIME, FME, and inter-frame prediction mode selection processes are based solely on the optimal CIMV and performed only on the optimal reference frame. This makes the computational workload of RIME, FME, and inter-frame prediction mode selection processes basically the same as when only a single reference frame exists, significantly reducing the computational workload of motion estimation and inter-frame prediction mode selection processes when multiple reference frames exist.

[0033] Step S3: Using the optimal CIMV as the search center, perform detailed integer pixel motion estimation (RIME) on the image block to be encoded and the optimal reference frame to obtain the optimal integer pixel precision motion vector (RIMV) of the image block to be encoded.

[0034] In this step, the search center is set to the optimal CIMV, the reference frame is set to the optimal reference frame, and each integer pixel position corresponds to a RIMV. The step size of the horizontal and vertical components of the RIMV is 1 pixel. When performing RIME search operations, a cost calculation method with lower computational cost and lower precision is generally used. The distortion cost (i.e., coding distortion D) is generally calculated using the SAD algorithm, and the coding bit rate cost (i.e., coding bit rate R) is generally calculated using a simplified bit rate estimation algorithm, such as the exponential Golomb algorithm. Then, according to the rate-distortion cost calculation formula J = D + λ × R, the prediction cost corresponding to each integer pixel position of the image block to be encoded within the search window of the optimal reference frame—that is, the search range of RIME—is calculated, and the RIMV corresponding to the integer pixel position with the smallest prediction cost is selected as the optimal RIMV of the image block to be encoded.

[0035] Step S4: Using the optimal RIMV as the search center, perform subpixel motion estimation (FME) on the image block to be encoded and the optimal reference frame to obtain the optimal subpixel precision motion vector (FMV) of the image block to be encoded.

[0036] In this step, the search center is set to the optimal RIMV, the reference frame is set to the optimal reference frame, and each sub-pixel position corresponds to an FMV. The step size of the horizontal and vertical components of the FMV is in sub-pixel units. When performing the FME search operation, a cost calculation method with lower computational cost and lower precision is generally used. The distortion cost (i.e., coding distortion D) is generally calculated using the SATD algorithm (Sum of Absolute Transformed Difference), and the coding bit rate cost (i.e., coding bit rate R) is generally calculated using a simplified bit rate estimation algorithm, such as the Exponential Golomb algorithm. Then, according to the rate-distortion cost calculation formula J = D + λ × R, the prediction cost corresponding to each sub-pixel position of the image block to be encoded within the search window of the optimal reference frame—that is, the search range of the FME—is calculated, and the FMV corresponding to the sub-pixel position with the minimum prediction cost is selected as the optimal FMV of the image block to be encoded.

[0037] Step S5: After obtaining the optimal FMV of the image block to be encoded, the inter-frame prediction mode of the image block to be encoded is selected by combining the motion vectors of the spatially and temporally adjacent encoded image blocks, so as to obtain the optimal inter-frame prediction mode and the optimal motion vector of the image block to be encoded.

[0038] In this step, the rate-distortion cost corresponding to the optimal FMV of the image block to be encoded, as well as the rate-distortion cost when the motion vectors of spatially and temporally neighboring encoded image blocks are applied to the current image block to be encoded, are calculated. During inter-frame prediction of the image block to be encoded, each motion vector has a corresponding rate-distortion cost. For each motion vector, a matching block is corresponding to it on the reference frame. Rate-distortion costs can be calculated between the image block to be encoded and each matching block, and there is a correspondence between this rate-distortion cost and the motion vector corresponding to that matching block. During inter-frame coding, each inter-frame prediction mode essentially corresponds to a motion vector. The motion vector corresponding to the minimum rate-distortion cost is selected as the optimal motion vector for the image block to be encoded, and the inter-frame prediction mode corresponding to the optimal motion vector is the optimal inter-frame prediction mode for the image block to be encoded.

[0039] In this step, the rate-distortion cost can be calculated using either a less computationally intensive and less precise method, such as the SAD or SATD algorithm, or the exponential Golomb algorithm, to calculate the distortion cost; or a more computationally intensive and more precise method, such as the SSD algorithm (Sum of Squared Differences), the CABAC algorithm (Context Adaptive Binary Arithmatic Coding), or the CAVLC algorithm (Context Adaptive Variable Length Coding), and transformation and quantization operations are introduced during the calculation process to make the selection of inter-frame prediction modes more accurate.

[0040] Please see Figure 5 The low-cost multi-reference frame motion estimation device proposed in this invention includes two CIME modules 1a and 1b, a CIMV selection module 2, a RIME module 3, an FME module 4, and an inter-frame prediction mode selection module 5. There are two CIME modules, and only one of each of the other modules. Figure 5 The device shown corresponds to Figure 4 The method shown.

[0041] The two CIME modules 1a and 1b are used to perform coarse integer pixel motion estimation (CIME) on the image block to be encoded and the two reference frames respectively, so as to obtain the optimal coarse pixel precision motion vector (CIMV) of the image block to be encoded on each reference frame and the corresponding prediction cost.

[0042] The CIMV selection module 2 is used to compare the prediction costs corresponding to the two optimal CIMVs of the image block to be encoded on two reference frames, and select the one with the smaller prediction cost as the optimal CIMV of the image block to be encoded, and the reference frame corresponding to the optimal CIMV of the image block to be encoded is used as the optimal reference frame of the image block to be encoded.

[0043] The RIME module 3 is used to perform detailed integer-pixel motion estimation (RIME) on the image block to be encoded and the optimal reference frame, with the optimal CIMV as the search center, to obtain the optimal integer-pixel precision motion vector (RIMV) of the image block to be encoded.

[0044] The FME module 4 is used to perform subpixel motion estimation (FME) on the image block to be encoded and the optimal reference frame with the optimal RIMV as the search center, so as to obtain the optimal subpixel precision motion vector (FMV) of the image block to be encoded.

[0045] The inter-frame prediction mode selection module 5 is used to select the inter-frame prediction mode of the image block to be encoded by combining the motion vectors of the spatially and temporally adjacent encoded image blocks, so as to obtain the optimal inter-frame prediction mode and the optimal motion vector of the image block to be encoded.

[0046] This invention proposes a low-cost multi-reference frame motion estimation method. By comparing the CIME prediction costs of the image block to be encoded across multiple reference frames, the optimal CIMV and corresponding optimal reference frame for the image block to be encoded are determined in advance. This ensures that subsequent RIME, FME, and inter-frame prediction mode selection processes are performed only based on the optimal CIMV and on the optimal reference frame, significantly reducing the computational load of motion estimation and inter-frame prediction mode selection. Therefore, this invention is suitable for implementation using hardware video encoders, effectively reducing the area of ​​the logic operation units and storage units of the hardware video encoder, significantly reducing the computational load and bandwidth consumption of motion estimation and inter-frame prediction mode selection processes, and substantially lowering the cost of the hardware video encoder.

[0047] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A low cost multi-reference frame motion estimation method, characterized in that, The method comprises the following steps: Step S1: performing coarse integer pixel motion estimation (CIME) on the to-be-encoded image block and each reference frame respectively to obtain an optimal coarse pixel precision motion vector (CIMV) of the to-be-encoded image block on each reference frame and a corresponding prediction cost; the CIME refers to performing down-sampling on the to-be-encoded image block and each reference frame, and then performing search operation on the down-sampled to-be-encoded image block on the down-sampled each reference frame with a preset point as a search center respectively; Step S2: comparing the prediction costs corresponding to the optimal CIMV of the to-be-encoded image block on each reference frame, and selecting the one with the minimum prediction cost as the optimal CIMV of the to-be-encoded image block, and the reference frame corresponding to the optimal CIMV of the to-be-encoded image block as the optimal reference frame of the to-be-encoded image block; subsequent steps are only based on the optimal CIMV and the optimal reference frame of the to-be-encoded image block; Step S3: performing refined integer pixel motion estimation (RIME) on the to-be-encoded image block and the optimal reference frame with the optimal CIMV as a search center to obtain an optimal integer pixel precision motion vector (RIMV) of the to-be-encoded image block; Step S4: performing fractional motion estimation (FME) on the to-be-encoded image block and the optimal reference frame with the optimal RIMV as a search center to obtain an optimal fractional pixel precision motion vector (FMV) of the to-be-encoded image block; Step S5: after obtaining the optimal FMV of the to-be-encoded image block, selecting an inter-frame prediction mode of the to-be-encoded image block in combination with motion vectors of spatially and temporally adjacent encoded image blocks to obtain an optimal inter-frame prediction mode and an optimal motion vector of the to-be-encoded image block.

2. The low cost multi-reference frame motion estimation method of claim 1, wherein, In the step S1, each down-sampled pixel position corresponds to a CIMV; the prediction cost corresponding to each down-sampled pixel position in a search window of each reference frame of the to-be-encoded image block is calculated, and the CIMV corresponding to the down-sampled pixel position with the minimum prediction cost is selected as the optimal CIMV of the to-be-encoded image block on the reference frame.

3. The low cost multi-reference frame motion estimation method of claim 1, wherein, In the step S3, the search center is set as the optimal CIMV, the reference frame is set as the optimal reference frame, and each integer pixel position corresponds to a RIMV; the prediction cost corresponding to each integer pixel position in a search window of the optimal reference frame of the to-be-encoded image block is calculated, and the RIMV corresponding to the integer pixel position with the minimum prediction cost is selected as the optimal RIMV of the to-be-encoded image block.

4. The method of claim 1, wherein, In the step S4, the search center is set as the optimal RIMV, the reference frame is set as the optimal reference frame, and each fractional pixel position corresponds to a FMV; the prediction cost corresponding to each fractional pixel position in a search window of the optimal reference frame of the to-be-encoded image block is calculated, and the FMV corresponding to the fractional pixel position with the minimum prediction cost is selected as the optimal FMV of the to-be-encoded image block. In the steps S1, S3 and S4, the distortion cost is calculated by using a sum of absolute differences (SAD) algorithm or a sum of absolute transformed differences (SATD) algorithm, and the encoding bit rate cost is calculated by using an exponential Golomb algorithm.

5. The method of claim 1, wherein the method further comprises: In the step S5, the distortion cost is calculated by using the SAD algorithm or the SATD algorithm, and the encoding bit rate cost is calculated by using the exponential Golomb algorithm.

6. The low cost multi-reference frame motion estimation method of claim 5, wherein, ​ Or alternatively, the distortion cost is calculated by using the sum of squared error and SSD algorithm, and the encoding bit rate cost is calculated by using the context adaptive binary arithmetic coding (CABAC) algorithm or the context adaptive variable length coding (CAVLC) algorithm.

7. The method of claim 1, wherein, In the step S5, the rate-distortion cost corresponding to the optimal FMV of the to-be-encoded image block is calculated, and the rate-distortion cost of the motion vector of the spatially and temporally adjacent encoded image block applied to the current to-be-encoded image block is calculated. When the to-be-encoded image block is subjected to inter-frame prediction, each rate-distortion cost corresponds to a motion vector, and each inter-frame prediction mode corresponds to a motion vector; the motion vector corresponding to the smallest rate-distortion cost is selected as the optimal motion vector of the to-be-encoded image block, and the inter-frame prediction mode corresponding to the optimal motion vector is the optimal inter-frame prediction mode of the to-be-encoded image block.

8. A low-cost multi-reference frame motion estimation apparatus, characterized by, The method comprises a plurality of CIME modules, a CIMV selection module, an RIME module, an FME module, and an inter-frame prediction mode selection module; the number of the CIME modules is equal to the number of reference frames, and the other modules each have only one. The plurality of CIME modules are respectively used to perform coarse integer pixel motion estimation (CIME) on the to-be-encoded image block and each reference frame to obtain the optimal coarse pixel precision motion vector (CIMV) of the to-be-encoded image block on each reference frame and the corresponding prediction cost; the CIME refers to performing down-sampling on the to-be-encoded image block and each reference frame, and then performing search operation on each down-sampled reference frame with the preset point as the search center. The CIMV selection module is used to compare the prediction costs corresponding to the optimal CIMV of the to-be-encoded image block on each reference frame, select the one with the smallest prediction cost as the optimal CIMV of the to-be-encoded image block, and select the reference frame corresponding to the optimal CIMV of the to-be-encoded image block as the optimal reference frame of the to-be-encoded image block; the subsequent modules are only based on the optimal CIMV and the optimal reference frame of the to-be-encoded image block. The RIME module is used to perform detailed integer pixel motion estimation (RIME) on the to-be-encoded image block and the optimal reference frame with the optimal CIMV as the search center to obtain the optimal integer pixel precision motion vector (RIMV) of the to-be-encoded image block. The FME module is used to perform sub-pixel motion estimation (FME) on the to-be-encoded image block and the optimal reference frame with the optimal RIMV as the search center to obtain the optimal sub-pixel precision motion vector (FMV) of the to-be-encoded image block. The inter-frame prediction mode selection module is used to select the inter-frame prediction mode of the to-be-encoded image block in combination with the motion vector of the spatially and temporally adjacent encoded image block to obtain the optimal inter-frame prediction mode and the optimal motion vector of the to-be-encoded image block.

Images