Video encoding method, video encoding system, and video encoder

Abstract

The application discloses a video coding method, a video coding system and a video coder. The video coding method comprises the following steps: using two motion estimation modules arranged in the video coder to perform double forward inter-frame prediction on P frames in a video bitstream; using the two motion estimation modules arranged in the video coder to perform forward inter-frame prediction, backward inter-frame prediction and double-direction inter-frame prediction on B frames in the video bitstream respectively. The video coding system comprises a first inter-frame prediction control module and a second inter-frame prediction control module. The video coder comprises a hardware driver, a hardware coder, a memory and the video coding system. The technical scheme of the application can reuse the same motion estimation module, fully utilize hardware resources to improve coding efficiency, and be applicable to all video coding standards.

Description

Technical Field

[0001] This invention relates to the field of video encoding technology, and in particular to a video encoding method, a video encoding system, and a video encoder. Background Technology

[0002] For frames with bidirectional prediction, we call them B-Frames (bi-directional interpolated prediction frames). A B-Frame has two reference frames: a forward reference frame and a backward reference frame. The block to be encoded in the current frame selects an optimal matching block from both the forward and backward reference frames. The two optimal matching blocks are then weighted and averaged to obtain the final matching block. The specific reference relationships are as follows: Figure 1 As shown, F n-1 For F n Forward reference frame, F n+1 For F n The backward reference frame.

[0003] Forward-predicted frames are called P-frames (predictive-frames, also known as forward-predictive coded frames). P-frames can only be used for forward reference; there can be only one or multiple forward reference frames. For example, if the current P-frame has two forward reference frames, the block to be coded will find an optimal matching block in each of the two forward reference frames, and then further compare the two optimal matching blocks to select the final optimal matching block. The specific reference relationship is as follows: Figure 2 As shown, F n-2 For F n The first forward reference frame, F n-1 For F n The first forward reference frame.

[0004] Table 1 below analyzes the impact of B-Frames on coding efficiency using the open-source encoder x265 (an open-source free software and library for encoding videos conforming to the High Efficiency Video Coding (HEVC / H.265) standard). The number of B-Frames indicates how many B-Frames are inserted between two P-Frames. Regardless of the number of B-Frames inserted, the improvement in coding efficiency is greater than 13%, demonstrating that supporting B-Frames has a significant impact on improving coding efficiency.

[0005] Table 1: Relationship between B-Frame Quantity and Encoding Efficiency Improvement To support B-Frames, there are two implementation schemes: Scheme 1: The motion estimation module of the P-Frame is reused in both reference directions of the B-Frame. Scheme 2: An additional motion estimation module is added. The forward reference frame of the B-Frame reuses the same motion estimation module as the P-Frame, while the backward reference frame of the B-Frame uses a separate motion estimation module.

[0006] Using the encoder in Scheme 1 results in a very slow encoding speed. Although it does not require additional hardware resources, its performance is too poor to be practical.

[0007] The encoder using Scheme 2 does not affect the encoding speed. Although it requires additional hardware resources, the increase in hardware resources is within an acceptable range considering the significant improvement in encoding efficiency brought about by B-Frame. When encoding P-Frame, if only single forward prediction is supported, the newly added motion estimation module will be completely idle, wasting hardware resources.

[0008] As can be seen from the encoders of the two schemes above, if the encoder does not support B-Frame, although it saves area, it has a significant impact on encoding efficiency. If the encoder supports B-Frame, an additional motion estimation module must be added. Then, when encoding P-Frame, the additional motion estimation module will be completely idle, wasting hardware resources. Summary of the Invention

[0009] One of the objectives of this invention is to overcome the shortcomings of the prior art. In view of the problem that adding a motion estimation module to the encoder in the prior art will greatly increase hardware resources and affect coding efficiency, this invention provides a video coding method, a video coding system and a video encoder.

[0010] To achieve the above objectives, the present invention is implemented through the following technical solution: In a first aspect, the present invention provides a video encoding method, the video encoding method comprising: Two motion estimation modules set in the video encoder are used to perform bi-forward inter-frame prediction on P-frames in the video bitstream. Two motion estimation modules set in the video encoder are used to perform forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction on the B-frames in the video bitstream, respectively.

[0011] In a preferred embodiment of this application, the dual forward inter-frame prediction performed on the P-frame specifically includes: The first motion estimation module of the two motion estimation modules is used to perform a first forward prediction of the P-frame for the first reference frame, and the second motion estimation module of the two motion estimation modules is used to perform a second forward prediction of the P-frame for the second reference frame. In the video stream, both the first reference frame and the second reference frame are located before the P-frame.

[0012] In a preferred embodiment of this application, the forward inter-frame prediction, the backward inter-frame prediction, and the bidirectional inter-frame prediction performed on the B-frame specifically include: The first motion estimation module of the two motion estimation modules is used to perform forward B-frame prediction of the forward reference frame for the B-frame; The second motion estimation module of the two motion estimation modules is used to perform B-frame backward prediction of the backward reference frame for the B-frame; The first motion estimation module is used to perform forward B-frame prediction of the forward reference frame for the B-frame, and the second motion estimation module is used to perform backward B-frame prediction of the backward reference frame for the B-frame. In the video stream, the forward reference frame is located before the B-frame, and the backward reference frame is located after the B-frame.

[0013] In a preferred embodiment of this application, before performing inter-frame prediction on the P-frame and the B-frame, the method further includes: Determine the type of frames in a video stream; If the type is a P-frame, then the dual forward inter-frame prediction and intra-frame prediction are performed on the P-frame. If the type is a B-frame, then the forward inter-frame prediction, the backward inter-frame prediction, the bidirectional inter-frame prediction, and the intra-frame prediction are performed on the B-frame. If the type is I-frame, then intra-frame prediction is performed on the I-frame.

[0014] In a preferred embodiment of this application, after performing the intra-frame prediction on the P-frame, B-frame, and I-frame, the method further includes: Based on the type of the current frame, perform mode determination on the P frame, the B frame, and the I frame respectively; For the P-frame, the results of the dual forward inter-frame prediction and the intra-frame prediction are compared to obtain the corresponding first optimal mode result; For the B-frame, the results of the forward inter-frame prediction, the backward inter-frame prediction, the bidirectional inter-frame prediction, and the intra-frame prediction are compared to obtain the corresponding second optimal mode result; For the I-frame, the result of the intra-frame prediction is taken as the third optimal mode result; Based on the type of the current frame, the first optimal mode result, the second optimal mode result, or the third optimal mode result is encoded into the video stream.

[0015] In a preferred embodiment of this application, the mode determination of the P-frame specifically includes: By using the first motion estimation module in the two motion estimation modules to perform the first forward prediction of the P-frame for the first reference frame, the first absolute error and value of the P-frame corresponding to the first forward prediction of the P-frame are obtained. By using the second motion estimation module in the two motion estimation modules to perform a second forward prediction of the P-frame for the second reference frame, the second absolute error and value of the P-frame corresponding to the second forward prediction of the P-frame are obtained. By performing intra-frame prediction on the P-frame, the corresponding P-frame third absolute error sum value is obtained; By comparing the sum of the first absolute errors of the P-frame, the sum of the second absolute errors of the P-frame, and the sum of the third absolute errors of the P-frame, the smallest value is taken as the first optimal mode result corresponding to the P-frame.

[0016] In a preferred embodiment of this application, the mode determination of the B-frame specifically includes: By using the first motion estimation module among the two motion estimation modules to perform forward prediction of the B-frame for the forward reference frame, the first absolute error and value of the B-frame corresponding to the forward prediction of the B-frame are obtained. By using the second motion estimation module among the two motion estimation modules to perform backward prediction of the B-frame from the backward reference frame, the second absolute error and value of the B-frame corresponding to the backward prediction of the B-frame are obtained. By using the first motion estimation module to perform forward prediction of the B-frame for the forward reference frame, and simultaneously using the second motion estimation module to perform backward prediction of the B-frame for the backward reference frame, the third absolute error and value of the B-frame corresponding to the bidirectional prediction of the B-frame are obtained. By performing intra-frame prediction on the B-frame, the corresponding fourth absolute error and value of the B-frame are obtained; By comparing the sum of the first, second, third, and fourth absolute errors of the B-frame, the smallest value is taken as the second optimal mode result corresponding to the B-frame.

[0017] In a second aspect, the present invention provides a video coding system, the video coding system comprising a first inter-frame prediction control module and a second inter-frame prediction control module; The first inter-frame prediction control module is used to perform bi-forward inter-frame prediction on P-frames in the video bitstream using two motion estimation modules located in the video encoder; the bi-forward inter-frame prediction on the P-frames specifically involves: The first motion estimation module of the two motion estimation modules is used to perform a first forward prediction of the P-frame for the first reference frame, and the second motion estimation module of the two motion estimation modules is used to perform a second forward prediction of the P-frame for the second reference frame. In the video stream, both the first reference frame and the second reference frame are located before the P frame; The second inter-frame prediction control module is used to perform forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction on B-frames in the video bitstream using two motion estimation modules located in the video encoder; the forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction performed on the B-frames are specifically as follows: The first motion estimation module of the two motion estimation modules is used to perform forward B-frame prediction of the forward reference frame for the B-frame; The second motion estimation module of the two motion estimation modules is used to perform B-frame backward prediction of the backward reference frame for the B-frame; The first motion estimation module is used to perform forward B-frame prediction of the forward reference frame for the B-frame, and the second motion estimation module is used to perform backward B-frame prediction of the backward reference frame for the B-frame. In the video stream, the forward reference frame is located before the B-frame, and the backward reference frame is located after the B-frame.

[0018] Thirdly, the present invention provides a video encoder, including a hardware driver, a hardware encoder, a memory, and a video encoding system as described in the second aspect, wherein the hardware encoder is connected to the hardware driver, the memory, and the video encoding system respectively. The hardware driver is used to transmit the video bitstream, the first and second reference frames corresponding to P frames in the video bitstream, and the forward and backward reference frames corresponding to B frames in the video bitstream to the hardware encoder. The hardware encoder includes two motion estimation modules; The video coding system includes a first inter-frame prediction control module and a second inter-frame prediction control module. The first inter-frame prediction control module is used to perform bi-forward inter-frame prediction on P-frames in the video bitstream using the two motion estimation modules. The second inter-frame prediction control module is used to perform forward inter-frame prediction, backward inter-frame prediction and bi-directional inter-frame prediction on B-frames in the video bitstream using the two motion estimation modules. The memory is used to store the reconstructed frames of the P frames and B frames in the video stream.

[0019] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the video encoding method as described in the first aspect.

[0020] Fifthly, the present invention provides a computer program product comprising a computer program that, when run on a computer, causes the computer to perform the video encoding method as described in the first aspect.

[0021] The video coding method, video coding system, and video encoder disclosed in this invention enable the second reference frame of a P-frame and the backward reference frame of a B-frame to reuse the same motion estimation module, making full use of hardware resources to improve coding efficiency, and are applicable to all video coding standards. Attached Figure Description

[0022] The present invention is described with reference to the following figures: Figure 1 This is a schematic diagram of a B-frame reference frame in the prior art; Figure 2 This is a schematic diagram of a P-frame reference frame in the prior art; Figure 3 This is a schematic diagram of the video coding method in Embodiment 1 of the present invention, which uses hardware resources to perform bidirectional prediction of B-frames and bidirectional forward prediction of P-frames. Figure 4 This is a basic flowchart of the video encoding method in Embodiment 1 of the present invention; Figure 5 This is an overall flowchart of the video encoding method in Embodiment 1 of the present invention; Figure 6 This is a flowchart illustrating the specific process of pattern determination for P-frames in the video encoding method of Embodiment 1 of the present invention. Figure 7 This is a flowchart illustrating the specific process of pattern determination for B-frames in the video coding method of Embodiment 1 of the present invention. Figure 8 This is a schematic diagram of the video encoding system in Embodiment 2 of the present invention; Figure 9 This is a schematic diagram of the internal structure of the video encoder in Embodiment 3 of the present invention; Figure 10 This is a schematic diagram of the motion compensation and transform quantization unit in the video encoder of Embodiment 3 of the present invention. Figure 11 This is a simplified schematic diagram of the video encoder performing encoding work in Embodiment 3 of the present invention. Detailed Implementation

[0023] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0024] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0025] Example 1 Embodiment 1 of the present invention discloses a video coding method that simultaneously achieves dual forward prediction and bidirectional prediction by reusing hardware resources.

[0026] The video coding method of this embodiment 1 includes: using two motion estimation modules set in the video encoder to perform dual forward inter-frame prediction on P frames in the video bitstream; and using two motion estimation modules set in the video encoder to perform forward inter-frame prediction, backward inter-frame prediction and bidirectional inter-frame prediction on B frames in the video bitstream respectively.

[0027] Specifically, if only one motion estimation module is set in the video encoder, this module can only search for one reference block within a certain period of time, meaning it can only search for reference blocks serially, which will significantly reduce the encoding speed. Adding multiple motion estimation modules to the video encoder can indeed support multiple forward prediction or bidirectional prediction, finding multiple reference blocks for the block to be encoded during motion estimation. However, since the motion estimation module is the most time-consuming and resource-intensive module in the entire encoder, adding multiple motion estimation modules will significantly increase hardware resources. Therefore, after weighing factors such as encoding quality, encoding speed, and hardware resources, in this embodiment 1, only one additional motion estimation module is added to the video encoder, such as... Figure 3 As shown, by reusing two motion estimation modules, dual forward prediction and bidirectional prediction of the current frame are achieved. This allows the video coding method of Embodiment 1 to be applicable to many video format coding standards, such as H.264 and HEVC, all of which support multiple forward prediction and bidirectional prediction of the current frame. Multiple forward prediction means that the current frame can have multiple forward reference frames, and during motion estimation, a reference block is searched in each forward reference frame. Bidirectional prediction means that the current frame can have one forward reference frame and one backward reference frame, and during motion estimation, each block to be encoded in the current frame can have one forward reference block and one backward reference block.

[0028] In video streams, frame types include I-frames (Intraframe Frames), P-frames, and B-frames. Since I-frames do not require inter-frame prediction, inter-frame prediction is only performed on P-frames and B-frames. See also Figure 3 When performing dual forward inter-frame prediction on P-frames and bidirectional inter-frame prediction on B-frames, the two motion estimation modules operate simultaneously. One inter-frame prediction in the dual forward inter-frame prediction of P-frames can reuse one motion estimation module with the forward inter-frame prediction of B-frames. Similarly, the other inter-frame prediction in the dual forward inter-frame prediction of P-frames can reuse another motion estimation module with the backward inter-frame prediction of B-frames. By reusing the motion estimation modules as described above to encode the current frame of the video stream, hardware resources are fully utilized, and encoding efficiency is improved.

[0029] In the video encoding method of this embodiment 1, the two motion estimation modules in the video encoder are the first motion estimation module ME0 and the second motion estimation module ME1.

[0030] Therefore Figure 3 In this embodiment, the video coding method uses a first motion estimation module ME0 and a second motion estimation module ME1 for dual forward inter-frame prediction of P-frames. Specifically, the first motion estimation module ME0 performs a first forward prediction of P-frames based on the first reference frame, and the second motion estimation module ME1 performs a second forward prediction of P-frames based on the second reference frame. In the video stream, both the first and second reference frames precede the P-frames. The order between the first and second reference frames is not strictly defined and can be flexibly configured according to actual conditions.

[0031] Similarly, in Figure 3 In this embodiment, the video coding method uses a first motion estimation module ME0 and a second motion estimation module ME1 to perform forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction on B-frames. Specifically, this means: using the first motion estimation module ME0 to perform forward prediction of B-frames based on forward reference frames; using the second motion estimation module ME1 to perform backward prediction of B-frames based on backward reference frames; and using the first motion estimation module ME0 to perform forward prediction of B-frames based on forward reference frames, while simultaneously using the second motion estimation module ME1 to perform backward prediction of B-frames based on backward reference frames. In the video stream, the forward reference frame precedes the B-frame, and the backward reference frame follows the B-frame.

[0032] Depend on Figure 3It can be seen that the first forward prediction of P-frame (with the first reference frame) and the forward prediction of B-frame (with the forward reference frame) reuse the first motion estimation module ME0, while the second forward prediction of P-frame (with the second reference frame) and the backward prediction of B-frame (with the backward reference frame) reuse the second motion estimation module ME1. Here, "first" and "second" are only used to facilitate the explanation of the video coding method of Embodiment 1, and do not indicate that the structures and working principles of the two motion estimation modules are different. When reusing the first motion estimation module ME0 and the second motion estimation module ME1 to perform inter-frame prediction for P-frames and B-frames, for P-frames, there are a first reference frame and a second reference frame preceding the current frame. The first motion estimation module ME0 selects an optimal matching block for the block to be encoded in the first reference frame, and the second motion estimation module ME1 selects another optimal matching block for the block to be encoded in the second reference frame. Then, the two optimal matching blocks are weighted and averaged to obtain the final matching block of the P-frame. For B-frames, there are a forward reference frame and a backward reference frame preceding and following the current frame, respectively. The first motion estimation module ME0 selects an optimal matching block for the block to be encoded in the forward reference frame, and the second motion estimation module ME1 selects another optimal matching block for the block to be encoded in the backward reference frame. Then, the two optimal matching blocks are weighted and averaged to obtain the final matching block of the B-frame. Throughout the inter-frame prediction process, both the first motion estimation module ME0 and the second motion estimation module ME1 can simultaneously perform inter-frame prediction for P-frames and B-frames. Compared to the scheme of setting only one motion estimation module in the video encoder and the scheme of setting multiple motion estimation modules but not reusing the added motion estimation modules during encoding, the video encoding method of this embodiment 1 ensures that the two motion estimation modules are always reused and in working state during encoding, without any idle state. This fully utilizes the hardware resources in the video encoder and also improves encoding efficiency.

[0033] In the video coding method of this embodiment 1, it is necessary to determine the type of the current frame in the video bitstream before performing inter-frame prediction on P-frames and B-frames.

[0034] See Figure 4 The video encoding method of this embodiment 1 includes: S100: Determine the type of the current frame in the video stream; if the current frame is a P-frame, execute S200 and S400 in sequence; if the current frame is a B-frame, execute S300 and S400 in sequence; if the current frame is an I-frame, execute S400 directly. S200: Performs dual forward inter-frame prediction on P-frames using two motion estimation modules set in the video encoder; S300: Uses two motion estimation modules set in the video encoder to perform forward inter-frame prediction, backward inter-frame prediction and bidirectional inter-frame prediction on B-frames respectively. S400: Perform intra-frame prediction on the current frame.

[0035] Specifically, in the video encoding method of this embodiment 1, the video encoder has a first motion estimation module ME0 and a second motion estimation module ME1. The video bitstream received by the video encoder contains the encoding configuration information of the current frame. The encoding configuration information includes the frame type information of each frame. For the current frame to be encoded, its type needs to be determined, that is, S100 is executed to determine whether the current frame to be encoded is a P frame, a B frame or an I frame. Inter-frame prediction and intra-frame prediction are performed for P frames and B frames respectively, while intra-frame prediction can be performed directly for I frames.

[0036] When the current frame type is P-frame, only forward inter-frame prediction can be performed, not backward inter-frame prediction. Therefore, S200 is executed to perform dual forward inter-frame prediction on the P-frame. See also Figure 3 In the received video stream, the reference frames for each P-frame are located before the P-frame, namely the first reference frame and the second reference frame. The first reference frame, the second reference frame, and the P-frame are arranged sequentially in the video stream. The first motion estimation module ME0 performs a first forward prediction of the P-frame from the first reference frame, and the second motion estimation module ME1 performs a second forward prediction of the P-frame from the second reference frame. In the first and second reference frames, a motion vector (mv) and a prediction block corresponding to the block to be encoded in the current frame are found, respectively.

[0037] When the current frame is a B-frame, forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction can be performed. Forward and backward inter-frame prediction are both unidirectional. Forward, backward, and bidirectional inter-frame predictions are performed on the B-frame by executing S300. See [link to documentation]. Figure 3 In the received video stream, the reference frames for B-frames are located before and after the B-frames, serving as forward reference frames and backward reference frames. These are arranged sequentially in the video stream. The first motion estimation module ME0 performs forward prediction of the B-frames from the forward reference frames, and the second motion estimation module ME1 performs backward prediction of the B-frames from the backward reference frames. In both the forward and backward reference frames, a motion vector and a prediction block corresponding to the block to be encoded in the current frame are found. Then, the first motion estimation module ME0 and the second motion estimation module ME1 simultaneously perform forward and backward prediction of the B-frames. In the forward and backward reference frames, two motion vectors and two prediction blocks corresponding to the block to be encoded in the current frame are found. Finally, a weighted average of the two prediction blocks is performed to obtain the final prediction block.

[0038] After completing the inter-frame prediction for P-frames and B-frames, regardless of whether the current frame is a P-frame, B-frame, or I-frame, S400 needs to be executed for intra-frame prediction. Following this, the video coding method of Embodiment 1 needs to calculate the prediction results obtained from the inter-frame and intra-frame predictions, and select the optimal result for the current frame as the basis for selecting the reference frame.

[0039] See Figure 5 The video encoding method of this embodiment 1 includes: S100: Determine the type of the current frame in the video stream; if the current frame is a P-frame, execute S200 and S400 in sequence; if the current frame is a B-frame, execute S300 and S400 in sequence; if the current frame is an I-frame, execute S400 directly. S200: Performs dual forward inter-frame prediction on P-frames using two motion estimation modules set in the video encoder; S300: Uses two motion estimation modules set in the video encoder to perform forward inter-frame prediction, backward inter-frame prediction and bidirectional inter-frame prediction on B-frames respectively. S400: Perform intra-frame prediction for the current frame; S500: Based on the type of the current frame, perform mode determination for P-frames, B-frames, and I-frames respectively; if the current frame is a P-frame, execute S600; if the current frame is a B-frame, execute S700; if the current frame is an I-frame, execute S800. S600: For P-frames, compare the results of dual forward inter-frame prediction and intra-frame prediction to obtain the corresponding first optimal mode result; S700: For B-frames, compare the results of forward inter-frame prediction, backward inter-frame prediction, bidirectional inter-frame prediction, and intra-frame prediction to obtain the corresponding second optimal mode result. S800: For I-frames, the intra-frame prediction result is used as the third best mode result; S900: Based on the type of the current frame, the first optimal mode result, the second optimal mode result, or the third optimal mode result is encoded into the video stream.

[0040] Specifically, after completing inter-frame prediction for P-frames and B-frames and intra-frame prediction for P-frames, B-frames and I-frames, the video coding method in this embodiment 1 needs to perform mode decision for different types of frames and select the optimal prediction mode (i.e., execute S600, S700, S800) so that the coding quality of the block to be coded in the current frame is as high as possible. In this way, the pixel difference between the reconstructed image in the reconstructed frame of the current frame and the original image of the current frame is as small as possible, the data loss of the video bitstream during transmission is less, and the quality of the reproduced video is higher.

[0041] The steps for pattern determination differ slightly depending on the type of frame.

[0042] See Figure 6 The video coding method in this embodiment 1 performs mode determination on P-frames in S600, specifically including: S601: By using the first motion estimation module ME0 in the two motion estimation modules to perform the first forward prediction of the P-frame for the first reference frame, the first absolute error and value of the P-frame corresponding to the first forward prediction of the P-frame are obtained. S602: By using the second motion estimation module ME1 in the two motion estimation modules to perform the second forward prediction of the second reference frame for the P-frame, the second absolute error and value of the P-frame corresponding to the second forward prediction of the P-frame are obtained. S603: Obtain the corresponding third absolute error value of the P-frame through intra-frame prediction; S604: By comparing the sum of the first absolute errors of P-frames, the sum of the second absolute errors of P-frames, and the sum of the third absolute errors of P-frames, the smallest value is taken as the first optimal mode result corresponding to P-frame.

[0043] S601, S602, and S603 are executed in no particular order, with S604 being executed last.

[0044] Specifically, the first reference frame marker address of a P-frame is ref0_addr_P, and the second reference frame marker address is ref1_addr_P. Encoding the current frame involves encoding each coded block within the current frame. During the inter-frame prediction process for the block to be encoded in the current frame, for a P-frame, the first motion estimation module ME0 performs the first forward prediction of the P-frame from the first reference frame ref0_addr_P to obtain the first motion vector P_mv0 of the P-frame. The first reference block of the P-frame is read from the first reference frame ref0_addr_P, and then the first absolute difference (sad) value P_sad0 between the block to be encoded in the current frame and the first reference block of the P-frame from the first reference frame ref0_addr_P is calculated. Then, the second motion estimation module ME1 performs the second forward prediction of the P-frame from the second reference frame ref1_addr_P to obtain the second motion vector P_mv1 of the P-frame. The second reference block of the P-frame is read from the second reference frame ref1_addr_P, and then the second absolute difference (sad) value P_sad1 between the block to be encoded in the current frame and the second reference block of the P-frame from the second reference frame ref1_addr_P is calculated. After completing inter-frame prediction, intra-frame prediction is performed on the current frame to obtain the third absolute error sum value P_sad2 of P-frame. Finally, the magnitudes of the first absolute error sum value P_sad0, the second absolute error sum value P_sad1, and the third absolute error sum value P_sad2 of P-frame are compared, and the smallest value is selected as the basis for selecting the prediction mode and reference frame. If the sum of the first absolute errors of P-frames, P_sad0, is minimized, then the optimal prediction mode is forward inter-frame prediction (i.e., P-frame first forward prediction), the optimal reference frame is the first reference frame ref0_addr_P, and the optimal motion vector is the first motion vector of P-frames, P_mv0. The corresponding first optimal mode result is {forward inter-frame prediction, ref0_addr_P, P_mv0}. If the sum of the second absolute errors of P-frames, P_sad1, is minimized, then the optimal prediction mode is forward inter-frame prediction (i.e., P-frame second forward prediction), the optimal reference frame is the second reference frame ref1_addr_P, and the optimal motion vector is the second motion vector of P-frames, P_mv1. The corresponding first optimal mode result is {forward inter-frame prediction, ref1_addr_P, P_mv1}. If the sum of the third absolute errors of P-frames, P_sad2, is minimized, then the optimal prediction mode is intra-frame prediction. There is no need to select an optimal reference frame, and there is no optimal motion vector. The corresponding first optimal mode result is {intra-frame prediction, 0, 0}.

[0045] See Figure 7 The video coding method in this embodiment 1 performs mode determination on B frames in step S700, specifically including: S701: By using the first motion estimation module ME0 in the two motion estimation modules to perform forward prediction of the B-frame for the forward reference frame, the first absolute error and value of the B-frame corresponding to the forward prediction of the B-frame are obtained. S702: By using the second motion estimation module ME1 in the two motion estimation modules to perform B-frame backward prediction of the backward reference frame, the second absolute error and value of the B-frame corresponding to the B-frame backward prediction are obtained. S703: By using the first motion estimation module ME0 to perform forward prediction of B-frames for forward reference frames, and simultaneously using the second motion estimation module ME1 to perform backward prediction of B-frames for backward reference frames, the third absolute error and value of B-frames corresponding to the bidirectional prediction of B-frames are obtained. S704: Obtain the corresponding fourth absolute error value of B-frame by performing intra-frame prediction on B-frame; S705: By comparing the sum of the first, second, third, and fourth absolute errors of B-frames, the smallest value is taken as the second optimal mode result corresponding to B-frame.

[0046] S701, S702 and S703 are executed in no particular order. S704 is executed after S701, S702 and S703 have been executed. S705 is executed last.

[0047] Specifically, the forward reference frame for the B-frame is marked with address ref0_addr_B, and the second reference frame is marked with address ref1_addr_B. Encoding the current frame involves encoding each coded block within it. During inter-frame prediction of the block to be encoded in the current frame, for the B-frame, the first motion estimation module ME0 performs forward prediction of the B-frame from the forward reference frame ref0_addr_B to obtain the first motion vector B_mv0 of the B-frame. The forward reference block of the B-frame is read from the forward reference frame ref0_addr_B, and then the first absolute error B_sad0 between the block to be encoded in the current frame and the forward reference block of the forward reference frame ref0_addr_B is calculated. Then, the second motion estimation module ME1 performs backward prediction of the B-frame from the backward reference frame ref1_addr_B to obtain the second motion vector B_mv1 of the B-frame. The backward reference block of the B-frame is read from the backward reference frame ref1_addr_B, and then the first absolute error B_sad0 between the block to be encoded in the current frame and the forward reference block of the forward reference frame ref0_addr_B is calculated. The second absolute error sum of B-frames, B_sad1, is calculated between the backward reference blocks of the backward reference frame ref1_addr_B. Simultaneously, the first motion estimation module ME0 performs forward prediction of the B-frame, and the second motion estimation module ME1 performs backward prediction. Based on the obtained first motion vector B_mv0 and second motion vector B_mv1, the forward reference blocks and backward reference blocks of the B-frames are read from the forward reference frame ref0_addr_B and the backward reference frame ref1_addr_B, respectively. Then, the third absolute error sum of B-frames, B_sad3, is calculated between the current frame's block to be encoded and the forward reference block of the forward reference frame ref0_addr_B, and between the current frame's block and the backward reference block of the backward reference frame ref1_addr_B. After completing inter-frame prediction, intra-frame prediction is performed on the current frame to obtain the fourth absolute error sum of B-frames, B_sad2. Finally, the sum of the first absolute errors of B-frame B_sad0, the sum of the second absolute errors of B-frame B_sad1, the sum of the third absolute errors of B-frame B_sad3, and the sum of the fourth absolute errors of B-frame B_sad2 are compared, and the smallest value is selected as the basis for selecting the prediction mode and the reference frame.If the sum of the first absolute errors of B-frame B_sad0 is minimized, then the optimal prediction mode is forward inter-frame prediction (i.e., forward prediction of B-frames), the optimal reference frame is the forward reference frame ref0_addr_B, and the optimal motion vector is the first motion vector of B-frame B_mv0. The corresponding second optimal mode result is {forward inter-frame prediction, ref0_addr_B, B_mv0}. If the sum of the second absolute errors of B-frame B_sad1 is minimized, then the optimal prediction mode is backward inter-frame prediction (i.e., backward prediction of B-frames), the optimal reference frame is the backward reference frame ref1_addr_B, and the optimal motion vector is the second motion vector of B-frame B_mv1. The corresponding second optimal mode result is {backward inter-frame prediction, ref1_addr_B, ... If the sum of the third absolute errors of B-frames, B_sad3, is the smallest, then the optimal prediction mode is bidirectional inter-frame prediction (i.e., simultaneous forward and backward prediction of B-frames). The optimal reference frame is the set of forward and backward reference frames (ref0_addr_B, ref1_addr_B), and the optimal motion vector is the set of the first and second motion vectors of B-frames (B_mv0, B_mv1). The corresponding second optimal mode result is {bidirectional inter-frame prediction, (ref0_addr_B, ref1_addr_B), (B_mv0, B_mv1)}. If the sum of the fourth absolute errors of B-frames, B_sad2, is the smallest, then the optimal prediction mode is intra-frame prediction. There is no need to select an optimal reference frame, and there is no optimal motion vector. The corresponding second optimal mode result is {intra-frame prediction, 0, 0}.

[0048] After obtaining the first, second, or third optimal mode result for the current frame, by executing S900, only the first optimal mode result is encoded into the information to be encoded when the current frame is a P-frame, only the second optimal mode result is encoded into the information to be encoded when the current frame is a B-frame, and only the third optimal mode result is encoded into the information to be encoded when the current frame is an I-frame. Simultaneously, for each absolute error and value generated during the mode decision process, these are transformed and quantized as residuals to obtain coding coefficients. Then, the information to be encoded and the coding coefficients are entropy-encoded together and written into the video bitstream. When image reconstruction is required, the coding coefficients in the video bitstream are inversely transformed, inversely quantized, and reconstructed, and based on the information to be encoded, a reconstructed frame (containing the reconstructed image) is obtained for the current frame. This reconstructed frame is also used as a reference frame for subsequent frames in the video bitstream.

[0049] Example 2 Embodiment 2 of the present invention discloses a video coding system for executing the video coding method disclosed in Embodiment 1, which simultaneously achieves dual forward prediction and bidirectional prediction by reusing hardware resources.

[0050] See Figure 8 The video coding system 10 of this embodiment 2 includes a first inter-frame prediction control module 11 and a second inter-frame prediction control module 12. The first inter-frame prediction control module 11 is used to perform dual forward inter-frame prediction on P frames in the video bitstream using two motion estimation modules set in the video encoder. The second inter-frame prediction control module 12 is used to perform forward inter-frame prediction, backward inter-frame prediction and bidirectional inter-frame prediction on B frames in the video bitstream using two motion estimation modules set in the video encoder.

[0051] Specifically, the video encoding system 10 of this embodiment 2 can be set in the video encoder. If only one motion estimation module is set in the video encoder, the motion estimation module can only search for one reference block in a certain period of time, which means that the reference block can only be searched serially, which will greatly reduce the encoding speed. If multiple motion estimation modules are added to the video encoder, it can indeed support multiple forward prediction or bidirectional prediction and find multiple reference blocks for the block to be encoded during the motion estimation process. However, since the motion estimation module is the most time-consuming and hardware resource-intensive module in the entire encoder, adding multiple motion estimation modules will greatly increase hardware resources. Therefore, after balancing factors such as encoding quality, encoding speed, and hardware resources, in this embodiment 2, only one additional motion estimation module is added to the video encoder. By reusing two motion estimation modules, dual forward prediction and bidirectional prediction of the current frame are achieved. In this way, the video encoding system 10 of this embodiment 2 can be applied to the encoding standards of many video formats, such as H.264 and HEVC, which all support multiple forward prediction and bidirectional prediction of the current frame. Multiple forward prediction means that the current frame can have multiple forward reference frames, and when performing motion estimation, a reference block is searched in each forward reference frame; bidirectional prediction means that the current frame can have one forward reference frame and one backward reference frame, and when performing motion estimation, each block to be encoded in the current frame can have one forward reference block and one backward reference block.

[0052] In a video stream, frame types include I-frames (Intraframe Frames), P-frames, and B-frames. Since I-frames do not require inter-frame prediction, inter-frame prediction is only performed on P-frames and B-frames. When performing bidirectional forward inter-frame prediction on P-frames and bidirectional inter-frame prediction on B-frames, two motion estimation modules operate simultaneously. One inter-frame prediction in the bidirectional forward inter-frame prediction of P-frames via the first inter-frame prediction control module 11 can reuse one motion estimation module for the forward inter-frame prediction of B-frames via the second inter-frame prediction control module 12. Similarly, the other inter-frame prediction in the bidirectional forward inter-frame prediction of P-frames via the first inter-frame prediction control module 11 can reuse the other motion estimation module for the backward inter-frame prediction of B-frames via the second inter-frame prediction control module 12. By reusing motion estimation modules as described above to encode the current frame of the video stream, hardware resources are fully utilized, and encoding efficiency is improved.

[0053] Example 3 Embodiment 3 of the present invention discloses a video encoder that simultaneously achieves dual forward prediction and bidirectional prediction by reusing hardware resources.

[0054] See Figure 9 The video encoder in this embodiment 3 includes a hardware driver, a hardware encoder, memory, and the video encoding system disclosed in embodiment 2 of this application. The hardware encoder is connected to the hardware driver, memory, and video encoding system respectively.

[0055] The hardware encoder includes a motion compensation and transform quantization unit, a SAD SRAM (Sum of Absolute Difference Static Random-Access Memory), an inter-frame prediction unit, an intra-frame prediction unit, a mode decision unit, an entropy-coded SRAM, and an entropy coding unit. The inter-frame prediction unit includes a first motion estimation module ME0, a second motion estimation module ME1, and a bidirectional prediction module. The video coding system includes a first inter-frame prediction control module and a second inter-frame prediction control module. The first inter-frame prediction control module uses the two motion estimation modules to perform bidirectional forward inter-frame prediction on P-frames in the video stream. The second inter-frame prediction control module uses the two motion estimation modules to perform forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction on B-frames in the video stream, respectively.

[0056] In the video encoder of this embodiment 3, the motion compensation and transform quantization unit is connected to the hardware driver, memory, SAD SRAM, intra-frame prediction unit, first motion estimation module ME0, second motion estimation module ME1 and entropy coding SRAM, respectively. The inter-frame prediction unit is connected to SAD SRAM. The mode decision unit is connected to the bidirectional prediction module and the intra-frame prediction unit, respectively. The entropy coding unit is connected to the entropy coding SAD SRAM and memory, respectively. Data can be transmitted between each unit and module.

[0057] Specifically, the hardware driver is used to drive the hardware encoder to encode the video stream. Specifically, it transmits encoding configuration information for each frame in the video stream to the hardware encoder. This encoding configuration information includes frame type information, which indicates whether the current frame to be encoded is an I-frame, P-frame, or B-frame. It is also used to transmit the addresses of the first reference frame (ref0_addr_P) and the second reference frame (ref1_addr_P) for P-frames, and the addresses of the forward and backward reference frames (ref0_addr_B) for B-frames to the motion compensation and transform quantization unit. The addresses of the first and forward reference frames (ref0_addr_P and ref0_addr_B) indicate multiplexing of the first motion estimation module ME0, while the addresses of the second and backward reference frames (ref1_addr_B) indicate multiplexing of the second motion estimation module ME1. I-frames do not have reference frames, therefore the reference frame for I-frames is an invalid value and no address is identified.

[0058] The inter-frame prediction unit is used for inter-frame prediction of P-frames and B-frames. After receiving the encoding configuration information of the video bitstream transmitted by the hardware driver, it first determines the frame type of the current frame. If the current frame is an I-frame, the inter-frame prediction unit ends its process and starts the intra-frame prediction unit to perform intra-frame prediction on the I-frame. If the current frame is a P-frame, the first motion estimation module ME0 is started to perform the first forward prediction of the P-frame based on the first reference frame ref0_addr_P, and the resulting first motion vector P_mv0 of the P-frame is transmitted to the motion compensation and transform quantization unit. The second motion estimation module ME1 is started to perform the second forward prediction of the P-frame based on the second reference frame ref1_addr_P, and the resulting second motion vector P_mv1 of the P-frame is transmitted to the motion compensation and transform quantization unit. If the current frame is a B-frame, the first motion estimation module ME0 is activated to perform forward B-frame prediction based on the forward reference frame ref0_addr_B, and the resulting first motion vector B_mv0 of the B-frame is transmitted to the motion compensation and transformation quantization unit. The second motion estimation module ME1 is activated to perform backward B-frame prediction based on the backward reference frame ref1_addr_B, and the resulting second motion vector B_mv1 of the B-frame is transmitted to the motion compensation and transformation quantization unit. The bidirectional prediction module is activated to control the first motion estimation module ME0 to perform forward B-frame prediction based on the forward reference frame ref0_addr_B, and simultaneously control the second motion estimation module ME1 to perform backward B-frame prediction based on the backward reference frame ref1_addr_B, and transmit the resulting first motion vector B_mv0 and second motion vector B_mv1 of the B-frame to the motion compensation and transformation quantization unit.

[0059] See Figure 10The motion compensation and transform quantization unit is used to receive the first motion vector P_mv0 of the P-frame and the first motion vector B_mv0 of the B-frame sent by the first motion estimation module ME0 during the inter-frame prediction process of P-frames and B-frames. Then, it reads a reference block from the address ref0_addr_P of the first reference frame of the P-frame, calculates the first absolute error sum P_sad0 between the block to be encoded in the P-frame and the first reference frame ref0_addr_P, and reads a reference block from the address ref0_addr_B of the preceding reference frame of the B-frame, calculates the first absolute error sum P_sad0 between the block to be encoded in the B-frame and the preceding reference frame ref0_addr_B. The first absolute error sum of the frame is B_sad0; the second motion vector P_mv1 of the P frame and the second motion vector B_mv1 of the B frame are sent by the second motion estimation module ME1. Then, the reference block is read from the address ref1_addr_P of the second reference frame of the P frame, and the second absolute error sum of the P frame between the block to be encoded of the P frame and the second reference frame ref1_addr_P is calculated. The reference block is read from the address ref1_addr_B of the backward reference frame of the B frame, and the second absolute error sum of the B frame between the block to be encoded of the B frame and the backward reference frame ref1_addr_B is calculated. When the motion compensation and transform quantization unit receives the first motion vector B_mv0 and the second motion vector B_mv1 of the B-frame simultaneously sent by the first motion estimation module ME0 and the second motion estimation module ME1, it indicates that the bidirectional prediction module has started working. The motion compensation and transform quantization unit reads the reference blocks from the address ref0_addr_B of the forward reference frame and the address ref1_addr_B of the backward reference frame of the B-frame, respectively, and calculates the third absolute error of the B-frame between the block to be encoded and the forward reference frame ref0_addr_B and the backward reference frame ref1_addr_B. Then, the motion compensation and transform quantization unit transmits the first absolute error sum P_sad0 and the second absolute error sum P_sad1 of P-frames, and the first absolute error sum B_sad0, the second absolute error sum B_sad1, and the third absolute error sum B_sad3 of B-frames to the SAD SRAM. This data is then used as residuals for transform and quantization to obtain coding coefficients. These coefficients are then subjected to inverse transform, inverse quantization, and reconstruction to obtain the reconstructed image. Finally, the first absolute error sum P_sad0 and the second absolute error sum P_sad1 of P-frames, the first absolute error sum B_sad0, the second absolute error sum B_sad1, and the third absolute error sum B_sad3 of B-frames, the coding coefficients, and the reconstructed image are transmitted to the entropy coding SRAM.

[0060] SAD SRAM is used to store a large number of absolute errors and values ​​generated by the motion compensation and transform quantization unit for each frame in the video bitstream. It also transmits the optimal motion vector and minimum absolute error and value of each frame to the inter-frame prediction unit by comparing the absolute errors and values ​​of different motion vectors obtained by the first motion estimation module ME0 and the second motion estimation module ME1 after performing inter-frame prediction.

[0061] The intra-frame prediction unit is used to perform intra-frame prediction on I-frames, P-frames, and B-frames, and obtain the third absolute error sum value P_sad2 for P-frames, the fourth absolute error sum value B_sad2 for B-frames, and the third optimal mode result for I-frames, respectively.

[0062] The mode decision unit is used to compare the prediction results of the intra-frame prediction unit and the inter-frame prediction unit to obtain the first optimal mode result, the second optimal mode result, and the third optimal mode result corresponding to the P-frame, B-frame, and I-frame, respectively. The first optimal mode result, the second optimal mode result, and the third optimal mode result, along with the related frame type information, reference frame address information, motion vector information, and absolute error of the SAD value, are transmitted to the entropy-coded SRAM.

[0063] Specifically, for the current frame (whether the frame type is P-frame or B-frame only affects the suffix identifier of the reference frame address and the prefix identifier of the motion vector and absolute error sum value), the following assumptions are made: ① The optimal motion vector of the block to be encoded obtained by the first motion estimation module ME0 is best_mv0, and the corresponding minimum absolute error sum value is min_sad0; ② The optimal motion vector of the block to be encoded obtained by the second motion estimation module ME1 is best_mv1, and the corresponding minimum absolute error sum value is min_sad1; ③ The optimal intra-prediction mode of the block to be coded obtained by intra-prediction is best_mode, and the corresponding minimum absolute error sum value is min_sad2; ④ Directly denote the motion vector obtained from bidirectional prediction as (best_mv0, best_mv1), and denote the corresponding absolute error sum of bidirectional prediction as min_sad3.

[0064] The decision method of the pattern decision unit is as follows: ① When the current frame is an I-frame, there is no inter-frame prediction, and the third optimal mode result of the intra-frame prediction is directly selected as the optimal result; ② When the current frame is a P-frame, compare the sizes of min_sad0, min_sad1, and min_sad2.

[0065] a) If min_sad0 is the smallest, then the result of the first optimal mode is: the optimal prediction mode is forward inter-frame prediction, the optimal reference frame is the first reference frame ref0_addr_P, and the optimal motion vector is best_P_mv0. b) If min_sad1 is the smallest, then the result of the first optimal mode is: the optimal prediction mode is forward inter-frame prediction, the optimal reference frame is the second reference frame ref1_addr_P, and the optimal motion vector is best_P_mv1. c) If min_sad2 is the smallest, then the first optimal mode result is: the optimal prediction mode is the best_P_mode of intra-frame prediction.

[0066] ③ When the current frame is a B-frame, compare the sizes of min_sad0, min_sad1, min_sad2, and min_sad3.

[0067] a) If min_sad0 is the smallest, then the result of the second optimal mode is: the optimal prediction mode is forward inter-frame prediction, the optimal reference frame is forward reference frame ref0_addr_B, and the optimal motion vector is best_B_mv0. b) If min_sad1 is the smallest, then the result of the second optimal mode is: the optimal prediction mode is backward inter-frame prediction, the optimal reference frame is the backward reference frame ref1_addr_B, and the optimal motion vector is best_B_mv1. c) If min_sad2 is the smallest, then the second best mode result is: the best prediction mode is the best_B_mode of intra-frame prediction; d) If min_sad3 is the smallest, then the second optimal mode result is: the optimal prediction mode is bidirectional inter-frame prediction, the optimal reference frame is the set of forward reference frame and backward reference frame (ref0_addr_B, ref1_addr_B), and the optimal motion vector is the set of the first motion vector of B frame and the second motion vector of B frame (best_B_mv0, best_B_mv1).

[0068] The entropy-coded SRAM is used to cache the information to be encoded by the entropy coding unit, including the first absolute error sum P_sad0 and the second absolute error sum P_sad1 of P-frames corresponding to P-frames generated by the motion compensation and transform quantization unit, the first absolute error sum B_sad0, the second absolute error sum B_sad1 and the third absolute error sum B_sad3 of B-frames corresponding to B-frames, coding coefficients and reconstructed images, and the first optimal mode result, the second optimal mode result and the third optimal mode result generated by the mode decision unit, as well as the related frame type information, reference frame address information, motion vector information and absolute error sum of SAD values.

[0069] The entropy coding unit is used to read information from the entropy coding SRAM, perform entropy coding, and write the encoded video stream into memory.

[0070] The memory is used to store the reconstructed frame of the current frame in the video stream, which will also be used as a reference frame for subsequent frames.

[0071] The process of the video encoder in Embodiment 3 encoding the video stream can be achieved through... Figure 11 The reference relationships between frames in the video stream are briefly explained below. Here, I0 represents the intra-predicted I-frame (the first frame of the video sequence is always an I-frame), P0, P1, and P2 represent P-frames, and B0, B1, and B2 represent B-frames; arrows point to the reference frame of the current frame, those pointing forward are forward reference frames, and those pointing backward are backward reference frames. Assuming P1 is currently being encoded: the first motion estimation module ME0 searches for the first forward reference frame I0 of P1, and the second motion estimation module ME1 searches for the second forward reference frame P0 of P1; next, B2 is encoded (since B2 references P1, the encoding order is...). Encoding P1, then B2: Using the first motion estimation module ME0, search for the forward reference frame P0 of B2, and using the second motion estimation module ME1, search for the backward reference frame P1 of B2; Next, encoding B3: Using the first motion estimation module ME0, search for the forward reference frame P0 of B3, and using the second motion estimation module ME1, search for the backward reference frame P1 of B3; Next, encoding P2: Using the first motion estimation module ME0, search for the first forward reference frame P0 of P2, and using the second motion estimation module ME1, search for the second forward reference frame P1 of P2. This process is repeated for the remaining frames in the video stream.

[0072] The video encoder of this embodiment 3 makes full use of the hardware resources in the hardware encoder and improves the encoding efficiency.

[0073] Example 4 Embodiment 4 of the present invention discloses a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the video encoding method disclosed in Embodiment 1.

[0074] Example 5 Embodiment 5 of the present invention discloses a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the video encoding method disclosed in Embodiment 1.

[0075] The video coding method, video coding system, and video encoder disclosed in this invention enable the second reference frame of a P-frame and the backward reference frame of a B-frame to reuse the same motion estimation module, making full use of hardware resources to improve coding efficiency, and are applicable to all video coding standards.

[0076] It should be understood that the above description of specific embodiments of the present invention is only for illustrating the technical approach and features of the present invention, and is intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. However, the present invention is not limited to the specific embodiments described above. All changes or modifications made within the scope of the claims of the present invention should be covered within the protection scope of the present invention.

Claims

1. A video encoding method, characterized in that, The video encoding method includes: Two motion estimation modules configured in the video encoder are used to perform bi-forward inter-frame prediction on P-frames in the video bitstream; the bi-forward inter-frame prediction for the P-frames specifically involves: The first motion estimation module of the two motion estimation modules is used to perform a first forward prediction of the P-frame for the first reference frame, and the second motion estimation module of the two motion estimation modules is used to perform a second forward prediction of the P-frame for the second reference frame. In the video stream, both the first reference frame and the second reference frame are located before the P frame; Two motion estimation modules configured in the video encoder are used to perform forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction on B-frames in the video bitstream, respectively; the forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction performed on the B-frames are specifically as follows: The first motion estimation module of the two motion estimation modules is used to perform forward B-frame prediction of the forward reference frame for the B-frame; The second motion estimation module of the two motion estimation modules is used to perform B-frame backward prediction of the backward reference frame for the B-frame; The first motion estimation module is used to perform forward B-frame prediction of the forward reference frame for the B-frame, and the second motion estimation module is used to perform backward B-frame prediction of the backward reference frame for the B-frame. In the video stream, the forward reference frame is located before the B-frame, and the backward reference frame is located after the B-frame.

2. The video encoding method according to claim 1, characterized in that, Before performing inter-frame prediction on the P-frame and the B-frame, the method further includes: Determine the type of frames in a video stream; If the type is a P-frame, then the dual forward inter-frame prediction and intra-frame prediction are performed on the P-frame. If the type is a B-frame, then the forward inter-frame prediction, the backward inter-frame prediction, the bidirectional inter-frame prediction, and the intra-frame prediction are performed on the B-frame. If the type is I-frame, then intra-frame prediction is performed on the I-frame.

3. The video encoding method according to claim 2, characterized in that, After performing the intra-frame prediction on the P-frame, B-frame, and I-frame, the method further includes: Based on the type of the current frame, perform mode determination on the P frame, the B frame, and the I frame respectively; For the P-frame, the results of the dual forward inter-frame prediction and the intra-frame prediction are compared to obtain the corresponding first optimal mode result; For the B-frame, the results of the forward inter-frame prediction, the backward inter-frame prediction, the bidirectional inter-frame prediction, and the intra-frame prediction are compared to obtain the corresponding second optimal mode result; For the I-frame, the result of the intra-frame prediction is taken as the third optimal mode result; Based on the type of the current frame, the first optimal mode result, the second optimal mode result, or the third optimal mode result is encoded into the video stream.

4. The video encoding method according to claim 3, characterized in that, The mode determination for the P-frame specifically includes: By using the first motion estimation module in the two motion estimation modules to perform the first forward prediction of the P-frame for the first reference frame, the first absolute error and value of the P-frame corresponding to the first forward prediction of the P-frame are obtained. By using the second motion estimation module in the two motion estimation modules to perform a second forward prediction of the P-frame for the second reference frame, the second absolute error and value of the P-frame corresponding to the second forward prediction of the P-frame are obtained. By performing intra-frame prediction on the P-frame, the corresponding P-frame third absolute error sum value is obtained; By comparing the sum of the first absolute errors of the P-frame, the sum of the second absolute errors of the P-frame, and the sum of the third absolute errors of the P-frame, the smallest value is taken as the first optimal mode result corresponding to the P-frame.

5. The video encoding method according to claim 3, characterized in that, The mode determination for the B-frame specifically includes: By using the first motion estimation module among the two motion estimation modules to perform forward prediction of the B-frame for the forward reference frame, the first absolute error and value of the B-frame corresponding to the forward prediction of the B-frame are obtained. By using the second motion estimation module among the two motion estimation modules to perform backward prediction of the B-frame from the backward reference frame, the second absolute error and value of the B-frame corresponding to the backward prediction of the B-frame are obtained. By using the first motion estimation module to perform forward prediction of the B-frame for the forward reference frame, and simultaneously using the second motion estimation module to perform backward prediction of the B-frame for the backward reference frame, the third absolute error and value of the B-frame corresponding to the bidirectional prediction of the B-frame are obtained. By performing intra-frame prediction on the B-frame, the corresponding fourth absolute error and value of the B-frame are obtained; By comparing the sum of the first, second, third, and fourth absolute errors of the B-frame, the smallest value is taken as the second optimal mode result corresponding to the B-frame.

6. A video encoding system, characterized in that, The video coding system includes a first inter-frame prediction control module and a second inter-frame prediction control module. The first inter-frame prediction control module is used to perform dual forward inter-frame prediction on P frames in the video bitstream using two motion estimation modules set in the video encoder. The specific steps of the dual forward inter-frame prediction for the P-frame are as follows: The first motion estimation module of the two motion estimation modules is used to perform a first forward prediction of the P-frame for the first reference frame, and the second motion estimation module of the two motion estimation modules is used to perform a second forward prediction of the P-frame for the second reference frame. In the video stream, both the first reference frame and the second reference frame are located before the P frame; The second inter-frame prediction control module is used to perform forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction on B-frames in the video bitstream using two motion estimation modules located in the video encoder; the forward inter-frame prediction, backward inter-frame prediction, and bidirectional inter-frame prediction performed on the B-frames are specifically as follows: The first motion estimation module of the two motion estimation modules is used to perform forward B-frame prediction of the forward reference frame for the B-frame; The second motion estimation module of the two motion estimation modules is used to perform B-frame backward prediction of the backward reference frame for the B-frame; The first motion estimation module is used to perform forward B-frame prediction of the forward reference frame for the B-frame, and the second motion estimation module is used to perform backward B-frame prediction of the backward reference frame for the B-frame. In the video stream, the forward reference frame is located before the B-frame, and the backward reference frame is located after the B-frame.

7. A video encoder, characterized in that, It includes a hardware driver, a hardware encoder, memory, and a video encoding system as described in claim 6, wherein the hardware encoder is connected to the hardware driver, the memory, and the video encoding system, respectively; The hardware driver is used to transmit the video bitstream, the first and second reference frames corresponding to P frames in the video bitstream, and the forward and backward reference frames corresponding to B frames in the video bitstream to the hardware encoder. The hardware encoder includes two motion estimation modules; The video coding system includes a first inter-frame prediction control module and a second inter-frame prediction control module. The first inter-frame prediction control module is used to perform bi-forward inter-frame prediction on P-frames in the video bitstream using the two motion estimation modules. The second inter-frame prediction control module is used to perform forward inter-frame prediction, backward inter-frame prediction and bi-directional inter-frame prediction on B-frames in the video bitstream using the two motion estimation modules. The memory is used to store the reconstructed frames of the P frames and B frames in the video stream.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the video encoding method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when run on a computer, causes the computer to perform the video encoding method as described in any one of claims 1 to 5.

Images