Video coding method, apparatus and device based on bidirectional prediction, and storage medium

Abstract

Embodiments of the present application provide a video coding method, apparatus and device based on bidirectional prediction, and a storage medium. According to the technical solution provided by an embodiment of the present application, the method comprises: determining a plurality of weight combinations on the basis of a first reference frame in a first direction and a plurality of candidate reference frames in a second direction of a unit to be coded; on the basis of the weight combinations, searching the plurality of candidate reference frames to determine a second reference frame and a second prediction block corresponding to the second direction; when the first reference frame and the second reference frame satisfy a set search end condition, retesting the first reference frame and the second reference frame to determine a first target weight corresponding to the first reference frame and a second target weight corresponding to the second reference frame; and on the basis of a first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame, performing coding processing on said unit, thereby effectively reducing the number bidirectional predictions and rate distortion optimization calculations, and improving the video coding efficiency.

Description

Video encoding method, device, equipment and storage medium based on bidirectional prediction

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 1, 2023, with application number 202310961680.2, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of video coding technology, and in particular to a video coding method, apparatus, device, and storage medium based on bidirectional prediction. Background Art

[0003] Current video coding standards are all based on a hybrid coding framework, in which each frame in the video is first divided into a maximum coding unit of a fixed size, and then further divided into coding units of variable sizes, and each coding unit is encoded separately.

[0004] Bi-prediction with CU-level weights (BCW) is a bidirectional inter-frame prediction technique often used in the coding process of coding units. In bidirectional prediction, the weighted average of the prediction blocks in the forward and backward directions is used as the prediction value, rather than the average value used in conventional bidirectional search. If the current block is encoded in AMVP (Advanced Motion Vector Prediction) mode, a weight is selected through prediction, and an index is encoded to represent the weight selected for the current block. Bidirectional inter-frame prediction requires performing a bidirectional prediction process for each coding unit using every possible weight and calculating the rate-distortion optimization (RDO). While this improves encoding quality, the multiple bidirectional predictions and RDO calculations require a large amount of computation, resulting in a reduction in encoding speed. In bidirectional inter-frame prediction, bidirectional predictions are performed for each coding unit using every possible weight. This multiple bidirectional predictions and RDO calculations require a large amount of computation, reducing video coding efficiency.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a video encoding method, apparatus, device and storage medium based on bidirectional prediction to solve the technical problem that the multiple bidirectional prediction and rate-distortion optimization calculation processes of existing video encoding require a large amount of calculation, which reduces the video encoding efficiency, effectively reduces the calculation amount of bidirectional prediction, and improves video encoding efficiency.

[0007] In a first aspect, an embodiment of the present application provides a video encoding method based on bidirectional prediction, comprising:

[0008] Determining multiple weight combinations based on a first reference frame of a unit to be coded and multiple candidate reference frames in a second direction in a first direction, where the second direction is opposite to the first direction, and the first direction is determined based on a unidirectional prediction result of the unit to be coded or a second direction determined by a previous search for candidate reference frames;

[0009] Searching the plurality of candidate reference frames according to the weight combination, and determining a second reference frame and a second prediction block corresponding to the second direction according to the search results;

[0010] When the first reference frame and the second reference frame meet a set search end condition, retesting the first reference frame and the second reference frame to determine a first target weight corresponding to the first reference frame and a second target weight corresponding to the second reference frame;

[0011] The unit to be encoded is encoded based on the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame, where the first prediction block is determined based on the unidirectional prediction result of the unit to be encoded, or the second prediction block corresponding to the second reference frame determined by the last search for the candidate reference frame.

[0012] In a second aspect, an embodiment of the present application provides a video encoding device based on bidirectional prediction, including a weight prediction module, a reference frame determination module, a weight determination module, and an encoding processing module, wherein:

[0013] The weight prediction module is configured to determine a plurality of weight combinations based on a first reference frame of a unit to be coded and a plurality of candidate reference frames in a second direction in a first direction opposite to the first direction, the first direction being determined based on a unidirectional prediction result of the unit to be coded or a second direction determined by a previous search for candidate reference frames;

[0014] The reference frame determination module is configured to search the plurality of candidate reference frames according to the weight combination, and determine the second reference frame and the second prediction block corresponding to the second direction according to the search results;

[0015] The weight determination module is configured to retest the first reference frame and the second reference frame to determine a first target weight corresponding to the first reference frame and a second target weight corresponding to the second reference frame when the first reference frame and the second reference frame meet a set search end condition;

[0016] The encoding processing module is configured to perform encoding processing on the unit to be encoded based on a first prediction block and a first target weight corresponding to the first reference frame, and a second prediction block and a second target weight corresponding to the second reference frame, wherein the first prediction block is determined based on a unidirectional prediction result of the unit to be encoded, or a second prediction block corresponding to the second reference frame determined by the last search for the candidate reference frame.

[0017] In a third aspect, an embodiment of the present application provides a video encoding device based on bidirectional prediction, comprising: a memory and one or more processors;

[0018] The memory is used to store one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the video encoding method based on bidirectional prediction as described in the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to perform the video encoding method based on bidirectional prediction as described in the first aspect.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the video encoding method based on bidirectional prediction as described in the first aspect.

[0022] The embodiment of the present application determines multiple weight combinations based on the first reference frame in the first direction and multiple candidate reference frames in the second direction of the unit to be encoded, searches the multiple candidate reference frames according to the weight combinations to determine the second reference frame and the second prediction block corresponding to the second direction, and retests the first reference frame and the second reference frame when the first reference frame and the second reference frame meet the set search end condition to determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame, and encodes the unit to be encoded according to the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame. In the bidirectional prediction process, multiple weight combinations are pre-selected according to the first reference frame in the first direction and the candidate reference frames in the second direction, and the second reference frame and the second prediction block are pre-selected according to the weight combinations. The second reference frame and the second prediction block are iteratively updated each time the search end condition is not met, and the search results are retested to determine the first target weight and the second target weight when the search end condition is met, thereby effectively reducing the number of bidirectional prediction and rate-distortion optimization calculations and improving video coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] FIG1 is a flowchart of a video encoding method based on bidirectional prediction provided by an embodiment of the present application;

[0024] FIG2 is a schematic diagram of a weight combination determination process provided by an embodiment of the present application;

[0025] FIG3 is a schematic diagram of a process for determining a second reference frame and a second prediction block provided by an embodiment of the present application;

[0026] FIG4 is a schematic structural diagram of a video encoding device based on bidirectional prediction provided by an embodiment of the present application;

[0027] FIG5 is a schematic structural diagram of a video encoding device based on bidirectional prediction provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only some, but not all, of the contents related to the present application are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0029] The video coding method based on bidirectional prediction provided in this application can be applied to video coding scenarios, such as during a video call or live video broadcast. The video coding method based on bidirectional prediction encodes the collected image data, aiming to pre-select multiple weight combinations based on the first reference frame in the first direction and the candidate reference frame in the second direction during the bidirectional prediction process, pre-select the second reference frame and the second prediction block based on the weight combination, and re-test the search results to determine the first target weight and the second target weight when the search end condition is met, thereby effectively reducing the number of bidirectional prediction and rate-distortion optimization calculations and improving video coding efficiency. In existing video coding schemes based on bidirectional prediction, for coding units encoded in the advanced motion vector prediction (AMVP) mode, the final selected weight needs to be determined based on the prediction results. If the actual value of the coding unit is X, then for each possible weight combination pair, the bidirectional prediction needs to first search one direction to obtain the optimal prediction block A1, and then calculate the target prediction value of the other direction based on X, A1 and weight, and then search for the optimal prediction block B1 closest to the target prediction value from the other direction, and then calculate the target prediction value A2 of the first direction based on X, B1 and weight, and repeat this process many times until the prediction result changes to meet the target requirements, and then calculate the rate-distortion cost under the current weight combination, and finally select the weight combination with the smallest rate-distortion cost for encoding. Bidirectional prediction requires a bidirectional prediction process for each coding unit according to each possible weight, and calculates the rate-distortion cost. Although the encoding quality is improved, the multiple bidirectional predictions and rate-distortion optimization calculation processes significantly increase the amount of calculation, resulting in a reduction in encoding speed. Based on this, a video encoding method based on bidirectional prediction is provided in an embodiment of the present application to solve the technical problem that the multiple bidirectional predictions and rate-distortion optimization calculation processes of the existing video encoding scheme based on bidirectional prediction require a large amount of calculation, which reduces the efficiency of video encoding.

[0030] Figure 1 shows a flowchart of a video encoding method based on bidirectional prediction provided in an embodiment of the present application. The video encoding method based on bidirectional prediction provided in an embodiment of the present application can be executed by a video encoding device based on bidirectional prediction, and the video encoding device based on bidirectional prediction can be implemented by hardware and / or software and integrated in a video encoding device based on bidirectional prediction.

[0031] The following description is based on an example of a video encoding method based on bidirectional prediction performed by a video encoding device based on bidirectional prediction. Referring to FIG1 , the video encoding method based on bidirectional prediction includes:

[0032] S110: Determine multiple weight combinations according to the first reference frame in the first direction and multiple candidate reference frames in the second direction of the unit to be encoded.

[0033] When encoding an image to be encoded, the image to be encoded may be divided into multiple coding units (CUs) of equal or different sizes. Each CU may be encoded using a video encoding method based on bidirectional prediction to obtain an encoding result for the corresponding CU. The encoding result for the entire image to be encoded is then determined based on the encoding results for each CU, and encoding continues with the next image to be encoded.

[0034] Exemplarily, a first reference frame in a first direction and multiple candidate reference frames in a second direction for a unit to be encoded are determined, wherein the number of candidate reference frames in different directions can be set in real time or in advance. For example, the forward candidate reference frames can be a first number (e.g., 5) of subsequent image frames, and the backward candidate reference frames can be a second number (e.g., 3) of subsequent image frames. The frame sequence number of the subsequent image frame is greater than the frame sequence number of the preceding image frame.

[0035] After determining the first reference frame in the first direction and multiple candidate reference frames in the second direction, multiple weight combinations are determined based on the first reference frame in the first direction and the multiple candidate reference frames in the second direction. The weight combinations correspond one-to-one to the candidate reference frames, and the weight combinations include a first prediction weight corresponding to the first reference frame and a second prediction weight corresponding to the candidate reference frame. In one embodiment, the weight combination corresponding to each candidate reference frame can be determined based on the difference between different weight combinations and the distance ratio between the candidate reference frame and the first reference frame (for example, the weight combination with the smallest difference is used as the weight combination corresponding to the candidate reference frame).

[0036] In one embodiment, the weights may be based on the selectable weights for bidirectional prediction BCW specified in the VVC (Versatile Video Coding) coding standard. For example, for low-delay images, 5 selectable weights (ω∈{-2,3,4,5,10}) are provided; for non-low-delay images, 3 selectable weights (ω∈{3,4,5}) are provided, and when the weight in one direction is ω, the weight in the other direction is 8-ω. Optionally, whether the image to be encoded is a low-delay image or a non-low-delay image has been determined in the image processing step before the image to be encoded is bidirectionally predicted, and this scheme can directly determine the image type of the image to be encoded. For the current coding block, if the pixel value of its reference block in direction 1 is P1 and the pixel value of the reference block in direction 2 is P2, then the pixel value of the prediction block is P bi-pred , calculated as follows: P bi-pred =(ω*P1+(8-ω)*P2+4)>>3

[0037] Among them, >>3 means shifting right by 3 digits (i.e. dividing by 8).

[0038] The second direction provided by this solution is opposite to the first direction, and the first direction can be the forward direction of the unit to be encoded in the image to be encoded, or it can be the backward direction. Optionally, the first direction provided by this solution can be determined based on the unidirectional prediction result of the unit to be encoded, or during the encoding process of the current unit to be encoded, it can be determined based on the second direction determined by the last search for the candidate reference frame (i.e., the second direction determined by the last search for the candidate reference frame is used as the current first direction), and the second direction determined last is used as the new first direction for iteration.

[0039] Optionally, when performing weighted combination prediction for the first time on the coding unit to be coded, unidirectional prediction may be performed on the coding unit to determine the first direction and the corresponding prediction block corresponding to the unidirectional prediction result. Subsequently, when the determined first reference frame and the second reference frame do not meet the set search end condition, the second direction is used as the new first direction to iteratively re-determine the second reference frame and the second prediction block corresponding to the new second direction (i.e., the last first direction) until the most recently determined first reference frame and the second reference frame meet the set search end condition. In one embodiment, when performing unidirectional prediction on the coding unit to be coded, each reference frame in the two directions may be searched separately to find the prediction block with the minimum prediction cost for the current frame, and then the direction and reference frame corresponding to the prediction block are used to determine the unidirectional search result.

[0040] In one possible embodiment, as shown in a schematic diagram of a weight combination determination process provided in FIG2 , the video coding method based on bidirectional prediction provided by this solution may include, when determining multiple weight combinations based on a first reference frame in a first direction and multiple candidate reference frames in a second direction of a unit to be coded:

[0041] S111: Determine a first time domain sequence number difference between a first reference frame of a unit to be encoded in a first direction and the frame where the unit to be encoded is located, and a second time domain sequence number difference between a plurality of candidate reference frames of the unit to be encoded in a second direction and the frame where the unit to be encoded is located.

[0042] S112: Determine the opposite number of the weight ratio corresponding to each candidate reference frame, and determine the weight combination of each candidate reference frame according to the opposite number of the weight ratio, the first time domain sequence number difference, and the second time domain sequence number difference.

[0043] Exemplarily, a first frame sequence number corresponding to a first reference frame of the unit to be encoded in the first direction and a second frame sequence number of the frame in which the unit to be encoded is located are determined, and the absolute value of the difference between the first frame sequence number and the second frame sequence number is determined as a first temporal sequence number difference (the temporal sequence number may also be represented by a playback sequence number). Furthermore, a third frame sequence number corresponding to each of multiple candidate reference frames of the unit to be encoded in the second direction is determined, and the absolute value of the difference between each third frame sequence number and the second frame sequence number is determined as a second temporal sequence number difference corresponding to each candidate reference frame.

[0044] For example, assuming that the first frame number corresponding to the first reference frame of the unit to be coded in the first direction D1 is POC1, and the second frame number of the frame where the unit to be coded is located is POC0, then the first time domain number difference is: L1 = abs(POC1-POC0). Traverse each candidate reference frame of the unit to be coded in the second direction D2, assuming that the third frame number of the i-th candidate reference frame is POC 2i , then the second time domain sequence number difference corresponding to the i-th candidate reference frame is: L 2i =abs(POC 2i -POC0).

[0045] Determine the opposite number of the weight ratio corresponding to each candidate reference frame, where the opposite number of the weight ratio can be understood as the opposite number of the ratio of the weight corresponding to the candidate reference frame to the weight corresponding to the first reference frame. For example, when the weight corresponding to the first reference frame is ω, the weight corresponding to the candidate reference frame is (8-ω), then the opposite number of the weight ratio S is (8-ω) / ω. After determining the opposite number of the weight ratio corresponding to each candidate reference frame, further determine the weight combination corresponding to each candidate reference frame according to the opposite number of the weight ratio, the first time domain sequence number difference, and the second time domain sequence number difference. This scheme determines the weight combination corresponding to each candidate reference frame according to the first time domain sequence number difference between the first reference frame and the current frame, the second time domain sequence number difference between each candidate reference frame and the current frame, and the opposite number of the weight ratio, thereby improving the efficiency of determining the weight combination, reducing the number of bidirectional predictions, and effectively improving the coding efficiency.

[0046] In one embodiment, the present solution, when determining the inverse weight ratio value corresponding to each candidate reference frame and determining the weight combination of each candidate reference frame according to the inverse weight ratio value, the first time domain sequence number difference, and the second time domain sequence number difference, includes:

[0047] S1121: Determine multiple weight ratio inverse numbers corresponding to each candidate reference frame based on multiple first candidate weights in the first direction.

[0048] S1122: Determine the first distance value of the ratio of the inverse of multiple weight ratios corresponding to each candidate reference frame to the first time domain sequence number difference and the second time domain sequence number difference, and determine the weight combination of each candidate reference frame according to the first candidate weight and the second candidate weight corresponding to the minimum first distance value corresponding to each candidate reference frame, and different first candidate weights correspond to different second candidate weights.

[0049] Among them, different first candidate weights correspond to different second candidate weights. After determining the first candidate weight, the corresponding second candidate weight can be determined. For example, when the first candidate weight is ω in one direction, the corresponding second candidate weight is 8-ω.

[0050] Exemplarily, multiple second candidate weights corresponding to each candidate reference frame are determined based on multiple first candidate weights in the first direction, and multiple weight ratio inverse numbers corresponding to each candidate reference frame are determined based on the multiple first candidate weights and the corresponding second candidate weights. The number of weight ratio inverse numbers corresponding to each candidate reference frame is consistent with the number of first candidate weights.

[0051] Optionally, the weights corresponding to the low-latency images provided by this solution include: ω∈{-2,3,4,5,10}, and the inverse numbers of the weight ratios corresponding to the candidate reference frames are Optionally, for low-latency images, positive weights (i.e., ω∈{3,4,5}) can be used to determine the opposite number of the weight ratio, which can reduce the amount of data processing, improve the efficiency of weight combination determination, and improve image coding efficiency while ensuring the accuracy of bidirectional prediction; the weights corresponding to non-low-latency images include: ω∈{3,4,5}, and the opposite numbers of the weight ratios corresponding to the candidate reference frames are

[0052] For each candidate reference frame, a first distance value of the ratio of the inverse of the weight ratios corresponding to each candidate reference frame to the first time domain sequence number difference and the second time domain sequence number difference is calculated. The distance value is an absolute value. For example, the distance value corresponding to the jth first candidate weight corresponding to the i-th candidate reference frame is: Dif = abs(S j -L1 / L 2i ). Where L 2i is the second time domain sequence number difference of the i-th candidate reference frame, L1 is the first time domain sequence number difference, S j is the opposite of the weight ratio corresponding to the j-th first candidate weight.

[0053] In one embodiment, for each candidate reference frame, the minimum first distance value among the multiple first distance values ​​corresponding thereto is determined, and the first candidate weight and the second candidate weight corresponding to the minimum first distance value are determined, and the weight combination corresponding to the candidate reference frame is determined based on the combination of the first candidate weight and the second candidate weight. In one embodiment, when there are two identical minimum first distance values ​​for candidate reference frames corresponding to different first candidate weights, the combination of the first candidate weight and the second candidate weight that are closer in distance with equal weight (for example, equal weight of 4) can be used as the weight combination of the candidate reference frame (in this case, there is no combination of two first candidate weights and second candidate weights with the same distance with equal weight).

[0054] This solution accurately determines the weight combination of each candidate reference frame based on the first distance value of the ratio of the opposite numbers of the multiple weight ratios corresponding to each candidate reference frame to the first time domain sequence number difference and the second time domain sequence number difference, effectively improving the efficiency and accuracy of determining the weight combination, reducing the amount of data processing, and improving image coding efficiency.

[0055] S120: Search multiple candidate reference frames according to the weight combination, and determine a second reference frame and a second prediction block corresponding to the second direction according to the search results.

[0056] Exemplarily, according to the weight combination corresponding to each candidate reference frame determined above, each candidate reference frame is searched to determine the candidate search results corresponding to each candidate reference frame, and the reference frame and prediction block in the second direction indicated by each candidate search result are determined, and the complexity information corresponding to the reference frame and the first reference frame corresponding to each candidate search result is calculated, and the reference frame with the smallest complexity information is used as the second reference frame corresponding to the second direction, and the corresponding prediction block is determined as the second prediction block corresponding to the second direction. Optionally, the complexity information can be represented by a combination of one or more of SATD (Sum of Absolute Transformed Differences, the sum of the absolute values ​​of the residuals after Hadamard transform between the predicted coding and the original image) value, SAD (Sum of Absolute Differences, the sum of the absolute values ​​of the residuals after Hadamard transform between the predicted coding and the original image) value, or SSE (Sum of Squared Error, the sum of the squares of the residuals after predicted coding and the original image) value.

[0057] In one possible embodiment, as shown in a schematic diagram of a flow chart for determining a second reference frame and a second prediction block provided in FIG3 , the video encoding method based on bidirectional prediction provided by this solution may include searching for multiple candidate reference frames based on a weight combination and determining the second reference frame and the second prediction block corresponding to the second direction based on the search results:

[0058] S121: Determine the search target value of each candidate reference frame according to the weight combination.

[0059] S122: Search each candidate reference frame according to the search target value of each candidate reference frame to determine the candidate prediction block and first complexity information corresponding to each candidate reference frame.

[0060] S123: Use the candidate reference frame and the candidate prediction block corresponding to the minimum first complexity information as the second reference frame and the second prediction block corresponding to the second direction.

[0061] Exemplarily, the search target value of each candidate reference frame is calculated based on the weight combination corresponding to each candidate reference frame determined above. In one embodiment, the search target value provided by this solution can be determined based on the following target value determination formula:

[0062] Among them, P0 is the pixel value corresponding to the unit to be coded, P1 is the first prediction block of the unit to be coded in the first direction, ω i The first target weight in the weight combination is . This solution accurately determines the search target value through the target value determination formula, improves the search efficiency for candidate reference frames in the second direction, and improves video coding efficiency.

[0063] In one embodiment, after determining the search target value, each candidate reference frame is searched in the second direction based on the search target value of each candidate reference frame (for example, a candidate prediction block having a pixel value closest to the search target value is determined among the candidate reference frames in the second direction). Based on the search results, the candidate prediction block corresponding to each candidate reference frame and the first complexity information are determined in the second direction. Optionally, the first complexity information may be represented by a combination of one or more of a SATD value, a SAD value, or a SSE value.

[0064] In one embodiment, after determining the first complexity information corresponding to each candidate reference frame, the first complexity information of each candidate reference frame is compared to determine the minimum first complexity therein, and the candidate reference frame and candidate prediction block corresponding to the minimum first complexity information are used as the second reference frame and second prediction block corresponding to the second direction.

[0065] This solution determines the search target value of each candidate reference frame according to a weight combination and searches each candidate reference frame to determine the candidate prediction block and first complexity information corresponding to each candidate reference frame, and uses the candidate reference frame and candidate prediction block corresponding to the minimum first complexity information as the second reference frame and second prediction block corresponding to the second direction, thereby improving the efficiency of determining the second reference frame and the second prediction block and effectively improving the video coding efficiency.

[0066] S130: When the first reference frame and the second reference frame meet the set search end conditions, retest the first reference frame and the second reference frame to determine a first target weight corresponding to the first reference frame and a second target weight corresponding to the second reference frame.

[0067] Exemplarily, after each determination of the second reference frame, it is determined whether the first reference frame and the second reference frame meet a set search end condition. If the first reference frame and the second reference frame meet the set search end condition, the first reference frame and the second reference frame are retested to determine a first target weight corresponding to the first reference frame and a second target weight corresponding to the second reference frame based on the retest results.

[0068] In a possible embodiment, the video encoding method based on bidirectional prediction provided by the present solution further includes, before retesting the first reference frame and the second reference frame, when the first reference frame and the second reference frame meet the set search end condition, determining that the first reference frame and the second reference frame meet the set search end condition when the second complexity information corresponding to the search result is less than a set threshold and / or the search result of searching multiple candidate reference frames remains unchanged.

[0069] Exemplarily, after each time multiple candidate reference frames are searched according to a weight combination to determine the second reference frame and the second prediction block corresponding to the second direction, it is determined whether the second complexity information corresponding to the current search result is less than a set threshold, and / or whether the search result of searching multiple candidate reference frames remains unchanged relative to the search result of the last search of multiple candidate reference frames.

[0070] If the second complexity information corresponding to the search result is less than a set threshold and / or the search result of searching multiple candidate reference frames remains unchanged, it is determined that the first reference frame and the second reference frame meet the set search end condition. If the second complexity information corresponding to the search result is greater than or equal to the set threshold or the search result of searching multiple candidate reference frames changes, it is determined that the first reference frame and the second reference frame do not meet the set search end condition. Optionally, the second complexity information provided by this solution can be represented by a combination of one or more of the SATD value, SAD value or SSE value corresponding to the first reference frame and the second reference frame. In one possible embodiment, for a unit to be encoded, when the number of searches for multiple candidate reference frames based on the weight combination reaches a set number, it can be considered that the set search end condition is met. This solution determines whether the search end condition is met based on the second complexity information corresponding to the search result and / or the change in the search result relative to the previous search result, accurately determines the time to end the bidirectional search and enter the retest, and directly reduces the number of bidirectional searches and rate-distortion cost calculations while ensuring the accuracy of the bidirectional prediction, thereby effectively improving the coding efficiency.

[0071] In one possible embodiment, the video encoding method based on bidirectional prediction provided by the present scheme, after searching for multiple candidate reference frames according to a weight combination and determining the second reference frame and the second prediction block corresponding to the second direction according to the search results, also includes: when the first reference frame and the second reference frame do not meet the set search end conditions, taking the second direction as the new first direction, and determining a new second reference frame in the second direction based on the new first direction.

[0072] Exemplarily, when the first reference frame and the second reference frame do not meet the set search end condition, the second direction is used as the new first direction, the second reference frame and the second prediction block are used as the new first reference frame and the first prediction block, and steps S110-S120 are returned to re-determine the new second reference frame and the new second prediction block corresponding to the new second direction. This solution uses the original second direction and the second reference frame as the new first direction and the second reference frame to search for the second reference frame that meets the set search end condition when the first reference frame and the second reference frame do not meet the set search end condition. There is no need to traverse all directions and reference frames for bidirectional search. Each iteration of the bidirectional search of this solution updates the weight combination, and after the bidirectional search is completed, the weights of the first reference frame and the second reference frame are retested according to the search results. Under the premise of ensuring the accuracy of the bidirectional prediction, the number of bidirectional searches and rate-distortion cost calculations is directly reduced, effectively improving the coding efficiency.

[0073] In one possible embodiment, the video encoding method based on bidirectional prediction provided by the present solution re-measures the first reference frame and the second reference frame, and when determining the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame, it can be: determining the third complexity information of the first prediction block corresponding to the first reference frame and the second prediction block corresponding to the second reference frame under different weight combinations, and combining the weights corresponding to the minimum third complexity information to determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame.

[0074] Exemplarily, after determining the second reference frame and the second prediction block corresponding to the second direction, the third complexity information of the first prediction block corresponding to the first reference frame and the second prediction block corresponding to the second reference frame under different weight combinations (for example, 5 first candidate weights corresponding to the low-latency image and 5 weight combinations corresponding to the corresponding second candidate weights, or 3 first candidate weights corresponding to the non-low-latency image and 3 weight combinations corresponding to the corresponding second candidate weights) is calculated. Optionally, the third complexity information provided by this scheme can be represented by a combination of one or more SATD values, SAD values ​​or SSE values ​​corresponding to the first reference frame and the second reference frame. The minimum third complexity information among the multiple third complexity information is determined, and the weight combination corresponding to the minimum third complexity information is used to determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame. This scheme accurately determines the first target weight and the second target weight by comparing the third complexity information of the first prediction block and the second prediction block under different weight combinations, accurately and efficiently realizes the retesting of the weights of the first reference frame and the second reference frame, and directly reduces the number of bidirectional searches and rate-distortion cost calculations while ensuring the accuracy of bidirectional prediction, thereby effectively improving the coding efficiency.

[0075] S140: Perform encoding processing on the unit to be encoded based on the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame.

[0076] The first prediction block provided by this solution is determined according to the unidirectional prediction result of the coding unit, or the second prediction block corresponding to the second reference frame determined by the last search for the candidate reference frame.

[0077] Exemplarily, after determining the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame, based on the set CU-level weighted bidirectional prediction algorithm, the encoding unit to be encoded is processed according to the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame.

[0078] In one embodiment, the present solution provides a video encoding method based on bidirectional prediction, wherein, based on a first prediction block and a first target weight corresponding to a first reference frame, and a second prediction block and a second target weight corresponding to a second reference frame, encoding a unit to be encoded includes:

[0079] S141: Determine a first rate-distortion optimization cost according to a first prediction block and a first target weight corresponding to a first reference frame, and a second prediction block and a second target weight corresponding to a second reference frame.

[0080] S142: When it is determined that the first rate-distortion optimization cost satisfies the bidirectional prediction condition, encoding processing is performed on the coding unit based on the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame.

[0081] Exemplarily, a first rate-distortion optimization cost is calculated based on the first prediction block and first target weight corresponding to the first reference frame, and the second prediction block and second target weight corresponding to the second reference frame. The first rate-distortion optimization cost is then compared with the second rate-distortion optimization cost corresponding to other coding modes (which can be output by the prediction module of the other coding modes). If the first rate-distortion optimization cost is less than the second rate-distortion optimization cost, it is determined that the first rate-distortion optimization cost meets the bidirectional prediction condition. The coding unit to be coded is then encoded based on the first prediction block and first target weight corresponding to the first reference frame, and the second prediction block and second target weight corresponding to the second reference frame. If the first rate-distortion optimization cost is greater than the second rate-distortion optimization cost, it is determined that the first rate-distortion optimization cost does not meet the bidirectional prediction condition. The coding unit to be coded can be encoded based on the coding mode corresponding to the minimum second rate-distortion optimization cost. This solution effectively improves video coding efficiency while ensuring video coding quality by determining the first rate-distortion optimization cost based on the first prediction block, first target weight, second prediction block, and second target weight, and encoding the coding unit based on the bidirectional search result determined by the dual number when the bidirectional prediction condition is met.

[0082] In the above, multiple weight combinations are determined based on the first reference frame in the first direction and multiple candidate reference frames in the second direction of the unit to be encoded, the multiple candidate reference frames are searched according to the weight combinations to determine the second reference frame and the second prediction block corresponding to the second direction, and the first reference frame and the second reference frame are retested when the first reference frame and the second reference frame meet the set search end condition to determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame, and the unit to be encoded is encoded according to the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame, in the bidirectional prediction process, multiple weight combinations are preselected based on the first reference frame in the first direction and the candidate reference frames in the second direction, the second reference frame and the second prediction block are preselected according to the weight combinations, and the second reference frame and the second prediction block are iteratively updated each time the search end condition is not met, and the search results are retested to determine the first target weight and the second target weight when the search end condition is met, thereby effectively reducing the number of bidirectional prediction and rate-distortion optimization calculations and improving video coding efficiency.

[0083] FIG4 is a schematic diagram of the structure of a video encoding device based on bidirectional prediction provided by an embodiment of the present application. Referring to FIG4 , the video encoding device based on bidirectional prediction includes a weight prediction module 41 , a reference frame determination module 42 , a weight determination module 43 and an encoding processing module 44 .

[0084] Among them, the weight prediction module 41 is configured to determine multiple weight combinations based on the first reference frame of the unit to be encoded in the first direction and multiple candidate reference frames in the second direction, the second direction is opposite to the first direction, and the first direction is determined according to the unidirectional prediction result of the unit to be encoded, or the second direction determined by the last search for the candidate reference frame; the reference frame determination module 42 is configured to search multiple candidate reference frames according to the weight combination, and determine the second reference frame and the second prediction block corresponding to the second direction according to the search results; the weight determination module 43 is configured to retest the first reference frame and the second reference frame when the first reference frame and the second reference frame meet the set search end conditions, and determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame; the encoding processing module 44 is configured to perform encoding processing on the unit to be encoded based on the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame, and the first prediction block is determined according to the unidirectional prediction result of the unit to be encoded, or the second prediction block corresponding to the second reference frame determined by the last search for the candidate reference frame.

[0085] In the above, multiple weight combinations are determined based on the first reference frame in the first direction and multiple candidate reference frames in the second direction of the unit to be encoded, the multiple candidate reference frames are searched according to the weight combinations to determine the second reference frame and the second prediction block corresponding to the second direction, and the first reference frame and the second reference frame are retested when the first reference frame and the second reference frame meet the set search end condition to determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame, and the unit to be encoded is encoded according to the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame, in the bidirectional prediction process, multiple weight combinations are preselected based on the first reference frame in the first direction and the candidate reference frames in the second direction, the second reference frame and the second prediction block are preselected according to the weight combinations, and the second reference frame and the second prediction block are iteratively updated each time the search end condition is not met, and the search results are retested to determine the first target weight and the second target weight when the search end condition is met, thereby effectively reducing the number of bidirectional prediction and rate-distortion optimization calculations and improving video coding efficiency.

[0086] In a possible embodiment, when the weight prediction module 41 determines multiple weight combinations based on the first reference frame in the first direction and the multiple candidate reference frames in the second direction of the unit to be coded, it is configured as follows:

[0087] Determine a first time domain sequence number difference between a first reference frame of a unit to be encoded in a first direction and a frame where the unit to be encoded is located, and a second time domain sequence number difference between a plurality of candidate reference frames of the unit to be encoded in a second direction and the frame where the unit to be encoded is located;

[0088] The opposite number of the weight ratio corresponding to each candidate reference frame is determined, and the weight combination of each candidate reference frame is determined according to the opposite number of the weight ratio, the first time domain sequence number difference, and the second time domain sequence number difference.

[0089] In one possible embodiment, the weight prediction module 41 is configured as follows when determining the inverse weight ratio corresponding to each candidate reference frame and determining the weight combination of each candidate reference frame according to the inverse weight ratio, the first temporal sequence number difference, and the second temporal sequence number difference:

[0090] Determine, based on the plurality of first candidate weights in the first direction, a plurality of weight ratio inverses corresponding to each candidate reference frame;

[0091] Determine the first distance value of the ratio of the inverse of multiple weight ratios corresponding to each candidate reference frame to the first time domain sequence number difference and the second time domain sequence number difference, and determine the weight combination of each candidate reference frame according to the first candidate weight and the second candidate weight corresponding to the minimum first distance value corresponding to each candidate reference frame, and different first candidate weights correspond to different second candidate weights.

[0092] In a possible embodiment, when the reference frame determination module 42 searches for multiple candidate reference frames according to the weight combination and determines the second reference frame and the second prediction block corresponding to the second direction according to the search results, the module is configured as follows:

[0093] Determine the search target value of each candidate reference frame according to the weight combination;

[0094] Searching each candidate reference frame according to a search target value of each candidate reference frame to determine a candidate prediction block and first complexity information corresponding to each candidate reference frame;

[0095] The candidate reference frame and the candidate prediction block corresponding to the minimum first complexity information are used as the second reference frame and the second prediction block corresponding to the second direction.

[0096] In one possible embodiment, the search target value is determined based on the following formula:

[0097] Among them, P0 is the pixel value corresponding to the unit to be coded, P1 is the first prediction block of the unit to be coded in the first direction, ω i is the first target weight in the weight combination.

[0098] In one possible embodiment, the video encoding device based on bidirectional prediction also includes a search end judgment module, which is configured to determine whether the first reference frame and the second reference frame meet the set search end condition when the second complexity information corresponding to the search result is less than a set threshold and / or the search result of searching multiple candidate reference frames remains unchanged.

[0099] In one possible embodiment, when the weight determination module 43 remeasures the first reference frame and the second reference frame to determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame, the weight determination module 43 is configured as follows:

[0100] Determine the third complexity information of the first prediction block corresponding to the first reference frame and the second prediction block corresponding to the second reference frame under different weight combinations, and determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame by combining the weights corresponding to the minimum third complexity information.

[0101] In one possible embodiment, the video encoding device based on bidirectional prediction also includes an iterative search module, which is configured to use the second direction as the new first direction when the first reference frame and the second reference frame do not meet the set search end condition, and determine the second reference frame of the new second direction based on the new first direction.

[0102] In one possible embodiment, when encoding the unit to be encoded based on the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame, the encoding processing module 44 is configured as follows:

[0103] Determining a first rate-distortion optimization cost based on a first prediction block and a first target weight corresponding to the first reference frame, and a second prediction block and a second target weight corresponding to the second reference frame;

[0104] When it is determined that the first rate-distortion optimization cost satisfies the bidirectional prediction condition, encoding processing is performed on the coding unit based on the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame.

[0105] It is worth noting that in the above-mentioned embodiment of the video encoding device based on bidirectional prediction, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0106] The embodiment of the present application also provides a video encoding device based on bidirectional prediction, and the video encoding device based on bidirectional prediction can integrate the video encoding apparatus based on bidirectional prediction provided by the embodiment of the present application. Figure 5 is a structural diagram of a video encoding device based on bidirectional prediction provided by the embodiment of the present application. Referring to Figure 5, the video encoding device based on bidirectional prediction includes: an input device 53, an output device 54, a memory 52 and one or more processors 51; the memory 52 is used to store one or more programs; when the one or more programs are executed by one or more processors 51, the one or more processors 51 implement the video encoding method based on bidirectional prediction as provided in the above embodiment. The above-mentioned video encoding apparatus, device and computer based on bidirectional prediction can be used to execute the video encoding method based on bidirectional prediction provided by any of the above embodiments, and have corresponding functions and beneficial effects.

[0107] The embodiment of the present application also provides a non-volatile storage medium that stores computer-executable instructions, and the computer-executable instructions are used to execute the video encoding method based on bidirectional prediction as provided in the above embodiment when executed by a computer processor. Of course, the non-volatile storage medium that stores computer-executable instructions provided in the embodiment of the present application, its computer-executable instructions are not limited to the video encoding method based on bidirectional prediction provided above, and can also execute related operations in the video encoding method based on bidirectional prediction provided in any embodiment of the present application. The video encoding apparatus, device and storage medium based on bidirectional prediction provided in the above embodiment can execute the video encoding method based on bidirectional prediction provided in any embodiment of the present application. For technical details not described in detail in the above embodiment, please refer to the video encoding method based on bidirectional prediction provided in any embodiment of the present application.

[0108] Based on the above embodiments, the embodiments of the present application also provide a computer program product. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the video encoding method based on bidirectional prediction provided in each embodiment of the present application.

Claims

1. A video coding method based on bidirectional prediction, wherein: include: Determine multiple weight combinations according to a first reference frame of the unit to be encoded in a first direction and multiple candidate reference frames in a second direction, wherein the second direction is opposite to the first direction, and the first direction is determined according to a unidirectional prediction result of the unit to be encoded, or a second direction determined by searching the candidate reference frames last time; Searching the plurality of candidate reference frames according to the weight combination, and determining a second reference frame and a second prediction block corresponding to the second direction according to the search results; When the first reference frame and the second reference frame meet the set search end condition, retest the first reference frame and the second reference frame to determine a first target weight corresponding to the first reference frame and a second target weight corresponding to the second reference frame; The unit to be encoded is encoded based on a first prediction block and a first target weight corresponding to the first reference frame, and a second prediction block and a second target weight corresponding to the second reference frame, wherein the first prediction block is determined according to a unidirectional prediction result of the unit to be encoded, or a second prediction block corresponding to the second reference frame determined by the last search for the candidate reference frame.

2. The video encoding method based on bidirectional prediction according to claim 1, wherein: The method of determining a plurality of weight combinations according to a first reference frame in a first direction and a plurality of candidate reference frames in a second direction of a unit to be encoded comprises: Determine a first time domain sequence number difference between a first reference frame of a unit to be encoded in a first direction and a frame where the unit to be encoded is located, and a second time domain sequence number difference between a plurality of candidate reference frames of the unit to be encoded in a second direction and the frame where the unit to be encoded is located; Determine the opposite number of the weight ratio corresponding to each of the candidate reference frames, and determine the weight combination of each of the candidate reference frames according to the opposite number of the weight ratio, the first time domain sequence number difference, and the second time domain sequence number difference.

3. The video encoding method based on bidirectional prediction according to claim 2, wherein: The step of determining the opposite number of the weight ratio corresponding to each candidate reference frame, and determining the weight combination of each candidate reference frame according to the opposite number of the weight ratio, the first time domain sequence number difference, and the second time domain sequence number difference includes: Determine a plurality of weight ratio inverse numbers corresponding to each of the candidate reference frames according to a plurality of first candidate weights in the first direction; Determine a first distance value of a ratio of a plurality of inverse numbers of the weight ratios corresponding to each of the candidate reference frames to the first time domain sequence number difference and the second time domain sequence number difference, and The first candidate weight corresponding to the minimum first distance value corresponding to the frame and the second candidate weight determine the weight combination of each candidate reference frame, and different first candidate weights correspond to different second candidate weights.

4. The video encoding method based on bidirectional prediction according to claim 1, wherein: The step of searching the plurality of candidate reference frames according to the weight combination, and determining the second reference frame and the second prediction block corresponding to the second direction according to the search results, includes: Determine the search target value of each of the candidate reference frames according to the weight combination; Searching each of the candidate reference frames according to the search target value of each of the candidate reference frames, and determining a candidate prediction block and first complexity information corresponding to each of the candidate reference frames; The candidate reference frame and the candidate prediction block corresponding to the minimum first complexity information are used as the second reference frame and the second prediction block corresponding to the second direction.

5. The video encoding method based on bidirectional prediction according to claim 4, wherein: The search target value is determined based on the following formula: Wherein, P0 is the pixel value corresponding to the unit to be encoded, P1 is the first prediction block of the unit to be encoded in the first direction, ω i is the first target weight in the weight combination.

6. The video encoding method based on bidirectional prediction according to claim 1, wherein: Before retesting the first reference frame and the second reference frame when the first reference frame and the second reference frame meet the set search end condition, the method further includes: When the second complexity information corresponding to the search result is less than a set threshold and / or the search result of searching the multiple candidate reference frames remains unchanged, it is determined that the first reference frame and the second reference frame meet the set search end condition.

7. The video encoding method based on bidirectional prediction according to claim 1, wherein: The retesting the first reference frame and the second reference frame to determine a first target weight corresponding to the first reference frame and a second target weight corresponding to the second reference frame includes: Determine the third complexity information of the first prediction block corresponding to the first reference frame and the second prediction block corresponding to the second reference frame under different weight combinations, and determine the first target weight corresponding to the first reference frame and the second target weight corresponding to the second reference frame by combining the weights corresponding to the minimum third complexity information.

8. The video encoding method based on bidirectional prediction according to any one of claims 1 to 7, wherein: After searching the plurality of candidate reference frames according to the weight combination and determining the second reference frame and the second prediction block corresponding to the second direction according to the search results, the method further includes: When the first reference frame and the second reference frame do not satisfy the set search end condition, the second direction is used as a new first direction, and a second reference frame in a new second direction is determined based on the new first direction.

9. The video encoding method based on bidirectional prediction according to any one of claims 1 to 7, wherein: The encoding process of the unit to be encoded based on the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame, includes: Determining a first rate-distortion optimization cost according to a first prediction block and a first target weight corresponding to the first reference frame, and a second prediction block and a second target weight corresponding to the second reference frame; When it is determined that the first rate-distortion optimization cost satisfies the bidirectional prediction condition, the unit to be encoded is encoded based on the first prediction block and the first target weight corresponding to the first reference frame, and the second prediction block and the second target weight corresponding to the second reference frame.

10. A video encoding device based on bidirectional prediction, wherein: It includes a weight prediction module, a reference frame determination module, a weight determination module and an encoding processing module, wherein: The weight prediction module is configured to determine a plurality of weight combinations according to a first reference frame of the unit to be encoded in a first direction and a plurality of candidate reference frames in a second direction, wherein the second direction is opposite to the first direction, and the first direction is determined according to a unidirectional prediction result of the unit to be encoded, or a second direction determined by searching for the candidate reference frames last time; The reference frame determination module is configured to search the plurality of candidate reference frames according to the weight combination, and determine the second reference frame and the second prediction block corresponding to the second direction according to the search results; The weight determination module is configured to retest the first reference frame and the second reference frame to determine a first target weight corresponding to the first reference frame and a second target weight corresponding to the second reference frame when the first reference frame and the second reference frame meet a set search end condition; The encoding processing module is configured to perform encoding processing on the unit to be encoded based on a first prediction block and a first target weight corresponding to the first reference frame, and a second prediction block and a second target weight corresponding to the second reference frame, wherein the first prediction block is determined based on a unidirectional prediction result of the unit to be encoded, or a second prediction block corresponding to the second reference frame determined by the last search for the candidate reference frame.

11. A video encoding device based on bidirectional prediction, wherein: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the video encoding method based on bidirectional prediction as described in any one of claims 1-9.

12. A non-volatile storage medium storing computer executable instructions, wherein: When the computer executable instructions are executed by a computer processor, they are used to perform the video encoding method based on bidirectional prediction as described in any one of claims 1 to 9.

13. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the video encoding method based on bidirectional prediction according to any one of claims 1 to 9 is implemented.

Images