Video processing method and related equipment

Abstract

The embodiment of the invention provides a video processing method and related equipment, and the method comprises the steps: carrying out the composite prediction of a current block in a video code stream, and obtaining N reference prediction values of the current block, N being an integer greater than 1; according to the importance of the N reference predicted values, determining a target weight group for weighted prediction for the current block, the target weight group comprising one or more weight values; and performing weighted prediction processing on the N reference predicted values based on the target weight group to obtain a predicted value of the current block, the predicted value of the current block being used for reconstructing a decoded image corresponding to the current block. According to the embodiment of the invention, the prediction accuracy of the coding block can be improved, so that the coding performance is improved.

Description

Technical Field

[0001] The present application relates to the field of audio and video technology, specifically to the field of video coding technology, and more particularly to a video processing method, a video processing device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Existing video coding technologies employ a block-based hybrid coding framework, dividing raw video data into a series of coding units (CUs). This framework combines prediction, transform, and entropy coding techniques to achieve video data compression. To achieve better prediction results, mainstream video coding standards such as AV1 (the first-generation video coding standard developed by the Alliance for Open Media, Video 1) and the next-generation AV2 (the second-generation video coding standard developed by the Alliance for Open Media, Video 2) currently under development by the Alliance for Open Media (AOM), include a prediction mode called compound prediction. This compound prediction mode allows for weighted prediction using multiple reference video signals. However, in practice, compound prediction has been found to be inaccurate when used for weighted prediction. Summary of the Invention

[0003] The embodiments of the present application provide a video processing method and related equipment, which can improve the accuracy of weighted prediction in the composite prediction process, thereby improving encoding performance.

[0004] In one aspect, an embodiment of the present application provides a video processing method, the method comprising:

[0005] Perform composite prediction on the current block in the video stream to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1;

[0006] Determining a target weight group for weighted prediction for the current block based on the importance of the N reference prediction values, the target weight group including one or more weight values;

[0007] The N reference prediction values ​​are weightedly predicted based on the target weight group to obtain the prediction value of the current block. The prediction value of the current block is used to reconstruct the decoded image corresponding to the current block.

[0008] In one aspect, an embodiment of the present application provides a video processing method, the method comprising:

[0009] Divide the current frame in the video to obtain the current block;

[0010] Perform composite prediction on the current block to obtain N reference prediction values ​​of the current block;

[0011] Determining a target weight group for weighted prediction for the current block based on the importance of the N reference prediction values, the target weight group including one or more weight values;

[0012] Performing weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block;

[0013] The video is encoded based on the predicted value of the current block to generate a video bitstream.

[0014] In one aspect, an embodiment of the present application provides a video processing device, comprising:

[0015] A processing unit, configured to perform composite prediction on a current block in a video stream to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1;

[0016] a determining unit, configured to determine a target weight group for weighted prediction for a current block according to the importance of the N reference prediction values, the target weight group comprising one or more weight values;

[0017] The processing unit is further configured to perform weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block.

[0018] In one embodiment, the N reference prediction values ​​are derived from N reference blocks; one reference prediction value corresponds to one reference block; the video frame where the reference block is located is the reference frame, and the video frame where the current block is located is the current frame;

[0019] The positional relationship between the N reference blocks and the current block includes any of the following:

[0020] The N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame belong to different video frames in the video stream;

[0021] The N reference blocks are located in the same reference frame, and the same reference frame and the current frame belong to different video frames in the video stream;

[0022] One or more of the N reference blocks are located in the current frame, and the remaining reference blocks in the N reference blocks are located in one or more reference frames, and the one or more reference frames and the current frame belong to different video frames in the video stream;

[0023] The N reference blocks and the current block are all located in the current frame.

[0024] In one embodiment, the processing unit is further configured to:

[0025] Determine whether the current block meets the conditions for adaptive weighted prediction;

[0026] If the current block meets the conditions for adaptive weighted prediction, a target weight group for weighted prediction is determined for the current block based on the importance of the N reference prediction values.

[0027] In one embodiment, the condition that the current block satisfies adaptive weighted prediction includes at least one of the following:

[0028] The sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that the coding block in the frame sequence allows the use of adaptive weighted prediction;

[0029] The slice header of the current slice to which the current block belongs includes a second indication field, and the second indication field indicates that the coding blocks in the current slice are allowed to use adaptive weighted prediction;

[0030] The frame header of the current frame where the current block is located includes a third indication field, and the third indication field indicates that the coding block in the current frame allows the use of adaptive weighted prediction;

[0031] In the composite prediction process, the current block uses at least two reference frames for inter-frame prediction;

[0032] In the composite prediction process, the current block uses at least one reference frame for inter prediction and the current frame for intra prediction;

[0033] The motion type of the current block is the specified motion type;

[0034] The current block uses a preset motion vector prediction mode;

[0035] The current block uses a preset interpolation filter;

[0036] The current block does not use a specific encoding tool;

[0037] The reference frame used in the composite prediction process of the current block meets specific conditions, which include one or more of the following: the orientation relationship between the used reference frame and the current frame in the video stream meets a preset relationship; the absolute value of the importance difference between the reference prediction values ​​corresponding to the used reference frames is greater than or equal to a preset threshold;

[0038] Among them, the orientation relationship satisfies the preset relationship including any of the following: all the reference frames used are located before the current frame; all the reference frames used are located after the current frame; some of the reference frames used are located before the current frame, and the remaining reference frames are located after the current frame.

[0039] In one embodiment, the video stream includes one or more weight lists, each weight list includes one or more weight groups, each weight group includes one or more weight values, the number of weight values ​​included in each weight group can be the same or different, and the weight values ​​included in each weight group can be the same or different; the determining unit is specifically configured to:

[0040] determining a target weight list from one or more weight lists based on the importance of the N reference prediction values; and

[0041] From the Target Weights list, select the target weight set to use for weighted predictions.

[0042] In one embodiment, the number of weight lists in the video stream is M+1, one weight list corresponds to one threshold interval, and M is a positive integer greater than or equal to 1; the determining unit is specifically configured to:

[0043] Obtain importance metrics of N reference prediction values, and calculate the importance differences between the N reference prediction values. The importance difference between any two reference prediction values ​​is measured by the difference between the importance metrics of the any two reference prediction values.

[0044] Determine a threshold interval where the absolute value of the significance difference between N reference prediction values ​​lies;

[0045] The weight list corresponding to the threshold interval where the absolute value of the importance difference between the N reference prediction values ​​lies is determined as the target weight list.

[0046] In one embodiment, the N reference prediction values ​​of the current block include a first reference prediction value and a second reference prediction value; the video stream includes a first weight list and a second weight list; and the determining unit is specifically configured to:

[0047] comparing the importance metric value of the first reference prediction value with the importance metric value of the second reference prediction value;

[0048] If the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, determining the first weight list as the target weight list;

[0049] If the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, determining the second weight list as the target weight list;

[0050] The weight values ​​in the first weight list are opposite to the weight values ​​in the second weight list.

[0051] In one embodiment, the N reference prediction values ​​of the current block include a first reference prediction value and a second reference prediction value; the video stream includes a first weight list, a second weight list, and a third weight list; and the determining unit is specifically configured to:

[0052] Calling a mathematical sign function to process a difference between the importance measure value of the first reference prediction value and the importance measure value of the second reference prediction value to obtain a sign value;

[0053] If the symbol value is the first preset value, determining the first weight list as the target weight list;

[0054] If the symbol value is the second preset value, determining the second weight list as the target weight list;

[0055] If the symbol value is the third preset value, determining the third weight list as the target weight list;

[0056] The first weight list, the second weight list and the third weight list are different weight lists respectively; or, two weight lists among the first weight list, the second weight list and the third weight list are allowed to be the same weight list.

[0057] In one embodiment, any one of the N reference prediction values ​​is denoted as reference prediction value i, where reference prediction value i is derived from reference block i, and the video frame where reference block i is located is reference frame i; i is an integer and i is less than or equal to N; and the video frame where the current block is located is the current frame;

[0058] The importance measure of the reference prediction value i is determined using any of the following methods:

[0059] Method 1: Calculated based on the frame display order of the current frame in the video stream and the frame display order of the reference frame i in the video stream;

[0060] Method 2: Calculated based on the frame display order of the current frame in the video stream, the frame display order of the reference frame i in the video stream, and the quality metric Q;

[0061] Method 3: Calculate the importance metric score of reference frame i based on the calculation results of method 1 and method 2, sort the importance metric scores of the reference frames corresponding to the N reference prediction values ​​in ascending order, and determine the index of reference frame i in the sort as the importance metric value of reference prediction value i;

[0062] Method 4: Based on the prediction model of the reference prediction value i, the calculation results of method 1, method 2 or method 3 are adjusted to obtain the importance measurement value of the reference prediction value i;

[0063] The prediction mode of the reference prediction value i includes any one of the following: inter-frame prediction mode and intra-frame prediction mode.

[0064] In one embodiment, the number of weight groups included in the target weight list is greater than 1; and the determining unit is specifically configured to:

[0065] Decoding the index of the target weight group for weighted prediction from the video bitstream;

[0066] According to the index of the target weight group, select the target weight group from the target weight list;

[0067] The index of the target weight group is encoded using a truncated unary code for binary encoding, or a multi-symbol entropy encoding method.

[0068] In one embodiment, the processing unit is specifically configured to:

[0069] The weight values ​​in the target weight group are used to perform weighted summation on the N reference prediction values ​​to obtain the prediction value of the current block; or

[0070] The N reference prediction values ​​are weighted by using the weight values ​​in the target weight group in the form of integer calculation to obtain the prediction value of the current block.

[0071] In one aspect, an embodiment of the present application provides a video processing device, comprising:

[0072] A processing unit, configured to divide a current frame in the video into a current block, where N is an integer greater than 1;

[0073] The processing unit is further configured to perform composite prediction on the current block to obtain N reference prediction values ​​of the current block;

[0074] a determining unit, configured to determine a target weight group for weighted prediction for a current block according to the importance of the N reference prediction values, the target weight group comprising one or more weight values;

[0075] The processing unit is further configured to perform weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, wherein the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block;

[0076] The processing unit is further configured to encode the video based on the prediction value of the current block to generate a video code stream.

[0077] In one embodiment, the determining unit is specifically configured to:

[0078] According to the importance of the N reference prediction values, a target weight list is determined from one or more weight lists; each weight list contains one or more weight groups, each weight group contains one or more weight values, the number of weight values ​​contained in each weight group can be the same or different, and the values ​​of the weight values ​​contained in each weight group can be the same or different;

[0079] From the Target Weights list, select the target weight set to use for weighted predictions.

[0080] In one embodiment, the bit rate consumed when performing weighted prediction processing on N reference prediction values ​​using the weight values ​​in the target weight group is less than a preset bit rate threshold; or,

[0081] Performing weighted prediction processing on the N reference prediction values ​​based on the weight values ​​in the target weight group so that the quality loss of the current block during the encoding process is less than a preset loss threshold; or

[0082] The bit rate consumed when performing weighted prediction processing on N reference prediction values ​​using the weight values ​​in the target weight group is less than a preset bit rate threshold, and weighted prediction processing is performed on the N reference prediction values ​​based on the weight values ​​in the target weight group, so that the quality loss of the current block during the encoding process is less than the preset loss threshold.

[0083] In another embodiment, the target weight group is the weight group with the best encoding performance in the target weight list;

[0084] Among them, the optimal coding performance includes: the bit rate consumed when the weight values ​​in the target weight group are used to perform weighted prediction processing on N reference prediction values ​​is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list; or, the quality loss of the corresponding current block in the encoding process when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list; or, the bit rate consumed when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list, and the quality loss of the corresponding current block in the encoding process when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list.

[0085] In one aspect, an embodiment of the present application provides a computer device, comprising:

[0086] a processor adapted to execute a computer program;

[0087] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the video processing method described above is implemented.

[0088] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is loaded by a processor and executes the above-mentioned video processing method.

[0089] In one aspect, embodiments of the present application provide a computer program product, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the above-described video processing method.

[0090] In an embodiment of the present application, a composite prediction is performed on the current block in a video stream to obtain N reference prediction values ​​for the current block. The current block refers to the coded block being decoded in the video stream. Based on the importance of the N reference prediction values, a target weight group for weighted prediction is determined for the current block. Based on the target weight group, a weighted prediction process is performed on the N reference prediction values ​​to obtain a prediction value for the current block. The prediction value for the current block is used to reconstruct the decoded image corresponding to the current block. Composite prediction is used in the video decoding process, and the importance of the reference prediction values ​​is fully considered in the composite prediction. Weighted prediction is performed by determining appropriate weights for the current block based on the importance of each reference prediction value in the composite prediction. This can improve the prediction accuracy of the current block, thereby improving codec performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0092] Figure 1a This is a basic workflow diagram of video encoding provided by an exemplary embodiment of the present application;

[0093] Figure 1b is a schematic diagram of inter-frame prediction provided by an exemplary embodiment of the present application;

[0094] Figure 2 is a structural diagram of a video processing system provided by an exemplary embodiment of the present application;

[0095] Figure 3is a flowchart of a video processing method provided by an exemplary embodiment of the present application;

[0096] Figure 4 is a flowchart of a video processing method provided by another exemplary embodiment of the present application;

[0097] Figure 5 is a structural diagram of a video processing device provided by an exemplary embodiment of the present application;

[0098] Figure 6 is a structural diagram of a video processing device provided by another exemplary embodiment of the present application;

[0099] Figure 7 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0100] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0101] The following is an introduction to the technical terms involved in this application:

[0102] 1. Video Encoding

[0103] A video can be composed of one or more frames, each of which contains a portion of the video signal. Video signals can be captured by a camera or generated by a computer. Due to the different statistical characteristics associated with these acquisition methods, the compression encoding methods used for the video may also differ.

[0104] Among mainstream video coding technologies, HEVC (High Efficiency Video Coding, international video coding standard HEVC / H.265), VVC (versatile video coding, international video coding standard VVC / H.266), AV1 (Alliance for Open Media Video 1, first-generation video coding standard developed by the Alliance for Open Media), AV2 (Alliance for Open Media Video 2, second-generation video coding standard developed by the Alliance for Open Media), and AVS3 (Audio and Video Source Coding Standard) all use a hybrid coding framework that allows the following operations and processing on videos:

[0105] 1) Block partition structure: According to the size of the current input frame (i.e., the video frame being encoded or decoded), the current frame can be divided into several non-overlapping processing units, and each processing unit will perform similar compression operations. This processing unit is called CTU (coding tree unit) or LCU (Largest Coding Unit). The CTU can be further divided into more refined parts to obtain one or more basic coding units, called CUs (coding units or coding blocks). Each CU is the most basic element in a coding link. The subsequent embodiments of this application describe various coding and decoding processing flows that may be used for each CU.

[0106] 2) Prediction coding: including intra-frame prediction and inter-frame prediction modes. The original video signal contained in the current CU in the current frame (i.e., the CU being encoded or decoded in the current frame) is predicted by the reconstructed video signal in the selected reference CU to obtain a residual video signal. Here, the current CU can also be called the current block, the video frame where the current block is located is called the current frame, the reference CU used for predicting the current block can also be called the reference block of the current block, and the video frame where the reference block is located is called the reference frame. Among them, the encoder needs to decide for the current CU to select the most appropriate prediction coding mode from many possible prediction coding modes and tell the decoder. Among them, the prediction coding mode may include:

[0107] a. Intra-picture Prediction: The reconstructed video signal used for prediction comes from an already coded and reconstructed area within the same video frame. That is, the current block and the reference block are located in the same video frame. The basic idea of ​​intra-frame prediction is to remove spatial redundancy by leveraging the correlation between adjacent pixels within the same video frame. In video coding, adjacent pixels refer to the reconstructed pixels of the coded CU surrounding the current CU within the same video frame.

[0108] b. Inter-picture prediction: The reconstructed video signal used for prediction comes from other video frames that have been encoded and are different from the current frame. That is, the current block and the reference block are located in different video frames.

[0109] 3) Transform Coding and Quantization: The residual video signal undergoes transforms such as the Discrete Fourier Transform (DFT) and Discrete Cosine Transform (DCT) to convert it into a transform domain, known as transform coefficients. The residual video signal in the transform domain undergoes a lossy quantization operation, which loses some information, making the quantized signal more suitable for compression.

[0110] In some video coding standards, there may be more than one transform method to choose from. Therefore, the encoder also needs to select one of the transforms for the current CU and inform the decoder. The degree of quantization is usually determined by QP (Quantization Parameters). When the QP value is large, the transform coefficients representing a larger value range will be quantized to the same output, which usually results in greater distortion and lower bit rate. Conversely, when the QP value is small, the transform coefficients representing a smaller value range will be quantized to the same output, which usually results in less distortion and correspondingly higher bit rate.

[0111] 4) Entropy Coding or Statistical Coding: The quantized transform domain signal is statistically compressed and encoded based on the frequency of each value, and a binary (0 or 1) video stream is output. At the same time, the encoding generates other information, such as the selected predictive coding mode and motion vectors. This other information also needs to be entropy encoded to reduce the bit rate. Statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).

[0112] 5) Loop Filtering: The decoded image corresponding to the encoded CU can be reconstructed after undergoing inverse quantization, inverse transformation and prediction compensation (the inverse operations of 2) to 4) above. Compared with the original image, the reconstructed decoded image has some information that is different from the original image due to the influence of quantization, resulting in distortion. Therefore, a filter can be used to filter the reconstructed decoded image to effectively reduce the degree of distortion caused by quantization. The filter can be, for example, a deblocking filter, SAO (sample adaptive offset) or ALF (adaptive loop filter), etc. Since these filtered reconstructed decoded images will serve as reference CUs for other CUs that need to be encoded subsequently and are used in the prediction process of other CUs, the above filtering operation is also called loop filtering, and the filtering operation within the encoding loop.

[0113] Based on the description of steps 1)-5) above, the present embodiment provides a basic workflow diagram of a video encoder. Figure 1a , Figure 1a The following is a basic working flow diagram of a video encoder. Figure 1a The current block is the kth CU in the current frame (current image) (e.g. Figure 1a The s shown k [x,y]) is used as an example for illustration, k is a positive integer, and k is less than or equal to the total number of CUs contained in the current frame. k [x, y] represents the pixel point (referred to as pixel) with coordinates [x, y] in the k-th CU, where x represents the horizontal coordinate of the pixel and y represents the vertical coordinate of the pixel; s k [x,y] can obtain the prediction signal after motion compensation or intra-frame prediction. and the original signal s k [x,y] is subtracted to obtain the residual video signal u k [x,y]; then the residual video signal u k [x,y] is transformed and quantized. The output data of the quantization process has two different destinations, A and B:

[0114] A: The data output by the quantization process can be sent to an entropy encoder for entropy encoding to obtain an encoded bit stream (i.e., a video bit stream), which is then output to a buffer for storage and waiting to be transmitted.

[0115] B. The data output by quantization processing can be dequantized and inversely transformed to obtain the residual video signal u after inverse transformation.′ k [x,y]. Then, the inverse transformed residual video signal u ′ k [x,y] and prediction signal Add up to get a new prediction signal And the new prediction signal Then, the new prediction signal can be sent to the buffer of the current image. After intra-frame prediction processing, we get The new prediction signal After loop filtering, the reconstructed signal s can be obtained ′ k [x,y], and will reconstruct the signal s ′ k [x,y] is sent to the decoded image buffer for storage to generate the reconstructed video. Reconstructed signal s ′ k [x,y] is obtained through motion compensation prediction in Can represent the reference block, m x and m y Represents the horizontal and vertical components of the motion vector of the reference block, respectively.

[0116] In some embodiments, mainstream video coding standards such as HEVC, VVC, AVS3, AV1, and AV2 all adopt a block-based hybrid coding framework, which can divide the original video data into a series of coding blocks, and combine video coding methods such as prediction, transform, and entropy coding to achieve video data compression. Among them, motion compensation is a commonly used prediction method for video coding. Motion compensation is based on the redundant characteristics of video content in the time domain or spatial domain, and derives the prediction value of the current block from the encoded reference block. This type of prediction method includes: inter-frame prediction, intra-frame block copy prediction, intra-frame string copy prediction, etc. In specific coding implementations, these prediction methods may be used alone or in combination. For coding blocks using these prediction methods, it is usually necessary to explicitly or implicitly encode one or more two-dimensional displacement vectors in the video code stream. The displacement vector is used to indicate the displacement of the current block (or the same-position block of the current block) relative to its one or more reference blocks.

[0117] It's important to note that displacement vectors may have different names in different prediction modes and implementations. This article uniformly describes them as follows: 1) The displacement vector in inter-frame prediction is called a motion vector (MV); 2) The displacement vector in intra-frame block copy prediction is called a block vector (BV); 3) The displacement vector in intra-frame string copy prediction is called a string vector (SV). The following uses inter-frame prediction as an example to introduce inter-frame prediction-related technologies.

[0118] Inter-frame prediction: Inter-frame prediction uses the correlation of the video time domain to use the adjacent coded image pixels to predict the pixels of the current image, so as to effectively remove the video time domain redundancy and effectively save the bits of the coded residual data. Figure 1b As shown, Figure 1b A schematic diagram of inter-frame prediction provided in an embodiment of the present application. Figure 1b In the example, P is the current frame, Pr is the reference frame, B is the current block, and Br is the reference block of B. B' and B have the same coordinate position in the image (i.e., B' is the same block as B), and the coordinates of Br are (x r ,y r ), the coordinates of B' are (x, y). The displacement between the current block and its reference block is called the motion vector (MV), that is: MV = (x t -x,y t -y).

[0119] Among them, considering that adjacent blocks in the time domain or spatial domain have a strong correlation, MV prediction technology can be used to further reduce the bits required for encoding MV. In H.265 / HEVC, inter-frame prediction includes two MV prediction technologies: Merge and AMVP (Advanced Motion Vector Prediction). The Merge mode will establish an MV candidate list for the current PU (Prediction Unit), in which there are 5 candidate MVs (and their corresponding reference images). Traverse these 5 candidate MVs and select the one with the lowest rate-distortion cost as the optimal MV. If the codec establishes the candidate list in the same way, the encoder only needs to transmit the index of the optimal MV in the candidate list. In AV1 and AV2, a technology called dynamic motion vector prediction (DMVP) is used to predict MV.

[0120] In order to achieve better prediction results, the current mainstream video coding standards allow the use of multiple reference frames for inter-frame prediction. The AV1 standard and the AOM next-generation standard AV2 currently under development include a prediction mode called compound prediction. The compound prediction mode allows the current block to use two reference frames for inter-frame prediction, and performs a weighted combination of the inter-frame prediction values ​​to derive the prediction value of the current block, or uses the inter-frame prediction value derived from a reference frame and the intra-frame prediction value derived from the current frame to perform a weighted combination to derive the prediction value of the current block. Among them, the current block refers to the coding block being encoded (or decoded). In the embodiment of the present application, the inter-frame prediction value and the intra-frame prediction value are subsequently referred to as reference prediction values. The following is the formula for deriving the prediction value of the current block during the compound prediction process:

[0121] P(x,y)=(w(x,y)·P0(x,y)+(1-w(x,y))·P1(x,y)) / 2

[0122] Where P(x,y) is the prediction value of the current block; P0(x,y) and P1(x,y) are the two reference prediction values ​​corresponding to the current block (x,y), and w(x,y) is the weight applied to the first reference prediction value P0(x,y).

[0123] Optionally, in video coding, to reduce the complexity of weighted prediction, integer calculations are often used instead of floating-point calculations. The following is a formula for deriving the predicted value of the current block using integer calculations:

[0124] P(x,y)=(w(x,y)·P0(x,y)+(64-w(x,y))·P1(x,y)+32)>>6

[0125] Among them, the weight value w(x,y) and the reference prediction values ​​P0(x,y) and P1(x,y) are all integers, and right shift operation is used instead of division. ">>6" means right shift by 6 bits. By right shifting by 6 bits, 64 can be divided by it. 32 is the bias added for rounding.

[0126] According to the current video coding standards, composite prediction uses a special weighting mode, that is, P0 and P1 have equal weight values, and the weights corresponding to the reference prediction values ​​corresponding to different positions are all set to fixed values. The specific formula is as follows:

[0127] P(x,y)=(32×P0(x,y)+32×P1(x,y)+32)>>6

[0128] 2. Video Decoding

[0129] On the decoding side, for each CU, after obtaining the video stream, on the one hand, the video stream is first entropy decoded to obtain information about various predictive coding modes and quantized transform coefficients. Each transform coefficient is then dequantized and inversely transformed to obtain a residual video signal. On the other hand, based on the known predictive coding mode information, the prediction signal corresponding to the CU (hereinafter referred to as the prediction value) can be obtained, and the residual video signal and the prediction signal are added to obtain a reconstructed video signal. This reconstructed video signal can be used to reconstruct the decoded image corresponding to the CU. Finally, the reconstructed video signal needs to undergo a loop filtering operation to produce the final output signal.

[0130] Based on the above description, the embodiments of the present application provide a video processing solution that can be applied to video encoders or video compression products that use composite prediction (or weighted prediction based on multiple reference frames). The general principles of the video processing solution are as follows:

[0131] On the encoder side, composite prediction is performed on the CUs contained in the video frame to obtain N reference prediction values ​​for the CU, where N is an integer greater than 1. Based on the importance of the N reference prediction values, weighted prediction is adaptively selected for the CU to obtain the CU's predicted value. Weighted prediction involves applying adaptively selected weights to the N reference prediction values ​​for the CU. The video is then encoded based on the CU's predicted values ​​to generate a video stream, which is then sent to the decoder.

[0132] At the decoding end: When decoding the CU in the video stream, a composite prediction of the CU in the video stream can be determined based on the information of the predictive coding mode, and appropriate weights can be adaptively selected for the CU based on the importance of the N reference prediction values ​​for weighted prediction; then, based on the adaptively selected weights, the N reference prediction values ​​are weighted predicted to obtain the prediction value of the CU, and the decoded image corresponding to the CU is reconstructed using the prediction value of the CU.

[0133] As mentioned above, the current video coding standard stipulates that equal weight values ​​are used to perform weighted prediction on the reference prediction values ​​derived from different reference blocks in composite prediction. However, in actual applications, the reference prediction values ​​derived from different reference blocks may have unequal importance. The use of equal weight values ​​cannot reflect the difference in importance of the predicted reference values. In this case, the use of existing standards will affect the prediction accuracy. The embodiment of the present application improves the current video coding standard, fully considers the importance of the reference prediction values ​​derived from different reference blocks in composite prediction, and allows the adaptive selection of appropriate weights for CU based on the importance of each reference prediction value for weighted prediction in composite prediction, which expands the weighted prediction method in the video coding standard, can improve the prediction accuracy of CU, and thus improve the coding performance.

[0134] Next, the video processing system provided by the embodiment of the present application is described. Figure 2 , Figure 2 The schematic diagram of the architecture of a video processing system provided in an embodiment of the present application. The video processing system 20 may include an encoding device 201 and a decoding device 202. The encoding device 201 is located at the encoding end, and the decoding device is located at the decoding end. The encoding device 201 can be a terminal or a server, and the decoding device 202 can be a terminal or a server, and a communication connection can be established between the encoding device 201 and the decoding device 202. Among them, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, a smart TV, etc., but is not limited to this. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0135] (1) For the encoding device 201:

[0136] The encoding device 201 can obtain a video to be encoded. The video can be captured by a camera or generated by a computer. The camera can be a hardware component provided in the encoding device 201. For example, the camera can be a common camera, a stereo camera, a light field camera, etc. provided in a terminal. The camera can also refer to a hardware device connected to the encoding device 201, such as a camera connected to a server.

[0137] Wherein, a video includes one or more video frames, and the encoding device 201 can divide each video frame into one or more CUs and encode each CU. When encoding any CU, a composite prediction can be performed on the CU being encoded (hereinafter referred to as the current block) to obtain N reference prediction values ​​of the current block, and the importance of each reference prediction value is determined by comprehensively considering factors such as the bit rate consumed during the weighted prediction processing and the quality loss of the current block during the encoding process. Then, based on the importance of each reference prediction value, a suitable target weight group is adaptively selected for the current block, and the target weight group may include one or more weight values; then, the weight values ​​in the target weight group are used to perform weighted prediction processing on the N reference prediction values ​​to obtain the prediction value of the current block. Wherein, the prediction value of the current block can be understood as the prediction signal corresponding to the current block, and the prediction value of the current block can be used to reconstruct the decoded image corresponding to the current block.

[0138] Among them, the N reference prediction values ​​of the current block are derived from the N reference blocks of the current block; one reference prediction value corresponds to one reference block. The video frame where the reference block is located is the reference frame, and the video frame where the current block is located is the current frame. The positional relationship between the N reference blocks and the current block may include but is not limited to any of the following: ① The N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame belong to different video frames in the video code stream; ② The N reference blocks are located in the same reference frame, and the same reference frame and the current frame belong to different video frames in the video code stream; ③ One or more reference blocks among the N reference blocks are located in the current frame, and the remaining reference blocks among the N reference blocks are located in one or more reference frames, and one or more reference frames and the current frame belong to different video frames in the video code stream; ④ The N reference blocks and the current block are both located in the current frame. It can be seen that the prediction mode of the composite prediction in the embodiment of the present application includes an inter-frame prediction mode, which allows the use of at least two reference frames for inter-frame prediction; it also includes a combined prediction mode, which allows the use of at least one reference frame for inter-frame prediction and allows the use of the current frame for intra-frame prediction; it also includes an intra-frame prediction mode, which allows the use of the current frame for intra-frame prediction.

[0139] Corresponding to different prediction modes of compound prediction, the N reference prediction values ​​of the current block may be derived in any of the following ways: ① The N reference prediction values ​​of the current block are derived by performing inter-frame prediction using the N reference blocks of the current block respectively. In this case, the N reference prediction values ​​can all be referred to as inter-frame prediction values. ② Among the N reference prediction values ​​of the current block, at least one reference prediction value is derived by performing inter-frame prediction using at least one reference block of the N reference blocks of the current block. Such reference prediction values ​​can be referred to as inter-frame prediction values; and the remaining reference prediction values ​​are derived by performing intra-frame prediction using the remaining reference blocks of the N reference blocks. Such reference prediction values ​​can be referred to as intra-frame prediction values.

[0140] Then, the encoding device 201 performs transform coding, quantization, entropy coding and other operations on the video based on the prediction value of the CU contained in the video frame to obtain a video code stream, and sends the video code stream to the decoding device 202 so that the decoding device 202 decodes the video code stream.

[0141] (2) For the decoding device 202:

[0142] After receiving the video stream sent by the encoding device 201, the decoding device 202 can decode the video stream and reconstruct the video corresponding to the video stream. Specifically, on the one hand, the decoding device 202 can perform entropy decoding on the video stream to obtain the prediction mode and quantized transform coefficients of each CU in the video stream, and perform composite prediction on the current block (i.e., the CU being decoded) based on the prediction mode of the current block to obtain N reference prediction values ​​for the current block; and determine whether the current block allows the use of adaptive weighted prediction.

[0143] If it is determined that adaptive weighted prediction is permitted for the current block, a target weight list may be determined from one or more weight lists based on the importance of the N reference prediction values, and a target weight group for weighted prediction may be determined for the current block from the target weight list, the target weight group containing one or more weight values. The weight values ​​in the target weight group are then used to directly perform weighted prediction processing on the N reference prediction values ​​to obtain a prediction value for the current block. If it is determined that adaptive weighted prediction is not permitted for the current block, a weighted prediction processing may be performed on the N reference prediction values ​​in accordance with existing video coding standards, for example, using equal weight values ​​for weighted prediction processing on each reference prediction value to obtain a prediction value for the current block.

[0144] On the other hand, the decoding device 202 performs inverse quantization and inverse transformation on the quantized transform coefficients to obtain the residual signal value of the current block. The predicted value of the current block and the residual signal value are superimposed to obtain the reconstructed value of the current block. The decoded image corresponding to the current block is then reconstructed based on the reconstructed value. The decoded image can be used as a reference image for decoding other CUs and can also be used to reconstruct video.

[0145] In the embodiments of the present application, composite prediction is used in the video encoding and decoding process, and the importance of reference prediction values ​​derived from different reference blocks is fully considered in the composite prediction, allowing the adaptive selection of appropriate weights for CU based on the importance of each reference prediction value in the composite prediction to perform weighted prediction, thereby expanding the weighted prediction method in the video coding standard, and can improve the prediction accuracy of the CU, thereby improving the encoding and decoding performance.

[0146] Next, the video processing method provided by the embodiment of the present application is described. Figure 3 , Figure 3 This is a flow chart of a video processing method provided in an embodiment of the present application. The video processing method can be executed by a decoding device in the above-mentioned video processing system. The video processing method described in this embodiment may include the following steps S301-S303:

[0147] S301. Perform composite prediction on a current block in a video stream to obtain N reference prediction values ​​of the current block, where the current block refers to a coding block being decoded in the video stream, and N is an integer greater than 1.

[0148] The video code stream contains one or more video frames, and each video frame may contain one or more coding blocks. The above-mentioned N reference prediction values ​​may be derived from N reference blocks, and one reference prediction value corresponds to one reference block. In the embodiment of the present application, the video frame where the reference block is located may be the reference frame, and the video frame where the current block is located may be the current frame. The positional relationship between the N reference blocks and the current block includes any of the following: ① The N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame may belong to different video frames in the video code stream. For example, N=2, one of the two reference blocks is located in reference frame one, and the other reference block is located in reference frame two. Reference frame one, reference frame two, and the current frame belong to different video frames in the video code stream. ② The N reference blocks are located in the same reference frame, and the same reference frame and the current frame may belong to different video frames in the video code stream. For example, N=2, both reference blocks are located in reference frame one, and reference frame one and the current frame belong to different video frames in the video code stream. ③ One or more of the N reference blocks are located in the current frame, and the remaining reference blocks in the N reference blocks are located in one or more reference frames, and one or more reference frames belong to different video frames in the video stream than the current frame. For example, N = 4, and the four reference blocks are: reference block 1, reference block 2, reference block 3, and reference block 4; reference block 1 is located in the current frame, and the remaining reference blocks 2, reference block 3, and reference block 4 are all located in reference frame 1, and reference frame 1 and reference frame 2 belong to different video frames in the video stream than the current frame; for another example, reference block 1 and reference block 2 are located in the current frame, and the remaining reference block 3 can be located in reference frame 1, and reference block 4 is located in a reference frame 2, and reference frame 1, reference frame 2, and the current frame belong to different video frames in the video stream. ④ N reference blocks and the current block are all located in the current frame, for example, N = 2, and both reference blocks are located in the current frame.

[0149] According to the positional relationship between the N reference blocks and the current block shown in ①-④ above, the prediction mode of the composite prediction in the embodiment of the present application includes an inter-frame prediction mode, that is, allowing the use of at least two reference frames for inter-frame prediction; it may also include a combined prediction mode, that is, allowing the use of at least one reference frame for inter-frame prediction, and allowing the use of the current frame for intra-frame prediction; it also includes an intra-frame prediction mode, that is, allowing the use of the current frame for intra-frame prediction.

[0150] Corresponding to different prediction modes of compound prediction, the N reference prediction values ​​of the current block may be derived in any of the following ways: ① The N reference prediction values ​​of the current block are derived by performing inter-frame prediction using the N reference blocks of the current block respectively. In this case, the N reference prediction values ​​can all be referred to as inter-frame prediction values. ② Among the N reference prediction values ​​of the current block, at least one reference prediction value is derived by performing inter-frame prediction using at least one reference block of the N reference blocks of the current block. Such reference prediction values ​​can be referred to as inter-frame prediction values; and the remaining reference prediction values ​​are derived by performing intra-frame prediction using the remaining reference blocks of the N reference blocks. Such reference prediction values ​​can be referred to as intra-frame prediction values.

[0151] In one embodiment, before executing step S302, it can be determined whether the current block meets the conditions for adaptive weighted prediction. If the current block meets the conditions for adaptive weighted prediction, step S302 is executed. By determining whether the current block meets the conditions for adaptive weighted prediction, adaptive selection of weights for weighted prediction can be achieved, further improving prediction accuracy and thus improving encoding performance.

[0152] The condition that the current block satisfies the adaptive weighted prediction includes at least one of the following:

[0153] a) The sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that all coding blocks in the frame sequence are allowed to use adaptive weighted prediction; wherein a frame sequence refers to a sequence of multiple video frames in sequence. It should be understood that when the sequence header of the frame sequence contains the first indication field, the first indication field is used to indicate that all coding blocks contained in the entire frame sequence are allowed to use adaptive weighted prediction.

[0154] As an implementation method, the first indication field can be expressed as seq_acp_flag. The first indication field can indicate whether the coding blocks in the frame sequence are allowed to use adaptive weighted prediction according to the value. If the above-mentioned first indication field is a first preset value (such as 1), it indicates that all coding blocks in the frame sequence are allowed to use adaptive weighted prediction, and it can be determined that the current block meets the adaptive weighted prediction conditions; if the above-mentioned first indication field is a second preset value (such as 0), it indicates that all coding blocks in the frame sequence are not allowed to use adaptive weighted prediction, and it can be determined that the current block does not meet the adaptive weighted prediction conditions.

[0155] b) The slice header of the current slice to which the current block belongs includes a second indication field, and the second indication field indicates that the coding blocks in the current slice allow the use of adaptive weighted prediction, wherein a video frame can be divided into multiple slices, each slice containing one or more coding blocks; the current slice refers to the slice to which the current block belongs, that is, the slice being decoded. It should be understood that when the slice header of the current slice includes the second indication field, the second indication field can be used to indicate that all coding blocks included in the current slice allow the use of adaptive weighted prediction.

[0156] As an implementation method, the second indication field can be expressed as slice_acp_flag. The second indication field can indicate whether the coding blocks in the current slice are allowed to use adaptive weighted prediction according to the value. If the above-mentioned second indication field is a first preset value (such as 1), it indicates that all coding blocks in the current slice are allowed to use adaptive weighted prediction, and it can be determined that the current block meets the adaptive weighted prediction conditions; if the above-mentioned second indication field is a second preset value (such as 0), it indicates that all coding blocks in the current slice are not allowed to use adaptive weighted prediction, and it can be determined that the current block does not meet the adaptive weighted prediction conditions.

[0157] c) The frame header of the current frame in which the current block is located includes a third indication field, and the third indication field indicates that the coding blocks in the current frame are allowed to use adaptive weighted prediction. It should be understood that when the frame header of the current frame includes the third indication field, the third indication field can be used to indicate that all coding blocks included in the current frame are allowed to use adaptive weighted prediction.

[0158] As an implementation method, the third indication field can be expressed as pic_acp_flag, and the third indication field indicates whether the current frame allows the use of adaptive weighted prediction based on its value. If the above-mentioned third indication field is a first preset value (such as 1), it indicates that the coding block in the current frame allows the use of adaptive weighted prediction, and it can be determined that the current block meets the adaptive weighted prediction conditions; if the third indication field is a second preset value (such as 0), it can indicate that the coding block in the current frame does not allow the use of adaptive weighted prediction, and it can be determined that the current block does not meet the adaptive weighted prediction conditions.

[0159] d) In the composite prediction process, the current block uses at least two reference frames for inter-frame prediction.

[0160] e) During the composite prediction process, the current block uses at least one reference frame for inter-frame prediction and the current frame for intra-frame prediction. For example, if the current block uses reference frame one for inter-frame prediction and the current frame for intra-frame prediction, it can be determined that the current block meets the conditions for adaptive weighted prediction.

[0161] f) The motion type of the current block is a specified motion type. For example, if the motion type of the current block is simple_translation (simple balance), it is determined that the current block meets the conditions for adaptive weighted prediction.

[0162] g) The current block uses a preset motion vector prediction mode. For example, the current block uses the preset motion vector prediction mode NEAR_NEARMV. Then, it can be determined that the current block meets the conditions of adaptive weighted prediction.

[0163] In the AV1 and AV2 standards, a technology called dynamic motion vector prediction is used to predict MV. MV can be predicted by spatially adjacent blocks in the current frame or temporally adjacent blocks in the reference frame. For single reference inter-frame prediction, each reference frame has a separate prediction MV list; for composite inter-frame prediction, the prediction MV lists corresponding to different reference frames form a prediction MV group list, and multiple prediction MV modes are allowed, such as NEAR_NEARMV, NEAR_NEWMV, NEW_NEAR_MV, NEW_NEWMV, GLOBAL_GLOBALMV, JOINT_NEWMV, etc. Among them:

[0164] NEAR_NEARMV: indicates that the MVs corresponding to the two reference frames are the MVs in the predicted MV group;

[0165] NEAR_NEWMV: The first MV is the first predicted MV in the predicted MV group. The second MV is derived from the MVD decoded from the video stream and the second predicted MV in the predicted MV group. The MVD (Motion Vector Difference) is the difference between the current MV and the predicted MV (Motion Vector Prediction, MVP).

[0166] NEW_NEARMV: The first MV is derived based on the MVD decoded from the video stream and the first predicted MV in the predicted MV group. The second MV is the second predicted MV in the predicted MV group.

[0167] NEW_NEWMV: The first MV is derived based on MVD1 decoded from the video stream and the first predicted MV in the predicted MV group. The second MV is derived based on MVD2 decoded from the video stream and the second predicted MV in the predicted MV group.

[0168] GLOBAL_GLOBALMV: derive MV based on per-frame global motion information.

[0169] NEW_NEWMV: Similar to NEW_NEWMV, but the video stream contains only one MVD, and the other MVD is derived based on the information between reference frames.

[0170] In the embodiments of the present application, the preset motion vector prediction mode may be one or more of the following: NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, NEW_NEWMV, GLOBAL_GLOBALMV, and JOINT_MV. It should be noted that the preset motion vector prediction mode is not limited to the above MV prediction mode. For other standards, such as H.265 and H.266, the motion vector prediction mode may also be determined in combination with merge and AMVP.

[0171] h) The current block uses a preset interpolation filter. The video encoding process typically includes many encoding tools, including various types of interpolation filters, such as linear interpolation filters, CIC (Integral-Comb Cascade) interpolation filters, and so on. In the embodiment of the present application, the preset interpolation filter can be any of these various types of interpolation filters. For example, the preset interpolation filter can be a linear interpolation filter. When the current block uses a linear interpolation filter, it can be determined that the current block meets the conditions for adaptive weighted prediction.

[0172] i) If the current block does not use a specific coding tool, such as an optical flow-based motion vector optimization method, then it can be determined that the current block meets the conditions for adaptive weighted prediction. Video coding standards such as AV2 and H.266 allow the use of an optical flow-based motion vector optimization method, which refines motion vectors based on the optical flow equation.

[0173] j) If the reference frame used in the composite prediction process of the current block meets specific conditions, then it can be determined that the current block meets the conditions for adaptive weighted prediction. The specific conditions include one or more of the following (i.e., the specific conditions may include one or more of ① and ②): ① The orientation relationship between the reference frame used in the composite prediction and the current frame in the video bitstream meets a preset relationship; wherein, the orientation relationship meeting the preset relationship includes any of the following: the reference frames used are all located before the current frame; the reference frames used are all located after the current frame; some of the reference frames used are located before the current frame, and the remaining reference frames are located after the current frame. The video bitstream contains multiple video frames, and any video frame corresponds to a frame display time in the video. The orientation relationship can actually be understood as the order of frame display time. For example, the reference frames used are all located before the current frame, which can be understood as: the frame display time of the reference frame is earlier than the frame display time of the current frame; the reference frames used are all located after the current frame, which can be understood as: the frame display time of the reference frame is later than the frame display time of the current frame.

[0174] ② If the absolute value of the importance difference between the reference prediction values ​​corresponding to the reference frames used in the composite prediction is greater than or equal to a preset threshold, it can be determined that the current block meets the conditions for adaptive weighted prediction; the preset threshold can be set as required. As an implementation method, the importance of the reference prediction values ​​corresponding to the reference frames can be measured by an importance metric. In this case, the importance difference between the reference prediction values ​​corresponding to the reference frames used in the composite prediction can be determined based on the importance metric between the reference prediction values ​​corresponding to the used reference frames. For example, N = 2, in the composite prediction, the two reference prediction values ​​are: reference prediction value 1 corresponding to reference frame 1 and reference prediction value 2 corresponding to reference frame 2; assuming that the importance metric of reference prediction value 1 is D0 and the importance metric of reference prediction value 2 is D1, then the importance difference between the reference prediction values ​​corresponding to the two reference frames is: the difference D0-D1 between the importance metric D0 of reference prediction value 1 and the importance metric D1 of reference prediction value 2. That is, the absolute value of the importance difference between the two reference prediction values ​​is: ΔD = abs(D0-D1), where abs() represents the absolute value.

[0175] It should be understood that the above-mentioned conditions for the current block to satisfy adaptive weighted prediction can be used individually or in combination. For example, the conditions for the current block to satisfy adaptive weighted prediction may include: the sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that the coding blocks in the frame sequence allow the use of adaptive weighted prediction, and the motion type of the current block is a specified motion type. For another example, the frame header of the current frame to which the current block belongs contains a third indication field, and the third indication field indicates that the coding blocks in the current frame allow the use of adaptive weighted prediction and the current block uses a preset motion vector prediction mode; etc. This application does not impose any limitations on this.

[0176] S302: Determine a target weight group for weighted prediction for the current block according to the importance of the N reference prediction values.

[0177] The importance of the reference prediction value can be comprehensively determined based on factors such as the bit rate consumed during weighted prediction processing and the quality loss of the current block during the encoding process. For example, when the current block uses a certain reference prediction value for weighted prediction, the bit rate consumption increases significantly, indicating that the reference prediction value is not very helpful in reducing bit rate consumption and its importance is low. For another example, when the current block uses a certain reference prediction value for weighted prediction, the quality loss increases significantly, indicating that the reference prediction value is not very helpful in reducing quality loss and its importance is low. The target weight group contains one or more weight values, which will be applied to each reference prediction value during the weighted prediction process. If the importance of a reference prediction value is low, then the reference prediction value corresponds to a smaller weight value in the target weight group. If the importance of a reference prediction value is high, then the reference prediction value corresponds to a larger weight value in the target weight group. That is to say, the target weight group is selected based on the influence of each reference prediction value on factors such as bit rate consumption and quality loss, so that the comprehensive cost of the weighted prediction process (i.e., the cost of bit rate consumption, the cost of quality loss, the cost of bit rate consumption and quality loss) is smaller and the encoding and decoding performance is better.

[0178] In one embodiment, the video stream includes one or more weight lists, each of which includes one or more weight groups, and each weight group includes one or more weight values. The number of weight values ​​included in each weight group can be the same or different, the weight values ​​included in each weight group can be the same or different, and the order of the weight values ​​included in each weight group can be the same or different. The following is an example of a weight list:

[0179] i. Weight list 1 is represented as: {2, 4, 6, 8, 10, 12, 14} / 16. This indicates that weight list 1 contains 7 weight groups, namely: weight group 1: {2} / 16; weight group 2: {4} / 16; weight group 3: {6} / 16; and so on, weight group 7: {14} / 16.

[0180] ii. Weight list 2 is represented as: {14, 8, 4, 12, 2} / 16. This indicates that weight list 2 contains five weight groups: Weight group 1: {14} / 16; Weight group 2: {8} / 16; and so on, Weight group 5: {2} / 16.

[0181] iii. Weight list 3 is represented as: {4, {8, 8}, 12} / 16. This indicates that weight list 3 contains three weight groups: Weight group 1: {4} / 16; Weight group 2: {8, 8} / 16; Weight group 3: {12} / 16.

[0182] From the above examples, we can see that: (1) the number of weight groups contained in each weight list is allowed to be different, such as the number of weight groups in weight list 1 is 7, the number of weight groups in weight list 2 is 5, and the number of weight groups in weight list 1 is 3. (2) the number of weight values ​​contained in each weight group is allowed to be the same, such as the number of weight values ​​contained in each weight group in weight list 1 is 1. (3) the data of weight values ​​contained in each weight group is also allowed to be different, such as the number of weight values ​​contained in weight group 1 in weight list 3 is 1, but the number of weight values ​​contained in weight group 2 in weight list 3 is 2. (4) the weight values ​​in each weight group are allowed to be the same, such as: weight group 1 in weight list 1 contains a weight value of 2 / 16, and weight group 1 in weight list 2 also contains a weight value of 2 / 16. (5) the weight values ​​in each weight group are also allowed to be different, such as weight group 1 in weight list 1 contains a weight value of 2 / 16, but weight group 2 in weight list 1 contains a weight value of 4 / 16. It is understandable that the sum of the weight values ​​provided by a weight group should be equal to 1. In the above example, although each weight group contains only one weight value, each weight value is less than 1. Therefore, each weight group implicitly contains another weight value, that is, in a specific application, each weight group actually provides two weight values, and the sum of these two weight values ​​is 1. For example, weight group 1 in weight list 1 only contains weight value 2 / 16, but in a specific application, the weight group 1 actually provides two weight values, namely weight value 2 / 16 and weight value 14 / 16. For another example: weight group 3 in weight list 3 only contains weight value 12 / 16, but in a specific application, the weight group 3 actually provides two weight values, namely weight value 12 / 16 and weight value 4 / 16. It can be seen that in an embodiment of the present application, when the sum of the weight values ​​contained in the weight group in the weight list is less than 1, the weight value implied by the weight group can be obtained by calculation.

[0183] Here is another example of a weight list:

[0184] i. Weight list 4 contains 4 weight groups, namely weight group 1: {2,14} / 16; weight group 2: {4,12} / 16; weight group 3: {6,10} / 16; weight group 4: {8,8}} / 16.

[0185] ii. Weight list 5 contains two weight groups, namely weight group 1: {4, 12} / 16; weight group 2: {10, 6} / 16.

[0186] The above examples show that: (1) the sum of all weight values ​​contained in the weight groups in the weight list is equal to 1. (2) the order of the weight values ​​contained in each weight group can be different. For example, although weight group 3 in weight list 4 and weight group 2 in weight list 5 contain the same weight values, the order of the weight values ​​is different.

[0187] In one embodiment, step S302 may include steps s31-s32:

[0188] s31. Determine a target weight list from one or more weight lists based on the importance of the N reference prediction values.

[0189] Specifically, the method for determining the target weight list from one or more weight lists according to the importance of the N reference prediction values ​​may include the following methods:

[0190] Method 1: When there is only one weight list in the video stream, the only weight list in the video stream can be directly determined as the target weight list.

[0191] Method 2: When there are multiple weight lists in the video stream, an importance metric of the reference prediction value can be introduced, and a target weight list can be determined from the multiple weight lists based on the importance metrics of N reference prediction values.

[0192] (1) Determine the target weight list based on the absolute value of the difference between the importance metrics of N reference prediction values.

[0193] When the number of weight lists in a video stream is M+1, where M is a positive integer greater than or equal to 1, the M+1 weight lists can be denoted as {w_list1, w_list2, ..., w_listM+1}. Each weight list corresponds to one threshold interval, meaning the number of threshold intervals is also M+1. As one implementation, M+1 threshold intervals can be directly set as needed. Another implementation involves obtaining M thresholds and dividing them into M+1 threshold intervals based on the M thresholds. For example, obtaining M thresholds, T1, T2, ..., TM; then dividing the M thresholds into M+1 threshold intervals, [0, T1], (T1, T2], (T2, T3], ..., (TM, +∞); wherein [0, T1] can be associated with w_list1, (T1, T2] with w_list2, and so on. T is an integer greater than or equal to 0.

[0194] The decoding device can obtain the importance metric values ​​of N reference prediction values, and calculate the importance difference between the N reference prediction values. The importance difference between any two reference prediction values ​​is measured by the difference between the importance metric values ​​of any two reference prediction values. Then, the decoding device can determine the threshold interval where the absolute value of the importance difference between the N reference prediction values ​​is located; and determine the weight list corresponding to the threshold interval where the absolute value of the importance difference between the N reference prediction values ​​is located as the target weight list. Among them, suppose the importance metric values ​​of any two reference prediction values ​​are D0 and D1 respectively, and the importance difference between the any two reference prediction values ​​is expressed as D0-D1, then the absolute value of the importance difference between the any two reference prediction values ​​is expressed as ΔD=abs(D0-D1). It should be noted that the importance metric value is an indicator used to measure the degree of importance, but there can be multiple situations for the measurement benchmark. For example, the measurement benchmark can be that the larger the importance metric value, the higher the degree of importance; for another example: the measurement benchmark can also be that the smaller the importance metric value, the higher the degree of importance; this application does not limit this measurement benchmark.

[0195] In one embodiment, if N=2, that is, only two reference prediction values ​​are included, and the importance metrics of the two reference prediction values ​​are D0 and D1, respectively, then the weight list corresponding to the threshold interval where ΔD=abs(D0-D1) is located is directly determined as the target weight list. For example: when the number of weight lists in the video stream is 2, the two weight lists are recorded as {w_list1, w_list2}, where w_list1 is {8,12,14} / 16 and w_list2 is {12,8,4} / 16. Let the threshold interval corresponding to w_list1 be [0,1] and the threshold interval corresponding to w_list2 be (1,+∞); then, when ΔD=abs(D0-D1) is less than or equal to 1, the threshold interval is [0,1], and w_list1 is determined as the target weight list; otherwise, when ΔD=abs(D0-D1) is greater than 1, the threshold interval is (1,+∞), and w_list2 is determined as the target weight list.

[0196] In another embodiment, if N>2, the absolute value of the importance difference between any two reference prediction values ​​can be calculated respectively, and the weight list corresponding to the threshold interval of each absolute value can be found respectively, and the weight list with the largest number of corresponding values ​​can be determined as the target weight list. For example, N=3, and the importance measurement values ​​of the three reference prediction values ​​are D0, D1 and D2 respectively, then the absolute value of the importance difference between any two reference prediction values ​​can be calculated respectively, that is, ΔD=abs(D0-D1), ΔD'=abs(D1-D2) and ΔD”=abs(D0-D1) can be calculated respectively, and then the weight lists corresponding to the threshold intervals of ΔD, ΔD' and ΔD” can be judged respectively. If there are two or more of the three that correspond to the same weight list, then the same weight list is determined as the target weight list.

[0197] In another embodiment, if N>2, the absolute value of the importance difference between any two reference prediction values ​​can be calculated respectively, and then the maximum value among the absolute values ​​can be found, and the weight list corresponding to the threshold interval where the maximum value is located is determined as the target weight list. For example, in the above example of N=3, after calculating ΔD, ΔD' and ΔD", the maximum value among ΔD, ΔD' and ΔD" is found. Assuming that the maximum value is ΔD', the weight list corresponding to the threshold interval where ΔD' is located is determined as the target weight list.

[0198] In another embodiment, if N>2, the absolute value of the importance difference between any two reference prediction values ​​can be calculated respectively, and then the minimum value among the absolute values ​​can be found, and the weight list corresponding to the threshold interval where the minimum value is located is determined as the target weight list. For example, in the above example of N=3, after calculating ΔD, ΔD' and ΔD", the minimum value among ΔD, ΔD' and ΔD" is found. Assuming that the minimum value is ΔD, the weight list corresponding to the threshold interval where ΔD is located is determined as the target weight list.

[0199] In another embodiment, if N>2, the absolute value of the importance difference between any two reference prediction values ​​can be calculated respectively, and then the average of each absolute value can be calculated, and the weight list corresponding to the threshold interval where the average value is located is determined as the target weight list. For example, in the above example of N=3, after calculating ΔD, ΔD' and ΔD", the average value of ΔD, ΔD' and ΔD" is calculated = (ΔD+ΔD'+ΔD") / 3, and the weight list corresponding to the threshold interval where the average value is located is determined as the target weight list.

[0200] It should be noted that the embodiments of the present application can also utilize other numerical characteristics of the absolute values ​​of the importance differences between N reference prediction values, such as the maximum value, minimum value, average value, etc. after squaring each absolute value; to determine the target weight list, and the present application is not limited to this.

[0201] (2) Compare the importance metrics of the reference prediction values ​​to determine the target weight list.

[0202] The N reference prediction values ​​of the current block may include a first reference prediction value and a second reference prediction value, and the video code stream may include a first weight list and a second weight list. The decoding device may compare the importance metric value of the first reference prediction value with the importance metric value of the second reference prediction value; if it is determined that the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, the first weight list is determined as the target weight list; if it is determined that the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, the second weight list is determined as the target weight list.

[0203] For example, the importance measure value of the first reference prediction value is D0, the importance measure value of the second reference prediction value is D1, the first weight list is w_list1, and the second weight list is w_list2; compare the sizes of D0 and D1; if D0>D1, the first weight list w_list1 can be determined as the target weight list; if D0≤D1, the first weight list w_list2 can be determined as the target weight list.

[0204] Optionally, the weight values ​​in the first weight list are opposite to the weight values ​​in the second weight list, that is, w_list2[x]=1-w_list1[x], where x represents the weight value in the weight list, such as w_list1={0.2, 0.4}, w_list2[x]=1-w_list1[x], that is, w_list2[x]={0.8, 0.6}. In addition, the weight values ​​in the first weight list and the weight values ​​in the second weight list can also be set separately.

[0205] (3) Use mathematical symbolic functions and the importance measure of reference prediction values ​​to determine the target weight list.

[0206] The N reference prediction values ​​of the current block include a first reference prediction value and a second reference prediction value; the video code stream includes a first weight list, a second weight list and a third weight list. The decoding device can call a mathematical symbol function to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain a symbol value. If the above symbol value is a first preset value (such as -1), the first weight list is determined as the target weight list; if the above symbol value is a second preset value (such as 0), the second weight list is determined as the target weight list; if the above symbol value is a third preset value (such as 1), the third weight list is determined as the target weight list. Among them, the above-mentioned first weight list, the second weight list and the third weight list are different weight lists respectively; or, two weight lists in the first weight list, the second weight list and the third weight list are allowed to be the same weight list.

[0207] Among them, the importance measure value of the above-mentioned first reference prediction value is D0, and the importance measure value of the second reference prediction value is D1. The data sign function is called to process the difference between D0 and D1 to obtain the sign value, that is, the sign value = sign(D0-D1), sign() represents the data sign function.

[0208] (4) The above methods (1), (2), and (3) can be used alone, or they can be combined to determine the target weight list. As an implementation method, there are M+1 weight lists. For any two reference prediction values, method (1) can be used to determine the candidate weight list by using the threshold interval of the absolute value of the importance difference between the two reference prediction values. Then, method (2) can be used to compare the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value. If it is determined that the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, the candidate weight list is directly determined as the target weight list; if it is determined that the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, the weight list corresponding to the weight value opposite to the weight value in the candidate weight list is determined as the target weight list.

[0209] For example, the importance metric value of the first reference prediction value is D0, and the importance metric threshold value of the second reference prediction value is D1; ​​the candidate weight list w_list1 can be determined by adopting method (1). If D0>D1, the weight list w_list1 is determined as the target weight list; if D0≤D1, the weight list w_list2 is determined as the target weight list; wherein the weight values ​​in w_list2 are opposite to the weight values ​​in w_list1, that is, w_list2[x]=1-w_list1[x].

[0210] As another implementation method, assume that there are 3*(M+1) weight lists in the video code stream, that is, one threshold interval can correspond to three weight lists; for any two reference prediction values, method (1) can be used to first determine the threshold interval where the absolute value of the importance difference between the first reference prediction value and the second reference prediction value is located, and the three weight lists corresponding to the threshold interval are all determined as candidate weight lists, that is, the candidate weight list can include the first weight list, the second weight list, and the third weight list. Then, method (2) is used to call the mathematical symbol function to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain a symbol value. If the symbol value is the first preset value, the first weight list is determined as the target weight list; if the symbol value is the second preset value, the second weight list is determined as the target weight list; if the symbol value is the third preset value, the third weight list is determined as the target weight list.

[0211] For example, method (1) is used to determine three weight lists {w_list1, w_list2, w_list3} as candidate weight lists, and then a mathematical symbol function is called to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain a symbol value. When the symbol value is -1, w_list1 is determined as the target weight list; when the symbol value is 0, w_list2 is determined as the target weight list; when the symbol value is 1, w_list3 is determined as the target weight list.

[0212] Any one of the N reference prediction values ​​is denoted as reference prediction value i. Reference prediction value i is derived from reference block i. The video frame where reference block i is located is reference frame i. i is an integer and i is less than or equal to N. The video frame where the current block is located is the current frame. The importance metric of reference prediction value i can be determined using any of the following methods:

[0213] Method 1: Calculated based on the Picture Order Count (POC) of the current frame in the video stream and the frame display order of the reference frame i in the video stream. Specifically, the difference between the frame display order of the current frame in the video stream and the frame display order of the reference frame i in the video stream can be calculated, and the absolute value of the difference is used as the importance metric of the reference prediction value i. For example, let the frame display number of the current frame in the video stream be cur_poc, and the frame display order of the reference frame i in the video stream be ref_poc; the importance metric of the reference prediction value i is D = abs(cur_poc-ref_poc), where abs() represents the absolute value.

[0214] Method 2: Calculated based on the frame display order of the current frame in the video stream, the frame display order of reference frame i in the video stream, and the quality metric Q. The quality metric Q can be determined based on a variety of circumstances, which are not limited in this application. Specifically, the quality metric Q of reference frame i can be derived based on the quantization information of the current block. For example, the quality metric Q can be set to the base_qindex (base quantization index) of reference frame i. The base_qindex of any reference frame can be different or the same. In another implementation, the quality metric Q of reference frame i can also be derived based on other coding information. For example, the quality metric Q of reference frame i can be derived based on the coding information difference between the coded CU in reference frame i and the coded CU in the current frame.

[0215] As an implementation method, the decoding device can calculate the difference between the frame display order of the current frame in the video code stream and the frame display order of the reference frame i in the video code stream; then, using the objective function, determine the importance metric value of the reference prediction value i based on the above difference, the quality metric Q and the importance metric value list.

[0216] The objective function is: D = f(cur_poc - ref_poc) + Q, where D represents the importance metric of reference prediction value i, f(x) is an increasing function, x = cur_poc - ref_poc, cur_poc represents the frame display sequence number of the current frame in the video stream, and ref_poc represents the frame display sequence of reference frame i in the video stream. The above importance metric list contains the correspondence between f(x) and the reference importance metric values, as shown in Table 1:

[0217] Table 1

[0218] x 0 1 2 3 4 5 6 7 8 9 f(x) 0 64 96 112 120 124 126 127 128 129

[0219] It should be noted that the above-mentioned function expression of f(x) is only an example. The embodiment of the present application allows the function expression of f(x) to vary, that is, the embodiment of the present application does not limit the specific expression form of f(x).

[0220] In one embodiment, the importance metric of the reference frame i can be calculated based on the orientation relationship between the reference frame i and the current frame and the quality metric Q. As an implementation method, a correspondence between the reference orientation relationship and the reference importance metric can be established. For example, if the reference orientation relationship indicates that the reference frame is before the current frame, the reference importance metric can correspond to a first value; if the reference orientation relationship indicates that the reference frame is after the current frame, the reference importance metric can correspond to a second value. The decoding device can then calculate the importance metric of the reference prediction value i based on the reference importance metric corresponding to the reference frame i and the quality metric Q.

[0221] Method 3: Calculate the importance metric score of reference frame i based on the calculation results of method 1 and method 2, and sort the importance metric scores of the reference frames corresponding to N reference prediction values ​​in ascending order, and determine the index of reference frame i in the sorting as the importance metric value of reference prediction value i.

[0222] As an implementation method, the first importance metric value of the reference prediction value i can be calculated by the above method one, and the second importance metric value of the reference prediction value i can be calculated by the above method two, and then the importance metric score (such as score) of the reference frame i is calculated based on the first importance metric value and the second importance metric value, and then the importance metric scores of the reference frames corresponding to the N reference prediction values ​​are sorted in order from small to large, and the index of the reference frame i in the sorting is determined as the importance metric value of the reference prediction value i.

[0223] For example, the importance metric score of reference frame one corresponding to reference prediction value 1 is 20, the importance metric score of reference frame two corresponding to reference prediction value 2 is 30, and the importance metric score of reference frame three corresponding to reference prediction value 3 is 40; then, the importance metric scores of the reference frames corresponding to the three reference prediction values ​​are sorted in ascending order, and the sorting result is: reference frame one, reference frame two, reference frame three; among them, the index of reference frame one in the sorting is 1, then the importance metric value of reference prediction value 1 is 1; the index of reference frame two in the sorting is 2, then the importance metric value of reference prediction value 2 is 2; the index of reference frame three in the sorting is 3, then the importance metric value of reference prediction value 3 is 3.

[0224] Calculating the importance metric score of reference frame i based on the first importance metric value and the second importance metric value can be done as follows: ① Taking a weighted sum of the first importance metric value and the second importance metric value to obtain the importance metric value of reference prediction value i. ② Taking an average of the first importance metric value and the second importance metric value to obtain the importance metric value of reference prediction value i.

[0225] Method 4: In order to obtain a more accurate importance metric value, the calculation results of method 1, method 2 or method 3 can be adjusted based on the prediction mode of the reference prediction value i to obtain the importance metric value of the reference prediction value i; wherein, the prediction mode of the reference prediction value i includes any one of the following: inter-frame prediction mode, intra-frame prediction mode. As an implementation method, the calculation results of method 1, method 2 or method 3 can be adjusted according to the adjustment function to obtain the importance metric value of the reference prediction value i. The adjustment function can be, for example, D'=g(D)=a*D+b; wherein, D' represents the importance metric value of the reference prediction value i; D is the calculation result of the above method 1, method 2 or method 3, and a and b can be adaptively set according to the prediction mode. For example, the following are 3 specific examples:

[0226] a) If the prediction mode of the reference prediction value i is inter prediction, then a=1, b=0;

[0227] b) If the prediction mode of the reference prediction value i is intra prediction, then a=2, b=0;

[0228] c) If the prediction mode of the reference prediction value i is intra prediction, then a=0, b=160.

[0229] It should be understood that, in actual practice, any one of the above methods 1 to 4 may be used to determine the importance metric of the reference prediction value as required, and this application does not impose any limitation thereto.

[0230] s32. Select the target weight group for weighted prediction from the target weight list.

[0231] The decoding device selects a target weight group from the target weight list, which can be divided into the following two cases:

[0232] (1) When the number of weight groups contained in the target weight list is equal to 1, there is no need to decode the index of the target weight group from the video stream, and the weight group in the target weight list is directly used as the target weight group for weighted prediction.

[0233] (2) When the number of weight groups included in the target weight list is greater than 1, that is, there are multiple weight groups in the target weight list. For example, the target weight list is expressed as {{2,14},{4,12},{6,10},{8,8}} / 16, and there are 4 weight groups in this target weight list. At this time, in the video bitstream, the index of the target weight group for weighted prediction of the current block can be indicated. The index of the target weight group is encoded using a binary encoding method for truncated unary code or an entropy encoding method with multiple symbols. Among them, the truncated unary code is in the case where the maximum value Max of the syntax element to be encoded is known. Assume the symbol to be encoded is x: If 0 < x < Max, x is binary-coded in the way of unary code; if x = Max, the binary string of x consists entirely of 1s and the length is Max. Then, the decoding device needs to decode the index of the target weight group for weighted prediction from the video bitstream, and select the target weight group from the target weight list according to the index of the target weight group. Among them, the index of the target weight group can indicate the position in the target weight list; for example, the above example target weight list contains 4 weight groups; the decoding device decodes the index of the target weight group from the video bitstream as 2, then determines the position in the target weight list according to the index of the target weight group, that is, it is in the second position in the target weight list (i.e., weight group 2), then the determined target weight group is {4,12} / 16.

[0234] S303. Perform weighted prediction processing on N reference prediction values based on the target weight group to obtain the prediction value of the current block, and the prediction value of the current block is used to reconstruct the decoded image corresponding to the current block.

[0235] The number of weight values actually provided by the target weight group should correspond to the number of reference prediction values. For example, the number of reference prediction values is N, and correspondingly, the number of weight values actually provided by the target weight group is also N. For example, N = 3, the 3 reference prediction values are reference prediction value 1, reference prediction value 2, and reference prediction value 3 respectively. The target weight group can include 3 weight values, which are weight value 1, weight value 2, and weight value 3 respectively; among them, reference prediction value 1 corresponds to weight value 1, reference prediction value 2 corresponds to weight value 2, and reference prediction value 3 corresponds to weight value 3. In the above example, the target weight group can also only include 2 weight values, which are weight value 1 and weight value 2 respectively, and the sum of weight value 1 and weight value 2 is less than 1, then the implied weight value 3 in the target weight group can be obtained by calculation: weight value 3 = 1 - weight value 1 - weight value 2.

[0236] In one embodiment, there are the following two ways (1) and (2) to perform weighted prediction processing on N reference prediction values based on the target weight to obtain the prediction value of the current block:

[0237] (1) The decoding device may use the weight values ​​in the target weight group to perform weighted summation on the N reference prediction values ​​to obtain the prediction value of the current block. The prediction value P(x, y) of the current block may be:

[0238] P(x,y)=(w1·P0(x,y)+w2·P1(x,y)+…+wn·P N-1 (x,y)) / N

[0239] Among them, P(x,y) is the predicted value of the current block, P0(x,y), P1(x,y)...P N-1 (x, y) represent N reference prediction values ​​respectively; w1 represents the weight value corresponding to the first reference prediction value corresponding to the current block (x, y), w2 represents the weight value corresponding to the second reference prediction value corresponding to the current block (x, y), and so on. wn represents the weight value corresponding to the Nth reference prediction value corresponding to the current block (x, y).

[0240] In one embodiment, when N=2, that is, the current block uses two reference prediction values ​​for weighted prediction. In this case, the number of weight values ​​included in the target weight group is one, that is, it includes an implicit weight value. The prediction value P(x, y) of the current block can be:

[0241] P(x,y)=(w(x,y)·P0(x,y)+(1-w(x,y))·P1(x,y)) / 2.

[0242] Among them, P(x,y) is the prediction value of the current block, P0(x,y) and P1(x,y) are the two reference prediction values ​​corresponding to the current block (x,y), w(x,y) is the weight value in the target weight group applied to the first reference prediction value P0(x,y), and 1-w(x,y) is the implicit weight value and is applied to the second reference prediction value P1(x,y).

[0243] (2) In video coding, considering the complexity of prediction value calculation, right shift operation can be used instead of division to implement integer calculation, and the weight values ​​in the target weight group are used to perform weighted processing on N reference prediction values ​​in the form of integer calculation to obtain the prediction value of the current block. The complexity of prediction value calculation can be reduced to a certain extent by using the form of integer calculation.

[0244] For example, if the number of reference prediction values ​​is 2 (i.e., N=2) and the number of weight values ​​in the target weight group is 1, the decoding device can use the weight values ​​in the target weight group to perform weighted processing on the two reference prediction values ​​in the form of integer calculation to obtain the prediction value of the current block. In this case, the prediction value P(x,y) of the current block can be:

[0245] P(x,y)=(w(x,y)×P0(x,y)+(16-w(x,y))×P1(x,y)+8)>>4

[0246] Among them, ">>4" indicates a right shift of 4 bits, that is, a right shift of 4 bits can make the data in the weighted processing process divisible by 16, and 8 is the bias added for rounding. The above P0(x,y), P1(x,y), w(x,y), and P(x,y) are all integers; P(x,y) is the prediction value of the current block, P0(x,y) and P1(x,y) are the two reference prediction values ​​corresponding to the current block (x,y), and w(x,y) is the weight value applied to the first prediction value P0(x,y) (that is, the weight value in the target weight group).

[0247] For another example, “>>6” means a 6-bit right shift, which means that the data in the weighted processing can be divided by 64. 32 is the bias added for rounding. At this time, the predicted value P(x,y) of the current block can be:

[0248] P(x,y)=(w·P0(x,y)+(64-w)·P1(x,y)+32)>>6

[0249] In one embodiment, after obtaining the predicted value for the current block, the predicted value can be superimposed on the residual video signal of the current block to obtain a reconstructed value for the current block. The decoded image corresponding to the current block can then be reconstructed based on the reconstructed value for the current block. The decoded image corresponding to the current block can be used as a reference image for weighted prediction of other coded blocks. Furthermore, the decoded image corresponding to the current block can also be used to reconstruct the current frame of the current block. Ultimately, a video can be reconstructed based on multiple reconstructed video frames.

[0250] In an embodiment of the present application, a composite prediction is performed on a current block in a video stream to obtain N reference prediction values ​​of the current block, where the current block refers to a coding block being decoded in the video stream, and N is an integer greater than 1; a target weight group for weighted prediction is determined for the current block based on the importance of the N reference prediction values; a weighted prediction process is performed on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block. Composite prediction is used in the video decoding process, and the importance of the reference prediction values ​​is fully considered in the composite prediction. In the composite prediction, appropriate weights are adaptively selected for weighted prediction of the current block based on the importance of each reference prediction value in the composite prediction, thereby improving the prediction accuracy of the current block and thus improving the encoding and decoding performance.

[0251] See also Figure 4 , Figure 4This is a flow chart of another video processing method provided in an embodiment of the present application. The video processing method can be executed by an encoding device in a video processing system. The video processing method described in this embodiment may include the following steps S401-S405:

[0252] S401: Divide a current frame in a video to obtain a current block. The current block refers to a coding block currently being encoded in the video. The video may include one or more video frames, and the current frame refers to the video frame currently being encoded. The encoding device may divide the current frame in the video to obtain one or more coding blocks, where the current block refers to any coding block currently being encoded among the one or more coding blocks in the current frame.

[0253] S402 . Perform composite prediction on the current block to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1.

[0254] The above-mentioned N reference prediction values ​​can be derived from N reference blocks, and one reference prediction value corresponds to one reference block. In an embodiment of the present application, the video frame where the reference block is located can be a reference frame, and the video frame where the current block is located is the current frame. Among them, the positional relationship between the N reference blocks and the current block includes any of the following: ① The N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame may belong to different video frames in the video. ② The N reference blocks are located in the same reference frame, and the same reference frame and the current frame may belong to different video frames in the video. ③ One or more reference blocks among the N reference blocks are located in the current frame, and the remaining reference blocks among the N reference blocks are located in one or more reference frames, and one or more reference frames and the current frame belong to different video frames in the video. ④ The N reference blocks and the current block are both located in the current frame.

[0255] According to the positional relationship between the N reference blocks and the current block shown in ①-④ above, the prediction mode of the composite prediction in the embodiment of the present application includes an inter-frame prediction mode, that is, allowing the use of at least two reference frames for inter-frame prediction; it may also include a combined prediction mode, that is, allowing the use of at least one reference frame for inter-frame prediction, and allowing the use of the current frame for intra-frame prediction; it also includes an intra-frame prediction mode, that is, allowing the use of the current frame for intra-frame prediction.

[0256] Corresponding to different modes of compound prediction, compound prediction is performed on the current block to obtain N reference prediction values ​​of the current block, which may include any of the following: ① In the process of compound prediction of the current block, the current block uses N reference blocks to perform inter-frame prediction to obtain N reference prediction values ​​of the current block. At this time, the N reference prediction values ​​of the current block are derived after performing inter-frame prediction using the N reference blocks of the current block respectively. ② In the process of compound prediction of the current block, the current block uses at least one reference prediction value among the N reference prediction values ​​to perform inter-frame prediction and uses the remaining reference blocks among the N reference blocks to perform intra-frame prediction, thereby obtaining N reference prediction values. At this time, some of the N reference prediction values ​​are derived after performing inter-frame prediction using at least one reference block among the N reference blocks of the current block; the remaining reference prediction values ​​are derived after performing intra-frame prediction using the remaining reference blocks among the N reference blocks.

[0257] S403: Determine a target weight group for weighted prediction for the current block according to the importance of the N reference prediction values.

[0258] The importance of the reference prediction value can be comprehensively determined based on factors such as the bit rate consumed during weighted prediction processing and the quality loss of the current block during the encoding process. For example, when the current block uses a certain reference prediction value for weighted prediction, the bit rate consumption increases significantly, indicating that the reference prediction value is not very helpful in reducing bit rate consumption and its importance is low. For another example, when the current block uses a certain reference prediction value for weighted prediction, the quality loss increases significantly, indicating that the reference prediction value is not very helpful in reducing quality loss and its importance is low. The target weight group contains one or more weight values, which will be applied to each reference prediction value during the weighted prediction process. If the importance of a reference prediction value is low, then the reference prediction value corresponds to a smaller weight value in the target weight group. If the importance of a reference prediction value is high, then the reference prediction value corresponds to a larger weight value in the target weight group. That is to say, the target weight group is selected based on the influence of each reference prediction value on factors such as bit rate consumption and quality loss, so that the comprehensive cost of the weighted prediction process (i.e., the cost of bit rate consumption, the cost of quality loss, the cost of bit rate consumption and quality loss) is small and the encoding and decoding performance is better. In one implementation, the bit rate consumed when the weight values ​​in the target weight group are used to perform weighted prediction processing on N reference prediction values ​​is less than a preset bit rate threshold; or, the N reference prediction values ​​are weighted predicted based on the weight values ​​in the target weight group, so that the quality loss of the current block in the encoding process is less than a preset loss threshold; or, the bit rate consumed when the N reference prediction values ​​are weighted predicted based on the weight values ​​in the target weight group is less than the preset bit rate threshold, and the N reference prediction values ​​are weighted predicted based on the weight values ​​in the target weight group, so that the quality loss of the current block in the encoding process is less than the preset loss threshold.

[0259] In one embodiment, one or more weight lists exist during the encoding process, which can be understood as follows: one or more weight lists can be used during the encoding process, each weight list contains one or more weight groups, each weight group contains one or more weight values, the number of weight values ​​contained in each weight group can be the same or different, and the weight values ​​contained in each weight group can be the same or different. In this case, based on the importance of the N reference prediction values, determining the target weight group for weighted prediction for the current block can include steps s41-s42:

[0260] s41. Determine a target weight list from one or more weight lists based on the importance of the N reference prediction values.

[0261] Specifically, the method for determining the target weight list from one or more weight lists according to the importance of the N reference prediction values ​​may include the following methods:

[0262] Method 1: When a weight list exists during the encoding process, the weight list can be directly used as the target weight list.

[0263] Method 2: When there are multiple weight lists during the encoding process, an importance metric of the reference prediction value can be introduced, and a target weight list can be determined from the multiple weight lists based on the importance metric values ​​of N reference prediction values.

[0264] (1) Determine the target weight list based on the absolute value of the difference between the importance metrics of N reference prediction values.

[0265] When the number of weight lists present in the encoding process is M+1, M is a positive integer greater than or equal to 1, and one weight list corresponds to one threshold interval, that is, the number of threshold intervals is also M+1. Then, the encoding device can obtain the importance metric values ​​of N reference prediction values, and calculate the importance difference between the N reference prediction values. The importance difference between any two reference prediction values ​​is measured by the difference between the importance metric values ​​of any two reference prediction values; specifically, the difference between the importance metrics of any two reference prediction values ​​can be calculated, and the difference between the importance metric values ​​of any two reference prediction values ​​can be used as the importance difference between any two reference prediction values; then, the threshold interval in which the absolute value of the importance difference between the N reference prediction values ​​is located is determined; and the weight list corresponding to the threshold interval in which the absolute value of the importance difference between the N reference prediction values ​​is located is determined as the target weight list.

[0266] (2) Compare the importance metrics of the reference prediction values ​​to determine the target weight list.

[0267] The N reference prediction values ​​of the current block may include a first reference prediction value and a second reference prediction value, and the weight lists existing in the encoding process include the first weight list and the second weight list. The encoding device may compare the importance metric value of the first reference prediction value with the importance metric value of the second reference prediction value; if it is determined that the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, the first weight list is determined as the target weight list; if it is determined that the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, the second weight list is determined as the target weight list.

[0268] Optionally, the weight values ​​in the first weight list are opposite to the weight values ​​in the second weight list.

[0269] (3) Use mathematical symbolic functions and the importance measure of reference prediction values ​​to determine the target weight list.

[0270] The N reference prediction values ​​of the current block include a first reference prediction value and a second reference prediction value; the weight lists existing in the encoding process include a first weight list, a second weight list, and a third weight list. The encoding device can call a mathematical symbol function to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain a symbol value. If the above symbol value is a first preset value (such as -1), the first weight list is determined as the target weight list; if the above symbol value is a second preset value (such as 0), the second weight list is determined as the target weight list; if the above symbol value is a third preset value (such as 1), the third weight list is determined as the target weight list.

[0271] (4) The above methods (1), (2), and (3) can be used alone, or they can be combined to determine the target weight list. As an implementation method, there are M+1 weight lists. Method (1) can be used to determine the weight list corresponding to the threshold interval where the absolute value of the importance difference is located as the candidate weight list. Then, the N reference prediction values ​​include the first reference prediction value and the second prediction value. Method (2) is used to compare the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value. If it is determined that the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, the candidate weight list is directly determined as the target weight list; if it is determined that the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, the weight list corresponding to the weight value opposite to the weight value in the candidate weight list is determined as the target weight list.

[0272] Wherein, any one of the N reference prediction values ​​is denoted as reference prediction value i, reference prediction value i is derived from reference block i, the video frame where reference block i is located is reference frame i; i is an integer and i is less than or equal to N; the video frame where the current block is located is the current frame; the importance metric of reference prediction value i can be determined using any of the following methods:

[0273] Method 1: Calculated based on the frame display order (POC) of the current frame in the video and the frame display order of the reference frame i in the video. Specifically, the difference between the frame display order of the current frame in the video and the frame display order of the reference frame i in the video can be calculated, and the absolute value of the difference is used as the importance measure of the reference prediction value i. For example, let the frame display order of the current frame in the video be cur_poc, and the frame display order of the reference frame i in the video be ref_poc; the importance measure of the reference prediction value i is D = abs(cur_poc-ref_poc), where abs() represents the absolute value.

[0274] Method 2: Calculated based on the frame display order of the current frame in the video, the frame display order of the reference frame i in the video, and the quality metric Q. The quality metric Q can be determined based on a variety of situations, which is not limited in this application. Specifically, the quality metric Q of the reference frame i can be derived based on the quantization information of the current block. For example, the quality metric Q can be set to the base_qindex (basic quantization index) of the reference frame i. The base_qindex of any reference frame can be different or the same. In another implementation, the quality metric Q of the reference frame i can also be derived based on other coding information. For example, the quality metric Q of the reference frame i can be derived based on the coding information difference between the coded CU in the reference frame i and the coded CU in the current frame.

[0275] As an implementation method, the encoding device can calculate the difference between the frame display order of the current frame in the video and the frame display order of the reference frame i in the video; then, using the objective function, determine the importance measure value of the reference prediction value i based on the above difference, quality measure Q and importance measure value list.

[0276] In one embodiment, the importance metric value of the reference frame i can be calculated based on the orientation relationship between the reference frame i and the current frame and the quality metric Q. As an implementation method, a correspondence between the reference orientation relationship and the reference importance metric value can be established. For example, if the reference orientation relationship is that the reference frame is before the current frame, the reference importance metric value can correspond to a first value; if the reference orientation relationship is that the reference frame is after the current frame, the reference importance metric value can correspond to a second value. Then, the encoding device can calculate the importance metric value of the reference prediction value i based on the reference importance metric value and the quality metric Q corresponding to the reference frame i.

[0277] Method 3: Calculate the importance metric score of reference frame i based on the calculation results of method 1 and method 2, and sort the importance metric scores of the reference frames corresponding to N reference prediction values ​​in ascending order, and determine the index of reference frame i in the sorting as the importance metric value of reference prediction value i.

[0278] As an implementation method, the first importance metric value of the reference prediction value i can be calculated by the above method one, and the second importance metric value of the reference prediction value i can be calculated by the above method two, and then the importance metric score (such as score) of the reference frame i is calculated based on the first importance metric value and the second importance metric value, and then the importance metric scores of the reference frames corresponding to the N reference prediction values ​​are sorted in order from small to large, and the index of the reference frame i in the sorting is determined as the importance metric value of the reference prediction value i.

[0279] Calculating the importance metric score of reference frame i based on the first importance metric value and the second importance metric value can be done as follows: ① Taking a weighted sum of the first importance metric value and the second importance metric value to obtain the importance metric value of reference prediction value i. ② Taking an average of the first importance metric value and the second importance metric value to obtain the importance metric value of reference prediction value i.

[0280] Method 4: In order to obtain a more accurate importance metric value, the calculation results of method 1, method 2 or method 3 can be adjusted based on the prediction mode of reference prediction value i to obtain the importance metric value of reference prediction value i; wherein the prediction mode of reference prediction value i includes any one of the following: inter-frame prediction mode, intra-frame prediction mode.

[0281] It should be understood that, in actual practice, any one of the above methods 1 to 4 may be used to determine the importance metric of the reference prediction value as required, and this application does not impose any limitation thereto.

[0282] s42. Select a target weight group for weighted prediction from the target weight list.

[0283] The target weight list contains one or more weight groups.

[0284] (1) When the number of weight groups included in the target weight list is equal to 1, the weight group in the target weight list is directly used as the target weight group for weighted prediction.

[0285] (2) The target weight list contains multiple weight groups, and the target weight group for weighted prediction is selected from the target weight list.

[0286] When performing weighted prediction processing, bit rate consumption may occur, and quality loss of the coding block may occur during the encoding process. Therefore, the encoding device can first obtain the encoding performance when performing weighted prediction processing on N reference prediction values ​​using each weight group in the target weight list, and then determine the weight group with the best encoding performance in the target weight list as the target weight group. As an implementation method, the bit rate consumed when performing weighted prediction processing on N reference prediction values ​​using the weight values ​​in the target weight group is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list; or, the quality loss of the corresponding current block during the encoding process when performing weighted prediction processing on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list; or, the bit rate consumed when performing weighted prediction processing on N reference prediction values ​​using the weight values ​​in the target weight group is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list, and the quality loss of the corresponding current block during the encoding process when performing weighted prediction processing on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list.

[0287] It should be understood that when selecting the target weight group for weighted prediction on the encoding side, it is necessary to continuously try various weight groups from the target weight list to obtain the target weight group for weighted prediction. On the decoding side, however, there is no need for continuous attempts. Instead, the index of the target weight group to be used is indicated in the video bitstream. The decoding side only needs to decode the target weight group index from the video bitstream and find the target weight group from the target weight list based on the target weight group index.

[0288] S404 : Perform weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block.

[0289] The specific implementation of step S404 can refer to the specific implementation of step S303 above, which will not be repeated here.

[0290] S405: Encode the video based on the predicted value of the current block to generate a video stream. The encoding device may encode the video based on the predicted value of the current block and the index of the target weight group to generate a video stream. The encoding based on the predicted value of the current block and the index of the target weight group to generate a video stream can be found in the above-mentioned corresponding encoding description and is not further described here.

[0291] In an embodiment of the present application, the current frame in the video is divided and processed to obtain the current block, which refers to the coding block being encoded in the video; composite prediction is performed on the current block to obtain N reference prediction values ​​of the current block; based on the importance of the N reference prediction values, a target weight group for weighted prediction is determined for the current block; based on the target weight group, the N reference prediction values ​​are weighted predicted to obtain the prediction value of the current block; and based on the prediction value of the current block, the video is encoded to generate a video bitstream. Composite prediction is used in the video encoding process, and the importance of the reference prediction values ​​derived in the composite prediction is used to implement weighted prediction by adaptively selecting appropriate weights for the current block based on the importance of each reference prediction value in the composite prediction, which can improve the prediction accuracy of the current block and thus improve the encoding and decoding performance.

[0292] See Figure 5 , Figure 5 It is a structural diagram of a video processing device provided in an embodiment of the present application. The video processing device can be set in the computer device provided in an embodiment of the present application. The computer device can be the decoding device mentioned in the above method embodiment. Figure 5 The video processing device shown can be a computer program (including program code) running in a computer device, and the video processing device can be used to perform Figure 3 Some or all of the steps in the method embodiment shown. Figure 5 , the video processing device may include the following units:

[0293] The processing unit 501 is configured to perform composite prediction on a current block in a video stream to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1.

[0294] A determining unit 502 is configured to determine a target weight group for weighted prediction for a current block based on the importance of the N reference prediction values, the target weight group including one or more weight values;

[0295] The processing unit 501 is further configured to perform weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block.

[0296] In one embodiment, the N reference prediction values ​​are derived from N reference blocks; one reference prediction value corresponds to one reference block; the video frame where the reference block is located is the reference frame, and the video frame where the current block is located is the current frame;

[0297] The positional relationship between the N reference blocks and the current block includes any of the following:

[0298] The N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame belong to different video frames in the video stream;

[0299] The N reference blocks are located in the same reference frame, and the same reference frame and the current frame belong to different video frames in the video stream;

[0300] One or more of the N reference blocks are located in the current frame, and the remaining reference blocks in the N reference blocks are located in one or more reference frames, and the one or more reference frames and the current frame belong to different video frames in the video stream;

[0301] The N reference blocks and the current block are all located in the current frame.

[0302] In one embodiment, the processing unit 501 is further configured to:

[0303] Determine whether the current block meets the conditions for adaptive weighted prediction;

[0304] If the current block meets the conditions for adaptive weighted prediction, a target weight group for weighted prediction is determined for the current block based on the importance of the N reference prediction values.

[0305] In one embodiment, the condition that the current block satisfies adaptive weighted prediction includes at least one of the following:

[0306] The sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that the coding block in the frame sequence allows the use of adaptive weighted prediction;

[0307] The slice header of the current slice to which the current block belongs includes a second indication field, and the second indication field indicates that the coding blocks in the current slice are allowed to use adaptive weighted prediction;

[0308] The frame header of the current frame where the current block is located includes a third indication field, and the third indication field indicates that the coding block in the current frame allows the use of adaptive weighted prediction;

[0309] In the composite prediction process, the current block uses at least two reference frames for inter-frame prediction;

[0310] In the composite prediction process, the current block uses at least one reference frame for inter prediction and the current frame for intra prediction;

[0311] The motion type of the current block is the specified motion type;

[0312] The current block uses a preset motion vector prediction mode;

[0313] The current block uses a preset interpolation filter;

[0314] The current block does not use a specific encoding tool;

[0315] The reference frame used in the composite prediction process of the current block meets specific conditions, which include one or more of the following: the orientation relationship between the used reference frame and the current frame in the video stream meets a preset relationship; the absolute value of the importance difference between the reference prediction values ​​corresponding to the used reference frames is greater than or equal to a preset threshold;

[0316] Among them, the orientation relationship satisfies the preset relationship including any of the following: all the reference frames used are located before the current frame; all the reference frames used are located after the current frame; some of the reference frames used are located before the current frame, and the remaining reference frames are located after the current frame.

[0317] In one embodiment, the video stream includes one or more weight lists, each weight list includes one or more weight groups, each weight group includes one or more weight values, the number of weight values ​​in each weight group can be the same or different, and the weight values ​​included in each weight group can be the same or different.

[0318] The determining unit 502 may be specifically configured to:

[0319] determining a target weight list from one or more weight lists based on the importance of the N reference prediction values;

[0320] From the Target Weights list, select the target weight set to use for weighted predictions.

[0321] In one embodiment, the number of weight lists in the video stream is M+1, one weight list corresponds to one threshold interval, and M is a positive integer greater than or equal to 1; the determining unit 502 may be specifically configured to:

[0322] Obtain importance metrics of N reference prediction values, and calculate the importance differences between the N reference prediction values. The importance difference between any two reference prediction values ​​is measured by the difference between the importance metrics of the any two reference prediction values.

[0323] Determine a threshold interval where the absolute value of the significance difference between N reference prediction values ​​lies;

[0324] The weight list corresponding to the threshold interval where the absolute value of the importance difference between the N reference prediction values ​​lies is determined as the target weight list.

[0325] In one embodiment, the N reference prediction values ​​of the current block include a first reference prediction value and a second reference prediction value; the video stream includes a first weight list and a second weight list; and the determination unit 502 may be specifically configured to:

[0326] comparing the importance metric value of the first reference prediction value with the importance metric value of the second reference prediction value;

[0327] If the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, determining the first weight list as the target weight list;

[0328] If the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, determining the second weight list as the target weight list;

[0329] The weight values ​​in the first weight list are opposite to the weight values ​​in the second weight list.

[0330] In one embodiment, the N reference prediction values ​​of the current block include a first reference prediction value and a second reference prediction value; the video stream includes a first weight list, a second weight list, and a third weight list; the determining unit 502 may be specifically configured to:

[0331] Calling a mathematical sign function to process a difference between the importance measure value of the first reference prediction value and the importance measure value of the second reference prediction value to obtain a sign value;

[0332] If the symbol value is the first preset value, determining the first weight list as the target weight list;

[0333] If the symbol value is the second preset value, determining the second weight list as the target weight list;

[0334] If the symbol value is the third preset value, determining the third weight list as the target weight list;

[0335] The first weight list, the second weight list and the third weight list are different weight lists respectively; or, two weight lists among the first weight list, the second weight list and the third weight list are allowed to be the same weight list.

[0336] In one embodiment, any one of the N reference prediction values ​​is denoted as reference prediction value i, where reference prediction value i is derived from reference block i, and the video frame where reference block i is located is reference frame i; i is an integer and i is less than or equal to N; and the video frame where the current block is located is the current frame;

[0337] The importance measure of the reference prediction value i is determined using any of the following methods:

[0338] Method 1: Calculated based on the frame display order of the current frame in the video stream and the frame display order of the reference frame i in the video stream;

[0339] Method 2: Calculated based on the frame display order of the current frame in the video stream, the frame display order of the reference frame i in the video stream, and the quality metric Q;

[0340] Method 3: Calculate the importance metric score of reference frame i based on the calculation results of method 1 and method 2, sort the importance metric scores of the reference frames corresponding to the N reference prediction values ​​in ascending order, and determine the index of reference frame i in the sort as the importance metric value of reference prediction value i;

[0341] Method 4: Based on the prediction model of the reference prediction value i, the calculation results of method 1, method 2 or method 3 are adjusted to obtain the importance measurement value of the reference prediction value i;

[0342] The prediction mode of the reference prediction value i includes any one of the following: inter-frame prediction mode and intra-frame prediction mode.

[0343] In one embodiment, the number of weight groups included in the target weight list is greater than 1; the determining unit 502 may be specifically configured to:

[0344] Decoding the index of the target weight group for weighted prediction from the video bitstream;

[0345] According to the index of the target weight group, select the target weight group from the target weight list;

[0346] The index of the target weight group is encoded using a truncated unary code for binary encoding, or a multi-symbol entropy encoding method.

[0347] In one embodiment, the processing unit 501 may be specifically configured to:

[0348] The weight values ​​in the target weight group are used to perform weighted summation on the N reference prediction values ​​to obtain the prediction value of the current block; or

[0349] The N reference prediction values ​​are weighted by using the weight values ​​in the target weight group in the form of integer calculation to obtain the prediction value of the current block.

[0350] In an embodiment of the present application, a composite prediction is performed on the current block in a video stream to obtain N reference prediction values ​​for the current block. The current block refers to the coded block being decoded in the video stream. Based on the importance of the N reference prediction values, a target weight group for weighted prediction is adaptively selected for the current block. Based on the target weight group, a weighted prediction process is performed on the N reference prediction values ​​to obtain a prediction value for the current block. The prediction value for the current block is used to reconstruct the decoded image corresponding to the current block. Composite prediction is used in the video decoding process, and the importance of the reference prediction values ​​is fully considered in the composite prediction. Weighted prediction is performed by determining appropriate weights for the current block based on the importance of each reference prediction value in the composite prediction. This can improve the prediction accuracy of the current block, thereby improving codec performance.

[0351] See Figure 6 , Figure 6 It is a structural diagram of a video processing device provided in an embodiment of the present application. The video processing device can be set in the computer device provided in an embodiment of the present application. The computer device can be the encoding device mentioned in the above method embodiment. Figure 6 The video processing device shown can be a computer program (including program code) running in a computer device, and the video processing device can be used to perform Figure 4 Some or all of the steps in the method embodiment shown. Figure 6 , the video processing device may include the following units:

[0352] The processing unit 601 is used to divide the current frame in the video into a current block;

[0353] The processing unit 601 is further configured to perform composite prediction on the current block to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1;

[0354] A determining unit 602 is configured to determine a target weight group for weighted prediction for a current block based on the importance of the N reference prediction values, where the target weight group includes one or more weight values;

[0355] The processing unit 601 is further configured to perform weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, where the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block;

[0356] The processing unit 601 is further configured to encode the video based on the prediction value of the current block to generate a video bit stream.

[0357] In one embodiment, the determining unit 602 may be specifically configured to:

[0358] According to the importance of the N reference prediction values, a target weight list is determined from one or more weight lists; each weight list contains one or more weight groups, each weight group contains one or more weight values, the number of weight values ​​contained in each weight group can be the same or different, and the values ​​of the weight values ​​contained in each weight group can be the same or different;

[0359] From the Target Weights list, select the target weight set to use for weighted predictions.

[0360] In one embodiment, the bit rate consumed when performing weighted prediction processing on N reference prediction values ​​using the weight values ​​in the target weight group is less than a preset bit rate threshold; or,

[0361] Performing weighted prediction processing on the N reference prediction values ​​based on the weight values ​​in the target weight group so that the quality loss of the current block during the encoding process is less than a preset loss threshold; or

[0362] The bit rate consumed when performing weighted prediction processing on N reference prediction values ​​using the weight values ​​in the target weight group is less than a preset bit rate threshold, and weighted prediction processing is performed on the N reference prediction values ​​based on the weight values ​​in the target weight group, so that the quality loss of the current block during the encoding process is less than the preset loss threshold.

[0363] In one embodiment, the target weight group is the weight group with the best encoding performance in the target weight list;

[0364] Among them, the optimal coding performance includes: the bit rate consumed when the weight values ​​in the target weight group are used to perform weighted prediction processing on N reference prediction values ​​is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list; or, the quality loss of the corresponding current block in the encoding process when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list; or, the bit rate consumed when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list, and the quality loss of the corresponding current block in the encoding process when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list.

[0365] In an embodiment of the present application, the current frame in the video is divided and processed to obtain the current block, which refers to the coding block being encoded in the video; a composite prediction is performed on the current block to obtain N reference prediction values ​​for the current block; based on the importance of the N reference prediction values, a target weight group for weighted prediction is determined for the current block; based on the target weight group, the N reference prediction values ​​are weighted predicted to obtain a prediction value for the current block; and based on the prediction value of the current block, the video is encoded to generate a video bitstream. Using composite prediction in the video encoding process and fully considering the importance of the reference prediction values ​​to determine appropriate weights for the current block based on the importance of each reference prediction value in the composite prediction can improve the prediction accuracy of the current block, thereby improving the encoding and decoding performance.

[0366] Furthermore, the present invention also provides a schematic diagram of the structure of a computer device. Figure 7The computer device may be the encoding device or decoding device described above. The computer device may include a processor 701, an input device 702, an output device 703, and a memory 704. The processor 701, input device 702, output device 703, and memory 704 are connected via a bus. The memory 704 is used to store a computer program, which includes program instructions. The processor 701 is used to execute the program instructions stored in the memory 704.

[0367] When the computer device is the above-mentioned decoding device, in the embodiment of the present application, the processor 701 performs the following operations by running the executable program code in the memory 704:

[0368] Perform composite prediction on the current block in the video stream to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1;

[0369] Determining a target weight group for weighted prediction for the current block based on the importance of the N reference prediction values, the target weight group including one or more weight values;

[0370] The N reference prediction values ​​are weightedly predicted based on the target weight group to obtain the prediction value of the current block. The prediction value of the current block is used to reconstruct the decoded image corresponding to the current block.

[0371] In one embodiment, the N reference prediction values ​​are derived from N reference blocks; one reference prediction value corresponds to one reference block; the video frame where the reference block is located is the reference frame, and the video frame where the current block is located is the current frame;

[0372] The positional relationship between the N reference blocks and the current block includes any of the following:

[0373] The N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame belong to different video frames in the video stream;

[0374] The N reference blocks are located in the same reference frame, and the same reference frame and the current frame belong to different video frames in the video stream;

[0375] One or more of the N reference blocks are located in the current frame, and the remaining reference blocks in the N reference blocks are located in one or more reference frames, and the one or more reference frames and the current frame belong to different video frames in the video stream;

[0376] The N reference blocks and the current block are all located in the current frame.

[0377] In one embodiment, the processor 701 is further configured to:

[0378] Determine whether the current block meets the conditions for adaptive weighted prediction;

[0379] If the current block meets the conditions for adaptive weighted prediction, a target weight group for weighted prediction is determined for the current block based on the importance of the N reference prediction values.

[0380] In one embodiment, the condition that the current block satisfies adaptive weighted prediction includes at least one of the following:

[0381] The sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that the coding block in the frame sequence allows the use of adaptive weighted prediction;

[0382] The slice header of the current slice to which the current block belongs includes a second indication field, and the second indication field indicates that the coding blocks in the current slice are allowed to use adaptive weighted prediction;

[0383] The frame header of the current frame where the current block is located includes a third indication field, and the third indication field indicates that the coding block in the current frame allows the use of adaptive weighted prediction;

[0384] In the composite prediction process, the current block uses at least two reference frames for inter-frame prediction;

[0385] In the composite prediction process, the current block uses at least one reference frame for inter prediction and the current frame for intra prediction;

[0386] The motion type of the current block is the specified motion type;

[0387] The current block uses a preset motion vector prediction mode;

[0388] The current block uses a preset interpolation filter;

[0389] The current block does not use a specific encoding tool;

[0390] The reference frame used in the composite prediction process of the current block meets specific conditions, which include one or more of the following: the orientation relationship between the used reference frame and the current frame in the video stream meets a preset relationship; the absolute value of the importance difference between the reference prediction values ​​corresponding to the used reference frames is greater than or equal to a preset threshold;

[0391] Among them, the orientation relationship satisfies the preset relationship including any of the following: all the reference frames used are located before the current frame; all the reference frames used are located after the current frame; some of the reference frames used are located before the current frame, and the remaining reference frames are located after the current frame.

[0392] In one embodiment, the video stream includes one or more weight lists, each weight list includes one or more weight groups, each weight group includes one or more weight values, the number of weight values ​​included in each weight group can be the same or different, and the weight values ​​included in each weight group can be the same or different. The processor 701 can be specifically configured to:

[0393] determining a target weight list from one or more weight lists based on the importance of the N reference prediction values;

[0394] From the Target Weights list, select the target weight set to use for weighted predictions.

[0395] In one embodiment, the number of weight lists in the video stream is M+1, one weight list corresponds to one threshold interval, and M is a positive integer greater than or equal to 1; the processor 701 may be specifically configured to:

[0396] Obtain importance metrics of N reference prediction values, and calculate the importance differences between the N reference prediction values. The importance difference between any two reference prediction values ​​is measured by the difference between the importance metrics of the any two reference prediction values.

[0397] Determine a threshold interval where the absolute value of the significance difference between N reference prediction values ​​lies;

[0398] The weight list corresponding to the threshold interval where the absolute value of the importance difference between the N reference prediction values ​​lies is determined as the target weight list.

[0399] In one embodiment, the N reference prediction values ​​of the current block include a first reference prediction value and a second reference prediction value; the video stream includes a first weight list and a second weight list; the processor 701 may be specifically configured to:

[0400] comparing the importance metric value of the first reference prediction value with the importance metric value of the second reference prediction value;

[0401] If the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, determining the first weight list as the target weight list;

[0402] If the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, determining the second weight list as the target weight list;

[0403] The weight values ​​in the first weight list are opposite to the weight values ​​in the second weight list.

[0404] In one embodiment, the N reference prediction values ​​of the current block include a first reference prediction value and a second reference prediction value; the video stream includes a first weight list, a second weight list, and a third weight list; when the processor 701 determines a target weight list from one or more weight lists based on the importance of the N reference prediction values, it can be specifically configured to:

[0405] Calling a mathematical sign function to process a difference between the importance measure value of the first reference prediction value and the importance measure value of the second reference prediction value to obtain a sign value;

[0406] If the symbol value is the first preset value, determining the first weight list as the target weight list;

[0407] If the symbol value is the second preset value, determining the second weight list as the target weight list;

[0408] If the symbol value is the third preset value, determining the third weight list as the target weight list;

[0409] The first weight list, the second weight list and the third weight list are different weight lists respectively; or, two weight lists among the first weight list, the second weight list and the third weight list are allowed to be the same weight list.

[0410] In one embodiment, any one of the N reference prediction values ​​is denoted as reference prediction value i, where reference prediction value i is derived from reference block i, and the video frame where reference block i is located is reference frame i; i is an integer and i is less than or equal to N; and the video frame where the current block is located is the current frame;

[0411] The importance measure of the reference prediction value i is determined using any of the following methods:

[0412] Method 1: Calculated based on the frame display order of the current frame in the video stream and the frame display order of the reference frame i in the video stream;

[0413] Method 2: Calculated based on the frame display order of the current frame in the video stream, the frame display order of the reference frame i in the video stream, and the quality metric Q;

[0414] Method 3: Calculate the importance metric score of reference frame i based on the calculation results of method 1 and method 2, sort the importance metric scores of the reference frames corresponding to the N reference prediction values ​​in ascending order, and determine the index of reference frame i in the sort as the importance metric value of reference prediction value i;

[0415] Method 4: adjusting the calculation results of method 1, method 2, or method 3 based on the prediction model of the reference prediction value i to obtain an importance measure value of the reference prediction value i;

[0416] The prediction mode of the reference prediction value i includes any one of the following: inter-frame prediction mode and intra-frame prediction mode.

[0417] In one embodiment, the number of weight groups included in the target weight list is greater than 1; the processor 701 may be specifically configured to:

[0418] Decoding the index of the target weight group for weighted prediction from the video bitstream;

[0419] According to the index of the target weight group, select the target weight group from the target weight list;

[0420] The index of the target weight group is encoded using a truncated unary code for binary encoding, or a multi-symbol entropy encoding method.

[0421] In one embodiment, the processor 701 may be specifically configured to:

[0422] The weight values ​​in the target weight group are used to perform weighted summation on the N reference prediction values ​​to obtain the prediction value of the current block; or

[0423] The N reference prediction values ​​are weighted by using the weight values ​​in the target weight group in the form of integer calculation to obtain the prediction value of the current block.

[0424] In an embodiment of the present application, a composite prediction is performed on the current block in a video stream to obtain N reference prediction values ​​for the current block. The current block refers to the coded block being decoded in the video stream. Based on the importance of the N reference prediction values, a target weight group for weighted prediction is determined for the current block. Based on the target weight group, a weighted prediction process is performed on the N reference prediction values ​​to obtain a prediction value for the current block. The prediction value for the current block is used to reconstruct the decoded image corresponding to the current block. Composite prediction is used in the video decoding process, and the importance of the reference prediction values ​​is fully considered in the composite prediction. Weighted prediction is performed by determining appropriate weights for the current block based on the importance of each reference prediction value in the composite prediction. This can improve the prediction accuracy of the current block, thereby improving codec performance.

[0425] Optionally, when the computer device is the above-mentioned encoding device, in the embodiment of the present application, the processor 701 performs the following operations by running the executable program code in the memory 704:

[0426] Divide the current frame in the video to obtain the current block;

[0427] Perform composite prediction on the current block to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1;

[0428] Determining a target weight group for weighted prediction for the current block based on the importance of the N reference prediction values, the target weight group including one or more weight values;

[0429] Performing weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, which is used to reconstruct a decoded image corresponding to the current block;

[0430] The video is encoded based on the predicted value of the current block to generate a video bitstream.

[0431] In one embodiment, the processor 701 may be specifically configured to:

[0432] According to the importance of the N reference prediction values, a target weight list is determined from one or more weight lists; each weight list contains one or more weight groups, each weight group contains one or more weight values, the number of weight values ​​contained in each weight group can be the same or different, and the values ​​of the weight values ​​contained in each weight group can be the same or different;

[0433] From the Target Weights list, select the target weight set to use for weighted predictions.

[0434] In one embodiment, the bit rate consumed when performing weighted prediction processing on the N reference prediction values ​​using the weight values ​​in the target weight group is less than a preset bit rate threshold; or

[0435] Performing weighted prediction processing on the N reference prediction values ​​based on the weight values ​​in the target weight group so that the quality loss of the current block during the encoding process is less than a preset loss threshold; or

[0436] The bit rate consumed when performing weighted prediction processing on N reference prediction values ​​using the weight values ​​in the target weight group is less than a preset bit rate threshold, and weighted prediction processing is performed on the N reference prediction values ​​based on the weight values ​​in the target weight group, so that the quality loss of the current block during the encoding process is less than the preset loss threshold.

[0437] In one embodiment, the target weight group is the weight group with the best encoding performance in the target weight list;

[0438] Among them, the optimal coding performance includes: the bit rate consumed when the weight values ​​in the target weight group are used to perform weighted prediction processing on N reference prediction values ​​is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list; or, the quality loss of the corresponding current block in the encoding process when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list; or, the bit rate consumed when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list, and the quality loss of the corresponding current block in the encoding process when the weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list.

[0439] In an embodiment of the present application, the current frame in the video is divided and processed to obtain the current block, which refers to the coding block being encoded in the video; a composite prediction is performed on the current block to obtain N reference prediction values ​​for the current block; based on the importance of the N reference prediction values, a target weight group for weighted prediction is determined for the current block; based on the target weight group, the N reference prediction values ​​are weighted predicted to obtain a prediction value for the current block; and based on the prediction value of the current block, the video is encoded to generate a video bitstream. Using composite prediction in the video encoding process and fully considering the importance of the reference prediction values ​​to determine appropriate weights for the current block based on the importance of each reference prediction value in the composite prediction can improve the prediction accuracy of the current block, thereby improving the encoding and decoding performance.

[0440] In addition, it should be noted that the present invention also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the above Figure 3 and Figure 4 The method in the corresponding embodiment will therefore not be described in detail here. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0441] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, so that the computer device can perform the above-mentioned Figure 3 and Figure 4 The method in the corresponding embodiment will therefore not be described in detail here.

[0442] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0443] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that implementing all or part of the processes of the above embodiment and making equivalent changes in accordance with the claims of this application still fall within the scope of the invention.

Claims

1. A video processing method, characterized in that: include: Predicting a current block in a current frame using at least two reference frames to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1; determining a target weight group for weighted prediction for the current block according to an orientation relationship between the reference frame and the current frame, the target weight group comprising one or more weight values; Performing weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block; The predicted value of the current block is used to reconstruct a decoded image corresponding to the current block.

2. The method according to claim 1, wherein The method further comprises: Determining whether the current block meets a condition for adaptive weighted prediction; If the current block meets the conditions for adaptive weighted prediction, a step of determining a target weight group for weighted prediction for the current block according to the orientation relationship between the reference frame and the current frame is performed.

3. The method according to claim 2, wherein The condition that the current block satisfies adaptive weighted prediction includes at least one of the following: The sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that the coding block in the frame sequence allows the use of adaptive weighted prediction; The slice header of the current slice to which the current block belongs includes a second indication field, and the second indication field indicates that the coding blocks in the current slice are allowed to use adaptive weighted prediction; The frame header of the current frame where the current block is located includes a third indication field, and the third indication field indicates that the coding block in the current frame allows the use of adaptive weighted prediction; The block header of the current block includes a fourth indication field, and the fourth indication field indicates that the current block allows the use of adaptive weighted prediction; In the prediction process, the current block performs inter-frame prediction using at least two reference frames; In the prediction process, the current block uses at least two reference frames for inter-frame prediction and uses the current frame for intra-frame prediction; The motion type of the current block is a specified motion type; The current block uses a preset motion vector prediction mode; The current block uses a preset interpolation filter; The current block does not use a specific coding tool; The absolute value of the importance difference between the reference prediction values ​​corresponding to the used reference frames is greater than or equal to a preset threshold.

4. The method according to claim 3, wherein Any one of the N reference prediction values ​​is denoted as reference prediction value i, where the reference prediction value i is derived from reference block i, and the video frame where the reference block i is located is reference frame i; i is an integer and i is less than or equal to N; The importance metric of the reference prediction value i is determined by any of the following methods: Method 1: Calculated based on the frame display order of the current frame in the video stream and the frame display order of the reference frame i in the video stream; Method 2: Calculated based on the frame display order of the current frame in the video stream, the frame display order of the reference frame i in the video stream, and the quality metric Q; Method 3: Calculate the importance metric score of the reference frame i based on the calculation results of method 1 and method 2, sort the importance metric scores of the reference frames corresponding to the N reference prediction values ​​in ascending order, and determine the index of the reference frame i in the sort as the importance metric value of the reference prediction value i; Method 4: adjusting the calculation results of method 1, method 2, or method 3 based on the prediction mode of the reference prediction value i to obtain an importance measure value of the reference prediction value i; The prediction mode of the reference prediction value i includes any one of the following: an inter-frame prediction mode and an intra-frame prediction mode.

5. The method according to any one of claims 1 to 4, characterized in that The step of determining a target weight group for weighted prediction for the current block according to the orientation relationship between the reference frame and the current frame includes: Determining a target weight list from at least one weight list according to the orientation relationship between the reference frame and the current frame; each of the weight lists includes one or more weight groups, each of the weight groups includes one or more weight values, the number of weight values ​​included in each weight group may be the same or different, and the weight values ​​included in each weight group may be the same or different; A target weight set for weighted prediction is selected from the target weight list.

6. The method according to claim 5, wherein The orientation relationship between the reference frame and the current frame includes any one of the following: The reference frames used are all located before the current frame; The reference frames used are all located after the current frame; Some of the reference frames used are located before the current frame, and the remaining reference frames are located after the current frame; Among them, if the reference frames used are all located before the current frame, or if the reference frames used are all located after the current frame, then the orientation relationship indicates that the reference frames used are from the same direction; if some of the reference frames used are located before the current frame and the remaining reference frames are located after the current frame, then the orientation relationship indicates that the reference frames used are from different directions.

7. The method according to claim 6, wherein The reference frames used are all located before the current frame, including: the frame display order of the reference frames used are all located before the frame display order of the current frame; or the frame display time of the reference frames used are all earlier than the frame display time of the current frame; The reference frames used are all located after the current frame, including: the frame display order of the reference frames used are all located after the frame display order of the current frame; or the frame display time of the reference frames used are all later than the frame display time of the current frame; Some of the reference frames used are located before the current frame, and the remaining reference frames are located after the current frame, including: the frame display order of some of the reference frames used is located before the current frame, and the frame display order of the remaining reference frames is located after the frame display order of the current frame; or, the frame display time of some of the reference frames used is earlier than the frame display time of the current frame, and the frame display time of the remaining reference frames is later than the frame display time of the current frame.

8. The method according to claim 6, wherein The at least one weight list includes a first weight list and a second weight list; The determining a target weight list from at least one weight list according to the orientation relationship between the reference frame and the current frame includes: If the reference frames used in the orientation relationship indication are from the same direction, determining the first weight list as the target weight list; If the orientation relationship indicates that the used reference frames are from different directions, the second weight list is determined as the target weight list.

9. The method according to claim 5, wherein The number of weight groups included in the target weight list is greater than 1; The step of selecting a target weight group for weighted prediction from the target weight list comprises: Decoding the index of the target weight group for weighted prediction from the video bitstream; The target weight group is selected from the target weight list according to the index of the target weight group.

10. The method according to claim 9, wherein The index of the target weight group is encoded using a truncated unary code for binary encoding, or an entropy encoding method using multiple symbols.

11. The method according to claim 1, wherein The performing weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain the prediction value of the current block includes: Using the weight values ​​in the target weight group to perform weighted summation processing on the N reference prediction values ​​respectively to obtain the prediction value of the current block; or, The N reference prediction values ​​are weighted respectively using the weight values ​​in the target weight group in the form of integer calculation to obtain the prediction value of the current block.

12. A video processing method, characterized in that: include: Divide the current frame in the video to obtain the current block; Predicting the current block using at least two reference frames to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1; determining a target weight group for weighted prediction for the current block according to an orientation relationship between the reference frame and the current frame, the target weight group comprising one or more weight values; Performing weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, where the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block; The video is encoded based on the prediction value of the current block.

13. The method according to claim 12, wherein: The step of determining a target weight group for weighted prediction for the current block according to the orientation relationship between the reference frame and the current frame includes: Determining a target weight list from at least one weight list according to the orientation relationship between the reference frame and the current frame; each of the weight lists includes one or more weight groups, each of the weight groups includes one or more weight values, the number of weight values ​​included in each weight group may be the same or different, and the weight values ​​included in each weight group may be the same or different; A target weight set for weighted prediction is selected from the target weight list.

14. The method according to claim 13, wherein The bit rate consumed when performing weighted prediction processing on the N reference prediction values ​​using the weight values ​​in the target weight group is less than a preset bit rate threshold; or performing weighted prediction processing on the N reference prediction values ​​based on the weight values ​​in the target weight group, so that the quality loss of the current block during the encoding process is less than a preset loss threshold; or, The bit rate consumed when weighted prediction processing is performed on the N reference prediction values ​​using the weight values ​​in the target weight group is less than a preset bit rate threshold, and weighted prediction processing is performed on the N reference prediction values ​​based on the weight values ​​in the target weight group, so that the quality loss of the current block during the encoding process is less than a preset loss threshold.

15. The method according to claim 13, wherein The target weight group is the weight group with the best encoding performance in the target weight list; Among them, the optimal encoding performance includes: the bit rate consumed when weighted prediction processing is performed on N reference prediction values ​​using the weight values ​​in the target weight group is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list; or, the quality loss of the corresponding current block in the encoding process when weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list; or, the bit rate consumed when weighted prediction processing is performed on N reference prediction values ​​using the weight values ​​in the target weight group is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list, and the quality loss of the corresponding current block in the encoding process when weighted prediction processing is performed on N reference prediction values ​​based on the weight values ​​in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list.

16. A method for storing or sending a video stream, characterized in that: The video code stream is generated according to the method according to any one of claims 12 to 15, or the video code stream is decoded based on the method according to any one of claims 1 to 11.

17. A video processing device, characterized in that: include: a processing unit, configured to predict a current block in a current frame using at least two reference frames to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1; a determining unit, configured to determine a target weight group for weighted prediction for the current block according to an orientation relationship between the reference frame and the current frame, the target weight group comprising one or more weight values; The processing unit is further configured to perform weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block; and to reconstruct a decoded image corresponding to the current block using the prediction value of the current block.

18. A video processing device, characterized in that: include: A processing unit, configured to divide a current frame in the video into a current block; The processing unit is further configured to predict the current block using at least two reference frames to obtain N reference prediction values ​​of the current block, where N is an integer greater than 1; a determining unit, configured to determine a target weight group for weighted prediction for the current block according to an orientation relationship between the reference frame and the current frame, the target weight group comprising one or more weight values; The processing unit is further configured to perform weighted prediction processing on the N reference prediction values ​​based on the target weight group to obtain a prediction value of the current block, wherein the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block; The processing unit is further configured to perform encoding processing on the video based on the prediction value of the current block.

19. A computer device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium storing a computer program, wherein when the computer program is executed by the processor, the video processing method according to any one of claims 1 to 15 is executed; or a computer-readable storage medium storing a video stream formed by the computer program, wherein when the computer program is executed by the processor, the method for storing or sending a video stream according to claim 16 is executed.

20. A computer-readable storage medium, characterized in that The computer storage medium stores a computer program, and when the computer program is executed by the processor, the video processing method according to any one of claims 1 to 15 is executed; or the computer-readable storage medium stores a video stream formed by the computer program, and when the computer program is executed by the processor, the method for storing or sending a video stream according to claim 16 is executed.

21. A computer program product, characterized in that The computer program product includes a computer program, which, when executed by a processor, implements the video processing method according to any one of claims 1 to 15; or, the computer program product includes a video stream formed by the computer program, which, when executed by a processor, implements the method for storing or sending a video stream according to claim 16.

Images