Video encoding method, device, electronic device and storage medium

Abstract

The present disclosure relates to a video encoding method, device, electronic device and storage medium, the method comprising: when the reference frame corresponding to the current encoding block of the video includes multiple frames, determining the first reference loss corresponding to each reference frame generated by the current encoding block referring to each reference frame respectively; assigning a corresponding propagation weight to each reference frame according to the first reference loss corresponding to each reference frame, the propagation weight is negatively correlated with the first reference loss; encoding the current encoding block based on the propagation weight. According to the technical solution of the present disclosure, dynamically adjusting the propagation weight of the CU‑tree based on sequence features can improve the compression performance of the video, so that each bit rate segment has a better effect.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a video encoding method, device, electronic device, storage medium and computer program product. Background Art

[0002] CU-tree (Coding Unit tree) is an adaptive quantization technology, which aims to adjust the QP (Quantization Parameter) value of the current block by estimating how much information the current block will be referenced by future reference frames in inter-frame prediction (expressed by propagate cost). If the current block contributes more information to the reference frame, the encoding quality of the current block should be improved and the QP value should be reduced; otherwise, the QP value of the current block should be increased.

[0003] In the related art, if the reference frame corresponding to the current frame to which the current block belongs is multiple frames, it is usually necessary to rely on the reference structure between the current frame and the reference frame, and assign corresponding propagation weights to each reference frame according to the distance between the current frame and each reference frame. According to the propagation weight corresponding to each reference frame, the total propagation cost is allocated to each reference frame to obtain the propagation cost of the current block to each reference frame. However, the reference structure is usually relatively fixed, which can easily lead to the propagation weights corresponding to different coding blocks in the same frame being fixed parameters, resulting in the inability to accurately propagate block-level features. Summary of the invention

[0004] The present disclosure provides a video encoding method, device, electronic device, storage medium and computer program product to at least solve the problem in the related art that CU-tree, as a block-level propagation tool, cannot accurately propagate block-level features. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, including:

[0006] In a case where the reference frame corresponding to the current coding block of the video includes multiple frames, determining first reference losses corresponding to each of the reference frames, which are generated by the current coding block referring to each of the reference frames respectively;

[0007] According to the first reference loss corresponding to each of the reference frames, a corresponding propagation weight is assigned to the reference block in each of the reference frames, wherein the propagation weight is negatively correlated with the first reference loss;

[0008] The current coding block is encoded based on the propagation weight.

[0009] In one of the embodiments, allocating corresponding propagation weights to reference blocks in each of the reference frames according to the first reference losses corresponding to each of the reference frames includes:

[0010] Determine a second reference loss generated by the current coding block simultaneously referring to the reference frames of multiple frames;

[0011] According to the first reference loss and the second reference loss corresponding to each of the reference frames, a corresponding propagation weight is assigned to the reference block in each of the reference frames.

[0012] In one of the embodiments, allocating corresponding propagation weights to reference blocks in each of the reference frames according to the first reference loss and the second reference loss corresponding to each of the reference frames includes:

[0013] Determine a loss difference value between a first reference loss and the second reference loss corresponding to each of the reference frames;

[0014] According to the loss difference value corresponding to each of the reference frames, a corresponding propagation weight is allocated to each of the reference frames.

[0015] In one of the embodiments, allocating corresponding propagation weights to reference blocks in each of the reference frames according to the loss difference values ​​corresponding to each of the reference frames includes:

[0016] For any reference frame, obtaining the sum of loss difference values ​​of other reference frames, where the other reference frames are reference frames other than the any reference frame among the multiple reference frames;

[0017] Obtaining a sum of loss difference values ​​of a plurality of reference frames;

[0018] According to the sum of the loss difference values ​​of other reference frames and the sum of the loss difference values ​​of multiple reference frames, a propagation weight corresponding to the reference block in any reference frame is generated.

[0019] In one embodiment, determining the loss difference value between the first reference loss and the second reference loss corresponding to each of the reference frames includes:

[0020] The difference between the first reference loss and the second reference loss corresponding to each of the reference frames is obtained as the loss difference value corresponding to each of the reference frames.

[0021] In one of the embodiments, determining the first reference loss corresponding to each of the reference frames, which is generated by the current coding block referring to each of the reference frames, includes:

[0022] Dividing the current frame to which the current coding block belongs and each of the reference frames to obtain a plurality of blocks corresponding to the current frame and each of the reference frames;

[0023] For any block among the multiple blocks of the current frame, perform motion search in the multiple blocks of each of the reference frames to determine the reference loss between the any block and each block in each of the reference frames;

[0024] The first reference loss is determined according to the reference loss between each block in the current frame and each block in the reference frame.

[0025] In one of the embodiments, when the current frame to which the current coding block belongs is a B frame, the multiple reference frames include a forward reference frame and a backward reference frame of the current frame.

[0026] According to a second aspect of an embodiment of the present disclosure, a video encoding device is provided, including:

[0027] A loss determination module is configured to determine, when a reference frame corresponding to a current coding block of a video includes multiple frames, first reference losses corresponding to each reference frame, generated by the current coding block referring to each reference frame respectively;

[0028] A weight determination module is configured to assign a corresponding propagation weight to a reference block in each of the reference frames according to a first reference loss corresponding to each of the reference frames, wherein the propagation weight is negatively correlated with the first reference loss;

[0029] The encoding module is configured to perform encoding on the current encoding block based on the propagation weight.

[0030] In one embodiment, the weight determination module includes:

[0031] A loss determining unit, configured to determine a second reference loss generated by the current coding block simultaneously referring to the reference frames of multiple frames;

[0032] The weight allocation unit is configured to allocate corresponding propagation weights to the reference blocks in each of the reference frames according to the first reference loss and the second reference loss corresponding to each of the reference frames.

[0033] In one embodiment, the weight allocation unit includes:

[0034] a difference value generating subunit, configured to determine a loss difference value between a first reference loss and the second reference loss corresponding to each of the reference frames;

[0035] The weight allocation subunit is configured to allocate corresponding propagation weights to the reference blocks in each of the reference frames according to the loss difference values ​​corresponding to each of the reference frames.

[0036] In one of the embodiments, the weight allocation subunit is configured to execute, for any reference frame, obtaining the sum of loss difference values ​​of other reference frames, where the other reference frames are reference frames other than the any reference frame among the multiple reference frames; obtaining the sum of loss difference values ​​of the multiple reference frames; and generating a propagation weight corresponding to a reference block in the any reference frame based on the sum of loss difference values ​​of the other reference frames and the sum of loss difference values ​​of the multiple reference frames.

[0037] In one of the embodiments, the difference value generating subunit is configured to obtain the difference between the first reference loss and the second reference loss corresponding to each of the reference frames as the loss difference value corresponding to each of the reference frames.

[0038] In one of the embodiments, the loss determination module is configured to perform division of the current frame to which the current coding block belongs and each of the reference frames to obtain a plurality of blocks related to the current frame and each of the reference frames;

[0039] For any block among the multiple blocks of the current frame, perform motion search in the multiple blocks of each of the reference frames to determine the reference loss between the any block and each block in each of the reference frames;

[0040] The first reference loss is determined according to the reference loss between each block in the current frame and each block in the reference frame.

[0041] In one of the embodiments, when the current frame to which the current coding block belongs is a B frame, the multiple reference frames include a forward reference frame and a backward reference frame of the current frame.

[0042] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0043] processor;

[0044] a memory for storing instructions executable by the processor;

[0045] The processor is configured to execute the instructions to implement the video encoding method described in any one of the first aspects above.

[0046] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video encoding method described in any one of the first aspects above.

[0047] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, wherein the computer program product includes instructions, and when the instructions are executed by a processor of an electronic device, the electronic device is able to execute the video encoding method described in any one of the first aspects above.

[0048] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0049] In the case where the reference frames corresponding to the current coding block of the video include multiple frames, first reference losses corresponding to the respective reference frames generated by the current coding block with reference to the respective reference frames are determined, and according to the first reference losses corresponding to the respective reference frames, the propagation weight of the CU-tree is dynamically adjusted based on sequence characteristics. Compared with the related art that uses the same propagation weight for all frame coding blocks based on the frame-level distance, this can improve the compression performance of the video and make each bit rate segment have a better effect (for example, BD-rate benefit, an objective indicator for measuring encoding performance).

[0050] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0052] Figure 1 The present invention is a flowchart of a video encoding method according to an exemplary embodiment.

[0053] Figure 2 The figure is a flowchart of another video encoding method according to an exemplary embodiment.

[0054] Figure 3 The invention is a block diagram of a video encoding device according to an exemplary embodiment.

[0055] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment.

[0056] Figure 5 is a block diagram of another electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0057] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0059] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0060] MB-Tree (MacroBlock-Tree) algorithm is a new technology introduced in x264 (a video coding standard). The main idea of ​​the algorithm is to consider the impact of various coding decisions on the coding quality of future image frames when encoding the current image frame, which can effectively improve the subjective and objective quality of the video. This algorithm is also inherited in x265 and renamed CU-tree.

[0061] CU-tree is a block-level QP (Quantization Parameter) control method. For each CU block (coding block), a certain number of image frames will be predicted forward, and the corresponding incremental QP will be used to quantize the coding block according to the reference of the coding block in the image frame. If the current coding block contributes more information to the subsequent image frames, it means that its importance is higher, and the coding quality of the coding block should be improved and the quantization parameter value should be reduced; otherwise, the quantization parameter value of the coding block should be increased.

[0062] The core of the CU-tree algorithm includes the calculation of propagation cost and QP increment. The calculation process is as follows:

[0063] (1) Obtain the intra_cost and inter_cost of the current coding block, and calculate the propagation factor (propagate_fraction) from the current coding block to its reference coding block based on the intra_cost and inter_cost:

[0064] propagate_fraction=1–inter_cost / intra_cost

[0065] Among them, inter_cost represents the SATD (Sum of Absolute Transformed Difference) loss cost of the predicted value and the original value estimated using the inter-frame mode; intra_cost represents the SATD loss cost of the predicted value and the original value estimated using the intra-frame mode.

[0066] (2) Calculate the total amount of propagation (propagate_cost) from the current coding block to its reference coding block (i.e., reference block) out ):

[0067] propagate_cost out =propagate_fraction*(intra_cost+propagate_cost in )

[0068] Among them, propagate_cost in Represents the total amount of propagation that is referenced and passed in for the current coding block.

[0069] (3) When the number of reference frames to which the reference block belongs is multiple, since the reference blocks are not located on the same coding unit, the total amount of propagation can be allocated to each reference block according to the area of ​​the reference region (overlap_area / CU_area) and the distance between the current frame to which the current coding block belongs and each reference frame, thereby obtaining the propagation cost of each reference block.

[0070] Take the current image frame as a B frame as an example. The B frame is a bidirectional prediction coding frame. The previous I frame (intra-frame coding frame) or P frame (forward prediction coding frame) and the following P frame are used as reference frames. The propagation cost corresponding to the reference blocks in the two reference frames (forward reference frame and backward reference frame) can be obtained as follows:

[0071] propagate_cost ref1 =(d2 / (d1+d2))*propagate_cost out *(overlap_area / CU_area)

[0072] propagate_cost ref2 =(d1 / (d1+d2))*propagate_cost out *(overlap_area / CU_area)

[0073] Among them, propagate_cost ref1Represents the propagation cost corresponding to the reference block in the forward reference frame; propagate_cost ref2 Represents the propagation cost corresponding to the reference block in the backward reference frame; d1 represents the inter-frame distance from the current frame to the forward reference frame; d2 represents the inter-frame distance from the current frame to the backward reference frame; d2 / (d1+d2) represents the propagation weight of the reference block in the forward reference frame; d1 / (d1+d2) represents the propagation weight of the reference block in the backward reference frame; overlap_area / CU_area represents the area of ​​the reference area.

[0074] (4) Determine the QP increment

[0075] Continuing with the above B frame as an example, after obtaining the propagation costs corresponding to the reference blocks in the forward reference frame and the backward reference frame, the QP increment corresponding to each reference block can be calculated based on the propagation costs:

[0076] DeltaQP_1=-strength*log2(1+propagate_cost ref1 / intra_cost)

[0077] DeltaQP_2=-strength*log2(1+propagate_cost ref2 / intra_cost)

[0078] Among them, DeltaQP_1 represents the QP increment corresponding to the reference block in the forward reference frame; DeltaQP_2 represents the QP increment corresponding to the reference block in the backward reference frame; strength is an adjustable strength value.

[0079] From the formula in step (3) above, we can see that for coding standards such as x264 and x265, CU-tree sets the propagation weight based on the distance between the current frame and its reference frame. The propagation weight depends on a fixed reference structure, which can easily lead to the same propagation weight for different coding blocks in the current frame. Taking the current frame as a B frame as an example, the structure of the bidirectional prediction coding frame is usually relatively fixed, mostly a binary frame structure, that is, the propagation weight is mostly a fixed value of 1 / 2. However, as a block-level propagation tool, CU-tree uses frame-level fixed parameters for all block levels and cannot accurately propagate block-level features.

[0080] In order to solve the above problems, the present disclosure provides a video encoding method, which can be applied to an electronic device, which can be a terminal, a server, or a system composed of a terminal and a server, and is implemented by the interaction between the terminal and the server. In one embodiment, the electronic device is installed with a client capable of publishing videos. The client can be a client of the live broadcast type, e-commerce type, social type, instant messaging type, etc. After the terminal obtains the video shot by the user, the video encoding method provided by the present disclosure can be used to compress and encode the video, and the compressed video can be sent to the server, and the server sends the compressed video to other clients. The other client is any client that plays the video. After receiving the compressed video, the other client can decode the video using a decoding method corresponding to the compression coding to play the video.

[0081] The above terminals may be, but are not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers.

[0082] Figure 1 is a flowchart of a video encoding method according to an exemplary embodiment. Figure 1 As shown, the video encoding method is used in an electronic device, and includes steps S110 to S130.

[0083] In step S110 , when the reference frames corresponding to the current coding block of the video include multiple frames, first reference losses corresponding to the reference frames, which are generated when the current coding block refers to the reference frames respectively, are determined.

[0084] The first reference loss can be characterized by loss indicators such as SATD, SSE (Sum of Squares due to Error), MSE (Mean Squared Error), and SAD (Sum of Absolute Difference) generated by the current coding block referring to each reference frame separately. It can be understood that the first reference loss is the loss in the inter-frame mode.

[0085] Specifically, when the current coding block corresponds to multiple reference coding blocks, and the reference frames to which the multiple reference blocks belong are multiple frames, after generating the total amount of propagation between the current coding block and the multiple reference blocks, the electronic device calculates the first reference loss corresponding to each reference frame generated by the current coding block referring to the reference blocks in each reference frame alone. The specific calculation method of the total amount of propagation can refer to the above content and will not be elaborated here.

[0086] The following is a description of a method for obtaining the first reference loss:

[0087] Downsample the current coding block and the current frame and each reference frame to which it belongs, and divide the current frame and each reference frame into blocks of a preset size. The preset size can be 8x8. For a certain reference frame, use the current block in the current frame to perform motion search in the divided reference frame, determine the reference loss between the current block and each block in the reference frame, and find the block with the smallest loss from the reference loss as the best matching block of the current block. After searching all blocks in the current frame, the sum of the reference losses between all blocks and their respective best matching blocks is used as the first reference loss.

[0088] In step S120, corresponding propagation weights are allocated to reference blocks in each reference frame according to the first reference losses corresponding to each reference frame.

[0089] Among them, the propagation weight is negatively correlated with the first reference loss, that is, the larger the first reference loss corresponding to the reference frame, the smaller the importance of the reference frame is, and a smaller propagation weight should be set for the reference block in the reference frame to reduce the QP quantization loss of the reference block.

[0090] In one possible embodiment, a mapping relationship between the propagation weight and the first reference loss can be preset, and the propagation weight obtained based on the mapping relationship is negatively correlated with the first reference loss. After obtaining the first reference loss corresponding to each reference frame, the propagation weight corresponding to the first reference loss can be obtained based on the mapping relationship.

[0091] In step S130 , encoding is performed for the current coding block based on the propagation weights.

[0092] Specifically, after obtaining the propagation weight corresponding to each reference frame, the electronic device can allocate the total propagation amount to the reference blocks in the reference frame according to the propagation weight of each reference frame, and obtain the propagation cost corresponding to each reference block. Then, the QP increment of each reference block is calculated according to the propagation cost of each reference block. The current coding block is encoded based on the QP increment of each reference block.

[0093] In the encoding method of the above video, when the reference frames corresponding to the current coding block of the video include multiple frames, the first reference losses corresponding to the respective reference frames generated by the current coding block with reference to the respective reference frames are determined, and according to the first reference losses corresponding to the respective reference frames, the propagation weight of the CU-tree is dynamically adjusted based on the sequence characteristics. Compared with the related technology that uses the same propagation weight for all frame coding blocks based on the frame-level distance, this can improve the compression performance of the video and enable each bit rate segment to have better benefits.

[0094] In an exemplary embodiment, step S120, according to the first reference loss corresponding to each reference frame, assigns a corresponding propagation weight to the reference block in each reference frame, including: determining the second reference loss generated by the current coding block simultaneously referring to multiple reference frames; according to the first reference loss and the second reference loss corresponding to each reference frame, assigning a corresponding propagation weight to the reference block in each reference frame.

[0095] Among them, the second reference loss can be characterized by indicators such as SATD, SSE, MSE, SAD, etc. generated by the current coding block simultaneously referring to multiple reference frames.

[0096] Specifically, if there is one reference block corresponding to the current coding block, the current coding block can be marked as "there are no multiple reference blocks", and the step of calculating the second reference loss is not performed. If there are multiple reference blocks corresponding to the current coding block, and the multiple reference blocks belong to multi-frame reference frames, and different reference blocks belong to different reference frames, then after generating the total amount of propagation between the current coding block and the multiple reference blocks, the electronic device calculates the second reference loss generated by the current coding block simultaneously referring to the multi-frame reference frames. Therefore, it can be understood that the second reference loss is less than or equal to the first reference loss corresponding to any reference frame. Based on the first reference loss and the second reference loss corresponding to each reference frame, the electronic device can calculate a propagation weight that has a negative correlation with the first reference loss corresponding to each reference frame.

[0097] In one possible embodiment, for any reference frame, the electronic device may determine the propagation weight of the reference blocks in each reference frame based on the proportion of the first reference loss of other reference frames in the second reference loss, wherein the other reference frames are all other reference frames in the multiple reference frames except the any reference frame.

[0098] In this embodiment, by calculating the propagation weight of each reference frame based on the second reference loss generated by the current coding block simultaneously referring to multiple reference frames, and the first reference loss of each reference frame, the loss caused by the actual reference of each reference frame can be reasonably utilized, and the propagation weight of the reference block in each reference frame can be dynamically adjusted, which is conducive to obtaining a propagation weight with higher accuracy.

[0099] In an exemplary embodiment, after obtaining the first reference loss and the second reference loss corresponding to each reference frame, the electronic device may calculate the loss difference value between the first reference loss and the second reference loss corresponding to each reference frame, and then calculate the propagation weight of the reference block in each reference frame according to the loss difference value corresponding to each reference frame. The loss difference value may be represented by a difference value, a ratio value, etc.

[0100] In one possible embodiment, for any reference frame, the electronic device can determine the propagation weight of the reference blocks in each reference frame based on the proportion of the total loss difference values ​​of other reference frames in the total loss difference values ​​of all reference frames. The total loss difference value can be calculated by addition, multiplication, or other calculation methods.

[0101] Take the loss difference value as difference and the total loss difference value as addition as an example. If the reference frame includes n (n is a positive integer greater than 1) frames, for any reference frame, the electronic device obtains the sum of the loss difference values ​​of other reference frames; obtains the sum of the loss difference values ​​of multiple reference frames; generates the propagation weight corresponding to the reference block in any reference frame based on the sum of the loss difference values ​​of other reference frames and the sum of the loss difference values ​​of multiple reference frames. The propagation weight w of the reference block in the i-th reference frame can be expressed by the following formula: i :

[0102]

[0103] Among them, ref_cost i represents the first reference loss corresponding to the i-th reference frame; ref_cost represents the second reference loss.

[0104] In this embodiment, by calculating the propagation weights of the reference blocks in each reference frame based on the second reference loss generated by the current coding block simultaneously referring to multiple reference frames, and the first reference loss of each reference frame, the loss caused by the actual reference of each reference frame can be reasonably utilized, and the propagation weights of each reference frame can be dynamically adjusted, which is conducive to obtaining propagation weights with higher accuracy.

[0105] In an exemplary embodiment, if Figure 2 As shown, a video encoding method is provided, in which the current frame to which the current encoding block belongs is described as a B frame. When the current frame is a B frame, the multi-frame reference frame includes a forward reference frame (assuming it is a P0 frame) and a backward reference frame (assuming it is a P1 frame, where P1 is greater than P0) of the current frame. This can be specifically implemented by the following steps:

[0106] In step S202, intra_cost and inter_cost of the current coding block are obtained, and a propagation factor from the current coding block to its reference coding block is calculated according to the intra_cost and inter_cost.

[0107] In step S204, the total amount of propagation from the current coding block to its forward reference frame and backward reference frame is calculated.

[0108] In step S206, determine the first reference loss ref_cost1=D(p0,b,b) corresponding to the forward reference frame generated by the current coding block with reference to the forward reference frame, the first reference loss ref_cost2=D(b,b,p1) corresponding to the backward reference frame generated by the current coding block with reference to the backward reference frame, and the second reference loss ref_cost=D(p0,b,p1) generated by the current coding block with reference to both the forward reference frame and the backward reference frame.

[0109] The following describes a method for determining the first reference loss of the P0 frame:

[0110] (1) Downsample the P0 frame and the B frame by 1 / 2, and divide the downsampled P0 frame and the B frame into 8×8 blocks respectively.

[0111] (2) Using the value of the current block I in the B frame as the search value, perform motion search in the P0 frame and find the block with the smallest SATD loss as the best matching block I'.

[0112] (3) When each block in the B frame is searched, the SATD loss of each block in the current frame is summed to obtain D(p0,b,b).

[0113] The method for determining the first reference loss of the P1 frame may refer to the above process and will not be elaborated in detail here.

[0114] The following is an explanation of how to determine the second reference loss:

[0115] (1) Downsample the P0 frame, B frame, and P1 frame by 1 / 2, and divide the downsampled P0 frame, B frame, and P1 frame into 8×8 blocks respectively.

[0116] (2) Using the value of the current block I in the B frame as the search value, perform motion search in the P0 frame and find the block with the smallest SATD as the best matching block I'. At this time, the SATD loss of the current block is recorded as SATD1.

[0117] (3) Use the value of 2*I - I’ as the search value, perform motion search in frame P1, find the block with the minimum SATD as the best matching block, and record the loss as SATD2. If SATD1 > SATD2, then use SATD2 as the SATD loss of the current block; otherwise, still use SATD1 as the SATD loss of the current block. Here, 2*I means multiplying each pixel in block I by 2; 2*I - I’ means subtracting the pixels in block I from the pixels in block I’ point by point.

[0118] (4) After each block in the B frame is searched, sum up the SATD losses of each block in the current frame to obtain D(p0, b, p1).

[0119] In step S208, according to the first reference loss ref_cost1 corresponding to the forward reference frame, the first reference loss ref_cost2 corresponding to the backward reference frame, and the second reference loss ref_cost, calculate the propagation weights of the reference blocks in the forward reference frame and the backward reference frame respectively:

[0120] w1 = (ref_cost2 - ref_cost) / (ref_cost1 + ref_cost2 – 2ref_cost)

[0121] w2 = (ref_cost1 - ref_cost) / (ref_cost1 + ref_cost2 – 2ref_cost)

[0122] Assume that ref_cost1 - ref_cost < ref_cost2 - ref_cost, which indicates that the reference block in the forward reference frame is more important to be referenced, and the propagation weight should be set larger to reduce the QP quantization loss of the forward reference frame, so that the current coding block can make the most use of its similarity for reference; on the contrary, it indicates that the reference block in the forward reference frame is less important to be referenced, and the propagation weight needs to be reduced.

[0123] In step S210, according to the propagation weights of the reference blocks in the forward reference frame and the backward reference frame, calculate the respective propagation costs of the reference blocks in the forward reference frame and the backward reference frame.

[0124] propagate_cost ref1 = w1 * propagate_cost out *(overlap_area / CU_area)

[0125] propagate_cost ref2 = w2 * propagate_cost out *(overlap_area / CU_area)

[0126] Among them, propagate_cost ref1 Represents the propagation cost corresponding to the forward reference frame; propagate_cost ref2 Represents the propagation cost corresponding to the backward reference frame.

[0127] In step S212, the QP increments corresponding to the reference blocks in the forward reference frame and the backward reference frame are determined.

[0128] DeltaQP_1=-strength*log2(1+propagate_cost ref1 / intra_cost)

[0129] DeltaQP_2=-strength*log2(1+propagate_cost ref2 / intra_cost)

[0130] Among them, DeltaQP_1 represents the QP increment corresponding to the reference block in the first reference frame; DeltaQP_2 represents the QP increment corresponding to the reference block in the second reference frame; strength is an adjustable strength value.

[0131] In step S214, encoding is performed for the current coding block based on the QP increments corresponding to the reference blocks in the forward reference frame and the backward reference frame.

[0132] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0133] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0134] Figure 3 FIG. 3 is a block diagram of a video encoding device 300 according to an exemplary embodiment. Figure 3, the device includes a loss determination module 302, a weight determination module 304 and an encoding module 306.

[0135] The loss determination module 302 is configured to determine the first reference losses corresponding to the respective reference frames, which are generated by the current coding block referring to the respective reference frames when the reference frames corresponding to the current coding block of the video include multiple frames; the weight determination module 304 is configured to assign corresponding propagation weights to the reference blocks in the respective reference frames according to the first reference losses corresponding to the respective reference frames, and the propagation weights are negatively correlated with the first reference losses; the encoding module 306 is configured to encode the current coding block based on the propagation weights.

[0136] In an exemplary embodiment, the weight determination module 304 includes: a loss determination unit, configured to determine the second reference loss generated by the current coding block simultaneously referring to multiple reference frames; a weight allocation unit, configured to allocate corresponding propagation weights to the reference blocks in each reference frame according to the first reference loss and the second reference loss corresponding to each reference frame.

[0137] In an exemplary embodiment, the weight allocation unit includes: a difference value generation subunit, configured to determine the loss difference value between the first reference loss and the second reference loss corresponding to each reference frame; a weight allocation subunit, configured to allocate corresponding propagation weights to the reference blocks in each reference frame according to the loss difference value corresponding to each reference frame.

[0138] In an exemplary embodiment, the weight allocation subunit is configured to execute, for any reference frame, obtaining the sum of loss difference values ​​of other reference frames, where the other reference frames are reference frames other than any reference frame in multiple reference frames; obtaining the sum of loss difference values ​​of multiple reference frames; and generating a propagation weight corresponding to a reference block in any reference frame based on the sum of loss difference values ​​of other reference frames and the sum of loss difference values ​​of multiple reference frames.

[0139] In an exemplary embodiment, the difference value generating subunit is configured to obtain the difference between the first reference loss and the second reference loss corresponding to each reference frame as the loss difference value corresponding to each reference frame.

[0140] In an exemplary embodiment, the loss determination module 302 is configured to perform division of the current frame to which the current coding block belongs and each reference frame to obtain multiple blocks related to the current frame and each reference frame; for any block among the multiple blocks of the current frame, perform motion search in the multiple blocks of each reference frame to determine the reference loss between any block and each block in each reference frame; and determine the first reference loss based on the reference loss between each block in the current frame and each block in each reference frame.

[0141] In an exemplary embodiment, when the current frame to which the current coding block belongs is a B frame, the multi-frame reference frame includes a forward reference frame and a backward reference frame of the current frame.

[0142] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0143] Figure 4 1 is a block diagram of an electronic device Z00 for encoding a video according to an exemplary embodiment. For example, the electronic device Z00 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0144] Reference Figure 4 The electronic device Z00 may include one or more of the following components: a processing component Z02, a memory Z04, a power component Z06, a multimedia component Z08, an audio component Z10, an input / output (I / O) interface Z12, a sensor component Z14, and a communication component Z16.

[0145] The processing component Z02 generally controls the overall operation of the electronic device Z00, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component Z02 may include one or more processors Z20 to execute instructions to complete all or part of the steps of the above-described method. In addition, the processing component Z02 may include one or more modules to facilitate interaction between the processing component Z02 and other components. For example, the processing component Z02 may include a multimedia module to facilitate interaction between the multimedia component Z08 and the processing component Z02.

[0146] The memory Z04 is configured to store various types of data to support operations on the electronic device Z00. Examples of such data include instructions for any application or method operating on the electronic device Z00, contact data, phone book data, messages, pictures, videos, etc. The memory Z04 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, optical disk, or graphene memory.

[0147] The power supply assembly Z06 provides power to various components of the electronic device Z00. The power supply assembly Z06 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device Z00.

[0148] The multimedia component Z08 includes a screen that provides an output interface between the electronic device Z00 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component Z08 includes a front camera and / or a rear camera. When the electronic device Z00 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0149] The audio component Z10 is configured to output and / or input audio signals. For example, the audio component Z10 includes a microphone (MIC), and when the electronic device Z00 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory Z04 or sent via the communication component Z16. In some embodiments, the audio component Z10 also includes a speaker for outputting an audio signal.

[0150] I / O interface Z12 provides an interface between processing component Z02 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0151] The sensor assembly Z14 includes one or more sensors for providing various aspects of status assessment for the electronic device Z00. For example, the sensor assembly Z14 can detect the open / closed state of the electronic device Z00, the relative positioning of the components, such as the display and keypad of the electronic device Z00, and the sensor assembly Z14 can also detect the position change of the electronic device Z00 or the components of the electronic device Z00, the presence or absence of contact between the user and the electronic device Z00, the orientation or acceleration / deceleration of the device Z00 and the temperature change of the electronic device Z00. The sensor assembly Z14 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly Z14 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly Z14 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0152] The communication component Z16 is configured to facilitate wired or wireless communication between the electronic device Z00 and other devices. The electronic device Z00 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component Z16 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component Z16 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0153] In an exemplary embodiment, the electronic device Z00 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0154] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory Z04 including instructions, and the above instructions can be executed by a processor Z20 of the electronic device Z00 to complete the above method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0155] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions. The instructions can be executed by the processor Z20 of the electronic device Z00 to implement the above method.

[0156] Figure 5 FIG. 1 is a block diagram of another electronic device S00 for encoding a video according to an exemplary embodiment. For example, the electronic device S00 may be a server. Figure 5 The electronic device S00 includes a processing component S20, which further includes one or more processors, and a memory resource represented by a memory S22 for storing instructions executable by the processing component S20, such as an application. The application stored in the memory S22 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component S20 is configured to execute the instructions to perform the above method.

[0157] The electronic device S00 may further include: a power supply component S24 configured to perform power management of the electronic device S00, a wired or wireless network interface S26 configured to connect the electronic device S00 to a network, and an input / output (I / O) interface S28. The electronic device S00 may operate based on an operating system stored in the memory S22, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD or the like.

[0158] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory S22 including instructions, and the above instructions can be executed by a processor of the electronic device S00 to complete the above method. The storage medium can be a computer-readable storage medium, for example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0159] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions. The instructions can be executed by a processor of the electronic device S00 to complete the above method.

[0160] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0161] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0162] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that: include: In a case where the reference frame corresponding to the current coding block of the video includes multiple frames, determining first reference losses corresponding to each of the reference frames, which are generated by the current coding block referring to each of the reference frames respectively; The first reference loss is a reference loss between a current frame to which the current coding block belongs and each of the reference frames; According to the first reference loss corresponding to each of the reference frames, a corresponding propagation weight is assigned to the reference block in each of the reference frames, wherein the propagation weight is negatively correlated with the first reference loss; encoding the current coding block based on the propagation weight; The encoding of the current coding block based on the propagation weight includes: allocating the total propagation amount to the reference blocks in the reference frame according to the propagation weights of each reference frame to obtain the propagation cost corresponding to each reference block; calculating the QP increment of each reference block according to the propagation cost of each reference block; encoding the current coding block based on the QP increment of each reference block.

2. The video encoding method according to claim 1, characterized in that: The allocating corresponding propagation weights to the reference blocks in each of the reference frames according to the first reference losses corresponding to each of the reference frames comprises: Determine a second reference loss generated by the current coding block simultaneously referring to the reference frames of multiple frames; According to the first reference loss and the second reference loss corresponding to each of the reference frames, a corresponding propagation weight is assigned to the reference block in each of the reference frames.

3. The video encoding method according to claim 2, characterized in that: The allocating corresponding propagation weights to the reference blocks in each of the reference frames according to the first reference loss and the second reference loss corresponding to each of the reference frames includes: Determine a loss difference value between a first reference loss and the second reference loss corresponding to each of the reference frames; According to the loss difference values ​​corresponding to the reference frames, corresponding propagation weights are allocated to the reference blocks in the reference frames.

4. The video encoding method according to claim 3, characterized in that: The allocating corresponding propagation weights to the reference blocks in each of the reference frames according to the loss difference values ​​corresponding to each of the reference frames comprises: For any reference frame, obtain the sum of loss difference values ​​corresponding to other reference frames, where the other reference frames are reference frames other than the any reference frame among the multiple reference frames; Obtaining a sum of loss difference values ​​of a plurality of reference frames; The propagation weight corresponding to the reference block in any reference frame is generated according to the sum of the loss difference values ​​of other reference frames and the sum of the loss difference values ​​of multiple reference frames.

5. The video encoding method according to claim 3, characterized in that: The determining a loss difference value between a first reference loss and the second reference loss corresponding to each of the reference frames includes: The difference between the first reference loss and the second reference loss corresponding to each of the reference frames is obtained as the loss difference value corresponding to each of the reference frames.

6. The video encoding method according to any one of claims 1 to 5, characterized in that: The determining the first reference loss corresponding to each of the reference frames, which is generated by the current coding block referring to each of the reference frames respectively, comprises: Dividing the current frame to which the current coding block belongs and each of the reference frames to obtain a plurality of blocks corresponding to the current frame and each of the reference frames; For any block among the multiple blocks of the current frame, perform motion search in the multiple blocks of each of the reference frames to determine the reference loss between the any block and each block in each of the reference frames; The first reference loss is determined according to the reference loss between each block in the current frame and each block in the reference frame.

7. The video encoding method according to any one of claims 1 to 5, characterized in that: In the case that the current frame to which the current coding block belongs is a B frame, the multiple reference frames include a forward reference frame and a backward reference frame of the current frame.

8. A video encoding device, characterized in that: include: A loss determination module is configured to determine, when a reference frame corresponding to a current coding block of a video includes multiple frames, first reference losses corresponding to each reference frame, generated by the current coding block referring to each reference frame respectively; The first reference loss is a reference loss between a current frame to which the current coding block belongs and each of the reference frames; A weight determination module is configured to assign a corresponding propagation weight to a reference block in each of the reference frames according to a first reference loss corresponding to each of the reference frames, wherein the propagation weight is negatively correlated with the first reference loss; An encoding module, configured to perform encoding of the current encoding block based on the propagation weight; The encoding of the current coding block based on the propagation weight includes: allocating the total propagation amount to the reference blocks in the reference frame according to the propagation weights of each reference frame to obtain the propagation cost corresponding to each reference block; calculating the QP increment of each reference block according to the propagation cost of each reference block; encoding the current coding block based on the QP increment of each reference block.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video encoding method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the video encoding method as described in any one of claims 1 to 7.

Images