Video coding method and device, electronic equipment, computer readable storage medium, computer program product and method for generating bit stream

Abstract

The invention relates to a video coding method and device, electronic equipment, a computer readable storage medium, a computer program product and a method for generating a bit stream. The video coding method comprises the steps that distortion information of each block in a current frame and a first subsequent frame is determined, the first subsequent frame is at least one actually available subsequent frame after the current frame, and the distortion information of the block is used for indicating the influence of distortion of a reference block of a previous frame on distortion of the block; based on the distortion information of each block, the distortion information of each block in a second subsequent frame is predicted, and the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; determining a quantization parameter increment of the current block on the basis of distortion information of the current block in the current frame and distortion information of predetermined blocks in the first subsequent frame and the second subsequent frame, the distortion of the predetermined blocks being affected by the distortion of the current block; and encoding the current block based on the determined quantization parameter increment.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing, and in particular to a video encoding method and device, an electronic device, a computer-readable storage medium, a computer program product, and a method for generating a bit stream. Background Art

[0002] Bitrate control is an important part of the video encoding process. It involves allocating bitrates to different frames of the video to be encoded, thereby ensuring the quality and compression efficiency of the video encoding. Bitrate control is generally achieved by adjusting the quantization parameter (QP). QP reflects the compression of spatial details. The QP value determines the quality of the encoded video to a certain extent. Generally, the smaller the QP value, the more details will be retained and the video clarity will be higher. The larger the QP value, the more details will be discarded and the video clarity will be lower.

[0003] Currently, in the open source encoders x264 and x265, the pre-analysis stage encoding tool cuTree is used to allocate bitrate for each frame, thereby improving compression efficiency. However, in low-latency scenarios (such as live broadcast scenarios), cuTree cannot obtain a good bitrate allocation result, which limits the compression performance. Summary of the invention

[0004] The present disclosure provides a video encoding method and apparatus, an electronic device, a computer-readable storage medium, a computer program product, and a method for generating a bit stream, so as to at least solve the problem that the related technology cannot obtain a good bit rate allocation result, resulting in limited compression performance.

[0005] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, comprising: determining distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame that is actually obtainable after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of a reference block of a previous frame on the distortion of the block; predicting the distortion information of each block in a second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; determining a quantization parameter increment of the current block based on the distortion information of the current block in the current frame, the distortion information of a predetermined block in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is affected by the distortion of the current block; encoding the current block based on the determined quantization parameter increment.

[0006] Optionally, determining the distortion information of each block in the current frame and the first subsequent frame includes: determining the distortion information of each block in the current frame and the first subsequent frame based on a distortion influence linear model and distortion information of a predetermined reference block, wherein the predetermined reference block is a block on the zeroth frame in the video to be encoded;

[0007] Predicting the distortion information of each block in the second subsequent frame based on the distortion information of each block includes: predicting the distortion information of each block in the second subsequent frame based on the influence factor of the reference block of the last frame in the first subsequent frame, and the distortion information of each block in the current frame and the first subsequent frame.

[0008] Optionally, based on the distortion information of the current block in the current frame and the distortion information of the predetermined block in the first subsequent frame and the second subsequent frame, determining the quantization parameter increment of the current block includes: obtaining a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is the rate-distortion cost without considering the influence of inter-frame propagation; obtaining a second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block, wherein the second rate-distortion cost is the rate-distortion cost considering the influence of inter-frame propagation; obtaining a quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; and adjusting an initial quantization parameter of the current block based on the quantization parameter increment to obtain the quantization parameter of the current block.

[0009] Optionally, based on the distortion information of the current block and the predetermined block, obtaining the second rate-distortion cost of the current block includes: adding the distortion information of the current block and the predetermined block to obtain a total distortion impact; and determining the second rate-distortion cost based on the total distortion impact and the bit rate.

[0010] Optionally, based on the first rate-distortion cost and the second rate-distortion cost, a quantization parameter increment of the current block is obtained, including: respectively derivatizing the first rate-distortion cost and the second rate-distortion cost with respect to the bit rate to obtain a first derivation result and a second derivation result; based on the second rate-distortion cost, obtaining a propagation influence factor of the current block; based on the first derivation result, the second derivation result and the propagation influence factor, obtaining a quantization parameter increment of the current block.

[0011] Optionally, based on the second rate-distortion cost, a propagation influence factor of the current block is obtained, including: based on the second rate-distortion cost, obtaining a propagation influence factor function, wherein the influence factors of the current block and the predetermined block are determined respectively by the intra-frame cost and the inter-frame cost of the current block and the predetermined block; based on the determined influence factor and the second rate-distortion cost, obtaining the propagation influence factor of the current block.

[0012] Optionally, based on the first derivative result, the second derivative result and the propagation influence factor, a quantization parameter increment of the current block is obtained, including: determining a first quantization parameter based on a first rate coefficient corresponding to a first rate-distortion cost; obtaining a second rate coefficient corresponding to a second rate-distortion cost based on the first derivative result, the second derivative result and the first rate coefficient; determining a second quantization parameter based on the second rate coefficient; and obtaining a quantization parameter increment based on the first quantization parameter, the second quantization parameter and the propagation influence factor.

[0013] Optionally, the impact factors of the current block and the predetermined block are determined according to the intra-frame cost and inter-frame cost of the current block and the predetermined block, including: for each block in the current block and the predetermined block, obtaining the difference between the intra-frame cost and the inter-frame cost of the block, and comparing the difference with the intra-frame cost of the block to obtain the impact factor of the block.

[0014] According to a second aspect of an embodiment of the present disclosure, a video encoding device is provided, comprising: a distortion information determination unit, configured to determine distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame that is actually obtainable after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of a reference block of a previous frame on the distortion of the block; a prediction unit, configured to predict the distortion information of each block in a second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; a quantization parameter increment determination unit, configured to determine a quantization parameter increment of the current block based on the distortion information of the current block in the current frame, the distortion information of a predetermined block in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is affected by the distortion of the current block; and an encoding unit, configured to encode the current block based on the determined quantization parameter increment.

[0015] Optionally, the distortion information determination unit is further configured to determine the distortion information of each block in the current frame and the first subsequent frame based on the distortion influence linear model and the distortion information of a predetermined reference block, wherein the predetermined reference block is a block on the zeroth frame in the video to be encoded;

[0016] The prediction unit is further configured to predict the distortion information of each block in the second subsequent frame based on the influence factor of the reference block of the last frame in the first subsequent frame and the distortion information of each block in the current frame and the first subsequent frame.

[0017] Optionally, the quantization parameter increment determination unit is further configured to obtain a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is the rate-distortion cost without considering the influence of inter-frame propagation; obtain a second rate-distortion cost of the current block based on the distortion information of the current block and a predetermined block, wherein the second rate-distortion cost is the rate-distortion cost considering the influence of inter-frame propagation; obtain a quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; and adjust an initial quantization parameter of the current block based on the quantization parameter increment to obtain the quantization parameter of the current block.

[0018] Optionally, the quantization parameter increment determination unit is further configured to add the distortion information of the current block and the predetermined block to obtain a total distortion impact; and determine a second rate-distortion cost based on the total distortion impact and the bit rate.

[0019] Optionally, the quantization parameter increment determination unit is also configured to respectively derive the first rate-distortion cost and the second rate-distortion cost with respect to the bit rate to obtain a first derivative result and a second derivative result; obtain a propagation influence factor of the current block based on the second rate-distortion cost; and obtain a quantization parameter increment of the current block based on the first derivative result, the second derivative result and the propagation influence factor.

[0020] Optionally, the quantization parameter increment determination unit is further configured to determine the influence factors of the current block and the predetermined block respectively through the intra-frame cost and inter-frame cost of the current block and the predetermined block; and obtain the propagation influence factor of the current block based on the determined influence factor and the second rate-distortion cost.

[0021] Optionally, the quantization parameter increment determination unit is also configured to determine the first quantization parameter based on the first rate coefficient corresponding to the first rate-distortion cost; obtain the second rate coefficient corresponding to the second rate-distortion cost based on the first derivative result, the second derivative result and the first rate coefficient; determine the second quantization parameter based on the second rate coefficient; and obtain the quantization parameter increment based on the first quantization parameter, the second quantization parameter and the propagation influence factor.

[0022] Optionally, the quantization parameter increment determination unit is further configured to obtain the difference between the intra-frame cost and the inter-frame cost of the block for each block in the current block and the predetermined block, and compare the difference with the intra-frame cost of the block to obtain the impact factor of the block.

[0023] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the video encoding method according to the present disclosure.

[0024] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the video encoding method according to the present disclosure as described above.

[0025] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, having instructions for storing a bit stream, wherein the bit stream includes: video data generated according to the video encoding method as described above.

[0026] According to a sixth aspect of an embodiment of the present disclosure, a method for generating a bit stream is provided, comprising: generating a bit stream according to the video encoding method as described above.

[0027] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0028] According to the video encoding method and device, electronic device, computer-readable storage medium, computer program product and method for generating bitstream disclosed in the present invention, the distortion information of each block in the actually obtainable frame is used to predict the distortion information of each block in the subsequent frame that cannot be actually obtained, so that the subsequent frame of the current frame not only includes the actually obtainable frame, but also includes the predicted frame, increasing the number of subsequent frames of the current frame, and then when determining the quantization parameter of the current block, not only the current frame and the frames actually transmitted thereafter are considered, but also the predicted subsequent frames are considered, and the correction of the quantization parameter is realized. Then, the current block is encoded based on the corrected quantization parameter, and a better code rate allocation result can be obtained, thereby effectively reducing the output bit rate of the encoder while ensuring the video quality, thereby achieving higher compression efficiency and saving bandwidth resources. Therefore, the present invention solves the problem that the related technology cannot obtain a better code rate allocation result, resulting in limited compression performance.

[0029] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0031] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure;

[0032] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure;

[0033] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure;

[0034] Figure 4 is a schematic diagram of the existing distortion propagation effects;

[0035] Figure 5 is a schematic diagram showing the influence of distortion propagation according to an exemplary embodiment of the present disclosure;

[0036] Figure 6 is a flowchart of a video encoding method according to an exemplary embodiment of the present disclosure;

[0037] Figure 7 is a block diagram of a video encoding device according to an exemplary embodiment of the present disclosure;

[0038] Figure 8 is a diagram illustrating a computing environment coupled with a user interface according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0039] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0041] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0042] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0043] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0044] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0045] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0046] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0047] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0048] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0049] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0050] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0051] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0052] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0053] like Figure 2 As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0054] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0055] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0056] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0057] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0058] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0059] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0060] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0061] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0062] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0063] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0064] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0065] In some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0066] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0067] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0068] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0069] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0070] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0071] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0072] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0073] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0074] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0075] Bitrate control is an important part of the video encoding process, which involves allocating bitrates to different image frames of the video to be encoded, so as to ensure the quality and compression efficiency of the video encoding. Different bitrate allocation schemes will have different effects on the quality and compression efficiency of the encoded output video. If an excellent bitrate allocation scheme is adopted, the output bitrate of the encoder can be effectively reduced while ensuring the video quality, thereby achieving higher compression efficiency and saving bandwidth resources. Currently, in the open source encoders x264 and x265, the pre-analysis stage encoding tool cuTree is used to allocate bitrates for each frame. The pre-analysis stage usually requires obtaining subsequent image frames in advance and analyzing the impact of quality changes in the current image frame (for example, encoding distortion) on the quality of subsequent image frames, thereby determining the bitrate allocated to the current image frame. For example, Figure 4 is a schematic diagram of the existing distortion propagation effects. Figure 4 As shown, the current image frame f n Need to be combined with subsequent image frames f n+1 ~f n+4 , analyze f n The effect of the coding distortion of the subsequent image frames on the coding distortion of the subsequent image frames determines the value assigned to f nHere, the bit rate control can be achieved by adjusting the quantization parameter, that is, by controlling f n In order for cuTree to obtain better bitrate allocation results, it is usually necessary to obtain a large number of subsequent frames in advance to determine the quantization parameters. However, in low-latency scenarios (such as live broadcast scenarios), the number of subsequent frames of the current frame is limited, which limits cuTree's bitrate allocation and restricts compression performance.

[0076] In order to solve the above problems, the present disclosure introduces a prediction propagation process to predict the subsequent propagation image frames based on the actual propagation image frames, so as to make up for the propagation limitation problem caused by the inability to obtain enough subsequent image frames in low-latency scenarios. The quantization parameters of the current image frame are corrected through the predicted subsequent image frames to allocate a suitable bit rate, thereby improving the encoder compression rate. For example, Figure 5 FIG. 1 is a schematic diagram showing a code rate allocation according to an exemplary embodiment of the present disclosure. Figure 5 As shown, it is necessary not only to analyze f n The coding distortion of the subsequent image frame f is actually transmitted n+1 ~f n+4 The influence of coding distortion, it is also necessary to predict f n The coding distortion of the subsequent frame f′ that has not yet been actually transmitted n+5 ~f′ n+8 The influence of coding distortion jointly determines the allocation to f n The bit rate.

[0077] Hereinafter, a video encoding method and apparatus, an electronic device, a computer-readable storage medium, a computer program product, and a method for generating a bitstream according to exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0078] Figure 6 is a flowchart of a video encoding method according to an exemplary embodiment of the present disclosure, such as Figure 6 As shown, the video encoding method includes the following steps:

[0079] In step S601, distortion information of each block in the current frame and the first subsequent frame is determined, wherein the first subsequent frame is at least one subsequent frame that is actually obtainable after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of the reference block of the previous frame on the distortion of the block.

[0080] As an example, taking the encoding tool cuTree as an example, in cuTree, the distortion of the current block is affected by the distortion of the reference block in the previous frame. Generally, the influence of the reference block on the previous frame on the distortion of the current block can be expressed as a linear model. Therefore, for the blocks on the frames that can actually be obtained, the distortion information of the block can be determined by the linear model. The present disclosure does not limit this.

[0081] According to an exemplary embodiment of the present disclosure, determining the distortion information of each block in the current frame and the first subsequent frame may include: determining the distortion information of each block in the current frame and the first subsequent frame based on the distortion influence linear model and the distortion information of the predetermined reference block, wherein the predetermined reference block is a block on the zeroth frame in the video to be encoded. Through this embodiment, the distortion information of each block in the current frame and the first subsequent frame can be conveniently and quickly determined.

[0082] As an example, based on the above linear model, that is, the distortion effect linear model, the distortion information of each block can be represented by the distortion of the block on the zeroth frame. The following systematic discussion is how to represent the distortion information of each block in the current frame and the first subsequent frame by the distortion of the block on the zeroth frame.

[0083] Specifically, the above distortion influence linear model can be expressed as follows:

[0084] D m =k m-1 D m-1 +b m-1 ;

[0085] D m-1 =k m-2 D m-2 +b m-2 ; ......;

[0087] D1=k0D0+b0(1)

[0088] Among them, D m represents the distortion information of the current block on the mth frame, k m represents the influence factor of the distortion of the reference block on the m-1th frame on the current block on the mth frame, b m represents a constant term, and D0 represents the distortion information of the block on the zeroth frame.

[0089] Based on the linear model of distortion effect in formula (1), we can deduce the effect of D0 on D m The influence of, that is, how to represent the distortion information of each block in the current frame and the first subsequent frame by the distortion of the block on the zeroth frame:

[0090]

[0091] Among them, b is a constant term.

[0092] return Figure 6 In step S602, based on the distortion information of each block, the distortion information of each block in the second subsequent frame is predicted, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame.

[0093] As an example, after obtaining the distortion information of each block in the current frame and the first subsequent frame, the distortion information of these blocks can be used to predict the distortion information of each block in the second subsequent frame that cannot be actually obtained. That is, the information of the subsequent frames that cannot be actually obtained is added, which is equivalent to increasing the number of subsequent frames of the current frame.

[0094] According to an exemplary embodiment of the present disclosure, based on the distortion information of each block, predicting the distortion information of each block in the second subsequent frame may include: predicting the distortion information of each block in the second subsequent frame based on the influence factor of the reference block of the last frame in the first subsequent frame, the distortion information of each block in the current frame and the first subsequent frame. Through this embodiment, since the influence factor of the last frame in the first subsequent frame that is closest to the second subsequent frame is adopted, combined with the distortion information of each block in the current frame and the first subsequent frame, the distortion information of each block in the second subsequent frame is predicted, so that the distortion information of the block on the second subsequent frame can be relatively accurately predicted.

[0095] As an example, due to the continuity of videos in most scenarios, it is assumed here that there is a trend of continuous transformation from the mth frame to the nth frame, and then the subsequent propagation is predicted based on the assumption. When predicting, the influence factor of the reference block of the last frame in the first subsequent frame is used, that is, the influence factor of the reference block of the frame that can be actually obtained is used to predict the distortion information of the corresponding current block in the corresponding second subsequent frame. At the same time, in the prediction process, the distortion of the block on the zeroth frame is combined with the above-mentioned influence factor to determine the distortion information of the corresponding current block in the corresponding second subsequent frame.

[0096] Assume that the last frame of the first subsequent frame is frame m, and the influence factor of the last frame m is k m-1 At this time, in the embodiment of the present disclosure, the influence factor of each frame in the second subsequent frame can be predicted as k m-1 , which is equivalent to using the influence factor of the last frame closest to the second subsequent frame in the first subsequent frame. Then D0 has no significant effect on the second subsequent frame D that cannot be actually obtained. m+n The impact can be expressed as:

[0097]

[0098] Among them, b′ is a constant term.

[0099] return Figure 7 In step S703, a quantization parameter increment of the current block is determined based on distortion information of the current block in the current frame and distortion information of predetermined blocks in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is affected by the distortion of the current block.

[0100] As an example, when determining the quantization parameter increment of the current block, the number of subsequent frames used can be set, but the subsequent frames used must include the second subsequent frame so that the number of subsequent frames used is sufficient and it is also convenient to use the second subsequent frame to correct the quantization parameter increment.

[0101] According to an exemplary embodiment of the present disclosure, determining the quantization parameter increment of the current block based on the distortion information of the current block in the current frame, the distortion information of the predetermined block in the first subsequent frame and the second subsequent frame may include: obtaining a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is a rate-distortion cost without considering the influence of inter-frame propagation; obtaining a second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block, wherein the second rate-distortion cost is a rate-distortion cost considering the influence of inter-frame propagation; obtaining a quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; adjusting the initial quantization parameter of the current block based on the quantization parameter increment to obtain the quantization parameter of the current block. Through this embodiment, the rate-distortion cost in two cases is used to perfectly introduce the information of the second subsequent frame, so that the quantization parameter increment can be corrected using the information of the second subsequent frame to obtain a relatively suitable quantization parameter, and then allocated to a suitable code rate.

[0102] As an example, after obtaining the corresponding rate-distortion cost based on the distortion information, the first rate-distortion cost and the second rate-distortion cost can be used to differentiate the bit rate respectively. According to the derivative results of the two rate-distortion costs, a quantization parameter function that takes subsequent frames into consideration can be obtained. Then, combined with the initial quantization parameter function, the corresponding quantization parameter increment function can be further obtained. The quantization parameter increment function can be applied in any low-latency scenario. It is only necessary to obtain the corresponding quantization parameter increment according to the quantization parameter increment function according to the corresponding scenario.

[0103] As an example, after obtaining the quantization parameter increment function, the influencing factors in the second rate-distortion cost can be further calculated, and the calculated influencing factors can be brought into the quantization parameter function to obtain the quantization parameter increment, so as to adjust the initial quantization parameter to obtain the final corrected quantization parameter for video encoding, so as to improve the compression performance.

[0104] According to an exemplary embodiment of the present disclosure, based on the distortion information of the current block and the predetermined block, obtaining the second rate-distortion cost of the current block includes: adding the distortion information of the current block and the predetermined block to obtain the total distortion effect; and determining the second rate-distortion cost based on the total distortion effect and the bit rate. Through the method of this embodiment, a rate-distortion cost that conforms to the actual situation can be determined.

[0105] As an example, taking the current frame as frame 0, without considering the influence of inter-frame propagation, the rate-distortion cost of the current block on frame 0 can be expressed as:

[0106] J1=D0+λR0(4)

[0107] Taking into account the influence of inter-frame propagation, the rate-distortion cost of the current block on the 0th frame can be expressed as:

[0108]

[0109] Wherein, R0 is the bit rate of the current block in the 0th frame, and λ is the bit rate coefficient.

[0110] As an example, taking the current frame as the third frame, without considering the influence of inter-frame propagation, the rate-distortion cost of the current block on the third frame can be expressed as:

[0111] J1=D3+λR3(6)

[0112] Taking into account the impact of inter-frame propagation, the rate-distortion cost of the current block on the third frame can be expressed as:

[0113]

[0114] Among them, R3 is the code rate and λ is the code rate coefficient.

[0115] According to an exemplary embodiment of the present disclosure, obtaining a quantization parameter increment of a current block based on a first rate-distortion cost and a second rate-distortion cost may include: respectively deriving the first rate-distortion cost and the second rate-distortion cost with respect to a bit rate to obtain a first derivative result and a second derivative result; obtaining a propagation influence factor of the current block based on the second rate-distortion cost; and obtaining a quantization parameter increment of the current block based on the first derivative result, the second derivative result, and the propagation influence factor.

[0116] Through this embodiment, by differentiating the bit rate with respect to the rate-distortion cost, the quantization parameter increment function containing the second subsequent frame information can be quickly and conveniently obtained. At the same time, the propagation influence factor containing the second subsequent frame information can also be conveniently and quickly obtained through the second rate-distortion cost. Then, based on the two, the quantization parameter increment containing the second subsequent frame information can be obtained, that is, the quantization parameter increment is corrected using the second subsequent frame.

[0117] As an example, still taking the current frame as frame 0, the rate-distortion costs J1 and J2 are respectively derived with respect to the bit rate,

[0118] The derivative result of J1 with respect to code rate R0 is:

[0119]

[0120] The derivative result of J2 with respect to code rate R0 is:

[0121]

[0122] in,

[0123] Transform the derivative result of J2 to obtain:

[0124]

[0125] The above derivatives of the rate-distortion cost J1 and J2 are used to solve the code rate coefficient corresponding to J2:

[0126]

[0127] Then we get the equivalent equation of J2, J3=D0+λ opt R0, that is, the optimization target changes from J2 to J3.

[0128] As an example, according to the derivative results of the above two rate-distortion costs, i.e., formula (8) and formula (9), a quantization parameter function taking into account subsequent frames can be obtained, and then combined with the initial quantization parameter function, the corresponding quantization parameter increment function can be further obtained. After obtaining the quantization parameter increment function, the propagation influence factor can be determined according to the influence factor in the second rate-distortion cost, and then based on the quantization parameter increment function and the propagation influence factor, the quantization parameter increment of the current block can be obtained.

[0129] According to an exemplary embodiment of the present disclosure, obtaining the propagation influence factor of the current block based on the second rate-distortion cost may include: determining the influence factors of the current block and the predetermined block respectively through the intra-frame cost and inter-frame cost of the current block and the predetermined block; and obtaining the propagation influence factor of the current block based on the determined influence factor and the second rate-distortion cost.

[0130] Through this embodiment, the second rate-distortion cost is used in combination with the intra-frame cost and the inter-frame cost of the block to further calculate the propagation impact factor.

[0131] As an example, based on the second rate-distortion cost, a propagation impact factor function may be obtained, wherein the propagation impact factor function includes an impact factor of the distortion of the reference block on the m-1th frame on the current block on the mth frame. For example, according to formula (9): The propagation impact factor function (1+α) can be obtained:

[0132]

[0133] and then,

[0134]

[0135] As an example, taking the case where 5 frames are actually available and 6, 7, and 8 frames are predicted, the derivation process of the propagation impact factor function (12) is as follows:

[0136]

[0137] According to an exemplary embodiment of the present disclosure, obtaining a quantization parameter increment of a current block based on a first derivation result, a second derivation result, and a propagation influence factor may include: determining a first quantization parameter based on a first rate coefficient corresponding to a first rate-distortion cost; obtaining a second rate coefficient corresponding to a second rate-distortion cost based on the first derivation result, the second derivation result, and the first rate coefficient; determining a second quantization parameter based on the second rate coefficient; and obtaining a quantization parameter increment based on the first quantization parameter, the second quantization parameter, and the propagation influence factor. Through this embodiment, combined with the derivation result, the quantization parameter increment can be obtained more conveniently.

[0138] As an example, since the code rate coefficient λ=C*2^(QP / 3), the first quantization parameter (QP) can be represented by λ, that is,

[0139] QP=3log2(λ / C) (14)

[0140] In the above formula (11), opt Substituting into formula (14), we can obtain the second quantization parameter that takes inter-frame propagation into account, namely:

[0141] QPnew=3log2(λ / C(1+α)) (15)

[0142] Furthermore, the quantization parameter increment △QP can be expressed as:

[0143]

[0144] Where s represents a constant. Substituting the value of the propagation impact factor (1+α) into formula (16), the quantization parameter increment can be obtained.

[0145] According to an exemplary embodiment of the present disclosure, determining the respective impact factors of the current block and the predetermined block through the intra-frame cost and inter-frame cost of the current block and the predetermined block may include: for each block in the current block and the predetermined block, obtaining the difference between the intra-frame cost and the inter-frame cost of the block, and comparing the difference with the intra-frame cost of the block to obtain the impact factor of the block.

[0146] As an example, in cuTree, the impact factor k i It can be expressed as a function of the intra-frame cost (intraCost) and the inter-frame cost (interCost).

[0147] ki=(intracost_i-intercost_i) / intracost_i (17)

[0148] According to the above formula (12), the propagation impact factor 1+α is composed of the impact factor ki. The corresponding ki is obtained using the above formula and then substituted into formula (12) to obtain the propagation impact factor 1+α.

[0149] return Figure 6 In step S604, the current block is encoded based on the determined quantization parameter increment.

[0150] In summary, the present invention predicts the propagation of subsequent frames, that is, uses the information of the first subsequent frame that has been acquired to predict the impact of subsequent frames, thereby predicting the information of the second subsequent frame, making up for the problem of limited number of subsequent frames acquired in low-latency scenarios, optimizing the quantization parameter increment in cuTree, greatly improving the compression performance of the encoder, and also improving the encoding performance.

[0151] Figure 7 is a block diagram of a video encoding device according to an exemplary embodiment of the present disclosure. Figure 7 The device includes a distortion information determination unit 70, a prediction unit 72, a quantization parameter increment determination unit 74 and an encoding unit 76.

[0152] The distortion information determination unit 70 is configured to determine the distortion information of each block in the current frame and the first subsequent frame, wherein the first subsequent frame is at least one subsequent frame that is actually obtainable after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of the reference block of the previous frame on the distortion of the block; the prediction unit 72 is configured to predict the distortion information of each block in the second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; the quantization parameter increment determination unit 74 is configured to determine the quantization parameter increment of the current block based on the distortion information of the current block in the current frame and the distortion information of the predetermined block in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is affected by the distortion of the current block; the encoding unit 76 is configured to encode the current block based on the determined quantization parameter increment.

[0153] According to an exemplary embodiment of the present disclosure, the distortion information determination unit 70 is further configured to determine the distortion information of each block in the current frame and the first subsequent frame based on the distortion influence linear model and the distortion information of the predetermined reference block, wherein the predetermined reference block is a block on the zeroth frame in the video to be encoded;

[0154] The prediction unit 72 is further configured to predict the distortion information of each block in the second subsequent frame based on the influence factor of the reference block of the last frame in the first subsequent frame and the distortion information of each block in the current frame and the first subsequent frame.

[0155] According to an exemplary embodiment of the present disclosure, the quantization parameter increment determination unit 74 is further configured to obtain a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is the rate-distortion cost without considering the influence of inter-frame propagation; obtain a second rate-distortion cost of the current block based on the distortion information of the current block and a predetermined block, wherein the second rate-distortion cost is the rate-distortion cost considering the influence of inter-frame propagation; obtain a quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; and adjust the initial quantization parameter of the current block based on the quantization parameter increment to obtain the quantization parameter of the current block.

[0156] According to an exemplary embodiment of the present disclosure, the quantization parameter increment determination unit 74 is further configured to add the distortion information of the current block and the predetermined block to obtain a total distortion impact; and determine a second rate-distortion cost based on the total distortion impact and the bit rate.

[0157] According to an exemplary embodiment of the present disclosure, the quantization parameter increment determination unit 74 is also configured to respectively derive the first rate-distortion cost and the second rate-distortion cost with respect to the bit rate to obtain a first derivative result and a second derivative result; based on the second rate-distortion cost, obtain a propagation influence factor of the current block; based on the first derivative result, the second derivative result and the propagation influence factor, obtain a quantization parameter increment of the current block.

[0158] According to an exemplary embodiment of the present disclosure, the quantization parameter increment determination unit 74 is also configured to determine the respective influence factors of the current block and the predetermined block through the intra-frame cost and inter-frame cost of the current block and the predetermined block; and obtain the propagation influence factor of the current block based on the determined influence factor and the second rate-distortion cost.

[0159] According to an exemplary embodiment of the present disclosure, the quantization parameter increment determination unit 74 is also configured to determine a first quantization parameter based on a first rate coefficient corresponding to a first rate-distortion cost; obtain a second rate coefficient corresponding to a second rate-distortion cost based on a first derivative result, a second derivative result and the first rate coefficient; determine a second quantization parameter based on the second rate coefficient; and obtain a quantization parameter increment function based on the first quantization parameter, the second quantization parameter and a propagation influence factor.

[0160] According to an exemplary embodiment of the present disclosure, the quantization parameter increment determination unit 74 is also configured to obtain the difference between the intra-frame cost and the inter-frame cost of the block for each block in the current block and the predetermined block, and compare the difference with the intra-frame cost of the block to obtain the impact factor of the block.

[0161] Figure 8A computing environment 810 is shown coupled to a user interface 850. The computing environment 810 may be part of a data processing server. The computing environment 810 includes a processor 820, a memory 830, and an input / output (I / O) interface 840.

[0162] The processor 820 generally controls the overall operation of the computing environment 810, such as operations associated with display, data acquisition, data communication, and image processing. The processor 820 may include one or more processors for executing instructions to perform all or some steps in the above method. In addition, the processor 820 may include one or more modules that facilitate the interaction between the processor 820 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0163] The memory 830 is configured to store various types of data to support the operation of the computing environment 810. The memory 830 may include predetermined software 832. Examples of such data include instructions for any application or method operating on the computing environment 810, video data sets, image data, etc. The memory 830 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0164] The I / O interface 840 provides an interface between the processor 820 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 840 may be coupled to an encoder and a decoder.

[0165] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 830 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 820 in a computing environment 810 to perform the above method. In one example, the plurality of programs can be executed by a processor 820 in a computing environment 810 to (for example, from Figure 2The video encoder 20 in the computing environment 810 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 820 in the computing environment 810 to perform the above-mentioned decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 820 in the computing environment 810 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 820 in the computing environment 810 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0166] In an embodiment, the computing environment 810 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0167] According to an embodiment of the present disclosure, an electronic device may be provided, which includes at least one memory and at least one processor, wherein a set of computer executable instructions is stored in the at least one memory, and when the set of computer executable instructions is executed by the at least one processor, a video encoding method according to an embodiment of the present disclosure is executed.

[0168] As an example, the electronic device can be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device 1000 is not necessarily a single electronic device, but can also be any device or circuit that can execute the above-mentioned instructions (or instruction sets) individually or jointly. The electronic device can also be part of an integrated control system or a system manager, or can be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0169] In addition, the electronic device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0170] According to an embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein when the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the video encoding method of the embodiment of the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0171] In an embodiment, a method for generating a bitstream is provided, the method comprising the bitstream generated by the above video encoding method. In an embodiment, a method for generating a bitstream is provided, the method comprising encoded video information generated by the above video encoding method.

[0172] According to an embodiment of the present disclosure, a computer program product having instructions for storing a bitstream is also provided, wherein the bitstream includes encoded video information generated by the above-mentioned video encoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 830 is also provided, wherein the plurality of programs can be executed by a processor 820 in a computing environment 810 to perform the above-mentioned method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0173] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0174] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0175] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that: include: Determine distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame that is actually obtainable after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of a reference block of a previous frame on the distortion of the block; Based on the distortion information of each block, predicting the distortion information of each block in a second subsequent frame, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; Determining a quantization parameter increment of the current block based on distortion information of the current block in the current frame and distortion information of predetermined blocks in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is affected by the distortion of the current block; The current block is encoded based on the determined quantization parameter increment.

2. The video encoding method according to claim 1, characterized in that: The determining of distortion information of each block in the current frame and the first subsequent frame comprises: Determining distortion information of each block in the current frame and the first subsequent frame based on a distortion influence linear model and distortion information of a predetermined reference block, wherein the predetermined reference block is a block on a zeroth frame in the video to be encoded; The predicting, based on the distortion information of each block, the distortion information of each block in the second subsequent frame comprises: Distortion information of each block in a second subsequent frame is predicted based on an influence factor of a reference block of a last frame in the first subsequent frame and distortion information of each block in the current frame and the first subsequent frame.

3. The video encoding method according to claim 1, wherein: The determining, based on the distortion information of the current block in the current frame and the distortion information of the predetermined blocks in the first subsequent frame and the second subsequent frame, a quantization parameter increment of the current block comprises: Based on the distortion information of the current block, obtaining a first rate-distortion cost of the current block, wherein the first rate-distortion cost is a rate-distortion cost without considering the influence of inter-frame propagation; Based on the distortion information of the current block and the predetermined block, obtaining a second rate-distortion cost of the current block, wherein the second rate-distortion cost is a rate-distortion cost in consideration of inter-frame propagation influence; Acquire a quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; An initial quantization parameter of the current block is adjusted based on the quantization parameter increment to obtain the quantization parameter of the current block.

4. The video encoding method according to claim 3, characterized in that: The acquiring, based on the distortion information of the current block and the predetermined block, a second rate-distortion cost of the current block includes: Adding the distortion information of the current block and the predetermined block to obtain a total distortion impact; The second rate-distortion cost is determined based on the total distortion impact and a bit rate.

5. The video encoding method according to claim 3, characterized in that: The acquiring, based on the first rate-distortion cost and the second rate-distortion cost, a quantization parameter increment of the current block includes: Deriving the first rate-distortion cost and the second rate-distortion cost with respect to the bit rate respectively to obtain a first derivation result and a second derivation result; Based on the second rate-distortion cost, obtaining a propagation impact factor of the current block; A quantization parameter increment of the current block is obtained based on the first derivation result, the second derivation result and the propagation influence factor.

6. The video encoding method according to claim 5, characterized in that: The obtaining, based on the second rate-distortion cost, a propagation impact factor of the current block includes: Determine the respective impact factors of the current block and the predetermined block according to the intra-frame cost and the inter-frame cost of the current block and the predetermined block; Based on the determined impact factor and the second rate-distortion cost, a propagation impact factor of the current block is obtained.

7. The video encoding method according to claim 5, characterized in that: The obtaining the quantization parameter increment of the current block based on the first derivation result, the second derivation result and the propagation influence factor includes: Determining a first quantization parameter based on a first rate coefficient corresponding to the first rate-distortion cost; Obtaining a second rate coefficient corresponding to a second rate-distortion cost based on the first derivation result, the second derivation result, and the first rate coefficient; Determining a second quantization parameter based on the second rate coefficient; The quantization parameter increment is obtained based on the first quantization parameter, the second quantization parameter and the propagation influence factor.

8. The video encoding method according to claim 6, characterized in that: The determining, by using the intra-frame cost and the inter-frame cost of the current block and the predetermined block, respective impact factors of the current block and the predetermined block comprises: For each of the current block and the predetermined block, a difference between the intra-frame cost and the inter-frame cost of the block is obtained, and the difference is compared with the intra-frame cost of the block to obtain an impact factor of the block.

9. A video encoding device, characterized in that: include: a distortion information determining unit, configured to determine distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame that is actually obtainable after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of the reference block of the previous frame on the distortion of the block; A prediction unit, configured to predict distortion information of each block in a second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; a quantization parameter increment determining unit, configured to determine a quantization parameter increment of the current block based on distortion information of the current block in the current frame and distortion information of predetermined blocks in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is affected by the distortion of the current block; The encoding unit is configured to encode the current block based on the determined quantization parameter increment.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video encoding method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the video encoding method according to any one of claims 1 to 8.

12. A computer program product having instructions for storing a bit stream, characterized in that The bitstream includes: Video data generated according to the video encoding method according to any one of claims 1 to 8.

13. A method for generating a bit stream, characterized in that: include: A bit stream is generated according to the video encoding method according to any one of claims 1 to 8.

Images