Video encoding method and device, electronic device, computer readable storage medium, computer program product and method for generating bitstream

Abstract

The present disclosure relates to a video encoding method and device, an electronic device, a computer readable storage medium, a computer program product and a method of generating a bitstream. The video encoding method comprises: determining distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame actually available after the current frame, and the distortion information of the block is used to indicate an influence of distortion of a reference block of a previous frame on distortion of the block; predicting distortion information of each block in a second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; determining a quantization parameter increment of a current block in the current frame based on the distortion information of the current block, and the distortion information of predetermined blocks in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined blocks is influenced by the distortion of the current block; and encoding the current block based on the determined quantization parameter increment.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of video processing, and in particular, to a video encoding method and device, an electronic device, a computer readable storage medium, a computer program product and a method for generating a bitstream. BACKGROUND

[0002] Rate control is an important part of the video encoding process, which involves allocating rate to different frames of the video to be encoded, so as to ensure the quality and compression efficiency of video encoding. Rate control is generally achieved by adjusting the quantization parameter (QP) to control the compression of spatial details. The QP value determines the quality of the encoded video to some extent. Generally, the smaller the QP value, the more details are retained, and the higher the video clarity. The larger the QP value, the more details are discarded, and the lower the video clarity.

[0003] Currently, in open source encoders x264 and x265, the cuTree encoding tool in the pre-analysis stage allocates rate to each frame to improve compression efficiency. However, in a low-latency scenario (such as a live scenario), cuTree cannot obtain a good rate allocation result, which limits the compression performance. SUMMARY

[0004] The present disclosure provides a video encoding method and device, an electronic device, a computer readable storage medium, a computer program product and a method for generating a bitstream to at least solve the problem that related technologies cannot obtain a good rate allocation result, resulting in limited compression performance.

[0005] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, comprising: determining distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame that can actually be obtained after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of a reference block of a previous frame on the distortion of the block; predicting the distortion information of each block in a second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; determining a quantization parameter increment of a current block in the current frame based on the distortion information of the current block, the distortion information of a predetermined block in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is affected by the distortion of the current block; and encoding the current block based on the determined quantization parameter increment.

[0006] Optionally, determining the distortion information of each block in the current frame and the first subsequent frame comprises: determining the distortion information of each block in the current frame and the first subsequent frame based on a distortion influence linear model and distortion information of a predetermined reference block, wherein the predetermined reference block is a block on the zeroth frame in the video to be encoded.

[0007] The distortion information of each block in the second subsequent frame is predicted based on the influence factor of the reference block of the last frame in the first subsequent frame, the distortion information of each block in the current frame and the first subsequent frame.

[0008] Optionally, the quantization parameter increment of the current block is determined based on the distortion information of the current block in the current frame and the distortion information of the predetermined block in the first subsequent frame and the second subsequent frame, comprising: obtaining a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is a rate-distortion cost without considering the influence of inter-frame propagation; obtaining a second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block, wherein the second rate-distortion cost is a rate-distortion cost considering the influence of inter-frame propagation; obtaining the quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; and adjusting the initial quantization parameter of the current block based on the quantization parameter increment to obtain the quantization parameter of the current block.

[0009] Optionally, the second rate-distortion cost of the current block is obtained based on the distortion information of the current block and the predetermined block, comprising: adding the distortion information of the current block and the predetermined block to obtain a total influence of distortion; and determining the second rate-distortion cost based on the total influence of distortion and the code rate.

[0010] Optionally, the quantization parameter increment of the current block is obtained based on the first rate-distortion cost and the second rate-distortion cost, comprising: deriving the first rate-distortion cost and the second rate-distortion cost with respect to the code rate to obtain a first derivative result and a second derivative result; obtaining a propagation influence factor of the current block based on the second rate-distortion cost; and obtaining the quantization parameter increment of the current block based on the first derivative result, the second derivative result and the propagation influence factor.

[0011] Optionally, the propagation influence factor of the current block is obtained based on the second rate-distortion cost, comprising: obtaining a propagation influence factor function based on the second rate-distortion cost, wherein the influence factor of the current block and the predetermined block is determined through the intra-frame cost and the inter-frame cost of the current block and the predetermined block; and obtaining the propagation influence factor of the current block based on the determined influence factor and the second rate-distortion cost.

[0012] Optionally, the quantization parameter increment of the current block is obtained based on the first derivative result, the second derivative result and the propagation influence factor, comprising: determining a first quantization parameter based on a first code rate coefficient corresponding to the first rate-distortion cost; obtaining a second code rate coefficient corresponding to the second rate-distortion cost based on the first derivative result, the second derivative result and the first code rate coefficient; determining a second quantization parameter based on the second code rate coefficient; and obtaining the quantization parameter increment based on the first quantization parameter, the second quantization parameter and the propagation influence factor.

[0013] Optionally, the determining the influence factor of each of the current block and the predetermined block based on the intra cost and the inter cost of the current block and the predetermined block comprises: obtaining a difference between the intra cost and the inter cost of each of the current block and the predetermined block, and performing a ratio operation on the difference and the intra cost of the block to obtain the influence factor of the block.

[0014] According to a second aspect of the embodiments of the present disclosure, a video encoding apparatus is provided, comprising: a distortion information determination unit configured to determine distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame actually available after the current frame, and the distortion information of the block is used to indicate an influence of distortion of a reference block of a previous frame on distortion of the block; a prediction unit configured to predict distortion information of each block in a second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; a quantization parameter increment determination unit configured to determine a quantization parameter increment of a current block based on the distortion information of the current block in the current frame and the distortion information of a predetermined block in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is influenced by the distortion of the current block; and an encoding unit configured to encode the current block based on the determined quantization parameter increment.

[0015] Optionally, the distortion information determination unit is further configured to determine the distortion information of each block in the current frame and the first subsequent frame based on a distortion influence linear model and the distortion information of a predetermined reference block, wherein the predetermined reference block is a block on a zeroth frame in the video to be encoded.

[0016] The prediction unit is further configured to predict the distortion information of each block in the second subsequent frame based on the influence factor of the reference block of the last frame in the first subsequent frame and the distortion information of each block in the current frame and the first subsequent frame.

[0017] Optionally, the quantization parameter increment determination unit is further configured to obtain a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is a rate-distortion cost without considering inter-frame propagation influence; obtain a second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block, wherein the second rate-distortion cost is a rate-distortion cost considering inter-frame propagation influence; obtain the quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; and obtain the quantization parameter of the current block based on the initial quantization parameter of the current block adjusted by the quantization parameter increment.

[0018] Optionally, the quantization parameter increment determination unit is further configured to add the distortion information of the current block and the predetermined block to obtain a total influence of distortion; and determine the second rate-distortion cost based on the total influence of distortion and a code rate.

[0019] Optionally, the quantization parameter increment determination unit is further configured to derive the first rate-distortion cost and the second rate-distortion cost with respect to the code rate to obtain a first derivative result and a second derivative result; obtain a propagation influence factor of the current block based on the second rate-distortion cost; and obtain the quantization parameter increment of the current block based on the first derivative result, the second derivative result, and the propagation influence factor.

[0020] Optionally, the quantization parameter increment determination unit is further configured to determine an influence factor of each of the current block and the predetermined block based on the intra-frame cost and the inter-frame cost of the current block and the predetermined block; and obtain a propagation influence factor of the current block based on the determined influence factor and the second rate-distortion cost.

[0021] Optionally, the quantization parameter increment determination unit is further configured to determine a first quantization parameter based on a first code rate coefficient corresponding to the first rate-distortion cost; obtain a second code rate coefficient corresponding to the second rate-distortion cost based on the first derivative result, the second derivative result, and the first code rate coefficient; determine a second quantization parameter based on the second code rate coefficient; and obtain the quantization parameter increment based on the first quantization parameter, the second quantization parameter, and the propagation influence factor.

[0022] Optionally, the quantization parameter increment determination unit is further configured to, for each of the current block and the predetermined block, obtain a difference between the intra-frame cost and the inter-frame cost of the block, and obtain an influence factor of the block by taking a ratio of the difference to the intra-frame cost of the block.

[0023] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement a video encoding method according to the present disclosure.

[0024] According to a fourth aspect of embodiments of the present disclosure, a computer-readable storage medium is provided, when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is caused to perform a video encoding method according to the present disclosure.

[0025] According to a fifth aspect of embodiments of the present disclosure, a computer program product is provided, having instructions for storing a bitstream, wherein the bitstream comprises: video data generated according to a video encoding method as described above.

[0026] According to a sixth aspect of embodiments of the present disclosure, a method for generating a bitstream is provided, comprising: generating a bitstream according to a video encoding method as described above.

[0027] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0028] According to the video coding method and device, the electronic device, the computer readable storage medium, the computer program product and the method for generating a bitstream provided in the present disclosure, the distortion information of each block in the subsequent frame that cannot be actually obtained is predicted by using the distortion information of each block in the actually obtainable frame, so that the subsequent frame of the current frame not only includes the actually obtainable frame but also can include the predicted frame, the number of the subsequent frame of the current frame is increased, and when the quantization parameter of the current block is determined, not only the current frame and the frame actually transmitted after the current frame are considered, but also the predicted subsequent frame is considered, the correction of the quantization parameter is realized, the current block is encoded based on the corrected quantization parameter, and a better code rate allocation result can be obtained, so that the output bit rate of the encoder is effectively reduced under the condition of guaranteeing the video quality, and higher compression efficiency is achieved, and the bandwidth resource is saved. Therefore, the present disclosure solves the problem that the related art cannot obtain a better code rate allocation result, and the compression performance is limited.

[0029] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings, which are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure, and do not constitute an improper limitation on the present disclosure.

[0031] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding a video block according to some embodiments of the present disclosure;

[0032] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure;

[0033] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure;

[0034] Figure 4 is a schematic diagram of an existing distortion propagation impact;

[0035] Figure 5 is a schematic diagram of a distortion propagation impact according to an exemplary embodiment of the present disclosure;

[0036] Figure 6 is a flowchart of a video coding method according to an exemplary embodiment of the present disclosure;

[0037] Figure 7 is a block diagram of a video coding device according to an exemplary embodiment of the present disclosure;

[0038] Figure 8 is a diagram illustrating a computing environment coupled with a user interface in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION

[0039] In order to make the ordinary person skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings.

[0040] It should be noted that the terms "first", "second", and the like in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0041] It should be noted herein that "at least one of a plurality of items" appearing in the present disclosure means that the three types of alternatives of "any one of the plurality of items", "a combination of any two or more of the plurality of items", and "all of the plurality of items" are included. For example, "including at least one of A and B" includes the following three alternatives: (1) including A; (2) including B; and (3) including A and B. For another example, "performing at least one of step one and step two" means the following three alternatives: (1) performing step one; (2) performing step two; and (3) performing step one and step two.

[0042] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, in accordance with some embodiments of the present disclosure. As shown in Figure 1 The system 10 includes a source device 12 that generates and encodes video data to be decoded later by a destination device 14, as shown in

[0043] In some embodiments, the destination device 14 can receive encoded video data to be decoded via a link 16. The link 16 can comprise any type of communication medium or device capable of moving the encoded video data from the source device 12 to the destination device 14.

[0044] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the target device 14 can access the encoded video data in the storage device 32 via the input interface 28.

[0045] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0046] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback.

[0047] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0048] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0049] Video encoder 20 and video decoder 30 can each be implemented as any of a variety of suitable encoder and / or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuitry, software, hardware, firmware or any combinations thereof. When implemented partially in software, an electronic device can store instructions for the software in a suitable, non- transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video coding / decoding operations disclosed in the present disclosure. Each of video encoder 20 and video decoder 30 can be included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0050] Figure 2 is a block diagram illustrating an example video encoder 20, in accordance with some embodiments described in the present disclosure. Video encoder 20 can perform intra-prediction encoding and inter-prediction encoding on video blocks within a video frame. Intra-prediction encoding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-prediction encoding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video coding, the term "frame" can be used as a synonym for the term "image" or "picture."

[0051] As Figure 2As shown in FIG. 1, video encoder 20 includes video data memory 40, prediction processing unit 41, decoded picture buffer (DPB) 64, summer 50, transform processing unit 52, quantization unit 54, and entropy encoding unit 56. Prediction processing unit 41 further includes motion estimation unit 42, motion compensation unit 44, partition unit 45, intra-prediction processing unit 46, and intra block copy (BC) unit 48. In some implementations, video encoder 20 also includes inverse quantization unit 58, inverse transform processing unit 60, and summer 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between summer 62 and DPB 64 to filter block boundaries to remove blockiness artifacts from reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of summer 62. In some examples, the loop filters can be omitted, and the decoded video blocks can be provided directly from summer 62 to DPB 64. Video encoder 20 can take the form of a fixed or programmable hardware encoder, or can be dispersed into one or more of the illustrated fixed or programmable hardware encoders.

[0052] Video data memory 40 can store video data to be encoded by the components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18 as shown in FIG. 1. DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) for use in encoding video data by video encoder 20 (e.g., in intra- or inter-coding modes). Figure 1

[0053] As shown in FIG. 1, video encoder 20 includes video data memory 40, prediction processing unit 41, decoded picture buffer (DPB) 64, summer 50, transform processing unit 52, quantization unit 54, and entropy encoding unit 56. Prediction processing unit 41 further includes motion estimation unit 42, motion compensation unit 44, partition unit 45, intra-prediction processing unit 46, and intra block copy (BC) unit 48. In some implementations, video encoder 20 also includes inverse quantization unit 58, inverse transform processing unit 60, and summer 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between summer 62 and DPB 64 to filter block boundaries to remove blockiness artifacts from reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of summer 62. In some examples, the loop filters can be omitted, and the decoded video blocks can be provided directly from summer 62 to DPB 64. Video encoder 20 can take the form of a fixed or programmable hardware encoder, or can be dispersed into one or more of the illustrated fixed or programmable hardware encoders. Figure 2

[0054] ​​Prediction processing unit 41 can select one of a plurality of possible predictive encoding modes, e.g., one of a plurality of intra-predictive encoding modes or one of a plurality of inter-predictive encoding modes, for the current video block based on the error results (e.g., coding rate and level of distortion). Prediction processing unit 41 can provide the resulting intra- or inter-predicted block to summer 50 to generate a residual block, and to summer 62 to reconstruct the encoded block for use as part of a reference frame at a later time. Prediction processing unit 41 also provides syntax elements, e.g., motion vectors, intra-mode indicators, partitioning information, and other such syntax information, to entropy encoding unit 56.

[0055] To select an appropriate intra-predictive encoding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 can perform intra-predictive encoding of the current video block in relation to one or more neighboring blocks in the same frame as the current block being encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-predictive encoding of the current video block in relation to one or more predictive blocks in one or more reference frames to provide temporal prediction. Video encoder 20 can perform multiple encoding passes, e.g., to select a suitable encoding mode for each block of video data.

[0056] In some implementations, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating motion vectors according to a predetermined pattern within a sequence of video frames, the motion vectors indicating displacement of video blocks within the current video frame relative to predictive blocks within a reference video frame. Motion estimation performed by motion estimation unit 42 is a process of generating motion vectors that estimate motion for video blocks. For example, a motion vector can indicate displacement of a video block within a current video frame or picture relative to a predictive block within a reference frame that is related to a current block being encoded within the current frame. The predetermined pattern can designate video frames in the sequence as P-frames or B-frames. Intra-BC unit 48 can determine vectors for intra-BC encoding (e.g., block vectors) in a similar manner as motion vectors determined by motion estimation unit 42 for inter-prediction, or can utilize block vectors determined by motion estimation unit 42.

[0057] Whether the predictive block is from the same frame according to intra-prediction or a different frame according to inter-prediction, video encoder 20 can form pixel difference values by subtracting pixel values of the predictive block from pixel values of the current video block being encoded, thereby forming a residual video block. The pixel difference values forming the residual video block can include both luma component differences and chroma component differences.

[0058] Intra prediction processing unit 46 can encode the current block using various intra prediction modes, e.g., during a separate encoding pass, and can select a suitable intra prediction mode to use from the tested intra prediction modes (or, in some examples, the mode selection unit). Intra prediction processing unit 46 can provide information indicating the selected intra prediction mode for the block to entropy encoding unit 56. Entropy encoding unit 56 can encode information indicating the selected intra prediction mode in the bitstream.

[0059] After prediction processing unit 41 determines a prediction block for the current video block via either inter prediction or intra prediction, summer 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block can be included in one or more TUs and is provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform, e.g., a discrete cosine transform (DCT) or a conceptually similar transform.

[0060] Transform processing unit 52 can send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting a quantization parameter. In some examples, quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Alternatively, entropy encoding unit 56 can perform the scan.

[0061] After quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients using, e.g., context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy encoding methodology or technique, into a video bitstream. The encoded bitstream can then be transmitted to video decoder 30 as shown in FIG. 3, or archived on a storage device 32 as shown in FIG. 3 for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 can also entropy encode motion vectors and other syntax elements for the current video frame being encoded. Figure 1 Figure 1

[0062] ​​The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for generating a reference block used to predict other video blocks. As noted above, the motion compensation unit 44 can generate a motion compensated prediction block from one or more reference blocks of a frame stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values for use in motion estimation.

[0063] The summer 62 adds the reconstructed residual block to the motion compensated prediction block produced by the motion compensation unit 44 to produce a reference block for storage in the DPB 64. The reference block can then be used by the intra BC unit 48, the motion estimation unit 42, and the motion compensation unit 44 as a prediction block to inter predict another video block in a subsequent video frame.

[0064] Figure 3 FIG. 1 shows a block diagram of an example video decoder 30 in accordance with some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, a summer 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra prediction unit 84, and an intra BC unit 85. The video decoder 30 can perform a decoding process substantially reciprocal to the encoding process described above in connection with the video encoder 20. Figure 2 The decoding process described in connection with the video encoder 20 is substantially reciprocal. For example, the motion compensation unit 82 can generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra prediction unit 84 can generate prediction data based on intra prediction mode indicators received from the entropy decoding unit 80.

[0065] In some examples, embodiments of the present disclosure can be dispersed among one or more of the units of the video decoder 30. For example, the intra BC unit 85 can perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 can not include the intra BC unit 85, and the functionality of the intra BC unit 85 can be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0066] The video data memory 79 can store video data, such as an encoded video bitstream, to be decoded by the other components of the video decoder 30. The video data stored in the video data memory 79 can be obtained, for example, from the storage device 32, from a local video source, such as a camera, via wired or wireless network communication of video data, or by accessing a physical data storage medium, such as a flash drive or hard disk.

[0067] During the decoding process, video decoder 30 receives an encoded video bitstream that represents encoded video frames and associated syntax elements of video blocks. Video decoder 30 can receive the syntax elements at the video frame level and / or the video block level. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors or intra prediction mode indicators, and other syntax elements. Entropy decoding unit 80 then forwards the motion vectors or intra prediction mode indicators, and other syntax elements to prediction processing unit 81.

[0068] When a video frame is coded as an intra-predicted coded (I) frame or an intra coded prediction block in other types of frames, intra prediction unit 84 of prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode signaled and reference data from previously decoded blocks of the current frame.

[0069] When a video frame is coded as an inter-predicted coded (i.e., B or P) frame, motion compensation unit 82 of prediction processing unit 81 produces one or more prediction blocks for a video block of the current video frame based on the motion vectors and other syntax elements received from entropy decoding unit 80. Each of the prediction blocks can be produced from a reference frame within one of the reference frame lists. Video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, using default construction techniques based on reference frames stored in DPB 92.

[0070] In some examples, when a video block is coded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 produces a prediction block for the current video block based on the block vectors and other syntax elements received from entropy decoding unit 80. The prediction block can be within a reconstructed region of the same picture as the current video block, as defined by video encoder 20.

[0071] Motion compensation unit 82 and / or intra BC unit 85 determine the prediction information for a video block of the current video frame by parsing the motion vectors and other syntax elements, and then use the prediction information to produce the prediction block for the current video block being decoded.

[0072] Motion compensation unit 82 can also perform interpolation to calculate interpolated values for sub-integer pixels of reference blocks using interpolation filters as used by video encoder 20 during encoding of the video blocks. In this case, motion compensation unit 82 can determine the interpolation filters used by video encoder 20 from the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0073] Inverse quantization unit 86 inverse quantizes quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80 using the same quantization parameter calculated by video encoder 20 for each video block in the video frame to determine a degree of quantization. Inverse transform processing unit 88 applies an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients in order to reconstruct the residual blocks in the pixel domain.

[0074] After motion compensation unit 82 or intra BC unit 85 generates the prediction block for the current video block based on the vectors and other syntax elements, adder 90 reconstructs the decoded video block for the current video block by adding the residual block from inverse transform processing unit 88 to the corresponding prediction block generated by motion compensation unit 82 and intra BC unit 85. In-loop filter 91, e.g., a de-blocking filter, a SAO filter, a CCSAO filter, and / or an ALF, can be located between adder 90 and DPB 92 to further process the decoded video block. In some examples, in-loop filter 91 can be omitted, and the decoded video block can be directly provided by adder 90 to DPB 92. The decoded video blocks in a given frame are then stored in DPB 92, which stores reference frames for subsequent motion compensation of video blocks that follow. DPB 92 or a memory device separate from DPB 92 can also store decoded video for later presentation on a display device (e.g., display device 34 of FIG. 1). Figure 1

[0075] Rate control is an important part of the video encoding process, which involves allocating code rate to different image frames of the video to be encoded, so as to ensure the quality and compression efficiency of the video encoding. Different code rate allocation schemes will have different effects on the quality and compression efficiency of the encoded output video. If an excellent code rate allocation scheme is adopted, the output bit rate of the encoder can be effectively reduced under the condition of ensuring the video quality, so as to achieve higher compression efficiency and save bandwidth resources. At present, in the open source encoders x264 and x265, the code rate of each frame is allocated by the cuTree encoding tool in the pre-analysis stage. The pre-analysis stage usually needs to obtain the subsequent image frames in advance, analyzes the influence of the quality change (such as encoding distortion) of the current image frame on the quality of the subsequent image frames, and then determines the code rate allocated to the current image frame. For example, Figure 4 is a schematic diagram of the existing distortion propagation influence. As shown in Figure 4 , the current image frame f n needs to be combined with the subsequent image frames f n+1 ~ f n+4 , to analyze the influence of the encoding distortion of f n on the encoding distortion of the subsequent image frames, and then determine the code rate allocated to f n ​rate control can be achieved by adjusting the quantization parameter, that is, the code rate control can be achieved by controlling the quantization parameter f n In order to make cuTree obtain better code rate allocation results, it is usually necessary to obtain a large number of subsequent frames in advance to determine the quantization parameter, but in a low-delay scene (such as a live scene), the number of subsequent frames of the current frame is limited, thus limiting the code rate allocation of cuTree, and the compression performance is limited.

[0076] To solve the above problems, the present disclosure introduces a prediction propagation process, predicts subsequent propagated image frames on the basis of actually propagated image frames, makes up for the propagation limitation problem caused by the inability to obtain sufficient subsequent image frames in a low-delay scene, corrects the quantization parameter of the current image frame by using the predicted subsequent image frames to allocate a suitable code rate, and thus improves the compression rate of the encoder. For example, Figure 5 is a schematic diagram of code rate allocation according to an example embodiment of the present disclosure. As shown in Figure 5 , not only the encoding distortion of f n needs to be analyzed to affect the encoding distortion of the actually propagated subsequent image frames f n+1 ~ f n+4 , but also the encoding distortion of f n needs to be predicted to affect the encoding distortion of the subsequent frames f' that have not been actually propagated f' n+5 ~ f' n+8 , and the code rate allocated to f n is determined jointly.

[0077] In the following, the video encoding method and device, electronic device, computer readable storage medium, computer program product and method of generating a bit stream according to the example embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0078] Figure 6 is a flowchart of a video encoding method according to an example embodiment of the present disclosure, as shown in Figure 6 , the video encoding method comprises the following steps:

[0079] In step S601, the distortion information of each block in the current frame and the first subsequent frame is determined, wherein the first subsequent frame is at least one subsequent frame that can be actually obtained after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of the reference block of the previous frame on the distortion of the block.

[0080] As an example, taking the encoding tool cuTree as an example, the distortion of the current block in cuTree is affected by the distortion of the reference block of the previous frame, and in general cases, the influence of the reference block on the distortion of the current block on the previous frame can be represented as a linear model, therefore, for the blocks on the actually obtainable frames, the distortion information of the block can be determined by the linear model, and the present disclosure is not limited thereto.

[0081] According to the example embodiment of the present disclosure, determining the distortion information of each block in the current frame and the first subsequent frame can comprise: determining the distortion information of each block in the current frame and the first subsequent frame based on the distortion influence linear model and the distortion information of a predetermined reference block, wherein the predetermined reference block is a block on the zeroth frame in the video to be encoded. Through the embodiment, the distortion information of each block in the current frame and the first subsequent frame can be determined conveniently and quickly.

[0082] As an example, based on the above linear model, i.e., the distortion influence linear model, the distortion information of each block can be represented by the distortion of the block on the zeroth frame. The following system describes how to represent the distortion information of each block in the current frame and the first subsequent frame by the distortion of the block on the zeroth frame.

[0083] Specifically, the above distortion influence linear model can be expressed as follows:

[0084] D m =k m-1 D m-1 +b m-1 ;

[0085] D m-1 =k m-2 D m-2 +b m-2 ; ......;

[0087] D1=k0D0+b0(1)

[0088] Wherein, D m m represents the distortion information of the current block on the mth frame, k m m-1 represents the influence factor of the distortion of the reference block on the m-1th frame on the current block on the mth frame, b m represents a constant term, and D0 represents the distortion information of the block on the zeroth frame.

[0089] Based on the distortion influence linear model of formula (1), the influence of D0 on D m , i.e., how to represent the distortion information of each block in the current frame and the first subsequent frame by the distortion of the block on the zeroth frame, can be derived as follows:

[0090]

[0091] Wherein, b is a constant term.

[0092] Returning to Figure 6 , in step S602, based on the distortion information of each block, the distortion information of each block in the second subsequent frame is predicted, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame.

[0093] As an example, after obtaining the distortion information of each block in the current frame and the first subsequent frame, the distortion information of each block in the second subsequent frame which cannot be actually obtained can be predicted by the distortion information of these blocks, that is, the information of the subsequent frame which cannot be actually obtained is increased, which is equivalent to increasing the number of subsequent frames of the current frame.

[0094] According to an example embodiment of the present disclosure, predicting the distortion information of each block in the second subsequent frame based on the distortion information of each block can include predicting the distortion information of each block in the second subsequent frame based on the influence factor of the reference block of the last frame in the first subsequent frame, the distortion information of each block in the current frame and the first subsequent frame. Through this embodiment, since the influence factor of the last frame in the first subsequent frame which is closest to the second subsequent frame is used to predict the distortion information of each block in the second subsequent frame in combination with the distortion information of each block in the current frame and the first subsequent frame, the distortion information of the block on the second subsequent frame can be relatively accurately predicted.

[0095] As an example, since there is continuity in most scenes of a video, it is assumed here that there is a trend of continuous transformation from the mth frame to the nth frame, and then the subsequent propagation is predicted based on the assumption. When predicting, the influence factor of the reference block of the last frame in the first subsequent frame is used, that is, the influence factor of the reference block of the frame which can be actually obtained is used to predict the distortion information of the corresponding current block in the corresponding second subsequent frame. At the same time, in the process of prediction, the distortion of the block on the zeroth frame is combined with the above influence factor to determine the distortion information of the corresponding current block in the corresponding second subsequent frame.

[0096] It is assumed that the last frame of the first subsequent frame is the mth frame, and the influence factor k of the last frame mth frame m-1 At this time, the influence factor of each frame in the second subsequent frame in the embodiment of the present disclosure can be predicted as k m-1 , which is equivalent to using the influence factor of the last frame in the first subsequent frame which is closest to the second subsequent frame, and then the influence of D0 on the second subsequent frame D m+n which cannot be actually obtained can be represented as:

[0097]

[0098] where b' is a constant term.

[0099] Returning to Figure 7 , in step S703, a quantization parameter increment of the current block is determined based on the distortion information of the current block in the current frame and the distortion information of predetermined blocks in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined blocks is affected by the distortion of the current block.

[0100] As an example, in determining the quantization parameter increment of the current block, the number of the employed subsequent frames can be set, but the employed subsequent frames should include the second subsequent frame, so that the number of the employed subsequent frames is enough and the second subsequent frame is also used to correct the quantization parameter increment.

[0101] According to an example embodiment of the present disclosure, in determining the quantization parameter increment of the current block based on the distortion information of the current block in the current frame, the distortion information of the predetermined block in the first subsequent frame and the second subsequent frame, the method can comprise: obtaining a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is a rate-distortion cost without considering the influence of inter-frame propagation; obtaining a second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block, wherein the second rate-distortion cost is a rate-distortion cost considering the influence of inter-frame propagation; obtaining the quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; and adjusting the initial quantization parameter of the current block based on the quantization parameter increment to obtain the quantization parameter of the current block. Through this embodiment, the information of the second subsequent frame is perfectly introduced by using the rate-distortion costs in two cases, so that the quantization parameter increment can be corrected by using the information of the second subsequent frame to obtain a relatively appropriate quantization parameter, and then the appropriate code rate is allocated.

[0102] As an example, after obtaining the corresponding rate-distortion cost based on the distortion information, the first rate-distortion cost and the second rate-distortion cost can be used to derive the code rate, and according to the derivation results of the two rate-distortion costs, a quantization parameter function considering the subsequent frame can be obtained, and then combined with the initial quantization parameter function, a corresponding quantization parameter increment function can be further obtained. The quantization parameter increment function can be applied in any low-delay scenario, and only the corresponding quantization parameter increment needs to be obtained according to the corresponding scene according to the quantization parameter increment function.

[0103] As an example, after obtaining the quantization parameter increment function, the influence factor in the second rate-distortion cost can be further calculated, and the calculated influence factor is brought into the quantization parameter function, so as to obtain the quantization parameter increment, so as to adjust the initial quantization parameter to obtain the final corrected quantization parameter, which is used for video encoding, so that the compression performance is improved.

[0104] According to an example embodiment of the present disclosure, in obtaining the second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block, the method can comprise: adding the distortion information of the current block and the predetermined block to obtain a total distortion influence; and determining the second rate-distortion cost based on the total distortion influence and the code rate. Through this embodiment, the rate-distortion cost conforming to the actual situation can be determined.

[0105] As an example, taking the current frame as the 0th frame, the rate-distortion cost of the current block on the 0th frame can be expressed as:

[0106] J1=D0+λR0(4)

[0107] In the case of considering the inter-frame propagation effect, the rate-distortion cost of the current block on the 0th frame can be expressed as:

[0108]

[0109] wherein R0 is the code rate of the current block on the 0th frame, and λ is the code rate coefficient.

[0110] As an example, taking the current frame as the 3rd frame, the rate-distortion cost of the current block on the 3rd frame can be expressed as:

[0111] J1=D3+λR3(6)

[0112] In the case of considering the inter-frame propagation effect, the rate-distortion cost of the current block on the 3rd frame can be expressed as:

[0113]

[0114] wherein R3 is the code rate, and λ is the code rate coefficient.

[0115] According to the exemplary embodiments of the present disclosure, based on the first rate-distortion cost and the second rate-distortion cost, obtaining the quantization parameter increment of the current block can comprise: deriving the first rate-distortion cost and the second rate-distortion cost with respect to the code rate to obtain a first derivative result and a second derivative result; obtaining a propagation influence factor of the current block based on the second rate-distortion cost; and obtaining the quantization parameter increment of the current block based on the first derivative result, the second derivative result and the propagation influence factor.

[0116] Through the present embodiment, by deriving the rate-distortion cost with respect to the code rate, the quantization parameter increment function containing the second subsequent frame information can be quickly and conveniently obtained, and the propagation influence factor containing the second subsequent frame information can also be quickly and conveniently obtained through the second rate-distortion cost, and then the quantization parameter increment containing the second subsequent frame information can be obtained according to the two, that is, the second subsequent frame is used to correct the quantization parameter increment.

[0117] As an example, still taking the current frame as the 0th frame, the rate-distortion cost J1 and J2 are derived with respect to the code rate,

[0118] The derivative result of J1 with respect to the code rate R0 is:

[0119]

[0120] The derivative of J2 with respect to the code rate R0 is:

[0121]

[0122] wherein,

[0123] The derivative of J2 is converted to obtain:

[0124]

[0125] The above derived formulas (8) and (10) of the rate-distortion cost J1 and J2 are solved to obtain the code rate coefficient corresponding to J2:

[0126]

[0127] Further, the J2 equivalent equation J3=D0+λ is obtained. opt R0, that is, the optimization target changes from J2 to J3.

[0128] As an example, according to the above two derivatives of the rate-distortion cost, that is, formula (8) and formula (9), the quantization parameter function considering the subsequent frame can be obtained, and further combined with the initial quantization parameter function, the corresponding quantization parameter increment function can be further obtained. After obtaining the quantization parameter increment function, the propagation influence factor can be determined according to the influence factor in the second rate-distortion cost, and further based on the quantization parameter increment function and the propagation influence factor, the quantization parameter increment of the current block can be obtained.

[0129] According to an example embodiment of the present disclosure, based on the second rate-distortion cost, the propagation influence factor of the current block can include: determining the influence factor of the current block and the predetermined block respectively through the intra-frame cost and the inter-frame cost of the current block and the predetermined block; and obtaining the propagation influence factor of the current block based on the determined influence factor and the second rate-distortion cost.

[0130] Through the present embodiment, the propagation influence factor is further calculated by using the second rate-distortion cost and combining the intra-frame cost and the inter-frame cost of the block.

[0131] As an example, based on the second rate-distortion cost, the propagation influence factor function can be obtained, wherein the propagation influence factor function contains the influence factor of the reference block on the m-1 frame on the current block on the m frame, for example, according to formula (9) The propagation influence factor function (1+α) can be obtained:

[0132]

[0133] Further,

[0134]

[0135] As an example, taking 5 frames actually available, 6, 7, 8 frames predicted as an example, the derivation process of the propagation influence factor function (12) is as follows:

[0136]

[0137] According to the example embodiment of the present disclosure, based on the first derivative result, the second derivative result and the propagation influence factor, obtaining the quantization parameter increment of the current block can comprise: determining a first quantization parameter based on the first code rate coefficient corresponding to the first rate-distortion cost; obtaining a second code rate coefficient corresponding to a second rate-distortion cost based on the first derivative result, the second derivative result and the first code rate coefficient; determining a second quantization parameter based on the second code rate coefficient; and obtaining the quantization parameter increment based on the first quantization parameter, the second quantization parameter and the propagation influence factor. Through this embodiment, the quantization parameter increment can be obtained more conveniently in combination with the derivative result.

[0138] As an example, since the code rate coefficient λ = C * 2 ^ (QP / 3), the first quantization parameter (QP) can be represented by λ, that is,

[0139] QP = 3log2(λ / C) (14)

[0140] Substituting λ opt in the above formula (11) into formula (14), the second quantization parameter considering inter-frame propagation can be obtained, that is:

[0141] QPnew = 3log2(λ / C(1+α)) (15)

[0142] Further, the quantization parameter increment ΔQP can be represented as:

[0143]

[0144] Where s represents a constant, substituting the value of the propagation influence factor (1+α) into formula (16) can obtain the quantization parameter increment.

[0145] According to the example embodiment of the present disclosure, by the intra-frame cost and the inter-frame cost of the current block and the predetermined block, determining the influence factor of each of the current block and the predetermined block can comprise: for each of the current block and the predetermined block, obtaining the difference value of the intra-frame cost and the inter-frame cost of the block, and taking the difference value and the intra-frame cost of the block as a ratio to obtain the influence factor of the block.

[0146] As an example, in the cuTree, the influence factor k i can be represented as a function of the intra-frame cost (intraCost) and the inter-frame cost (interCost). That is,

[0147] ki = (intracost_i - intercost_i) / intracost_i (17)

[0148] According to the above formula (12), the propagation influence factor 1 + a is composed of the influence factor ki. The corresponding ki is obtained by using the above formula, and the propagation influence factor 1 + a is obtained by inputting the formula (12).

[0149] Return Figure 6 In step S604, the current block is encoded based on the determined quantization parameter increment.

[0150] In summary, the disclosure predicts the propagation of the subsequent frame, that is, the influence of the subsequent frame is predicted by using the obtained information of the first subsequent frame, so as to predict the information of the second subsequent frame, thereby making up for the problem that the number of subsequent frames obtained in the low delay scene is limited, optimizing the quantization parameter increment in cuTree, greatly improving the compression performance of the encoder, and also improving the encoding performance.

[0151] Figure 7 is a block diagram of a video encoding device according to an example embodiment of the disclosure. Referring to Figure 7 The device includes a distortion information determination unit 70, a prediction unit 72, a quantization parameter increment determination unit 74, and an encoding unit 76.

[0152] The distortion information determination unit 70 is configured to determine the distortion information of each block in the current frame and the first subsequent frame, wherein the first subsequent frame is at least one subsequent frame that can be actually obtained after the current frame, and the distortion information of the block is used to indicate the influence of the distortion of the reference block of the previous frame on the distortion of the block; the prediction unit 72 is configured to predict the distortion information of each block in the second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; the quantization parameter increment determination unit 74 is configured to determine the quantization parameter increment of the current block based on the distortion information of the current block in the current frame, and the distortion information of the predetermined block in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is affected by the distortion of the current block; and the encoding unit 76 is configured to encode the current block based on the determined quantization parameter increment.

[0153] According to an example embodiment of the disclosure, the distortion information determination unit 70 is further configured to determine the distortion information of each block in the current frame and the first subsequent frame based on a distortion influence linear model and the distortion information of a predetermined reference block, wherein the predetermined reference block is a block on the zeroth frame in the video to be encoded.

[0154] The prediction unit 72 is further configured to predict the distortion information of each block in the second subsequent frame based on the influence factor of the reference block of the last frame in the first subsequent frame, and the distortion information of each block in the current frame and the first subsequent frame.

[0155] According to an example embodiment of the present disclosure, the quantization parameter increment determination unit 74 is further configured to obtain a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is a rate-distortion cost without considering the inter-frame propagation impact; obtain a second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block, wherein the second rate-distortion cost is a rate-distortion cost considering the inter-frame propagation impact; obtain the quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; and adjust the initial quantization parameter of the current block based on the quantization parameter increment to obtain the quantization parameter of the current block.

[0156] According to an example embodiment of the present disclosure, the quantization parameter increment determination unit 74 is further configured to add the distortion information of the current block and the predetermined block to obtain a total impact of distortion; and determine the second rate-distortion cost based on the total impact of distortion and the code rate.

[0157] According to an example embodiment of the present disclosure, the quantization parameter increment determination unit 74 is further configured to obtain a first derivative result and a second derivative result by respectively deriving the first rate-distortion cost and the second rate-distortion cost with respect to the code rate; obtain a propagation impact factor of the current block based on the second rate-distortion cost; and obtain the quantization parameter increment of the current block based on the first derivative result, the second derivative result, and the propagation impact factor.

[0158] According to an example embodiment of the present disclosure, the quantization parameter increment determination unit 74 is further configured to determine an impact factor of each of the current block and the predetermined block based on the intra-frame cost and the inter-frame cost of the current block and the predetermined block; and obtain a propagation impact factor of the current block based on the determined impact factors and the second rate-distortion cost.

[0159] According to an example embodiment of the present disclosure, the quantization parameter increment determination unit 74 is further configured to determine a first quantization parameter based on a first code rate coefficient corresponding to the first rate-distortion cost; obtain a second code rate coefficient corresponding to the second rate-distortion cost based on the first derivative result, the second derivative result, and the first code rate coefficient; determine a second quantization parameter based on the second code rate coefficient; and obtain a quantization parameter increment function based on the first quantization parameter, the second quantization parameter, and the propagation impact factor.

[0160] According to an example embodiment of the present disclosure, the quantization parameter increment determination unit 74 is further configured to, for each of the current block and the predetermined block, obtain a difference between the intra-frame cost and the inter-frame cost of the block, and obtain an impact factor of the block by taking a ratio of the difference to the intra-frame cost of the block.

[0161] Figure 8A computing environment 810 is shown coupled with a user interface 850. The computing environment 810 can be part of a data processing server. The computing environment 810 includes a processor 820, a memory 830, and an input / output (I / O) interface 840.

[0162] The processor 820 generally controls the overall operation of the computing environment 810, such as operations associated with displaying, data acquisition, data communication, and image processing. The processor 820 can include one or more processors for executing instructions to perform all or some of the steps in the above-described methods. In addition, the processor 820 can include one or more modules that facilitate interaction with other components of the computing environment 810. The processor can be a central processing unit (CPU), a microprocessor, a microcontroller, a graphics processing unit (GPU), etc.

[0163] The memory 830 is configured to store various types of data to support the operation of the computing environment 810. The memory 830 can include predetermined software 832. Examples of such data include instructions for any applications or methods operating on the computing environment 810, video data sets, image data, etc. The memory 830 can be implemented by using any type of volatile or non-volatile memory devices, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0164] The I / O interface 840 provides an interface between the processor 820 and peripheral interface modules (e.g., a keyboard, a click wheel, a button, etc.). The button can include, but is not limited to, a home button, a start scanning button, and a stop scanning button. The I / O interface 840 can be coupled with an encoder and a decoder.

[0165] In an embodiment, a non-transitory computer readable storage medium including, for example, a plurality of programs in the memory 830 and / or storing a bitstream generated by the encoding method described above or a bitstream to be decoded by the decoding method described above, which can be executed by the processor 820 in the computing environment 810 for performing the above-described methods is also provided. In one example, the plurality of programs can be executed by the processor 820 in the computing environment 810 to (e.g., from Figure 2The video encoder 20 in the non-transitory computer-readable storage medium can receive a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) and can also be executed by the processor 820 in the computing environment 810 for performing the above-described decoding method from the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 820 in the computing environment 810 for performing the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or associated one or more syntax elements, etc.) into a bitstream or data stream and can also be executed by the processor 820 in the computing environment 810 for transmitting (e.g., to a decoder, such as the video decoder 30 in the computing environment 810) the bitstream or data stream. Figure 3 The video decoder 30 in the non-transitory computer-readable storage medium can receive a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) and can also be executed by the processor 820 in the computing environment 810 for performing the above-described decoding method from the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 820 in the computing environment 810 for performing the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or associated one or more syntax elements, etc.) into a bitstream or data stream and can also be executed by the processor 820 in the computing environment 810 for transmitting (e.g., to a decoder, such as the video decoder 30 in the computing environment 810) the bitstream or data stream. Figure 2 The video encoder 20 in the non-transitory computer-readable storage medium can receive a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) and can also be executed by the processor 820 in the computing environment 810 for performing the above-described decoding method from the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 820 in the computing environment 810 for performing the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or associated one or more syntax elements, etc.) into a bitstream or data stream and can also be executed by the processor 820 in the computing environment 810 for transmitting (e.g., to a decoder, such as the video decoder 30 in the computing environment 810) the bitstream or data stream. Figure 3 The video decoder 30 in the non-transitory computer-readable storage medium can receive a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) and can also be executed by the processor 820 in the computing environment 810 for performing the above-described decoding method from the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 820 in the computing environment 810 for performing the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or associated one or more syntax elements, etc.) into a bitstream or data stream and can also be executed by the processor 820 in the computing environment 810 for transmitting (e.g., to a decoder, such as the video decoder 30 in the computing environment 810) the bitstream or data stream. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, an optical data storage device, and so on.

[0166] In embodiments, the computing environment 810 can be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described methods.

[0167] According to embodiments of the present disclosure, an electronic device can be provided, which includes at least one memory having stored therein a set of computer-executable instructions, and at least one processor, wherein, when the set of computer-executable instructions is executed by the at least one processor, the video encoding method according to embodiments of the present disclosure is performed.

[0168] As an example, the electronic device can be a PC computer, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-described set of instructions. Here, the electronic device 1000 is not necessarily a single electronic device, but can be a collection of any devices or circuits capable of executing the above-described instructions (or set of instructions) individually or jointly. The electronic device can also be part of an integrated control system or system manager, or can be configured as a portable electronic device that interfaces with local or remote (e.g., via wireless transmission) devices.

[0169] In addition, electronic devices may include video displays (such as liquid crystal displays) and user interaction interfaces (such as keyboards, mice, touch input devices, etc.). All components of the electronic device may be interconnected via buses and / or networks.

[0170] According to embodiments of this disclosure, a computer-readable storage medium may also be provided, wherein when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the at least one processor to perform the video encoding method of the embodiments of this disclosure. Examples of computer-readable storage media herein include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0171] In one embodiment, a method for generating a bitstream is provided, the method comprising a bitstream generated by the video encoding method described above. In another embodiment, encoded video information comprising encoded video information generated by the video encoding method described above is provided.

[0172] According to embodiments of the present disclosure, there is also provided a computer program product having instructions for storing a bitstream including encoded video information generated by the above-mentioned video encoding method. In embodiments, there is also provided a computer program product including a plurality of programs, for example, in the memory 830, which can be executed by the processor 820 in the computing environment 810 for performing the above-mentioned method. For example, the computer program product can include a non-transitory computer-readable storage medium.

[0173] The order of steps of the method according to the present disclosure is only intended to be illustrative, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to the actual situation. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0174] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the description hereof, particularly the specification and practices of the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including other known ins and customary technical practices not specifically disclosed in the present disclosure. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0175] It should be understood that the present disclosure is not limited to the precise structures herein described and illustrated above, and that various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A method of video coding, the method comprising: The method comprises: determining distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame actually available after the current frame, and the distortion information of the block is used to indicate an influence of distortion of a reference block of a previous frame on distortion of the block; predicting distortion information of each block in a second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; determining a quantization parameter increment of a current block in the current frame based on the distortion information of the current block, and the distortion information of a predetermined block in the first subsequent frame and the second subsequent frame, wherein the distortion of the predetermined block is influenced by the distortion of the current block; encoding the current block based on the determined quantization parameter increment.

2. The video coding method of claim 1, wherein, The determining of the distortion information of each block in the current frame and the first subsequent frame comprises: determining the distortion information of each block in the current frame and the first subsequent frame based on a distortion influence linear model and distortion information of a predetermined reference block, wherein the predetermined reference block is a block on a zeroth frame in the video to be encoded. The predicting of the distortion information of each block in the second subsequent frame based on the distortion information of each block comprises: predicting the distortion information of each block in the second subsequent frame based on an influence factor of a reference block of a last frame in the first subsequent frame, and the distortion information of each block in the current frame and the first subsequent frame.

3. The video coding method of claim 1, wherein, The determining of the quantization parameter increment of the current block based on the distortion information of the current block, and the distortion information of the predetermined block in the first subsequent frame and the second subsequent frame comprises: obtaining a first rate-distortion cost of the current block based on the distortion information of the current block, wherein the first rate-distortion cost is a rate-distortion cost without considering an inter-frame propagation influence; obtaining a second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block, wherein the second rate-distortion cost is a rate-distortion cost considering the inter-frame propagation influence; obtaining the quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost; adjusting an initial quantization parameter of the current block based on the quantization parameter increment to obtain a quantization parameter of the current block.

4. The video coding method of claim 3, wherein, The obtaining of the second rate-distortion cost of the current block based on the distortion information of the current block and the predetermined block comprises: adding the distortion information of the current block and the predetermined block to obtain a total influence of distortion; determining the second rate-distortion cost based on the total influence of distortion and a code rate.

5. The video coding method of claim 3, wherein, The obtaining of the quantization parameter increment of the current block based on the first rate-distortion cost and the second rate-distortion cost comprises: deriving the first rate-distortion cost and the second rate-distortion cost with respect to a code rate to obtain a first derivation result and a second derivation result; obtaining a propagation influence factor of the current block based on the second rate-distortion cost; obtaining the quantization parameter increment of the current block based on the first derivation result, the second derivation result and the propagation influence factor.

6. The video coding method of claim 5, wherein, The obtaining of the propagation influence factor of the current block based on the second rate-distortion cost comprises: determine an influence factor of each of the current block and the predetermined block based on the intra cost and the inter cost of the current block and the predetermined block; obtain a propagation influence factor of the current block based on the determined influence factor and the second rate-distortion cost.

7. The video coding method of claim 5, wherein, The obtaining of the quantization parameter increment of the current block based on the first derivative result, the second derivative result and the propagation influence factor comprises: determining a first quantization parameter based on a first rate coefficient corresponding to the first rate-distortion cost; obtaining a second rate coefficient corresponding to a second rate-distortion cost based on the first derivative result, the second derivative result and the first rate coefficient; determining a second quantization parameter based on the second rate coefficient; obtaining the quantization parameter increment based on the first quantization parameter, the second quantization parameter and the propagation influence factor.

8. The video coding method of claim 6, wherein, The determining of the influence factor of each of the current block and the predetermined block based on the intra cost and the inter cost of the current block and the predetermined block comprises: for each of the current block and the predetermined block, obtaining a difference value of the intra cost and the inter cost of the block, and obtaining the influence factor of the block by ratio of the difference value to the intra cost of the block.

9. A video encoding apparatus, comprising: comprise: a distortion information determination unit configured to determine distortion information of each block in a current frame and a first subsequent frame, wherein the first subsequent frame is at least one subsequent frame actually available after the current frame, and the distortion information of the block is used to indicate an influence of distortion of a reference block of a previous frame on distortion of the block; a prediction unit configured to predict distortion information of each block in a second subsequent frame based on the distortion information of each block, wherein the second subsequent frame is a predetermined number of subsequent frames after the first subsequent frame; a quantization parameter increment determination unit configured to determine a quantization parameter increment of a current block in the current frame based on the distortion information of the current block and distortion information of a predetermined block in the first subsequent frame and the second subsequent frame, wherein distortion of the predetermined block is influenced by distortion of the current block; an encoding unit configured to encode the current block based on the determined quantization parameter increment.

10. An electronic device, comprising: comprise: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the video encoding method of any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is caused to perform the video encoding method of any one of claims 1 to 8.

12. A computer program product having instructions for storing a bitstream, characterized in that, The bitstream comprises: video data generated according to the video encoding method of any one of claims 1 to 8.

13. A method of generating a bitstream, the method comprising: comprise: generating a bitstream according to the video encoding method of any one of claims 1 to 8.

Images