Video coding method and device, electronic equipment, storage medium, computer program product and method for generating bit stream

Abstract

The invention relates to a video coding method and device, electronic equipment, a storage medium, a computer program product and a method for generating a bit stream. The video coding method comprises the following steps: acquiring real coding information of a coded coding frame in a video to be coded; the quantization scale and complexity information of parallel frames which are encoded in parallel with the current frame in the video to be encoded are obtained, and the encoding sequence of the current frame is behind the parallel frames; respectively predicting coding information of each parallel frame based on the quantization scale and complexity information of each parallel frame; predicting the quantization scale of the current frame based on the real coding information of the coding frame and the predicted coding information of each parallel frame; and encoding the current frame based on the predicted quantization scale of the current frame.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing, and in particular to a video encoding method and device, an electronic device, a storage medium, a computer program product, and a method for generating a bit stream. Background Art

[0002] In order to make the bit rate of the encoded output stream reach the target bit rate, the traditional video encoding ABR (Average Bitrate) rate control algorithm (single-frame serial encoding) often needs to use the actual encoding information obtained after encoding the previous frame to update the rate control parameters and adjust the QP (Quantization Parameters) of subsequent frames.

[0003] At present, in real-time video transmission scenarios such as live video broadcasting and remote conferencing, in order to ensure that the end-to-end transmission delay is within a controllable range, video encoders often enable multi-frame parallel encoding to increase the encoding speed. However, after enabling multi-frame parallel encoding, the real encoding information of the previous frame in the encoding sequence cannot be obtained when encoding the current frame. Therefore, compared with single-frame serial encoding, multi-frame parallel encoding cannot adjust the encoding QP of subsequent frames in time, resulting in increased bit rate control errors and reduced encoding performance. Summary of the invention

[0004] The present disclosure provides a video encoding method and apparatus, an electronic device, a storage medium, a computer program product, and a method for generating a bit stream, so as to at least solve the problems of high rate control error and low encoding performance in multi-frame parallel encoding in the related art.

[0005] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, comprising: obtaining real encoding information of an encoded frame in a video to be encoded; obtaining quantization scale and complexity information of a parallel frame encoded in parallel with a current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frame; based on the quantization scale and complexity information of each parallel frame, respectively predicting the encoding information of each parallel frame; based on the real encoding information of the encoded frame and the predicted encoding information of each parallel frame, predicting the quantization scale of the current frame; and encoding the current frame based on the predicted quantization scale of the current frame.

[0006] Optionally, based on the quantization scale and complexity information of each parallel frame, the coding information of each parallel frame is predicted respectively, including: for each parallel frame, the following processing is performed: based on the quantization scale, complexity information and prediction parameters of the current parallel frame, the number of coding bits of the current parallel frame is predicted; based on the predicted number of coding bits and complexity information of the current parallel frame, the coding information of the current parallel frame is predicted.

[0007] Optionally, the complexity information includes inter-frame complexity information and smoothing complexity information; based on the quantization scale, complexity information and prediction parameters of the current parallel frame, predicting the number of coding bits of the current parallel frame, including: based on the quantization scale, inter-frame complexity information and prediction parameters of the current parallel frame, predicting the number of coding bits of the current parallel frame; based on the predicted number of coding bits and complexity information of the current parallel frame, predicting the coding information of the current parallel frame, including: based on the predicted number of coding bits, smoothing complexity information and quantization scale of the current parallel frame, predicting the coding information of the current parallel frame.

[0008] Optionally, the prediction parameters include a first prediction parameter and a second prediction parameter, and based on the quantization scale of the current parallel frame, inter-frame complexity information and the prediction parameters, predict the number of coding bits of the current parallel frame, including: adjusting the inter-frame complexity information of the current parallel frame based on the first prediction parameter and the second prediction parameter, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information and the second prediction parameter is the intercept coefficient of the inter-frame complexity information; and predicting the number of coding bits of the current parallel frame based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame.

[0009] Optionally, after encoding the current frame based on the predicted quantization scale of the current frame, the method further includes: updating prediction parameters based on the actual number of encoding bits, the actual quantization scale and inter-frame complexity information of the current frame.

[0010] Optionally, after encoding the current frame based on the predicted quantization scale of the current frame, it also includes: updating the first prediction parameter based on the actual number of coding bits of the current frame, the actual quantization scale, the inter-frame complexity information and the second prediction parameter; updating the second prediction parameter based on the actual number of coding bits of the current frame, the actual quantization scale, the inter-frame complexity information and the updated first prediction parameter.

[0011] Optionally, based on the real coding information of the coded frame and the predicted coding information of each parallel frame, the quantization scale of the current frame is predicted, including: determining a rate factor based on the real coding information of the coded frame and the predicted coding information of each parallel frame; and predicting the quantization scale of the current frame based on the rate factor.

[0012] According to a second aspect of an embodiment of the present disclosure, a video encoding device is provided, comprising: a first acquisition unit, configured to acquire real encoding information of an encoded frame in a video to be encoded; a second acquisition unit, configured to acquire quantization scale and complexity information of a parallel frame encoded in parallel with a current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frame; a first prediction unit, configured to predict the encoding information of each parallel frame respectively based on the quantization scale and complexity information of each parallel frame; a second prediction unit, configured to predict the quantization scale of the current frame based on the real encoding information of the encoded frame and the predicted encoding information of each parallel frame; and an encoding unit, configured to encode the current frame based on the predicted quantization scale of the current frame.

[0013] Optionally, the first prediction unit is further configured to perform the following processing for each parallel frame: predicting the number of coding bits of the current parallel frame based on the quantization scale, complexity information and prediction parameters of the current parallel frame; predicting the coding information of the current parallel frame based on the predicted number of coding bits and complexity information of the current parallel frame.

[0014] Optionally, the complexity information includes inter-frame complexity information and smoothing complexity information; the first prediction unit is also configured to predict the number of coding bits of the current parallel frame based on the quantization scale, inter-frame complexity information and prediction parameters of the current parallel frame; and predict the coding information of the current parallel frame based on the predicted number of coding bits, smoothing complexity information and quantization scale of the current parallel frame.

[0015] Optionally, the prediction parameters include a first prediction parameter and a second prediction parameter, and the first prediction unit is further configured to adjust the inter-frame complexity information of the current parallel frame based on the first prediction parameter and the second prediction parameter, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information and the second prediction parameter is the intercept coefficient of the inter-frame complexity information; based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame, predict the number of coding bits of the current parallel frame.

[0016] Optionally, the updating unit is configured to update the prediction parameters based on the actual number of coding bits, the actual quantization scale and inter-frame complexity information of the current frame after encoding the current frame based on the predicted quantization scale of the current frame.

[0017] Optionally, the updating unit is further configured to, after encoding the current frame based on the predicted quantization scale of the current frame, update the first prediction parameter based on the actual number of coding bits, the actual quantization scale, the inter-frame complexity information and the second prediction parameter of the current frame; and update the second prediction parameter based on the actual number of coding bits, the actual quantization scale, the inter-frame complexity information and the updated first prediction parameter of the current frame.

[0018] Optionally, the second prediction unit is further configured to determine a rate factor based on real coding information of the coded frame and predicted coding information of each parallel frame; and predict a quantization scale of the current frame based on the rate factor.

[0019] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video encoding method as described above.

[0020] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the video encoding method as described above.

[0021] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising instructions for storing a bit stream, wherein the bit stream comprises: video data generated according to the video encoding method as described above.

[0022] According to a sixth aspect of an embodiment of the present disclosure, a method for generating a bit stream is provided, comprising: generating a bit stream according to the video encoding method as described above.

[0023] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0024] According to the video encoding method and device, electronic device, storage medium, computer program product and method for generating bitstream disclosed in the present invention, based on the quantization scale and complexity information of each parallel frame, the encoding information of each parallel frame can be predicted, so that when predicting the quantization size of the current frame, on the basis of considering the real encoding information of the frames that have been encoded in the video to be encoded, the predicted encoding information of each parallel frame can also be considered, that is, the predicted encoding information of several frames that have not been encoded before the current frame can also be considered, so as to timely adjust the encoding QP of the subsequent frames, and avoid the problem of increased rate control error and decreased encoding performance caused by multi-frame parallel encoding. Therefore, the present invention solves the problem of high rate control error and low encoding performance of multi-frame parallel encoding in the related art.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0027] Figure 1 is a block diagram of an exemplary system for encoding and decoding video blocks according to an exemplary embodiment of the present disclosure;

[0028] Figure 2 is a block diagram of a video encoder according to an exemplary embodiment of the present disclosure;

[0029] Figure 3 is a block diagram of a video decoder according to an exemplary embodiment of the present disclosure;

[0030] Figure 4 is a schematic diagram showing single-threaded encoding and multi-threaded encoding according to an exemplary embodiment of the present disclosure;

[0031] Figure 5 is a flowchart of a video encoding method according to an exemplary embodiment of the present disclosure;

[0032] Figure 6 is a schematic diagram of a rate control process according to an exemplary embodiment of the present disclosure;

[0033] Figure 7 is a block diagram of a video encoding device according to an exemplary embodiment of the present disclosure;

[0034] Figure 8 is a diagram of a computing environment coupled with a user interface according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the drawings are used to distinguish objects, but not to describe any specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0037] Figure 1 1 is a block diagram of an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0038] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0039] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0040] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0041] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0042] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0043] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0044] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0045] Figure 2 A block diagram of an exemplary video encoder 20 according to some embodiments described in the present application is shown. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0046] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0047] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0048] like Figure 2 As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0049] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0050] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0051] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0052] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0053] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0054] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0055] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0056] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0057] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0058] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0059] Figure 3 A block diagram of an exemplary video decoder 30 according to some embodiments of the present application is shown. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned operations in combination with the above-mentioned operations. Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0060] In some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0061] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0062] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0063] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0064] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0065] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0066] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0067] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0068] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0069] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0070] At present, the traditional video coding ABR (Average Bitrate, that is, average bit rate) rate control algorithm (single frame serial coding) is as follows Figure 4 In order to make the bit rate of the encoded output stream reach the target bit rate, it is often necessary to use the real encoding information obtained after encoding the previous frame to update the bit rate control parameters and adjust the QP (Quantization Parameters) of the subsequent frames. In real-time video transmission scenarios such as live video broadcasting and remote conferencing, in order to ensure that the end-to-end transmission delay is within a controllable range, video encoders often enable multi-frame parallel encoding to increase the encoding speed, such as Figure 4 As shown in the figure, multi-threaded parallel encoding is performed. However, after multi-frame parallel encoding is turned on, in the ABR rate control algorithm mode, when the i-th frame begins to be encoded, since the i-N+1 to i-1 frames are also being encoded, the encoder can only obtain the real encoding information after the iN-th frame is encoded (N is the number of parallel threads) to update the rate control model, which means that the real encoding information of the previous frame in the encoding sequence cannot be obtained when encoding the current frame. Therefore, compared with single-frame serial encoding, multi-frame parallel encoding cannot adjust the encoding QP of subsequent frames in time, resulting in an increase in the rate control error and a decrease in encoding performance.

[0071] In order to overcome the above problems, the present invention can predict the coding information of each parallel frame based on the quantization scale and complexity information of each parallel frame. Therefore, when predicting the quantization size of the current frame, on the basis of considering the actual coding information of the frames that have been encoded in the video to be encoded, the predicted coding information of each parallel frame can also be considered, that is, the predicted coding information of several frames that have not been encoded before the current frame can also be considered, so as to timely adjust the coding QP of subsequent frames to avoid the problem of increased bit rate control error and decreased coding performance caused by parallel encoding of multiple frames.

[0072] Hereinafter, a video encoding method and apparatus, an electronic device, a storage medium, a computer program product, and a method of generating a bitstream according to exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0073] Figure 5 is a flowchart of a video encoding method according to an exemplary embodiment of the present disclosure, such as Figure 5 As shown, the video encoding method includes the following steps:

[0074] In step S501, real encoding information of encoded frames in the video to be encoded is obtained.

[0075] As an example, the above real coding information can be obtained based on the real coding bit number, quantization scale and smoothing complexity information of the encoded coding frame. In order to facilitate understanding of the above real coding information, the following briefly introduces the ABR bit rate control process of traditional video coding:

[0076] like Figure 6 As shown, the process can be divided into the following three steps: 1) Calculate the smoothing complexity of each frame (BlurCplx i ); 2) Calculate the smoothing complexity rate factor (Rate_factor) of each frame; 3) Calculate the quantization scale (Qscale) of each frame based on the smoothing complexity obtained in step 1) and the rate factor in step 2). i ).

[0077] 1) According to the inter-frame complexity (SATD) obtained during pre-analysis of the i-th frame i ), the inter-frame complexity of the i-th frame and the cumulative inter-frame complexity of the i-1-th frame are weighted by exponential smoothing method to obtain the cumulative inter-frame complexity of the i-th frame (CplxSum i ), and according to the number of frames that have started to be encoded, the cumulative number of frames corresponding to the i-th frame (CplxCount i ), the specific calculation method is as follows:

[0078]

[0079] After obtaining the above information, the smoothing complexity of the i-th frame can be calculated based on the cumulative inter-frame complexity and the cumulative number of frames of the i-th frame:

[0080]

[0081] 2) Rate_factor here needs to be calculated based on the target bit rate and the number of coded bits of the coded frame. The specific calculation method can be as follows:

[0082]

[0083] Among them, fps represents the set frame rate, Bitrate Target Represents the target bit rate, WantedBitsWindow represents the average target encoding bits per frame calculated based on the target bit rate, bits i-1 is the actual number of coded bits for the i-1th frame, Qscale i-1 is the quantization scale (Qscale) used for the i-1th frame, and qCompress represents the rate control model parameter. It should be noted that the denominator here is actually the real coding information determined based on the real coding bits, quantization scale and smoothing complexity of the coded frame. If the total real coding bits are greater than the average target coding bits, it is equivalent to increasing the Qscale of the current frame, and the expected reduction of the coding bits of the current frame; conversely, if the total real coding bits are less than the average target coding bits, it is equivalent to lowering the Qscale of the current frame to increase the coding bits of the current frame.

[0084] 3) After obtaining the smoothing complexity and bit rate factor of the i-th frame, the Qscale of the current frame can be calculated. The specific calculation formula can be as follows:

[0085]

[0086] Wherein, qCompress represents a rate control model parameter, and its default value is 0.6. It should be noted that the parameter can be set as required, and the setting of 0.6 in the present disclosure is only an example.

[0087] Therefore, the actual coding information of the coded frames in the video to be coded in this embodiment may be as shown in the denominator in the above formula (4), which is not limited in the present disclosure.

[0088] From the above ABR rate control process, it can be seen that the encoding information of historical frames plays a very important role in the rate control model, which greatly affects the accuracy of rate control. However, when N-frame parallel encoding is enabled, when encoding the i-th frame, the i-1-th frame is often also encoded in parallel, and the actual number of encoded bits after encoding the i-1-th frame cannot be obtained. Therefore, in order to ensure the consistency of multiple encodings, when encoding the i-th frame, only the actual number of encoded bits of the iN-th frame and the frames before it can be used to calculate the Rate_factor. The calculation formula can be as follows:

[0089]

[0090] Therefore, the calculation delay of Rate_factor in multi-frame parallel encoding will lead to a decrease in the accuracy of rate control, and the more frames are encoded in parallel, the more serious the decrease in rate control accuracy will be, which will also lead to a decrease in encoding efficiency. In order to avoid the calculation delay of Rate_factor in multi-frame parallel encoding leading to a decrease in the accuracy of rate control, the present disclosure predicts the encoding information of parallel frames.

[0091] return Figure 5 In step S502, the quantization scale and complexity information of the parallel frame encoded in parallel with the current frame in the video to be encoded are obtained, wherein the encoding order of the current frame is after the parallel frame.

[0092] As an example, the above complexity information may be obtained during pre-analysis, and the complexity information may include but is not limited to: inter-frame complexity information and smoothing complexity information. The calculation methods of these two types of complexity information have been introduced above and will not be discussed in detail here.

[0093] As an example, the quantization scale of the above parallel frame may be predicted in the same manner as the quantization scale of the current frame.

[0094] return Figure 5 In step S503, based on the quantization scale and complexity information of each parallel frame, the coding information of each parallel frame is predicted respectively.

[0095] As an example, the number of coding bits of each parallel frame may be predicted based on the quantization scale and complexity information, and then the coding information of the corresponding parallel frame may be predicted based on the predicted number of coding bits and complexity information.

[0096] According to an exemplary embodiment of the present disclosure, based on the quantization scale and complexity information of each parallel frame, respectively predicting the coding information of each parallel frame may include: for each parallel frame, performing the following processing: based on the quantization scale, complexity information and prediction parameters of the current parallel frame, predicting the number of coding bits of the current parallel frame; based on the predicted number of coding bits and complexity information of the current parallel frame, predicting the coding information of the current parallel frame. Through this embodiment, the number of coding bits can be conveniently and quickly predicted through a model containing prediction parameters, so that the coding information of the parallel frames can be conveniently and quickly predicted.

[0097] As an example, the above-mentioned number of coded bits can be obtained using a prediction model, that is, the quantization scale and complexity information are input into the prediction model to obtain the coded bit integer of the corresponding parallel frame. The above-mentioned prediction model can be the R-lambda model in the HEVC (High Efficiency Video Codin) reference software, or other applicable models, which are not limited by the present disclosure.

[0098] As an example, after obtaining the predicted number of coding bits, the coding information of the corresponding parallel frame is predicted based on the number of coding bits and complexity information. For example, for each frame k in the parallel frames i-N+1 to i-1, the predicted coding information can be obtained by the following formula, which is not limited in the present disclosure:

[0099]

[0100] in, is the actual number of coded bits for the kth frame, Qscale k is the quantization scale (Qscale) used for the kth frame, qCompress represents the rate control model parameters, BlurCplx k Represents the smoothness complexity of the kth frame.

[0101] According to an exemplary embodiment of the present disclosure, complexity information may include inter-frame complexity information and smoothing complexity information; predicting the number of coding bits of the current parallel frame based on the quantization scale, complexity information and prediction parameters of the current parallel frame may include: predicting the number of coding bits of the current parallel frame based on the quantization scale, inter-frame complexity information and prediction parameters of the current parallel frame; predicting the coding information of the current parallel frame based on the predicted number of coding bits and complexity information of the current parallel frame may include: predicting the coding information of the current parallel frame based on the predicted number of coding bits, smoothing complexity information and quantization scale of the current parallel frame.

[0102] As an example, after N frames are encoded in parallel, when encoding the i-th frame (i.e., the current frame), although only the actual number of encoded bits of the iN-th frame and its previous frames can be obtained, the inter-frame complexity information and smoothing complexity information obtained during the pre-analysis of each frame in the i-N+1 to i-1 frames can be obtained, because these values ​​need to be calculated before the corresponding frame starts encoding. At the same time, the predicted quantization scale of each frame in the i-N+1 to i-1 frames, that is, the quantization scale of the parallel frames, can also be obtained, so that the present disclosure can use these values ​​to predict the number of encoded bits of the i-N+1 to i-1 frames, and the prediction process can be shown in the following formula:

[0103]

[0104] Wherein, coeff represents the first prediction parameter, which is the prediction parameter of the prediction model, and the initial value can be 1.0, which is not limited in the present disclosure. It should be noted that the method of predicting the number of coding bits is not limited to formula (8), and any method that can predict the number of coding bits relatively accurately can be applied in the present disclosure.

[0105] As an example, the above prediction parameters are not limited to the first prediction parameters, and may also include other parameters, which is not limited in the present disclosure.

[0106] According to an exemplary embodiment of the present disclosure, after encoding the current frame based on the predicted quantization scale of the current frame, the prediction parameters can also be updated based on the actual number of coded bits, the actual quantization scale, and the inter-frame complexity information of the current frame. Through this embodiment, the prediction parameters are updated in real time to make them more consistent with the actual scene, so as to predict a relatively accurate estimated number of coded bits.

[0107] As an example, after encoding the i-th frame (i.e., the current frame), the parameters of the prediction model can be updated according to the Qscale used in the actual encoding and the actual number of bits obtained, that is, the first prediction parameter and the second prediction parameter are updated. The specific updating method can be as follows;

[0108]

[0109] It should be noted that the updating method of the present disclosure is not limited to the above formula, and other updating methods that can achieve corresponding effects can also be applied to the present disclosure.

[0110] According to an exemplary embodiment of the present disclosure, the above-mentioned prediction parameters may also include a first prediction parameter and a second prediction parameter, and based on the quantization scale of the current parallel frame, the inter-frame complexity information and the prediction parameters, predicting the number of coded bits of the current parallel frame may include: adjusting the inter-frame complexity information of the current parallel frame based on the first prediction parameter and the second prediction parameter, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information, and the second prediction parameter is the intercept coefficient of the inter-frame complexity information; predicting the number of coded bits of the current parallel frame based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame. Through this embodiment, the slope coefficient and the intercept coefficient are used to adjust the inter-frame complexity information, and based on the adjusted inter-frame complexity information and the quantization scale, relatively accurate coding information of the parallel frame can be predicted.

[0111] As an example, after N frames are encoded in parallel, when encoding the i-th frame (i.e., the current frame), although only the actual number of encoded bits of the iN-th frame and its previous frames can be obtained, the inter-frame complexity information and smoothing complexity information obtained during the pre-analysis of each frame in the i-N+1 to i-1 frames can be obtained, because these values ​​need to be calculated before the corresponding frame starts encoding. At the same time, the predicted quantization scale of each frame in the i-N+1 to i-1 frames can also be obtained, so the present disclosure can use these values ​​to predict the number of encoded bits of the i-N+1 to i-1 frames, and the prediction process can be shown in the following formula:

[0112]

[0113] Wherein, coeff represents the first prediction parameter, and offset represents the second prediction parameter, both of which are prediction parameters of the prediction model, and the initial values ​​can be 1.0 and 0.15, which are not limited in the present disclosure. It should be noted that the method of predicting the number of coding bits is not limited to formula (10), and any method that can predict the number of coding bits relatively accurately can be applied in the present disclosure.

[0114] According to an exemplary embodiment of the present disclosure, after encoding the current frame based on the predicted quantization scale of the current frame, the first prediction parameter can also be updated based on the actual number of coded bits, the actual quantization scale, the inter-frame complexity information, and the second prediction parameter of the current frame; the second prediction parameter can be updated based on the actual number of coded bits, the actual quantization scale, the inter-frame complexity information, and the updated first prediction parameter of the current frame. Through this embodiment, the first prediction parameter and the second prediction parameter are updated in real time to make them more consistent with the actual scenario, so as to predict a relatively accurate estimated number of coded bits.

[0115] As an example, after encoding the i-th frame (i.e., the current frame), the parameters of the prediction model can be updated according to the Qscale used in the actual encoding and the actual number of bits obtained, that is, the first prediction parameter and the second prediction parameter are updated. The specific updating method can be as follows;

[0116]

[0117] It should be noted that the updating method of the present disclosure is not limited to the above formula, and other updating methods that can achieve corresponding effects can also be applied to the present disclosure.

[0118] In step S504, the quantization scale of the current frame is predicted based on the actual coding information of the coded frame and the predicted coding information of each parallel frame.

[0119] As an example, after obtaining the real coding information of the coded frame and the predicted coding information of each parallel frame, the quantization scale of the current frame can be further predicted based on information such as the target bit rate.

[0120] According to an exemplary embodiment of the present disclosure, predicting the quantization scale of the current frame based on the real coding information of the coded frame and the predicted coding information of each parallel frame may include: determining a rate factor based on the real coding information of the coded frame and the predicted coding information of each parallel frame; and predicting the quantization scale of the current frame based on the rate factor.

[0121] As an example, after obtaining the predicted coding bit numbers of frames i-N+1 to i-1, the Qscale of the i-th frame (i.e., the current frame) can be calculated by combining the actual coding bit numbers of the iN-th frame and its previous frames. The calculation method can be as follows:

[0122]

[0123] The denominator in the formula consists of two parts. The first part is the real coding information of the iNth frame and its previous frames, that is, the real coding information of the coded frames; the second part is the predicted coding information of the i-N+1th to i-1th frames, that is, the sum of the predicted coding information of each parallel frame. Compared with the related technical solutions, the solution proposed in the present disclosure uses the estimated coding information to compensate for the missing coding information caused by the parallelization of multiple frames to a certain extent, thereby improving the accuracy of rate control and coding efficiency.

[0124] In step S505, the current frame is encoded based on the predicted quantization scale of the current frame.

[0125] As an example, after the predicted quantization scale of the current frame is obtained, the quantization parameter can be adjusted according to the predicted quantization scale, and then the current frame is quantized and subsequently processed.

[0126] The present invention discloses a multi-thread-friendly video encoding method. After enabling multi-frame parallel encoding, the number of encoding bits of each parallel frame can be predicted according to the quantization scale of the parallel frames being encoded in parallel and the complexity obtained during pre-analysis. The predicted number of encoding bits can be used to compensate for the lack of encoding information caused by multi-frame parallelism to a certain extent, that is, the input of the rate control model can be updated in time, and the influence of the rate control model update delay introduced by multi-threading on the rate control accuracy and encoding performance can be reduced, thereby improving the accuracy of rate control and encoding efficiency.

[0127] The present disclosure can achieve a BD-rate PSNR of about -1.023% and a BD-rate SSIM of about -0.814% on a K265 mobile encoder, and the bit rate control accuracy is improved by about 2.32%. The encoding time is almost unchanged. rate) represents the bit rate difference between two encoding algorithms under the same quality evaluation index, and a negative number represents bit rate saving; PSNR (Peak signal-to-noise ratio) represents peak signal-to-noise ratio; SSIM (Structural Similarity) represents structural similarity measurement.

[0128] Figure 7 is a block diagram of a video encoding device according to an exemplary embodiment of the present disclosure. Figure 7 The device includes a first acquisition unit 70, a second acquisition unit 72, a first prediction unit 74, a second prediction unit 76 and an encoding unit 78.

[0129] The first acquisition unit 70 is configured to obtain the real coding information of the encoded frame in the video to be encoded; the second acquisition unit 72 is configured to obtain the quantization scale and complexity information of the parallel frame encoded in parallel with the current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frame; the first prediction unit 74 is configured to predict the coding information of each parallel frame based on the quantization scale and complexity information of each parallel frame; the second prediction unit 76 is configured to predict the quantization scale of the current frame based on the real coding information of the encoded frame and the predicted coding information of each parallel frame; the encoding unit 78 is configured to encode the current frame based on the predicted quantization scale of the current frame.

[0130] According to an exemplary embodiment of the present disclosure, the first prediction unit 74 is also configured to perform the following processing for each parallel frame: predicting the number of coding bits of the current parallel frame based on the quantization scale, complexity information and prediction parameters of the current parallel frame; predicting the coding information of the current parallel frame based on the predicted number of coding bits and complexity information of the current parallel frame.

[0131] According to an exemplary embodiment of the present disclosure, the complexity information includes inter-frame complexity information and smoothing complexity information; the first prediction unit 74 is also configured to predict the number of coding bits of the current parallel frame based on the quantization scale of the current parallel frame, the inter-frame complexity information and the prediction parameters: based on the predicted number of coding bits of the current parallel frame, the smoothing complexity information and the quantization scale, predict the coding information of the current parallel frame.

[0132] According to an exemplary embodiment of the present disclosure, the prediction parameters include a first prediction parameter and a second prediction parameter, and the first prediction unit 74 is further configured to adjust the inter-frame complexity information of the current parallel frame based on the first prediction parameter and the second prediction parameter, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information, and the second prediction parameter is the intercept coefficient of the inter-frame complexity information; based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame, predict the number of coding bits of the current parallel frame.

[0133] According to an exemplary embodiment of the present disclosure, the updating unit is configured to update the prediction parameters based on the actual coding bit number, the actual quantization scale and the inter-frame complexity information of the current frame after encoding the current frame based on the predicted quantization scale of the current frame.

[0134] According to an exemplary embodiment of the present disclosure, the updating unit is further configured to, after encoding the current frame based on the predicted quantization scale of the current frame, update the first prediction parameter based on the actual number of coding bits, the actual quantization scale, the inter-frame complexity information and the second prediction parameter of the current frame; and update the second prediction parameter based on the actual number of coding bits, the actual quantization scale, the inter-frame complexity information and the updated first prediction parameter of the current frame.

[0135] According to an exemplary embodiment of the present disclosure, the second prediction unit 76 is further configured to determine a rate factor based on real coding information of the coded frame and predicted coding information of each parallel frame; and predict a quantization scale of the current frame based on the rate factor.

[0136] Figure 8 A computing environment 810 is shown coupled to a user interface 850. The computing environment 810 may be part of a data processing server. The computing environment 810 includes a processor 820, a memory 830, and an input / output (I / O) interface 840.

[0137] The processor 820 generally controls the overall operation of the computing environment 810, such as operations associated with display, data acquisition, data communication, and image processing. The processor 820 may include one or more processors for executing instructions to perform all or some steps in the above method. In addition, the processor 820 may include one or more modules that facilitate the interaction between the processor 820 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0138] The memory 830 is configured to store various types of data to support the operation of the computing environment 810. The memory 830 may include predetermined software 832. Examples of such data include instructions for any application or method operating on the computing environment 810, video data sets, image data, etc. The memory 830 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0139] The I / O interface 840 provides an interface between the processor 820 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 840 may be coupled to an encoder and a decoder.

[0140] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 830 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 820 in a computing environment 810 to perform the above method. In one example, the plurality of programs can be executed by a processor 820 in a computing environment 810 to (for example, from Figure 2 The video encoder 20 in the computing environment 810 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 820 in the computing environment 810 to perform the above-mentioned decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 820 in the computing environment 810 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 820 in the computing environment 810 to (e.g., to Figure 3Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0141] In an embodiment, the computing environment 810 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0142] According to an embodiment of the present disclosure, an electronic device may be provided, which includes at least one memory and at least one processor, wherein a set of computer executable instructions is stored in the at least one memory, and when the set of computer executable instructions is executed by the at least one processor, a video encoding method according to an embodiment of the present disclosure is executed.

[0143] As an example, the electronic device can be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device 1000 is not necessarily a single electronic device, but can also be any device or circuit that can execute the above-mentioned instructions (or instruction sets) individually or jointly. The electronic device can also be part of an integrated control system or system manager, or can be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0144] In addition, the electronic device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0145] According to an embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein when the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the video encoding method of the embodiment of the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0146] In an embodiment, a method for generating a bitstream is provided, the method comprising the bitstream generated by the above video encoding method. In an embodiment, a method for generating a bitstream is provided, the method comprising encoded video information generated by the above video encoding method.

[0147] According to an embodiment of the present disclosure, a computer program product having instructions for storing a bitstream is also provided, wherein the bitstream includes encoded video information generated by the above-mentioned video encoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 830 is also provided, wherein the plurality of programs can be executed by a processor 820 in a computing environment 810 to perform the above-mentioned method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0148] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0149] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0150] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that: include: Obtaining real encoding information of encoded frames in the video to be encoded; Obtaining quantization scale and complexity information of a parallel frame encoded in parallel with the current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frame; Based on the quantization scale and complexity information of each parallel frame, respectively predict the coding information of each parallel frame; Predicting a quantization scale of the current frame based on real coding information of the coding frame and predicted coding information of each parallel frame; The current frame is encoded based on the predicted quantization scale of the current frame.

2. The video encoding method according to claim 1, characterized in that: The predicting the encoding information of each parallel frame based on the quantization scale and complexity information of each parallel frame respectively includes: For each parallel frame, the following processing is performed: Predicting the number of coded bits of the current parallel frame based on the quantization scale, complexity information and prediction parameters of the current parallel frame; Based on the predicted coding bit number of the current parallel frame and the complexity information, coding information of the current parallel frame is predicted.

3. The video encoding method according to claim 2, characterized in that: The complexity information includes inter-frame complexity information and smoothing complexity information; The predicting the number of coded bits of the current parallel frame based on the quantization scale, complexity information and prediction parameters of the current parallel frame includes: Predicting the number of coding bits of the current parallel frame based on the quantization scale of the current parallel frame, the inter-frame complexity information and the prediction parameter; The predicting the coding information of the current parallel frame based on the predicted coding bit number of the current parallel frame and the complexity information includes: The coding information of the current parallel frame is predicted based on the predicted coding bit number, smoothing complexity information and quantization scale of the current parallel frame.

4. The video encoding method according to claim 3, characterized in that: The prediction parameters include a first prediction parameter and a second prediction parameter. The predicting the number of coding bits of the current parallel frame based on the quantization scale of the current parallel frame, the inter-frame complexity information and the prediction parameter includes: Adjusting the inter-frame complexity information of the current parallel frame based on the first prediction parameter and the second prediction parameter, wherein the first prediction parameter is a slope coefficient of the inter-frame complexity information, and the second prediction parameter is an intercept coefficient of the inter-frame complexity information; The number of coding bits of the current parallel frame is predicted based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame.

5. The video encoding method according to claim 2, wherein: After encoding the current frame based on the predicted quantization scale of the current frame, the method further includes: The prediction parameters are updated based on the actual number of coded bits, the actual quantization scale, and inter-frame complexity information of the current frame.

6. The video encoding method according to claim 4, characterized in that: After encoding the current frame based on the predicted quantization scale of the current frame, the method further includes: Update the first prediction parameter based on the actual number of coded bits, the actual quantization scale, the inter-frame complexity information and the second prediction parameter of the current frame; The second prediction parameter is updated based on the actual number of coding bits, the actual quantization scale, the inter-frame complexity information and the updated first prediction parameter of the current frame.

7. The video encoding method according to claim 1, wherein: The predicting the quantization scale of the current frame based on the real coding information of the coding frame and the predicted coding information of each parallel frame comprises: Determining a rate factor based on the actual coding information of the coded frame and the predicted coding information of each parallel frame; Based on the rate factor, a quantization scale of the current frame is predicted.

8. A video encoding device, characterized in that: include: A first acquisition unit is configured to acquire real encoding information of an encoded frame in the video to be encoded; A second acquisition unit is configured to acquire quantization scale and complexity information of a parallel frame encoded in parallel with the current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frame; A first prediction unit is configured to predict encoding information of each parallel frame based on a quantization scale and complexity information of each parallel frame; A second prediction unit is configured to predict a quantization scale of the current frame based on real coding information of the coding frame and predicted coding information of each parallel frame; The encoding unit is configured to encode the current frame based on the predicted quantization scale of the current frame.

9. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, Wherein, when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to perform the video encoding method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the video encoding method according to any one of claims 1 to 7.

11. A computer program product having instructions for storing a bit stream, characterized in that The bitstream includes: Video data generated according to the video encoding method according to any one of claims 1 to 7.

12. A method for generating a bit stream, characterized in that: include: A bit stream is generated according to the video encoding method according to any one of claims 1 to 7.

Images