Video encoding method and device, electronic device, storage medium, computer program product and method for generating a bitstream

Abstract

The present disclosure relates to a video encoding method and device, electronic equipment, storage medium, computer program product and a method of generating a bitstream. The video encoding method comprises: obtaining real encoding information of an encoded frame in a to-be-encoded video; obtaining quantization scales and complexity information of parallel frames in the to-be-encoded video which are encoded in parallel with a current frame, wherein the encoding order of the current frame is after the parallel frames; predicting encoding information of each parallel frame based on the quantization scales and complexity information of each parallel frame; predicting a quantization scale of the current frame based on the real encoding information of the encoded frame and the predicted encoding information of each parallel frame; and encoding the current frame based on the predicted quantization scale of the current frame.

Description

Technical Field

[0001] This disclosure relates to the field of video processing, and more particularly to a video encoding method and apparatus, electronic device, storage medium, computer program product, and method for generating bitstreams. Background Technology

[0002] In order to achieve the target bitrate, the ABR (Average Bitrate) bitrate control algorithm (single-frame serial coding) of traditional video coding often requires using the actual coding information obtained from the previous frame to update the bitrate control parameters and adjust the QP (Quantization Parameters) of subsequent frames.

[0003] Currently, in real-time video transmission scenarios such as live streaming and remote conferencing, video encoders often enable multi-frame parallel encoding to improve encoding speed in order to ensure that end-to-end transmission latency is within a controllable range. However, when multi-frame parallel encoding is enabled, the actual encoded information of the previous frame in the encoding sequence cannot be obtained when encoding the current frame. Therefore, compared with single-frame serial encoding, multi-frame parallel encoding cannot adjust the encoding QP of subsequent frames in a timely manner, resulting in increased bitrate control error and decreased encoding performance. Summary of the Invention

[0004] This disclosure provides a video encoding method and apparatus, electronic device, storage medium, computer program product, and method for generating bitstreams, to at least solve the problems of high bitrate control error and low encoding performance in multi-frame parallel encoding in related technologies.

[0005] According to a first aspect of the present disclosure, a video coding method is provided, comprising: acquiring real coding information of encoded frames already encoded in a video to be encoded; acquiring quantization scale and complexity information of parallel frames encoded in parallel with the current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frames; predicting coding information of each parallel frame based on the quantization scale and complexity information of each parallel frame; predicting the quantization scale of the current frame based on the real coding information of the encoded frames and the predicted coding information of each parallel frame; and encoding the current frame based on the predicted quantization scale of the current frame.

[0006] Optionally, based on the quantization scale and complexity information of each parallel frame, the encoding information of each parallel frame is predicted, including: for each parallel frame, the following processing is performed: based on the quantization scale, complexity information and prediction parameters of the current parallel frame, the number of encoded bits of the current parallel frame is predicted; based on the predicted number of encoded bits and complexity information of the current parallel frame, the encoding information of the current parallel frame is predicted.

[0007] Optionally, the complexity information includes inter-frame complexity information and smoothing complexity information; based on the quantization scale, complexity information, and prediction parameters of the current parallel frame, the number of encoded bits of the current parallel frame is predicted, including: based on the quantization scale, inter-frame complexity information, and prediction parameters of the current parallel frame, the number of encoded bits of the current parallel frame is predicted; based on the predicted number of encoded bits and complexity information of the current parallel frame, the encoding information of the current parallel frame is predicted, including: based on the predicted number of encoded bits, smoothing complexity information, and quantization scale of the current parallel frame, the encoding information of the current parallel frame is predicted.

[0008] Optionally, the prediction parameters include a first prediction parameter and a second prediction parameter. Based on the quantization scale, inter-frame complexity information, and prediction parameters of the current parallel frame, the prediction of the number of encoded bits of the current parallel frame includes: adjusting the inter-frame complexity information of the current parallel frame based on the first prediction parameter and the second prediction parameter, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information, and the second prediction parameter is the intercept coefficient of the inter-frame complexity information; and predicting the number of encoded bits of the current parallel frame based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame.

[0009] Optionally, after encoding the current frame based on the predicted quantization scale of the current frame, the method further includes updating the prediction parameters based on the actual number of encoded bits, the actual quantization scale, and the inter-frame complexity information of the current frame.

[0010] Optionally, after encoding the current frame based on the predicted quantization scale of the current frame, the method further includes: updating the first prediction parameter based on the actual number of encoded bits, the actual quantization scale, the inter-frame complexity information, and the second prediction parameter of the current frame; and updating the second prediction parameter based on the actual number of encoded bits, the actual quantization scale, the inter-frame complexity information, and the updated first prediction parameter of the current frame.

[0011] Optionally, based on the actual coding information of the coded frame and the predicted coding information of each parallel frame, the quantization scale of the current frame is predicted, including: determining the bit rate factor based on the actual coding information of the coded frame and the predicted coding information of each parallel frame; and predicting the quantization scale of the current frame based on the bit rate factor.

[0012] According to a second aspect of the present disclosure, a video encoding apparatus is provided, comprising: a first acquisition unit configured to acquire real encoding information of encoded frames already encoded in a video to be encoded; a second acquisition unit configured to acquire quantization scale and complexity information of parallel frames encoded in parallel with the current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frames; a first prediction unit configured to predict encoding information of each parallel frame based on the quantization scale and complexity information of each parallel frame; a second prediction unit configured to predict the quantization scale of the current frame based on the real encoding information of the encoded frames and the predicted encoding information of each parallel frame; and an encoding unit configured to encode the current frame based on the predicted quantization scale of the current frame.

[0013] Optionally, the first prediction unit is further configured to perform the following processing for each parallel frame: predict the number of encoded bits of the current parallel frame based on the quantization scale, complexity information and prediction parameters of the current parallel frame; and predict the encoding information of the current parallel frame based on the predicted number of encoded bits and complexity information of the current parallel frame.

[0014] Optionally, the complexity information includes inter-frame complexity information and smoothing complexity information; the first prediction unit is further configured to predict the number of encoded bits of the current parallel frame based on the quantization scale, inter-frame complexity information and prediction parameters of the current parallel frame; and to predict the encoding information of the current parallel frame based on the predicted number of encoded bits, smoothing complexity information and quantization scale of the current parallel frame.

[0015] Optionally, the prediction parameters include a first prediction parameter and a second prediction parameter. The first prediction unit is further configured to adjust the inter-frame complexity information of the current parallel frame based on the first prediction parameter and the second prediction parameter, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information, and the second prediction parameter is the intercept coefficient of the inter-frame complexity information; and to predict the number of encoded bits of the current parallel frame based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame.

[0016] Optionally, the update unit is configured to update the prediction parameters based on the actual number of encoded bits, the actual quantization scale, and inter-frame complexity information of the current frame after encoding the current frame based on the prediction quantization scale of the current frame.

[0017] Optionally, the updating unit is further configured to, after encoding the current frame based on the predicted quantization scale of the current frame, update the first prediction parameter based on the actual number of encoded bits, the actual quantization scale, the inter-frame complexity information, and the second prediction parameter of the current frame; and update the second prediction parameter based on the actual number of encoded bits, the actual quantization scale, the inter-frame complexity information, and the updated first prediction parameter of the current frame.

[0018] Optionally, the second prediction unit is also configured to determine the bit rate factor based on the real coding information of the coded frame and the predicted coding information of each parallel frame; and to predict the quantization scale of the current frame based on the bit rate factor.

[0019] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the video encoding method as described above.

[0020] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by at least one processor, causes at least one processor to perform the video encoding method as described above.

[0021] According to a fifth aspect of the present disclosure, a computer program product is provided, including instructions for storing a bitstream, the bitstream comprising: video data generated according to the video encoding method described above.

[0022] According to a sixth aspect of the present disclosure, a method for generating a bitstream is provided, comprising: generating a bitstream according to the video encoding method described above.

[0023] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0024] According to the video coding method, apparatus, electronic device, storage medium, computer program product, and bitstream generation method disclosed herein, based on the quantization scale and complexity information of each parallel frame, the coding information of each parallel frame can be predicted. Therefore, when predicting the quantization size of the current frame, in addition to considering the actual coding information of the frames already encoded in the video to be encoded, the predicted coding information of each parallel frame can also be considered. That is, the predicted coding information of several frames before the current frame that have not yet been encoded can also be considered, thereby allowing timely adjustment of the coding QP of subsequent frames and avoiding the problems of increased rate control error and decreased coding performance caused by multi-frame parallel coding. Therefore, this disclosure solves the problems of high rate control error and low coding performance in multi-frame parallel coding in related technologies.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0027] Figure 1 This is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to exemplary embodiments of the present disclosure;

[0028] Figure 2 This is a block diagram of a video encoder illustrated according to exemplary embodiments of the present disclosure;

[0029] Figure 3 This is a block diagram of a video decoder illustrated according to exemplary embodiments of the present disclosure;

[0030] Figure 4 This is a schematic diagram illustrating single-threaded encoding and multi-threaded encoding according to exemplary embodiments of the present disclosure;

[0031] Figure 5 This is a flowchart illustrating a video encoding method according to exemplary embodiments of the present disclosure;

[0032] Figure 6 This is a schematic diagram illustrating a rate control process according to exemplary embodiments of the present disclosure;

[0033] Figure 7 This is a block diagram of a video encoding apparatus illustrated according to exemplary embodiments of the present disclosure;

[0034] Figure 8 This is a diagram illustrating a computing environment coupled to a user interface according to exemplary embodiments of the present disclosure. Detailed Implementation

[0035] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this disclosure are used to distinguish objects and are not used to describe any specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in sequences other than those shown in the drawings or described in this disclosure.

[0037] Figure 1 A block diagram of an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure, is shown. Figure 1As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0038] In some implementations, the target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14.

[0039] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the target device 14 can access the encoded video data in the storage device 32 via the input interface 28.

[0040] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0041] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback.

[0042] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0043] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0044] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0045] Figure 2 A block diagram of an exemplary video encoder 20 according to some embodiments described in this application is shown. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0046] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or it can be distributed among one or more of the fixed or programmable hardware units described.

[0047] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... Figure 1 The video source 18 shown obtains video data from the video data storage 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 (e.g., in intra-frame or inter-frame predictive coding mode) when encoding the video data.

[0048] like Figure 2 As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., sets of video blocks) or other larger coding units (CUs) according to a predefined splitting structure (e.g., a quadtree (QT) structure) associated with the video data. It should be noted that the term "block" or "video block" as used herein can be a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring to, for example, HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0049] The prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0050] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0051] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block within the current video frame or picture relative to a prediction block within a reference frame associated with the current block being encoded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0052] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0053] Intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, during individual encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) can select a suitable intra-prediction mode from the tested intra-prediction modes for use. Intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to entropy coding unit 56. Entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0054] After prediction processing unit 41 determines the prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0055] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0056] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1 The video decoder 30 shown, or archived in, for example Figure 1 The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame being encoded.

[0057] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0058] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0059] Figure 3 A block diagram of an exemplary video decoder 30 according to some embodiments of this application is shown. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame prediction (BC) unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0060] In some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame prediction (BC) unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30 (e.g., motion compensation unit 82, intra-frame prediction unit 84, and entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame prediction (BC) unit 85, and the functionality of the intra-frame prediction (BC) unit 85 may be performed by other components of the prediction processing unit 81 (e.g., motion compensation unit 82).

[0061] Video data storage 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of video decoder 30. The video data stored in video data storage 79 can be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0062] During the decoding process, the video decoder 30 receives a encoded video bitstream representing video blocks and associated syntax elements of encoded video frames. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0063] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0064] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0065] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within a reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0066] The motion compensation unit 82 and / or the intra-frame BC unit 85 determine the prediction information for the video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0067] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0068] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0069] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be directly provided to the DPB 92 by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., ...). Figure 1 On the display device 34).

[0070] Currently, traditional video coding ABR (Average Bitrate) rate control algorithms (single-frame serial coding) such as Figure 4 The single-threaded serial encoding shown often requires updating the bitrate control parameters using the actual encoded information from the previous frame to adjust the QP (Quantization Parameters) of subsequent frames in order to achieve the target bitrate in the encoded output stream. However, in real-time video transmission scenarios such as live streaming and remote conferencing, video encoders often employ multi-frame parallel encoding to improve encoding speed and ensure that end-to-end transmission latency remains within a controllable range. Figure 4 The diagram illustrates multi-threaded parallel encoding. However, with multi-frame parallel encoding enabled, in the ABR rate control algorithm mode, when encoding begins at the start of the i-th frame, since frames i-N+1 to i-1 are also in progress, the encoder can only obtain the actual encoded information of the iN-th frame (N is the number of parallel threads) to update the rate control model. This means that the actual encoded information of the previous frame in the encoded sequence cannot be obtained when encoding the current frame. Therefore, compared to single-frame serial encoding, multi-frame parallel encoding cannot adjust the encoding QP of subsequent frames in a timely manner, leading to increased rate control error and decreased encoding performance.

[0071] To overcome the above problems, this disclosure can predict the coding information of each parallel frame based on the quantization scale and complexity information of each parallel frame. Thus, when predicting the quantization size of the current frame, in addition to considering the actual coding information of the frames that have been encoded in the video to be encoded, it can also consider the predicted coding information of each parallel frame, that is, it can also consider the predicted coding information of several frames that have not yet been encoded before the current frame. This allows for timely adjustment of the coding QP of subsequent frames, avoiding the problems of increased bitrate control error and decreased coding performance caused by multi-frame parallel coding.

[0072] The video encoding method and apparatus, electronic device, storage medium, computer program product, and method for generating bitstream according to exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0073] Figure 5 This is a flowchart illustrating a video encoding method according to exemplary embodiments of the present disclosure, such as... Figure 5 As shown, the video encoding method includes the following steps:

[0074] In step S501, the actual encoding information of the encoded frames in the video to be encoded is obtained.

[0075] As an example, the aforementioned actual encoding information can be obtained based on the actual number of encoded bits, quantization scale, and smoothing complexity information of the encoded frames. To facilitate understanding of the aforementioned actual encoding information, the traditional ABR rate control process for video coding is briefly introduced below:

[0076] like Figure 6 As shown, the process can be divided into the following three steps: 1) Calculate the smoothing complexity (BlurCplx) for each frame. i 1) Calculate the smoothing complexity rate factor (Rate_factor) for each frame; 2) Calculate the quantization scale (Qscale) for each frame based on the smoothing complexity obtained in step 1) and the rate factor obtained in step 2). i ).

[0077] 1) The inter-frame complexity (SATD) obtained during pre-analysis of the i-th frame can be used as a basis. i The inter-frame complexity of frame i is weighted by exponential smoothing and the cumulative inter-frame complexity of frame i-1, resulting in the cumulative inter-frame complexity (CplxSum) of frame i. i And based on the number of frames that have already started encoding, calculate the cumulative frame count (CplxCount) corresponding to the i-th frame. i The specific calculation method is as follows:

[0078]

[0079] After obtaining the above information, the smooth complexity of the i-th frame can be calculated based on the cumulative inter-frame complexity and the cumulative number of frames:

[0080]

[0081] 2) The Rate_factor here needs to be calculated based on the target bitrate and the number of encoded bits in the encoded frame. The specific calculation method is as follows:

[0082]

[0083] Where fps represents the set frame rate, and Bitrate Target The target bitrate is represented by `WantedBitsWindow`, which represents the average number of target encoded bits per frame calculated based on the target bitrate. i-1 Qscale represents the actual number of encoded bits in the (i-1)th frame. i-1 Here, q is the quantization scale (Qscale) used for the (i-1)th frame, and qCompress represents the bitrate control model parameters. It's important to note that the denominator here is actually the actual encoded information determined based on the actual number of encoded bits, quantization scale, and smoothing complexity of the encoded frames. If the total actual number of encoded bits is greater than the average target number of encoded bits, it's equivalent to increasing the Qscale of the current frame, which is expected to reduce the number of encoded bits in the current frame; conversely, if the total actual number of encoded bits is less than the average target number of encoded bits, it's equivalent to decreasing the Qscale of the current frame, which is expected to increase the number of encoded bits in the current frame.

[0084] 3) After obtaining the smoothing complexity and bitrate factor of the i-th frame, the Qscale of the current frame can be calculated. The specific calculation formula is as follows:

[0085]

[0086] Here, qCompress represents the bitrate control model parameter, with a default value of 0.6. It should be noted that this parameter can be set as needed; the setting of 0.6 in this disclosure is merely an example.

[0087] Therefore, the actual encoding information of the encoded frames in the video to be encoded in this embodiment can be as shown in the denominator of the above formula (4), and this disclosure does not limit this.

[0088] As can be seen from the ABR rate control process described above, the encoding information of historical frames plays a crucial role in the rate control model, significantly impacting the accuracy of rate control. However, when enabling N-frame parallel encoding, the (i-1)th frame is often being encoded in parallel while the i-th frame is being encoded, making it impossible to obtain the actual encoded bit count of the (i-1)th frame. Therefore, to ensure consistency across multiple encoding iterations, the Rate_factor can only be calculated using the actual encoded bit count of the i-th frame and the frames preceding it when encoding the i-th frame. The calculation formula is as follows:

[0089]

[0090] Therefore, the computational delay of Rate_factor in multi-frame parallel coding leads to a decrease in the accuracy of rate control. Moreover, the more frames encoded in parallel, the more severe the decrease in rate control accuracy, which in turn leads to a reduction in coding efficiency. To avoid the decrease in rate control accuracy caused by the computational delay of Rate_factor in multi-frame parallel coding, this disclosure predicts the coding information of parallel frames.

[0091] return Figure 5 In step S502, the quantization scale and complexity information of the parallel frames encoded in parallel with the current frame in the video to be encoded are obtained, wherein the encoding order of the current frame is after the parallel frames.

[0092] As an example, the above complexity information can be obtained during pre-analysis, and this complexity information may include, but is not limited to, inter-frame complexity information and smoothness complexity information. The calculation methods for these two types of complexity information have been introduced above and will not be discussed further here.

[0093] As an example, the quantization scale of the above parallel frames can be predicted in the same way as the quantization scale of the current frame.

[0094] return Figure 5 In step S503, the encoding information of each parallel frame is predicted based on the quantization scale and complexity information of each parallel frame.

[0095] As an example, the number of encoded bits for each parallel frame can be predicted first based on quantization scale and complexity information, and then the encoding information of the corresponding parallel frame can be predicted based on the predicted number of encoded bits and complexity information.

[0096] According to an exemplary embodiment of this disclosure, predicting the coding information of each parallel frame based on the quantization scale and complexity information of each parallel frame may include: for each parallel frame, performing the following processing: predicting the number of coded bits of the current parallel frame based on the quantization scale, complexity information, and prediction parameters; and predicting the coding information of the current parallel frame based on the predicted number of coded bits and complexity information. Through this embodiment, the number of coded bits can be conveniently and quickly predicted using a model containing prediction parameters, thereby enabling convenient and rapid prediction of the coding information of parallel frames.

[0097] As an example, the aforementioned number of encoded bits can be obtained using a prediction model, which involves inputting the quantization scale and complexity information into the prediction model to obtain the integer number of encoded bits for the corresponding parallel frame. The prediction model can be the R-lambda model in the HEVC (High Efficiency Video Coding) reference software, or other applicable models; this disclosure does not limit its application.

[0098] As an example, after obtaining the predicted number of encoded bits, the encoding information of the corresponding parallel frame is predicted based on the number of encoded bits and complexity information. For example, for each frame k in parallel frames i-N+1 to i-1, the predicted encoding information can be obtained by the following formula. This disclosure does not limit the scope of the prediction.

[0099]

[0100] in, Qscale represents the actual number of encoded bits in the k-th frame. k The quantization scale (Qscale) used for the k-th frame, qCompress represents the bitrate control model parameter, and BlurCplx... k This represents the smoothness complexity of the k-th frame.

[0101] According to an exemplary embodiment of this disclosure, the complexity information may include inter-frame complexity information and smoothing complexity information; predicting the number of encoded bits of the current parallel frame based on the quantization scale, complexity information, and prediction parameters may include: predicting the number of encoded bits of the current parallel frame based on the quantization scale, inter-frame complexity information, and prediction parameters; predicting the encoding information of the current parallel frame based on the predicted number of encoded bits and complexity information may include: predicting the encoding information of the current parallel frame based on the predicted number of encoded bits, smoothing complexity information, and quantization scale.

[0102] As an example, after enabling N-frame parallel encoding, when encoding the i-th frame (i.e., the current frame), although only the actual number of encoded bits for the i-N-th frame and the frames preceding it can be obtained, the inter-frame complexity and smoothing complexity information obtained during the pre-analysis of each frame from i-N+1 to i-1 can be obtained. This is because these values ​​need to be calculated before the corresponding frame begins encoding. Simultaneously, the predictive quantization scale for each frame from i-N+1 to i-1, i.e., the quantization scale for parallel frames, can also be obtained. Therefore, this disclosure can use these values ​​to predict the number of encoded bits for frames from i-N+1 to i-1. The prediction process can be represented by the following formula:

[0103]

[0104] Wherein, coeff represents the first prediction parameter, which is the prediction parameter of the prediction model. The initial value can be 1.0, and this disclosure does not limit it. It should be noted that the method of predicting the number of coded bits is not limited to formula (8). Any method that can predict the number of coded bits with relative accuracy can be applied to this disclosure.

[0105] As an example, the above prediction parameters are not limited to the first prediction parameter, and may include other parameters, which are not limited in this disclosure.

[0106] According to an exemplary embodiment of this disclosure, after encoding the current frame based on the predicted quantization scale of the current frame, the prediction parameters can be updated based on the actual number of encoded bits, the actual quantization scale, and inter-frame complexity information of the current frame. This embodiment updates the prediction parameters in real time, making them more consistent with the actual scenario, so as to predict a relatively accurate number of estimated encoded bits.

[0107] As an example, after encoding the i-th frame (i.e. the current frame), the parameters of the prediction model can be updated according to the Qscale used in the actual encoding and the number of bits obtained in the actual ratio, that is, the first prediction parameter and the second prediction parameter can be updated. The specific update method is as follows;

[0108]

[0109] It should be noted that the update method disclosed herein is not limited to the above formula, and other update methods that can achieve the same effect can also be applied to this disclosure.

[0110] According to an exemplary embodiment of this disclosure, the prediction parameters may further include a first prediction parameter and a second prediction parameter. Predicting the number of encoded bits in the current parallel frame based on the quantization scale, inter-frame complexity information, and prediction parameters may include: adjusting the inter-frame complexity information of the current parallel frame based on the first and second prediction parameters, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information, and the second prediction parameter is the intercept coefficient of the inter-frame complexity information; and predicting the number of encoded bits in the current parallel frame based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame. Through this embodiment, by using the slope coefficient and intercept coefficient to adjust the inter-frame complexity information, and based on the adjusted inter-frame complexity information and quantization scale, relatively accurate encoding information for the parallel frame can be predicted.

[0111] As an example, after enabling N-frame parallel encoding, when encoding the i-th frame (i.e., the current frame), although only the actual number of encoded bits for the i-N-th frame and its preceding frames can be obtained, the inter-frame complexity and smoothing complexity information obtained during the pre-analysis of each frame from i-N+1 to i-1 can be obtained. This is because these values ​​need to be calculated before the corresponding frame begins encoding. Simultaneously, the predicted quantization scale for each frame from i-N+1 to i-1 can also be obtained. Therefore, this disclosure can use these values ​​to predict the number of encoded bits for frames from i-N+1 to i-1. The prediction process can be represented by the following formula:

[0112]

[0113] Wherein, coeff represents the first prediction parameter and offset represents the second prediction parameter. Both are prediction parameters of the prediction model, and their initial values ​​can be 1.0 and 0.15, respectively. This disclosure does not limit these values. It should be noted that the method for predicting the number of encoded bits is not limited to formula (10). Any method that can predict a relatively accurate number of encoded bits can be applied to this disclosure.

[0114] According to an exemplary embodiment of this disclosure, after encoding the current frame based on the predicted quantization scale of the current frame, the first prediction parameter can be updated based on the actual number of encoded bits, the actual quantization scale, inter-frame complexity information, and the second prediction parameter of the current frame; the second prediction parameter is updated based on the actual number of encoded bits, the actual quantization scale, the inter-frame complexity information, and the updated first prediction parameter of the current frame. Through this embodiment, the first and second prediction parameters are updated in real time, making them more consistent with the actual scenario, so as to predict a relatively accurate estimated number of encoded bits.

[0115] As an example, after encoding the i-th frame (i.e. the current frame), the parameters of the prediction model can be updated according to the Qscale used in the actual encoding and the number of bits obtained in the actual ratio, that is, the first prediction parameter and the second prediction parameter can be updated. The specific update method is as follows;

[0116]

[0117] It should be noted that the update method disclosed herein is not limited to the above formula, and other update methods that can achieve the same effect can also be applied to this disclosure.

[0118] In step S504, the quantization scale of the current frame is predicted based on the actual coding information of the coded frame and the predicted coding information of each parallel frame.

[0119] As an example, after obtaining the actual coding information of the encoded frame and the predicted coding information of each parallel frame, the quantization scale of the current frame can be further predicted based on information such as the target bit rate.

[0120] According to an exemplary embodiment of this disclosure, predicting the quantization scale of the current frame based on the actual coding information of the coded frame and the predicted coding information of each parallel frame may include: determining a bitrate factor based on the actual coding information of the coded frame and the predicted coding information of each parallel frame; and predicting the quantization scale of the current frame based on the bitrate factor.

[0121] As an example, after obtaining the predicted coding bit count of frames i-N+1 to i-1, the Qscale of the i-th frame (i.e., the current frame) can be calculated by combining the actual coding bit count of the i-th frame and the frames preceding it. The calculation method is as follows:

[0122]

[0123] In this formula, the denominator consists of two parts: the first part is the actual coding information of the i-th frame and the frames preceding it, i.e., the actual coding information of the already coded frames; the second part is the predicted coding information of the (i-N+1)th to (i-1)th frames, i.e., the sum of the predicted coding information of each parallel frame. Compared with related technical solutions, the scheme proposed in this disclosure utilizes the predicted coding information to a certain extent to compensate for the lack of coding information caused by multi-frame parallelism, thereby improving the accuracy of bitrate control and coding efficiency.

[0124] In step S505, the current frame is encoded based on the predicted quantization scale of the current frame.

[0125] As an example, after obtaining the predicted quantization scale of the current frame, the quantization parameters can be adjusted using the predicted quantization scale, thereby performing quantization processing and subsequent processing on the current frame.

[0126] This disclosure proposes a multi-threaded friendly video coding method. After enabling multi-frame parallel coding, the number of encoded bits for each parallel frame can be predicted based on the quantization scale of the parallel frames being encoded and the complexity obtained during pre-analysis. This predicted number of encoded bits can, to some extent, compensate for the lack of coding information caused by multi-frame parallelism, thereby updating the input of the rate control model in a timely manner. This reduces the impact of the rate control model update delay introduced by multi-threading on the accuracy of rate control and coding performance, thus improving the accuracy of rate control and coding efficiency.

[0127] This disclosure achieves a BD-rate PSNR of approximately -1.023% and a BD-rate SSIM of approximately -0.814% on the K265 mobile encoder, with a rate control accuracy improvement of approximately 2.32%. Furthermore, the encoding time remains almost unchanged. Among these, the BD-rate ( (rate) represents the difference in bit rate between two coding algorithms under the same quality evaluation metric, with negative numbers indicating bit rate savings; PSNR (Peak signal-to-noise ratio) represents the peak signal-to-noise ratio; SSIM (Structural Similarity) represents the structural similarity metric.

[0128] Figure 7 This is a block diagram of a video encoding apparatus illustrated according to exemplary embodiments of the present disclosure. (Refer to...) Figure 7 The device includes a first acquisition unit 70, a second acquisition unit 72, a first prediction unit 74, a second prediction unit 76, and an encoding unit 78.

[0129] The first acquisition unit 70 is configured to acquire the actual encoding information of the encoded frames already encoded in the video to be encoded; the second acquisition unit 72 is configured to acquire the quantization scale and complexity information of the parallel frames encoded in parallel with the current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frames; the first prediction unit 74 is configured to predict the encoding information of each parallel frame based on the quantization scale and complexity information of each parallel frame; the second prediction unit 76 is configured to predict the quantization scale of the current frame based on the actual encoding information of the encoded frame and the predicted encoding information of each parallel frame; and the encoding unit 78 is configured to encode the current frame based on the predicted quantization scale of the current frame.

[0130] According to an exemplary embodiment of the present disclosure, the first prediction unit 74 is further configured to perform the following processing for each parallel frame: predict the number of encoded bits of the current parallel frame based on the quantization scale, complexity information and prediction parameters of the current parallel frame; and predict the encoding information of the current parallel frame based on the predicted number of encoded bits and complexity information of the current parallel frame.

[0131] According to an exemplary embodiment of the present disclosure, the complexity information includes inter-frame complexity information and smoothing complexity information; the first prediction unit 74 is further configured to predict the number of encoded bits of the current parallel frame based on the quantization scale of the current parallel frame, the inter-frame complexity information and the prediction parameters; and to predict the encoding information of the current parallel frame based on the predicted number of encoded bits of the current parallel frame, the smoothing complexity information and the quantization scale.

[0132] According to an exemplary embodiment of this disclosure, the prediction parameters include a first prediction parameter and a second prediction parameter. The first prediction unit 74 is further configured to adjust the inter-frame complexity information of the current parallel frame based on the first prediction parameter and the second prediction parameter, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information and the second prediction parameter is the intercept coefficient of the inter-frame complexity information; and to predict the number of encoded bits of the current parallel frame based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame.

[0133] According to an exemplary embodiment of the present disclosure, the updating unit is configured to update the prediction parameters based on the actual number of encoded bits, the actual quantization scale, and inter-frame complexity information of the current frame after encoding the current frame based on the prediction quantization scale of the current frame.

[0134] According to an exemplary embodiment of this disclosure, the updating unit is further configured to, after encoding the current frame based on the predicted quantization scale of the current frame, update the first prediction parameter based on the actual number of encoded bits, the actual quantization scale, inter-frame complexity information, and the second prediction parameter of the current frame; and update the second prediction parameter based on the actual number of encoded bits, the actual quantization scale, the inter-frame complexity information, and the updated first prediction parameter of the current frame.

[0135] According to an exemplary embodiment of this disclosure, the second prediction unit 76 is further configured to determine a bitrate factor based on the actual coding information of the coded frame and the predicted coding information of each parallel frame; and to predict the quantization scale of the current frame based on the bitrate factor.

[0136] Figure 8 A computing environment 810 coupled to a user interface 850 is shown. The computing environment 810 may be part of a data processing server. The computing environment 810 includes a processor 820, a memory 830, and an input / output (I / O) interface 840.

[0137] Processor 820 typically controls the overall operation of computing environment 810, such as operations associated with display, data acquisition, data communication, and image processing. Processor 820 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 820 may include one or more modules that facilitate interaction between processor 820 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0138] Memory 830 is configured to store various types of data to support the operation of computing environment 810. Memory 830 may include predefined software 832. Examples of such data include instructions for any application or method operating on computing environment 810, video datasets, image data, etc. Memory 830 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0139] I / O interface 840 provides an interface between processor 820 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 840 can be coupled to encoders and decoders.

[0140] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a plurality of programs in a memory 830 and / or a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by a processor 820 in a computing environment 810 to perform the above-described methods. In one example, the plurality of programs can be executed by a processor 820 in a computing environment 810 to (e.g., from...) Figure 2 The video encoder 20 in the computing environment 810 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 820 in the computing environment 810 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 820 in the computing environment 810 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 820 in the computing environment 810 to (e.g., to...) Figure 3The video decoder 30 in the middle sends the bit stream or data stream. Alternatively, a non-transitory computer-readable storage medium may store data generated by the encoder (e.g., Figure 2 The video encoder 20 in the video is generated using, for example, the encoding method described above, for use by the decoder (e.g., Figure 3 The video decoder 30 in the video decoder uses a bitstream or data stream that includes encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) when decoding video data. Non-transitory computer-readable storage media can be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage devices, etc.

[0141] In an embodiment, the computing environment 810 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0142] According to embodiments of the present disclosure, an electronic device may be provided, the electronic device including at least one memory and at least one processor, wherein the at least one memory stores a set of computer-executable instructions, and when the set of computer-executable instructions is executed by the at least one processor, a video encoding method according to embodiments of the present disclosure is performed.

[0143] As an example, the electronic device can be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, the electronic device 1000 is not necessarily a single electronic device, but can be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. The electronic device can also be part of an integrated control system or system manager, or can be configured to interconnect locally or remotely (e.g., via wireless transmission) via an interface.

[0144] In addition, electronic devices may include video displays (such as liquid crystal displays) and user interaction interfaces (such as keyboards, mice, touch input devices, etc.). All components of the electronic device may be interconnected via buses and / or networks.

[0145] According to embodiments of this disclosure, a computer-readable storage medium may also be provided, wherein when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the at least one processor to perform the video encoding method of the embodiments of this disclosure. Examples of computer-readable storage media herein include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0146] In one embodiment, a method for generating a bitstream is provided, the method comprising a bitstream generated by the video encoding method described above. In another embodiment, encoded video information comprising encoded video information generated by the video encoding method described above is provided.

[0147] According to embodiments of this disclosure, a computer program product having instructions for storing a bitstream, the bitstream including encoded video information generated by the video encoding method described above, is also provided. In embodiments, a computer program product comprising, for example, a plurality of programs stored in a memory 830, the plurality of programs being executable by a processor 820 in a computing environment 810 to perform the methods described above, is also provided. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0148] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0149] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0150] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that, include: Obtain the actual encoding information of the encoded frames in the video to be encoded, wherein the actual encoding information is determined based on the actual number of encoded bits, quantization scale, and smoothing complexity of the encoded frames. Obtain the quantization scale and complexity information of the parallel frames encoded in parallel with the current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frames; Based on the quantization scale and complexity information of each parallel frame, the encoding information of each parallel frame is predicted. Based on the actual coding information of the coded frame and the predicted coding information of each parallel frame, the quantization scale of the current frame is predicted; The current frame is encoded based on the predicted quantization scale of the current frame; The step of predicting the quantization scale of the current frame based on the actual coding information of the coded frame and the predicted coding information of each parallel frame includes: determining the bitrate factor based on the actual coding information of the coded frame and the predicted coding information of each parallel frame; and predicting the quantization scale of the current frame based on the bitrate factor.

2. The video encoding method as described in claim 1, characterized in that, The prediction of the encoding information for each parallel frame based on the quantization scale and complexity information of each parallel frame includes: For each parallel frame, perform the following processing: Based on the quantization scale, complexity information, and prediction parameters of the current parallel frame, predict the number of encoded bits of the current parallel frame; Based on the predicted number of encoded bits of the current parallel frame and the complexity information, predict the encoding information of the current parallel frame.

3. The video encoding method as described in claim 2, characterized in that, The complexity information includes inter-frame complexity information and smoothness complexity information; The prediction of the number of encoded bits for the current parallel frame based on the quantization scale, complexity information, and prediction parameters of the current parallel frame includes: Based on the quantization scale, inter-frame complexity information, and prediction parameters of the current parallel frame, predict the number of encoded bits of the current parallel frame; The prediction of the encoding information of the current parallel frame based on the predicted number of encoded bits and the complexity information includes: Based on the predicted number of coding bits, smoothness complexity information, and quantization scale of the current parallel frame, predict the coding information of the current parallel frame.

4. The video encoding method as described in claim 3, characterized in that, The prediction parameters include a first prediction parameter and a second prediction parameter. The prediction of the number of encoded bits for the current parallel frame based on the quantization scale, inter-frame complexity information, and prediction parameters includes: Based on the first prediction parameter and the second prediction parameter, the inter-frame complexity information of the current parallel frame is adjusted, wherein the first prediction parameter is the slope coefficient of the inter-frame complexity information, and the second prediction parameter is the intercept coefficient of the inter-frame complexity information. The number of encoded bits for the current parallel frame is predicted based on the ratio of the adjusted inter-frame complexity information to the quantization scale of the current parallel frame.

5. The video encoding method as described in claim 2, characterized in that, After encoding the current frame based on the predictive quantization scale of the current frame, the method further includes: The prediction parameters are updated based on the actual number of encoded bits, the actual quantization scale, and the inter-frame complexity information of the current frame.

6. The video encoding method as described in claim 4, characterized in that, After encoding the current frame based on the predictive quantization scale of the current frame, the method further includes: Based on the actual number of encoded bits, actual quantization scale, inter-frame complexity information, and the second prediction parameter of the current frame, update the first prediction parameter; The second prediction parameter is updated based on the actual number of encoded bits, the actual quantization scale, the inter-frame complexity information, and the updated first prediction parameter of the current frame.

7. A video encoding device, characterized in that, include: The first acquisition unit is configured to acquire the real encoding information of the encoded frames in the video to be encoded, wherein the real encoding information is determined based on the real number of encoded bits, quantization scale and smoothing complexity of the encoded frames. The second acquisition unit is configured to acquire the quantization scale and complexity information of the parallel frames encoded in parallel with the current frame in the video to be encoded, wherein the encoding order of the current frame is after the parallel frames; The first prediction unit is configured to predict the encoded information of each parallel frame based on the quantization scale and complexity information of each parallel frame. The second prediction unit is configured to predict the quantization scale of the current frame based on the actual coding information of the coded frame and the predicted coding information of each parallel frame. The encoding unit is configured to encode the current frame based on the predictive quantization scale of the current frame; The second prediction unit is further configured to determine a bitrate factor based on the actual coding information of the coded frame and the predicted coding information of each parallel frame; and to predict the quantization scale of the current frame based on the bitrate factor.

8. An electronic device, characterized in that, include: At least one processor; At least one memory that stores computer-executable instructions. The computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the video encoding method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program and a bit stream, characterized in that, When a computer program in the computer-readable storage medium is executed by at least one processor, the bitstream is generated according to the video encoding method of any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by at least one processor, implements the video encoding method as described in any one of claims 1 to 6.

11. A method for generating a bitstream, characterized in that, include: A bitstream is generated according to the video encoding method as described in any one of claims 1 to 6.

Images