Video encoding method, device, electronic equipment, storage medium, program product and method for storing bitstream

Abstract

The present disclosure provides a video coding method, device, electronic equipment, storage medium, program product and a method for storing a bitstream, the video coding method comprising: obtaining a frame set, wherein the frame set comprises a current frame and a plurality of frames subsequent to the current frame; performing time domain correlation analysis on the frame set to determine a correlation category of the frame set from a plurality of preset correlation categories, wherein the time domain correlation is a time correlation between adjacent frames, the plurality of preset correlation categories are used to represent different correlation degrees of the time domain correlation, and each of the plurality of preset correlation categories has a corresponding code rate prediction parameter; performing code rate prediction on the frame set according to the code rate prediction parameter corresponding to the correlation category of the frame set to obtain a predicted code rate of the frame set; determining a quantization parameter according to the predicted code rate of the frame set; and encoding the current frame using the quantization parameter. The method helps to improve the accuracy of code rate prediction.

Description

Technical Field

[0001] This disclosure relates to video encoding / decoding and compression. More specifically, this disclosure relates to a video encoding method, apparatus, electronic device, storage medium, program product, and method of storing bitstreams. Background Technology

[0002] In real-time video transmission scenarios such as live streaming and remote conferencing, the smoothness of bitrate fluctuations in the encoded video stream significantly impacts the user's playback experience. Frames with higher bitrates often require more decoding time, and drastic changes in bitrate can cause significant variations in decoding time for each frame, resulting in playback stuttering. To reduce bitrate fluctuations, a common industry practice is to establish a virtual video buffer at the encoding end. This buffer simulates the series of operations performed at the decoding end from receiving the bitrate to decoding, thereby smoothing the output bitrate at the encoding end. This module is called a Video Buffer Verifier (VBV).

[0003] VBV's bitrate control can be summarized as a leaky bucket model. Since it simulates the decoding process, the input bitrate of the VBV buffer is constant, which is the maximum bitrate set at the encoding end, while the output is the actual bitrate encoded for each frame. To ensure smooth bitrate fluctuations, the number of bits in the VBV buffer needs to be within a reasonable range, that is, between the overflow and underflow warning thresholds, to avoid buffer overflow and underflow.

[0004] In existing technical solutions, the VBV buffer fullness after encoding all N+1 frames is estimated by obtaining the encoding complexity of the current frame and the subsequent N frames in advance. Then, the quantization parameters (QP) used to encode the current frame are determined based on the buffer fullness. However, existing solutions use the same set of bitrate prediction parameters for the entire video. This one-size-fits-all approach may affect the accuracy of bitrate prediction, and therefore there is still room for optimization. Summary of the Invention

[0005] Embodiments of this disclosure provide a video encoding method, apparatus, electronic device, storage medium, program product, and method for storing bitstreams, for solving at least one of the above-described problems.

[0006] According to one aspect of this disclosure, a video encoding method is provided, the video encoding method comprising: acquiring a frame set, wherein the frame set includes a current frame and a plurality of subsequent frames; performing temporal correlation analysis on the frame set to determine a correlation category of the frame set from a plurality of preset correlation categories, wherein temporal correlation is the temporal correlation between adjacent frames, the plurality of preset correlation categories are used to represent different degrees of temporal correlation, and each of the plurality of preset correlation categories has a corresponding bitrate prediction parameter; performing bitrate prediction on the frame set according to the bitrate prediction parameter corresponding to the correlation category of the frame set to obtain a predicted bitrate of the frame set; determining a quantization parameter according to the predicted bitrate of the frame set; and encoding the current frame using the quantization parameter.

[0007] Optionally, performing temporal correlation analysis on the frame set includes: performing temporal correlation analysis on the frame set based on the feature information of each frame in the frame set, wherein the feature information includes at least one of the following: intra-frame prediction loss, inter-frame prediction loss, length of motion vector, and texture complexity.

[0008] Optionally, when the feature information includes the intra-frame prediction loss and the inter-frame prediction loss, the step of performing temporal correlation analysis on the frame set based on the feature information of each frame in the frame set includes: for each frame in the frame set, determining a correlation metric for each frame by comparing the intra-frame prediction loss with the inter-frame prediction loss; determining a statistical value of the correlation metric for each frame in the frame set; and performing temporal correlation analysis on the frame set by comparing the statistical value of the correlation metric with a statistical value threshold and comparing the intra-frame prediction loss of the current frame with an intra-frame prediction loss threshold.

[0009] Optionally, the video encoding method further includes: updating the bitrate prediction parameter corresponding to the relevance category of the frame set according to the quantization parameter and the actual bitrate for encoding the current frame, wherein the updated bitrate prediction parameter is applied to other frames encoded after the current frame in the current video.

[0010] Optionally, each of the plurality of preset correlation categories has corresponding multiple sets of bitrate prediction parameters, wherein the multiple sets of bitrate prediction parameters correspond one-to-one with multiple frame types; wherein, the step of predicting the bitrate of the frame set according to the bitrate prediction parameters corresponding to the correlation category of the frame set includes: for each frame in the frame set, performing bitrate prediction for each frame according to the correlation category of the frame set and the bitrate prediction parameters corresponding to the frame type of each frame, so as to predict the bitrate of the frame set; wherein, the step of updating the bitrate prediction parameters corresponding to the correlation category of the frame set according to the quantization parameters and the actual bitrate of encoding the current frame includes: updating the bitrate prediction parameters corresponding to the correlation category of the frame set and the frame type of the current frame according to the quantization parameters and the actual bitrate of encoding the current frame.

[0011] Optionally, the bitrate prediction parameter corresponding to each of the plurality of preset correlation categories is obtained through the following steps: for each of the plurality of preset correlation categories, obtain a plurality of sample frame sets of the preset correlation category; encode the plurality of sample frame sets respectively using preset quantization parameters to obtain a plurality of target encoded data, wherein the target encoded data includes encoded data related to the bitrate prediction parameter; perform fitting processing on the plurality of target encoded data to obtain the bitrate prediction parameter corresponding to the preset correlation category.

[0012] Optionally, the plurality of sample frame sets include a set of frames from sample videos of a preset video category.

[0013] According to another aspect of this disclosure, a video encoding apparatus is provided, the video encoding apparatus comprising: an acquisition unit configured to acquire a frame set, wherein the frame set includes a current frame and a plurality of subsequent frames; an analysis unit configured to perform temporal correlation analysis on the frame set to determine a correlation category of the frame set from a plurality of preset correlation categories, wherein temporal correlation is the temporal correlation between adjacent frames, the plurality of preset correlation categories are used to represent different degrees of temporal correlation, and each of the plurality of preset correlation categories has a corresponding bitrate prediction parameter; a prediction unit configured to perform bitrate prediction on the frame set according to the bitrate prediction parameter corresponding to the correlation category of the frame set to obtain a predicted bitrate of the frame set; a determination unit configured to determine a quantization parameter according to the predicted bitrate of the frame set; and an encoding unit configured to encode the current frame using the quantization parameter.

[0014] Optionally, the analysis unit is further configured to perform temporal correlation analysis on the frame set based on the feature information of each frame in the frame set, wherein the feature information includes at least one of the following: intra-frame prediction loss, inter-frame prediction loss, length of motion vector, and texture complexity.

[0015] Optionally, when the feature information includes the intra-frame prediction loss and the inter-frame prediction loss, the analysis unit is further configured to: for each frame in the frame set, determine the correlation metric of each frame by comparing the intra-frame prediction loss with the inter-frame prediction loss; determine the statistical value of the correlation metric of each frame in the frame set; and perform temporal correlation analysis on the frame set by comparing the statistical value of the correlation metric with a statistical value threshold and comparing the intra-frame prediction loss of the current frame with the intra-frame prediction loss threshold.

[0016] Optionally, the video encoding apparatus further includes an update unit configured to update the bitrate prediction parameter corresponding to the relevance category of the frame set based on the quantization parameter and the actual bitrate for encoding the current frame, wherein the updated bitrate prediction parameter is applied to other frames encoded after the current frame in the current video.

[0017] Optionally, each of the plurality of preset correlation categories has corresponding multiple sets of bitrate prediction parameters, wherein the multiple sets of bitrate prediction parameters correspond one-to-one with multiple frame types; the prediction unit is further configured to perform bitrate prediction for each frame in the frame set according to the correlation category of the frame set and the bitrate prediction parameters corresponding to the frame type of each frame, so as to perform bitrate prediction for the frame set; the update unit is further configured to update the correlation category of the frame set and the bitrate prediction parameters corresponding to the frame type of the current frame according to the quantization parameters and the actual bitrate of encoding the current frame.

[0018] Optionally, the bitrate prediction parameter corresponding to each of the plurality of preset correlation categories is obtained through the following steps: for each of the plurality of preset correlation categories, obtain a plurality of sample frame sets of the preset correlation category; encode the plurality of sample frame sets respectively using preset quantization parameters to obtain a plurality of target encoded data, wherein the target encoded data includes encoded data related to the bitrate prediction parameter; perform fitting processing on the plurality of target encoded data to obtain the bitrate prediction parameter corresponding to the preset correlation category.

[0019] Optionally, the plurality of sample frame sets include a set of frames from sample videos of a preset video category.

[0020] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the video encoding method as described above.

[0021] According to another aspect of this disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by at least one processor, causes at least one processor to perform the video encoding method as described above.

[0022] According to another aspect of this disclosure, a computer program product is provided, including computer instructions that, when executed by at least one processor, implement the video encoding method described above.

[0023] According to another aspect of this disclosure, a method for generating a bitstream is provided, comprising: generating a bitstream according to the video encoding method described above.

[0024] The video coding method, apparatus, electronic device, storage medium, program product, and method for storing bitstream according to exemplary embodiments of the present disclosure analyze the temporal correlation of a set of frames to classify the set of frames accordingly, and configure different bitrate prediction parameters for different correlation categories. This enables reasonable classification of the set of frames and differentiated bitrate prediction in subsequent steps, which helps to improve the accuracy of bitrate prediction, coding efficiency, and coding quality.

[0025] It will be understood that the above general description and the following detailed description are merely examples and do not limit this disclosure. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate examples according to this disclosure and, together with this description, serve to explain the principles of this disclosure.

[0027] Figure 1 This is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0028] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0029] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0030] Figure 4 This is a schematic diagram illustrating a rate control leaky bucket model according to some embodiments of the present disclosure.

[0031] Figure 5 This is a schematic diagram illustrating the algorithm flow of a video cache verifier according to some embodiments of the present disclosure.

[0032] Figure 6 This is a flowchart illustrating a video encoding method according to some embodiments of the present disclosure.

[0033] Figure 7 This is a schematic diagram illustrating the algorithm flow of a video cache verifier according to some other embodiments of the present disclosure.

[0034] Figure 8 This is a block diagram illustrating a video encoding apparatus according to some embodiments of the present disclosure.

[0035] Figure 9 This is a diagram illustrating a computing environment coupled to a user interface according to some embodiments of the present disclosure. Detailed Implementation

[0036] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this disclosure are used to distinguish objects and are not used to describe any specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in sequences other than those shown in the drawings or described in this disclosure.

[0038] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0039] In some implementations, the target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14.

[0040] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the target device 14 can access the encoded video data in the storage device 32 via the input interface 28.

[0041] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0042] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback.

[0043] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0044] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this disclosure is not limited to any particular video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0045] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0046] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this disclosure. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0047] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or it can be distributed among one or more of the fixed or programmable hardware units described.

[0048] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... Figure 1 The video source 18 shown obtains video data from the video data storage 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 (e.g., in intra-frame or inter-frame predictive coding mode) when encoding the video data.

[0049] like Figure 2 As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a collection of video blocks) or other larger coding units (CUs) according to a predefined splitting structure (e.g., a quadtree (QT) structure) associated with the video data. It should be noted that the term "block" or "video block" as used herein can be a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring to, for example, HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0050] The prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0051] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0052] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block within the current video frame or picture relative to a prediction block within a reference frame associated with the current block being encoded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0053] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0054] Intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, during individual encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) can select a suitable intra-prediction mode from the tested intra-prediction modes for use. Intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to entropy coding unit 56. Entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0055] After prediction processing unit 41 determines the prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0056] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0057] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1 The video decoder 30 shown, or archived in, for example Figure 1 The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame being encoded.

[0058] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0059] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0060] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of the present disclosure. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0061] In some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform embodiments of this disclosure alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra-frame prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0062] The video data storage 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of the video decoder 30. The video data stored in the video data storage 79 can be obtained, for example, from the storage device 32, a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0063] During the decoding process, the video decoder 30 receives a encoded video bitstream representing video blocks and associated syntax elements of encoded video frames. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0064] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-predictive unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0065] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vectors and other syntax elements received from the entropy decoding unit 80.

[0066] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within a reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0067] The motion compensation unit 82 and / or the intra-frame BC unit 85 determine the prediction information for the video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0068] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0069] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. In some examples, the loop filter 91 may be omitted.

[0070] In real-time video transmission scenarios such as live streaming and remote conferencing, the smoothness of bitrate fluctuations in the encoded video stream significantly impacts the user's playback experience. Frames with higher bitrates often require more decoding time, and drastic changes in bitrate can cause significant variations in decoding time for each frame, resulting in playback stuttering. To reduce bitrate fluctuations, a common industry practice is to establish a virtual video buffer at the encoding end. This buffer simulates the series of operations performed at the decoding end from receiving the bitrate to decoding, thereby smoothing the output bitrate at the encoding end. This module is called a Video Buffer Verifier (VBV).

[0071] VBV's bitrate control can be summarized as follows: Figure 4 The leaky bucket model shown has a constant input bitrate, which is the maximum bitrate set at the encoder, and the output is the actual bitrate for each frame. To ensure smooth bitrate fluctuations, the number of bits in the VBV buffer needs to be within a reasonable range, that is, between the overflow and underflow warning thresholds, to avoid buffer overflow and underflow. In existing technical solutions, the number of bits in the VBV buffer after all N+1 frames have been encoded is estimated by obtaining the complexity of the next N+1 frames in advance. If the number of bits in the buffer exceeds the overflow warning threshold, the quantization parameter (QP) of the current frame needs to be decreased. Conversely, if the number of bits in the buffer is below the underflow warning threshold, the QP of the current frame needs to be increased.

[0072] The technical solution process currently used by the open-source software x264 / x265 is as follows: Figure 5 As shown, the first step is to analyze the encoding complexity and frame type (including I-frames, P-frames, and B-frames) of the current frame and the subsequent N frames using a pre-analysis module. Real-time scenes often employ a fixed frame type structure, making it relatively easy to determine the frame type of subsequent frames during the pre-analysis stage. The encoding complexity is determined by downsampling the source image by 1 / 2 and dividing it into 8×8 blocks. Pre-analysis is performed on a block-by-block basis. The pre-analysis process includes calculating the intra-frame prediction loss for each block (…). ), and inter-frame prediction loss ( The encoding complexity of the i-th frame is measured using the SATD (Sum of Absolute Transformed Difference) metric. It is the sum of the complexity of all m×n 8×8 blocks in the frame, which can be expressed by the following formula.

[0073]

[0074] After obtaining the encoding complexity of all frames from frame i to frame i+N, the rate control module first calculates the initial QP of the current frame based on the target bitrate or CRF (Constant Rate Factor) value set by the user, and converts it into Qscale. The conversion formula is as follows.

[0075]

[0076] After obtaining the initial Qscale of the i-th frame, it is necessary to calculate the number of bits in the current VBV buffer. Next, the bitrates of frames i to i+N are predicted and summed using the encoding complexity and Qscale of the current frame (i.e., frame i) and subsequent frames (i.e., frames i+1 to i+N). The bitrate prediction formula for frame i is as follows.

[0077]

[0078] in, This represents the predicted number of coded bits for the i-th frame (the bitrate is specifically the number of bits transmitted per second, thus allowing for bitrate prediction). The Qscale of subsequent frames can be obtained based on the Qscale of the current frame and a fixed ratio of Qscale between different frame types. The Qscale ratios in x264 / x265 are as follows: and It can be customized by the user, with default values ​​of 1.40 and 1.30.

[0079]

[0080]

[0081] After obtaining the predicted bitrate for all frames within the entire sliding window (of length N+1), it is necessary to estimate the number of bits in the VBV buffer after encoding these N+1 frames. The specific calculation formula is as follows.

[0082]

[0083] in, This indicates the maximum bitrate of the input VBV. This represents the frame rate (Frames Per Second). The goal of bitrate control is to keep the estimated number of bits in the buffer between the VBV overflow and underflow warning thresholds; otherwise, the Qscale of the current frame needs to be adjusted as follows.

[0084]

[0085] in, This represents the adjusted Qscale for the i-th frame. It is the capacity of the VBV cache, that is, the maximum number of bits it can hold. and The ratio thresholds for underflow and overflow warnings are fixed values ​​in x264 / x265, set to 0.5 and 0.8 respectively. This is based on the estimated number of bits currently cached. If the requirements are met, exit the loop and use the values ​​from the current loop. Encode the i-th frame. Otherwise, use the obtained... Restart the next round of bit estimation and Qscale adjustment.

[0086] After encoding the i-th frame, it is necessary to determine the Qscale used in the actual encoding (represented as...). ) and the number of bits obtained (represented as To update the bitrate prediction parameters (i.e., the parameters coeff and offset in the bitrate prediction formula), the specific update method is as follows.

[0087]

[0088]

[0089] In this context, the index "old" in the bitrate prediction parameter indicates the state before the update, and "new" indicates the state after the update.

[0090] The existing schemes described above use a single set of bitrate prediction parameters and update mechanisms for different frames. This one-size-fits-all approach may affect the accuracy of bitrate prediction, and therefore there is still room for optimization.

[0091] Figure 6 This is a flowchart illustrating an exemplary video encoding method according to some embodiments of the present disclosure. This video encoding method can be implemented in the video encoder 20 described above.

[0092] Reference Figure 6 In step S601, the frame set is obtained.

[0093] The frame set includes the current frame and multiple subsequent frames. It is used to estimate the buffer usage of a future event before encoding the current frame. Refer to frames i to i+N in the existing scheme described above, and it can be obtained using a sliding window. As an example, the length N of the sliding window can be a fixed value or an adjustable value; this disclosure does not limit this. Of course, other reasonable methods besides sliding windows can also be used to obtain the frame set, and this disclosure also does not limit this.

[0094] In step S602, temporal correlation analysis is performed on the frame set to determine the correlation category of the frame set from multiple preset correlation categories.

[0095] Temporal correlation refers to the temporal relationship between adjacent frames. Multiple predefined correlation categories represent different degrees of temporal correlation, and each category has a corresponding bitrate prediction parameter. Analysis reveals that different temporal complexities (i.e., the degree of change in the time dimension) have varying impacts on the final bitrate during encoding, thus affecting the accuracy of bitrate prediction. Therefore, by analyzing the temporal correlation of a frame set to classify it accordingly, and configuring different bitrate prediction parameters for different correlation categories, the frame set can be reasonably classified, and bitrate prediction can be performed selectively in subsequent steps. This helps improve the accuracy of bitrate prediction, encoding efficiency, and encoding quality. As an example, the multiple predefined correlation categories include, but are not limited to, static, continuous, and complex categories, with the degree of temporal correlation decreasing progressively among them.

[0096] Optionally, the operation of performing temporal correlation analysis on the frame set in step S602 includes: performing temporal correlation analysis on the frame set based on the feature information of each frame in the frame set, wherein the feature information includes at least one of the following: intra-frame prediction loss, inter-frame prediction loss, length of motion vector, and texture complexity. As mentioned above, temporal correlation is the temporal correlation between adjacent frames. By obtaining and analyzing the related feature information, such as including but not limited to intra-frame prediction loss, inter-frame prediction loss, length of motion vector, and texture complexity, temporal correlation analysis can be achieved. As an example, the smaller the intra-frame prediction loss, the more suitable the intra-frame prediction coding mode is for encoding the frame set, which means the lower the temporal correlation and the higher the temporal complexity. The inter-frame prediction loss is exactly the opposite of the intra-frame prediction loss; the larger its value, the more suitable the intra-frame prediction coding mode is for encoding the frame set, which means the lower the temporal correlation and the higher the temporal complexity. The larger the length of the motion vector, or the greater the texture complexity, the more intense the motion of each frame in the frame set, and the less likely it is to find the corresponding prediction block in the reference frame. This means that the temporal correlation is lower and the temporal complexity is higher, similar to the inter-frame prediction loss.

[0097] As an example, when the feature information includes intra-frame prediction loss and inter-frame prediction loss, the above-mentioned operation of performing temporal correlation analysis on the frame set based on the feature information of each frame in the frame set includes: for each frame in the frame set, determining the correlation metric of each frame by comparing the intra-frame prediction loss and the inter-frame prediction loss; determining the statistical value of the correlation metric of each frame in the frame set; and performing temporal correlation analysis on the frame set by comparing the statistical value of the correlation metric with a statistical threshold and comparing the intra-frame prediction loss of the current frame with the intra-frame prediction loss threshold. Considering that from a coding perspective, both intra-frame prediction loss and inter-frame prediction loss have better coding performance as the smaller the value, by comparing the two feature information with opposite correlations to temporal correlation, a relative value can be obtained as a correlation metric to characterize whether the inter-frame prediction coding mode or the intra-frame prediction coding mode is more suitable, thereby reducing the impact of the absolute value of the two losses on the temporal correlation analysis. Building upon this foundation, by introducing statistical thresholds, a benchmark can be provided for the statistical values ​​of correlation measures (such as, but not limited to, sum, mean, mode, and median), enabling temporal correlation analysis. Furthermore, if the intra-frame prediction loss of the current frame is large, it indicates complex texture within the current frame. Even if comparing the two losses reveals that the current frame is more suitable for inter-frame predictive coding, it may be difficult to find suitable reference frames and blocks. In other words, the actual temporal correlation may be low. Therefore, by additionally assessing the intra-frame prediction loss of the current frame—specifically by comparing it with an intra-frame prediction loss threshold—we can enhance the analysis of the spatial complexity of the current frame, thereby optimizing the assessment of temporal correlation and improving the analysis results. For example, taking the aforementioned preset correlation categories—including the stationary category, continuous category, and complex category with gradually decreasing temporal correlation—as an example, the ratio of inter-frame prediction loss to intra-frame prediction loss can be used as a correlation metric. In this case, the smaller the correlation metric value, the higher the temporal correlation. Two statistical threshold values ​​can be configured, denoted as the first statistical threshold and the second statistical threshold, with the former being greater than the latter. An intra-frame prediction loss threshold is also configured. When the statistical value of the correlation metric is less than the second statistical threshold, the frame is classified as stationary. When the statistical value of the correlation metric is greater than the first statistical threshold, or when the intra-frame prediction loss of the current frame is greater than the intra-frame prediction loss threshold (i.e., the spatial complexity of the current frame is too high, making it difficult to find reference frames and reference blocks), the frame is classified as complex. Otherwise, the frame is classified as continuous. Of course, other reasonable methods can also be used to analyze the intra-frame prediction loss and inter-frame prediction loss of each frame in the frame set; this disclosure does not impose any limitations on this.

[0098] As an example, when the feature information includes the length of the motion vector, two statistical thresholds can be configured, denoted as the first statistical threshold and the second statistical threshold, respectively. The former is greater than the latter. When the length of the motion vector is less than the second statistical threshold, it is classified as a stationary category; when it is greater than the first statistical threshold, it is classified as a complex category; and when it is between the first and second statistical thresholds, it is classified as a continuous category. The same applies when the feature information includes texture complexity, and the specific values ​​of the statistical thresholds need to be adjusted accordingly. Of course, other reasonable methods can also be used to analyze the length of the motion vector or texture complexity, or even other feature information, of each frame in the frame set, and this disclosure does not limit this.

[0099] Regarding the determination of bitrate prediction parameters, optionally, the bitrate prediction parameters corresponding to each of the multiple preset correlation categories are obtained through the following steps: For each of the multiple preset correlation categories, obtain multiple sample frame sets of the preset correlation category; encode the multiple sample frame sets separately using preset quantization parameters to obtain multiple sets of target encoded data, wherein the target encoded data includes encoded data related to the bitrate prediction parameters; perform fitting processing on the multiple sets of target encoded data to obtain the bitrate prediction parameters corresponding to the preset correlation category. By obtaining a sample frame set of a specific preset correlation category and encoding it using preset quantization parameters, the target encoded data for actually encoding these sample frame sets can be obtained. The bitrate prediction parameters are specifically used to predict these target encoded data (e.g., including but not limited to Qscale, encoding complexity, and actual bitrate mentioned above), so the corresponding bitrate prediction parameters can be backfitted to achieve the determination of the bitrate prediction parameters.

[0100] As an example, a sample video can be acquired first. Based on each frame in the sample video, multiple sample frame sets are obtained. Then, the operation in step S602 is performed on each of these sample frame sets to determine the correlation category of each sample frame set. These sample frame sets can be grouped according to their corresponding correlation categories, and then the bitrate prediction parameters are calculated for each group of sample frame sets. It should be understood that if the number of sample frame sets for a certain correlation category is insufficient, new sample videos can be acquired.

[0101] As an example, there can be multiple preset quantization parameters to use different quantization parameters for encoding, so that each sample frame set can obtain multiple sets of target encoded data. Accordingly, when fitting multiple sets of target encoded data, specifically, multiple sets of target encoded data from the same sample frame set can be fitted first to obtain the fitting result of that sample frame set (specifically, the fitted bitrate prediction parameters), and then the fitting results of multiple sample frame sets can be statistically processed, for example, by averaging each bitrate prediction parameter to obtain the bitrate prediction parameters for that preset correlation category.

[0102] Optionally, the multiple sample frame sets include a set of frames from sample videos of a preset video category. By using sample videos of a preset video category and extracting sample frame sets from them, more representative sample videos can be used specifically based on the encoding requirements of the specific scenario corresponding to the preset video category, thereby obtaining better encoding results in the specific scenario. As an example, as mentioned above, the Video Buffer Verifier (VBV) is often used in real-time video transmission scenarios such as live video streaming and remote conferencing. Based on this, for live video streaming scenarios, live screen recordings can be selected as sample videos, or for remote conferencing scenarios, remote conferencing recordings can be selected as sample videos. This disclosure does not impose any limitations on this.

[0103] In step S603, the bitrate of the frame set is predicted according to the bitrate prediction parameters corresponding to the correlation category of the frame set, and the predicted bitrate of the frame set is obtained.

[0104] In step S604, the quantization parameters are determined based on the predicted bitrate of the frame set.

[0105] In step S605, the current frame is encoded using quantization parameters.

[0106] Once the specific bitrate prediction parameters are determined, the bitrate prediction, quantization parameter determination, and encoding performed in steps S603 to S605 can all refer to existing schemes, and will not be elaborated here.

[0107] The exemplary video coding method according to this disclosure will be further described below.

[0108] Optionally, in some embodiments, the exemplary video encoding method according to this disclosure further includes: updating the bitrate prediction parameters corresponding to the relevance categories of the frame set based on quantization parameters and the actual bitrate for encoding the current frame, wherein the updated bitrate prediction parameters are applied to other frames encoded after the current frame in the current video. By updating the corresponding bitrate prediction parameters according to the actual encoding quantization parameters and the actual bitrate for the corresponding relevance categories, the bitrate prediction parameters can be continuously optimized, thereby continuously improving the accuracy of bitrate prediction and enhancing encoding efficiency and quality. It should be understood that the bitrate prediction parameters corresponding to each preset relevance category obtained by fitting the sample frame set as described above can be used as the initial bitrate prediction parameters for each preset relevance category. When encoding a frame of a certain preset relevance category in the currently encoded video, the initial bitrate prediction parameters of the preset relevance category are used for bitrate prediction, and the bitrate prediction parameters can be updated as needed. When encoding other frames of the same preset relevance category, the latest updated bitrate prediction parameters of the preset relevance category are used for bitrate prediction, and updates can continue. It should also be understood that the bitrate prediction parameters are only applied to the current video, in addition to being updated according to different relevance categories. This means that when encoding another video, since the video content has changed, the initial bitrate prediction parameters need to be reused, but the bitrate prediction parameters can still be updated.

[0109] Optionally, each of the multiple preset correlation categories has corresponding multiple sets of bitrate prediction parameters, wherein the multiple sets of bitrate prediction parameters correspond one-to-one with multiple frame types. Accordingly, step S603 includes: for each frame in the frame set, performing bitrate prediction on each frame according to the correlation category of the frame set and the bitrate prediction parameters corresponding to the frame type of each frame, to perform bitrate prediction on the frame set. The aforementioned step of updating the bitrate prediction parameters corresponding to the correlation category of the frame set according to the quantization parameters and the actual bitrate of encoding the current frame includes: updating the bitrate prediction parameters corresponding to the correlation category of the frame set and the frame type of the current frame according to the quantization parameters and the actual bitrate of encoding the current frame. By using different bitrate prediction parameters for different correlation categories, and further using different bitrate prediction parameters for different frame types under the same correlation category to perform bitrate prediction, and performing differentiated bitrate prediction parameter updates, that is, updating the bitrate prediction parameters of the frame type of the current frame only according to the actual encoding situation of the current frame, more detailed and accurate bitrate prediction can be achieved. It should be understood that, taking the aforementioned preset correlation categories—including the three categories of stationary, continuous, and complex—as an example, since there are also three frame types (I-frame, P-frame, and B-frame), a total of nine sets of bitrate prediction parameters can be formed. As an example, different frame types within the same preset correlation category can use the same initial bitrate prediction parameters, and then update the bitrate prediction parameters separately for each frame type. Of course, different initial bitrate prediction parameters can also be used for different frame types; this disclosure does not impose any restrictions on this.

[0110] Next, combine Figure 7 This disclosure introduces a video encoding method according to a specific embodiment.

[0111] like Figure 7 As shown, in this specific embodiment, when the encoder starts encoding the current frame, it calculates the inter-frame prediction loss based on the current frame (the i-th frame) and the subsequent frames (the N subsequent frames). and intra-frame prediction loss The ratio (as a correlation measure for each frame) is used to classify the temporal correlation of all frames (i.e., the frame set) within the current sliding window. First, the cumulative temporal correlation ratio of the current frame and subsequent frames is calculated. (Statistical value as a measure of correlation).

[0112]

[0113] Based on the calculated cumulative time-domain correlation ratio The specific classification method is as follows.

[0114]

[0115] in, The value of can be 0, 1, or 2, representing that the current set of frames in the sliding window is classified into three temporal correlation categories: static, continuous, and complex. m×n represents the number of 8x8 blocks that the current frame can be divided into. and Corresponding to the first and second statistical thresholds introduced earlier, Corresponding to the intra-prediction loss threshold introduced earlier, experiments were conducted to... , and Set them to 0.6, 0.3 and 2 respectively. 11 .

[0116] After determining the category to which the frame set belongs, the predictive coding bits (bits) for each frame in the frame set are calculated according to the following formula, where the subscript... The superscript "type" indicates the relevance category to which a set of frames belongs, and indicates the frame type of the frames whose predicted coding bits need to be calculated.

[0117]

[0118] After obtaining the predicted number of bits for subsequent frames, the following algorithm is similar to the existing scheme described above, which estimates the number of bits in the VBV buffer after encoding N+1 frames in the frame set.

[0119]

[0120] After encoding the current frame, based on the actual number of encoded bits in the current frame. and This is used to update the bitrate prediction parameters, and only updates the bitrate prediction parameters based on the correlation category shown in the frame set and the frame type of the current frame. and This means that the parameter updates for each correlation category and frame type are independent of each other, reducing the mutual influence of parameter update processes for different correlation categories and frame types, and improving prediction accuracy.

[0121]

[0122]

[0123] For different relevance categories, it is necessary to... and Initial values ​​are fitted, and parameters can be updated separately for different frame types. Here, the first 200 frames of each sequence in the HEVC standard sequence are selected as sample frames, and their corresponding sample frame sets are determined. Temporal correlation analysis is then performed to determine the correlation category. Then, four QP points (22, 27, 32, and 37) are selected to encode these 200 frames respectively. The initial values ​​fitted by the bitrate prediction formula are as follows.

[0124]

[0125] This specific embodiment innovatively proposes a buffered check rate control method based on temporal correlation. Utilizing information obtained from pre-analysis of the current frame and subsequent frames, it classifies the frame set corresponding to the frame to be encoded according to temporal correlation, fitting different initial rate prediction parameters to different correlation categories, thus making the rate prediction process more accurate. This specific embodiment also innovatively designs a rate prediction parameter update mechanism based on correlation category and frame type. After the current frame is encoded, the rate prediction parameters are updated according to its correlation category and frame type, reducing rate prediction errors and improving encoding performance. Testing shows that this specific embodiment can achieve a PSNR (Peak Signal-to-Noise Rate) decrease of approximately 0.357% and an SSIM (Structural Similarity Index) decrease of approximately 0.260% on the BD-rate (Bjøntegaard-Delta rate) of the K265 mobile encoder, with almost no change in encoding time.

[0126] Figure 8 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment of the present disclosure.

[0127] Reference Figure 8 The video encoding device 800 includes an acquisition unit 801, an analysis unit 802, a prediction unit 803, a determination unit 804, and an encoding unit 805.

[0128] The acquisition unit 801 can acquire a frame set, wherein the frame set includes the current frame and multiple subsequent frames.

[0129] The analysis unit 802 can perform temporal correlation analysis on the frame set to determine the correlation category of the frame set from multiple preset correlation categories. The temporal correlation is the temporal correlation between adjacent frames. The multiple preset correlation categories are used to represent different degrees of temporal correlation. Each preset correlation category has a corresponding bitrate prediction parameter.

[0130] The prediction unit 803 can predict the bitrate of the frame set according to the bitrate prediction parameters corresponding to the correlation category of the frame set, and obtain the predicted bitrate of the frame set.

[0131] The determining unit 804 can determine the quantization parameters based on the predicted bit rate of the frame set.

[0132] The coding unit 805 can use quantization parameters to encode the current frame.

[0133] Optionally, the analysis unit 802 may also: perform temporal correlation analysis on the frame set based on the feature information of each frame in the frame set, wherein the feature information includes at least one of the following: intra-frame prediction loss, inter-frame prediction loss, length of motion vector, and texture complexity.

[0134] Optionally, when the feature information includes intra-frame prediction loss and inter-frame prediction loss, the analysis unit 802 may further: for each frame in the frame set, determine the correlation metric of each frame by comparing the intra-frame prediction loss with the inter-frame prediction loss; determine the statistical value of the correlation metric of each frame in the frame set; and perform temporal correlation analysis on the frame set by comparing the statistical value of the correlation metric with a statistical value threshold and comparing the intra-frame prediction loss of the current frame with the intra-frame prediction loss threshold.

[0135] Optionally, the video encoding apparatus 800 further includes an update unit (not shown in the figure), which can update the bitrate prediction parameters corresponding to the correlation category of the frame set according to the quantization parameters and the actual bitrate for encoding the current frame. The updated bitrate prediction parameters are applied to other frames encoded after the current frame in the current video.

[0136] Optionally, each of the multiple preset correlation categories has a corresponding set of bitrate prediction parameters, wherein the multiple sets of bitrate prediction parameters correspond one-to-one with multiple frame types; the prediction unit 803 can also perform bitrate prediction for each frame in the frame set according to the correlation category of the frame set and the bitrate prediction parameters corresponding to the frame type of each frame, so as to perform bitrate prediction for the frame set; the update unit can also update the correlation category of the frame set and the bitrate prediction parameters corresponding to the frame type of the current frame according to the quantization parameters and the actual bitrate of the current frame.

[0137] Optionally, the bitrate prediction parameter corresponding to each of the multiple preset correlation categories is obtained through the following steps: for each of the multiple preset correlation categories, obtain multiple sample frame sets of the preset correlation category; encode the multiple sample frame sets respectively using preset quantization parameters to obtain multiple sets of target encoded data, wherein the target encoded data includes encoded data related to the bitrate prediction parameter; and perform fitting processing on the multiple sets of target encoded data to obtain the bitrate prediction parameter corresponding to the preset correlation category.

[0138] Optionally, the multiple sample frame sets include a set of frames from sample videos of a preset video category.

[0139] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0140] Figure 9 A computing environment 910 coupled to a user interface 950 is shown. The computing environment 910 may be part of a data processing server. The computing environment 910 includes a processor 920, a memory 930, and an input / output (I / O) interface 940.

[0141] Processor 920 typically controls the overall operation of computing environment 910, such as operations associated with display, data acquisition, data communication, and image processing. Processor 920 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 920 may include one or more modules that facilitate interaction between processor 920 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0142] Memory 930 is configured to store various types of data to support the operation of computing environment 910. Memory 930 may include predefined software 932. Examples of such data include instructions for any application or method operating on computing environment 910, video datasets, image data, etc. Memory 930 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0143] I / O interface 940 provides an interface between processor 920 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 940 can be coupled to encoders and decoders.

[0144] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a plurality of programs in a memory 930 and / or a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by a processor 920 in a computing environment 910 to perform the above-described methods. In one example, the plurality of programs can be executed by a processor 920 in a computing environment 910 to (e.g., from...) Figure 2 The video encoder 20 in the computing environment 910 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 920 in the computing environment 910 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 920 in the computing environment 910 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 920 in the computing environment 910 to (e.g., to...) Figure 3 The video decoder 30 in the middle sends the bitstream or data stream. Alternatively, a non-transitory computer-readable storage medium may store data generated by the encoder (e.g., Figure 2 The video encoder 20 in the video encoder uses, for example, the encoding method described above to generate the video for the decoder (e.g., Figure 3 The video decoder 30 in the video decoder uses a bitstream or data stream that includes encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) when decoding video data. Non-transitory computer-readable storage media can be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage devices, etc.

[0145] In one embodiment, a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method is provided. In another embodiment, a bitstream comprising encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method is provided.

[0146] In one embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 920); and a non-transitory computer-readable storage medium or memory 930 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0147] In one embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, the bitstream including encoded video information generated by the encoding method described above or encoded video information to be decoded by the decoding method described above. In another embodiment, a computer program product including, for example, a plurality of programs in a memory 930, the plurality of programs being executable by a processor 920 in a computing environment 910 to perform the methods described above. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0148] In an embodiment, the computing environment 910 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0149] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0150] In one embodiment, a method for transmitting a bitstream generated by the encoder described above is also provided. In another embodiment, a method for receiving a bitstream to be decoded by the decoder described above is also provided.

[0151] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0152] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0153] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A video encoding method, characterized in that, The video encoding method includes: Obtain a frame set, wherein the frame set includes the current frame and multiple subsequent frames; Based on the feature information of each frame in the frame set, a temporal correlation analysis is performed on the frame set to determine the correlation category of the frame set from multiple preset correlation categories. The feature information includes at least one of the following: intra-frame prediction loss, inter-frame prediction loss, length of motion vector, and texture complexity. Temporal correlation is the temporal correlation between adjacent frames. The multiple preset correlation categories are used to represent different degrees of temporal correlation. The intra-frame prediction loss is positively correlated with the degree of temporal correlation, while the inter-frame prediction loss, the length of motion vector, and the texture complexity are negatively correlated with the degree of temporal correlation. Each preset correlation category has a corresponding bitrate prediction parameter. Based on the bitrate prediction parameters corresponding to the correlation category of the frame set, the bitrate of the frame set is predicted to obtain the predicted bitrate of the frame set. The quantization parameters are determined based on the predicted bitrate of the frame set; The current frame is encoded using the quantization parameters.

2. The video encoding method as described in claim 1, characterized in that, When the feature information includes the intra-frame prediction loss and the inter-frame prediction loss, the step of performing temporal correlation analysis on the frame set based on the feature information of each frame in the frame set includes: For each frame in the frame set, a correlation metric for each frame is determined by comparing the intra-frame prediction loss with the inter-frame prediction loss. Determine the statistical value of the correlation metric for each frame in the frame set; Temporal correlation analysis is performed on the frame set by comparing the statistical value of the correlation metric with the statistical value threshold, and by comparing the intra-frame prediction loss of the current frame with the intra-frame prediction loss threshold.

3. The video encoding method as described in claim 1 or 2, characterized in that, The video encoding method further includes: Based on the quantization parameters and the actual bitrate used to encode the current frame, the bitrate prediction parameters corresponding to the relevance category of the frame set are updated, wherein the updated bitrate prediction parameters are applied to other frames encoded after the current frame in the current video.

4. The video encoding method as described in claim 3, characterized in that, Each of the multiple preset correlation categories has a corresponding set of bitrate prediction parameters, wherein the multiple sets of bitrate prediction parameters correspond one-to-one with multiple frame types; Wherein, the step of predicting the bitrate of the frame set based on the bitrate prediction parameters corresponding to the relevance category of the frame set includes: For each frame in the frame set, bitrate prediction is performed on each frame according to the correlation category of the frame set and the bitrate prediction parameters corresponding to the frame type of each frame, so as to perform bitrate prediction on the frame set. The step of updating the bitrate prediction parameter corresponding to the relevance category of the frame set based on the quantization parameter and the actual bitrate of encoding the current frame includes: Based on the quantization parameters and the actual bitrate used to encode the current frame, update the correlation category of the frame set and the bitrate prediction parameters corresponding to the frame type of the current frame.

5. The video encoding method as described in claim 1 or 2, characterized in that, The bitrate prediction parameters corresponding to each of the multiple preset correlation categories are obtained through the following steps: For each of the multiple preset correlation categories, obtain a set of multiple sample frames for that preset correlation category; The multiple sample frame sets are encoded using preset quantization parameters to obtain multiple sets of target encoded data, wherein the target encoded data includes encoded data related to the bitrate prediction parameters; The multiple sets of target encoded data are fitted to obtain the bitrate prediction parameters corresponding to the preset correlation category.

6. The video encoding method as described in claim 5, characterized in that, The multiple sample frame sets include a set of frames from sample videos of a preset video category.

7. A video encoding device, characterized in that, The video encoding device includes: The acquisition unit is configured to acquire a set of frames, wherein the set of frames includes the current frame and multiple subsequent frames; An analysis unit is configured to perform temporal correlation analysis on the frame set based on the feature information of each frame in the frame set, so as to determine the correlation category of the frame set from multiple preset correlation categories. The feature information includes at least one of the following: intra-frame prediction loss, inter-frame prediction loss, length of motion vector, and texture complexity. Temporal correlation is the temporal correlation between adjacent frames. The multiple preset correlation categories are used to represent different degrees of temporal correlation. The intra-frame prediction loss is positively correlated with the degree of temporal correlation, while the inter-frame prediction loss, the length of motion vector, and the texture complexity are negatively correlated with the degree of temporal correlation. Each preset correlation category has a corresponding bitrate prediction parameter. The prediction unit is configured to predict the bitrate of the frame set based on the bitrate prediction parameters corresponding to the correlation category of the frame set, and obtain the predicted bitrate of the frame set. The determining unit is configured to determine quantization parameters based on the predicted bitrate of the frame set; The encoding unit is configured to encode the current frame using the quantization parameters.

8. An electronic device, characterized in that, include: At least one processor; At least one memory that stores computer-executable instructions. The computer-executable instructions, when executed by at least one processor, cause at least one processor to perform the video encoding method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by at least one processor, they cause at least one processor to perform the video encoding method as described in any one of claims 1 to 6.

10. A computer program product comprising computer instructions, characterized in that, The computer instructions, when executed by at least one processor, implement the video encoding method as described in any one of claims 1 to 6.

11. A method for storing a bit stream, characterized in that, include: Generate a bitstream by performing the video encoding method as described in any one of claims 1 to 6; Store the bit stream.

Images