Video encoding methods and apparatus, methods for transmitting bit streams, electronic devices, storage media and software products

Abstract

This disclosure relates to a video encoding method and apparatus, a method for generating a bitstream, an electronic device, a storage medium, and a program product. The video encoding method includes: determining a target video frame in a video to be encoded; determining the complexity of the current video scene corresponding to the target video frame; determining the encoding bitrate of the target video frame based on the complexity of the current video scene; and encoding the target video frame based on the encoding bitrate. This disclosure solves the problem of unsatisfactory bitrate allocation by the encoder for different video scenes, and can adaptively allocate the bitrate according to the video scene, thereby improving the subjective image quality experience of videos in different video scenes.

Description

Technical Field

[0001] This disclosure relates to the field of video technology, and in particular to a video encoding method and apparatus, a method for generating bitstreams, electronic devices, storage media, and program products. Background Technology

[0002] Various electronic devices (such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc.) support digital video. Electronic devices send and receive, or otherwise transmit, digital video data via communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited storage resources of storage devices, video data can be compressed using one or more video codec standards before it is transmitted or stored. For example, video codec standards include Universal Video Codec (VVC), Joint Explore Test Model (JEM), High Efficiency Video Codec (HEVC / H.265), Advanced Video Codec (AVC / H.264), Moving Picture Experts Group (MPEG) codec, etc. Video codecs typically employ prediction methods that utilize the inherent redundancy in video data (e.g., inter-frame prediction, intra-frame prediction, etc.). Video codecs aim to compress video data to a form using a lower bitrate while avoiding or minimizing degradation in video quality.

[0003] In videos, the image quality may vary depending on the video scene. However, during the video encoding process, the bitrate allocation strategy of mainstream hardware encoders for different video scenes is not ideal, which may affect the subjective image quality experience. It may also lead to high transmission bandwidth costs due to unreasonable bitrate allocation, thus affecting the viewing experience.

[0004] Furthermore, hardware encoders can only be configured and used through the peripheral interfaces provided by the official website. Their internal implementation mechanism is a black box, with limited adjustable parameters, low algorithm flexibility and accuracy, and the inability to adaptively allocate bitrate according to video scenarios. Summary of the Invention

[0005] This disclosure provides a video encoding method and apparatus, a method for generating bitstreams, an electronic device, a storage medium, and a program product, to at least solve the problem of unsatisfactory bitrate allocation for different video scenes by encoders in related technologies. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, a video encoding method is provided, the video encoding method comprising: determining a target video frame in a video to be encoded; determining the complexity of a current video scene corresponding to the target video frame; determining the encoding bitrate of the target video frame based on the complexity of the current video scene; and encoding the target video frame based on the encoding bitrate of the target video frame.

[0006] Optionally, determining the encoding bitrate of the target video frame based on the complexity of the current video scene includes: determining the complexity distribution information of the video to be encoded; and determining the encoding bitrate of the target video frame based on the complexity of the current video scene and the complexity distribution information.

[0007] Optionally, determining the encoding bitrate of the target video frame based on the complexity of the current video scene includes: obtaining the encoding bitrate of the target video frame by increasing the original encoding bitrate of the video to be encoded based on the complexity of the current video scene and the complexity distribution information.

[0008] Optionally, the complexity distribution information is determined by: determining the complexity of the second video frame following the first video frame in the video to be encoded, wherein the first video frame is an intra-predicted frame, and the second video frame is a forward-predicted frame a preset number of frames after the first video frame; and determining the complexity distribution information based on the complexity of multiple first video frames and multiple second video frames in the video to be encoded.

[0009] Optionally, the step of obtaining the encoding bitrate of the target video frame by increasing the original encoding bitrate of the video to be encoded based on the complexity of the current video scene and the complexity distribution information includes: determining an enhancement factor for increasing the original encoding bitrate based on the complexity of the current video scene and the complexity distribution information; and obtaining the encoding bitrate of the target video frame based on the enhancement factor and the original encoding bitrate.

[0010] Optionally, the complexity distribution information includes a maximum value and a minimum value of the complexity distribution. The maximum value of the complexity distribution is determined based on the maximum complexity of the plurality of first video frames and the plurality of second video frames, and the minimum value of the complexity distribution is determined based on the minimum complexity of the plurality of first video frames and the plurality of second video frames. The boosting coefficient is determined by adjusting a preset coefficient based on the maximum value of the complexity distribution, the minimum value of the complexity distribution, and the complexity of the current video scene. The preset coefficient represents the maximum multiple by which the original coding bitrate is increased.

[0011] Optionally, determining the encoding bitrate of the target video frame based on the complexity of the current video scene further includes: responding to the current video scene complexity being higher than a first preset value, performing the step of increasing the original encoding bitrate of the video to be encoded to obtain the encoding bitrate of the target video frame based on the complexity of the current video scene and the complexity distribution information; responding to the current video scene complexity and the complexity of the first second video frame after the target video frame both being lower than a second preset value, obtaining the encoding bitrate of the target video frame by decreasing the original encoding bitrate; responding to the current video scene complexity being lower than the first preset value and higher than the second preset value, determining the original encoding bitrate as the encoding bitrate of the target video frame, wherein the first preset value is greater than the second preset value.

[0012] Optionally, the complexity of the current video scene can be determined by statistically analyzing the complexity of video frames within a first preset number of frames starting from the target video frame.

[0013] Optionally, the target video frame is a transition frame in the video to be encoded, wherein the target video frame is determined by: identifying video frames in the video to be encoded whose difference between inter-frame complexity and intra-frame complexity exceeds a difference threshold as the target video frame.

[0014] Optionally, the video encoding method further includes: based on the original encoding bitrate of the video to be encoded, obtaining the encoding bitrate of a plurality of related video frames after the target video frame by increasing the original encoding bitrate; encoding the plurality of related video frames based on their respective encoding bitrates, wherein the encoding bitrate of the target video frame is greater than or equal to the encoding bitrate of the plurality of related video frames, and the encoding bitrate of the preceding video frame in any adjacent video frame among the plurality of related video frames is greater than or equal to the encoding bitrate of the following video frame in the adjacent video frames.

[0015] According to a second aspect of the present disclosure, a video encoding apparatus is provided, comprising: a first determining unit configured to determine a target video frame in a video to be encoded; a second determining unit configured to determine the complexity of a current video scene corresponding to the target video frame; a third determining unit configured to determine the encoding bitrate of the target video frame based on the complexity of the current video scene; and an encoding unit configured to encode the target video frame based on the encoding bitrate of the target video frame.

[0016] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor-executable instructions, when executed by the processor, cause the processor to perform a video encoding method according to an exemplary embodiment of the present disclosure.

[0017] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform a video encoding method according to exemplary embodiments of the present disclosure.

[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer instructions that, when executed by a processor, implement the video encoding method according to exemplary embodiments of the present disclosure.

[0019] According to a sixth aspect of the present disclosure, a method for generating a bitstream is provided, comprising: generating a bitstream according to a video encoding method as described in exemplary embodiments of the present disclosure.

[0020] According to a seventh aspect of the present disclosure, a computer program product is provided having instructions for storing a bitstream, the bitstream comprising: video data generated according to a video encoding method as described in exemplary embodiments of the present disclosure.

[0021] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: In the embodiments of this disclosure, the complexity of the video scene corresponding to the target video frame in the video to be encoded can be determined. Based on the complexity of the video scene, the encoding bitrate of the target video frame can be determined for encoding. In this way, the encoding bitrate can be determined by combining the complexity analysis of the video scene, solving the problem of unsatisfactory bitrate allocation for different video scenes. This allows for adaptive allocation of bitrate based on the video scene, improving the subjective picture quality experience of the video in different video scenes.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0024] Figure 1 This is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to exemplary embodiments of the present disclosure.

[0025] Figure 2 This is a block diagram of an exemplary video encoder illustrated according to exemplary embodiments of the present disclosure.

[0026] Figure 3 This is a block diagram of an exemplary video decoder illustrated according to exemplary embodiments of the present disclosure.

[0027] Figure 4 This is a schematic flowchart illustrating a video encoding method according to exemplary embodiments of the present disclosure.

[0028] Figure 5 This is a flowchart illustrating a video encoding method according to exemplary embodiments of the present disclosure.

[0029] Figure 6 This is a block diagram illustrating a video encoding apparatus according to exemplary embodiments of the present disclosure.

[0030] Figure 7 This is a block diagram illustrating an electronic device according to exemplary embodiments of the present disclosure.

[0031] Figure 8 This is a diagram illustrating a computing environment coupled to a user interface according to exemplary embodiments of the present disclosure. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0033] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0034] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0035] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0036] In some implementations, the target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14.

[0037] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the target device 14 can access the encoded video data in the storage device 32 via the input interface 28.

[0038] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0039] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback.

[0040] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0041] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to any particular video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0042] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0043] Figure 2This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0044] like Figure 2 As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or it can be distributed among one or more of the fixed or programmable hardware units described.

[0045] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... Figure 1 The video source 18 shown obtains video data from the video data storage 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 (e.g., in intra-frame or inter-frame predictive coding mode) when encoding the video data.

[0046] like Figure 2As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a collection of video blocks) or other larger coding units (CUs) according to a predefined splitting structure (e.g., a quadtree (QT) structure) associated with the video data. It should be noted that the term "block" or "video block" as used herein can be a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring to, for example, HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0047] The prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0048] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0049] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block within the current video frame or picture relative to a prediction block within a reference frame associated with the current block being encoded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0050] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0051] Intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, during individual encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) can select a suitable intra-prediction mode from the tested intra-prediction modes for use. Intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to entropy coding unit 56. Entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0052] After prediction processing unit 41 determines the prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0053] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0054] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1 The video decoder 30 shown, or archived in, for example Figure 1 The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame being encoded.

[0055] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0056] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0057] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0058] In some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame prediction (BC) unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra-frame prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra-frame prediction (BC) unit 85, and the functionality of the intra-frame prediction (BC) unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0059] The video data storage device 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of the video decoder 30. The video data stored in the video data storage device 79 can be obtained, for example, from the storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0060] During the decoding process, the video decoder 30 receives a encoded video bitstream representing video blocks and associated syntax elements of encoded video frames. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0061] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-predictive unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0062] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0063] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within a reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0064] The motion compensation unit 82 and / or the intra-frame BC unit 85 determine the prediction information for the video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0065] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0066] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0067] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be directly provided to the DPB 92 by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., ...). Figure 1 On the display device 34).

[0068] As described above, in related technologies, hardware encoders have unsatisfactory bitrate allocation strategies for different video scenarios and cannot adaptively allocate bitrate according to the video scenario.

[0069] Specifically, taking video transition scenarios as an example, scene transitions are a common video scenario. The human eye is highly sensitive to the image quality of such scenes. However, because the content of the transitioned scene has no informational connection with the preceding frame, it is difficult to utilize temporal correlation for encoding. Therefore, a large amount of code is usually required. In related technologies, the bitrate allocation strategies of mainstream hardware encoders in such scenarios are not ideal, affecting the subjective image quality experience and incurring high transmission bandwidth costs.

[0070] Furthermore, hardware encoders can only be configured and used through the peripheral interfaces provided by the official website. The internal implementation mechanism is a black box, with limited adjustable parameters, low algorithm flexibility and accuracy, and the inability to customize bitrate allocation strategies for different scenarios according to requirements.

[0071] Taking the multimedia codec framework VideoToolbox as an example, it is mainly used for hardware-accelerated audio and video encoding and decoding on iOS and Mac operating system platforms. It provides developers with a set of application programming interfaces (APIs) and tools, enabling applications to utilize the device's hardware acceleration capabilities for efficient audio and video processing.

[0072] However, this multimedia codec framework exposes limited peripheral code control interfaces, resulting in limited adjustment space and low flexibility. Furthermore, it cannot recognize scenarios requiring high bitrates, such as scene transitions, and its adaptive bitrate allocation based on varying video complexity is limited.

[0073] To address at least some of the aforementioned problems, exemplary embodiments of this disclosure provide a video encoding method, a video encoding apparatus, a method for generating bitstreams, an electronic device, a computer-readable storage medium, and a computer program product. Reference will be made below. Figures 4 to 8 Provide a detailed description.

[0074] According to one aspect of an exemplary embodiment of the present disclosure, a video coding method is proposed that can be applied to any video coding scenario. An example implementation scenario of the video coding method according to an exemplary embodiment of the present disclosure is given below.

[0075] For example, when a user requests video content to watch from a server or video distribution platform via a network using a video application client on their user terminal, the server or video distribution platform can send the requested video content to the user terminal via the network, and the user can watch the received video content through the video application client.

[0076] In the above process, the server or video distribution platform may use a video encoding method according to an exemplary embodiment of the present disclosure to encode the video content to be sent.

[0077] It should be noted that although the above description uses a server or video distribution platform as an example, it is only an example. The execution subject of this video encoding method can be any electronic device. The electronic device can include physical devices such as smartphones, tablets, laptops, digital assistants, wearable devices, and in-vehicle terminals as hardware codecs, or software running on physical devices as software video encoders.

[0078] It should also be noted that although the video encoding application scenario of a user terminal requesting to watch a video is used as an example here, it should be understood that the application scenarios of the video encoding method according to this disclosure are not limited to this, and it can also be applied to any other application scenario involving video encoding.

[0079] Figure 4 This is a flowchart illustrating a video encoding method according to an exemplary embodiment. As described above, the video encoding method according to an exemplary embodiment of this disclosure can determine the encoding bitrate by combining video scene complexity analysis, solving the problem of unsatisfactory bitrate allocation for different video scenes, thereby adaptively allocating the bitrate according to the video scene and improving the subjective image quality experience of the video in different video scenes.

[0080] like Figure 4 As shown, a video encoding method according to an exemplary embodiment of this disclosure may include the following steps: In step S410, the target video frame in the video to be encoded can be determined.

[0081] In this step, as an example, the target video frame can be any video frame in the video to be encoded, such as the current video frame to be encoded during the process of encoding the video frame by frame.

[0082] As another example, the target video frame can be a transition frame in the video to be encoded. Specifically, scene transitions may be video scenes that are highly sensitive to image quality. During the encoding process, focusing on the image quality of the transition frame or several frames after the transition frame can better allocate the encoding bitrate and improve the overall image quality experience of the video.

[0083] For example, a target video frame can be determined as follows: A video frame in the video to be encoded whose difference between inter-frame complexity and intra-frame complexity exceeds a difference threshold is identified as the target video frame. Here, inter-frame complexity can represent the information that the current video frame needs to encode with reference to a reference video frame; intra-frame complexity can represent the information that the current video frame needs to encode with reference to itself, where the reference video frame can be a video frame preceding the current video frame. The reference video frame can be any video frame preceding the current video frame, such as the frame before the current video frame.

[0084] Specifically, intra-frame prediction and inter-frame prediction can be performed on the current video frame using coding techniques. The codewords and distortion required to encode the current video frame under each prediction mode are calculated, thus obtaining the aforementioned intra-frame complexity and inter-frame complexity, respectively. Here, the coding complexity of a video frame can characterize the amount of information that needs to be encoded in the video frame. The complexity can be represented, for example, but not limited to, by SAD (Sum of Absolute Difference) or SATD (Sum of Absolute Transformed Difference). Here, for example, but not limited to, typical prediction methods in HEVC can be used to perform intra-frame prediction and inter-frame prediction. As examples, intra-frame prediction methods can be, but not limited to, 33 angle modes, DC mode, planar mode, etc., while inter-frame prediction methods can be, but not limited to, motion estimation, motion compensation, etc.

[0085] After entering the formal encoding process, the video frames in the video to be encoded are sent to the encoder one by one, such as... Figure 5 As shown, before the frame to be encoded is sent to the encoder, the bitrate allocation model of the hardware encoder can calculate the intra-frame prediction complexity information (intra_cost) and inter-frame prediction complexity information (inter_cost) of the current frame in real time, and can also calculate the difference between the inter-frame complexity and the intra-frame complexity. Here, in Figure 5 In this model, the bitrate allocation model can include a bitrate decision module. The bitrate decision module can obtain the target video frame detected by the scene detection module and the video scene information in which the target video frame is located, such as the transition frame and the transition scene. The bitrate decision module can also determine the configuration bitrate for encoding the current video frame based on the complexity information of the current video scene obtained from real-time complexity analysis (or based on the complexity distribution information described below).

[0086] Here, the complexity difference between inter-frame complexity and intra-frame complexity can be represented, for example, but not limited to, by a ratio, difference, or other mathematical means. As an example, it can be represented by the ratio in the following equation (1). ratio To characterize the difference in complexity: (1) in, inter_cost It represents the inter-frame complexity, which indicates the information that the current video frame needs to encode with reference to a reference video frame. intra_cost Intra-frame complexity represents the information that the current video frame needs to encode with itself as a reference.

[0087] For equation (1), the ratio ratio The larger the value, the less information the current video frame can reference from the reference video frame, the greater the scene difference between the two frames, the higher the inter-frame complexity, the lower the correlation between the current video frame and the reference video frame, and the greater the probability that the current video frame is a transition frame. However, the way to represent the complexity difference is not limited to equation (1), for example, it can also be expressed as a ratio. ratio It is represented by the reciprocal of .

[0088] As an example, the current video frame can be determined to be a transition frame if the complexity difference exceeds a difference threshold; or it can be determined to be a non-transition frame if the complexity difference does not exceed a difference threshold.

[0089] Specifically, when the inter-frame complexity is significantly greater than the intra-frame complexity, such as exceeding a difference threshold, it indicates that using a preceding reference video frame as a reference does not yield higher coding performance compared to referencing only itself. This suggests that the current video frame has low correlation with its preceding reference video frames, thus classifying it as a transition frame. Determining transition frames by analyzing complexity differences allows for more accurate detection of transition scenes in the video. This serves as the basis for bitrate enhancement of transition scenes, guiding subsequent more rational allocation of coding bitrate.

[0090] As an example, the difference threshold can be a configurable parameter, which can be an empirical value based on industry standards. Taking the above equation (1) as an example, according to the Scenecut decision algorithm in High Efficiency Video Coding (HEVC), the difference threshold scenecut_th can be set. This difference threshold scenecut_th can be, for example, but not limited to, 40%. Furthermore, in the embodiments of this disclosure, in order to avoid a large increase in the final encoded bitrate, the difference threshold scenecut_th can be set to, for example, 70%. If the ratio in the above equation (1) is... ratioIf the difference threshold `scenecut_th` is exceeded, the scene difference between the two frames can be considered significant, thus indicating a transition has occurred. However, the difference threshold is not limited to a constant; it can also change dynamically during the encoding process, or more specifically, during the detection of transition frames. For example, according to embodiments of this disclosure, in determining whether the current video frame is a transition frame, the video encoding method may further include: updating the difference threshold; and based on the updated difference threshold, determining whether the third video frame following the current video frame is a transition frame.

[0091] Here, the step of updating the difference threshold may include: increasing the difference threshold in response to the number of transition frames determined within a video frame sequence of a preset window size being greater than a first preset number of transition frames; and decreasing the difference threshold in response to the number of transition frames determined within a video frame sequence of a preset window size being less than a second preset number of transition frames, wherein the second preset number of transition frames is less than or equal to the first preset number of transition frames.

[0092] Specifically, transition frames can be detected for each video frame in the order of the video to be encoded. An initial difference can be set as the difference threshold for initial detection; for example, the difference threshold `scenecut_th` can be set to `scenecut_th_base`. During the sequential detection of the video frame sequence, if the number of detected transition frames within a preset detection window is greater than a first preset number of transition frames, the difference threshold can be increased. For example, the initial difference `scenecut_th_base` can be increased to an increased difference threshold `scenecut_th_max`. This increased difference threshold is then used to detect transition frames in subsequent video frames, thus strengthening the judgment condition for transition frames and avoiding excessive detection. The difference threshold can be decreased until the number of detected transition frames within the detection window is less than a second preset number of transition frames. For example, the increased difference threshold `scenecut_th_max` can be adjusted back to the initial difference threshold `scenecut_th_base`. This decreased difference threshold is then used to detect transition frames in subsequent video frames.

[0093] The main reason for dynamically adjusting the difference threshold is that the judgment of the transition frame will affect the number of bitrate boosts. Excessive boosts can easily lead to a large increase in the final bitrate. Here, the preset window size, the first preset number of transition frames and the second preset number of transition frames can be set according to actual needs. For example, for the above formula (1), the initial difference scenecut_th_base can be set to 70%, the increased difference threshold scenecut_th_max can be 85%, the preset window size can be, for example, but not limited to 20 frames, the first preset number of transition frames can be, for example, but not limited to 4 frames, and the second preset number of transition frames can be, for example, but not limited to 3 frames. In this way, bitrate boosting can be performed only when the difference in the picture is extremely large or when the bitrate is more likely to be insufficient.

[0094] For example, if more than the first preset number of transition frames (b) are detected within the range of frame a after bitrate boosting (e.g., the detection window), it can be assumed that the current scene is undergoing continuous and drastic changes. In this case, the difference threshold can be increased to reduce the number of detected transitions and control the number of bitrate boosts. After increasing the difference threshold, if the number of detected transition frames within the range of frame a after bitrate boosting (e.g., the detection window) is lower than the second preset number of transition frames (c), it can be assumed that the current scene has stabilized. In this case, the difference threshold can be decreased, for example, by adjusting the difference back to the initial difference.

[0095] By employing the above method, the detection threshold for transition frames can be dynamically adjusted, avoiding frequent bitrate increases caused by excessive detection of transition frames, thus reducing computational pressure and communication burden. Furthermore, setting the second preset number of transition frames to be less than the first preset number of transition frames avoids the possibility of repeated jumps in the difference threshold when the two are equal, thereby improving the stability of the detection process.

[0096] The process of determining the target video frame described above can be, for example, by: Figure 5 The scene detection module in the video can be used to achieve this. For example, a series of scene switching frame information can be obtained based on the above transition detection method, that is, a series of transition frames can be obtained as target video frames.

[0097] In step S420, the complexity of the current video scene corresponding to the target video frame can be determined.

[0098] In this step, the current video scene corresponding to the target video frame can refer to a video segment that includes the target video frame. In one example, the target video frame can be any frame in its corresponding video scene.

[0099] In another example, a video segment starting from the target video frame and including a first preset number of frames can be used as the video scene corresponding to the target video frame.

[0100] In this example, the complexity of the current video scene can be determined as follows: by statistically analyzing the complexity of video frames within a first preset number of frames starting from the target video frame, the complexity of the current video scene can be obtained. Here, the complexity of a segment of video, including the target video frame, can be determined, thereby comprehensively judging the complexity level of the video scene in which the target video frame is located, and thus more reasonably determining the complexity of the target video frame. However, the embodiments of this disclosure are not limited to this; the complexity of the target video frame can also be used as the complexity of the current video scene.

[0101] Here, the inter-frame complexity of a video frame can be, for example, the information that a video frame needs to encode with reference to the previous frame. Statistical analysis of the complexity of video frames within a first preset number of frames starting from the target video frame can be, for example, by averaging, or by using other statistical methods, such as calculating the standard deviation or variance.

[0102] The first preset frame count can be set according to actual needs. In one example, the first preset frame count can be less than or equal to the maximum video frame capacity of the lookahead buffer. Specifically, during the encoding process of the video to be encoded, each video frame is stored sequentially in the lookahead buffer. The video frames in the lookahead buffer are encoded in a first-in-first-out (FIFO) order. Therefore, when the first preset frame count is less than or equal to the maximum video frame capacity of the lookahead buffer, the video frames in the lookahead buffer (e.g., all video frames) can be directly used to calculate the above statistical values, which facilitates statistical calculation.

[0103] In step S430, the encoding bitrate of the target video frame can be determined based on the complexity of the current video scene.

[0104] As an example, the encoding bitrate of the target video frame can be obtained by increasing the original encoding bitrate of the video to be encoded, depending on the complexity of the current video scene.

[0105] For example, in response to the fact that the complexity of the current video scene is higher than the first preset value, the encoding bitrate of the target video frame can be obtained by increasing the original encoding bitrate of the video to be encoded.

[0106] The original bitrate of the video to be encoded can be preset, such as the desired bitrate. This original bitrate can be used as a benchmark for bitrate upscaling the target video frame, and upscaling can be performed on this basis to obtain the bitrate of the target video frame.

[0107] Using the above method, when the encoding complexity of the current video scene is relatively high, a process can be performed to increase the original encoding bitrate to obtain the encoding bitrate of the transition frame. This allows for targeted improvement of the encoding bitrate of the target video frame, ensuring its image quality. For example, by combining the complexity information of the video scene in which the target video frame is located, a complexity threshold can be set. Cplx_th If the complexity of the current scene Cplx_cur Above this complexity threshold Cplx_th This will increase the encoding bitrate of the current frame to achieve better encoding quality. In the example where the target video frame is a transition frame, the current scene complexity is... X i It can reflect the complexity of the visuals at the beginning of a new scene after a transition.

[0108] Here, the first preset value can be set according to actual needs, or it can be determined according to the block complexity or block size of the transition frame.

[0109] Specifically, since frame complexity is related to frame resolution, the more coded blocks a frame contains, the greater the complexity information. Therefore, block complexity is also related to block size; the larger the block size, the more complexity information is contained, and correspondingly, the preset complexity... Cplx_th It can also be related to the block size. For example, referring to the initialization method of code control model complexity in HEVC, the preset complexity can be... Cplx_th It can be expressed by the following formula (2): (2) in, and This represents a preset coefficient, which can be set according to actual needs, for example... It can be 7000. It can be 5; This indicates the number of blocks in the transition frame.

[0110] As an example, ,in, This indicates the number of blocks in the width direction within the transition frame. This indicates the number of blocks in the height direction within the transition frame. For example, and This can be the number of coding units (e.g., 8×8 CUs) contained in the width and height directions of a video frame.

[0111] In this example, in some cases, the encoding bitrate of the target video frame can be determined based on a preset multiple of the original encoding bitrate of the video to be encoded, according to the complexity of the current video scene. This allows for the allocation of more bitrate to complex video scenes, thereby improving the overall subjective quality of the video.

[0112] In other cases, different levels of bitrate compensation can be applied to each video scene that meets the bitrate compensation criteria, allowing for a more reasonable allocation of bitrate based on the complexity distribution throughout the sequence. Specifically, methods such as... Figure 5 The pre-analysis module shown first obtains the approximate complexity distribution of the video as a whole, and then adjusts the bitrate increase for the corresponding video scene according to the complexity distribution. More bitrate is compensated for relatively complex scenes, and less bitrate is compensated for relatively simple scenes, in order to save the overall encoding bitrate.

[0113] As an example, according to an exemplary embodiment of this disclosure, in response to the current video scene having a complexity higher than a first preset value, the encoding bitrate of the target video frame can be obtained by increasing the original encoding bitrate of the video to be encoded, based on the complexity and complexity distribution information of the current video scene.

[0114] Specifically, the complexity distribution information of the video to be encoded can be determined, and the encoding bitrate of the target video frame can be determined based on the complexity and complexity distribution information of the current video scene.

[0115] As an example, the encoding bitrate of the target video frame can be obtained by increasing the original encoding bitrate of the video to be encoded, based on the complexity and complexity distribution information of the current video scene.

[0116] Here, the complexity distribution information of the video to be encoded can characterize the complexity distribution of different video frames. In one example, this complexity distribution information can be determined by pre-analyzing the complexity of each frame in the video to be encoded.

[0117] According to embodiments of this disclosure, in order to improve encoding speed and reduce encoding computation while determining complexity distribution information, in another example, complexity distribution information can be determined by: determining the complexity of the second video frame after the first video frame in the video to be encoded; and determining complexity distribution information based on the complexity of multiple first video frames and multiple second video frames in the video to be encoded.

[0118] Here, the first video frame can be an intra-coded frame (I-frame), and the second video frame can be a forward-predictive frame (P-frame) that is a preset number of frames after the first video frame.

[0119] Specifically, such as Figure 5As shown, for an input video, it can first be pre-analyzed by a pre-analysis module to obtain the distribution of content complexity in the video. Ideally, direct frame-by-frame analysis of the video can obtain the most accurate complexity distribution, but frame-by-frame analysis brings huge computational complexity, which will seriously affect the encoding and output time of the first frame, thus affecting the user experience.

[0120] To mitigate the increased computational complexity, the complexity can be calculated only for the I-frame and one or more P-frames following it. The rationale behind this design is that the I-frame is an intra-predictive frame, its encoding using itself as a reference only and not depending on other frames. Therefore, it can be used as a random access frame, meaning that the I-frame can be decoded at any time without needing to decode preceding frames, significantly reducing computational latency. Furthermore, to obtain the temporal complexity of the content, P-frames following the I-frame at a predetermined number of frames can also be decoded, such as a P-frame immediately after the I-frame. This P-frame can be decoded normally using only the preceding I-frame as a reference, thus yielding the corresponding intra-frame and inter-frame complexities. Here, the number of P-frames following the I-frame can be any integer greater than or equal to 1.

[0121] Furthermore, since the interval between I-frames in most videos is fixed, decoding similar combinations of IP frames (i.e., combinations of I-frames and one or more P-frames following I-frames) in the video is equivalent to performing a uniform sampling of the video, which can obtain the approximate complexity distribution information of the entire video.

[0122] The intra-frame complexity and inter-frame complexity mentioned above refer to the encoding cost of the frame to be encoded after being encoded using intra-frame prediction or inter-frame prediction. The higher the encoding cost, the higher the complexity. Here, typical prediction methods from the HEVC standard can be used to implement intra-frame and inter-frame prediction. Furthermore, as an example, if the number of I-frames in the video to be encoded is small or their distribution is uneven, video pre-analysis can be omitted, or the pre-analysis information can be considered unusable and discarded. Testing showed that the sampled and parallel-processed video pre-analysis module has virtually no perceptible impact on encoding speed.

[0123] As an example, based on the complexity and complexity distribution information of the current video scene, the target video frame's encoding bitrate can be obtained by increasing the original encoding bitrate of the video to be encoded. This can include: determining an enhancement factor for increasing the original encoding bitrate based on the complexity and complexity distribution information of the current video scene; and obtaining the target video frame's encoding bitrate based on the enhancement factor and the original encoding bitrate. In this way, the complexity level of the current video scene can be measured by combining the overall complexity distribution of the video, thus allowing for a more reasonable determination of the extent to which the original encoding bitrate should be increased.

[0124] Specifically, after meeting the conditions for increasing the bitrate, the original encoded bitrate can be weighted based on an increase factor, thereby increasing the encoded bitrate of the target video frame to a multiple of the original bitrate. In one example, a table of increase factors corresponding to the complexity of different video scenes and the distribution of different complexity can be pre-set. The increase factor corresponding to the complexity of the current video scene and the current complexity distribution can be obtained by looking up the table, and thus the original encoded bitrate can be increased using the increase factor to obtain the encoded bitrate of the target video frame.

[0125] In another example, the complexity distribution information may include a maximum value and a minimum value of the complexity distribution, wherein the maximum value of the complexity distribution is determined based on the maximum value of the complexity of multiple first video frames and multiple second video frames, and the minimum value of the complexity distribution is determined based on the minimum value of the complexity of multiple first video frames and multiple second video frames.

[0126] Specifically, it can be derived from, for example Figure 5 The pre-analysis module shown obtains the complexity of all first video frames (e.g., I-frames) and the second video frames (e.g., P-frames) following each first video frame, and calculates the maximum value among them. and minimum value .

[0127] As an example, the minimum value mentioned above can be... and the above complexity threshold Cplx_th The smaller of the two is taken as the minimum value of the complexity distribution, i.e. = max( , Cplx_th Furthermore, as an example, the maximum value mentioned above can be... The complexity of the detected target video frames The larger of the two is taken as the maximum value of the complexity distribution, i.e. = max( , During the real-time complexity calculation of the target video frame, the response to the complexity of the target video frame... X i If the maximum value is greater than the current complexity, it can be updated. = .

[0128] In this example, the boost factor can be determined as follows: based on the maximum value of the complexity distribution, the minimum value of the complexity distribution, and the complexity of the current video scene, the preset factor is adjusted to obtain the boost factor, where the preset factor represents the maximum multiple by which the original coding bitrate is increased.

[0129] Here, the enhancement factor for the current video scene can be determined by combining the complexity of the current video scene and the complexity distribution information in the pre-analysis. For example, the enhancement factor can be determined by the following formula (3). n : (3) in, Cplx_cur Given the complexity of the current video scene, and These are the maximum and minimum values ​​of the complexity distribution, respectively. N The maximum boost factor is preset and can be set according to actual needs. Here, under the constraint of the above equation (3), the boost factor is... n The value can be in the range of [0.5N, N].

[0130] In the above method, by setting a maximum and minimum complexity value, and based on the relationship between the complexity of the current video scene and this maximum and minimum value, the preset maximum boost factor can be reasonably adjusted, quantifying the degree of bitrate boost that matches the complexity of the current video scene, thereby allowing for more flexible bitrate boosting and allocation.

[0131] Although the above reference formula (3) describes an example of how to determine the boost coefficient, it is not limited to this. Any expression that can use the above maximum value, minimum value, and current scene complexity to adjust the preset coefficient can replace the above formula (3) to calculate the boost coefficient.

[0132] In embodiments of this disclosure, the video to be encoded may include multiple target video frames, and at least two target video frames may have different boosting factors. The boosting factor may be positively correlated with the complexity of the video scene corresponding to the target video frame. Specifically, by employing the method of this disclosure, the encoding bitrate of the target video frames can be increased according to the complexity of the video scene, thereby allowing for a more reasonable design of the encoding bitrate for different target video frames and improving the image quality of target video frames with higher complexity.

[0133] In step S430, as another example, in response to the fact that the complexity of the current video scene and the complexity of the first second video frame after the target video frame are both lower than the second preset value, the encoding bitrate of the target video frame can be obtained by reducing the original encoding bitrate, wherein the first preset value is greater than the second preset value.

[0134] Specifically, for a target video frame, if the complexity of its surrounding video scene and the complexity of its immediate second-to-last video frame are both lower than a second preset value, it indicates that the overall complexity of the current video scene is relatively low. Therefore, the encoding bitrate for that scene can be reduced to achieve reasonable bitrate adjustment and allocation. For example, if the target video frame is identified as a transition frame, and the complexity of both the current transition frame and the most recent second-to-last video frame are lower than the second preset value... Cplx_static_th This indicates that the overall complexity of the scene after the transition may be low, and the encoding bitrate of the scene can be appropriately reduced.

[0135] As an example, the second preset value can be determined by the following formula (4). Cplx_static_th : (4) in, This represents the preset coefficient, which can be set according to actual needs, for example, it can be 5000.

[0136] In this example, for instance, a reduction coefficient could be used. m The original encoded bitrate is weighted, and the encoded bitrate of the target video frame is configured to be BR times the original encoded bitrate. m times, of which, m <1, m Configurable parameters, for example m It can be set to 0.7.

[0137] In step S430, as another example, in response to the current video scene complexity being lower than a first preset value and higher than a second preset value, the original encoding bitrate is determined as the encoding bitrate of the target video frame.

[0138] In this example, if the complexity of the current video scene is lower than the first preset value but higher than the second preset value, the complexity level of the current scene can be considered to be at a medium level, and there is no need to adjust the encoding bitrate of the target video frame.

[0139] By employing the methods described above, different strategies can be adopted to determine the encoding bitrate of the target video frame when the complexity of the video scene varies. This aims to minimize the encoding bitrate, reducing computational load and communication burden while ensuring video quality. However, the embodiments of this disclosure are not limited to this; the above-described examples of increasing the bitrate can also be applied directly without determining the complexity of the scene, thereby optimizing video quality as much as possible.

[0140] In step S440, the target video frame can be encoded based on the encoding bitrate of the target video frame.

[0141] In this step, the target video frame can be encoded using, for example, existing encoding methods, based on the encoding bitrate of the target video frame.

[0142] The above describes an example process for determining the encoding bitrate of a target video frame based on the complexity of the video scene in which the target video frame is located. In addition, in the embodiments of this disclosure, the encoding bitrate of multiple related video frames following the target video frame can also be determined in a similar manner as described above.

[0143] Specifically, the video encoding method may further include: based on the original encoding bitrate of the video to be encoded, obtaining the encoding bitrate of multiple related video frames after the target video frame by increasing the original encoding bitrate; and encoding the multiple related video frames based on their respective encoding bitrates.

[0144] Here, the encoding bitrate of the target video frame is greater than or equal to the encoding bitrate of multiple related video frames, and the encoding bitrate of the preceding video frame in any adjacent video frame among the multiple related video frames is greater than or equal to the encoding bitrate of the following video frame in the adjacent video frames.

[0145] In this example, the bitrate of the preceding video frame in any adjacent video frame among the target video frame, such as a transition frame, and multiple related video frames is greater than or equal to the bitrate of the following video frame in the adjacent video frame.

[0146] Specifically, because the human eye has limited ability to distinguish consecutive video frames, when watching a video, the target video frame (such as a transition frame) and several subsequent video frames may be perceived as the same video scene. Therefore, the bitrate of the target video frame and several subsequent video frames can be increased. Since video frames closer to the target video frame (e.g., transition frames) are likely to have a stronger correlation with the video scene (e.g., the transition scene) and vice versa, the bitrate of related video frames closer to the target video frame can be increased more significantly. The increase in bitrate can be gradually reduced as the distance between related video frames and the target video frame increases, causing the bitrate to gradually decrease from the target video frame until it returns to the original bitrate. In this way, the bitrate of a sequence of video frames starting from the target video frame can be increased, improving the overall viewing experience of the video scene.

[0147] As an example, the bitrate of the target video frame and multiple related video frames can be gradually reduced from the bitrate of the target video frame to the original bitrate with a preset decay step size. For example, when the bitrate of the target video frame is increased to n times the original bitrate, the bitrate of each related video frame after the target video frame can be represented by the following equations (5) and (6): (5) (6) in, This indicates the bitrate of the current frame (i.e., the target video frame or related video frame). BR Indicates the original encoded bitrate. n The aforementioned boost factor represents the percentage increase in the bitrate of the target video frame compared to its original bitrate. n times, L This indicates a preset sliding window. Indicates the decay step size. dist This indicates the frame number between the current frame and the target video frame. It can be the difference between the frame number of the current frame and the frame number of the target video frame. floor This indicates rounding down. Equations (5) and (6) above indicate rounding down within a sliding window. L Within, the encoding bitrate of related video frames after the target video frame is determined according to the attenuation step size. It gradually returns to the original coding rate in a step-like manner.

[0148] By reducing the bitrate of video frames in the current video scene back to the original bitrate in a stepwise manner, the bitrate of video frames within the same attenuation step is the same. There is no need to calculate the bitrate for each frame. Therefore, the amount of coding computation can be reduced as much as possible while improving the coding quality of multiple video frames close to the target video frame, such as a transition frame.

[0149] Although the above describes a step-like attenuation method with a preset attenuation step size to gradually reduce the encoding bitrate of the target video frame and multiple related video frames, the embodiments of this disclosure are not limited to this, and other attenuation methods may also be used, such as linear or non-linear attenuation.

[0150] In addition, in the example of determining the encoding bitrate of the target video frame by reducing the original encoding bitrate, the encoding bitrate of multiple related video frames after the target video frame can also be determined in a manner similar to that shown in Equation (5) above, so that the bitrate can be gradually restored (specifically, increased) to the original encoding bitrate after the bitrate is reduced.

[0151] For example, as mentioned above, the reduction coefficient can be utilized. m The original encoded bitrate is weighted, and the encoded bitrate of the target video frame is configured to be BR times the original encoded bitrate. m The bitrate of the target video frame can be determined by the following formula (7): (x times). (7) Furthermore, the above describes an example process for increasing the encoding bitrate of the detected target video frame. As an example, in the embodiments of this disclosure, the number or frequency of bitrate increases for the target video frame can also be limited to eliminate interference from some noisy scenarios where bitrate increases are unnecessary. In such noisy scenarios, multiple target video frames may be detected frequently. By introducing a bitrate increase limiting mechanism, the frequent bitrate increases can be avoided, which could lead to an excessively high final output bitrate.

[0152] In one example, flickering scenes can be detected. Specifically, the video encoding method may further include: encoding the current target video frame based on the original encoding bitrate in response to the number of target video frames determined within a second preset number of frames starting from the current target video frame being greater than or equal to a preset number; and performing a step of increasing the encoding bitrate of the target video frame by increasing the original encoding bitrate in response to the number of target video frames determined within a second preset number of frames starting from the current target video frame being less than a preset number.

[0153] Specifically, when a target video frame is detected, it can be determined whether the number of target video frames contained within the second preset number of frames starting from the target video frame is greater than a preset number. If it is greater than the preset number, it can be considered that the current target video frame may belong to a flickering scene. Since the human eye has a weak visual discrimination of flickering scenes, the bitrate of the target video frames in the flickering scene can be not increased, thereby reducing the amount of encoding calculation and bandwidth cost.

[0154] Here, the second preset frame number and preset quantity can be set according to actual needs. For example, the second preset frame number can be, but is not limited to, 3 frames, and the preset quantity can be, but is not limited to, 2.

[0155] As an example, a pre-read buffer can be used to implement the above-mentioned flicker scene detection. Specifically, as mentioned above, during the encoding process of the video to be encoded, each video frame is stored sequentially into the pre-read buffer, and the video frames in the pre-read buffer are encoded according to the FIFO order. Here, the maximum video frame capacity of the pre-read buffer can be the second preset number of frames mentioned above.

[0156] Thus, after performing complexity analysis on the input frames, the complexity information of each frame can be stored sequentially in the pre-read buffer. Once the pre-read buffer is full, encoding can be performed starting from the earliest inserted frame according to the FIFO order. If multiple target video frames are detected in the pre-read buffer, it indicates that the current target video frame is in a flickering scenario, and the bitrate of the current target video frame can be left unenhanced.

[0157] Specifically, in response to the number of target video frames in the pre-read buffer being greater than or equal to a preset number, the current target video frame is encoded based on the original encoding bitrate; in response to the number of target video frames in the pre-read buffer being less than the preset number, for the current target video frame, the step of obtaining the encoding bitrate of the target video frame by increasing the original encoding bitrate is performed.

[0158] Furthermore, as an example, the second preset frame number may be greater than or equal to the first preset frame number described above. It can be seen that, in the embodiments of this disclosure, the first preset frame number and the second preset frame number can be determined with reference to the maximum video frame capacity of the pre-read buffer, thereby limiting the number of bitrate boosts in different ways, such as by limiting the inter-frame complexity statistics within the first preset frame number and by limiting the number of target video frames such as transition frames detected within the second preset frame number.

[0159] Additionally or alternatively, in another example, the number or frequency of bitrate boosting can be limited by setting a minimum frame interval for bitrate boosting. Specifically, the video encoding method may further include: in response to the current target video frame being more than a preset minimum frame interval from a reference target video frame, performing a step of increasing the original encoding bitrate to obtain the encoding bitrate of the current target video frame; in response to the current target video frame being less than or equal to the preset minimum frame interval from a reference target video frame, encoding the current target video frame based on the original encoding bitrate, or encoding the current target video frame based on the distance to the reference target video frame to determine the encoding bitrate for the current target video frame.

[0160] Here, the reference target video frame can be the target video frame preceding the current target video frame. Specifically, after triggering a bitrate boost on the target video frame, the frame distance for the next bitrate boost can be greater than the preset minimum frame interval. Here, the frame interval refers to the number of video frames between two video frames (excluding the two endpoint video frames). The preset minimum frame interval can be set according to actual needs, for example, it can be 20 frames.

[0161] As an example, the preset minimum frame interval can be greater than the number of multiple related video frames after the target video frame, for example, it can be greater than the preset sliding window described above. L Decrease by 1. In this way, when performing bitrate boosting on related video frames after the target video frame, it can avoid the inability to correctly perform bitrate boosting on related video frames due to the simultaneous setting of a preset minimum frame interval limit.

[0162] In addition, the number of related video frames following the target video frame where the number of frames between the current target video frame and the reference target video frame is less than or equal to the preset minimum frame interval and greater than the target video frame (e.g., the above)L In the case of reducing by 1), the current target video frame can be encoded based on the original encoding bitrate.

[0163] The number of frames between the current target video frame and the reference target video frame is less than the preset minimum frame interval and less than the number of multiple related video frames following the reference target video frame (e.g., the above). L In the case of subtracting 1), the current target video frame can be one of the related video frames after the reference target video frame. In this case, the encoding bitrate of the current target video frame can be determined based on the number of frames between the current target video frame and the reference target video frame, using the method of increasing the encoding bitrate of related video frames described above. For example, the encoding bitrate of the current target video frame can be determined by the above formula (4), where the number of frames between the current target video frame and the reference target video frame is the value in formula (4). dist This allows the current target video frame to be encoded based on the determined encoding bitrate.

[0164] Additionally or alternatively, in another example, the video encoding method may further include: in response to the number of times the step of obtaining the encoding bitrate of the target video frame by increasing the original encoding bitrate is performed within a first preset video frame range being greater than a first preset number of times, encoding the target video frame within a second preset video frame range based on the original encoding bitrate.

[0165] Here, the first preset video frame range can be greater than the aforementioned preset minimum frame interval. Specifically, after triggering a bitrate boost, if a first preset number of bitrate boosts are triggered within the first preset video frame range, then no further bitrate boosts will be performed within the second preset video frame range. Here, the first and second preset video frame ranges can be empirical values, for example, the first preset video frame range can be 50 frames, and the second preset video frame range can be 100 frames. Thus, even when the minimum frame interval is met, multiple compensations may occur, such as multiple boosts within a 50-frame range. In this case, bitrate boosts within a certain video frame range (e.g., 100 frames) can be stopped to further limit the number of bitrate boosts and reduce computational and communication burdens.

[0166] When the target video frame is a transition frame, some videos may contain a large number of transitions that meet the conditions within a certain period of time. The large number of transitions mentioned here does not refer to short-term flickering, but to multiple transitions over a longer period of time, such as frequent camera switching in e-commerce product displays. In this case, by adopting the above-mentioned restriction method, subsequent transition scenes can be sacrificed for a certain period of time, such as transition scenes within the range of the second preset video frame mentioned above, to avoid a sudden increase in bitrate and control the encoding calculation of the entire video.

[0167] Additionally or alternatively, in another example, the video encoding method may further include: in response to the second preset number of times the step of obtaining the encoding bitrate of the target video frame by increasing the original encoding bitrate has been performed within a time window of a preset length, encoding subsequent target video frames within the current time window based on the original encoding bitrate until the next time window is reached.

[0168] Specifically, you can set the maximum number of bitrate boosts within a preset time window. If the number of bitrate boosts reaches the threshold limit within the time window, no further bitrate boosts will be performed until the next unit time window opens.

[0169] In this way, the number of bitrate boosts can be limited from a time perspective. It can be used in conjunction with the above-mentioned limiting mechanisms to limit bitrate boosts from different dimensions.

[0170] It should be noted that the bitrate boosting mechanisms described above can be used individually or in any combination. Furthermore, all variables in each bitrate boosting mechanism have external setting interfaces, allowing for configuration of different parameter combinations based on varying scenario requirements through upper-layer control.

[0171] Furthermore, as an example, in the embodiments of this disclosure, the prediction algorithm can be optimized by fast algorithm and assembly during the process of determining intra-frame complexity and inter-frame complexity, thereby reducing the overall complexity of bitrate allocation and ensuring that the prediction algorithm maintains high accuracy without introducing excessive computational power consumption.

[0172] Here, fast algorithms can include intra-frame fast algorithms and inter-frame fast algorithms. Intra-frame fast algorithms can reduce the number of prediction modes, for example, including but not limited to reducing the prediction modes to five directions: bottom left, left, top left, top, and top right, as well as DC mode and Planar mode; inter-frame fast algorithms can reduce the number of reference frames, for example, each frame can only refer to the previous frame for motion estimation.

[0173] In addition, fast algorithms may also include down-resolution processing of video frames, such as reducing the length and width of video frames to 1 / 2, 1 / 4 or other proportions of the original size according to preset configuration, in order to reduce the amount of computation.

[0174] In addition, fast algorithms may also include reducing the inter-frame prediction search range. For example, during the inter-frame prediction search process, the algorithm may exit the inter-frame prediction search process in response to the inter-frame complexity of the coded block being less than a preset threshold.

[0175] Here, assembly optimization can include intra-frame assembly optimization and inter-frame assembly optimization. Intra-frame assembly optimization can, for example, but not limited to, performing arm64 assembly optimization on all prediction modes to increase computational parallelism. Inter-frame assembly optimization can, for example, but not limited to performing Single Instruction Multiple Data (SIMD) assembly optimization on operations such as pixel interpolation in motion estimation and motion compensation, which can also increase computational parallelism.

[0176] By incorporating fast algorithms and assembly optimizations, the computation of this method has no significant impact on the overall coding complexity, thus significantly improving the user experience when producing and consuming videos.

[0177] The above describes a video coding method according to exemplary embodiments of the present disclosure. This video coding method proposes to incorporate video pre-analysis and transition detection into the hardware encoder, which effectively improves the bitrate allocation efficiency in transition scenarios, saves bitrate in simple scenarios, improves the overall subjective image quality of the video, reduces the overall bitrate, and the model has no significant impact on the overall complexity, thus significantly improving the user experience when producing and consuming videos.

[0178] The video encoding method according to embodiments of this disclosure was verified on an iOS VideoToolBox hardware encoder. In 100 test sequences, the average bitrate decreased by 4.55%, and the subjective evaluation index KVQ (Kuaishou VideoQuality) increased by 0.0105, achieving a visually noticeable improvement in image quality. Simultaneously, some bad cases in transition scenes were improved, and the overall computational complexity was extremely low, with an increase of less than 4% compared to the VTB hardware encoder. This solves the pain point of commercially available hardware encoders being unable to adjust bitrate control to improve image quality according to scene requirements. Furthermore, this method can also be adapted to other hardware encoders besides VTB, such as, but not limited to, MediaCodec on the Android platform, as long as the hardware encoder supports a real-time bitrate setting interface.

[0179] In one application example employing the video encoding method according to an exemplary embodiment of the present disclosure, compared with the video output by the original encoder that does not employ the present method, the overall image quality of the video output by the encoder after bitrate enhancement by the above-described video encoding method can be observed to be improved.

[0180] Therefore, the video encoding method according to the embodiments of this disclosure can propose a content-adaptive encoder bitrate determination scheme, which can reduce the overall encoding bitrate while obtaining better subjective image quality by more reasonably allocating codewords to scenes with different complexities.

[0181] Furthermore, the video encoding method according to the embodiments of this disclosure can obtain the approximate distribution of the complexity of the entire video scene by introducing a simplified complexity pre-analysis module, which only performs complexity analysis on the first video frame and the subsequent second video frame in the video, so as to measure the overall complexity level of the video and thus more reasonably determine the encoding bitrate of the current target video frame.

[0182] Furthermore, the video encoding method according to the embodiments of this disclosure can use real-time complexity analysis and transition detection to provide more codewords for video scenes with complex visuals, such as transition scenes, which are sensitive to human eyes and have high bitrate requirements, thereby improving the subjective viewing experience of videos in multiple scenarios.

[0183] Furthermore, the video encoding method according to the embodiments of this disclosure can be combined with the obtained prior information on complexity to finely adjust the bitrate compensation range for transition scenes. For video scenes with relatively low complexity, the compensation intensity can be appropriately reduced, and the encoding bitrate can be lowered for simple scenes, thereby achieving overall bitrate savings.

[0184] Figure 6 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment. (Refer to...) Figure 6 The video encoding device 600 includes a first determining unit 610, a second determining unit 620, a third determining unit 630, and an encoding unit 640.

[0185] The first determining unit 610 is configured to determine the target video frame in the video to be encoded.

[0186] The second determining unit 620 is configured to determine the complexity of the current video scene corresponding to the target video frame.

[0187] The third determining unit 630 is configured to determine the encoding bitrate of the target video frame based on the complexity of the current video scene.

[0188] The encoding unit 640 is configured to encode the target video frame based on the encoding bitrate of the target video frame.

[0189] As an example, the third determining unit 630 is also configured to: determine the complexity distribution information of the video to be encoded; and determine the encoding bitrate of the target video frame based on the complexity and complexity distribution information of the current video scene.

[0190] As an example, the third determining unit 630 is also configured to: obtain the encoding bitrate of the target video frame by increasing the original encoding bitrate of the video to be encoded, based on the complexity and complexity distribution information of the current video scene.

[0191] As an example, the video encoding apparatus 600 further includes a fourth determining unit 650, which is configured to determine complexity distribution information by: determining the complexity of a second video frame following a first video frame in the video to be encoded, wherein the first video frame is an intra-predicted frame and the second video frame is a forward-predicted frame following the first video frame at a preset number of frames away from the second video frame; and determining complexity distribution information based on the complexity of multiple first video frames and multiple second video frames in the video to be encoded.

[0192] As an example, the third determining unit 630 is also configured to: determine an enhancement factor for increasing the original coding bitrate based on the complexity and complexity distribution information of the current video scene; and obtain the coding bitrate of the target video frame based on the enhancement factor and the original coding bitrate.

[0193] As an example, the complexity distribution information includes the maximum value and the minimum value of the complexity distribution. The maximum value of the complexity distribution is determined based on the maximum value of the complexity of multiple first video frames and multiple second video frames, and the minimum value of the complexity distribution is determined based on the minimum value of the complexity of multiple first video frames and multiple second video frames. The third determining unit 630 is also configured to determine the boosting coefficient by adjusting the preset coefficient based on the maximum value of the complexity distribution, the minimum value of the complexity distribution, and the complexity of the current video scene, to obtain the boosting coefficient, wherein the preset coefficient represents the maximum multiple by which the original coding bitrate is increased.

[0194] As an example, the video to be encoded includes multiple target video frames, and at least two target video frames have different boosting coefficients. The boosting coefficient is positively correlated with the complexity of the video scene corresponding to the target video frame.

[0195] As an example, the third determining unit 630 is further configured to: respond to the current video scene complexity being higher than a first preset value, perform the step of increasing the original encoding bitrate of the video to be encoded to obtain the encoding bitrate of the target video frame based on the complexity and complexity distribution information of the current video scene; respond to the current video scene complexity and the complexity of the first second video frame after the target video frame being lower than the second preset value, obtain the target video frame encoding bitrate by decreasing the original encoding bitrate; respond to the current video scene complexity being lower than the first preset value and higher than the second preset value, determine the original encoding bitrate as the encoding bitrate of the target video frame, wherein the first preset value is greater than the second preset value.

[0196] As an example, the second determining unit 620 is also configured to determine the complexity of the current video scene by statistically analyzing the complexity of video frames within a first preset number of frames starting from the target video frame.

[0197] As an example, the first determining unit 610 is also configured to determine the target video frame by identifying video frames in the video to be encoded whose difference between inter-frame complexity and intra-frame complexity exceeds a difference threshold.

[0198] As an example, the encoding unit 640 is configured to: based on the original encoding bitrate of the video to be encoded, obtain the encoding bitrate of multiple related video frames after the target video frame by increasing the original encoding bitrate; and encode the multiple related video frames based on their respective encoding bitrates, wherein the encoding bitrate of the target video frame is greater than or equal to the encoding bitrate of the multiple related video frames, and the encoding bitrate of the preceding video frame in any adjacent video frame among the multiple related video frames is greater than or equal to the encoding bitrate of the following video frame in any adjacent video frame.

[0199] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0200] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 includes a processor 710 and a memory 720 for storing processor-executable instructions. Here, when executed by the processor, the processor executes the video encoding method as described in the exemplary embodiments above.

[0201] As an example, electronic device 700 can be a single device or a collection of any means or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Processor 710 may include a central processing unit (CPU), graphics processing unit (GPU), programmable logic device, dedicated processor system, microcontroller, or microprocessor. Processor 710 can execute instructions or code stored in memory 720. Instructions and data can also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0202] The memory 720 can be integrated with the processor 710. The memory 720 and the processor 710 can be operationally coupled, or can communicate with each other, for example, through I / O ports, network connections, etc. Furthermore, the electronic device 700 may also include a video display and a user interface.

[0203] In an exemplary embodiment, a computer-readable storage medium may also be provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the video encoding method as described in the exemplary embodiment above. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: a read-only memory (ROM), a random access memory (RAM), a random access programmable read-only memory (PROM), an electrically erasable programmable read-only memory (EEPROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), flash memory, non-volatile memory, a CD-ROM, a CD-R, a CD+R, a CD-RW, a CD+RW, a DVD-ROM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, a DVD-RAM, a BD-ROM, a BD-R, or a BD-R... LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia card, secure digital (SD) card, or ultra-fast digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store computer programs and any associated data, data files, and data structures in a non-transitory manner and to provide the computer programs and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer programs.

[0204] In an exemplary embodiment, a computer program product may also be provided, which includes computer instructions that, when executed by a processor, implement the video encoding method as described in the exemplary embodiment above.

[0205] In an exemplary embodiment, a method for generating a bitstream may also be provided, the method comprising: generating a bitstream according to the video encoding method described in the exemplary embodiment above.

[0206] In an exemplary embodiment, a computer program product may also be provided having instructions for storing a bitstream, the bitstream including video data generated according to the video encoding method described in the exemplary embodiment above.

[0207] Figure 8 A computing environment 810 coupled to a user interface 850 is shown. The computing environment 810 may be part of a data processing server. The computing environment 810 includes a processor 820, a memory 830, and an input / output (I / O) interface 840.

[0208] Processor 820 typically controls the overall operation of computing environment 810, such as operations associated with display, data acquisition, data communication, and image processing. Processor 820 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 820 may include one or more modules that facilitate interaction between processor 820 and other components.

[0209] Memory 830 is configured to store various types of data to support the operation of computing environment 810. Memory 830 may include predefined software 832. Examples of such data include instructions for any application or method operating on computing environment 810, video datasets, image data, etc. Memory 830 can be implemented using any type of volatile or non-volatile memory device or a combination thereof.

[0210] I / O interface 840 provides an interface between processor 820 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 840 can be coupled to encoders and decoders.

[0211] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a memory 830 containing a plurality of programs and / or storing a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by a processor 820 in a computing environment 810 to perform the above-described methods. In one example, the plurality of programs can be executed by a processor 820 in a computing environment 810 to (e.g., from...) Figure 2 The video encoder 20 in the computing environment 810 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 820 in the computing environment 810 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 820 in the computing environment 810 to perform the above-described encoding method to encode video information into a bitstream or data stream, and can also be executed by the processor 820 in the computing environment 810 to (e.g., to...) Figure 3 The video decoder 30 in the middle sends the bitstream or data stream. Alternatively, a non-transitory computer-readable storage medium may store data generated by the encoder (e.g., Figure 2 The video encoder 20 in the video encoder uses, for example, the encoding method described above to generate the video for the decoder (e.g., Figure 3 The video decoder 30 in the middle is a bit stream or data stream that includes encoded video information used when decoding video data.

[0212] In one embodiment, a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method is provided. In another embodiment, a bitstream comprising encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method is provided.

[0213] In one embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 820); and a non-transitory computer-readable storage medium or memory 830 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0214] In one embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, the bitstream including encoded video information generated by the encoding method described above or encoded video information to be decoded by the decoding method described above. In another embodiment, a computer program product including, for example, a plurality of programs in a memory 830 is also provided, the plurality of programs being executable by a processor 820 in a computing environment 810 to perform the methods described above. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0215] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0216] In one embodiment, a method for transmitting a bitstream generated by the encoder described above is also provided. In another embodiment, a method for receiving a bitstream to be decoded by the decoder described above is also provided.

[0217] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0218] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0219] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

[0220] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0221] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that, The video encoding method includes: Identify the target video frame in the video to be encoded; Determine the complexity of the current video scene corresponding to the target video frame; The encoding bitrate of the target video frame is determined based on the complexity of the current video scene. The target video frame is encoded based on its encoding bitrate. The step of determining the encoding bitrate of the target video frame based on the complexity of the current video scene includes: determining the complexity distribution information of the video to be encoded; and determining the encoding bitrate of the target video frame based on the complexity of the current video scene and the complexity distribution information. The complexity distribution information is determined in the following way: The complexity of the second video frame following the first video frame in the video to be encoded is determined, wherein the first video frame is an intra-predicted frame and the second video frame is a forward-predicted frame that is a preset number of frames after the first video frame. The complexity distribution information is determined based on the complexity of multiple first video frames and multiple second video frames in the video to be encoded.

2. The video encoding method according to claim 1, characterized in that, Determining the encoding bitrate of the target video frame based on the complexity of the current video scene and the complexity distribution information includes: Based on the complexity of the current video scene and the complexity distribution information, the encoding bitrate of the target video frame is obtained by increasing the original encoding bitrate of the video to be encoded.

3. The video encoding method according to claim 2, characterized in that, The step of obtaining the encoding bitrate of the target video frame by increasing the original encoding bitrate of the video to be encoded, based on the complexity of the current video scene and the complexity distribution information, includes: Based on the complexity of the current video scene and the complexity distribution information, an improvement coefficient for increasing the original coding bitrate is determined; The encoding bitrate of the target video frame is obtained based on the boost factor and the original encoding bitrate.

4. The video encoding method according to claim 3, characterized in that, The complexity distribution information includes a maximum value and a minimum value of the complexity distribution. The maximum value of the complexity distribution is determined based on the maximum complexity of the plurality of first video frames and the plurality of second video frames, and the minimum value of the complexity distribution is determined based on the minimum complexity of the plurality of first video frames and the plurality of second video frames. The lifting factor is determined in the following manner: Based on the maximum value of the complexity distribution, the minimum value of the complexity distribution, and the complexity of the current video scene, the preset coefficient is adjusted to obtain the enhancement coefficient, wherein the preset coefficient represents the maximum multiple by which the original encoding bitrate is increased.

5. The video encoding method according to claim 1, characterized in that, Determining the encoding bitrate of the target video frame based on the complexity of the current video scene and the complexity distribution information includes: In response to the current video scene having a complexity higher than a first preset value, the encoding bitrate of the target video frame is obtained by increasing the original encoding bitrate of the video to be encoded, based on the complexity of the current video scene and the complexity distribution information. In response to the fact that the complexity of the current video scene and the complexity of the first second video frame after the target video frame are both lower than the second preset value, the encoding bitrate of the target video frame is obtained by reducing the original encoding bitrate. In response to the fact that the complexity of the current video scene is lower than the first preset value but higher than the second preset value, the original encoding bitrate is determined as the encoding bitrate of the target video frame. Wherein, the first preset value is greater than the second preset value.

6. The video encoding method according to claim 1, characterized in that, The complexity of the current video scene is determined in the following ways: The complexity of the current video scene is obtained by statistically analyzing the complexity of video frames within a first preset number of frames starting from the target video frame.

7. The video encoding method according to claim 1, characterized in that, The target video frame is a transition frame in the video to be encoded, wherein the target video frame is determined in the following manner: The video frames whose difference between inter-frame complexity and intra-frame complexity exceeds a difference threshold are identified as the target video frames.

8. The video encoding method according to claim 1, characterized in that, The video encoding method further includes: Based on the original encoding bitrate of the video to be encoded, the encoding bitrate of multiple related video frames after the target video frame is obtained by increasing the original encoding bitrate. Encode the multiple related video frames based on their respective encoding bitrates. Wherein, the encoding bitrate of the target video frame is greater than or equal to the encoding bitrate of the plurality of related video frames, and the encoding bitrate of the preceding video frame in any adjacent video frame among the plurality of related video frames is greater than or equal to the encoding bitrate of the following video frame in the adjacent video frames.

9. A video encoding device, characterized in that, The video encoding device includes: The first determining unit is configured to determine the target video frame in the video to be encoded; The second determining unit is configured to determine the complexity of the current video scene corresponding to the target video frame; The third determining unit is configured to determine the encoding bitrate of the target video frame based on the complexity of the current video scene. The encoding unit is configured to encode the target video frame based on the encoding bitrate of the target video frame. The third determining unit is configured to: determine the complexity distribution information of the video to be encoded; and determine the encoding bitrate of the target video frame based on the complexity of the current video scene and the complexity distribution information. The video encoding device further includes a fourth determining unit, which is configured to determine the complexity distribution information in the following manner: The complexity of the second video frame following the first video frame in the video to be encoded is determined, wherein the first video frame is an intra-predicted frame and the second video frame is a forward-predicted frame that is a preset number of frames after the first video frame. The complexity distribution information is determined based on the complexity of multiple first video frames and multiple second video frames in the video to be encoded.

10. An electronic device, characterized in that, The electronic device includes: Processor; and Memory used to store processor-executable instructions. Wherein, when the processor executes the processor, the processor causes the processor to perform the video encoding method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the video encoding method according to any one of claims 1 to 8.

12. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the video encoding method according to any one of claims 1 to 8.

13. A method for transmitting a bit stream, characterized in that, include: A bitstream is generated according to the video encoding method according to any one of claims 1 to 8; Send the bit stream.

Images