Video coding method and device, method for generating bit stream, electronic equipment, storage medium and program product

Abstract

The invention relates to a video coding method and device, a bit stream generation method, electronic equipment, a storage medium and a program product. The video coding method comprises the following steps: determining a target video frame in a video to be coded; determining the complexity of a current video scene corresponding to the target video frame; determining a coding rate of a target video frame according to the complexity of the current video scene; and coding the target video frame based on the coding rate of the target video frame. The problem that the code rate distribution of an encoder for different video scenes is not ideal is solved, the code rate can be adaptively distributed according to the video scenes, and the subjective image quality experience of videos in different video scenes is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video technology, and in particular to a video encoding method and apparatus, a method for generating a bit stream, an electronic device, a storage medium, and a program product. Background Art

[0002] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices send and receive or otherwise transmit digital video data through a communication network, and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited storage resources of the storage device, before the video data is transmitted or stored, the video data can be compressed using a video codec according to one or more video codec standards. For example, video codec standards include general video codec (VVC), joint exploration test model (JEM), high efficiency video codec (HEVC / H.265), advanced video codec (AVC / H.264), moving picture expert group (MPEG) codec, etc. Video codecs generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize the inherent redundancy in video data. Video codecs are intended to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0003] In videos, the image quality of different video scenes may be different. However, during the video encoding process, the bitrate allocation strategy of mainstream hardware encoders for different video scenes is not ideal, which may affect the subjective image quality experience. It may also lead to higher transmission bandwidth costs due to unreasonable bitrate allocation, affecting the viewing experience.

[0004] In addition, the hardware encoder can only be configured and used through the official peripheral interface. Its internal implementation mechanism is in a black box state, with limited adjustable parameters, low algorithm flexibility and accuracy, and it is unable to adaptively allocate bitrates according to video scenarios. Summary of the invention

[0005] The present disclosure provides a video encoding method and apparatus, a method for generating a bit stream, an electronic device, a storage medium and a program product, so as to at least solve the problem that the encoder in the related art has unsatisfactory bit rate allocation for different video scenes. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, the video encoding method comprising: determining a target video frame in a video to be encoded; determining the complexity of a current video scene corresponding to the target video frame; determining an encoding bit rate of the target video frame according to the complexity of the current video scene; and encoding the target video frame based on the encoding bit rate of the target video frame.

[0007] Optionally, determining the encoding bit rate of the target video frame according to the complexity of the current video scene includes: determining complexity distribution information of the video to be encoded; and determining the encoding bit rate of the target video frame according to the complexity of the current video scene and the complexity distribution information.

[0008] Optionally, determining the encoding bit rate of the target video frame according to the complexity of the current video scene includes: obtaining the encoding bit rate of the target video frame by increasing the original encoding bit rate of the video to be encoded according to the complexity of the current video scene and the complexity distribution information.

[0009] Optionally, the complexity distribution information is determined in the following manner: determining the complexity of a second video frame after a first video frame in the video to be encoded, wherein the first video frame is an intra-frame prediction frame, and the second video frame is a forward prediction frame that is a preset number of frames after the first video frame and away from the second video frame; determining the complexity distribution information based on the complexity of multiple first video frames and multiple second video frames in the video to be encoded.

[0010] Optionally, the encoding bit rate of the target video frame is obtained by improving the original encoding bit rate of the video to be encoded according to the complexity of the current video scene and the complexity distribution information, including: determining a boosting coefficient for improving the original encoding bit rate based on the complexity of the current video scene and the complexity distribution information; and obtaining the encoding bit rate of the target video frame based on the boosting coefficient and the original encoding bit rate.

[0011] Optionally, the complexity distribution information includes a maximum value of the complexity distribution and a minimum value of the complexity distribution, wherein the maximum value of the complexity distribution is determined based on the maximum value of the complexity of the multiple first video frames and the multiple second video frames, and the minimum value of the complexity distribution is determined based on the minimum value of the complexity of the multiple first video frames and the multiple second video frames, wherein the improvement coefficient is determined in the following manner: based on the maximum value of the complexity distribution, the minimum value of the complexity distribution and the complexity of the current video scene, a preset coefficient is adjusted to obtain the improvement coefficient, wherein the preset coefficient represents the maximum multiple of improving the original encoding bit rate.

[0012] Optionally, determining the encoding bit rate of the target video frame according to the complexity of the current video scene further includes: in response to the complexity of the current video scene being higher than a first preset value, executing the step of obtaining the encoding bit rate of the target video frame by increasing the original encoding bit rate of the video to be encoded according to the complexity of the current video scene and the complexity distribution information; in response to the complexity of the current video scene and the complexity of the first second video frame after the target video frame being lower than a second preset value, obtaining the encoding bit rate of the target video frame by reducing the original encoding bit rate; in response to the complexity of the current video scene being lower than the first preset value and higher than the second preset value, determining the original encoding bit rate as the encoding bit rate of the target video frame, wherein the first preset value is greater than the second preset value.

[0013] Optionally, the complexity of the current video scene is determined in the following manner: the complexity of the current video scene is obtained by counting the complexity of video frames within a first preset number of frames starting from the target video frame.

[0014] Optionally, the target video frame is a transition frame in the video to be encoded, wherein the target video frame is determined in the following manner: a video frame in which the difference between inter-frame complexity and intra-frame complexity in the video to be encoded exceeds a difference threshold is determined as the target video frame.

[0015] Optionally, the video encoding method also includes: based on the original encoding bit rate of the video to be encoded, obtaining the encoding bit rate of multiple related video frames after the target video frame by improving the original encoding bit rate; based on the respective encoding bit rates of the multiple related video frames, encoding the multiple related video frames, wherein the encoding bit rate of the target video frame is greater than or equal to the encoding bit rate of the multiple related video frames, and the encoding bit rate of a previous video frame in any adjacent video frames among the multiple related video frames is greater than or equal to the encoding bit rate of a subsequent video frame among the adjacent video frames.

[0016] According to the second aspect of the embodiments of the present disclosure, a video encoding device is provided, which includes: a first determination unit, configured to determine a target video frame in a video to be encoded; a second determination unit, configured to determine the complexity of a current video scene corresponding to the target video frame; a third determination unit, configured to determine the encoding bit rate of the target video frame according to the complexity of the current video scene; and an encoding unit, configured to encode the target video frame based on the encoding bit rate of the target video frame.

[0017] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor-executable instructions, when executed by the processor, enable the processor to execute the video encoding method described in the exemplary embodiment of the present disclosure.

[0018] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the video encoding method according to the exemplary embodiment of the present disclosure.

[0019] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the video encoding method according to the exemplary embodiment of the present disclosure.

[0020] According to a sixth aspect of an embodiment of the present disclosure, a method for generating a bit stream is provided, comprising: generating a bit stream according to the video encoding method as described in the exemplary embodiment of the present disclosure.

[0021] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, having instructions for storing a bit stream, wherein the bit stream comprises: video data generated according to the video encoding method as described in the exemplary embodiment of the present disclosure.

[0022] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0023] In an embodiment of the present disclosure, the complexity of the video scene corresponding to the target video frame in the video to be encoded can be determined, so that the encoding bit rate of the target video frame can be determined according to the complexity of the video scene to encode the target video frame. In this way, the encoding bit rate can be determined in combination with the complexity analysis of the video scene to solve the problem of unsatisfactory bit rate allocation for different video scenes, so that the bit rate can be adaptively allocated according to the video scene, thereby improving the subjective picture quality experience of the video in different video scenes.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0026] Figure 1is a block diagram showing an exemplary system for encoding and decoding video blocks according to an exemplary embodiment of the present disclosure.

[0027] Figure 2 is a block diagram showing an exemplary video encoder according to an exemplary embodiment of the present disclosure.

[0028] Figure 3 is a block diagram showing an exemplary video decoder according to an exemplary embodiment of the present disclosure.

[0029] Figure 4 It is a flowchart of a video encoding method according to an exemplary embodiment of the present disclosure.

[0030] Figure 5 It is a flowchart of a video encoding method according to an exemplary embodiment of the present disclosure.

[0031] Figure 6 is a block diagram showing a video encoding device according to an exemplary embodiment of the present disclosure.

[0032] Figure 7 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

[0033] Figure 8 is a diagram of a computing environment coupled with a user interface according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0036] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0037] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0038] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0039] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0040] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0041] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0042] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0043] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0044] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0045] Figure 2is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0046] like Figure 2 As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0047] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0048] like Figure 2As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0049] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0050] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0051] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0052] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0053] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0054] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0055] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0056] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0057] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0058] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0059] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0060] In some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0061] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0062] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0063] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0064] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0065] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0066] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0067] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0068] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0069] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0070] As described above, in the related art, the bit rate allocation strategy of the hardware encoder for different video scenes is not ideal, and the bit rate cannot be adaptively allocated according to the video scene.

[0071] Specifically, taking the video transition scene as an example, the picture transition is a common video scene, and the human eye is more sensitive to the image quality of this kind of scene. However, since the picture content after the transition has no information association with the previous frame, it is difficult to use the time domain correlation for encoding, so more codewords are usually required. In the relevant technology, the bit rate allocation strategy of mainstream hardware encoders in such scenes is not ideal, which affects the subjective picture quality experience and brings higher transmission bandwidth costs.

[0072] In addition, the hardware encoder can only be configured and used through the official peripheral interface. The internal implementation mechanism is in a black box state, the adjustable parameters are limited, the algorithm flexibility and accuracy are low, and it is impossible to customize the bit rate allocation strategy in different scenarios according to needs.

[0073] Taking the multimedia codec framework VideoToolbox as an example, it is mainly used for hardware-accelerated encoding and decoding of audio and video on iOS and mac operating system platforms. It provides developers with a set of application programming interfaces (APIs) and tools, allowing applications to use the hardware acceleration function of the device to perform efficient audio and video processing.

[0074] However, the multimedia codec framework exposes limited peripheral code control interfaces, limited adjustment space, and low flexibility. In addition, the multimedia codec framework cannot recognize scenes that require high bit rates, such as screen switching, and has limited effect in adaptively adjusting bit rate allocation according to different video complexity scenes.

[0075] In order to solve at least part of the above problems, exemplary embodiments of the present disclosure provide a video encoding method, a video encoding device, a method for generating a bit stream, an electronic device, a computer-readable storage medium, and a computer program product. Figures 4 to 8 Describe in detail.

[0076] According to an aspect of an exemplary embodiment of the present disclosure, a video encoding method is proposed. The video encoding method can be applied to any video encoding scenario. An example implementation scenario of the video encoding method according to the exemplary embodiment of the present disclosure is given below.

[0077] For example, when a user uses a video application client on a user terminal to request to watch video content from a server or video distribution platform through the network, the server or video distribution platform can send the requested video content to the user terminal through the network, and the user can watch the received video content through the video application client.

[0078] In the above process, the server or the video distribution platform may use the video encoding method according to the exemplary embodiment of the present disclosure to encode the transmitted video content.

[0079] It should be noted that although the above description uses a server or video distribution platform as an example, it is only an example. The executor of the video encoding method can be any electronic device, which may include, for example, smart phones, tablet computers, laptops, digital assistants, wearable devices, vehicle terminals, etc., which are physical devices that serve as hardware codecs, and may also include software that runs on physical devices as software video encoders.

[0080] It should also be noted that although the video encoding application scenario in which a user terminal requests to watch a video is described here as an example, it should be understood that the application scenario of the video encoding method according to the present disclosure is not limited to this, and it can also be applied to any other application scenario involving video encoding.

[0081] Figure 4 It is a flow chart of a video encoding method according to an exemplary embodiment. As described above, according to the video encoding method of the exemplary embodiment of the present disclosure, it is possible to determine the encoding bit rate in combination with the complexity analysis of the video scene, solve the problem of unsatisfactory bit rate allocation for different video scenes, and thus can adaptively allocate the bit rate according to the video scene, and improve the subjective picture quality experience of the video in different video scenes.

[0082] like Figure 4 As shown, the video encoding method according to an exemplary embodiment of the present disclosure may include the following steps:

[0083] In step S410, a target video frame in a video to be encoded may be determined.

[0084] In this step, as an example, the target video frame may be any video frame in the video to be encoded, for example, may be a video frame currently to be encoded in the process of encoding the video frame by frame.

[0085] As another example, the target video frame may be a transition frame in the video to be encoded. Specifically, the scene transition may be a video scene that is highly sensitive to image quality. During the encoding process, paying attention to the image quality of the transition frame or several frames after the transition frame can better allocate the encoding bit rate and improve the overall image quality experience of the video.

[0086] For example, the target video frame may be determined in the following manner: a video frame whose difference between the inter-frame complexity and the intra-frame complexity in the video to be encoded exceeds a difference threshold is determined as the target video frame. Here, the inter-frame complexity may represent the information that needs to be encoded for the current video frame with the reference video frame as a reference; the intra-frame complexity may represent the information that needs to be encoded for the current video frame with itself as a reference, wherein the reference video frame may be a video frame before the current video frame. The reference video frame may be any video frame before the current video frame, for example, it may be the previous frame of the current video frame.

[0087] Specifically, the intra-frame prediction and inter-frame prediction in the encoding technology can be performed on the current video frame, and the codewords and distortion sizes required to encode the current video frame in the two prediction modes are calculated, so as to obtain the above-mentioned intra-frame complexity and inter-frame complexity respectively. Here, the encoding complexity of the video frame can characterize the amount of information that needs to be encoded for the video frame, and the complexity can be, for example, but not limited to, represented by SAD (Sum of Absolute Difference) or SATD (Sum of Absolute Transformed Difference). Here, for example, it is possible but not limited to use a typical prediction method in HEVC to perform intra-frame prediction and inter-frame prediction. As an example, the intra-frame prediction method can be, but not limited to, 33 angle modes, direct current (DC) mode, planar mode, etc., and the inter-frame prediction method can be, but not limited to, motion estimation, motion compensation, etc.

[0088] After entering the formal encoding, the video frames in the video to be encoded will be sent to the encoder one by one, such as Figure 5 As shown, before the frame to be encoded is sent to the encoder, in the bit rate allocation model of the hardware encoder, the intra-frame prediction complexity information intra_cost and the inter-frame prediction complexity information inter_cost of the current frame can be calculated in real time, and the difference between the inter-frame complexity and the intra-frame complexity can be calculated. Figure 5 In the embodiment, the bit rate allocation model may include a bit rate decision module, which can obtain the target video frame detected by the scene detection module and the video scene information where the target video frame is located, such as obtaining the transition frame and the transition scene. The bit rate decision module can also decide the configuration bit rate for encoding the current video frame based on the complexity information of the current video scene obtained by real-time complexity analysis (or based on the complexity distribution information described below).

[0089] Here, the complexity difference between the inter-frame complexity and the intra-frame complexity may be represented by, for example but not limited to, a ratio, a difference, or other mathematical methods. As an example, the complexity difference may be represented by the ratio in the following formula (1):

[0090]

[0091] Among them, inter_cost represents the inter-frame complexity, which represents the information that needs to be encoded when the current video frame takes the reference video frame as a reference; intra_cost represents the intra-frame complexity, which represents the information that needs to be encoded when the current video frame takes itself as a reference.

[0092] For this formula (1), the larger the ratio ratio is, the less information the current video frame can refer to from the reference video frame, the greater the scene difference between the two frames, the higher the complexity between the frames, the lower the correlation between the current video frame and the reference video frame, and the greater the possibility that the current video frame is a transition frame. However, the representation method of complexity difference is not limited to this formula (1), for example, it can also be represented by the inverse of the ratio ratio.

[0093] As an example, in response to the complexity difference exceeding the difference threshold, the current video frame may be determined to be a transition frame; in response to the complexity difference not exceeding the difference threshold, the current video frame may be determined to be a non-transition frame.

[0094] Specifically, when the inter-frame complexity is relatively large relative to the intra-frame complexity, for example, when the difference threshold is exceeded, it can be indicated that the current video frame does not obtain higher encoding performance using the previous reference video frame as a reference compared to only referring to itself, indicating that the current video frame has a low correlation with the previous reference video frame, and thus it can be judged as a transition frame. By determining the transition frame by analyzing the complexity difference, the transition scene in the video can be detected more accurately, which serves as the basis for improving the bit rate of the transition scene, and is conducive to guiding the subsequent more reasonable allocation of the encoding bit rate.

[0095] As an example, the difference threshold may be a configurable parameter, which may be an empirical value according to industry standards. Taking the above formula (1) as an example, according to the Scenecut decision algorithm in High Efficiency Video Coding (HEVC), a difference threshold scenecut_th may be set, and the difference threshold scenecut_th may be, for example, but not limited to, 40%. In addition, in the embodiment of the present disclosure, in order to avoid a large increase in the final encoding bit rate, the difference threshold scenecut_th may be set to 70%, for example. If the ratio ratio in the above formula (1) exceeds the difference threshold scenecut_th, it can be considered that the difference between the two frames of scenes is large, so that a transition may occur. However, the difference threshold is not limited to a constant, and it may also change dynamically during the encoding process, or specifically during the detection of the transition frame. For example, according to an embodiment of the present disclosure, in the process of determining whether the current video frame is a transition frame, the video encoding method may further include: updating the difference threshold; and determining whether the third video frame after the current video frame is a transition frame based on the updated difference threshold.

[0096] Here, the step of updating the difference threshold may include: in response to the number of transition frames determined in a video frame sequence of a preset window size being greater than a first preset number of transition frames, increasing the difference threshold; in response to the number of transition frames determined in a video frame sequence of a preset window size being less than a second preset number of transition frames, reducing the difference threshold, wherein the second preset number of transition frames is less than or equal to the first preset number of transition frames.

[0097] Specifically, according to the sequence of video frames in the video to be encoded, the transition frame can be detected for each video frame, and an initial difference can be set as a difference threshold at the time of initial detection, for example, an initial difference scenecut_th_base can be set for the above-mentioned difference threshold scenecut_th. In the sequential detection of the video frame sequence, in the preset detection window, if the number of the detected transition frames is greater than the first preset transition frame number, the difference threshold can be increased, for example, the initial difference scenecut_th_base is increased to the increased difference threshold scenecut_th_max, so that the subsequent video frames are subjected to transition frame detection based on the increased difference threshold, so that the judgment condition of the transition frame can be strengthened to avoid excessive detection of the transition frame. Until the number of the transition frames detected in the detection window is less than the second preset transition frame number, the difference threshold can be reduced, for example, the increased difference threshold scenecut_th_max can be adjusted back to the initial difference scenecut_th_base, so that the subsequent video frames are subjected to transition frame detection based on the reduced difference threshold.

[0098] The main reason for dynamically adjusting the difference threshold in this way is that the judgment of the transition frame will affect the number of times the bit rate is increased. Excessive increase will easily lead to a large increase in the final bit rate. Here, the preset window size, the first preset number of transition frames and the second preset number of transition frames can be set according to actual needs. For example, for the above formula (1), the initial difference scenecut_th_base can be set to 70%, the increased difference threshold scenecut_th_max can be 85%, the preset window size can be, for example, but not limited to 20 frames, the first preset number of transition frames can be, for example, but not limited to 4 frames, and the second preset number of transition frames can be, for example, but not limited to 3 frames. In this way, the bit rate can be increased only when the picture difference is extremely large or when the bit rate is more likely to be insufficient.

[0099] For example, if more transition frames than the first preset transition frame number b are detected within the a frame range (e.g., detection window) after the bit rate is increased, it can be considered that the current scene continues to change dramatically, and the difference threshold can be increased to reduce the number of transition detections and control the number of bit rate increases. After the difference threshold is increased, if the number of transition frames detected within the a frame range (e.g., detection window) after the bit rate is increased is lower than the second preset transition frame number c, it can be considered that the current scene has stabilized, and the difference threshold can be reduced, for example, the difference can be adjusted back to the initial difference.

[0100] By the above method, the detection threshold of the transition frame can be dynamically adjusted to avoid frequent bit rate increase caused by excessive detection of transition frames, thereby reducing the calculation pressure and communication burden. In addition, when the second preset transition frame number is set to be less than the first preset transition frame number, the situation that the difference threshold may repeatedly jump when the two are equal can be avoided, thereby improving the stability of the detection process.

[0101] The process of determining the target video frame can be performed as follows, for example: Figure 5 The scene detection module in the embodiment can be used to implement the above-mentioned scene detection. For example, a series of scene switching frame information can be obtained according to the above-mentioned scene transition detection method, that is, a series of scene transition frames can be obtained as target video frames.

[0102] In step S420, the complexity of the current video scene corresponding to the target video frame may be determined.

[0103] In this step, the current video scene corresponding to the target video frame may refer to a video segment including the target video frame. In one example, the target video frame may be any frame in the corresponding video scene.

[0104] In another example, a video segment starting from the target video frame and including a first preset number of frames may be used as a video scene corresponding to the target video frame.

[0105] In this example, the complexity of the current video scene can be determined in the following manner: by taking statistics on the complexity of video frames within a first preset number of frames starting from the target video frame, the complexity of the current video scene is obtained. Here, the complexity of a video segment including the target video frame can be determined, so as to comprehensively judge the complexity level of the video scene where the target video frame is located, thereby more reasonably determining the complexity of the target video frame. However, the embodiments of the present disclosure are not limited to this, and the complexity of the target video frame can also be used as the complexity of the current video scene.

[0106] Here, the inter-frame complexity of the video frame may be, for example, information that needs to be encoded using a previous frame as a reference. The complexity of the video frames within the first preset number of frames starting from the target video frame may be calculated by, for example, but not limited to, averaging, or by other statistical methods, such as standard deviation or variance.

[0107] The first preset number of frames can be set according to actual needs. In one example, the first preset number of frames can be less than or equal to the maximum video frame capacity of the lookahead buffer. Specifically, in the process of encoding the video to be encoded, each video frame is sequentially stored in the lookahead buffer, and the video frames in the lookahead buffer are encoded in a first-in-first-out (Fifo) order. In this regard, when the first preset number of frames is less than or equal to the maximum video frame capacity of the lookahead buffer, the video frames in the lookahead buffer (e.g., all video frames) can be directly used to calculate the above statistical values, so that statistical calculations can be facilitated.

[0108] In step S430, the encoding bit rate of the target video frame may be determined according to the complexity of the current video scene.

[0109] As an example, the encoding bit rate of the target video frame may be obtained by increasing the original encoding bit rate of the video to be encoded according to the complexity of the current video scene.

[0110] For example, in response to the complexity of the current video scene being higher than a first preset value, the encoding bit rate of the target video frame may be obtained by increasing the original encoding bit rate of the video to be encoded.

[0111] The original encoding bit rate of the video to be encoded may be preset, for example, an expected encoding bit rate. The original encoding bit rate may be used as a benchmark for increasing the bit rate of the target video frame, and the bit rate of the target video frame may be increased based on the benchmark to obtain the encoding bit rate of the target video frame.

[0112] In the above manner, when the encoding complexity of the current video scene itself is relatively high, the process of increasing the original encoding bit rate to obtain the encoding bit rate of the transition frame can be performed to specifically increase the encoding bit rate of the target video frame to ensure the image quality of the target video frame. For example, in combination with the complexity information of the video scene itself where the target video frame is located, by setting the complexity threshold Cplx_th, if the current scene complexity Cplx_cur is higher than the complexity threshold Cplx_th, the encoding bit rate of the current frame is increased to obtain better encoding quality. In the example where the target video frame is a transition frame, the current scene complexity Cplx_th is higher than the complexity threshold Cplx_th. i It can reflect the complexity of the picture at the beginning of the new scene after the transition.

[0113] Here, the first preset value may be set according to actual needs, for example, or may be determined according to the block complexity or block size of the transition frame.

[0114] Specifically, since the frame complexity is related to the frame resolution, the more coding blocks a frame contains, the greater the complexity information. Therefore, the block complexity is also related to the size of the block. The larger the block size, the more complexity information. Accordingly, the preset complexity Cplx_th can also be related to the size of the block. For example, referring to the initialization method of the complexity of the code control model in HEVC, the preset complexity Cplx_th can be expressed by the following formula (2):

[0115]

[0116] Among them, s1 and s2 represent preset coefficients, which can be set according to actual needs, for example, s1 can be 7000, and s2 can be 5; Cu_count represents the number of blocks in the transition frame.

[0117] As an example, Cu_count=cu_width×cu_height, where cu_width represents the number of blocks in the width direction of the transition frame, and cu_height represents the number of blocks in the height direction of the transition frame. For example, cu_width and cu_height may be the number of coding units (e.g., 8×8 CUs) included in the video frame in the width and height directions, respectively.

[0118] In this example, in some cases, the encoding bitrate of the target video frame can be determined based on the complexity of the current video scene and a preset multiple of the original encoding bitrate of the video to be encoded. In this way, more bitrate can be allocated to complex video scenes, thereby improving the overall subjective quality of the video.

[0119] In other cases, different rates can be compensated for each video scene that meets the rate compensation conditions, and the rate can be more reasonably allocated based on the complexity distribution of the entire sequence. Figure 5 The pre-analysis module shown first obtains the approximate complexity distribution of the entire video, and then adjusts the bit rate improvement strength in the corresponding video scene according to the complexity distribution, compensating more bit rate for relatively complex scenes and less bit rate for relatively simple scenes to save the overall encoding bit rate.

[0120] As an example, according to an exemplary embodiment of the present disclosure, in response to the complexity of the current video scene being higher than a first preset value, the encoding bit rate of the target video frame can be obtained by improving the original encoding bit rate of the video to be encoded based on the complexity of the current video scene and complexity distribution information.

[0121] Specifically, the complexity distribution information of the video to be encoded may be determined, and the encoding bit rate of the target video frame may be determined according to the complexity of the current video scene and the complexity distribution information.

[0122] As an example, the encoding bit rate of the target video frame may be obtained by increasing the original encoding bit rate of the video to be encoded according to the complexity and complexity distribution information of the current video scene.

[0123] Here, the complexity distribution information of the video to be encoded may represent the complexity distribution of different video frames of the video to be encoded. In one example, the complexity distribution information may be determined by pre-analyzing the complexity of each frame in the video to be encoded.

[0124] According to an embodiment of the present disclosure, in order to improve the encoding speed and reduce the encoding calculation amount while determining the complexity distribution information, in another example, the complexity distribution information can be determined in the following manner: determining the complexity of the second video frame after the first video frame in the video to be encoded; determining the complexity distribution information based on the complexity of multiple first video frames and multiple second video frames in the video to be encoded.

[0125] Here, the first video frame may be an intra-coded frame (I frame), and the second video frame may be a forward-coded frame (P frame) that is a preset number of frames behind the first video frame.

[0126] Specifically, if Figure 5As shown in the figure, for an input video, it can be pre-analyzed by the pre-analysis module first, with the goal of obtaining the distribution of content complexity in the video. Ideally, directly analyzing the video frame by frame can obtain the most accurate complexity distribution, but frame by frame analysis will bring huge computational complexity, which will seriously affect the time of encoding and outputting the first frame, thereby affecting the user experience.

[0127] In order to alleviate the increase in computational complexity, the complexity can be calculated only for the I frame and one or more P frames after the I frame. The starting point of this design is: the I frame is an intra-frame prediction frame, and its encoding only uses itself as a reference and does not rely on other frames. Therefore, it can be used as a random access frame, that is, the I frame can be decoded at any time without decoding the previous frame, which greatly reduces the computational delay. In addition, in order to obtain the temporal complexity of the content, the P frame that is a preset number of frames away from the I frame can also be decoded, such as a P frame immediately after the I frame. This P frame can only use the previous I frame as a reference, so it can also be decoded normally, and the corresponding intra-frame and inter-frame complexity can be obtained. Here, the number of P frames after the selected I frame can be any integer greater than or equal to 1.

[0128] In addition, since the intervals between I frames in most videos are fixed, after decoding similar IP frame combinations in the video (i.e., a combination of an I frame and one or more P frames following the I frame), it is equivalent to uniformly sampling the video, and the approximate complexity distribution information of the entire video can be obtained.

[0129] The intra-frame complexity and inter-frame complexity mentioned in the above process refer to the coding cost of the frame to be encoded through intra-frame prediction or inter-frame prediction. The higher the coding cost, the higher the complexity. Here, the typical prediction method in the HEVC standard can be used to implement intra-frame prediction and inter-frame prediction. In addition, as an example, if the number of I frames in the video to be encoded is small or the distribution is uneven, video pre-analysis may not be performed, or the pre-analysis information may be discarded as unavailable. After testing, the sampled and parallel processed video pre-analysis module has basically no perceptible effect on the encoding speed.

[0130] As an example, according to the complexity and complexity distribution information of the current video scene, by improving the original encoding rate of the video to be encoded, the encoding rate of the target video frame can be obtained, which can include: determining the improvement coefficient for improving the original encoding rate based on the complexity and complexity distribution information of the current video scene; and obtaining the encoding rate of the target video frame based on the improvement coefficient and the original encoding rate. In this way, the complexity level of the current video scene can be measured in combination with the overall complexity distribution of the video, so that the degree of improvement of the original encoding rate can be more reasonably determined.

[0131] Specifically, after the conditions for increasing the bit rate are met, the original encoding bit rate can be weighted based on the boosting coefficient, so that the encoding bit rate of the target video frame can be increased to a multiple of the original encoding bit rate. In one example, a boosting coefficient table corresponding to the complexity of different video scenes and different complexity distribution information can be preset, and the boosting coefficient corresponding to the complexity of the current video scene and the current complexity distribution information can be obtained by looking up the table, so that the original encoding bit rate can be increased by using the boosting coefficient to obtain the encoding bit rate of the target video frame.

[0132] In another example, the complexity distribution information may include a maximum value of the complexity distribution and a minimum value of the complexity distribution, wherein the maximum value of the complexity distribution is determined based on the maximum value of the complexity of multiple first video frames and multiple second video frames, and the minimum value of the complexity distribution is determined based on the minimum value of the complexity of multiple first video frames and multiple second video frames.

[0133] Specifically, it can be Figure 5 The pre-analysis module shown obtains the complexity of all first video frames (such as I frames) and second video frames (such as P frames) after each first video frame, and counts the maximum value X max_IP and minimum value X min_IP .

[0134] As an example, the above minimum value X min_IP The smaller of the complexity threshold Cplx_th and the above complexity threshold Cplx_th is taken as the minimum value of the complexity distribution, that is, X min =max(X min_IP , Cplx_th). In addition, as an example, the above maximum value X max_IP and the complexity X of the detected target video frame i The larger of the two is taken as the maximum value of the complexity distribution, that is, X max =max(X max_IP ,X i ). In the process of calculating the complexity of the target video frame in real time, in response to the complexity X of the target video frame i If it is greater than the current maximum complexity, X can be updated. max =X i .

[0135] In this example, the boost coefficient can be determined in the following manner: based on the maximum value of the complexity distribution, the minimum value of the complexity distribution and the complexity of the current video scene, the preset coefficient is adjusted to obtain the boost coefficient, wherein the preset coefficient represents the maximum multiple of the original encoding bit rate.

[0136] Here, the lifting coefficient for the current video scene can be determined by combining the complexity of the current video scene and the complexity distribution information in the pre-analysis. For example, the lifting coefficient n can be determined by the following formula (3):

[0137]

[0138] Among them, Cplx_cur is the complexity of the current video scene, X max and X min are the maximum and minimum values ​​of the complexity distribution, respectively, and N is the preset maximum boost factor, which can be set according to actual needs. Here, under the constraint of the above formula (3), the value of the boost factor n can be in the range of [0.5N, N].

[0139] In the above method, by setting the maximum and minimum values ​​of complexity, according to the relationship between the complexity of the current video scene and the maximum and minimum values, the preset maximum improvement multiple can be reasonably adjusted to quantify the degree of bit rate improvement that matches the complexity of the current video scene, so that the bit rate improvement and allocation can be performed more flexibly.

[0140] Although the above describes an example method for determining the boost factor with reference to formula (3), it is not limited to this. As long as the expression of the preset coefficient can be adjusted using the above maximum value, minimum value, and complexity of the current scene, the above formula (3) can be used to calculate the boost factor.

[0141] In an embodiment of the present disclosure, the video to be encoded may include multiple target video frames, and the lifting coefficients corresponding to at least two target video frames may be different, wherein the lifting coefficient may be positively correlated with the complexity of the video scene corresponding to the target video frame. Specifically, by adopting the method of the present disclosure, the encoding bit rate of the target video frame can be improved according to the complexity of the video scene, so that the encoding bit rates of different target video frames can be more reasonably designed, and the image quality experience of the target video frame with higher complexity can be improved.

[0142] In step S430, as another example, in response to the complexity of the current video scene and the complexity of the first second video frame after the target video frame being lower than a second preset value, the encoding bit rate of the target video frame can be obtained by reducing the original encoding bit rate, wherein the first preset value is greater than the second preset value.

[0143] Specifically, for a target video frame, when the complexity of the video scene in which it is located and the complexity of the first and second video frames that follow it are both lower than the second preset value, it can be said that the overall complexity of the current video scene is relatively low, and the encoding bit rate of the scene can be lowered to achieve reasonable bit rate adjustment and allocation. For example, when the target video frame is identified as a transition frame, if the complexity of the current transition frame and the complexity of the second video frame that is closest to the sequence are both lower than the second preset value Cplx_static_th, it means that the overall complexity of the scene after the transition may be low, and the encoding bit rate of the scene can be appropriately reduced.

[0144] As an example, the second preset value Cplx_static_th may be determined by the following formula (4):

[0145]

[0146] Among them, s3 represents a preset coefficient, which can be set according to actual needs, for example, it can be 5000.

[0147] In this example, for example, the original encoding bit rate can be weighted by using the reduction coefficient m, and the encoding bit rate of the target video frame can be configured to be m times the original encoding bit rate BR, where m<1, and m can be a configurable parameter, for example, m can be set to 0.7.

[0148] In step S430, as yet another example, in response to the complexity of the current video scene being lower than a first preset value and higher than a second preset value, the original encoding bit rate is determined as the encoding bit rate of the target video frame.

[0149] In this example, when the complexity of the current video scene is lower than the first preset value but higher than the second preset value, it can be considered that the complexity level of the current scene is at a medium level, and there is no need to adjust the encoding bit rate of the target video frame.

[0150] Through the above method, different strategies can be adopted to determine the encoding bit rate of the target video frame under different complexity of the video scene, and the encoding bit rate can be reduced as much as possible while ensuring the video quality experience, thereby reducing the encoding operation amount and communication burden. However, the embodiments of the present disclosure are not limited to this, and the above example of increasing the bit rate can also be directly applied without judging the complexity level of the scene, so as to optimize the video quality as much as possible.

[0151] In step S440, the target video frame may be encoded based on the encoding bit rate of the target video frame.

[0152] In this step, the target video frame may be encoded based on the encoding bit rate of the target video frame using, for example, an existing encoding method.

[0153] The above describes an example process of determining the encoding bit rate of a target video frame based on the complexity of the video scene in which the target video frame is located. In addition, in an embodiment of the present disclosure, the encoding bit rates of multiple related video frames after the target video frame can also be determined in a manner similar to the above manner.

[0154] Specifically, the video encoding method may also include: based on the original encoding bit rate of the video to be encoded, obtaining the encoding bit rate of multiple related video frames after the target video frame by improving the original encoding bit rate; encoding multiple related video frames based on their respective encoding bit rates.

[0155] Here, the encoding bit rate of the target video frame is greater than or equal to the encoding bit rates of multiple related video frames, and the encoding bit rate of the previous video frame in any adjacent video frames in the multiple related video frames is greater than or equal to the encoding bit rate of the next video frame in the adjacent video frames.

[0156] In this example, the encoding bit rate of a target video frame such as a transition frame and a previous video frame in any adjacent video frames among the multiple related video frames is greater than or equal to the encoding bit rate of a subsequent video frame in the adjacent video frames.

[0157] Specifically, since the human eye has a weak ability to distinguish consecutively appearing video frames, when watching a video, a target video frame such as a transition frame and several video frames after the target video frame may be perceived as the same video scene. Therefore, the bit rate of the target video frame and several video frames after the target video frame can be increased. Since the video frame closer to the target video frame (such as a transition frame) may have a stronger correlation with the video scene (such as a transition scene), and vice versa, the encoding bit rate of the relevant video frame close to the target video frame can be increased more, and the degree of improvement of the encoding bit rate can be gradually reduced as the distance between the relevant video frame and the target video frame increases, so that the encoding bit rate gradually decreases from the target video frame until it falls back to the original encoding bit rate. In this way, the encoding bit rate of a video frame sequence starting from the target video frame can be increased, improving the overall viewing experience of the video scene.

[0158] As an example, the encoding bitrates of the target video frame and the multiple related video frames may be gradually reduced from the encoding bitrate of the target video frame to the original encoding bitrate at a preset attenuation step. For example, when the encoding bitrate of the target video frame is increased to n times the original encoding bitrate, the encoding bitrates of the related video frames after the target video frame may be expressed by the following equations (5) and (6):

[0159]

[0160] Wherein, br represents the encoding bit rate of the current frame (i.e., the target video frame or the related video frame), BR represents the original encoding bit rate, n is the above-mentioned improvement factor, indicating that the encoding bit rate of the target video frame is increased to n times of the original encoding bit rate, L represents the preset sliding window, l step represents the attenuation step length, dist represents the number of frames between the current frame and the target video frame, which can be the difference between the frame number of the current frame and the frame number of the target video frame, and floor represents rounding down. The above equations (5) and (6) indicate that within the sliding window L, the encoding bit rate of the relevant video frames after the target video frame is adjusted according to the attenuation step length l. step The bit rate is gradually reduced to the original encoding bit rate in a step-down manner.

[0161] By making the encoding bit rate of the video frames in the current video scene fall back to the original encoding bit rate in a step-like manner, the encoding bit rate of the video frames within the same attenuation step is the same, and there is no need to calculate the bit rate for each frame. Therefore, the encoding calculation amount can be minimized while achieving the improvement of the encoding quality of multiple video frames close to the target video frame such as the transition frame.

[0162] Although the above describes that the encoding bit rate of the target video frame and multiple related video frames is gradually reduced in a step-like attenuation manner with a preset attenuation step length, the embodiments of the present disclosure are not limited thereto, and other reduction methods may also be used, such as linear or nonlinear reduction.

[0163] In addition, in the example of determining the encoding bit rate of the target video frame by reducing the original encoding bit rate, the encoding bit rates of multiple related video frames after the target video frame can also be determined in a manner similar to the manner shown in the above formula (5), so that after the bit rate is reduced, the bit rate can be gradually restored (specifically, increased) to the original encoding bit rate.

[0164] For example, as described above, the original encoding bit rate can be weighted by using the reduction coefficient m, and the encoding bit rate of the target video frame can be configured to be m times the original encoding bit rate BR. In this case, the encoding bit rates of multiple related video frames after the target video frame can be determined by the following formula (7):

[0165]

[0166] In addition, the above describes an example process of improving the encoding bit rate of the detected target video frame. As an example, in an embodiment of the present disclosure, the number of times or frequency of bit rate improvement for the target video frame can also be limited to eliminate interference from some noisy scenes that do not require bit rate improvement. In such a noisy scene, multiple target video frames may be frequently detected. By introducing a bit rate improvement limitation mechanism, frequent bit rate improvements that result in a final exported bit rate that is too high can be avoided.

[0167] In one example, a stroboscopic scene can be detected. Specifically, the video encoding method can also include: in response to the number of target video frames determined within a second preset frame number starting from the current target video frame being greater than or equal to a preset number, encoding the current target video frame based on the original encoding bit rate; in response to the number of target video frames determined within the second preset frame number starting from the current target video frame being less than a preset number, executing, for the current target video frame, a step of obtaining the encoding bit rate of the target video frame by increasing the original encoding bit rate.

[0168] Specifically, when a target video frame is detected, it can be determined whether the number of target video frames contained in a second preset number of frames starting from the target video frame is greater than the preset number. If it is greater than the preset number, it can be considered that the current target video frame may belong to a stroboscopic scene. Since the human eye has weak visual resolution for stroboscopic scenes, the bit rate of the target video frame in the stroboscopic scene may not be increased, thereby reducing the encoding calculation amount and bandwidth cost.

[0169] Here, the second preset frame number and the preset number can be set according to actual needs. The second preset frame number can be, for example, but not limited to, 3 frames, and the preset number can be, for example, but not limited to, 2.

[0170] As an example, the above-mentioned stroboscopic scene detection can be realized by using a pre-read buffer. Specifically, as described above, in the process of encoding the video to be encoded, each video frame is sequentially stored in the pre-read buffer, and the video frames in the pre-read buffer are encoded in a Fifo order, where the maximum video frame capacity of the pre-read buffer can be the above-mentioned second preset number of frames.

[0171] In this way, after performing complexity analysis on the input frame, the complexity information of each frame can be stored in the pre-read buffer in sequence. After the pre-read buffer is full, encoding can be started from the earliest frame according to the Fifo order. If multiple target video frames are detected in the pre-read buffer, it means that the current target video frame is in a stroboscopic scene, and the bit rate of the current target video frame does not need to be increased.

[0172] Specifically, in response to the number of target video frames in the pre-read buffer being greater than or equal to a preset number, the current target video frame is encoded based on the original encoding bit rate; in response to the number of target video frames in the pre-read buffer being less than a preset number, for the current target video frame, a step of obtaining the encoding bit rate of the target video frame by increasing the original encoding bit rate is executed.

[0173] In addition, as an example, the second preset frame number may be greater than or equal to the first preset frame number described above. It can be seen that in an embodiment of the present disclosure, the first preset frame number and the second preset frame number may be determined with reference to the maximum video frame capacity of the pre-read buffer, so that the number of bit rate increases may be limited in different ways, such as by limiting the inter-frame complexity statistics within the first preset frame number and limiting the number of target video frames such as transition frames detected within the second preset frame number.

[0174] Additionally or alternatively, in another example, the number of times or frequency of performing bit rate boosting can be limited by setting a minimum frame number interval for bit rate boosting. Specifically, the video encoding method may also include: in response to the number of frames from the current target video frame to the reference target video frame being greater than a preset minimum frame number interval, for the current target video frame, performing a step of obtaining the encoding bit rate of the current target video frame by boosting the original encoding bit rate; in response to the number of frames from the current target video frame to the reference target video frame being less than or equal to a preset minimum frame number interval, encoding the current target video frame based on the original encoding bit rate, or, based on the distance from the reference target video frame, determining the encoding bit rate for the current target video frame and encoding the current target video frame.

[0175] Here, the reference target video frame may be a target video frame before the current target video frame. Specifically, after triggering the bit rate increase of the target video frame, the frame number distance for performing the next bit rate increase may be greater than the preset minimum frame number interval, where the frame number interval may refer to the number of video frames between two video frames (excluding the two endpoint video frames). The preset minimum frame number interval may be set according to actual needs, for example, it may be 20 frames.

[0176] As an example, the preset minimum frame number interval may be greater than the number of multiple related video frames after the target video frame, for example, may be greater than the preset sliding window L described above minus 1. In this way, when the bit rate of the related video frames after the target video frame is increased, it can be avoided that the bit rate of the related video frames cannot be correctly increased due to the limitation of the preset minimum frame number interval being set at the same time.

[0177] In addition, when the number of frames between the current target video frame and the reference target video frame is less than or equal to the preset minimum frame interval and is greater than the number of multiple related video frames after the target video frame (for example, the above-mentioned L minus 1), the current target video frame can be encoded based on the original encoding bit rate.

[0178] When the number of frames between the current target video frame and the reference target video frame is less than the preset minimum frame interval and less than the number of multiple related video frames after the reference target video frame (for example, L minus 1 above), the current target video frame may be one of the related video frames after the reference target video frame. In this case, the encoding bit rate of the current target video frame may be determined based on the number of frames between the current target video frame and the reference target video frame using the method of improving the encoding bit rate of the related video frames described above. For example, the encoding bit rate of the current target video frame may be determined using the above formula (4). The number of frames between the current target video frame and the reference target video frame is dist in formula (4), so that the current target video frame may be encoded based on the encoding bit rate determined thereby.

[0179] Additionally or alternatively, in another example, the video encoding method may further include: in response to the number of times the step of obtaining the encoding bit rate of the target video frame by increasing the original encoding bit rate is performed within the first preset video frame range is greater than the first preset number of times, encoding the target video frame within a second preset video frame range after the first preset video frame range based on the original encoding bit rate.

[0180] Here, the first preset video frame range may be greater than the above-mentioned preset minimum frame number interval. Specifically, after triggering a bit rate increase, if the first preset number of bit rate increases are triggered within the first preset video frame range, the bit rate increase will no longer be performed within the second preset video frame range. Here, the first preset video frame range and the second preset video frame range may be test experience values, for example, the first preset video frame range may be 50 frames, and the second preset video frame range may be 100 frames. In this way, even if the minimum frame number interval is met, multiple compensations may occur, for example, multiple increases are performed within a range of 50 frames. In this case, the bit rate increase of a video frame range (for example, 100 frames) can be stopped, further limiting the number of bit rate increases and reducing the computational and communication burdens.

[0181] When the target video frame is a transition frame, in some videos, there may be a large number of transitions that meet the conditions within a period of time. The large number of transitions mentioned here does not mean frequent strobing in a short period of time, but multiple transitions in a longer period of time, such as frequent switching of lenses in e-commerce product displays. In this case, by adopting the above-mentioned restriction method, subsequent transition scenes can be sacrificed within a certain period of time, such as the transition scenes within the above-mentioned second preset video frame range, to avoid a sudden increase in bit rate and control the encoding calculation amount of the entire video.

[0182] Additionally or alternatively, in another example, the video encoding method may further include: in response to the number of times the step of obtaining the encoding bit rate of the target video frame by increasing the original encoding bit rate is performed within a time window of preset length has reached a second preset number, encoding subsequent target video frames within the current time window based on the original encoding bit rate until reaching the next time window.

[0183] Specifically, the maximum number of bit rate increases within a time window of a preset length can be set. If the number of bit rate increases within the time window reaches the upper threshold, the bit rate will no longer be increased until the next unit time window is opened.

[0184] In this way, the number of times the bit rate is increased can be limited from the time dimension, which can be used together with the above-mentioned restriction mechanisms to limit the bit rate increase from different dimensions.

[0185] It should be noted that the above-mentioned rate increase limiting mechanisms can be used alone or in any combination. In addition, all variables in each rate increase limiting mechanism have an external setting interface, so that they can be issued through upper layer control and different parameter combinations can be configured according to different scene requirements.

[0186] In addition, as an example, in an embodiment of the present disclosure, in the process of determining the intra-frame complexity and the inter-frame complexity, the prediction algorithm can be optimized by fast algorithm and assembly, thereby reducing the overall complexity of the bit rate allocation, thereby ensuring that the prediction algorithm maintains a high degree of accuracy without introducing excessive computing power consumption.

[0187] Here, the fast algorithm may include an intra-frame fast algorithm and an inter-frame fast algorithm. The intra-frame fast algorithm may reduce the types of prediction modes, for example, including but not limited to reducing the prediction types to five directions: lower left, left, upper left, upper and upper right, as well as a DC mode and a Planar mode; the inter-frame fast algorithm may reduce the number of reference frames, for example, each frame may only refer to the previous frame for motion estimation.

[0188] In addition, the fast algorithm may also include reducing the resolution of the video frame. For example, the length and width of the video frame may be reduced to 1 / 2, 1 / 4 or other proportions of the original size according to a preset configuration to reduce the amount of calculation.

[0189] In addition, the fast algorithm may also include reducing the inter-frame prediction search range. For example, during the inter-frame prediction search process, in response to the inter-frame complexity of the coding block being less than a preset threshold, the inter-frame prediction search process may be skipped.

[0190] Here, assembly optimization may include intra-frame assembly optimization and inter-frame assembly optimization. Intra-frame assembly optimization may, for example, but is not limited to, performing arm64 assembly optimization on all prediction modes to increase computational parallelism. Inter-frame assembly optimization may, for example, but is not limited to, performing single instruction multiple data (SIMD) assembly optimization on operations such as pixel interpolation in motion estimation and motion compensation, which may also increase computational parallelism.

[0191] By adding fast algorithms and assembly optimization, the calculation of this method can have no obvious impact on the overall encoding complexity, significantly improving the user experience when producing and consuming videos.

[0192] The above describes a video encoding method according to an exemplary embodiment of the present disclosure. The video encoding method proposes to combine video pre-analysis and transition detection for the hardware encoder, effectively improves the bit rate allocation efficiency in transition scenarios, while saving the encoding bit rate in simple scenarios, improves the overall subjective picture quality of the video, and reduces the overall encoding bit rate. The model has no obvious effect on the overall complexity, which significantly improves the user experience when producing and consuming videos.

[0193] The video encoding method according to the embodiment of the present disclosure was verified on the iOS VideoToolBox hardware encoder. Under 100 test sequences, the average bit rate dropped by 4.55%, and the subjective evaluation index KVQ (Kuaishou VideoQuality, Kuaishou video quality evaluation index) increased by 0.0105. The image quality improvement reached the level visible to the human eye. At the same time, the image quality blur cases (bad cases) of some transition scenes were improved, and the overall calculation complexity was extremely low. The complexity increased by only less than 4% with the VTB hardware encoder as the reference benchmark, which solved the pain point that the hardware encoders on the market could not adjust the code control to improve the image quality according to the scene requirements. In addition, this method can also be adapted to other hardware encoders other than VTB, such as but not limited to MediaCodec of the Android platform. As long as the hardware encoder supports the real-time bit rate setting interface, this method can be applied.

[0194] In an application example of a video encoding method according to an exemplary embodiment of the present disclosure, compared with a video picture exported by an original encoder that does not adopt this method, it can be observed that the overall picture quality of the video is improved after the bit rate is increased by the above-mentioned video encoding method.

[0195] It can be seen that according to the video encoding method of the embodiment of the present disclosure, a content-adaptive encoder bit rate determination scheme can be proposed, which reduces the overall encoding bit rate while obtaining better subjective picture quality by more reasonably allocating codewords to scenes of different complexity.

[0196] In addition, according to the video encoding method of the embodiment of the present disclosure, a simplified complexity pre-analysis module can be introduced to perform complexity analysis only on the first video frame and the subsequent second video frame in the video, so as to obtain a rough distribution of the complexity of the entire video scene, which is used to measure the overall complexity level of the video, thereby more reasonably determining the encoding bit rate of the current target video frame.

[0197] In addition, according to the video encoding method of the embodiments of the present invention, real-time complexity analysis and transition detection can be used to provide more codewords for video scenes with complex pictures, such as transition scenes, which are sensitive to the human eye and have high bit rate requirements, thereby improving the subjective viewing experience of videos in multiple scenarios.

[0198] In addition, according to the video encoding method of the embodiment of the present invention, the obtained complexity prior information can be combined to fine-tune the bit rate compensation amplitude of the transition scene. For video scenes with relatively low complexity, the compensation intensity can be appropriately reduced, and the encoding bit rate can be lowered for simple scenes, thereby achieving overall bit rate savings.

[0199] Figure 6 is a block diagram of a video encoding device according to an exemplary embodiment. Figure 6 , the video encoding device 600 includes a first determining unit 610, a second determining unit 620, a third determining unit 630 and an encoding unit 640.

[0200] The first determining unit 610 is configured to determine a target video frame in a video to be encoded.

[0201] The second determining unit 620 is configured to determine the complexity of the current video scene corresponding to the target video frame.

[0202] The third determining unit 630 is configured to determine the encoding bit rate of the target video frame according to the complexity of the current video scene.

[0203] The encoding unit 640 is configured to encode the target video frame based on the encoding bit rate of the target video frame.

[0204] As an example, the third determining unit 630 is further configured to: determine complexity distribution information of the video to be encoded; and determine the encoding bit rate of the target video frame according to the complexity of the current video scene and the complexity distribution information.

[0205] As an example, the third determining unit 630 is further configured to obtain the encoding bit rate of the target video frame by increasing the original encoding bit rate of the video to be encoded according to the complexity and complexity distribution information of the current video scene.

[0206] As an example, the video encoding device 600 also includes a fourth determination unit 650, and the fourth determination unit 650 is further configured to determine the complexity distribution information in the following manner: determine the complexity of a second video frame after a first video frame in the video to be encoded, wherein the first video frame is an intra-frame prediction frame, and the second video frame is a forward prediction frame that is a preset number of frames after the first video frame; determine the complexity distribution information based on the complexity of multiple first video frames and multiple second video frames in the video to be encoded.

[0207] As an example, the third determination unit 630 is further configured to: determine a boosting coefficient for improving the original encoding bit rate based on the complexity and complexity distribution information of the current video scene; and obtain the encoding bit rate of the target video frame based on the boosting coefficient and the original encoding bit rate.

[0208] As an example, the complexity distribution information includes a maximum value of the complexity distribution and a minimum value of the complexity distribution, wherein the maximum value of the complexity distribution is determined based on the maximum value of the complexity of multiple first video frames and multiple second video frames, and the minimum value of the complexity distribution is determined based on the minimum value of the complexity of multiple first video frames and multiple second video frames, and the third determination unit 630 is also configured to determine the enhancement coefficient in the following manner: based on the maximum value of the complexity distribution, the minimum value of the complexity distribution and the complexity of the current video scene, a preset coefficient is adjusted to obtain the enhancement coefficient, wherein the preset coefficient represents the maximum multiple of the original encoding bit rate.

[0209] As an example, the video to be encoded includes multiple target video frames, and at least two target video frames correspond to different lifting coefficients, wherein the lifting coefficient is positively correlated with the complexity of the video scene corresponding to the target video frame.

[0210] As an example, the third determination unit 630 is also configured to: in response to the complexity of the current video scene being higher than a first preset value, execute the step of obtaining the encoding bit rate of the target video frame by increasing the original encoding bit rate of the video to be encoded according to the complexity and complexity distribution information of the current video scene; in response to the complexity of the current video scene and the complexity of the first second video frame after the target video frame being lower than a second preset value, obtain the encoding bit rate of the target video frame by reducing the original encoding bit rate; in response to the complexity of the current video scene being lower than the first preset value and higher than the second preset value, determine the original encoding bit rate as the encoding bit rate of the target video frame, wherein the first preset value is greater than the second preset value.

[0211] As an example, the second determination unit 620 is further configured to determine the complexity of the current video scene in the following manner: by counting the complexity of video frames within a first preset number of frames starting from the target video frame, the complexity of the current video scene is obtained.

[0212] As an example, the first determining unit 610 is further configured to determine the target video frame in the following manner: determining a video frame whose difference between the inter-frame complexity and the intra-frame complexity in the to-be-encoded video exceeds a difference threshold as the target video frame.

[0213] As an example, the encoding unit 640 is configured to: based on the original encoding bit rate of the video to be encoded, obtain the encoding bit rate of multiple related video frames after the target video frame by improving the original encoding bit rate; based on the respective encoding bit rates of the multiple related video frames, encode the multiple related video frames, wherein the encoding bit rate of the target video frame is greater than or equal to the encoding bit rate of the multiple related video frames, and the encoding bit rate of the previous video frame in any adjacent video frames in the multiple related video frames is greater than or equal to the encoding bit rate of the next video frame in the adjacent video frames.

[0214] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0215] Figure 7 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 includes a processor 710 and a memory 720 for storing processor executable instructions. Here, when the processor executable instructions are executed by the processor, the processor executes the video encoding method as described in the above exemplary embodiments.

[0216] As an example, the electronic device 700 may be a single device, or may be any device or circuit that can execute the above instructions (or instruction sets) individually or jointly. The processor 710 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. The processor 710 may run instructions or codes stored in the memory 720. Instructions and data may also be sent and received through a network via a network interface device, which may use any known transmission protocol.

[0217] The memory 720 may be integrated with the processor 710. The memory 720 and the processor 710 may be operatively coupled or may communicate with each other, for example, via an I / O port, a network connection, etc. In addition, the electronic device 700 may further include a video display and a user interaction interface.

[0218] In an exemplary embodiment, a computer-readable storage medium may also be provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can perform the video encoding method as described in the above exemplary embodiment. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: a read-only memory (ROM), a random access memory (RAM), a random access programmable read-only memory (PROM), an electrically erasable programmable read-only memory (EEPROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a non-volatile memory, a CD-ROM, a CD-R, a CD+R, a CD-RW, a CD+RW, a DVD-ROM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, a DVD-RAM, a BD-ROM, a BD-R, a BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, MultiMediaCard, Secure Digital (SD) card or Extreme Digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device configured to store the computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program.

[0219] In an exemplary embodiment, a computer program product may also be provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the video encoding method as described in the exemplary embodiment above is implemented.

[0220] In an exemplary embodiment, a method for generating a bit stream may also be provided. The method includes: generating a bit stream according to the video encoding method as described in the above exemplary embodiment.

[0221] In an exemplary embodiment, a computer program product may also be provided, having instructions for storing a bit stream, the bit stream comprising: video data generated according to the video encoding method as described in the above exemplary embodiments.

[0222] Figure 8 A computing environment 810 is shown coupled to a user interface 850. The computing environment 810 may be part of a data processing server. The computing environment 810 includes a processor 820, a memory 830, and an input / output (I / O) interface 840.

[0223] The processor 820 generally controls the overall operation of the computing environment 810, such as operations associated with display, data acquisition, data communication, and image processing. The processor 820 may include one or more processors for executing instructions to perform all or some steps in the above method. In addition, the processor 820 may include one or more modules that facilitate the interaction between the processor 820 and other components.

[0224] The memory 830 is configured to store various types of data to support the operation of the computing environment 810. The memory 830 may include predetermined software 832. Examples of such data include instructions for any application or method operating on the computing environment 810, video data sets, image data, etc. The memory 830 may be implemented using any type of volatile or non-volatile memory device or combination thereof.

[0225] The I / O interface 840 provides an interface between the processor 820 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 840 may be coupled to an encoder and a decoder.

[0226] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 830 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 820 in a computing environment 810 to perform the above method. In one example, the plurality of programs can be executed by a processor 820 in a computing environment 810 to (for example, from Figure 2 The video encoder 20 in the computing environment 810 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 820 in the computing environment 810 to perform the above decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 820 in the computing environment 810 to perform the above encoding method to encode the video information into a bit stream or data stream, and can also be executed by the processor 820 in the computing environment 810 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information used by the video decoder 30 in FIG. 3 when decoding video data.

[0227] In an embodiment, a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is provided. In an embodiment, a bit stream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0228] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 820); and a non-transitory computer-readable storage medium or memory 830 in which multiple programs that can be executed by the one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0229] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, wherein the bitstream includes coded video information generated by the above encoding method or coded video information to be decoded by the above decoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 830 is also provided, wherein the plurality of programs can be executed by a processor 820 in a computing environment 810 for performing the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0230] In an embodiment, a method for storing a bit stream is also provided, comprising: storing the bit stream on a digital storage medium, wherein the bit stream comprises encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method.

[0231] In an embodiment, a method for transmitting a bit stream generated by the above encoder is also provided. In an embodiment, a method for receiving a bit stream to be decoded by the above decoder is also provided.

[0232] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0233] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0234] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

[0235] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0236] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that: The video encoding method comprises: Determining a target video frame in a video to be encoded; Determining the complexity of the current video scene corresponding to the target video frame; Determining the encoding bit rate of the target video frame according to the complexity of the current video scene; The target video frame is encoded based on the encoding bit rate of the target video frame.

2. The video encoding method according to claim 1, characterized in that Determining the encoding bit rate of the target video frame according to the complexity of the current video scene includes: Determining complexity distribution information of the video to be encoded; The encoding bit rate of the target video frame is determined according to the complexity of the current video scene and the complexity distribution information.

3. The video encoding method according to claim 2, characterized in that: The step of determining the encoding bit rate of the target video frame according to the complexity of the current video scene includes: According to the complexity of the current video scene and the complexity distribution information, the encoding bit rate of the target video frame is obtained by improving the original encoding bit rate of the video to be encoded.

4. The video encoding method according to claim 2 or 3, characterized in that: The complexity distribution information is determined by: Determine the complexity of a second video frame after a first video frame in the video to be encoded, wherein the first video frame is an intra-frame prediction frame, and the second video frame is a forward prediction frame that is a preset number of frames after the first video frame and away from the second video frame; The complexity distribution information is determined based on the complexity of a plurality of first video frames and a plurality of second video frames in the video to be encoded.

5. The video encoding method according to claim 4, characterized in that: The step of obtaining the encoding bit rate of the target video frame by improving the original encoding bit rate of the video to be encoded according to the complexity of the current video scene and the complexity distribution information includes: Determining, based on the complexity of the current video scene and the complexity distribution information, an improvement coefficient for improving the original encoding bit rate; Based on the lifting coefficient and the original encoding bit rate, the encoding bit rate of the target video frame is obtained.

6. The video encoding method according to claim 5, characterized in that: The complexity distribution information includes a maximum value of the complexity distribution and a minimum value of the complexity distribution, wherein the maximum value of the complexity distribution is determined based on a maximum value of the complexity of the multiple first video frames and the multiple second video frames, and the minimum value of the complexity distribution is determined based on a minimum value of the complexity of the multiple first video frames and the multiple second video frames, The lifting coefficient is determined in the following manner: Based on the maximum value of the complexity distribution, the minimum value of the complexity distribution and the complexity of the current video scene, the preset coefficient is adjusted to obtain the boost coefficient, wherein the preset coefficient represents the maximum multiple of boosting the original encoding bit rate.

7. The video encoding method according to claim 4, characterized in that: The step of determining the encoding bit rate of the target video frame according to the complexity of the current video scene also includes: In response to the complexity of the current video scene being higher than a first preset value, executing the step of obtaining the encoding bit rate of the target video frame by increasing the original encoding bit rate of the video to be encoded according to the complexity of the current video scene and the complexity distribution information; In response to the complexity of the current video scene and the complexity of the first second video frame after the target video frame being lower than a second preset value, obtaining the encoding bit rate of the target video frame by reducing the original encoding bit rate; In response to the complexity of the current video scene being lower than the first preset value and higher than the second preset value, determining the original encoding bit rate as the encoding bit rate of the target video frame, Wherein, the first preset value is greater than the second preset value.

8. The video encoding method according to claim 1, characterized in that: The complexity of the current video scene is determined in the following manner: The complexity of the current video scene is obtained by counting the complexity of video frames within a first preset number of frames starting from the target video frame.

9. The video encoding method according to claim 1, characterized in that: The target video frame is a transition frame in the video to be encoded, wherein the target video frame is determined by: A video frame in the to-be-encoded video whose difference between the inter-frame complexity and the intra-frame complexity exceeds a difference threshold is determined as the target video frame.

10. The video encoding method according to claim 1, characterized in that: The video encoding method further comprises: Based on the original encoding bit rate of the video to be encoded, the encoding bit rates of multiple related video frames after the target video frame are obtained by increasing the original encoding bit rate; encoding the multiple related video frames based on respective encoding bit rates of the multiple related video frames, Among them, the encoding bit rate of the target video frame is greater than or equal to the encoding bit rates of the multiple related video frames, and the encoding bit rate of the previous video frame in any adjacent video frames among the multiple related video frames is greater than or equal to the encoding bit rate of the next video frame in the adjacent video frames.

11. A video encoding device, characterized in that: The video encoding device comprises: A first determining unit is configured to determine a target video frame in a video to be encoded; A second determining unit is configured to determine the complexity of the current video scene corresponding to the target video frame; A third determining unit is configured to determine an encoding bit rate of a target video frame according to the complexity of the current video scene; The encoding unit is configured to encode the target video frame based on the encoding bit rate of the target video frame.

12. An electronic device, characterized in that: The electronic device comprises: Processor; and a memory for storing processor executable instructions, When the processor executable instructions are executed by the processor, the processor executes the video encoding method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the video encoding method according to any one of claims 1 to 10.

14. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the video encoding method according to any one of claims 1 to 10 is implemented.

15. A method for generating a bit stream, characterized in that: include: A bit stream is generated according to the video encoding method according to any one of claims 1 to 10.

Images