Video encoding method and device, bit stream generation method, electronic device, storage medium and program product

Abstract

The present disclosure relates to a video encoding method and apparatus, a bitstream generation method, an electronic device, a storage medium and a program product. The video encoding method includes: determining the block energy of the current encoding block, wherein the block energy represents the amount of information contained in the encoding block; determining whether to divide the current encoding block into sub-blocks according to the block energy of the current encoding block; and encoding the current encoding block based on the determination result. The present disclosure solves the problem of reduced encoding quality due to unreasonable division of encoding blocks, and can perform block division more reasonably, avoiding the problem of reduced encoding quality due to unreasonable division of encoding blocks, thereby improving the quality of video encoding.

Description

Technical Field

[0001] The present disclosure relates to the field of video technology, and in particular to a video encoding method and apparatus, a bit stream generation method, an electronic device, a storage medium, and a program product. Background Art

[0002] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices send and receive or otherwise transmit digital video data through a communication network, and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited storage resources of the storage device, before the video data is transmitted or stored, the video data can be compressed using a video codec according to one or more video codec standards. For example, video codec standards include general video codec (VVC), joint exploration test model (JEM), high efficiency video codec (HEVC / H.265), advanced video codec (AVC / H.264), moving picture expert group (MPEG) codec, etc. Video codecs generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize the inherent redundancy in video data. Video codecs are intended to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0003] In existing encoders, due to considerations of computational complexity, many fast algorithms are usually designed to accelerate encoding. For example, the RDO process of sub-block division of some blocks can be skipped by combining the rate distortion optimization (RDO) cost of the current coding block, the division depth of neighboring blocks and other information.

[0004] Specifically, since the coding units (CodingUnit, CU) adjacent to the current coding block have all been encoded, the division depth of the current coding block will be directly estimated according to the division depth of the adjacent coding units, thereby skipping the subsequent sub-block division RDO process; or the subsequent sub-block division RDO process may be skipped directly according to the size of the RDO cost of the current coding block.

[0005] However, in such a block division method, since the angle of reference information is relatively limited, there may be a problem of reduced encoding quality due to unreasonable division of encoding blocks. Summary of the invention

[0006] The present disclosure provides a video encoding method and apparatus, a bit stream generation method, an electronic device, a storage medium and a program product, so as to at least solve the problem in the related art that the encoding quality is reduced due to the unreasonable division of the encoding block. The technical solution of the present disclosure is as follows:

[0007] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, the video encoding method comprising: determining a block energy of a current encoding block, wherein the block energy represents an amount of information contained in the encoding block; determining whether to perform sub-block division on the current encoding block based on the block energy of the current encoding block; and encoding the current encoding block based on the determination result.

[0008] Optionally, determining whether to perform sub-block division on the current coding block according to the block energy of the current coding block includes: in response to the block energy of the current coding block being within a preset energy range, determining whether to perform sub-block division on the current coding block according to block information of the current coding block, wherein the block information includes: a frame type of a coding frame in which the current coding block is located and / or a division depth of the current coding block; and in response to the block energy of the current coding block not being within the preset energy range, determining not to perform sub-block division on the current coding block.

[0009] Optionally, determining whether to divide the current coding block into sub-blocks according to the block information of the current coding block includes: in response to the block information of the current coding block satisfying a preset condition, determining to divide the current coding block into sub-blocks; in response to the block information of the current coding block not satisfying the preset condition, determining not to divide the current coding block into sub-blocks, the preset condition including at least one of the following items: the frame type of the coding frame in which the current coding block is located is not a bidirectional prediction frame; the division depth of the current coding block is less than a first preset depth.

[0010] Optionally, the current coding block is an intra-frame coding block or an inter-frame coding block, and the preset energy range includes a first block energy range for the intra-frame coding block and a second block energy range for the inter-frame coding block, wherein the second block energy range is included in the first block energy range.

[0011] Optionally, the first block energy range is 2.7<E<3.6, the second block energy range is 2.7<E<3.2, and E represents the block energy of the coding block.

[0012] Optionally, the block energy of the current coding block is determined in the following manner: statistics are performed on the pixel value of each pixel in the current coding block to obtain a statistical result; and based on the statistical result, the block energy of the current coding block is determined.

[0013] Optionally, the statistical result includes a sum of squares of pixel values ​​of each pixel in the current coding block and a sum of pixel values ​​of each pixel in the current coding block.

[0014] Optionally, determining the block energy of the current coding block based on the statistical result includes: determining the ratio between the square of the sum value and the number of pixels in the current coding block; and determining the difference between the square sum and the ratio as the block energy of the current coding block.

[0015] Optionally, the video encoding method also includes an adaptive quantization process, wherein the block energy of the current coding block is determined in the following manner: obtaining a quantization result of the adaptive quantization process; determining the block energy of a target coding block based on the quantization result, wherein the target coding block is a coding block corresponding to the position of the current coding block and having a size less than or equal to the size of the current coding block; and determining the block energy of the current coding block based on the relationship between the size of the current coding block and the size of the target coding block and the block energy of the target coding block.

[0016] Optionally, in response to determining to divide the current coding block into sub-blocks, the video encoding method further includes: in response to satisfying at least one of the following conditions, increasing the encoding cost of the intra-frame prediction mode of the sub-blocks obtained by dividing the current coding block: the division depth of the current coding block is greater than a second preset depth; the prediction mode of the parent block of the current coding block is an inter-frame prediction mode; the block energy of the current coding block is within a preset block energy range.

[0017] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0018] According to the video encoding method of the embodiment of the present disclosure, the block energy of the current coding block can be determined, and based on the block energy, it is determined whether to divide the current coding block into sub-blocks to encode the current coding block. Here, since the block energy can characterize the amount of information contained in the coding block, the higher the block energy, the more complex the picture content of the coding block may be. Therefore, the coding blocks with complex contents can be identified by determining the block energy, and the block division can be performed more reasonably, avoiding the problem of reduced coding quality due to unreasonable division of the coding blocks, thereby improving the video coding quality.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0021] Figure 1 is a block diagram showing an exemplary system for encoding and decoding video blocks according to an exemplary embodiment of the present disclosure.

[0022] Figure 2 is a block diagram showing an exemplary video encoder according to an exemplary embodiment of the present disclosure.

[0023] Figure 3 is a block diagram showing an exemplary video decoder according to an exemplary embodiment of the present disclosure.

[0024] Figure 4A and Figure 4B The block diagram shows how to recursively divide a frame into a plurality of video blocks of different sizes and shapes in a video encoding method according to an exemplary embodiment of the present disclosure.

[0025] Figure 5 It is a schematic diagram of an example implementation scenario of a video encoding method according to an exemplary embodiment of the present disclosure.

[0026] Figure 6 It is a flowchart of a video encoding method according to an exemplary embodiment of the present disclosure.

[0027] Figure 7 The figure is a flowchart showing the steps of determining block energy in a video encoding method according to an exemplary embodiment of the present disclosure.

[0028] Figure 8 It is a schematic diagram comparing the difference in mode selection before and after block partition optimization in an application example of a video encoding method according to an exemplary embodiment of the present disclosure.

[0029] Fig. 9 is a block diagram showing a video encoding device according to an exemplary embodiment of the present disclosure.

[0030] Fig.10 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

[0031] Fig.11 is a diagram of a computing environment coupled with a user interface according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0034] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0035] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the drawings are used to distinguish objects, but not to describe any specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0037] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0038] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0039] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0040] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0041] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0042] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0043] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0044] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0045] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0046] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0047] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0048] like Figure 2 As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0049] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0050] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0051] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0052] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0053] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0054] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0055] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0056] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0057] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0058] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0059] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0060] In some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0061] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0062] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0063] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0064] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0065] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0066] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0067] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0068] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0069] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0070] As described above, in the related art, since the angle of reference information in block division decision is relatively limited, only coding information such as the depth of the neighboring block and the RDO cost of the current coding block that is not divided is used for reference. Therefore, there may be a problem of reduced coding quality due to unreasonable division of the coding block.

[0071] For example, without considering the content characteristics of the current coding block itself, if the division depth in the coded block is shallow, it is easy to propagate and cause the subsequent blocks to tend to choose a lower level of division. In particular, for areas with complex texture information, large-sized coding units are more likely to cause texture smearing and artifacts, resulting in poor coding quality and affecting the subjective picture quality experience.

[0072] In order to solve at least part of the above problems, exemplary embodiments of the present disclosure provide a video encoding method, a video encoding device, a bit stream generation method, an electronic device, a computer readable storage medium, and a computer program product. FIG. 4A to FIG. 11 Describe in detail.

[0073] In order to better understand the embodiments of the present disclosure, reference is first made to Figure 4A and Figure 4B An example process of dividing coding blocks in video codec is described.

[0074] Take the video coding and decoding technology of High Efficiency Video Coding (HEVC) as an example. Figure 4A As shown in the figure, a frame of image can be first divided into 64×64 coding tree units (Coding Tree Unit, CTU), each CTU consists of a luminance coding tree block (Coding Tree Block, CTB) and two corresponding chrominance CTBs and corresponding syntax elements. CTU can be further divided into multiple square coding units (Coding Unit, CU) through a recursive quadtree. CU is the most basic coding unit of HEVC, with a maximum size of 64×64 and a minimum size of 8×8.

[0075] In the bitstream, the split_cu_flag flag can be used to indicate whether the current CU should continue to be divided into sub-CUs. Figure 4B As shown in the figure, a 64×64 CTU can be divided into four 32×32 CUs, and each 32×32 CU can be divided into four 16×16 CTUs, and so on. The smallest luminance CTU is 8×8, so it can only be divided into 3 layers at most. In quadtree partitioning, the size of the block can be determined as long as the partition depth is given.

[0076] It should be noted that here, the division of the video block is exemplarily described by taking quadtree division as an example, so as to facilitate a better understanding of the exemplary embodiments of the present invention, and the exemplary embodiments of the present invention may also be applied to other division methods.

[0077] According to an exemplary embodiment of the present disclosure, a video encoding method is provided. The video encoding method can be applied to any video encoding scenario. Figure 5 An example implementation scenario of a video encoding method according to an exemplary embodiment of the present disclosure is given.

[0078] like Figure 5 As shown, when a user uses a video application client on a user terminal (e.g., mobile phone 511, desktop computer 512, tablet computer 513, etc.) to request to watch video content from a server or video distribution platform 530 through network 520, the server or video distribution platform 530 can send the requested video content to user terminals 511, 512 and / or 513 through network 520, and the user can watch the received video content through the video application client.

[0079] In the above process, the server or video distribution platform 530 can encode the transmitted video content, and in the encoding process, the transmitted video can be encoded according to the video encoding method of the exemplary embodiment of the present disclosure. Specifically, the server or video distribution platform 530 can determine the block energy of the current coding block, wherein the block energy represents the amount of information contained in the coding block; determine whether to divide the current coding block into sub-blocks according to the block energy of the current coding block; and encode the current coding block based on the determination result.

[0080] It should be noted that although the above description uses a server or video distribution platform as an example, it is only an example. The executor of the video encoding method can be any electronic device, which may include, for example, smart phones, tablet computers, laptops, digital assistants, wearable devices, vehicle terminals, etc., which are physical devices that serve as hardware codecs, and may also include software that runs on physical devices as software video encoders.

[0081] It should also be noted that although the video encoding application scenario in which a user terminal requests to watch a video is described here as an example, it should be understood that the application scenario of the video encoding method according to the present disclosure is not limited to this, and it can also be applied to any other application scenario involving video encoding.

[0082] Figure 6 It is a flow chart of a video encoding method according to an exemplary embodiment. As described above, according to the video encoding method of the exemplary embodiment of the present disclosure, it is possible to determine the encoding bit rate in combination with the complexity analysis of the video scene, solve the problem of unsatisfactory bit rate allocation for different video scenes, and thus can adaptively allocate the bit rate according to the video scene, and improve the subjective picture quality experience of the video in different video scenes.

[0083] like Figure 6 As shown, the video encoding method according to an exemplary embodiment of the present disclosure may include the following steps:

[0084] In step S610, the block energy of the current coding block may be determined.

[0085] Here, the block energy can represent the amount of information contained in the coding block. Specifically, for a coding block, the more complex the picture content is, the higher the energy is, so the complexity of the content of the coding block can be identified by the energy.

[0086] As an example, the block energy of the current coding block may be determined in the following manner: performing statistics on the pixel value of each pixel in the current coding block to obtain a statistical result; and determining the block energy of the current coding block based on the statistical result.

[0087] Specifically, there is a correlation between the amount of information in the coding block and the pixel value of each pixel in the coding block. By using this correlation, the block energy can be determined by statistically analyzing the pixel values, so that the amount of information in the coding block can be evaluated more accurately.

[0088] As an example, the statistical result may include a square sum of pixel values ​​of each pixel in the current coding block and a sum of pixel values ​​of each pixel in the current coding block.

[0089] For example, the block energy of the current coding block can be determined by: determining the ratio between the square of the sum of pixel values ​​and the number of pixels in the current coding block; and determining the difference between the sum of squares of pixel values ​​and the ratio as the block energy of the current coding block.

[0090] Specifically, for a coding block, the larger the variance of each pixel value, the higher the macroblock complexity; the smaller the variance of each pixel value, the lower the macroblock complexity. Based on this, a relationship between the variance of each pixel value and the block complexity or block energy can be established.

[0091] For example, suppose the coding block includes n pixels, and the pixel values ​​are x0, x1, x2, ..., x n-1 , the average value of each pixel value is M, and the variance of each pixel value can be expressed by the following formula (1):

[0092]

[0093] Among them, s 2 represents the variance, and n represents the number of pixels.

[0094] Here, the variance of each pixel value refers to the variance between pixels included in the coding block itself.

[0095] By expanding the above formula (1), we can get the following formula (2):

[0096]

[0097] Based on the above formula (2), by transforming the expression, we can get the following formula (3):

[0098]

[0099] Based on the formula (3), it can be seen that the block energy can be correlated with the sum of squared differences (SSD) of the pixel values ​​of each pixel in the current coding block and the sum of the pixel values ​​(SUM) of each pixel in the current coding block. For example, by transforming the formula (3), the following formula (4) can be obtained:

[0100]

[0101] Therefore, an example expression of the block energy of the coding block can be obtained, as shown in the following equation (5):

[0102]

[0103] Taking a 16×16 CU as an example, the CU may include 256 pixels, and the block energy of the CU can be expressed as the following formula (6):

[0104]

[0105] In the video encoder, the pixel square sum SSD and the sum function are called frequently. Therefore, by establishing the relationship between block energy and the pixel square sum SSD and the pixel sum value SUM, the calculation of block energy can be simplified. While realizing the block energy calculation, the calculation efficiency is also well optimized. For example, the pixel square sum SSD and the pixel sum value SUM calculated in other encoding processes can be directly used to determine the block energy of the encoding block.

[0106] In the embodiment of the present disclosure, the manner of determining the block energy is not limited to the above example. As another example, the video encoding method may further include an adaptive quantization process. In this case, Figure 7 As shown, the block energy of the current coding block can be determined in the following way:

[0107] In step S710, a quantization result of an adaptive quantization process may be obtained.

[0108] In step S720, block energy of the target coding block may be determined based on the quantization result.

[0109] Here, the target coding block may be a coding block corresponding to the position of the current coding block and having a size smaller than or equal to the size of the current coding block.

[0110] Specifically, in a video encoder, an adaptive quantization (AQ) mode algorithm can be used to perform quantization processing to obtain corresponding quantization results. In this algorithm, the block energy information of a coding block of a preset size is counted, for example, the energy information of a 16×16 CU can be counted. Therefore, based on the quantization result of the algorithm, the block energy of a target coding block whose size corresponding to the current coding block position is less than or equal to the current coding block size can be obtained.

[0111] In step S730, the block energy of the current coding block may be determined based on the relationship between the size of the current coding block and the size of the target coding block and the block energy of the target coding block.

[0112] In this step, since the size of the current coding block is greater than or equal to the size of the target coding block, the block energy of the current coding block can be measured based on the block energy of the target coding block by analyzing the size relationship between the current coding block and the target coding block.

[0113] Here, the energy average of the block energies of the target coding blocks included in the current coding block can be used as the block energy of the current coding block. For example, the block energy of the current coding block can be determined by the following formula (7):

[0114]

[0115] Among them, E n×n represents the block energy of the current coding block of size n, for example, the current coding block is an n×n block; E m_i Represents the i-th target coding block of size m. For example, if the target coding block is an m×m block, the value of i is [1, k 2 Here, k=n / m, which may indicate the relationship between the size of the current coding block and the size of the target coding block, or it may indicate that the current coding block of size n contains m target coding blocks.

[0116] Taking the current coding block as n×n CU as an example, the block energy of the i-th 16×16 target coding block contained in the CU is E 16_i , then the block energy of the current coding block containing such a target coding block can be expressed as the following formula (8):

[0117]

[0118] In the above manner, the block energy obtained in the AQ process is used to simplify the calculation amount of the block energy of the current coding block, reuse the calculation results in the AQ process, and improve the calculation efficiency.

[0119] It should be noted that, although several examples of determining the block energy of the current coding block are described above, the embodiments of the present disclosure are not limited thereto, and other methods may also be used as long as the block energy can be determined.

[0120] In step S620, it may be determined whether to divide the current coding block into sub-blocks according to the block energy of the current coding block.

[0121] Since the block energy of a coding block can characterize the complexity of the content in the block, when the block energy is at a relatively high level, the current coding block can be divided into sub-blocks according to actual conditions; when the block energy is at a relatively low level, the current coding block may not be divided into sub-blocks.

[0122] In one example, in response to the block energy of the current coding block being within a preset energy range, it can be determined to divide the current coding block into sub-blocks; in response to the block energy of the current coding block not being within the preset energy range, it can be determined not to divide the current coding block into sub-blocks. Here, the preset energy range can be set according to actual needs.

[0123] In the embodiments of the present disclosure, in addition to block energy, other information of the coding block can also be considered, and the block energy information can be combined to jointly decide whether to perform sub-block division, so that the coding block can be divided more accurately and reasonably, further improving the encoding quality.

[0124] Specifically, step S620 may include: in response to the block energy of the current coding block being within a preset energy range, determining whether to perform sub-block division on the current coding block according to the block information of the current coding block; in response to the block energy of the current coding block not being within the preset energy range, determining not to perform sub-block division on the current coding block.

[0125] Here, when the block energy of the current coding block is within the preset energy range, it is also possible to further decide whether to perform sub-block division based on other information of the block itself. For example, the block information may include: the frame type of the coding frame in which the current coding block is located and / or the division depth of the current coding block.

[0126] As an example, the step of determining whether to divide the current coding block into sub-blocks based on the block information of the current coding block may include: in response to the block information of the current coding block satisfying a preset condition, determining to divide the current coding block into sub-blocks; in response to the block information of the current coding block not satisfying the preset condition, determining not to divide the current coding block into sub-blocks.

[0127] Here, the preset condition may include at least one of the following items: the frame type of the coding frame where the current coding block is located is not a bidirectional prediction frame; the partition depth of the current coding block is less than the first preset depth.

[0128] Specifically, the embodiment of the present disclosure, under the condition of turning off the fast algorithm of all skipped sub-blocks, counts the correlation between the result of the encoder's final selection of sub-block division and the block energy under different frame types, wherein the correlation with the non-bidirectionally predicted frame (Bidirectionally Predicted Frame, B frame) is higher, while for the B frame, the correlation between the result of the sub-block division and the block energy is lower. Therefore, when the frame type of the coding frame where the current coding block is located is not a bidirectionally predicted frame, it can be determined that the current coding block is divided into sub-blocks; when the frame type of the coding frame where the current coding block is located is a bidirectionally predicted frame, the current coding block may not be divided into sub-blocks.

[0129] In addition, during the encoding process, a larger coding block (such as CU) has a greater impact on the subjective image quality, while a smaller coding block has a relatively insignificant impact on the image quality. If a smaller coding block is forced to undergo sub-block division RDO, the overall encoding speed will be greatly affected, while the improvement in subjective image quality will be limited. Therefore, when the division depth cu_depth of the current coding block is less than the first preset depth, it may be determined that the current coding block is to be divided into sub-blocks; when the division depth of the current coding block is less than the first preset depth, the current coding block may not be divided into sub-blocks.

[0130] Here, the first preset depth can be set according to actual needs, for example, it can be 2, that is, when the division depth cu_depth of the current coding block is <2, it is determined to divide the current coding block into sub-blocks; when the division depth cu_depth of the current coding block is ≥2, the current coding block may not be divided into sub-blocks.

[0131] In addition, in the above example, the preset energy range can be set according to actual needs. Here, according to the embodiment of the present disclosure, considering the coding blocks under different prediction modes, the prediction mode of the current coding block can be distinguished to set the preset energy range, thereby further improving the encoding quality.

[0132] As an example, the current coding block may be an intra-coding block or an inter-coding block, and the preset energy range may include a first block energy range for the intra-coding block and a second block energy range for the inter-coding block, wherein the second block energy range is included in the first block energy range.

[0133] Specifically, the intra prediction (intra) mode can easily cause smearing and artifacts in texture areas. Therefore, for coding blocks that select the intra mode, dividing them into sub-blocks will effectively improve the texture blurring problem. For coding blocks that select the inter prediction (inter) mode, the intra mode may be selected after division, which will affect the subjective image quality. Therefore, the energy range of the inter coding block can be smaller than that of the intra coding block. In addition, for scenes with particularly complex textures, sub-block RDO may not be forced after the inter coding block is selected.

[0134] For example, the first block energy range may be 2.7<E<3.6, and the second block energy range may be 2.7<E<3.2, where E represents the block energy of the coding block. The first block energy range and the second block energy range given here can better measure whether the coding block needs to be divided into sub-blocks, thereby making a more reasonable decision on sub-block division.

[0135] In step S630, the current coding block may be encoded based on the determination result.

[0136] In this step, the current coding block may be encoded based on the determination result indicating whether to divide the current coding block into sub-blocks, for example, using an existing coding method.

[0137] The above describes an example process of determining whether to divide the current coding block into sub-blocks based on the block energy of the current coding block. In addition, in an embodiment of the present disclosure, when it is determined to divide the current coding block into sub-blocks, the coding cost of the intra-frame prediction mode of the sub-blocks obtained by dividing the current coding block can also be improved based on the block energy or block information of the current coding block.

[0138] As an example, when the current coding block is an inter-frame coding block that selects the inter-frame prediction mode, the coding cost of the intra-frame prediction mode of the divided sub-blocks can be increased. In this way, blurring of the picture texture caused by all sub-blocks selecting the intra-frame prediction mode can be avoided.

[0139] Specifically, in response to determining to divide the current coding block into sub-blocks, the video encoding method may also include: in response to satisfying at least one of the following conditions, increasing the encoding cost of the intra-frame prediction mode of the sub-blocks obtained by dividing the current coding block: the division depth of the current coding block is greater than the second preset depth; the prediction mode of the parent block of the current coding block is the inter-frame prediction mode; the block energy of the current coding block is within the preset block energy range.

[0140] Here, the second preset depth can be set according to actual needs, and can be smaller than the first preset depth, for example, 1. In addition, the preset block energy range can be, for example, but not limited to, the second block energy range.

[0141] In this example, the coding cost of the inter-frame prediction mode of the divided sub-block can be multiplied by the lifting coefficient a, where a>1, and a can be configured with different coefficient values ​​according to different scenarios. For example, a can be but is not limited to 1.05.

[0142] The above describes a video encoding method according to an exemplary embodiment of the present disclosure. The video encoding method can make decisions on the division of coding blocks based on block energy, and can avoid skipping the sub-block division of larger coding blocks due to the block division size being affected by other fast algorithms in the target scene, thereby improving the subjective picture quality experience.

[0143] The video encoding method according to the embodiment of the present disclosure was verified on the K265 mobile encoder, and the obtained BD-rate PSNR was about -0.205%, the BD-rate SSIM was about -0.240%, and the encoding speed was reduced by 0.66%, but the subjective picture quality was significantly improved. Figure 8 A comparison of mode selection differences before and after block partition optimization in an application example of a video encoding method according to an exemplary embodiment of the present disclosure is shown.

[0144] from Figure 8 It can be seen that after optimization, the number of CUs in the intra mode of larger coding blocks is reduced, and the image blur problem in the corresponding texture area is repaired to a certain extent.

[0145] It can be seen that according to the video encoding method of the embodiment of the present disclosure, by introducing energy information as the basis for judging whether to perform sub-block division, it is possible to identify areas with more complex texture information through energy. For the CUs in these areas, the subsequent sub-block division RDO process is not skipped, and a more appropriate division mode is selected, thereby improving the subjective quality.

[0146] Fig. 9 is a block diagram of a video encoding device according to an exemplary embodiment. Fig. 9 , the video encoding device 900 includes a first determination unit 910, a second determination unit 920 and an encoding unit 930.

[0147] The first determining unit 910 is configured to determine a block energy of a current coding block, wherein the block energy represents an amount of information contained in the coding block.

[0148] The second determining unit 920 is configured to determine whether to perform sub-block division on the current coding block according to the block energy of the current coding block.

[0149] The encoding unit 930 is configured to encode the current coding block based on the determination result.

[0150] As an example, the second determination unit 920 is further configured to: in response to the block energy of the current coding block being within a preset energy range, determine whether to perform sub-block division on the current coding block according to the block information of the current coding block, wherein the block information includes: the frame type of the coding frame in which the current coding block is located and / or the division depth of the current coding block; in response to the block energy of the current coding block not being within the preset energy range, determine not to perform sub-block division on the current coding block.

[0151] As an example, the second determination unit 920 is also configured to: in response to the block information of the current coding block satisfying a preset condition, determine to divide the current coding block into sub-blocks; in response to the block information of the current coding block not satisfying the preset condition, determine not to divide the current coding block into sub-blocks, and the preset condition includes at least one of the following items: the frame type of the coding frame in which the current coding block is located is not a bidirectional prediction frame; the division depth of the current coding block is less than the first preset depth.

[0152] As an example, the current coding block is an intra-frame coding block or an inter-frame coding block, and the preset energy range includes a first block energy range for the intra-frame coding block and a second block energy range for the inter-frame coding block, wherein the second block energy range is included in the first block energy range.

[0153] As an example, the first block energy range is 2.7<E<3.6, and the second block energy range is 2.7<E<3.2, where E represents the block energy of the coding block.

[0154] As an example, the first determination unit 910 is further configured to determine the block energy of the current coding block in the following manner: performing statistics on the pixel value of each pixel in the current coding block to obtain a statistical result; and determining the block energy of the current coding block based on the statistical result.

[0155] As an example, the statistical result includes the square sum of the pixel values ​​of each pixel in the current coding block and the sum of the pixel values ​​of each pixel in the current coding block.

[0156] As an example, the first determination unit 910 is further configured to: determine a ratio between the square of the sum value and the number of pixels in the current coding block; and determine a difference between the square sum and the ratio as the block energy of the current coding block.

[0157] As an example, the video encoding method also includes an adaptive quantization process, wherein the first determination unit 910 is further configured to determine the block energy of the current coding block in the following manner: obtaining a quantization result of the adaptive quantization process; determining the block energy of the target coding block based on the quantization result, wherein the target coding block is a coding block corresponding to the position of the current coding block and having a size less than or equal to the size of the current coding block; determining the block energy of the current coding block based on the relationship between the size of the current coding block and the size of the target coding block and the block energy of the target coding block.

[0158] As an example, the video encoding device also includes an improving unit 940. In response to determining that the current coding block is divided into sub-blocks, the improving unit 940 is configured to: in response to satisfying at least one of the following conditions, improve the encoding cost of the intra-frame prediction mode of the sub-block obtained by dividing the current coding block: the division depth of the current coding block is greater than the second preset depth; the prediction mode of the parent block of the current coding block is the inter-frame prediction mode; the block energy of the current coding block is within the preset block energy range.

[0159] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0160] Fig.10 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Fig.10As shown, the electronic device 1000 includes a processor 1010 and a memory 1020 for storing processor executable instructions. Here, when the processor executable instructions are executed by the processor, the processor executes the video encoding method as described in the above exemplary embodiments.

[0161] As an example, the electronic device 1000 may be a single device, or may be any device or circuit that can execute the above instructions (or instruction sets) individually or jointly. The processor 1010 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. The processor 1010 may run instructions or codes stored in the memory 1020. Instructions and data may also be sent and received over a network via a network interface device, which may use any known transmission protocol.

[0162] The memory 1020 may be integrated with the processor 1010. The memory 1020 and the processor 1010 may be operatively coupled or may communicate with each other, for example, via an I / O port, a network connection, etc. In addition, the electronic device 1000 may further include a video display and a user interaction interface.

[0163] In an exemplary embodiment, a computer-readable storage medium may also be provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can perform the video encoding method as described in the above exemplary embodiment. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: a read-only memory (ROM), a random access memory (RAM), a random access programmable read-only memory (PROM), an electrically erasable programmable read-only memory (EEPROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a non-volatile memory, a CD-ROM, a CD-R, a CD+R, a CD-RW, a CD+RW, a DVD-ROM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, a DVD-RAM, a BD-ROM, a BD-R, a BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, MultiMediaCard, Secure Digital (SD) card or Extreme Digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device configured to store the computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program.

[0164] In an exemplary embodiment, a computer program product may also be provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the video encoding method as described in the exemplary embodiment above is implemented.

[0165] In an exemplary embodiment, a method for generating a bitstream may be provided. The method includes: generating a bitstream according to the video encoding method as described in the exemplary embodiment above.

[0166] In an exemplary embodiment, a computer program product may also be provided, having instructions for storing a bit stream, the bit stream comprising: video data generated according to the video encoding method as described in the above exemplary embodiments.

[0167] Fig.11 A computing environment 1110 is shown coupled to a user interface 1150. The computing environment 1110 may be part of a data processing server. The computing environment 1110 includes a processor 1120, a memory 1130, and an input / output (I / O) interface 1140.

[0168] The processor 1120 generally controls the overall operation of the computing environment 1110, such as operations associated with display, data acquisition, data communication, and image processing. The processor 1120 may include one or more processors for executing instructions to perform all or some of the steps in the above method. In addition, the processor 1120 may include one or more modules that facilitate the interaction between the processor 1120 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0169] The memory 1130 is configured to store various types of data to support the operation of the computing environment 1110. The memory 1130 may include predetermined software 1132. Examples of such data include instructions for any application or method operating on the computing environment 1110, video data sets, image data, etc. The memory 1130 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0170] The I / O interface 1140 provides an interface between the processor 1120 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 1140 may be coupled to an encoder and a decoder.

[0171] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 1130 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 1120 in a computing environment 1110 to perform the above method. In one example, the plurality of programs can be executed by a processor 1120 in a computing environment 1110 to (for example, from Figure 2 The video encoder 20 in the computing environment 1110 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1120 in the computing environment 1110 to perform the above-mentioned decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 1120 in the computing environment 1110 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 1120 in the computing environment 1110 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0172] In an embodiment, a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is provided. In an embodiment, a bit stream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0173] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 1120); and a non-transitory computer-readable storage medium or memory 1130 in which multiple programs that can be executed by one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0174] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, wherein the bitstream includes the encoded video information generated by the above encoding method or the encoded video information to be decoded by the above decoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 1130 is also provided, wherein the plurality of programs can be executed by a processor 1120 in a computing environment 1110 for performing the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0175] In an embodiment, the computing environment 1110 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0176] In an embodiment, a method for storing a bit stream is also provided, comprising: storing the bit stream on a digital storage medium, wherein the bit stream comprises encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method.

[0177] In an embodiment, a method for transmitting a bit stream generated by the above encoder is also provided. In an embodiment, a method for receiving a bit stream to be decoded by the above decoder is also provided.

[0178] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0179] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0180] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

[0181] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0182] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that: The video encoding method comprises: Determining a block energy of a current coding block, wherein the block energy represents an amount of information contained in the coding block; Determining whether to divide the current coding block into sub-blocks according to the block energy of the current coding block; Based on the determination result, the current coding block is encoded.

2. The video encoding method according to claim 1, characterized in that The determining, according to the block energy of the current coding block, whether to divide the current coding block into sub-blocks comprises: In response to the block energy of the current coding block being within a preset energy range, determining whether to perform sub-block division on the current coding block according to block information of the current coding block, wherein the block information includes: a frame type of a coding frame in which the current coding block is located and / or a division depth of the current coding block; In response to the block energy of the current coding block not being within the preset energy range, it is determined not to perform sub-block division on the current coding block.

3. The video encoding method according to claim 2, characterized in that: The determining, according to the block information of the current coding block, whether to divide the current coding block into sub-blocks includes: In response to the block information of the current coding block satisfying a preset condition, determining to divide the current coding block into sub-blocks; In response to the block information of the current coding block not satisfying the preset condition, determining not to divide the current coding block into sub-blocks, The preset condition includes at least one of the following items: The frame type of the coding frame in which the current coding block is located is not a bidirectional prediction frame; The division depth of the current coding block is less than a first preset depth.

4. The video encoding method according to claim 2 or 3, characterized in that: The current coding block is an intra-frame coding block or an inter-frame coding block, and the preset energy range includes a first block energy range for the intra-frame coding block and a second block energy range for the inter-frame coding block, wherein the second block energy range is included in the first block energy range.

5. The video encoding method according to claim 4, characterized in that: The first block energy range is 2.7<E<3.6, and the second block energy range is 2.7<E<3.2, where E represents the block energy of the coding block.

6. The video encoding method according to claim 1, characterized in that: The block energy of the current coding block is determined by: Performing statistics on the pixel value of each pixel in the current coding block to obtain a statistical result; Based on the statistical result, a block energy of the current coding block is determined.

7. The video encoding method according to claim 6, characterized in that: The statistical result includes a square sum of pixel values ​​of each pixel in the current coding block and a sum of pixel values ​​of each pixel in the current coding block.

8. The video encoding method according to claim 7, characterized in that: The determining, based on the statistical result, the block energy of the current coding block comprises: determining a ratio between the square of the sum and the number of pixels in the current coding block; A difference between the square sum and the ratio is determined as the block energy of the current coding block.

9. The video encoding method according to claim 1, characterized in that: The video encoding method further comprises an adaptive quantization process, wherein the block energy of the current encoding block is determined by: Obtaining a quantization result of the adaptive quantization process; Determine a block energy of a target coding block based on the quantization result, wherein the target coding block is a coding block corresponding to a position of the current coding block and having a size smaller than or equal to a size of the current coding block; Based on the relationship between the size of the current coding block and the size of the target coding block and the block energy of the target coding block, the block energy of the current coding block is determined.

10. The video encoding method according to claim 4, characterized in that: In response to determining to divide the current coding block into sub-blocks, the video coding method further includes: in response to satisfying at least one of the following conditions, increasing the coding cost of the intra-frame prediction mode of the sub-blocks obtained by dividing the current coding block: The division depth of the current coding block is greater than the second preset depth; The prediction mode of the parent block of the current coding block is the inter-frame prediction mode; The block energy of the current coding block is within a preset block energy range.

11. A video encoding device, characterized in that: The video encoding device comprises: A first determining unit is configured to determine a block energy of a current coding block, wherein the block energy represents an amount of information contained in the coding block; A second determining unit is configured to determine whether to perform sub-block division on the current coding block according to the block energy of the current coding block; The encoding unit is configured to encode the current encoding block based on the determination result.

12. An electronic device, characterized in that: The electronic device comprises: Processor; and a memory for storing processor executable instructions, When the processor executable instructions are executed by the processor, the processor executes the video encoding method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the video encoding method according to any one of claims 1 to 10.

14. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the video encoding method according to any one of claims 1 to 10 is implemented.

15. A method for generating a bit stream, characterized in that: include: A bit stream is generated according to the video encoding method according to any one of claims 1 to 10.

Images