Video coding method and device, electronic equipment and storage medium

Abstract

The invention provides a video coding method and device, electronic equipment and a storage medium. The video coding method comprises: constructing a mode pool for a current block, the mode pool comprising a plurality of current block coding modes or the plurality of current block coding modes and a sub-block coding mode; determining a mode queue comprising at least one mode for the current block based on the mode pool; executing the at least one mode according to a mode sequence in the mode queue, and determining a coding mode for the current block or a plurality of sub-blocks divided from the current block; and coding the current block or the plurality of sub-blocks based on the determined coding mode, the plurality of current block coding modes being related to coding of the current block, and the sub-block coding modes being related to coding of the plurality of sub-blocks.

Description

Technical Field

[0001] The present application relates to video coding and compression. More specifically, the present application relates to a video coding method, a bit stream generation method, a device, an electronic device, a storage medium and a program product. Background Art

[0002] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices send and receive or otherwise transmit digital video data through a communication network, and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited storage resources of the storage device, before the video data is transmitted or stored, the video data can be compressed using a video codec according to one or more video codec standards. For example, video codec standards include general video codec (VVC), joint exploration test model (JEM), high efficiency video codec (HEVC / H.265), advanced video codec (AVC / H.264), moving picture expert group (MPEG) codec, etc. Video codecs generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize the inherent redundancy in video data. Video codecs are intended to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0003] The H.265 / HEVC standard was formally proposed in 2013. Its compression performance has been greatly improved compared with the H.264 standard. The widespread application of HEVC technology can save storage and bandwidth costs, and has broad application prospects. The H.265 / HEVC standard adds new codec tools based on the original codec structure and codec tools of the H.264 standard, which greatly improves the compression performance while bringing a huge increase in complexity. The increase in the codec complexity of H.265 / HEVC is largely due to the selection of coding modes during the encoding process. However, there are many problems in the selection process of related coding modes (for example, the coding division order of video blocks is fixed, the mode traversal order of video blocks is fixed, and the mode execution process of video blocks is fixed, etc.), which makes the selection process of coding modes less flexible, the types or number of algorithms that can be added in the mode selection process are limited, and the coding and decoding complexity is large. Summary of the invention

[0004] An exemplary embodiment of the present disclosure is to provide a video encoding method, a bitstream generation method, an apparatus, an electronic device, a storage medium and a program product, which can enhance the flexibility of selecting encoding and decoding modes, increase the upper limit of the algorithm, and be applicable to various types of hybrid coding framework encoders, etc.

[0005] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, comprising: constructing a mode pool for a current block, wherein the mode pool includes multiple current block encoding modes or the multiple current block encoding modes and sub-block encoding modes; based on the mode pool, determining a mode queue including at least one mode for the current block; executing the at least one mode in the mode sequence in the mode queue to determine the encoding mode for the current block or multiple sub-blocks divided from the current block; encoding the current block or the multiple sub-blocks based on the determined encoding mode, wherein the multiple current block encoding modes are related to the encoding of the current block, and the sub-block encoding mode is related to the encoding of the multiple sub-blocks.

[0006] Optionally, the step of executing the sub-block encoding mode includes: obtaining the multiple sub-blocks by dividing the current block; for each of the multiple sub-blocks, determining the encoding mode of the corresponding sub-block by constructing a mode pool and determining a mode queue, wherein the mode pools and / or mode queues of different sub-blocks are the same or different.

[0007] Optionally, when the mode pool includes only the multiple current block coding modes, the mode queue includes the sub-block coding mode according to the coding division order, wherein the position of the sub-block coding mode in the mode queue is related to the coding division order.

[0008] Optionally, the coding division order includes a small block-large block order, a large block-small block order and / or a large block-small block-large block order, wherein, when the coding division order is a small block-large block order, the sub-block coding mode is located at the first position in the mode queue; when the coding division order is a large block-small block order, the sub-block coding mode is located at the last position in the mode queue; and / or when the coding division order is a large block-small block-large block order, the sub-block coding mode is located in the middle position in the mode queue.

[0009] Optionally, the step of determining a mode queue including at least one mode for the current block based on the mode pool includes: selecting the at least one mode from the mode pool and determining the order of the at least one mode in the mode queue by referring to the encoding information of the neighboring blocks of the current block, the encoding information of the parent block or the child block of the current block and / or the pre-analyzed encoding information of the video frame where the current block is located.

[0010] Optionally, the multiple current block encoding modes include multiple encoding modes for performing encoding processing on the current block or multiple first encoding sub-modes and multiple second encoding sub-modes respectively split from the multiple encoding modes, wherein the first encoding sub-mode is used to perform prediction processing in the encoding processing of the corresponding encoding mode, and the second encoding sub-mode is used to perform transformation processing, quantization processing and entropy coding processing in the encoding processing of the corresponding encoding mode.

[0011] Optionally, the step of executing at least one mode according to the mode order in the mode queue to determine the encoding mode for the current block or multiple sub-blocks divided from the current block includes: skipping the execution of at least one subsequent mode in the mode queue based on the execution result of the previous mode in the mode queue.

[0012] Optionally, the step of skipping the execution of at least one subsequent mode in the mode queue based on the execution result of the previous mode in the mode queue includes: when the previous mode is a coding mode, skipping the execution of at least one subsequent mode in the mode queue based on the rate-distortion optimization result and / or distortion result obtained by performing coding processing using the coding mode; when the previous mode is a first coding sub-mode, skipping the execution of the corresponding second coding sub-mode in the mode queue based on the distortion result obtained by performing prediction processing using the first coding sub-mode; and / or when the previous mode is a sub-block coding mode, skipping the execution of at least one subsequent mode in the mode queue based on the sum of the rate-distortion optimization results and / or the sum of the distortion results of the multiple sub-blocks obtained by performing coding processing using the sub-block coding mode.

[0013] According to a second aspect of an embodiment of the present disclosure, a video encoding device is provided, comprising: a mode pool construction unit, configured to construct a mode pool for a current block, wherein the mode pool includes multiple current block encoding modes or the multiple current block encoding modes and sub-block encoding modes; a queue determination unit, configured to determine a mode queue including at least one mode for the current block based on the mode pool; a mode determination unit, configured to execute the at least one mode in the mode queue according to the mode order, and determine the encoding mode for the current block or the multiple sub-blocks divided from the current block; an encoding unit, configured to encode the current block or the multiple sub-blocks based on the determined encoding mode, wherein the multiple current block encoding modes are related to the encoding of the current block, and the sub-block encoding mode is related to the encoding of the multiple sub-blocks.

[0014] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video encoding method as described above.

[0015] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the video encoding method as described above.

[0016] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the video encoding method as described above when executed by at least one processor.

[0017] According to a sixth aspect of an embodiment of the present disclosure, a method for generating a bitstream is provided, comprising: generating a bitstream according to the video encoding method as described above.

[0018] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, having instructions for storing a bit stream, wherein the bit stream includes: video information generated according to the video encoding method as described above.

[0019] According to the video encoding method, bitstream generation method, device, electronic device, storage medium and program product of the exemplary embodiments of the present invention, a mode pool is constructed for the current block and a mode queue is determined. The encoding mode is determined according to the execution mode of the mode queue and the determined encoding mode is used for encoding. This can ensure the encoding quality while improving the flexibility of the encoding architecture, increasing the diversity of algorithms that can be added, and improving the encoding efficiency, thereby improving the overall encoding and decoding performance.

[0020] It is to be understood that both the foregoing general description and the following detailed description are examples only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate examples according to the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0022] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0023] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0024] Figure 3is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0025] Figure 4 is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0026] Figure 5 and Figure 6 is an exemplary diagram illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0027] Figure 7 is an exemplary diagram illustrating a coding mode and a coding sub-mode according to an exemplary embodiment of the present disclosure.

[0028] Figure 8 and Fig. 9 is an exemplary diagram illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0029] Fig.10 A structural block diagram of a video encoding apparatus according to an exemplary embodiment of the present disclosure is shown.

[0030] Fig.11 is a diagram illustrating a computing environment coupled with a user interface according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0031] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the drawings are used to distinguish objects, but not to describe any specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0033] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0034] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0035] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0036] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0037] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0038] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0039] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0040] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0041] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0042] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0043] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0044] like Figure 2 As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments (hereinafter, also referred to as "partition") the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks) or other larger coding units (CUs) according to a predefined splitting structure associated with the video data (e.g., a quadtree (QT) structure). It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU) or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB) or a transform block (TB)) and / or a sub-block.

[0045] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0046] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0047] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0048] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0049] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0050] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0051] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0052] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0053] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0054] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0055] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0056] In some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0057] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0058] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0059] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0060] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0061] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0062] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0063] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0064] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0065] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0066] As described above, on the encoder side, in order to select a suitable prediction coding mode (hereinafter, also referred to as a "coding mode") for a current video block (hereinafter, also referred to as a "current block"), a video encoder (hereinafter, also referred to as an "encoder") may perform multiple encoding passes to select a suitable coding mode for each block of video data. For example, the encoder uses various prediction coding modes to encode the current block and test their performance during a separate encoding pass, and selects a suitable optimal coding mode to use among the various tested prediction coding modes.

[0067] In the related art, when determining the optimal coding mode for the current block, the encoder may use each coding mode of the current block to perform coding one by one (i.e., traverse each coding mode) to obtain multiple coding results of each coding mode of the current block. In addition to the coding of each coding mode for the current block, the encoder may also divide (or split) the current block to obtain sub-blocks, and use each coding mode of the sub-block to perform coding one by one according to the execution mode of each coding mode of the current block to obtain multiple coding results of each coding mode of the sub-block. Finally, the encoder determines the optimal coding mode applicable to the current block and / or sub-block based on the multiple coding results of the current block and / or sub-block. In the following, the relevant process or architecture may be understood as a "coding mode selection process" or a "coding architecture" regarding the coding mode selection process, respectively.

[0068] Typically, in the encoding process of related encoders (e.g., HM encoder, x265 encoder, etc.), the encoding mode selection process for determining the optimal encoding mode for the current block has problems such as a fixed encoding division order of video blocks, a fixed mode selection order of video blocks, a fixed mode selection process of video blocks, a limited number of algorithms that can be added in the encoding mode selection process, and excessive encoding and decoding complexity.

[0069] Specifically, when selecting the optimal coding mode for the current block, the relevant encoder often adopts a fixed coding mode selection strategy. For example, a fixed coding partitioning order (i.e., a fixed traversal order of blocks at different layers) is used for the current block, such as a large block-small block order of a top-down coding architecture, a small block-large block order of a bottom-up coding architecture, or a large block-small block-large block order of a parent block-child block-parent block coding architecture. Such a traversal order of blocks of different sizes according to a fixed architecture is very inflexible, resulting in the encoder being unable to quickly decide on the optimal block size.

[0070] In addition, the relevant encoders often execute various coding modes in a fixed mode selection order (i.e., the traversal order of each mode), such as executing the skip mode first, executing the merge mode, executing the inter mode, and finally executing the intra mode for all video blocks and their sub-blocks in a fixed order to determine the optimal coding mode without considering the coding and decoding scene and / or sequence characteristics. Such a mode traversal order is also not flexible, resulting in the encoder being unable to quickly execute the optimal coding mode. For example, even if the skip mode executed first is not the optimal coding mode for the current block, the previous skip mode must be executed before executing other more optimal modes. This fixed mode selection order does not consider the possibility of giving priority to the optimal coding mode.

[0071] In addition to the fixed coding division order and mode traversal order, the relevant encoder must also follow a fixed mode execution process when selecting a coding mode. For example, when the encoder executes a coding mode during the mode traversal process, the entire coding process of the coding mode (for example, prediction, transformation, quantization and entropy coding) must be fully executed, and only partial processing cannot be executed. Therefore, the coding process of the next coding mode can only be executed when the complete coding process of the current coding mode is completed. In this case, even if the intermediate result of the current coding mode does not perform well, the execution process of the current coding mode cannot be terminated in advance. Such a fixed mode execution process is also not flexible, which reduces the coding and decoding efficiency.

[0072] In addition, the inflexibility of the above-mentioned fixed mode selection process also makes the types and number of algorithms that can be added in the process very limited, which is not conducive to the adaptive change of the coding architecture for different coding and decoding scenarios.

[0073] That is to say, the mode selection process of the related technology requires that all video sequences go through the same fixed mode selection process to determine the optimal encoding mode. Each encoding mode cannot be terminated early, and the algorithms that can be added under the corresponding encoding architecture are limited, which makes it impossible for the mode selection process of the related technology to adapt to the encoding and decoding scenario.

[0074] At least in view of the above-mentioned defects in the related art, the exemplary embodiments of the present disclosure provide a video coding technology with an adaptive mode selection process that has high flexibility, a high upper limit of algorithms that can be added, and high universality. In the exemplary embodiments of the present disclosure, the coding mode selection process (or coding architecture) for selecting the optimal coding mode is reconstructed, and the coding division order, mode traversal order and / or mode execution process of blocks of different sizes in the reconstructed mode selection process can be adaptively changed according to the coding and decoding scenario, thereby greatly reducing the coding and decoding complexity, improving the flexibility of the coding architecture, and greatly improving the upper limit of the algorithms that can be added, while ensuring the accuracy of the mode selection, thereby improving the overall coding and decoding performance. This will be described in detail with reference to the accompanying drawings.

[0075] In the present disclosure, the video encoding method according to the exemplary embodiment of the present disclosure includes various steps performed on the current block to be encoded. Therefore, it should be understood that after the video encoding method according to the exemplary embodiment of the present disclosure is performed on the current block, the video encoding method according to the exemplary embodiment of the present disclosure can continue to be performed on the next video block to be encoded (i.e., the next current block), or the video encoding method according to the exemplary embodiment of the present disclosure can be not performed on the next current block, but the encoding mode determined by referring to the video encoding method according to the exemplary embodiment of the present disclosure that has been performed.

[0076] As described above, the mode traversal order of the relevant encoder for the video block (such as the CU block corresponding to the CU) is fixed. However, in the actual coding and decoding scenario, for example, assuming that the optimal coding mode of a current block is the intra-frame mode for the current block and the fixed mode traversal order is skip mode-merge mode-inter-frame mode-intra-frame mode, then for the current block, the previous other modes except the intra-frame mode must be executed before the intra-frame mode is executed. In this case, the previous encoding process of executing other modes except the intra-frame mode has a waste of coding and decoding complexity, and this inflexible mode traversal order also leads to an increase in decision complexity, making it difficult to quickly decide on the optimal coding mode. In this regard, the present disclosure proposes to construct a more flexible coding mode selection process by using a mode pool to determine a mode queue.

[0077] Figure 4 is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure. Figure 5 and Figure 6 is an exemplary diagram illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0078] Reference Figure 4 , in step S410, a pattern pool for the current block is constructed.

[0079] According to the exemplary embodiments of the present disclosure, the mode pool can be understood as a multi-unit mode pool including multiple units (or modes), each mode in the mode pool coexists, and can be associated with each other according to the use of different algorithms, or can be independent of each other through decoupling. That is, a related algorithm can be used to associate at least one mode in the mode pool. However, the present disclosure does not impose any restrictions on the algorithms used in the exemplary embodiments, and any algorithm in the relevant field may be applicable to the present disclosure. In the present disclosure, modes are "independent" of each other, which may mean that the previously executed mode has no effect on the subsequently executed mode, and modes are "associated" with each other, which may mean that the execution result of the previously executed mode may have an impact on the subsequently executed mode.

[0080] As an exemplary embodiment, the mode pool for the current block may include multiple current block coding modes, wherein each of the multiple current block coding modes is related to the coding of the current block. As an exemplary embodiment, in addition to including multiple current block coding modes, the mode pool for the current block may also include a sub-block coding mode, wherein the sub-block coding mode is related to the coding of each of the multiple sub-blocks divided from the current block. That is, the mode pool may include only multiple current block coding modes, or may also include multiple current block coding modes and sub-block coding modes. The modes in the mode pool are described in detail below.

[0081] As an exemplary embodiment, the current block encoding mode may include an encoding mode directly used to perform encoding processing on the current block, an encoding sub-mode split from the encoding mode used to perform encoding processing on the current block, and / or any other mode or processing related to the encoding of the current block, but the present disclosure is not limited thereto.

[0082] As an example, the encoding mode directly used to perform encoding processing on the current block may be a feasible prediction encoding mode as described above, for example, an intra-frame prediction encoding mode (in the present disclosure, may also be referred to as "intra-frame mode" or "intra-frame prediction mode"), an inter-frame prediction encoding mode (in the present disclosure, may also be referred to as "inter-frame mode" or "inter-frame prediction mode"), a merge mode and / or a skip mode, etc.

[0083] It should be understood that the feasible prediction coding modes may include but are not limited to the above, for example, may also include any coding modes such as Advanced Motion Vector Prediction (AMVP) mode, Inter-Intra combination prediction mode, Affine Inter mode, Current Picture Reference mode, etc. In the following, for the convenience of description, only the feasible prediction coding modes including Intra mode, Inter mode, Merge mode and Skip mode are described as examples, and it should be understood that the present disclosure is not limited thereto.

[0084] As an example, the coding sub-modes split from the coding mode may include a plurality of coding sub-modes split from each coding mode in the feasible prediction coding modes, which will be described in detail later below.

[0085] In an exemplary embodiment of the present disclosure, in order to obtain the optimal coding mode that is most suitable for the area corresponding to the current block in the video frame, it is possible to consider traversing the coding modes of the sub-blocks recursively divided from the current block. Accordingly, when determining the optimal coding mode for the current block, the determined result can be the optimal coding mode for the current block or the optimal coding mode for the multiple sub-blocks recursively divided from the current block.

[0086] Considering this possible determination result, the present disclosure proposes to regard the process of recursively dividing the current block and determining the coding mode of the recursively divided sub-blocks as a selectable mode, and put the selectable mode into the mode pool for mode selection. That is, in the present disclosure, the sub-block coding mode may represent a mode of dividing the current block to obtain sub-blocks and determining the coding mode of the sub-blocks, which will be described in detail later.

[0087] As an example of a pattern pool, see Figure 5 , for the current block, a 5-unit mode pool including 4 current block coding modes (for example, 4 coding modes, such as intra mode, inter mode, merge mode and skip mode) and 1 sub-block coding mode may be constructed. For another example, referring to Figure 6, for the current block, a 4-unit mode pool including only 4 current block coding modes (eg, 4 coding modes, such as intra mode, inter mode, merge mode, and skip mode) may be constructed. For another example, referring to Figure 8 For the current block, a 9-unit mode pool including 8 current block coding modes (eg, a plurality of coding sub-modes to be described later) and 1 sub-block coding mode may be constructed.

[0088] Like this, the exemplary embodiments of the present disclosure can treat multiple current block coding modes and / or sub-block coding processes equally, and unitize (or modularize) them into modes in a mode pool, thereby constructing a mode pool for the current block. Through the construction of the mode pool, the execution of each coding mode and the coding mode of the sub-block will not have a fixed traversal (or selection) order, which can improve the flexibility of the mode selection process of the coding process, and can be combined with the algorithm to provide the advantage of greatly reducing the complexity of the decision on the optimal mode.

[0089] In step S420 , based on the mode pool, a mode queue including at least one mode for the current block is determined.

[0090] As an exemplary embodiment, step S420 may include: selecting at least one mode from the mode pool and determining the order of at least one mode in the mode queue by referring to the encoding information of the neighboring blocks of the current block, the encoding information of the parent block or the child block of the current block, and / or the pre-analyzed encoding information of the video frame in which the current block is located. That is, selecting the mode based on the encoding reference information that can be used by the current block, and determining the priority of executing these selected modes.

[0091] As an example, at least one mode and its order can be determined by using an algorithm to refer to the coding reference information associated with the current block, thereby constructing a mode queue. Here, the present disclosure does not limit the algorithm or method for constructing the mode queue, as long as the algorithm or method used can achieve the corresponding function.

[0092] As an example, at least one mode in the mode queue and its order (or priority) can be determined by using an algorithm to refer to the encoding information (such as encoding modes, etc.) of one or more neighboring blocks (e.g., already encoded) within the video frame in which the current block to be encoded is located.

[0093] As an example, at least one mode in the mode queue and its order (or priority) may be determined by using an algorithm with reference to encoding information (such as encoding mode, etc.) of one or more parent blocks or child blocks of the current block to be encoded (e.g., already encoded). Here, the parent block of the current block may be understood as an upper block or upper block of the current block, that is, a video block from which the current block can be divided. In addition, the child block of the current block may be understood as a lower block or lower block of the current block, that is, a video block that can be divided from the current block.

[0094] As an example, at least one mode in the mode queue and its order (or priority) may be determined by using a preprocessing result (i.e., pre-analyzed encoding information, etc.) for the current video frame or picture in which the current block to be encoded is located. For example, a pre-analysis process may be performed using a look-ahead process to obtain the preprocessing result. Here, the present disclosure does not limit the manner in which the preprocessing is performed.

[0095] That is, by using an algorithm to refer to the coding information (such as the coding mode, etc.) of the block associated with the current block (for example, already encoded) and / or the pre-analyzed coding information, at least one mode that is a candidate for the optimal coding mode can be adaptively adjusted according to the coding and decoding scenario (for example, sequence characteristics (such as sequence features, sequence complexity, the static state or motion state of the video frame and the amplitude of its motion, etc.)) to construct a mode queue suitable for selecting the optimal coding mode. In this way, by adaptively adjusting the mode queue, the encoder can execute in advance a mode that has a higher probability of being selected as the optimal mode relative to other modes, thereby accelerating the decision of the optimal coding mode.

[0096] As an exemplary embodiment, when a mode pool includes multiple current block coding modes and sub-block coding modes, correspondingly, the selected at least one mode may include at least one current block coding mode and / or sub-block coding mode. That is, the mode queue in this case may only include at least one current block coding mode or sub-block coding mode, or may include at least one current block coding mode and sub-block coding mode. This embodiment is described in detail below.

[0097] Reference Figure 5 , when the mode pool for the current block includes 4 coding modes (such as intra mode, inter mode, merge mode and skip mode) and 1 sub-block coding mode, a mode queue including 5 modes can be determined from the 5-unit mode pool by using an algorithm based on the above coding reference information: intra mode - sub-block coding mode - skip mode - merge mode - inter mode. Here, although Figure 5The mode queue in is shown to include 5 modes in the above order, but the present disclosure does not limit the number, category, order (or priority) of the modes in the queue, and the mode queue determined using the relevant algorithm may include any number of modes, any mode category, and any mode order. For example, the mode queue determined from the mode pool can also be: sub-block coding mode - skip mode - merge mode - intra-frame mode.

[0098] By constructing a mode pool including the coding mode of the current block and the sub-block coding mode, the flexibility of the coding architecture can be greatly improved. For example, for a 5-unit mode pool, up to 325 mode queues can be determined. Compared with a fixed mode selection order in the related technology, the types of mode queues are significantly increased, and the applicable mode queue can be arbitrarily determined by the relevant algorithm.

[0099] In addition, during the video encoding and decoding process, different encoding and decoding scenarios may require different encoding division orders for traversing blocks of each layer. For example, complex encoding and decoding scenarios (such as encoding and decoding scenarios with complex motion or more textures) may be suitable for executing the encoding mode in the encoding order of traversing small blocks-large blocks, while simple encoding and decoding scenarios (such as encoding and decoding scenarios with static or less textures) may be suitable for executing the encoding mode in the encoding order of traversing large blocks-small blocks. However, the single fixed encoding division order in the related art may cause the encoding and decoding efficiency to be reduced or the encoding and decoding calculation complexity to be wasted in different encoding and decoding scenarios. Therefore, the present disclosure proposes a method for constructing a mode selection process of a bidirectional encoding division order.

[0100] As an example, a block traversal direction (i.e., coding division order) suitable for the coding scene can be determined by using an algorithm (e.g., considering the above coding scene complexity), and the position of the sub-block coding mode can be set based on the determined block traversal direction. Here, the present disclosure does not specifically limit the method for determining the coding division order, and the coding division order suitable for the current coding scene can be determined according to any relevant technology or algorithm, for example, as described above, a suitable coding division order is determined according to the scene complexity.

[0101] In the present disclosure, in order to realize the setting of the coding division order that can be changed according to the coding and decoding scene, there is a sub-block coding mode related to block division in the mode queue to realize the division of blocks at different levels and the determination of coding modes. In this regard, based on the above-mentioned conception about the mode pool, the present disclosure also proposes the conception of selecting only the current block coding mode from the mode pool when the sub-block coding mode is fixedly used.

[0102] As an exemplary embodiment, when the mode pool for the current block includes only multiple current block coding modes, correspondingly, the selected at least one mode may include one or more current block coding modes but not include a sub-block coding mode. The mode queue may include sub-block coding modes to traverse the coding modes of blocks at different levels according to the coding division order, that is, different priorities of determining the coding mode of the current block and determining the coding mode of the sub-block may be considered, and the sub-block coding mode may be directly placed in the mode queue instead of selecting the sub-block coding mode from the mode pool. This is described in detail below with reference to the accompanying drawings.

[0103] Reference Figure 6 In the case of traversing the coding modes of blocks at different levels, it can be determined that a sub-block coding mode related to the division of the block exists fixedly in the mode queue, and one or more current block coding modes determined from a 4-unit mode pool (including intra mode, inter mode, merge mode, and skip mode) may exist before or after, or before and after, the fixedly existing sub-block coding mode.

[0104] As an exemplary embodiment, the position of the sub-block coding mode in the mode queue is related to the coding partition order. That is, the coding partition order is considered to determine the order between the sub-block coding mode and one or more current block coding modes determined from the mode pool.

[0105] As an exemplary embodiment, the coding partitioning order may include a small block-large block order of a bottom-up coding architecture, a large block-small block order of a top-down coding architecture, and / or a large block-small block-large block order of a parent block-child block-parent block coding architecture.

[0106] As an example, the small block-large block order may represent the order of traversal of such coding modes: the small block is first encoded by traversing the mode that can be used by the small block (or child block, lower layer block or lower-level block), and then the large block is encoded by traversing the mode that can be used by the large block (or parent block, upper layer block or upper-level block).

[0107] As an example, the large block-small block order may represent an order of traversal of coding modes in which the large block is first encoded by traversing modes usable by the large block, and then the small block is encoded by traversing modes usable by the small block.

[0108] As an example, the large block-small block-big block order may represent an order of traversal of such coding modes: the large block is first encoded by traversing a portion of the modes that can be used by the large block, and then the small block is encoded by traversing the modes that can be used by the small block, and then the large block is encoded by traversing another portion of the modes that can be used by the large block.

[0109] In the present disclosure, for a current block and a sub-block divided from the current block, the current block may be regarded as a large block and the sub-block may be regarded as a small block, and similarly, for a sub-block and a further recursively divided sub-block divided from the sub-block, the sub-block may be regarded as a large block and the further recursively divided sub-block may be regarded as a small block.

[0110] As an exemplary embodiment, when the coding division order is a small block-large block order, the sub-block coding mode is located at the first position in the mode queue, that is, referring to Figure 6 , there is no current block coding mode (or no selection) before the sub-block coding mode. In this case, the coding modes of blocks of different layers can be traversed in the order of small block-large block.

[0111] As another exemplary embodiment, when the coding division order is a large block-small block order, the sub-block coding mode is located at the last position in the mode queue, that is, referring to Figure 6 , there is no (or no selected) current block coding mode after the sub-block coding mode. In this case, the coding modes of blocks of different layers can be traversed in the order of large block-small block.

[0112] As another exemplary embodiment, when the coding division order is a large block-small block-large block order, the sub-block coding mode is located in the middle position of the mode queue, that is, referring to Figure 6 , there are (or are selected) one or more current block coding modes both after and after the sub-block coding mode. In this case, the coding modes of blocks of different layers can be traversed in the order of large block-small block-large block.

[0113] In addition, despite Figure 6 The case where the mode pool includes only four coding modes is shown, but this mode pool is only an example, and various mode pools according to exemplary embodiments of the present disclosure described above and below may be combined here.

[0114] Through the construction of this bidirectional dynamic coding division order, the encoder can adaptively configure the coding division order according to the requirements of the coding and decoding scenario. It can traverse from small blocks to large blocks, or from large blocks to small blocks. It can also perform a part of the traversal of large blocks, perform the traversal of small blocks and then return to continue the traversal of large blocks. In this way, the optimal block size or block division and coding strategy can be decided more quickly according to the actual application scenario, and the space and upper limit of the algorithms that can be added in this dynamic bidirectional coding division order will also be larger.

[0115] In addition, as described above, when executing each coding mode in sequence, the encoder often needs to perform a complete coding process including prediction, transformation, quantization and entropy coding. When executing these processes, even if it is found midway that the currently executed coding mode will not be determined as the optimal coding mode due to poor performance, it is impossible to jump out of the execution process of the coding mode in advance, which will lead to a waste of coding and decoding complexity. In this regard, the present disclosure also proposes a method for reducing the execution of unnecessary subsequent processing in the coding mode by using the intermediate prediction results of the coding process.

[0116] As an exemplary embodiment, a feasible prediction coding mode (for example, the intra-frame mode, inter-frame mode, merge mode and / or skip mode as described above, or other coding modes) can be split into multiple coding sub-modes. Due to the characteristics of the encoding process, there may be associations between the split multiple coding sub-modes.

[0117] As an exemplary embodiment, the multiple current block coding modes include multiple prediction modes (hereinafter, the prediction mode is referred to as "PRED" for short, or referred to as the first coding sub-mode) and multiple rate-distortion optimization modes (hereinafter, the rate-distortion optimization mode is referred to as "RDO" for short, or referred to as the second coding sub-mode) respectively split from the multiple coding modes. That is, each coding mode can be split into two coding sub-modes—PRED and RDO, and the input of the execution process of RDO can depend on the result of the execution process of PRED. For example, the result of the execution process of PRED can be the input of the execution process of RDO, such as by calculating the difference between the prediction result and the original image as the residual to perform subsequent transformation processing, quantization processing and entropy coding processing. Accordingly, the PRED of a coding mode can exist alone in the mode queue, but the corresponding RDO will not exist alone in the mode queue. In addition, since the RDO depends on the execution result of the corresponding PRED, if there is a PRED and an RDO corresponding to the PRED in the pattern queue, the RDO is located in a subsequent position of the PRED in the pattern queue, for example, a position after the PRED that is adjacent to the PRED or a position after the PRED that is separated from the PRED by one or more patterns.

[0118] As an exemplary embodiment, PRED (i.e., the first encoding sub-mode) is used to perform a prediction process in an encoding process of a corresponding encoding mode, and RDO (i.e., the second encoding sub-mode) is used to perform a transform process, a quantization process, and an entropy encoding process in an encoding process of a corresponding encoding mode. Figure 7 This is described in detail.

[0119] Figure 7 is an exemplary diagram showing a coding mode and a coding sub-mode according to an exemplary embodiment of the present disclosure. Figure 7 , a coding mode can be split into two coding sub-modes.

[0120] As an example, after performing the prediction process, PRED may output a distortion result (ie, distortion amount DISTORTION) indicating the distortion amount between the predicted value and the original value. The distortion result here may represent an intermediate result used to measure the encoding performance in the encoding process.

[0121] As an example, after performing subsequent transformation processing, quantization processing, and entropy coding processing, RDO may output a rate-distortion (optimization) result (i.e., rate-distortion cost RDcost) representing the amount of distortion in consideration of the bit rate. The rate-distortion optimization result (also referred to as rate-distortion result) here may represent the final result used to measure the coding mode in the coding process.

[0122] As an example, the pros and cons of coding modes may be measured based on multiple distortion results and / or rate-distortion (optimization) results of multiple coding sub-modes. For example, the optimal coding mode may be the coding mode with the minimum distortion result and / or rate-distortion optimization result.

[0123] Refer to the following Figure 8 and Fig. 9 Based on Figure 7 The video encoding method of the encoding sub-mode is described in detail.

[0124] Figure 8 and Fig. 9 is an exemplary diagram illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0125] Reference Figure 8 When multiple coding modes are respectively split into two coding sub-modes, for the current block (e.g., CU block), a 9-unit mode pool including 8 coding sub-modes (such as intra-PRED, intra-RDO, inter-PRED, inter-RDO, merged PRED, merged RDO, skip PRED and skip RDO) and sub-block coding modes can be constructed.

[0126] Similar to the operation of the mode pool including coding modes, by using an algorithm based on coding reference information, a mode queue including 6 modes "Intra-frame PRED—Sub-block coding mode—Skip PRED—Skip RDO—Merge PRED—Merge RDO" can be determined from the 9-unit mode pool. Here, although the mode queue is shown to include 6 modes in the above order, the present disclosure does not limit the number, category, order (or priority) of the modes in the queue, and the mode queue determined using the relevant algorithm may include any number of modes, any mode category and any mode order. For example, the mode queue determined from the mode pool may also be: Intra-frame PRED—Inter-frame PRED—Inter-frame RDO—Intra-frame RDO—Skip PRED.

[0127] In addition, refer to Fig. 9 , when the mode pool for the current block only includes multiple coding sub-modes of the current block, the same as the above reference Figure 6 Similarly to the description, considering the traversal of coding modes of blocks at different levels, the mode queue may have fixed sub-block coding modes, and one or more coding sub-modes determined from the 9-unit mode pool may exist before or after the fixed sub-block coding mode, or before and after, thereby realizing a bidirectional coding division order that varies according to the coding and decoding scenario.

[0128] By constructing a mode pool including the coding sub-mode of the current block and the sub-block coding mode, the flexibility of the coding architecture can be greatly improved. For example, for a 9-unit mode pool, up to 65,137 mode queues can be determined, which further greatly increases the types of mode queues compared to a fixed mode selection order in the related art. In addition, compared to the mode pool including the coding mode, the capacity of the mode pool including the coding sub-mode is increased, and the possibility of applying algorithms to make decisions is also increased accordingly. Furthermore, due to the use of the coding sub-mode, the coding complexity can be reduced, which will be described later.

[0129] When the mode queue is determined in step S420, the optimal encoding mode may be determined based on the determined mode queue. Figure 4 In step S430, at least one mode is executed according to the mode sequence in the mode queue to determine a coding mode for the current block or a plurality of sub-blocks divided from the current block.

[0130] Specifically, when a mode queue including at least one mode is determined, at least one mode in the ordered mode queue can be executed one by one in sequence to obtain at least one execution result corresponding to the at least one mode, and the optimal encoding mode can be determined based on the at least one execution result obtained.

[0131] As an example, based on at least one execution result respectively corresponding to at least one mode, a mode with an optimal execution result among the at least one mode may be selected as the optimal encoding mode.

[0132] For example, if the mode with the minimum rate-distortion cost is the current block coding mode (for example, the intra mode or intra RDO for the current block has the minimum RDcost compared to other modes), the coding mode corresponding to the current block coding mode (i.e., the intra mode) is determined as the coding mode for the current block.

[0133] For another example, if the mode with the minimum rate-distortion cost is a sub-block coding mode (for example, the sum of the RDcosts of the multiple sub-blocks divided from the current block is less than the RDcost in other coding modes or the RDcost of other RDOs), the respective coding modes of the multiple sub-blocks corresponding to the sub-block coding mode are determined as coding modes for the multiple sub-blocks. That is, in this case, the coding mode for the current block is not determined, but the coding mode for the sub-blocks divided from the current block is determined. In this case, the determined coding modes of the multiple sub-blocks may also indicate a division method (such as division direction, division size and / or division shape, etc.) for dividing the sub-blocks from the current block. Here, the present disclosure does not impose specific restrictions on the division method.

[0134] In addition, the present disclosure also proposes that the execution of some subsequent modes can be terminated early or skipped when the execution results of the previous mode are obtained. That is, before completing the execution of all modes in the mode queue, some modes in the mode queue can be adaptively selectively skipped (or pruned) to speed up the mode decision process.

[0135] As an exemplary embodiment, at least one mode is executed in the mode sequence in the mode queue, and the step of determining the coding mode for the current block or the plurality of sub-blocks divided from the current block includes: based on the execution result of the previous mode in the mode queue, skipping the execution of at least one subsequent mode in the mode queue. This is described in detail below.

[0136] As an exemplary embodiment, when the previous mode is a coding mode, based on the rate-distortion optimization result and / or distortion result obtained by performing the coding process using the coding mode, the execution of at least one subsequent mode in the mode queue is skipped. For example, based on the rate-distortion optimization result obtained by the coding process of the previously executed coding mode, it is determined to skip the execution of at least one subsequent mode in the mode queue, but this is only an example, and the method of skipping some modes disclosed in the present invention is not limited to this.

[0137] As an example, multiple rate-distortion optimization results as the execution results of multiple coding modes can be used to measure the coding performance of each coding mode. The smaller the RDcost of a coding mode, the better the performance of the coding mode for the current block. As an example, multiple distortion results determined based on the execution results of multiple coding modes can also be used to measure the coding performance of each coding mode. For example, referring to Figure 5 Based on the execution result of the intra mode in the mode queue, you can directly jump to the skip mode in the mode queue. Here, the skipping or pruning method based on the execution result of the coding mode is only an example, and the present disclosure does not limit this.

[0138] As an exemplary embodiment, when the previous mode is the first encoding submode, execution of the corresponding second encoding submode in the mode queue is skipped based on a distortion result obtained by performing a prediction process using the first encoding submode.

[0139] As an example, multiple distortion results as intermediate results of multiple coding modes may be used to measure each coding mode. For example, the larger the DISTORTION of a coding mode is, the worse the performance of the coding mode is for the current block.

[0140] Since the rate-distortion optimization result of the coding mode is highly correlated with the distortion result, when the DISTORTION of a certain coding mode is significantly larger than that of other modes, the possibility that the coding mode is determined as the optimal coding mode for the current block is relatively low, and the corresponding RDO may not be executed. In other words, since the purpose of the coding mode decision is to select a coding mode with the best coding performance, once it is determined that the coding mode has no advantage over other modes based on the execution result of the PRED of a coding mode, the execution of the subsequent RDO of the coding mode may be skipped to reduce unnecessary processing. In other words, the multiple distortion results determined by using multiple PREDs can be regarded as selection reference factors, which are used to select part of the RDOs to be executed from the multiple RDOs corresponding to the multiple PREDs, thereby eliminating another part of the RDOs that are unnecessary to execute.

[0141] For example, refer to Figure 8 , based on the execution result of the skip PRED of the skip mode in the mode queue, it is determined that the encoding performance of the skip mode for the current block is poor relative to other modes, and the skip RDO may not be executed, but the merged PRED in the mode queue may be directly jumped to. Here, the skipping or pruning method based on the execution result of the encoding sub-mode is only an example, and the present disclosure is not limited to this.

[0142] As an exemplary embodiment, when the previous mode is a sub-block coding mode, based on the sum of rate-distortion optimization results and / or the sum of distortion results of multiple sub-blocks obtained by performing coding processing using the sub-block coding mode, the execution of at least one subsequent mode in the mode queue is skipped. As described above, the sum of rate-distortion optimization results and / or the sum of distortion results of multiple sub-blocks of the coding mode of multiple sub-blocks can be compared with the rate-distortion optimization results and / or distortion results of other current block coding modes to measure the coding performance of the sub-block coding mode. Here, the sum of rate-distortion optimization results based on multiple sub-blocks can be understood as: the sum of rate-distortion optimization results of multiple sub-blocks is regarded as the rate-distortion optimization result of the block corresponding to the multiple sub-blocks for dividing the multiple sub-blocks. Therefore, the manner of skipping the execution of at least one subsequent mode based on the sum of rate-distortion optimization results of multiple sub-blocks is similar to the manner of skipping the execution of at least one subsequent mode based on the rate-distortion optimization result of the current block, and no repeated description will be given. In addition, similarly, the sum of the distortion results based on multiple sub-blocks can be understood as the corresponding distortion result of the block used to divide the multiple sub-blocks. Therefore, the method of skipping the execution of at least one subsequent mode based on the sum of the distortion results of multiple sub-blocks is similar to the method of skipping the execution of at least one subsequent mode based on the distortion result of the current block, and will not be repeated here.

[0143] In addition, the execution result of the previous mode can also affect the skipping or pruning of the subsequent mode in other appropriate ways, and the present disclosure does not limit this. In addition, the present disclosure does not limit the specific details of skipping some modes based on the execution result of the previous mode. The algorithm in the related field can be used to adaptively skip some modes, which is not described in detail here.

[0144] Optionally, the video encoding method according to the exemplary embodiment of the present disclosure may not skip any mode, but execute each mode one by one according to the mode queue to determine the encoding mode for the current block or multiple sub-blocks.

[0145] By skipping unnecessary execution modes in the mode queue, the encoding process complexity can be reduced while ensuring the encoding quality, and the upper limit of the algorithm that can be added can also be increased accordingly.

[0146] Furthermore, as described above, in an exemplary embodiment of the present disclosure, the subblock encoding mode includes determination of encoding modes of a plurality of subblocks divided from a current block.

[0147] As an exemplary embodiment, the step of executing the sub-block coding mode includes: obtaining multiple sub-blocks by dividing the current block; for each of the multiple sub-blocks, determining the coding mode of the corresponding sub-block by constructing a mode pool and determining a mode queue.

[0148] That is to say, for each of the multiple sub-blocks divided from the current block, steps corresponding to step 410 to step S430 can be performed respectively, that is, a mode pool for the current sub-block is constructed, and based on the mode pool for the current sub-block, a mode queue including at least one mode for the current sub-block is determined, at least one mode is executed according to the mode order in the mode queue, and the encoding mode for the current sub-block or multiple further divided sub-blocks further divided from the current sub-block is determined.

[0149] As an example, each of the multiple sub-blocks can be regarded as a current block and operations similar to those described with reference to the above-mentioned drawings can be performed, and the encoding mode for each sub-block or the encoding mode of the sub-blocks further divided from the corresponding sub-block can be determined in sequence by repeating the above-mentioned operations for each sub-block.

[0150] For example, when the current block is a block with a size of 64×64, in the sub-block coding mode, the current block can be first divided into four sub-blocks with a size of 32×32, for example, according to four partitions, and for each 32×32 sub-block, steps corresponding to steps S410 to S430 are performed, and each mode for each sub-block can be determined respectively, for example, the intra-frame mode for the first sub-block and the second sub-block, the coding mode for multiple sub-blocks with a size of 16×16 further divided into the third sub-block, and the inter-frame mode for the fourth sub-block.

[0151] In addition, when the coding mode for the current block or multiple sub-blocks divided from the current block is determined in step S43 for the current block, the execution result in the sub-block coding mode may include the sum of the rate-distortion optimization results of the coding modes of the multiple sub-blocks, that is, the RDcost of the intra-frame mode for the first sub-block, the RDcost of the intra-frame mode for the second sub-block, the sum of the RDcost of the coding modes of multiple sub-blocks of size 16×16 further divided from the third sub-block, and the sum of the RDcost of the inter-frame mode for the fourth sub-block.

[0152] Details of the operation of determining the encoding mode of the current block of the exemplary embodiment of the present disclosure may be applicable to the operation of determining the encoding modes of various blocks including subblocks, and thus, the present disclosure will not describe them in detail.

[0153] In the present disclosure, the above series of operations on determining the coding mode may be performed for the recursively divided blocks layer by layer. As an exemplary embodiment, the mode pools and / or mode queues of different sub-blocks may be the same or different. That is, different mode pools may be used for different sub-blocks, and different mode queues may be determined for different sub-blocks.

[0154] As such, the determination of the coding mode of the sub-block has great flexibility and algorithm addition space similar to the determination of the coding mode of the current block, and therefore, the flexibility of the coding architecture of the embodiment of the present disclosure can be further greatly improved.

[0155] As described above, when the applicable encoding mode is determined in step S430, encoding can be performed using the determined encoding mode. Figure 4 In step S440, the current block or multiple sub-blocks are encoded based on the determined encoding mode. Here, the present disclosure will not further describe in detail the specific operation of performing encoding using the encoding mode.

[0156] For video encoding and decoding, the coding architecture is the basis of the entire encoder. The video coding method according to the exemplary embodiment of the present invention constructs a highly adaptive video coding architecture that is different from the traditional video coding architecture, which can greatly improve the flexibility of the encoder's video coding process and increase the upper limit of the algorithm.

[0157] According to the exemplary embodiments of the video encoding method of the present disclosure, the mode pool construction, mode queue determination and / or encoding mode determination are not fixed, and can be adaptively adjusted according to the sequence characteristics in combination with the algorithm. This flexibility enables the video encoding method according to the exemplary embodiments of the present disclosure to have excellent performance when facing various video sequences.

[0158] The video encoding method according to the exemplary embodiments of the present disclosure can be applied to or migrated to various types of encoders and / or hybrid coding framework encoders without limiting the encoding standard and has high universality.

[0159] The video encoding method according to the exemplary embodiment of the present disclosure can significantly improve the encoding speed while ensuring the encoding quality, and has a relatively high encoding efficiency.

[0160] Fig.10 A structural block diagram of a video encoding apparatus 1000 according to an exemplary embodiment of the present disclosure is shown.

[0161] like Fig.10 As shown, the video encoding device 1000 according to an exemplary embodiment of the present disclosure may include: a mode pool construction unit 1010, a queue determination unit 1020, a mode determination unit 1030 and an encoding unit 1040.

[0162] Specifically, the mode pool construction unit 1010 is configured to construct a mode pool for the current block, wherein the mode pool includes multiple current block coding modes or multiple current block coding modes and sub-block coding modes. The queue determination unit 1020 is configured to determine a mode queue including at least one mode for the current block based on the mode pool. The mode determination unit 1030 is configured to execute at least one mode according to the mode order in the mode queue to determine the coding mode for the current block or multiple sub-blocks divided from the current block. The encoding unit 1040 is configured to encode the current block or multiple sub-blocks based on the determined coding mode. Among them, multiple current block coding modes are related to the coding of the current block, and the sub-block coding mode is related to the coding of multiple sub-blocks.

[0163] As an exemplary embodiment, the mode determination unit 1030 is configured to execute the sub-block coding mode through the following operations: by dividing the current block, a plurality of sub-blocks are obtained; for each of the plurality of sub-blocks, the coding mode of the corresponding sub-block is determined by constructing a mode pool and determining a mode queue, wherein the mode pools and / or mode queues of different sub-blocks are the same or different.

[0164] As an exemplary embodiment, when the mode pool includes only multiple current block coding modes, the mode queue includes sub-block coding modes according to the coding division order, wherein the position of the sub-block coding mode in the mode queue is related to the coding division order.

[0165] As an exemplary embodiment, the coding division order includes a small block-large block order, a large block-small block order and / or a large block-small block-large block order, wherein, when the coding division order is a small block-large block order, the sub-block coding mode is located at the first position in the mode queue; when the coding division order is a large block-small block order, the sub-block coding mode is located at the last position in the mode queue; and / or when the coding division order is a large block-small block-large block order, the sub-block coding mode is located in the middle position in the mode queue.

[0166] As an exemplary embodiment, the queue determination unit 1020 is configured to determine a mode queue including at least one mode for the current block based on the mode pool by the following operations: selecting at least one mode from the mode pool and determining the order of at least one mode in the mode queue by referring to the encoding information of the neighboring blocks of the current block, the encoding information of the parent block or child block of the current block and / or the pre-analyzed encoding information of the video frame in which the current block is located.

[0167] As an exemplary embodiment, the multiple current block encoding modes include multiple encoding modes for performing encoding processing on the current block or multiple first encoding sub-modes and multiple second encoding sub-modes respectively split from the multiple encoding modes, wherein the first encoding sub-mode is used to perform prediction processing in the encoding processing of the corresponding encoding mode, and the second encoding sub-mode is used to perform transformation processing, quantization processing and entropy coding processing in the encoding processing of the corresponding encoding mode.

[0168] As an exemplary embodiment, the mode determination unit 1030 is configured to determine the encoding mode for the current block or multiple sub-blocks divided from the current block by executing at least one mode in the mode order in the mode queue through the following operations: based on the execution result of the previous mode in the mode queue, skipping the execution of at least one subsequent mode in the mode queue.

[0169] As an exemplary embodiment, the mode determination unit 1030 is configured to skip the execution of at least one subsequent mode in the mode queue based on the execution result of the previous mode in the mode queue through the following operations: when the previous mode is a coding mode, based on the rate-distortion optimization result and / or distortion result obtained by performing the coding process using the coding mode, skip the execution of at least one subsequent mode in the mode queue; when the previous mode is a first coding sub-mode, based on the distortion result obtained by performing the prediction process using the first coding sub-mode, skip the execution of the corresponding second coding sub-mode in the mode queue; and / or when the previous mode is a sub-block coding mode, based on the sum of the rate-distortion optimization results and / or the sum of the distortion results of multiple sub-blocks obtained by performing the coding process using the sub-block coding mode, skip the execution of at least one subsequent mode in the mode queue.

[0170] Regarding the video encoding device 1000 in the above embodiment, the specific manner in which each unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0171] In addition, it should be understood that the various units in the video encoding device 1000 according to the exemplary embodiment of the present disclosure may be implemented as hardware components and / or software components. Those skilled in the art may implement the various units, for example, using a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), according to the processing performed by the defined various units.

[0172] In addition, the present disclosure also provides an electronic device, which includes: at least one memory and at least one processor, wherein a computer executable instruction set is stored in the at least one memory, and when the computer executable instruction set is executed by the at least one processor, the video encoding method described in the above exemplary embodiment is executed.

[0173] As an example, the electronic device may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instruction set. Here, the electronic device is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instruction (or instruction set) individually or in combination. The electronic device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0174] In electronic devices, the processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0175] The processor can execute instructions or codes stored in the memory, wherein the memory can also store data. Instructions and data can also be sent and received through the network via the network interface device, wherein the network interface device can adopt any known transmission protocol.

[0176] The memory may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor or the like. In addition, the memory may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the memory.

[0177] In addition, the electronic device may also include a multimedia resource display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0178] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may further be provided, wherein when the instructions are executed by at least one processor, the at least one processor is prompted to perform the video encoding method as described in the above exemplary embodiments.

[0179] Examples of computer-readable storage media include read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0180] Fig.11 A computing environment 1110 is shown coupled to a user interface 1150. The computing environment 1110 may be part of a data processing server. The computing environment 1110 includes a processor 1120, a memory 1130, and an input / output (I / O) interface 1140.

[0181] The processor 1120 generally controls the overall operation of the computing environment 1110, such as operations associated with display, data acquisition, data communication, and image processing. The processor 1120 may include one or more processors for executing instructions to perform all or some of the steps in the above method. In addition, the processor 1120 may include one or more modules that facilitate the interaction between the processor 1120 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0182] The memory 1130 is configured to store various types of data to support the operation of the computing environment 1110. The memory 1130 may include predetermined software 1132. Examples of such data include instructions for any application or method operating on the computing environment 1110, video data sets, image data, etc. The memory 1130 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0183] The I / O interface 1140 provides an interface between the processor 1120 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 1140 may be coupled to an encoder and a decoder.

[0184] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 1130 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 1120 in a computing environment 1110 to perform the above method. In one example, the plurality of programs can be executed by a processor 1120 in a computing environment 1110 to (for example, from Figure 2 The video encoder 20 in the computing environment 1110 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1120 in the computing environment 1110 to perform the above-mentioned decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 1120 in the computing environment 1110 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 1120 in the computing environment 1110 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0185] In an embodiment, a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is provided. In an embodiment, a bit stream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0186] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 1120); and a non-transitory computer-readable storage medium or memory 1130 in which multiple programs that can be executed by one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0187] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, wherein the bitstream includes the encoded video information generated by the above encoding method or the encoded video information to be decoded by the above decoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 1130 is also provided, wherein the plurality of programs can be executed by a processor 1120 in a computing environment 1110 for performing the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0188] In an embodiment, the computing environment 1110 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0189] In an embodiment, a method for storing a bit stream is also provided, comprising: storing the bit stream on a digital storage medium, wherein the bit stream comprises encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method.

[0190] In an embodiment, a method for transmitting a bit stream generated by the above encoder is also provided. In an embodiment, a method for receiving a bit stream to be decoded by the above decoder is also provided.

[0191] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0192] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0193] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video encoding method, characterized in that: include: Constructing a mode pool for a current block, wherein the mode pool includes a plurality of current block coding modes or the plurality of current block coding modes and a sub-block coding mode; Based on the mode pool, determining a mode queue including at least one mode for a current block; Execute the at least one mode according to the mode order in the mode queue to determine a coding mode for a current block or a plurality of sub-blocks divided from the current block; encoding the current block or the plurality of sub-blocks based on the determined encoding mode, The multiple current block encoding modes are related to the encoding of the current block, and the sub-block encoding modes are related to the encoding of the multiple sub-blocks.

2. The video encoding method according to claim 1, characterized in that The steps of executing the sub-block coding mode include: By dividing the current block, the multiple sub-blocks are obtained; For each of the plurality of sub-blocks, a coding mode of the corresponding sub-block is determined by constructing a mode pool and determining a mode queue, The mode pools and / or mode queues of different sub-blocks are the same or different.

3. The video encoding method according to claim 1, characterized in that: When the mode pool includes only the multiple current block coding modes, the mode queue includes the sub-block coding modes according to the coding division order, The position of the sub-block coding mode in the mode queue is related to the coding division order.

4. The video encoding method according to claim 3, characterized in that: The coding division order includes a small block-large block order, a large block-small block order and / or a large block-small block-large block order. Wherein, when the coding division order is a small block-large block order, the sub-block coding mode is located at the first position in the mode queue; When the coding division order is a large block-small block order, the sub-block coding mode is located at the last position in the mode queue; and / or When the coding division sequence is a large block-small block-large block sequence, the sub-block coding mode is located in the middle position of the mode queue.

5. The video encoding method according to claim 1, characterized in that: The step of determining a mode queue including at least one mode for the current block based on the mode pool comprises: By referring to the encoding information of the neighboring blocks of the current block, the encoding information of the parent block or child block of the current block and / or the pre-analyzed encoding information of the video frame where the current block is located, at least one mode is selected from the mode pool and the order of the at least one mode in the mode queue is determined.

6. The video encoding method according to claim 1, characterized in that: The multiple current block encoding modes include multiple encoding modes for performing encoding processing on the current block or multiple first encoding sub-modes and multiple second encoding sub-modes respectively split from the multiple encoding modes, The first coding sub-mode is used to perform prediction processing in the coding process of the corresponding coding mode, and the second coding sub-mode is used to perform transformation processing, quantization processing and entropy coding processing in the coding process of the corresponding coding mode.

7. The video encoding method according to claim 6, characterized in that: The step of executing the at least one mode according to the mode order in the mode queue to determine the coding mode for the current block or the plurality of sub-blocks divided from the current block comprises: Based on the execution result of the previous pattern in the pattern queue, the execution of at least one subsequent pattern in the pattern queue is skipped.

8. The video encoding method according to claim 7, characterized in that: The step of skipping the execution of at least one subsequent mode in the mode queue based on the execution result of the previous mode in the mode queue comprises: When the previous mode is a coding mode, skipping execution of at least one subsequent mode in the mode queue based on a rate-distortion optimization result and / or a distortion result obtained by performing a coding process using the coding mode, When the previous mode is a first coding submode, skipping execution of a corresponding second coding submode in the mode queue based on a distortion result obtained by performing a prediction process using the first coding submode, and / or When the previous mode is a subblock coding mode, execution of at least one subsequent mode in the mode queue is skipped based on the sum of rate-distortion optimization results and / or the sum of distortion results of the multiple subblocks obtained by performing encoding processing using the subblock coding mode.

9. A video encoding device, characterized in that: include: A mode pool construction unit, configured to construct a mode pool for a current block, wherein the mode pool includes a plurality of current block coding modes or the plurality of current block coding modes and a sub-block coding mode; a queue determination unit configured to determine a mode queue including at least one mode for a current block based on the mode pool; a mode determination unit, configured to execute the at least one mode according to the mode order in the mode queue, and determine a coding mode for a current block or a plurality of sub-blocks divided from the current block; an encoding unit, configured to encode the current block or the plurality of sub-blocks based on the determined encoding mode, The multiple current block encoding modes are related to the encoding of the current block, and the sub-block encoding modes are related to the encoding of the multiple sub-blocks.

10. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video encoding method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the video encoding method according to any one of claims 1 to 8.

12. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by at least one processor, the video encoding method according to any one of claims 1 to 8 is implemented.

13. A method for generating a bit stream, characterized in that: include: A bit stream is generated according to the video encoding method according to any one of claims 1 to 8.

Images