Video encoding methods, apparatus, electronic devices and storage media

Abstract

This disclosure provides a video encoding method, apparatus, electronic device, and storage medium. The video encoding method includes: constructing a mode pool for a current block, wherein the mode pool includes a plurality of current block encoding modes or the plurality of current block encoding modes and sub-block encoding modes; determining a mode queue for the current block, including at least one mode, based on the mode pool; executing the at least one mode in the mode queue in mode order to determine an encoding mode for the current block or a plurality of sub-blocks divided from the current block; encoding the current block or the plurality of sub-blocks based on the determined encoding modes, wherein the plurality of current block encoding modes are related to the encoding of the current block, and the sub-block encoding modes are related to the encoding of the plurality of sub-blocks.

Description

Technical Field

[0001] This application relates to video encoding / decoding and compression. More specifically, this application relates to a video encoding method, a bitstream generation method, an apparatus, an electronic device, a storage medium, and a program product. Background Technology

[0002] Various electronic devices (such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc.) support digital video. Electronic devices send and receive, or otherwise transmit, digital video data via communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited storage resources of storage devices, video data can be compressed using one or more video codec standards before it is transmitted or stored. For example, video codec standards include Universal Video Codec (VVC), Joint Explore Test Model (JEM), High Efficiency Video Codec (HEVC / H.265), Advanced Video Codec (AVC / H.264), Moving Picture Experts Group (MPEG) codec, etc. Video codecs typically employ prediction methods that utilize the inherent redundancy in video data (e.g., inter-frame prediction, intra-frame prediction, etc.). Video codecs aim to compress video data to a form using a lower bitrate while avoiding or minimizing degradation in video quality.

[0003] The H.265 / HEVC standard was officially proposed in 2013. Its compression performance is significantly improved compared to the H.264 standard. The widespread application of HEVC technology can save storage and bandwidth costs, and its application prospects are broad. The H.265 / HEVC standard adds new encoding and decoding tools to the original encoding and decoding structure and tools of the H.264 standard, significantly improving compression performance while also bringing a huge increase in complexity. The increased complexity of H.265 / HEVC encoding and decoding largely stems from the selection of encoding modes during the encoding process. However, the related encoding mode selection process has many problems (e.g., the fixed order of video block encoding division, the fixed order of video block mode traversal, and the fixed execution flow of video block modes), making the encoding mode selection process less flexible, limiting the types or number of algorithms that can be added during mode selection, and resulting in high encoding and decoding complexity. Summary of the Invention

[0004] Exemplary embodiments of this disclosure provide a video encoding method, a bitstream generation method, an apparatus, an electronic device, a storage medium, and a program product, which can improve the flexibility of encoding and decoding mode selection, increase the upper limit of the algorithm, and are applicable to various hybrid encoding framework encoders, etc.

[0005] According to a first aspect of the present disclosure, a video encoding method is provided, comprising: constructing a mode pool for a current block, wherein the mode pool includes a plurality of current block encoding modes or the plurality of current block encoding modes and sub-block encoding modes; determining a mode queue for the current block, including at least one mode, based on the mode pool; executing the at least one mode in the mode queue in mode order to determine encoding modes for the current block or a plurality of sub-blocks divided from the current block; encoding the current block or the plurality of sub-blocks based on the determined encoding modes, wherein the plurality of current block encoding modes are related to the encoding of the current block, and the sub-block encoding modes are related to the encoding of the plurality of sub-blocks.

[0006] Optionally, the step of executing the sub-block encoding pattern includes: obtaining the plurality of sub-blocks by dividing the current block; and determining the encoding pattern of the corresponding sub-block by constructing a pattern pool and determining a pattern queue for each of the plurality of sub-blocks, wherein the pattern pools and / or pattern queues of different sub-blocks are the same or different.

[0007] Optionally, when the pattern pool only includes the plurality of current block encoding patterns, the pattern queue includes the sub-block encoding patterns according to the encoding partitioning order, wherein the position of the sub-block encoding pattern in the pattern queue is related to the encoding partitioning order.

[0008] Optionally, the encoding partitioning order includes a small-block-large-block order, a large-block-small-block order, and / or a large-block-small-block-large-block order, wherein when the encoding partitioning order is small-block-large-block order, the sub-block encoding pattern is located at the first position in the pattern queue; when the encoding partitioning order is large-block-small-block order, the sub-block encoding pattern is located at the last position in the pattern queue; and / or when the encoding partitioning order is large-block-small-block-large-block order, the sub-block encoding pattern is located at the middle position in the pattern queue.

[0009] Optionally, the step of determining a mode queue including at least one mode for the current block based on the mode pool includes: selecting the at least one mode from the mode pool and determining the order of the at least one mode in the mode queue by referring to the encoding information of neighboring blocks of the current block, the encoding information of the parent or child blocks of the current block and / or the pre-analyzed encoding information of the video frame in which the current block is located.

[0010] Optionally, the plurality of current block encoding modes include a plurality of encoding modes for performing encoding processing on the current block, or a plurality of first encoding sub-modes and a plurality of second encoding sub-modes respectively split from the plurality of encoding modes, wherein the first encoding sub-modes are used to perform prediction processing in the encoding processing of the corresponding encoding mode, and the second encoding sub-modes are used to perform transform processing, quantization processing and entropy encoding processing in the encoding processing of the corresponding encoding mode.

[0011] Optionally, the step of executing the at least one mode in the pattern queue according to the pattern order to determine the encoding mode for the current block or multiple sub-blocks divided from the current block includes: skipping the execution of at least one subsequent mode in the pattern queue based on the execution result of the previous mode in the pattern queue.

[0012] Optionally, the step of skipping the execution of at least one subsequent mode in the mode queue based on the execution result of a previous mode in the mode queue includes: when the previous mode is an encoding mode, skipping the execution of at least one subsequent mode in the mode queue based on the rate-distortion optimization result and / or distortion result obtained by performing encoding processing using the encoding mode; when the previous mode is a first encoding sub-mode, skipping the execution of the corresponding second encoding sub-mode in the mode queue based on the distortion result obtained by performing prediction processing using the first encoding sub-mode; and / or when the previous mode is a sub-block encoding mode, skipping the execution of at least one subsequent mode in the mode queue based on the sum of the rate-distortion optimization results and / or the sum of the distortion results of the plurality of sub-blocks obtained by performing encoding processing using the sub-block encoding mode.

[0013] According to a second aspect of the present disclosure, a video encoding apparatus is provided, comprising: a mode pool construction unit configured to construct a mode pool for a current block, wherein the mode pool includes a plurality of current block encoding modes or the plurality of current block encoding modes and sub-block encoding modes; a queue determination unit configured to determine a mode queue for the current block, including at least one mode, based on the mode pool; a mode determination unit configured to execute the at least one mode in the mode queue in mode order to determine an encoding mode for the current block or a plurality of sub-blocks divided from the current block; and an encoding unit configured to encode the current block or the plurality of sub-blocks based on the determined encoding modes, wherein the plurality of current block encoding modes are related to the encoding of the current block, and the sub-block encoding modes are related to the encoding of the plurality of sub-blocks.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein, when executed by the at least one processor, the computer-executable instructions cause the at least one processor to perform the video encoding method as described above.

[0015] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by at least one processor, causes the at least one processor to perform the video encoding method as described above.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer instructions that, when executed by at least one processor, implement the video encoding method as described above.

[0017] According to a sixth aspect of the present disclosure, a bitstream generation method is provided, comprising: generating a bitstream according to the video encoding method described above.

[0018] According to a seventh aspect of the present disclosure, a computer program product is provided having instructions for storing a bit stream, the bit stream including video information generated according to the video encoding method described above.

[0019] According to the exemplary embodiments of the present disclosure, the video encoding method, bitstream generation method, apparatus, electronic device, storage medium, and program product determine the encoding mode for the current block component mode pool and the mode queue, and perform encoding using the determined encoding mode according to the mode queue execution mode. This can improve the encoding quality while increasing the flexibility of the encoding architecture, increasing the diversity of algorithms that can be added, and improving the encoding efficiency, thereby improving the overall encoding and decoding performance.

[0020] It will be understood that the above general description and the following detailed description are merely examples and do not limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate examples according to this disclosure and, together with this description, serve to explain the principles of this disclosure.

[0022] Figure 1 This is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0023] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0024] Figure 3This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0025] Figure 4 This is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0026] Figure 5 and Figure 6 This is an exemplary diagram illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0027] Figure 7 This is an exemplary diagram illustrating encoding modes and encoding sub-modes according to exemplary embodiments of the present disclosure.

[0028] Figure 8 and Figure 9 This is an exemplary diagram illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0029] Figure 10 A structural block diagram of a video encoding apparatus according to an exemplary embodiment of the present disclosure is shown.

[0030] Figure 11 This is a diagram illustrating a computing environment coupled to a user interface according to some embodiments of the present disclosure. Detailed Implementation

[0031] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this disclosure are used to distinguish objects and are not used to describe any specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in sequences other than those shown in the drawings or described in this disclosure.

[0033] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0034] In some implementations, the target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14.

[0035] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the target device 14 can access the encoded video data in the storage device 32 via the input interface 28.

[0036] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0037] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback.

[0038] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0039] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0040] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0041] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0042] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or it can be distributed among one or more of the fixed or programmable hardware units described.

[0043] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... Figure 1 The video source 18 shown obtains video data from the video data storage 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 (e.g., in intra-frame or inter-frame predictive coding mode) when encoding the video data.

[0044] like Figure 2 As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments (also referred to below as “partitioning”) the video data into video blocks. This segmentation may also include dividing the video frame into strips, tiles (e.g., sets of video blocks) or other larger coding units (CUs) according to a predefined splitting structure (e.g., a quadtree (QT) structure) associated with the video data. It should be noted that the term “block” or “video block” as used herein can be a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring to, for example, HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0045] The prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0046] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0047] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block within the current video frame or picture relative to a prediction block within a reference frame associated with the current block being encoded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0048] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0049] Intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, during individual encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) can select a suitable intra-prediction mode from the tested intra-prediction modes for use. Intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to entropy coding unit 56. Entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0050] After prediction processing unit 41 determines the prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0051] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0052] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1 The video decoder 30 shown, or archived in, for example Figure 1 The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame being encoded.

[0053] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0054] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0055] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0056] In some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame prediction (BC) unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30 (e.g., motion compensation unit 82, intra-frame prediction unit 84, and entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame prediction (BC) unit 85, and the functionality of the intra-frame prediction (BC) unit 85 may be performed by other components of the prediction processing unit 81 (e.g., motion compensation unit 82).

[0057] The video data storage device 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of the video decoder 30. The video data stored in the video data storage device 79 can be obtained, for example, from the storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0058] During the decoding process, the video decoder 30 receives a encoded video bitstream representing video blocks and associated syntax elements of encoded video frames. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0059] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0060] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0061] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within a reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0062] The motion compensation unit 82 and / or the intra-frame BC unit 85 determine the prediction information for the video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0063] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0064] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0065] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be directly provided to the DPB 92 by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., ...). Figure 1 On the display device 34).

[0066] As described above, on the encoder side, in order to select a suitable predictive coding mode (hereinafter also referred to as the "coding mode") for the current video block (hereinafter also referred to as the "current block"), the video encoder (hereinafter also referred to as the "encoder") can perform multiple encoding passes to select a suitable coding mode for each block of video data. For example, the encoder uses various predictive coding modes to encode the current block and tests their performance during individual encoding passes, and selects the appropriate optimal coding mode to use from the various tested predictive coding modes.

[0067] In related technologies, when determining the optimal encoding mode for the current block, the encoder can perform encoding one by one using each encoding mode of the current block (i.e., traversing each encoding mode) to obtain multiple encoding results for each encoding mode of the current block. In addition to encoding for each encoding mode of the current block, the encoder may also divide (or segment) the current block to obtain sub-blocks, and perform encoding one by one using each encoding mode of the sub-blocks according to the execution method of each encoding mode of the current block, to obtain multiple encoding results for each encoding mode of the sub-blocks. Finally, the encoder determines the optimal encoding mode applicable to the current block and / or sub-blocks based on the multiple encoding results of the current block and / or sub-blocks. In the following text, the related process or architecture can be understood as an "encoding mode selection process" or a "coding architecture" relating to the encoding mode selection process, respectively.

[0068] Typically, in the encoding process of relevant encoders (e.g., HM encoder, x265 encoder, etc.), the encoding mode selection process used to determine the optimal encoding mode for the current block has problems such as fixed encoding division order of video blocks, fixed mode selection order of video blocks, fixed mode selection process of video blocks, limited algorithms that can be added in the encoding mode selection process, and excessive encoding and decoding complexity.

[0069] Specifically, encoders often employ a fixed encoding mode selection strategy when choosing the optimal encoding mode for the current block. For example, they might use a fixed encoding partitioning order for the current block (i.e., a fixed traversal order of blocks at different levels), such as a large-block-small-block order in a top-down encoding architecture, a small-block-large-block order in a bottom-up encoding architecture, or a large-block-small-block-large-block order in a parent-child-parent encoding architecture. This fixed traversal order of blocks of different sizes is highly inflexible, preventing the encoder from quickly determining the optimal block size.

[0070] Furthermore, encoders often execute encoding modes in a fixed mode selection order (i.e., the traversal order of modes). For example, for all video blocks and their sub-blocks, the skip mode is executed first, followed by the merge mode, then the inter mode, and finally the intra mode to determine the optimal encoding mode, without considering the encoding / decoding scenario and / or sequence characteristics. This mode traversal order lacks flexibility, preventing the encoder from quickly executing the optimal encoding mode. For instance, even if the skip mode executed first is not the optimal encoding mode for the current block, it must still be executed before any subsequent better modes can be executed. This fixed mode selection order completely disregards the possibility of prioritizing the execution of potentially optimal encoding modes.

[0071] Besides the fixed encoding partitioning order and pattern traversal order, the encoder must also follow a fixed pattern execution flow when selecting encoding modes. For example, when executing an encoding mode during pattern traversal, the encoder must complete the entire encoding process for that mode (e.g., prediction, transform, quantization, and entropy encoding), and cannot perform only partial processing. Therefore, the encoding process for the next encoding mode can only be executed after the complete encoding process for the current encoding mode has been completed. In this case, even if the intermediate results of the current encoding mode are not good, the execution flow of the current encoding mode cannot be terminated early. Such a fixed pattern execution flow lacks flexibility, resulting in reduced encoding and decoding efficiency.

[0072] Furthermore, the inflexibility of the aforementioned fixed mode selection process limits the types and number of algorithms that can be added during the process, which is not conducive to the adaptive changes of the encoding architecture for different encoding and decoding scenarios.

[0073] In other words, the mode selection process of related technologies requires all sequences of video to go through the same fixed mode selection process to determine the optimal encoding mode. Each encoding mode cannot end prematurely, and the limited number of algorithms that can be added under the corresponding encoding architecture makes it impossible for the mode selection process of related technologies to adapt to changes in encoding and decoding scenarios.

[0074] At least in view of the aforementioned deficiencies in related technologies, the exemplary embodiments of this disclosure provide a video coding technique with an adaptive mode selection process that offers high flexibility, a high upper limit for achievable algorithms, and high versatility. In the exemplary embodiments of this disclosure, the coding mode selection process (or coding architecture) for selecting the optimal coding mode is reconstructed. The coding partitioning order, mode traversal order, and / or mode execution flow of blocks of different sizes in the reconstructed mode selection process can be adaptively changed according to the encoding / decoding scenario. This significantly reduces encoding / decoding complexity, enhances the flexibility of the coding architecture, and significantly increases the upper limit of addable algorithms while ensuring the accuracy of mode selection, thereby improving overall encoding / decoding performance. This will be described in detail below with reference to the accompanying drawings.

[0075] In this disclosure, the video encoding method according to an exemplary embodiment of the present disclosure includes various steps performed for the current block to be encoded. Therefore, it should be understood that after the video encoding method according to an exemplary embodiment of the present disclosure is performed for the current block, the video encoding method according to an exemplary embodiment of the present disclosure may continue to be performed for the next video block to be encoded (i.e., the next current block), or the video encoding method according to an exemplary embodiment of the present disclosure may not be performed for the next current block, but rather the encoding mode determined by the video encoding method according to an exemplary embodiment of the present disclosure may be referenced instead.

[0076] As mentioned above, the mode traversal order of the relevant encoder for video blocks (such as CU blocks corresponding to CUs) is fixed. However, in actual encoding and decoding scenarios, for example, assuming that the optimal encoding mode for a current block is the intra-frame mode for that current block and the fixed mode traversal order is skip mode—merge mode—inter-frame mode—intra-frame mode, then for that current block, all previous modes other than the intra-frame mode must be executed before the intra-frame mode is executed. In this case, the encoding process of executing other modes other than the intra-frame mode in the previous step wastes encoding and decoding complexity, and this inflexible mode traversal order also increases decision complexity, making it difficult to quickly determine the optimal encoding mode. To address this, this disclosure proposes to construct a more flexible encoding mode selection process by using a mode pool to determine the mode queue.

[0077] Figure 4 This is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure. Figure 5 and Figure 6 This is an exemplary diagram illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0078] Reference Figure 4 In step S410, a pattern pool is constructed for the current block.

[0079] The pattern pool according to exemplary embodiments of this disclosure can be understood as a multi-unit pattern pool comprising multiple units (or patterns), each pattern in the pattern pool coexisting and may be associated with each other depending on the use of different algorithms, or may be independent of each other through decoupling. That is, related algorithms can be used to associate at least one pattern in the pattern pool. However, this disclosure does not impose any limitations on the algorithms used in the exemplary embodiments, and any algorithm in the relevant art is applicable to this disclosure. In this disclosure, "independent" patterns may mean that a previously executed pattern has no effect on a subsequently executed pattern, and "associated" patterns may mean that the execution result of a previously executed pattern can affect a subsequently executed pattern.

[0080] As an exemplary embodiment, the pattern pool for the current block may include multiple current block encoding patterns, wherein each of the multiple current block encoding patterns is associated with the encoding of the current block. As an exemplary embodiment, in addition to including multiple current block encoding patterns, the pattern pool for the current block may also include sub-block encoding patterns, wherein each sub-block encoding pattern is associated with the encoding of each of the multiple sub-blocks divided from the current block. That is, the pattern pool may include only multiple current block encoding patterns, or it may include multiple current block encoding patterns and sub-block encoding patterns. The patterns in the pattern pool are described in detail below.

[0081] As an exemplary embodiment, the current block encoding mode may include an encoding mode directly used to perform encoding processing on the current block, an encoding sub-mode split from the encoding mode used to perform encoding processing on the current block, and / or any other mode or processing related to the encoding of the current block, and this disclosure is not limited thereto.

[0082] As an example, the encoding mode directly used to perform encoding processing on the current block can be a feasible predictive encoding mode as described above, such as intra-frame predictive encoding mode (also referred to as "intra-frame mode" or "intra-frame prediction mode" in this disclosure), inter-frame predictive encoding mode (also referred to as "inter-frame mode" or "inter-frame prediction mode" in this disclosure), merge mode and / or skip mode, etc.

[0083] It should be understood that feasible predictive coding modes may include, but are not limited to, those shown above. For example, they may also include any coding mode such as Advanced Motion Vector Prediction (AMVP), Inter-Frame / Intra-Frame Combined Prediction, Affine Inter-Frame, Current Frame Reference, etc. In the following description, for ease of description, feasible predictive coding modes including intra-frame, inter-frame, merge, and skip modes are used as examples only. It should be understood that this disclosure is not limited thereto.

[0084] As an example, the coding sub-patterns split from the coding pattern may include multiple coding sub-patterns split from each coding pattern in the feasible predictive coding pattern, which will be described in detail later below.

[0085] In an exemplary embodiment of this disclosure, in order to obtain the optimal coding mode best suited for the region corresponding to the current block within the video frame, it is possible to consider traversing the coding modes of the sub-blocks recursively divided from the current block. Accordingly, when determining the optimal coding mode for the current block, the determined result can be the optimal coding mode for the current block or the optimal coding mode for multiple sub-blocks recursively divided from the current block.

[0086] Considering this possible outcome, this disclosure proposes to treat the process of recursively partitioning the current block and determining the encoding patterns of the recursively partitioned sub-blocks as a selectable mode, and to place this selectable mode in a mode pool for mode selection. That is, in this disclosure, the sub-block encoding mode can represent the mode of partitioning the current block to obtain sub-blocks and determining the encoding patterns of the sub-blocks, which will be described in detail later.

[0087] As an example of a pattern pool, refer to Figure 5 For the current block, a 5-unit mode pool can be constructed, comprising 4 current block coding modes (e.g., 4 coding modes such as intra-frame mode, inter-frame mode, merge mode, and skip mode) and 1 sub-block coding mode. For example, refer to... Figure 6For the current block, a 4-unit mode pool can be constructed, consisting of only the four current block coding modes (e.g., four coding modes such as intra-frame mode, inter-frame mode, merge mode, and skip mode). For example, refer to... Figure 8 For the current block, a 9-unit pattern pool can be constructed, which includes 8 current block encoding patterns (e.g., multiple encoding sub-patterns, which will be described later) and 1 sub-block encoding pattern.

[0088] In this way, exemplary embodiments of this disclosure can treat multiple current block encoding patterns and / or sub-block encoding processes equally, unitizing (or modularizing) them into patterns in a pattern pool, thereby constructing a pattern pool for the current block. By constructing the pattern pool, the execution of individual encoding patterns and sub-block encoding patterns does not have a fixed traversal (or selection) order, which improves the flexibility of the pattern selection process in the encoding process and can be combined with algorithms to provide the advantage of significantly reducing the decision complexity regarding the optimal pattern.

[0089] In step S420, based on the pattern pool, a pattern queue including at least one pattern is determined for the current block.

[0090] As an exemplary embodiment, step S420 may include: selecting at least one mode from the mode pool and determining the order of at least one mode in the mode queue by referring to the encoding information of neighboring blocks of the current block, the encoding information of the parent or child blocks of the current block, and / or the pre-analyzed encoding information of the video frame in which the current block is located. That is, the mode is selected based on the encoding reference information that can be used by the current block, and the priority of the modes that perform these selections is determined.

[0091] As an example, a pattern queue can be constructed by using an algorithm to determine at least one pattern and its order by referring to the encoding reference information associated with the current block. This disclosure does not limit the algorithm or method used to construct the pattern queue, as long as the algorithm or method used can achieve the corresponding function.

[0092] As an example, at least one mode in the mode queue and its order (or priority) can be determined by using an algorithm to refer to the encoding information (such as encoding modes) of one or more neighboring blocks (e.g., already encoded) within the video frame in which the current block to be encoded is located.

[0093] As an example, at least one mode and its order (or priority) in the mode queue can be determined by using an algorithm to reference the encoding information (such as encoding modes) of one or more parent or child blocks (e.g., already encoded) of the current block to be encoded. Here, the parent block of the current block can be understood as the upper-level block or superior block of the current block, i.e., the video block from which the current block can be divided. Furthermore, the child blocks of the current block can be understood as the lower-level blocks or subordinate blocks of the current block, i.e., video blocks that can be divided from the current block.

[0094] As an example, at least one mode and its order (or priority) in the mode queue can be determined by using the preprocessing results (i.e., pre-analyzed encoding information, etc.) of the current video frame or image containing the current block to be encoded. For example, look-ahead processing can be used for pre-analysis to obtain the preprocessing results. Here, this disclosure does not limit the manner in which preprocessing is performed.

[0095] In other words, an algorithm can adaptively adjust at least one candidate mode as the optimal coding mode based on the encoding / decoding scenario (e.g., sequence characteristics, sequence complexity, the stillness or motion state of video frames, and the magnitude of their motion) by referencing the encoding information (such as encoding modes) of blocks associated with the current block (e.g., already encoded blocks) and / or pre-analyzed encoding information, to construct a mode queue suitable for selecting the optimal coding mode. In this way, by adaptively adjusting the mode queue, the encoder can pre-execute modes with a higher probability of being selected as the optimal mode compared to other modes, thereby accelerating the decision-making process for the optimal coding mode.

[0096] As an exemplary embodiment, when the pattern pool includes multiple current block encoding patterns and sub-block encoding patterns, the selected at least one pattern may accordingly include at least one current block encoding pattern and / or sub-block encoding pattern. That is, the pattern queue in this case may contain only at least one current block encoding pattern or sub-block encoding pattern, or it may contain at least one current block encoding pattern and sub-block encoding pattern. This embodiment will be described in detail below.

[0097] Reference Figure 5 When the mode pool for the current block includes four coding modes (such as intra-block mode, inter-block mode, merge mode, and skip mode) and one sub-block coding mode, an algorithm can be used to determine a mode queue containing five modes from the five-unit mode pool based on the aforementioned coding reference information: intra-block mode—sub-block coding mode—skip mode—merge mode—inter-block mode. Here, although... Figure 5The mode queue shown is comprised of five modes in the order described above. However, this disclosure does not limit the number, category, or order (or priority) of modes in the queue. The mode queue determined using relevant algorithms may include any number of modes, any mode category, and any mode order. For example, the mode queue determined from the mode pool may also be: sub-block coding mode—skip mode—merge mode—intra-frame mode.

[0098] By constructing a pattern pool that includes the encoding mode of the current block and the encoding modes of sub-blocks, the flexibility of the encoding architecture can be greatly improved. For example, for a 5-unit pattern pool, up to 325 pattern queues can be determined. Compared with the fixed pattern selection order in related technologies, this significantly increases the variety of pattern queues, and the applicable pattern queues can be arbitrarily determined by related algorithms.

[0099] Furthermore, in the process of video encoding and decoding, different encoding and decoding scenarios may require different encoding partitioning orders for traversing each layer of blocks. For example, complex encoding and decoding scenarios (such as encoding and decoding scenarios with complex motion or a lot of texture) may be suitable for executing the encoding mode in the order of traversing small blocks to large blocks, while simple encoding and decoding scenarios (such as encoding and decoding scenarios with static or less texture) may be suitable for executing the encoding mode in the order of traversing large blocks to small blocks. However, the single fixed encoding partitioning order in related technologies may lead to reduced encoding and decoding efficiency or wasted encoding and decoding computational complexity in different encoding and decoding scenarios. Therefore, this disclosure proposes a method for constructing a bidirectional encoding partitioning order mode selection process.

[0100] As an example, a suitable block traversal direction (i.e., encoding partitioning order) for the encoding / decoding scenario can be determined using an algorithm (e.g., considering the complexity of the encoding / decoding scenario described above), and the positions of the sub-block encoding modes can be set based on the determined block traversal direction. Here, this disclosure does not specifically limit the method of determining the encoding partitioning order; the encoding partitioning order suitable for the current encoding / decoding scenario can be determined according to any relevant technology or algorithm, for example, as described above, determining the suitable encoding partitioning order based on the scenario complexity.

[0101] In this disclosure, to enable the setting of encoding partitioning order that can change according to the encoding / decoding scenario, a sub-block encoding pattern related to block partitioning exists in the pattern queue, so as to realize the partitioning of blocks at different levels and the determination of encoding patterns. In this regard, based on the above-mentioned concept of the pattern pool, this disclosure further proposes the concept of selecting the current block encoding pattern only from the pattern pool when the sub-block encoding pattern is used in a fixed manner.

[0102] As an exemplary embodiment, when the pattern pool for the current block includes only multiple current block encoding patterns, the selected at least one pattern may include one or more current block encoding patterns but exclude sub-block encoding patterns. The pattern queue may traverse the encoding patterns of blocks at different levels, including sub-block encoding patterns according to the encoding partitioning order. That is, different priorities for determining the encoding pattern of the current block and determining the encoding pattern of sub-blocks may be considered, and sub-block encoding patterns may be directly placed in the pattern queue instead of selecting them from the pattern pool. This will be described in detail below with reference to the accompanying drawings.

[0103] Reference Figure 6 When traversing the encoding modes of blocks at different levels, it can be determined that there are sub-block encoding modes that are fixedly related to the block division in the mode queue. One or more current block encoding modes determined from the 4-unit mode pool (including intra-frame mode, inter-frame mode, merge mode and skip mode) can exist before or after the fixedly existing sub-block encoding modes, or before and after.

[0104] As an exemplary embodiment, the position of the sub-block encoding pattern in the pattern queue is related to the encoding partitioning order. That is, the order of the encoding partitioning order is taken into account to determine the order between the sub-block encoding pattern and one or more current block encoding patterns determined from the pattern pool.

[0105] As an exemplary embodiment, the encoding partitioning order may include a bottom-up encoding architecture block-to-block order, a top-down encoding architecture block-to-small block order, and / or a parent-child-parent encoding architecture block-to-small-block-to-large block order.

[0106] As an example, the small-block-large-block order can represent the order in which such encoding patterns are traversed: first, small blocks are encoded by traversing patterns that can be used by small blocks (or child blocks, lower-level blocks, or lower-level blocks), and then large blocks are encoded by traversing patterns that can be used by large blocks (or parent blocks, upper-level blocks, or upper-level blocks).

[0107] As an example, the block-to-block order can represent the order in which such encoding patterns are traversed: first, large blocks are encoded by traversing the patterns that can be used by large blocks, and then small blocks are encoded by traversing the patterns that can be used by small blocks.

[0108] As an example, the block-small-block-block sequence can represent the order in which such encoding patterns are traversed: first, the block is encoded by traversing a subset of the patterns that can be used by the block, then the small block is encoded by traversing the patterns that can be used by the small block, and then the block is encoded by traversing another subset of the patterns that can be used by the block.

[0109] In this disclosure, for the current block and the sub-blocks partitioned from the current block, the current block can be considered a large block and the sub-blocks can be considered small blocks, and similarly, for the sub-blocks and the further recursively partitioned sub-blocks partitioned from the sub-blocks, the sub-blocks can be considered large blocks and the further recursively partitioned sub-blocks can be considered small blocks.

[0110] As an exemplary embodiment, when the encoding division order is small block-large block order, the sub-block encoding pattern is located at the first position in the pattern queue, that is, referring to... Figure 6 If there is no (or no selected) current block encoding mode before the sub-block encoding mode, the encoding modes of different layers of blocks can be traversed in the order of small blocks to large blocks.

[0111] As another exemplary embodiment, when the encoding division order is block-to-block, the sub-block encoding pattern is located at the last position in the pattern queue, i.e., referring to... Figure 6 If there is no (or no selected) current block encoding pattern after the sub-block encoding pattern, the encoding patterns of blocks at different levels can be traversed in the order of large block to small block.

[0112] As another exemplary embodiment, when the encoding division order is large block-small block-large block, the sub-block encoding pattern is located in the middle position in the pattern queue, that is, referring to... Figure 6 If there are (or one) or more current block encoding patterns after and after the sub-block encoding pattern, the encoding patterns of blocks at different levels can be traversed in the order of large block-small block-large block.

[0113] Furthermore, despite Figure 6 The example shown is a pattern pool containing only four encoding patterns, but this pattern pool is merely an example, and various pattern pools according to the exemplary embodiments described above and below according to this disclosure may be combined herein.

[0114] By constructing this bidirectional dynamic encoding partitioning order, the encoder can adaptively configure the encoding partitioning order according to the needs of the encoding and decoding scenario. It can traverse from small blocks to large blocks, or from large blocks to small blocks, or it can perform a partial traversal of large blocks, perform a traversal of small blocks, and then return to continue performing a traversal of large blocks. This allows for a faster decision on the optimal block size or block partitioning and encoding strategy based on the actual application scenario. Furthermore, the space and upper limit of algorithms that can be added in this dynamic bidirectional encoding partitioning order are also greater.

[0115] Furthermore, as mentioned above, when executing each encoding mode sequentially, the encoder often needs to perform a complete encoding process including prediction, transformation, quantization, and entropy coding. Even if it is discovered midway through the execution of this process that the currently executing encoding mode is not optimal due to poor performance, it is impossible to exit the execution process of that encoding mode in advance, leading to a waste of encoding and decoding complexity. To address this, this disclosure also proposes a method to reduce the execution of unnecessary subsequent processing in encoding modes by utilizing intermediate prediction results from the encoding process.

[0116] As an exemplary embodiment, a feasible predictive coding mode (e.g., intra-frame mode, inter-frame mode, merge mode and / or skip mode, or other coding modes as described above) can be split into multiple coding sub-modes. Due to the characteristics of the coding process, the multiple split coding sub-modes may be related to each other.

[0117] As an exemplary embodiment, the multiple current block coding modes include multiple prediction modes (hereinafter referred to as "PRED" or "first coding sub-mode") and multiple rate-distortion optimization modes (hereinafter referred to as "RDO" or "second coding sub-mode"), each separately split from multiple coding modes. That is, each coding mode can be split into two coding sub-modes—PRED and RDO—and the input to the execution of RDO can depend on the result of the execution of PRED. For example, the result of the execution of PRED can be the input to the execution of RDO, such as by calculating the difference between the prediction result and the original image as a residual to perform subsequent transform processing, quantization processing, and entropy coding processing. Accordingly, a PRED for a coding mode can exist independently in the mode queue, but the corresponding RDO will not exist independently in the mode queue. Furthermore, since RDOs depend on the execution results of the corresponding PREDs, if a PRED and its corresponding RDO exist in the pattern queue, the RDO is located in a subsequent position of the PRED in the pattern queue, for example, in a position adjacent to the PRED after the PRED, or in a position one or more patterns away from the PRED after the PRED.

[0118] As an exemplary embodiment, PRED (i.e., the first coding sub-mode) is used to perform prediction processing in the coding process of the corresponding coding mode, and RDO (i.e., the second coding sub-mode) is used to perform transform processing, quantization processing, and entropy coding processing in the coding process of the corresponding coding mode. Referring below... Figure 7 This will be described in detail.

[0119] Figure 7 This is an exemplary diagram illustrating encoding modes and encoding sub-modes according to exemplary embodiments of the present disclosure. (Refer to...) Figure 7 It can split an encoding pattern into two encoding sub-patterns.

[0120] As an example, after performing prediction processing, PRED can output a distortion result (i.e., distortion amount) representing the amount of distortion between the predicted and original values. This distortion result can represent an intermediate result used in the coding process to measure coding performance.

[0121] As an example, after performing subsequent transform, quantization, and entropy coding processes, RDO can output a rate-distortion (optimized) result (i.e., rate-distortion cost RDcost) that represents the amount of distortion considering the bit rate. This rate-distortion optimization result (also referred to as the rate-distortion result) represents the final result used in the coding process to measure the coding mode.

[0122] As an example, the quality of a coding pattern can be measured based on multiple distortion results and / or rate-distortion (optimization) results of multiple coding sub-patterns. For example, the optimal coding pattern may be the coding pattern with the minimum distortion result and / or rate-distortion optimization result.

[0123] The following reference Figure 8 and Figure 9 Based on Figure 7 The video coding method of the coding sub-mode is described in detail.

[0124] Figure 8 and Figure 9 This is an exemplary diagram illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0125] Reference Figure 8 When multiple coding modes are split into two coding sub-modes respectively, a 9-unit mode pool can be constructed for the current block (e.g., CU block) including 8 coding sub-modes (such as intra-frame PRED, intra-frame RDO, inter-frame PRED, inter-frame RDO, merged PRED, merged RDO, skipped PRED and skipped RDO) and sub-block coding modes.

[0126] Similar to the operation of a mode pool including coding modes, a mode queue comprising six modes, "intra-frame PRED—sub-block coding mode—skip PRED—skip RDO—merge PRED—merge RDO," can be determined from the 9-unit mode pool using an algorithm based on coding reference information. Here, although the mode queue is shown as including six modes in the above-described order, this disclosure does not limit the number, category, or order (or priority) of modes in the queue. The mode queue determined using the relevant algorithm can include any number of modes, any mode category, and any mode order. For example, the mode queue determined from the mode pool could also be: intra-frame PRED—inter-frame PRED—inter-frame RDO—intra-frame RDO—skip PRED.

[0127] In addition, refer to Figure 9 When the pattern pool for the current block only includes multiple encoded sub-patterns for the current block, as mentioned above... Figure 6 Similarly, considering the traversal of encoding patterns of blocks at different levels, the pattern queue can have fixed sub-block encoding patterns, and one or more encoding sub-patterns determined from the 9-unit pattern pool can exist before or after the fixed sub-block encoding patterns, or before and after, thereby realizing a bidirectional encoding partitioning order that varies according to the encoding and decoding scenario.

[0128] By constructing a pattern pool that includes the encoding sub-patterns of the current block and the encoding patterns of sub-blocks, the flexibility of the encoding architecture can be greatly improved. For example, for a 9-unit pattern pool, up to 65,137 pattern queues can be determined, significantly increasing the variety of pattern queues compared to a fixed pattern selection order in related technologies. Furthermore, compared to a pattern pool that includes encoding sub-patterns, the increased capacity of the pattern pool also enhances the possibility of applying algorithms for decision-making. Moreover, the use of encoding sub-patterns reduces encoding complexity, which will be described later.

[0129] Once the pattern queue is determined in step S420, the optimal encoding pattern can be determined based on the determined pattern queue. Return Figure 4 In step S430, at least one mode is executed in the order of the modes in the mode queue to determine the encoding mode for the current block or multiple sub-blocks divided from the current block.

[0130] Specifically, when a pattern queue including at least one pattern is determined, at least one pattern in the ordered pattern queue can be executed one by one in sequence to obtain at least one execution result corresponding to at least one pattern, and the optimal encoding pattern can be determined based on the at least one execution result obtained.

[0131] As an example, the optimal encoding mode can be selected from the at least one modes based on at least one execution result corresponding to at least one mode.

[0132] For example, if the mode with the lowest rate distortion cost is the current block coding mode (e.g., the intra-frame mode or intra-frame RDO with the lowest RDcost compared to other modes), then the coding mode corresponding to that current block coding mode (i.e., the intra-frame mode) is determined as the coding mode for the current block.

[0133] For example, if the mode with the lowest rate-distortion cost is a sub-block encoding mode (e.g., the sum of the RDcosts of multiple sub-blocks divided from the current block is less than the RDcost of other encoding modes or the RDcost of other RDOs), then the encoding modes of the multiple sub-blocks corresponding to this sub-block encoding mode are determined as the encoding modes for the multiple sub-blocks. That is, in this case, the encoding mode for the current block is not determined, but the encoding mode for the sub-blocks divided from the current block is determined. In this case, the determined encoding modes for the multiple sub-blocks may also indicate the division method of the sub-blocks from the current block (such as division direction, division size, and / or division shape, etc.), and here, this disclosure does not impose specific limitations on the division method.

[0134] Furthermore, this disclosure proposes to terminate or skip the execution of some subsequent modes early if the execution results of previous modes are obtained. In other words, before the execution of all modes in the mode queue is completed, some modes in the mode queue can be adaptively and selectively skipped (or pruned) to speed up the mode decision-making process.

[0135] As an exemplary embodiment, the step of determining the encoding pattern for the current block or a plurality of sub-blocks divided from the current block by executing at least one pattern in the pattern queue according to the pattern order includes: skipping the execution of at least one subsequent pattern in the pattern queue based on the execution result of the previous pattern in the pattern queue. This will be described in detail below.

[0136] As an exemplary embodiment, when the previous mode is an encoding mode, the execution of at least one subsequent mode in the mode queue is skipped based on the rate-distortion optimization result and / or distortion result obtained by performing encoding processing using the encoding mode. For example, it is determined to skip the execution of at least one subsequent mode in the mode queue based on the rate-distortion optimization result obtained by performing encoding processing of the previously executed encoding mode satisfying (e.g., less than or equal to) a threshold of the algorithm. However, this is only an example, and the ways in which the present disclosure skips some modes are not limited thereto.

[0137] As an example, the coding performance of each coding mode can be measured using multiple rate-distortion optimization results, which are the results of execution of multiple coding modes. The smaller the RDcost of a coding mode, the better the coding mode performs for the current block. As an example, the coding performance of each coding mode can also be measured using multiple distortion results determined based on the execution results of multiple coding modes. For example, refer to... Figure 5 Based on the execution result of the intra-frame mode in the mode queue, one can directly jump to the skipped mode in the same mode queue. Here, the skipping or pruning method based on the execution result of the encoding mode is merely an example, and this disclosure does not impose any limitations on it.

[0138] As an exemplary embodiment, when the previous pattern is the first coded sub-pattern, the execution of the corresponding second coded sub-pattern in the pattern queue is skipped based on the distortion result obtained by performing prediction processing using the first coded sub-pattern.

[0139] As an example, multiple distortion results, which are intermediate results of multiple encoding modes, can be used to measure each encoding mode. For example, the larger the DISTORTION of an encoding mode, the worse the encoding mode performs for the current block.

[0140] Because the rate-distortion optimization (RDD) results of an encoding pattern are highly correlated with its distortion results, when the discount of a particular encoding pattern is significantly larger than other patterns, the likelihood of that pattern being determined as the optimal encoding pattern for the current block is relatively low, and the corresponding RDO can be skipped. In other words, since the purpose of encoding pattern decision-making is to select the encoding pattern with the best encoding performance, once it is determined from the execution results of a pattern's RRED that the pattern does not have an advantage over other patterns, the execution of subsequent RDOs for that pattern can be skipped to reduce unnecessary processing. That is, multiple distortion results determined by multiple PREDs can be considered as selection reference factors, used to select some RDOs to be executed from the multiple RDOs corresponding to multiple PREDs, thereby eliminating other unnecessary RDOs.

[0141] For example, refer to Figure 8 If the execution result of skip PRED for skipped modes in the pattern queue determines that the skipped mode has poor encoding performance for the current block compared to other modes, then skipping RDO can be skipped, and the merge PRED in the pattern queue can be skipped directly. Here, the skipping or pruning method based on the execution result of the encoding sub-mode is only an example, and this disclosure does not limit it.

[0142] As an exemplary embodiment, when the previous mode is a sub-block encoding mode, the execution of at least one subsequent mode in the mode queue is skipped based on the sum of rate-distortion optimization results and / or the sum of distortion results of multiple sub-blocks obtained by performing encoding processing using the sub-block encoding mode. As described above, the sum of rate-distortion optimization results and / or the sum of distortion results of multiple sub-blocks in a multi-sub-block encoding mode can be compared with the rate-distortion optimization results and / or distortion results of other current block encoding modes to measure the encoding performance of the sub-block encoding mode. Here, the sum of rate-distortion optimization results of multiple sub-blocks can be understood as: the sum of rate-distortion optimization results of multiple sub-blocks is regarded as the rate-distortion optimization result of the block corresponding to the multiple sub-blocks used to divide the multiple sub-blocks. Therefore, the way to skip the execution of at least one subsequent mode based on the sum of rate-distortion optimization results of multiple sub-blocks is similar to the way to skip the execution of at least one subsequent mode based on the rate-distortion optimization result of the current block, and will not be described again. Similarly, the sum of distortion results based on multiple sub-blocks can be understood as the corresponding distortion result of the block used to divide the multiple sub-blocks. Therefore, the execution method of skipping at least one subsequent mode based on the sum of distortion results of multiple sub-blocks is similar to the execution method of skipping at least one subsequent mode based on the distortion result of the current block, and will not be described again.

[0143] Furthermore, the execution results of previous patterns can also influence the skipping or pruning of subsequent patterns in other appropriate ways, and this disclosure does not limit this. Moreover, this disclosure does not impose excessive restrictions on the specific details of skipping certain patterns based on the execution results of previous patterns; algorithms in related fields can be used to adaptively skip certain patterns, which will not be described in detail here.

[0144] Alternatively, the video encoding method according to an exemplary embodiment of this disclosure may also not skip any mode, but instead execute each mode one by one according to the mode queue to determine the encoding mode for the current block or multiple sub-blocks.

[0145] By skipping unnecessary execution patterns in the pattern queue, the complexity of the encoding process can be reduced while ensuring encoding quality, and the upper limit of the algorithm that can be implemented can also be increased accordingly.

[0146] Furthermore, as described above, in an exemplary embodiment of this disclosure, the sub-block encoding pattern includes determining the encoding patterns of multiple sub-blocks divided from the current block.

[0147] As an exemplary embodiment, the step of executing a sub-block encoding mode includes: dividing the current block to obtain multiple sub-blocks; and for each of the multiple sub-blocks, determining the encoding mode of the corresponding sub-block by constructing a mode pool and determining a mode queue.

[0148] In other words, for each of the multiple sub-blocks divided from the current block, the steps corresponding to steps S410 to S430 can be executed respectively, namely, constructing a pattern pool for the current sub-block, determining a pattern queue for the current sub-block including at least one pattern based on the pattern pool for the current sub-block, executing at least one pattern in the order of the patterns in the pattern queue, and determining the encoding pattern for the current sub-block or multiple further sub-blocks further divided from the current sub-block.

[0149] As an example, each of the multiple sub-blocks can be regarded as the current block and an operation similar to that described above with reference to the accompanying drawings can be performed. The encoding mode for each sub-block or the encoding mode for sub-blocks further subdivided from the corresponding sub-block can be determined sequentially by repeating the above operation for each sub-block.

[0150] For example, when the current block is a 64×64 block, in the sub-block encoding mode, the current block can first be divided into four 32×32 sub-blocks according to the four partitions. For each 32×32 sub-block, the steps corresponding to steps S410 to S430 are executed to determine the respective modes for each sub-block. For example, the intra-frame mode for the first and second sub-blocks, the encoding mode for the further division of the third sub-block into multiple 16×16 sub-blocks, and the inter-frame mode for the fourth sub-block are determined.

[0151] Furthermore, when determining the encoding mode for the current block or multiple sub-blocks divided from the current block in step S43, the execution result under the sub-block encoding mode may include the sum of the rate-distortion optimization results of the encoding modes of multiple sub-blocks, that is, the sum of the RDcost of the intra-frame mode for the first sub-block, the RDcost of the intra-frame mode for the second sub-block, the sum of the RDcost of the encoding modes of multiple sub-blocks of size 16×16 further divided for the third sub-block, and the sum of the RDcost of the inter-frame mode for the fourth sub-block.

[0152] The details of the operation for determining the encoding mode of the current block in the exemplary embodiments of this disclosure are applicable to the operation for determining the encoding modes of various blocks including sub-blocks, and therefore will not be described in detail here.

[0153] In this disclosure, the aforementioned series of operations regarding encoding pattern determination can be performed on blocks that are recursively divided layer by layer. As an exemplary embodiment, the pattern pools and / or pattern queues for different sub-blocks may be the same or different. That is, different pattern pools may be used for different sub-blocks, and different pattern queues may be determined for different sub-blocks.

[0154] In this way, the determination of the encoding mode of the sub-block has great flexibility and room for algorithm addition, similar to the determination of the encoding mode of the current block. Therefore, the flexibility of the encoding architecture of the embodiments of this disclosure can be further greatly improved.

[0155] As described above, once the applicable encoding mode is determined in step S430, encoding can be performed using the determined encoding mode. Return Figure 4 In step S440, the current block or multiple sub-blocks are encoded based on the determined encoding pattern. Here, this disclosure will not describe in detail the specific operations of performing encoding using the encoding pattern.

[0156] For video encoding and decoding, the encoding architecture is the foundation of the entire encoder. The video encoding method according to the exemplary embodiments of this disclosure constructs a highly adaptive video encoding architecture that differs from traditional video encoding architectures, which can greatly improve the flexibility of the encoder in the video encoding process and increase the upper limit of the algorithm.

[0157] The mode pool construction, mode queue determination, and / or encoding mode determination of the video coding method according to the exemplary embodiments of the present disclosure are not fixed and can be adaptively adjusted according to the sequence characteristics and the algorithm. This flexibility enables the video coding method according to the exemplary embodiments of the present disclosure to have excellent performance when facing various types of video sequences.

[0158] The video coding method according to exemplary embodiments of this disclosure is applicable to or can be migrated to various types of encoders and / or hybrid coding framework encoders, without limiting the coding standard, and has high versatility.

[0159] The video encoding method according to exemplary embodiments of this disclosure can significantly improve encoding speed while ensuring encoding quality, and has relatively high encoding efficiency.

[0160] Figure 10 A structural block diagram of a video encoding apparatus 1000 according to an exemplary embodiment of the present disclosure is shown.

[0161] like Figure 10 As shown, a video encoding apparatus 1000 according to an exemplary embodiment of the present disclosure may include: a mode pool construction unit 1010, a queue determination unit 1020, a mode determination unit 1030, and an encoding unit 1040.

[0162] Specifically, the pattern pool construction unit 1010 is configured to construct a pattern pool for the current block, wherein the pattern pool includes multiple current block encoding patterns or multiple current block encoding patterns and sub-block encoding patterns. The queue determination unit 1020 is configured to determine a pattern queue for the current block, including at least one pattern, based on the pattern pool. The pattern determination unit 1030 is configured to execute at least one pattern according to the pattern order in the pattern queue to determine the encoding pattern for the current block or multiple sub-blocks divided from the current block. The encoding unit 1040 is configured to encode the current block or multiple sub-blocks based on the determined encoding patterns. The multiple current block encoding patterns are related to the encoding of the current block, and the sub-block encoding patterns are related to the encoding of multiple sub-blocks.

[0163] As an exemplary embodiment, the pattern determination unit 1030 is configured to perform sub-block encoding patterns by: dividing the current block to obtain multiple sub-blocks; and for each of the multiple sub-blocks, determining the encoding pattern of the corresponding sub-block by constructing a pattern pool and determining a pattern queue, wherein the pattern pools and / or pattern queues of different sub-blocks are the same or different.

[0164] As an exemplary embodiment, when the pattern pool includes only multiple current block encoding patterns, the pattern queue includes sub-block encoding patterns according to the encoding partitioning order, wherein the position of the sub-block encoding pattern in the pattern queue is related to the encoding partitioning order.

[0165] As an exemplary embodiment, the encoding partitioning order includes small block-large block order, large block-small block order, and / or large block-small block-large block order, wherein when the encoding partitioning order is small block-large block order, the sub-block encoding pattern is located at the first position in the pattern queue; when the encoding partitioning order is large block-small block order, the sub-block encoding pattern is located at the last position in the pattern queue; and / or when the encoding partitioning order is large block-small block-large block order, the sub-block encoding pattern is located at the middle position in the pattern queue.

[0166] As an exemplary embodiment, the queue determination unit 1020 is configured to determine a mode queue including at least one mode for the current block based on the mode pool by: selecting at least one mode from the mode pool and determining the order of at least one mode in the mode queue by referring to the encoding information of the neighboring blocks of the current block, the encoding information of the parent or child blocks of the current block and / or the pre-analyzed encoding information of the video frame in which the current block is located.

[0167] As an exemplary embodiment, the multiple current block encoding modes include multiple encoding modes for performing encoding processing on the current block, or multiple first encoding sub-modes and multiple second encoding sub-modes respectively split from the multiple encoding modes, wherein the first encoding sub-modes are used to perform prediction processing in the encoding processing of the corresponding encoding mode, and the second encoding sub-modes are used to perform transform processing, quantization processing and entropy encoding processing in the encoding processing of the corresponding encoding mode.

[0168] As an exemplary embodiment, the pattern determination unit 1030 is configured to execute at least one pattern in the pattern queue according to the pattern order in the pattern queue by the following operation: determining the encoding pattern for the current block or multiple sub-blocks divided from the current block by skipping the execution of at least one subsequent pattern in the pattern queue based on the execution result of the previous pattern in the pattern queue.

[0169] As an exemplary embodiment, the pattern determination unit 1030 is configured to skip the execution of at least one subsequent pattern in the pattern queue based on the execution result of a previous pattern in the pattern queue by: when the previous pattern is an encoding pattern, skipping the execution of at least one subsequent pattern in the pattern queue based on the rate-distortion optimization result and / or distortion result obtained by performing encoding processing using the encoding pattern; when the previous pattern is a first encoding sub-pattern, skipping the execution of the corresponding second encoding sub-pattern in the pattern queue based on the distortion result obtained by performing prediction processing using the first encoding sub-pattern; and / or when the previous pattern is a sub-block encoding pattern, skipping the execution of at least one subsequent pattern in the pattern queue based on the sum of the rate-distortion optimization results and / or the sum of the distortion results of multiple sub-blocks obtained by performing encoding processing using the sub-block encoding pattern.

[0170] Regarding the video encoding device 1000 in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0171] Furthermore, it should be understood that the various units in the video encoding apparatus 1000 according to exemplary embodiments of this disclosure may be implemented as hardware components and / or software components. Those skilled in the art, based on the processes performed by the defined various units, may implement the various units, for example, using a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0172] Furthermore, this disclosure also provides an electronic device. The electronic device includes at least one memory and at least one processor, wherein the at least one memory stores a set of computer-executable instructions, which, when executed by the at least one processor, perform the video encoding method as described in the exemplary embodiments above.

[0173] As an example, an electronic device can be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, the electronic device is not necessarily a single device; it can be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. The electronic device can also be part of an integrated control system or system manager, or can be configured to interconnect locally or remotely (e.g., via wireless transmission) through an interface.

[0174] In electronic devices, processors may include central processing units (CPUs), graphics processing units (GPUs), programmable logic devices, dedicated processor systems, microcontrollers, or microprocessors. By way of example and not limitation, processors may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0175] The processor can execute instructions or code stored in memory, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transport protocol.

[0176] Memory can be integrated with the processor; for example, RAM or flash memory can be housed within an integrated circuit microprocessor. Alternatively, memory can comprise a separate device, such as an external disk drive, storage array, or other storage device that can be used by any database system. Memory and processor can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor to read files stored in the memory.

[0177] In addition, electronic devices may include multimedia resource displays (such as liquid crystal displays) and user interaction interfaces (such as keyboards, mice, touch input devices, etc.). All components of the electronic device can be interconnected via buses and / or networks.

[0178] According to exemplary embodiments of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one processor, they cause at least one processor to perform the video encoding method as described in the exemplary embodiments above.

[0179] Examples of computer-readable storage media herein include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0180] Figure 11 A computing environment 1110 coupled to a user interface 1150 is shown. The computing environment 1110 may be part of a data processing server. The computing environment 1110 includes a processor 1120, memory 1130, and input / output (I / O) interface 1140.

[0181] Processor 1120 typically controls the overall operation of computing environment 1110, such as operations associated with display, data acquisition, data communication, and image processing. Processor 1120 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 1120 may include one or more modules that facilitate interaction between processor 1120 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0182] Memory 1130 is configured to store various types of data to support the operation of computing environment 1110. Memory 1130 may include predefined software 1132. Examples of such data include instructions for any application or method operating on computing environment 1110, video datasets, image data, etc. Memory 1130 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0183] I / O interface 1140 provides an interface between processor 1120 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 1140 can be coupled to encoders and decoders.

[0184] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a plurality of programs in memory 1130 and / or a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by processor 1120 in computing environment 1110 to perform the above-described methods. In one example, the plurality of programs can be executed by processor 1120 in computing environment 1110 to (e.g., from...) Figure 2 The video encoder 20 in the computing environment 1110 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1120 in the computing environment 1110 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 1120 in the computing environment 1110 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 1120 in the computing environment 1110 to (e.g., to...) Figure 3 The video decoder 30 in the middle sends the bitstream or data stream. Alternatively, a non-transitory computer-readable storage medium may store data generated by the encoder (e.g., Figure 2 The video encoder 20 in the video is generated using, for example, the encoding method described above, for use by the decoder (e.g., Figure 3The video decoder 30 in the video decoder uses a bitstream or data stream that includes encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) when decoding video data. Non-transitory computer-readable storage media can be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage devices, etc.

[0185] In one embodiment, a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method is provided. In another embodiment, a bitstream comprising encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method is provided.

[0186] In one embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 1120); and a non-transitory computer-readable storage medium or memory 1130 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0187] In one embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, the bitstream including encoded video information generated by the encoding method described above or encoded video information to be decoded by the decoding method described above. In another embodiment, a computer program product including, for example, a plurality of programs in a memory 1130 is also provided, the plurality of programs being executable by a processor 1120 in a computing environment 1110 to perform the methods described above. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0188] In an embodiment, the computing environment 1110 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0189] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0190] In one embodiment, a method for transmitting a bitstream generated by the encoder described above is also provided. In another embodiment, a method for receiving a bitstream to be decoded by the decoder described above is also provided.

[0191] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0192] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0193] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A video encoding method, characterized in that, include: Construct a pattern pool for the current block, wherein the pattern pool includes multiple current block encoding patterns or the multiple current block encoding patterns and sub-block encoding patterns; By selecting at least one pattern from the pattern pool and determining the order of the at least one pattern, a pattern queue including the at least one pattern is determined for the current block; Execute at least one mode in the order of modes in the mode queue to determine the encoding mode for the current block or multiple sub-blocks divided from the current block; The current block or the multiple sub-blocks are encoded based on the determined encoding pattern. Wherein, the plurality of current block encoding patterns are related to the encoding of the current block, and the sub-block encoding patterns are related to the encoding of the plurality of sub-blocks. Wherein, when the mode pool only includes the plurality of current block encoding modes, the mode queue includes the sub-block encoding modes, and The step of executing the sub-block encoding mode includes dividing the current block to obtain multiple sub-blocks, and for each of the multiple sub-blocks, determining the encoding mode of the corresponding sub-block by constructing a mode pool and determining a mode queue.

2. The video encoding method according to claim 1, characterized in that, The pattern pools and / or pattern queues of different sub-blocks may be the same or different.

3. The video encoding method according to claim 1, characterized in that, When the pattern pool only includes the multiple current block encoding patterns, the pattern queue includes the sub-block encoding patterns according to the encoding partitioning order. The position of the sub-block encoding pattern in the pattern queue is related to the encoding division order.

4. The video encoding method according to claim 3, characterized in that, The encoding partitioning order includes small block-large block order, large block-small block order, and / or large block-small block-large block order. Wherein, when the encoding division order is small block-large block order, the sub-block encoding pattern is located at the first position in the pattern queue; When the encoding division order is large-block-small-block order, the sub-block encoding pattern is located at the last position in the pattern queue; and / or When the encoding division order is large block-small block-large block, the sub-block encoding pattern is located in the middle position of the pattern queue.

5. The video encoding method according to claim 1, characterized in that, The step of determining a pattern queue including the at least one pattern for the current block by selecting at least one pattern from the pattern pool and determining the order of the at least one pattern includes: By referring to the encoding information of neighboring blocks of the current block, the encoding information of the parent or child blocks of the current block, and / or the pre-analyzed encoding information of the video frame in which the current block is located, the at least one mode is selected from the mode pool and the order of the at least one mode in the mode queue is determined.

6. The video encoding method according to claim 1, characterized in that, The plurality of current block encoding modes include a plurality of encoding modes for performing encoding processing on the current block, or a plurality of first encoding sub-modes and a plurality of second encoding sub-modes respectively split from the plurality of encoding modes. The first encoding sub-mode is used to perform prediction processing in the encoding process of the corresponding encoding mode, and the second encoding sub-mode is used to perform transformation processing, quantization processing and entropy encoding processing in the encoding process of the corresponding encoding mode.

7. The video encoding method according to claim 6, characterized in that, The step of executing at least one mode in the pattern queue to determine the encoding mode for the current block or multiple sub-blocks divided from the current block includes: Based on the execution results of previous modes in the mode queue, the execution of at least one subsequent mode in the mode queue is skipped.

8. The video encoding method according to claim 7, characterized in that, The step of skipping the execution of at least one subsequent pattern in the pattern queue based on the execution result of a previous pattern in the pattern queue includes: When the previous mode is an encoding mode, based on the rate-distortion optimization result and / or distortion result obtained by performing encoding processing using the encoding mode, the execution of at least one subsequent mode in the mode queue is skipped. When the previous pattern is the first coded sub-pattern, based on the distortion result obtained by performing prediction processing using the first coded sub-pattern, the execution of the corresponding second coded sub-pattern in the pattern queue is skipped, and / or When the previous mode is a sub-block encoding mode, the execution of at least one subsequent mode in the mode queue is skipped based on the sum of rate-distortion optimization results and / or the sum of distortion results of the plurality of sub-blocks obtained by performing encoding processing using the sub-block encoding mode.

9. A video encoding device, characterized in that, include: A pattern pool construction unit is configured to construct a pattern pool for the current block, wherein the pattern pool includes multiple current block encoding patterns or the multiple current block encoding patterns and sub-block encoding patterns; The queue determination unit is configured to determine a pattern queue including the at least one pattern for the current block by selecting at least one pattern from the pattern pool and determining the order of the at least one pattern. The pattern determination unit is configured to execute at least one pattern in the pattern queue according to the pattern order to determine the encoding pattern for the current block or multiple sub-blocks divided from the current block; The encoding unit is configured to encode the current block or the plurality of sub-blocks based on the determined encoding pattern. Wherein, the plurality of current block encoding patterns are related to the encoding of the current block, and the sub-block encoding patterns are related to the encoding of the plurality of sub-blocks. Wherein, when the mode pool only includes the plurality of current block encoding modes, the mode queue includes the sub-block encoding modes, and The step of executing the sub-block encoding mode includes dividing the current block to obtain multiple sub-blocks, and for each of the multiple sub-blocks, determining the encoding mode of the corresponding sub-block by constructing a mode pool and determining a mode queue.

10. An electronic device, characterized in that, include: At least one processor; At least one memory that stores computer-executable instructions. Wherein, when the computer-executable instructions are executed by the at least one processor, they cause the at least one processor to perform the video encoding method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the at least one processor to perform the video encoding method as described in any one of claims 1 to 8.

12. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by at least one processor, they implement the video encoding method as described in any one of claims 1 to 8.

13. A method for transmitting a bit stream, characterized in that, include: Perform the video encoding method as described in any one of claims 1 to 8 to generate the bitstream; as well as Send the bit stream.

Images