Method and apparatus for decoding a current video block of a video bitstream

Abstract

Embodiments of the present application disclose a method and apparatus for decoding a current video block of a video bitstream. The method comprises receiving the encoded video bitstream; extracting first transform coefficients of a first color component of the current video block from the encoded video bitstream; determining second transform coefficients of a second color component of the current video block; deriving an offset value based on at least one of a quantization step size or the first transform coefficients; adding the offset value to the second transform coefficients to generate modified second transform coefficients of the second color component; and decoding the current video block based on the first transform coefficients and the modified second transform coefficients of the second color component.

Description

[0001] By incorporating via reference

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 250,832, filed September 30, 2021, which is incorporated herein by reference in its entirety. This application also claims the benefit of priority to U.S. Non-Provisional Patent Application No. 17 / 939,199, filed September 7, 2022, which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to video encoding / decoding technology, and specifically to a method and apparatus for decoding the current video block of a video bitstream. Background Technology

[0004] The background description provided herein is for the purpose of presenting the general context of this disclosure. To the extent that the work described in this background section is applicable, neither the work of the currently identified inventors nor aspects of the description that are not otherwise considered prior art at the time of filing this application are expressly or implicitly acknowledged as prior art to this disclosure.

[0005] Video encoding and decoding can be performed using inter-frame picture prediction with motion compensation. Uncompressed digital video can comprise a series of pictures, each with a spatial dimension of, for example, 1920×1080 luminance samples and associated full-color or subsampled chrominance samples. The series of pictures can have a fixed or variable picture rate (alternatively referred to as the frame rate), such as 60 pictures per second or 60 frames per second. Uncompressed video has specific bitrate requirements for streaming or data processing. For example, video with a pixel resolution of 1920×1080, a frame rate of 60 frames per second, and chrominance subsampling of 4:2:0 at 8 bits per pixel per color channel requires approximately 1.5 Gbit / s of bandwidth. One hour of such video would require over 600 Gbytes of storage.

[0006] One purpose of video encoding and decoding is to reduce redundancy in the uncompressed input video signal through compression. In some cases, compression can help reduce the aforementioned bandwidth and / or storage requirements by two orders of magnitude or more. Lossless compression, lossy compression, and combinations thereof can be employed. Lossless compression refers to a technique that allows an exact copy of the original signal to be reconstructed from the compressed original signal via a decoding process. Lossy compression refers to an encoding / decoding process in which the original video information is not fully preserved during encoding and cannot be fully recovered during decoding. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal is useful for the intended application despite some information loss. In the case of video, lossy compression is widely used in many applications. The amount of distortion tolerated depends on the application. For example, users of some consumer video streaming applications may tolerate higher distortion than users of film or television broadcasting applications. The compression ratio achievable by a particular encoding algorithm can be selected or adjusted to reflect various distortion tolerances: higher tolerable distortion generally allows encoding algorithms that produce higher losses and higher compression ratios.

[0007] Video encoders and video decoders can utilize techniques from several broad categories and steps, including, for example, motion compensation, Fourier transform, quantization, and entropy coding.

[0008] Video codec techniques can include techniques known as intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, this picture can be called an intra-frame picture. Intra-frame pictures and their derivatives (e.g., independent decoder refresh pictures) can be used to reset the decoder state and can therefore be used as the first picture in the encoded video bitstream and video session or as a still image. The samples of the block following intra-frame prediction can then be transformed into the frequency domain, and the resulting transform coefficients can be quantized before entropy coding. Intra-frame prediction represents a technique that minimizes the sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value and the smaller the AC coefficients, the fewer bits are needed to represent the block following entropy coding at a given quantization step size.

[0009] Traditional intra-frame coding techniques, such as those known from MPEG-2 generation coding technologies, do not use intra-frame prediction. However, some newer video compression techniques include those that attempt to encode / decode blocks based on metadata and / or surrounding sample data, obtained during encoding and / or decoding that are spatially adjacent and precede the data blocks being intra-frame encoded or decoded in decoding order. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that, in at least some cases, intra-frame prediction uses only reference data from the current frame being reconstructed, and not reference data from other reference frames.

[0010] There can be many different forms of intra-prediction. When more than one such technique can be used in a given video coding technique, the technique used can be called an intra-prediction mode. A particular codec can provide one or more intra-prediction modes. In some cases, a mode can have sub-modes and / or can be associated with various parameters, and the mode / sub-mode information of the video block and the intra-coding parameters can be encoded individually or collectively included in the mode codeword. The codeword to be used for a given mode, sub-mode, and / or parameter combination can affect the coding efficiency gain through intra-prediction, and therefore can affect the entropy coding technique used to convert the codeword into a bitstream.

[0011] Certain patterns of intra-frame prediction were introduced in H.264, refined in H.265, and further refined in newer coding techniques such as the Joint Development Model (JEM), Universal Video Coding (VVC), and Baseline Set (BMS). Typically, for intra-frame prediction, prediction value blocks can be formed using neighboring sample values ​​that have become available. For example, the available values ​​of a specific set of neighboring samples along a specific direction and / or line can be copied into the prediction value block. The reference to the direction used can be encoded in the bitstream, or it can be predicted itself.

[0012] The joint chroma coding scheme assumes a statistical correlation between the transform coefficients of the Cr and Cb transform blocks, which can introduce distortion in some cases. In particular, this assumption fails and does not save any coding bits when one of the color coefficients in the transform block is non-zero and the coefficient of the other color component is zero. Summary of the Invention

[0013] This disclosure describes various implementations of methods, apparatuses, and computer-readable storage media for deriving offsets in cross-component transform coefficient level reconstruction.

[0014] According to one aspect, embodiments of this disclosure provide a method for deriving an offset in a cross-component transform coefficient level reconstruction. The method includes receiving an encoded video bitstream by means of an apparatus. The apparatus includes a memory storing instructions and a processor communicating with the memory. The method further includes extracting first transform coefficients of a first color component of a current video block from the encoded video bitstream by the apparatus; determining second transform coefficients of a second color component of the current video block by the apparatus; deriving an offset value by the apparatus based on at least one of a quantization step size or the first transform coefficients; adding the offset value to the second transform coefficients by the apparatus to generate modified second transform coefficients of the second color component; and decoding the current video block by the apparatus based on the first transform coefficients of the first color component and the modified second transform coefficients of the second color component.

[0015] According to another aspect, embodiments of this disclosure provide an apparatus for deriving offsets in cross-component transform coefficient level reconstruction. The apparatus includes: a memory storing instructions; and a processor communicating with the memory. When the processor executes the instructions, the processor is configured to cause the apparatus to perform the methods described above for video decoding and / or encoding.

[0016] In another aspect, embodiments of this disclosure provide a non-transitory computer-readable medium storing instructions that, when executed by a computer for video decoding and / or encoding, cause the computer to perform the methods described above for video decoding and / or encoding.

[0017] In summary, the embodiments of this application describe a coefficient-level cross-component coding scheme that utilizes some correlations between the juxtaposed transform coefficients of color components. It is particularly effective for transform blocks or transform units where the transform coefficient of one color component is zero and the transform coefficient of another color component is non-zero. To a certain extent, it can recover some information lost during the coding and quantization process without generating significant additional coding costs.

[0018] The foregoing and other aspects, and their implementations, are described in more detail in the accompanying drawings, specification, and claims. Attached Figure Description

[0019] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0020] Figure 1A A schematic illustration of an exemplary subset of intra-frame predicted directional patterns is shown;

[0021] Figure 1B An illustration of an exemplary intra-frame prediction direction is shown;

[0022] Figure 2A schematic illustration of the current block and its surrounding spatial merging candidates for motion vector prediction according to an embodiment of this application is shown;

[0023] Figure 3 A schematic illustration of a simplified block diagram of a communication system according to an embodiment of this application is shown;

[0024] Figure 4 A schematic illustration of a simplified block diagram of a communication system according to another embodiment of this application is shown;

[0025] Figure 5 A schematic illustration of a simplified block diagram of a video decoder according to an embodiment of this application is shown;

[0026] Figure 6 A schematic illustration of a simplified block diagram of a video encoder according to an embodiment of this application is shown;

[0027] Figure 7 A block diagram of a video encoder according to another embodiment of this application is shown;

[0028] Figure 8 A block diagram of a video decoder according to another embodiment of this application is shown;

[0029] Figure 9 A scheme for dividing coded blocks according to an embodiment of this application is shown;

[0030] Figure 10 A scheme for dividing coded blocks according to another embodiment of this application is shown;

[0031] Figure 11 A scheme for dividing coded blocks according to yet another embodiment of this application is shown;

[0032] Figure 12 An example of dividing a basic block into coded blocks according to an embodiment of this application is shown;

[0033] Figure 13 An example ternary partitioning scheme is shown;

[0034] Figure 14 An example quadtree / binary tree coding block partitioning scheme is shown;

[0035] Figure 15 This paper illustrates a scheme for dividing a coding block into multiple transform blocks according to an embodiment of the present application, and the coding order of the transform blocks;

[0036] Figure 16 A scheme for dividing a coding block into multiple transform blocks and the coding order of the transform blocks are shown according to another embodiment of this application;

[0037] Figure 17A scheme for dividing a coding block into multiple transform blocks according to yet another embodiment of this application is shown;

[0038] Figure 18 A flowchart of a method according to an embodiment of this application is shown; and

[0039] Figure 19 A schematic illustration of a computer system according to an embodiment of this application is shown. Detailed Implementation

[0040] Throughout the specification and claims, terms may have nuanced meanings beyond those explicitly stated, implied or suggested in the context. The phrases “in one embodiment” or “in some embodiments” as used herein do not necessarily refer to the same embodiment, and the phrases “in another embodiment” or “in other embodiments” as used herein do not necessarily refer to different embodiments. Similarly, the phrases “in one implementation” or “in some implementations” as used herein do not necessarily refer to the same implementation, and the phrases “in another implementation” or “in other implementations” as used herein do not necessarily refer to different implementations. For example, it is intended that the claimed subject matter includes a combination of all or part of the exemplary embodiments / implementations.

[0041] Generally, terms can be understood, at least in part, from their usage in context. For example, terms such as “and,” “or,” or “and / or,” as used herein, can come in a variety of meanings, which can depend at least in part on the context in which such terms are used. Generally, if “or” is used with an associative list, such as A, B, or C, then “or” is intended to mean A, B, and C, used here in an inclusive sense, and A, B, or C, used here in an exclusive sense. Additionally, terms such as “one or more” or “at least one,” as used herein, can be used, at least in part, to describe any feature, structure, or characteristic in a singular sense, or can be used, at least in a plural sense, to describe a combination of features, structures, or characteristics. Similarly, terms such as “a,” “one,” or “the” can also be understood to convey either a singular or a plural usage, which depends at least in part on the context. Furthermore, the terms “based on” or “determined by” can be understood to not necessarily convey an exclusive set of factors, and can alternatively allow for additional factors that are not necessarily explicitly described, which also depends at least in part on the context. Figure 3 A simplified block diagram of a communication system (300) according to an embodiment of the present disclosure is shown. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and terminal device (320) interconnected via the network (350). Figure 3 In the example, the first pair of terminal devices (310) and (320) can perform one-way data transmission. For example, terminal device (310) can encode video data (e.g., a video image stream captured by terminal device (310)) for transmission via network (350) to another terminal device (320). The encoded video data can be transmitted in the form of one or more encoded video bitstreams. Terminal device (320) can receive the encoded video data from network (350), decode the encoded video data to recover the video images, and display the video images based on the recovered video images. One-way data transmission can be implemented in media service applications, etc.

[0042] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) performing bidirectional transmission of encoded video data, which may be implemented, for example, during a video conferencing application. For bidirectional data transmission, in this example, each of the terminal devices (330) and (340) may encode video data (e.g., a stream of video images captured by the terminal device) for transmission via a network (350) to the other terminal device (330) and (340). Each of the terminal devices (330) and (340) may also receive encoded video data transmitted by the other terminal device (330) and (340), and may decode the encoded video data to recover video images, and may display the video images at an accessible display device based on the recovered video data.

[0043] Reference Figure 1A The lower right corner depicts a subset of nine prediction directions specified among the 33 possible intra-frame prediction directions in H.265 (corresponding to 33 angular modes out of the 35 intra-frame modes specified in H.265). The point (101) where the arrows converge represents the sample being predicted. The arrows indicate the direction in which neighboring samples are used to predict the sample at 101. For example, arrow (102) indicates that sample (101) is predicted from one or more neighboring samples at a 45-degree angle to the horizontal from the upper right. Similarly, arrow (103) indicates that sample (101) is predicted from one or more neighboring samples at a 22.5-degree angle to the horizontal from the lower left of sample (101).

[0044] Still refer to Figure 1AThe top left depicts a 4×4 square block (104) of samples (indicated by a bold dashed line). The square block (104) comprises 16 samples, each labeled with “S”, its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from top) and the first sample in the X dimension (from left). Similarly, sample S44 is the fourth sample in both the Y and X dimensions of block (104). Since the block size is 4×4 samples, S44 is located in the bottom right. An example reference sample following a similar numbering scheme is further shown. The reference sample is labeled with R, its Y position (e.g., row index) relative to block (104), and its X position (column index). In both H.264 and H.265, predicted samples that are adjacent to the block being reconstructed are used.

[0045] Intra-frame image prediction for block 104 can begin by copying reference sample values ​​from neighboring samples according to the prediction direction indicated by a signal. For example, suppose the encoded video bitstream includes the following signaling for block 104, which indicates the prediction direction of arrow (102), that is, the samples are predicted based on one or more prediction samples at a 45-degree angle to the horizontal direction in the upper right. In such a case, samples S41, S32, S23, and S14 are predicted based on the same reference sample R05. Then, sample S44 is predicted based on reference sample R08.

[0046] In some cases, the values ​​of multiple reference samples can be combined, for example, by interpolation, to calculate the reference sample; especially when the direction is not divisible by 45 degrees.

[0047] As video coding technologies continue to develop, the number of possible directions is also increasing. For example, in H.264 (2003), nine different directions were available for intra-frame prediction. This increased to 33 in H.265 (2013), and JEM / VVC / BMS could support up to 65 directions at the time of publication. Experimental studies have been conducted to help identify the most suitable intra-frame prediction directions, and certain techniques in entropy coding can be used to encode those most suitable directions with a small number of bits, thus accepting a specific bit penalty for the direction. Furthermore, sometimes the direction itself can be predicted based on the neighboring directions used in the intra-frame prediction of decoded neighboring blocks.

[0048] Figure 1B A schematic diagram (180) depicting 65 intra-frame prediction directions according to JEM is shown to illustrate the increasing number of prediction directions in various coding techniques developed over time.

[0049] The way bits representing intra-prediction directions are mapped to prediction directions in a encoded video bitstream can vary depending on the video coding technique; and this mapping can range from a simple direct mapping from prediction direction to intra-prediction mode to codeword to a complex adaptive scheme involving the most probable mode and similar techniques. However, in all cases, there may exist some intra-prediction directions that are statistically less likely to appear in the video content compared to some other directions. Since the goal of video compression is to reduce redundancy, in well-functioning video coding techniques, those less probable directions can be represented by a larger number of bits compared to the more probable directions.

[0050] Inter-frame image prediction, or inter-frame prediction, can be based on motion compensation. In motion compensation, after spatial shifting in a direction indicated by a motion vector (hereafter referred to as MV), sample data from a previously reconstructed image or a portion thereof (the reference image) can be used to predict a newly reconstructed image or image portion (e.g., a patch). In some cases, the reference image can be the same as the currently reconstructed image. MV can have two dimensions, X and Y, or three dimensions, where the third dimension is an indication of the reference image in use (similar to the temporal dimension).

[0051] In some video compression techniques, the current MV applicable to a specific region of the sample data can be predicted from other MVs, such as those other MVs that are spatially adjacent to other regions of the sample data that are being reconstructed and that precede the current MV in the decoding order. This can significantly reduce the total amount of data required to encode the MV by relying on removing redundancy in the associated MVs, thereby improving compression efficiency. MV prediction can work effectively, for example, because when encoding an input video signal derived from a camera device (referred to as natural video), there is a statistical probability that regions in the video sequence larger than the region applicable to a single MV move in similar directions, and therefore, in some cases, similar motion vectors derived from MVs in neighboring regions can be used for prediction. This results in the actual MV for a given region being similar to or the same as the MV predicted from the surrounding MVs. Such MVs can then be represented after entropy coding with fewer bits than if the MV were directly encoded from neighboring MVs instead of predicted from neighboring MVs. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, such as due to rounding errors when calculating predictions from several surrounding MVs, MV predictions themselves can be lossy.

[0052] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016). Among the various MV prediction mechanisms specified in H.265, the following describes a technique subsequently known as “spatial combining”.

[0053] Specifically, refer to Figure 2 The current block (201) includes samples that have been discovered by the encoder during the motion search process, and these samples can be predicted based on previous blocks of the same size that have generated spatial offsets. Instead of directly encoding the MV, the MV can be derived from metadata associated with one or more reference images. For example, the MV can be derived from the metadata of the nearest reference image (in decoding order) using the MV associated with any of five surrounding samples represented as A0, A1 and B0, B1, B2 (corresponding to 202 to 206, respectively). In H.265, MV prediction can use predictions from the same reference image that is also being used in neighboring blocks.

[0054] exist Figure 3 In the examples, the terminal devices (310), (320), (330), and (340) can be implemented as servers, personal computers, and smartphones, but the applicability of the basic principles of this disclosure is not limited thereto. Implementations of this disclosure can be implemented in desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated video conferencing equipment, etc. Network (350) refers to any number or type of network that transmits encoded video data between the terminal devices (310), (320), (330), and (340), including, for example, wired (connected) and / or wireless communication networks. Communication networks (350) can exchange data in circuit-switched, packet-switched, and / or other types of channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, unless expressly stated herein, the architecture and topology of the network (350) may be irrelevant to the operation of this disclosure.

[0055] As an example of the application of the disclosed topic Figure 4 The placement of a video encoder and video decoder in a video streaming environment is illustrated. The disclosed subject matter is equally applicable to other video applications, including, for example, video conferencing, digital television broadcasting, gaming, virtual reality, and storage of compressed video on digital media including CDs, DVDs, Memory Sticks, etc.

[0056] The video streaming system may include a video capture subsystem (413) that may include a video source (401), such as a digital camera device, for creating uncompressed video pictures or image streams (402). In the example, the video picture stream (402) includes samples recorded by the digital camera device of the video source 401. The video picture stream (402) is depicted as a thick line to emphasize the high data volume compared to encoded video data (404) (or encoded video bitstream), and the video picture stream (402) may be processed by an electronic device (420) including a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to implement or carry out aspects of the disclosed subject matter as described in more detail below. Encoded video data (404) (or encoded video bitstream (404)) is depicted as a thin line to emphasize the lower data volume compared to the uncompressed video picture stream (402). The encoded video data (404) can be stored on a streaming server (405) for future use or directly stored to a downstream video device (not shown). One or more streaming client subsystems, such as... Figure 4 Client subsystems (406) and (408) can access a streaming server (405) to retrieve copies (407) and (409) of encoded video data (404). Client subsystem (406) may include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the incoming copy (407) of encoded video data and creates an uncompressed outgoing video picture stream (411) that can be presented on a display (412) (e.g., a display screen) or other presentation device (not depicted). The video decoder 410 may be configured to perform some or all of the various functions described in this disclosure. In some streaming systems, encoded video data (404), (407), and (409) (e.g., video bitstreams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In this example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The topics disclosed can be used in environments with VVC and other video coding standards.

[0057] It should be noted that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) may include a video decoder (not shown), and electronic device (430) may also include a video encoder (not shown).

[0058] Figure 5A block diagram of a video decoder (510) according to any embodiment of the present disclosure is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuitry system). The video decoder (510) may be used instead of... Figure 4 The example video decoder (410).

[0059] The receiver (531) can receive one or more encoded video sequences to be decoded by the video decoder (510). In the same or another embodiment, one encoded video sequence can be decoded at a time, wherein the decoding of each encoded video sequence is independent of other encoded video sequences. Each video sequence can be associated with multiple video frames or images. Encoded video sequences can be received from a channel (501), which can be a hardware / software link to a storage device storing encoded video data or a streaming source transmitting encoded video data. The receiver (531) can receive encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective processing circuitry systems (not depicted). The receiver (531) can separate the encoded video sequences from other data. To combat network jitter, a buffer memory (515) can be provided between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) can be implemented as part of the video decoder (510). In other applications, the buffer memory (515) may be external to and separate from the video decoder (510) (not depicted). In still other applications, a buffer memory (not depicted) may exist external to the video decoder (510) for purposes such as combating network jitter, and another buffer memory (515) may exist internally to the video decoder (510) for purposes such as handling playback timing. When the receiver (531) is receiving data from a store / forward device with sufficient bandwidth and controllability or from an equal synchronization network, the buffer memory (515) may not be required, or the buffer memory (515) may be small. For use on best-effort packet networks such as the Internet, a buffer memory (515) of sufficient size may be required, and the size of the buffer memory (515) may be relatively large. Such a buffer memory may be implemented with an adaptive size and may be implemented at least partially in a similar component (not depicted) external to the operating system or the video decoder (510).

[0060] The video decoder (510) may include a parser (520) to reconstruct symbols (521) based on the encoded video sequence. These symbols may include information for managing the operation of the video decoder (510) and may include information for controlling a presentation device such as a display (512) (e.g., a screen), which may or may not be part of the electronic device (530) but may be coupled to it, such as... Figure 5 As shown. Control information for the presentation device may take the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (520) may perform parsing / entropy decoding on the encoded video sequence received by the parser (520). The entropy coding of the encoded video sequence may conform to video coding techniques or video coding standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a subgroup parameter set for at least one subgroup of pixel subgroups in the video decoder from the encoded video sequence based on at least one parameter corresponding to the subgroup. Subgroups may include Group of Pictures (GOP), pictures, tiles, slices, macroblocks, Coding Units (CU), blocks, Transform Units (TU), Prediction Units (PU), etc. The parser (520) can also extract information from the encoded video sequence, such as transform coefficients (e.g., Fourier transform coefficients), quantizer parameter values, motion vectors, etc.

[0061] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

[0062] Depending on the type of the encoded video picture or its components (such as inter-frame pictures and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of the symbol (521) can involve multiple different processing units or functional units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (520). For the sake of brevity, the flow of such subgroup control information between the parser (520) and the multiple processing units or functional units below is not depicted.

[0063] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into a number of functional units as described below. In a practical implementation operating under commercial constraints, many of these functional units interact closely with each other and can be integrated with each other at least partially. However, for the purpose of clearly describing the various functions of the disclosed subject matter, the conceptual subdivision into functional units is adopted in the following disclosure.

[0064] The first unit may include a scaler / inverse transform unit (551). The scaler / inverse transform unit (551) can receive quantization transform coefficients as symbols (521) from the parser (520) and control information, including information indicating which type of inverse transform to use, block size, quantization factor / parameter, quantization scaling matrix, etc. The scaler / inverse transform unit (551) can output a block including sample values, which can be input into the aggregator (555).

[0065] In some cases, the output samples of the scaler / inverse transform unit (551) may belong to intra-coded blocks, i.e., blocks that do not use predictive information from previously reconstructed images, but instead use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) may use information from surrounding blocks that have already been reconstructed and stored in the current picture buffer (558) to generate blocks of the same size and shape as the block being reconstructed. For example, the current picture buffer (558) buffers the partially reconstructed current image and / or the fully reconstructed current image. In some implementations, the aggregator (555) may add the prediction information already generated by the intra-picture prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) on a per-sample basis.

[0066] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to inter-frame coded blocks that may have undergone motion compensation. In such cases, the motion compensation prediction unit (553) may access the reference image memory (557) to obtain samples for inter-frame image prediction. After motion compensation of the obtained samples according to the symbols (521) belonging to the blocks, these samples may be added by the aggregator (555) to the output of the scaler / inverse transform unit (551) (the output of unit 551 may be referred to as residual samples or residual signals) to generate output sample information. The address in the reference image memory (557) from which the motion compensation prediction unit (553) obtains the predicted samples may be controlled by a motion vector, which may be obtained by the motion compensation prediction unit (553) in the form of symbols (521) that may have, for example, an X component, a Y component (offset), and a reference image component (time). Motion compensation may also include interpolation of sample values ​​obtained from a reference image memory (557) when using subsampled precise motion vectors, and may also be associated with motion vector prediction mechanisms, etc.

[0067] The output samples of the aggregator (555) can be subjected to various loop filtering techniques in the loop filter unit (556). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also known as the encoded video bitstream) and obtainable by the loop filter unit (556) as symbols (521) from the parser (520), but may also respond to metadata obtained during decoding of previous (in decoding order) portions of the encoded picture or encoded video sequence, as well as to previously reconstructed and loop-filtered sample values. Several types of loop filters may be included as part of the loop filter unit 556 in various orders, as will be described in further detail below.

[0068] The output of the loop filter unit (556) can be a sample stream, which can be output to the presentation device (512) and stored in the reference image memory (557) for future inter-frame image prediction.

[0069] Once a certain coded image has been fully reconstructed, it can be used as a reference image for future inter-frame image prediction. For example, once the coded image corresponding to the current image has been fully reconstructed and that coded image (by, for example, the parser (520)) is identified as the reference image, the current image buffer (558) can become part of the reference image memory (557), and a new current image buffer can be reallocated before the reconstruction of subsequent coded images begins.

[0070] The video decoder (510) can perform decoding operations according to a predetermined video compression technique adopted in, for example, the ITU-T H.265 Recommendation. The encoded video sequence can conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under said configuration file. For compliance, the complexity of the encoded video sequence can be within limits defined by a level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (measured, for example, megasamples per second), maximum reference picture size, etc. In some cases, the limitations set by the level can be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata signaled in the encoded video sequence for HRD buffer management.

[0071] In some example implementations, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The video decoder (510) may use the additional data to correctly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0072] Figure 6 A block diagram of a video encoder (603) according to an example embodiment of the present disclosure is shown. The video encoder (603) may be included in an electronic device (620). The electronic device (620) may also include a transmitter (640) (e.g., a transmission circuitry system). The video encoder (603) may be used instead of Figure 4 The example video encoder (403).

[0073] The video encoder (603) can capture video images from a video source (601) that can be encoded by the video encoder (603). Figure 6 In one example, the video source (601) is not part of the electronic device (620) and receives video samples. In another example, the video source (601) can be implemented as part of the electronic device (620).

[0074] The video source (601) can provide a sequence of source video in the form of a digital video sample stream to be encoded by the video encoder (603). The digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit…), any color space (e.g., BT.601YCrCB, RGB, XYZ…), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (601) can be a storage device capable of storing previously prepared video. In a video conferencing system, the video source (601) can be a camera device that captures local image information as a video sequence. The video data can be provided as multiple individual pictures or images that are given motion when viewed sequentially. The pictures themselves can be organized as spatial pixel arrays, where each pixel can include one or more samples, depending on the sampling structure, color space, etc., used. Those skilled in the art will readily understand the relationship between pixels and samples. The following description focuses on samples.

[0075] According to some example implementations, the video encoder (603) can encode and compress images of a source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate constitutes a function of the controller (650). In some implementations, the controller (650) may be functionally coupled to and control other functional units as described below. For brevity, the coupling is not depicted. Parameters set by the controller (650) may include rate control related parameters (image skipping, quantizer, λ value of rate distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured to have other suitable functions related to the video encoder (603) optimized for a particular system design.

[0076] In some example implementations, the video encoder (603) may be configured to operate within an encoding loop. As a highly simplified description, in this example, the encoding loop may include: a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image); and a (local) decoder (633), which is embedded within the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder would create sample data, even though the embedded decoder 633 processes the encoded video stream of the source encoder 630 without entropy coding (because in the video compression techniques considered in the disclosed subject matter, any compression between the symbols and the encoded video bitstream in entropy coding may be lossless). The reconstructed sample stream (sample data) is input to a reference image memory (634). Since the decoding of the symbol stream results in a bit-exact result regardless of the decoder location (local or remote), the contents of the reference image memory (634) are also bit-exact between the local and remote encoders. In other words, the encoder's prediction part "treats" the exact same sample values ​​that the decoder will "see" during decoding as the reference image sample. This fundamental principle of reference image synchronization (and the resulting offset in cases where synchronization cannot be maintained, for example, due to channel errors) is used to improve encoding quality.

[0077] The operation of the "local" decoder (633) can be the same as that of the "remote" decoder, such as the video decoder (510), as already combined above. Figure 5 The video decoder (510) is described in detail. However, a brief additional reference is provided. Figure 5 Since symbols are available and the encoding of symbols into a encoded video sequence by the entropy encoder (645) and the decoding of symbols by the parser (520) can be lossless, the entropy decoding part of the video decoder (510), which includes the buffer memory (515) and the parser (520), can be fully implemented in the local decoder (633) in the encoder.

[0078] It can be observed that any decoder technique, except for parsing / entropy decoding which may exist solely in the decoder, must also exist in the corresponding encoder in essentially the same functional form. For this reason, the disclosed topics may sometimes focus on decoder operations, which are related to the decoding portion of the encoder. Since encoder techniques are the opposite of the fully described decoder techniques, the description of encoder techniques can be simplified. A more detailed description of the encoder is provided below, focusing only on certain areas or aspects.

[0079] In some example implementations, during operation, the source encoder (630) may perform motion-compensated predictive coding, which predictively encodes the input image with reference to one or more previously encoded images from the video sequence designated as "reference images." In this way, the encoding engine (632) encodes the difference (or residual) in the color channels between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as the prediction reference for the input image. The term "residual" and its adjective form "residual" are used interchangeably.

[0080] The local video decoder (633) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (630). The operation of the encoding engine (632) can advantageously be a lossy process. When it is possible to decode the video decoder (633) Figure 6 When decoding encoded video data at a location (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding process performed on the reference image by the video decoder and can store the reconstructed reference image in a reference image memory (634). In this way, the video encoder (603) can locally store a copy of the reconstructed reference image that shares common content (no transmission errors) with the reconstructed reference image to be obtained by a remote video decoder.

[0081] The predictor (635) can perform a prediction search against the encoding engine (632). That is, for a new image to be encoded, the predictor (635) can search the reference image memory (634) for sample data (as candidate reference pixel blocks) or specific metadata such as reference image motion vectors, block shapes, etc., which can be used as suitable prediction references for the new image. The predictor (635) can operate on a sample-block-to-pixel basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (634).

[0082] The controller (650) can manage the encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters used to encode video data.

[0083] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into a coded video sequence by performing lossless compression on the symbols according to techniques such as Huffman coding, variable-length coding, arithmetic coding, etc.

[0084] The transmitter (640) can buffer the encoded video sequence created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which may be a hardware / software link to a storage device capable of storing the encoded video data. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0085] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a specific encoded image type to each encoded image, which may affect the encoding techniques that can be applied to the corresponding image. For example, one of the following image types can typically be assigned to an image:

[0086] An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand those variations of I-pictures and their respective applications and characteristics.

[0087] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, wherein the inter-frame prediction uses at most one motion vector and reference index to predict sample values ​​for each block.

[0088] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using either intra-frame prediction or inter-frame prediction, where inter-frame prediction uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can be used to reconstruct a single block using more than two reference images and associated metadata.

[0089] Source images can typically be spatially subdivided into multiple sample coding blocks (e.g., blocks with 4×4, 8×8, 4×8, or 16×16 samples respectively) and coded block by block. These blocks can be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to the corresponding images. For example, blocks of image I can be non-predictively coded, or blocks of image I can be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of image P can be predictively coded with reference to a previously coded reference image via spatial prediction or via temporal prediction. Blocks of image B can be predictively coded with reference to one or two previously coded reference images via spatial prediction or via temporal prediction. For other purposes, source images or intermediate images can be subdivided into other types of blocks. The partitioning of coding blocks and other types of blocks may or may not follow the same manner, as described in further detail below.

[0090] The video encoder (603) can perform encoding operations according to a predetermined video coding technique or standard such as ITU-T H.265 Recommendation. In its operation, the video encoder (603) can perform various compression operations, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard in use.

[0091] In some example implementations, the transmitter (640) may send additional data along with the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0092] Video can be captured temporally as multiple source images (video frames). Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes temporal or other correlations between images. For example, a specific image being encoded / decoded, referred to as the current image, can be divided into blocks. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference image.

[0093] In some example implementations, bidirectional prediction techniques can be used for inter-frame image prediction. According to such bidirectional prediction techniques, two reference images are used, such as a first reference image and a second reference image that both precede the current image in the video in decoding order (but may be in the past and future in display order, respectively). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be predicted jointly using a combination of the first and second reference blocks.

[0094] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0095] According to some example implementations of this disclosure, predictions such as inter-frame picture prediction and intra-frame picture prediction are performed on a block-by-block basis. For example, pictures in a video picture sequence are segmented into coding tree units (CTUs) for compression. CTUs in the pictures can have the same size, for example, 64×64 pixels, 32×32 pixels, or 16×16 pixels. Typically, a CTU can include three parallel coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively split into one or more coding units (CUs) in a quadtree. For example, a 64×64 pixel CTU can be split into one 64×64 pixel CU, or four 32×32 pixel CUs. Each of one or more of the 32×32 blocks can also be split into four 16×16 pixel CUs. In some example implementations, each CU can be analyzed during encoding to determine the prediction type of that CU among various prediction types, such as inter-frame prediction type or intra-frame prediction type. Depending on temporal and / or spatial predictability, a CU can be split into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma prediction blocks. In implementations, prediction operations in encoding (encoding / decoding) are performed on a block-by-block basis. Splitting the CU into PUs (or PBs for different color channels) can be performed in various spatial modes. A luma or chroma PB can, for example, contain a matrix of values ​​(e.g., luma values) of samples such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0096] Figure 7A diagram of a video encoder (703) according to another exemplary embodiment of this disclosure is shown. The video encoder (703) is configured to receive a processing block (e.g., a prediction block) of sample values ​​within a current video frame in a video frame sequence, and to encode the processing block into a coded frame that is part of a coded video sequence. The example video encoder (703) can be used instead. Figure 4 The video encoder (403) in the example.

[0097] For example, a video encoder (703) receives a matrix of sample values ​​for processing blocks, such as prediction blocks of 8×8 samples. The video encoder (703) then uses, for example, rate distortion optimization (RDO) to determine whether to use intra-frame mode, inter-frame mode, or bidirectional prediction mode to optimally encode the processing block. When it is determined that the processing block should be encoded in intra-frame mode, the video encoder (703) can encode the processing block into an encoded picture using intra-frame prediction techniques; and when it is determined that the processing block should be encoded in inter-frame mode or bidirectional prediction mode, the video encoder (703) can encode the processing block into an encoded picture using inter-frame prediction techniques or bidirectional prediction techniques, respectively. In some example implementations, a merge mode can be used as a sub-mode for inter-frame picture prediction, wherein motion vectors are derived from one or more motion vector prediction values ​​without the aid of encoded motion vector components outside the prediction values. In some other example implementations, motion vector components applicable to the object block may exist. Therefore, the video encoder (703) may include Figure 7 Components not explicitly shown include, for example, a pattern decision module for determining the prediction pattern of a processing block.

[0098] exist Figure 7 In the example, the video encoder (703) includes, for example, Figure 7 The example arrangement shows an inter-frame encoder (730), an intra-frame encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725) coupled together.

[0099] The inter-frame encoder (730) is configured to: receive a sample of the current block (e.g., the processed block); compare the block with one or more reference blocks in a reference image (e.g., blocks in previous and subsequent images in display order); generate inter-frame prediction information (e.g., motion vectors, merging mode information, descriptions of redundancy information according to the inter-frame coding technique); and compute inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some examples, the reference images are based on encoded video information embedded in... Figure 6 Decoding unit 633 in example encoder 620 (shown as) Figure 7The residual decoder 728 (described in further detail below) is used to decode the reference image.

[0100] The intra encoder (722) is configured to: receive samples of the current block (e.g., the processed block); compare the block with already encoded blocks in the same image; generate quantization coefficients after transformation; and in some cases, generate intra prediction information (e.g., intra prediction direction information based on one or more intra coding techniques). The intra encoder (722) can compute intra prediction results (e.g., the predicted block) based on the intra prediction information and a reference block in the same image.

[0101] A general controller (721) can be configured to determine general control data and control other components of the video encoder (703) based on the general control data. In the example, the general controller (721) determines the prediction mode of the block and provides control signals to the switch (726) based on the prediction mode. For example, if the prediction mode is intra-frame mode, the general controller (721) controls the switch (726) to select the intra-frame mode result for use by the residual calculator (723) and controls the entropy encoder (725) to select intra-frame prediction information and include it in the bitstream; and if the prediction mode of the block is inter-frame mode, the general controller (721) controls the switch (726) to select the inter-frame prediction result for use by the residual calculator (723) and controls the entropy encoder (725) to select inter-frame prediction information and include it in the bitstream.

[0102] A residual calculator (723) can be configured to calculate the difference (residual data) between the received block and the prediction result of a block selected from the intra encoder (722) or the inter encoder (730). A residual encoder (724) can be configured to encode the residual data to generate transform coefficients. For example, the residual encoder (724) can be configured to transform the residual data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various example embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be appropriately used by the intra encoder (722) and the inter encoder (730). For example, the inter encoder (730) can generate decoded blocks based on the decoded residual data and inter-frame prediction information, and the intra encoder (722) can generate decoded blocks based on the decoded residual data and intra-frame prediction information. The decoded blocks are processed appropriately to generate decoded images, which can be buffered in a memory circuit (not shown) and used as reference images.

[0103] An entropy encoder (725) can be configured to format a bitstream to include coded blocks and perform entropy encoding. The entropy encoder (725) is configured to include various types of information in the bitstream. For example, the entropy encoder (725) can be configured to include general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. Residual information may not be present when encoding blocks in inter-frame mode or in a merged sub-mode of bidirectional prediction mode.

[0104] Figure 8 A diagram of an example video decoder (810) according to another embodiment of this disclosure is shown. The video decoder (810) is configured to receive an encoded picture as part of an encoded video sequence and decode the encoded picture to generate a reconstructed picture. In the example, the video decoder (810) can be used instead of Figure 4 The example video decoder (410).

[0105] exist Figure 8 In the example, the video decoder (810) includes, for example, Figure 8 The example arrangement shows an entropy decoder (871), an inter-frame decoder (880), a residual decoder (873), a reconstruction module (874), and an intra-frame decoder (872) coupled together.

[0106] The entropy decoder (871) can be configured to reconstruct certain symbols from the encoded picture, which represent the syntax elements constituting the encoded picture. Such symbols may include, for example, the mode encoding the block (e.g., intra-mode, inter-mode, bidirectional prediction mode, merged sub-mode, or another sub-mode), prediction information (e.g., intra-prediction information or inter-prediction information) that can identify certain samples or metadata used by the intra-decoder (872) or inter-decoder (880) for prediction, residual information in the form of, for example, quantized transform coefficients, etc. In the example, when the prediction mode is inter-mode or bidirectional prediction mode, inter-prediction information is provided to the inter-decoder (880); and when the prediction type is intra-prediction type, intra-prediction information is provided to the intra-decoder (872). The residual information may be inversely quantized and provided to the residual decoder (873).

[0107] The inter-frame decoder (880) can be configured to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0108] The intra-frame decoder (872) can be configured to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0109] The residual decoder (873) can be configured to perform inverse quantization to extract the dequantized transform coefficients and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) can also utilize certain control information (including quantizer parameters (QP)), which can be provided by the entropy decoder (871) (the data path is not depicted because this may only be low-data-volume control information).

[0110] The reconstruction module (874) can be configured to combine the residual output by the residual decoder (873) with the prediction result (depending on the situation, output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block, which forms part of a reconstructed image, which is part of a reconstructed video. It should be noted that other suitable operations, such as deblocking, can also be performed to improve visual quality.

[0111] It should be noted that any suitable technology can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810). In some example embodiments, one or more integrated circuits can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810). In another embodiment, one or more processors executing software instructions can be used to implement the video encoder (403).

[0112] (603) and (703) and video decoders (410), (510) and (810).

[0113] Turning to block partitioning for encoding and decoding, general partitioning can begin with basic blocks and can follow a predefined set of rules, a specific pattern, a partition tree, or any partitioning structure or scheme. Partitioning can be hierarchical and recursive. After splitting or partitioning the basic blocks according to any of the example partitioning processes described below, or other processes, or combinations thereof, a set of final partitions or coded blocks can be obtained. Each of these partitions can be at one of various partitioning levels in the partitioning hierarchy and can have various shapes. Each of the partitions can be called a coded block (CB). For the various example partitioning implementations further described below, each resulting CB can be of any allowed size and partitioning level. Such partitions are called coded blocks because they can form units against which some basic encoding / decoding decisions can be made and can be optimized, determined, and signaled to the encoding / decoding parameters in the encoded video bitstream. The highest or deepest level in the final partition represents the depth of the tree-like coded block partitioning structure. Coded blocks can be luma-coded blocks or chroma-coded blocks. The CB tree structure for each color can be called a coded block tree (CBT).

[0114] The coding blocks for all color channels can be collectively referred to as coding units (CUs). The hierarchical structure of all color channels can be collectively referred to as coding tree units (CTUs). The partitioning patterns or structures of various color channels within a CTU may or may not be the same.

[0115] In some implementations, the partitioning schemes or structures used for the luma and chroma channels may not need to be the same. In other words, the luma and chroma channels can have separate coding tree structures or patterns. Furthermore, whether the luma and chroma channels use the same or different coding tree structures, and the actual coding tree structure used, can depend on whether the slice being encoded is a P-slice, a B-slice, or an I-slice. For example, for an I-slice, the chroma and luma channels can have separate coding tree structures or coding tree structure patterns, while for P-slices or B-slices, the luma and chroma channels can share the same coding tree scheme. When applying separate coding tree structures or patterns, the luma channel can be partitioned into a CB by one coding tree structure, while the chroma channel can be partitioned into a chroma CB by another coding tree structure.

[0116] In some example implementations, a predefined partitioning pattern can be applied to basic blocks. For example... Figure 9As shown, the example 4-way partitioning tree can start from a first predefined level (e.g., a 64×64 block level or another size, as the basic block size), and the basic blocks can be partitioned hierarchically down to the lowest predefined level (e.g., a 4×4 level). For example, the basic blocks can go through four predefined partitioning options or patterns indicated by 902, 904, 906, and 908, where partitions designated as R are allowed for recursive partitioning because... Figure 9 The same partitioning options indicated in the document can be repeated at lower levels, up to the lowest level (e.g., a 4x4 level). In some implementations, this can be... Figure 9 The partitioning scheme applies additional restrictions. Figure 9 The implementation allows rectangular partitioning (e.g., 1:2 / 2:1 rectangular partitioning), but recursive rectangular partitioning is not allowed; recursive square partitioning is permitted. If needed, according to... Figure 9 The recursive partitioning generates the final set of coded blocks. The depth of the coding tree can be further defined to indicate the depth at which the partition originates from the root node or root block. For example, the depth of the coding tree for the root node or root block, such as a 64×64 block, can be set to 0, and the depth of the root block can be determined according to... Figure 9 After a further split, the coding tree depth increases by 1. For the above scheme, the maximum or deepest level from the 64×64 basic blocks to the 4×4 minimum partition will be 4 (starting from level 0). This partitioning scheme can be applied to one or more color channels. Each color channel can be divided according to... Figure 9 The partitioning scheme is performed independently (e.g., the partitioning mode or option in a predefined pattern can be determined independently for each color channel at each level). Alternatively, two or more color channels can share... Figure 9 The same hierarchical pattern tree (for example, the same partitioning pattern or option in a predefined pattern can be selected for two or more color channels at each level).

[0117] Figure 10 Another example of a predefined partitioning pattern that allows recursive partitioning to form a partition tree is shown. For example... Figure 10 As shown, the example 10-way partitioning structure or pattern can be predefined. The root block can start at a predefined level (e.g., a basic block at a 128×128 level or a 64×64 level). Figure 10 The example partitioning structures include various 2:1 / 1:2 and 4:1 / 1:4 rectangular partitions. Figure 10 The partitioning type with three sub-partitions, indicated as 1002, 1004, 1006, and 1008 in the second row, can be called a "T-shaped" partition. The "T-shaped" partitions 1002, 1004, 1006, and 1008 can be referred to as left T-shaped, top T-shaped, right T-shaped, and bottom T-shaped. In some example implementations, Figure 10 Rectangular partitions are not allowed to be further subdivided. The encoding tree depth can also be defined to indicate the split depth starting from the root node or root block. For example, the encoding tree depth of the root node or root block (e.g., a 128×128 block) can be set to 0, and the split depth of the root block can be determined according to... Figure 10 After being further split, the coding tree depth increases by 1. In some implementations, only full square partitions within 1010 are allowed to be divided according to... Figure 10 The pattern is recursively partitioned to the next level of the partition tree. In other words, for square partitions within T-patterns 1002, 1004, 1006, and 1008, recursive partitioning may not be allowed. If necessary, according to Figure 10 The recursive partitioning process generates the final set of coded blocks. This scheme can be applied to one or more color channels. Some implementations offer greater flexibility for using partitions below 8x8 levels. For example, 2x2 chroma inter-frame prediction can be used in certain cases.

[0118] In some other implementations of block partitioning, quadtree structures can be used to split basic blocks or intermediate blocks into quadtree partitions. Such quadtree splitting can be applied hierarchically and recursively to any square-shaped partition. Whether a basic block, intermediate block, or partition is further quadtree-split can be adapted to various local characteristics of the basic block or intermediate block / partition. Quadtree partitioning at image boundaries can be further adjusted. For example, implicit quadtree splitting can be performed at image boundaries, such that blocks will maintain quadtree splitting until their size fits the image boundaries.

[0119] In some other example implementations, hierarchical binary partitioning starting from a base block can be used. In such a scheme, the base block or intermediate level block can be divided into two partitions. The binary partition can be horizontal or vertical. For example, a horizontal binary partition can split the base block or intermediate block into equal right and left partitions. Similarly, a vertical binary partition can split the base block or intermediate block into equal upper and lower partitions. Such binary partitioning can be hierarchical and recursive. At each point in the base block or intermediate block, it can be decided whether the binary partitioning scheme should continue, and if so, whether to use a horizontal or vertical binary partition. In some implementations, further partitioning can stop at a predefined minimum partition size (in one or two dimensions). Alternatively, further partitioning can stop once a predefined partitioning level or depth starting from the base block is reached. In some implementations, the aspect ratio of the partitions can be restricted. For example, the aspect ratio of the partitions cannot be less than 1:4 (or greater than 4:1). Therefore, a vertical strip partition with a vertical to horizontal aspect ratio of 4:1 can be further divided into an upper partition and a lower partition only in the vertical direction, with each partition having a vertical to horizontal aspect ratio of 2:1.

[0120] In other examples, the ternary partitioning scheme can be used to partition basic blocks or any intermediate blocks, such as... Figure 13 As shown in the diagram. This ternary pattern can be as follows: Figure 13 The vertical implementation shown in 1302, or as... Figure 13 The level implementation shown in 1304. Although Figure 13 The example vertical or horizontal split ratio is shown as 1:2:1, but other ratios can be predefined. In some implementations, two or more different ratios can be predefined. Such a ternary partitioning scheme can be used to complement quadtree or binary partitioning structures because such a ternary partition can capture objects located at the center of a block in a continuous partition, while quadtrees and binary trees always split along the center of the block and thus split objects into different partitions. In some implementations, the width and height of the partitions in the example ternary tree are always powers of 2 to avoid additional transformations.

[0121] The partitioning schemes described above can be combined in any way at different partitioning levels. As an example, the quadtree and binary partitioning schemes described above can be combined to partition basic blocks into a quadtree-binary tree (QTBT) structure. In such a scheme, basic blocks or intermediate blocks / partitions can be quadtree-split or binary-split, depending on a set of predefined conditions, if specified. Figure 14 A specific example is shown in the image. Figure 14In the examples, the basic block is first split into four partitions by a quadtree, as shown in 1402, 1404, 1406, and 1408. Subsequently, each of the resulting partitions is either split into four additional partitions by the quadtree (as in 1408), or binary split into two additional partitions at the next level (horizontally or vertically, as in 1402 or 1406, both of which are symmetrical, for example), or not split at all (as in 1404). For partitions that are square in shape, recursive binary or quadtree splits are allowed, as shown in the overall example partitioning pattern of 1410 and the corresponding tree structure / representation in 1420, where solid lines represent quadtree splits and dashed lines represent binary splits. Flags can be used for each binary split node (non-leaf binary partition) to indicate whether the binary split is horizontal or vertical. For example, as shown in 1420, consistent with the partitioning structure of 1410, the flag "0" can represent a horizontal binary split, and the flag "1" can represent a vertical binary split. For quadtree partitioning, it is not necessary to specify the partition type, because quadtree partitioning always splits blocks or partitions in both the horizontal and vertical directions to produce 4 sub-blocks / partitions of the same size. In some implementations, the flag "1" can represent a horizontal binary partition, while the flag "0" can represent a vertical binary partition.

[0122] In some example implementations of QTBT, quadtrees and binary splitting rule sets can be represented by the following predefined parameters and associated functions:

[0123] -CTU size: The size of the root node of the quadtree (the size of the basic block).

[0124] -MinQTSize: The minimum allowed size of a quadtree leaf node.

[0125] -MaxBTSize: The maximum allowed size of the root node of the binary tree.

[0126] -MaxBTDepth: Maximum allowed binary tree depth

[0127] -MinBTSize: The minimum allowed size of a binary leaf node.

[0128] In some example implementations of the QTBT partitioning structure, the CTU size can be set to 128×128 luminance samples with two corresponding 64×64 chrominance sample blocks (when considering and using example chrominance subsampling), MinQTSize can be set to 16×16, MaxBTSize can be set to 64×64, MinBTSize (for both width and height) can be set to 4×4, and MaxBTDepth can be set to 4. Quadtree partitioning can first be applied to the CTU to generate quadtree leaf nodes. The size of the quadtree leaf nodes can range from their minimum allowed size of 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If a node is 128×128, it will not be partitioned into a binary tree first because its size exceeds MaxBTSize (i.e., 64×64). Otherwise, nodes not exceeding MaxBTSize can be partitioned into a binary tree. Figure 14 In the example, the basic block is 128×128. According to the predefined rule set, the basic block can only be split into quadtrees. The partitioning depth of the basic block is 0. Each of the four resulting partitions is 64×64, not exceeding MaxBTSize, and can be further split into quadtrees or binary trees at level 1. The process continues. When the binary tree depth reaches MaxBTDepth (i.e., 4), further splitting can be disregarded. When the width of a binary tree node equals MinBTSize (i.e., 4), further horizontal splitting may not be considered. Similarly, when the height of a binary tree node equals MinBTSize, further vertical splitting is not considered.

[0129] In some example implementations, the QTBT scheme described above can be configured to support the flexibility of having the same QTBT structure for luma and chroma, or separate QTBT structures. For example, for P-slices and B-slices, the luma CTB and chroma CTB in a CTU can share the same QTBT structure. However, for I-slices, the luma CTB can be divided into CBs by a QTBT structure, while the chroma CTB can be divided into chroma CBs by another QTBT structure. This means that CUs can be used to refer to different color channels in an I-slice; for example, an I-slice can consist of coded blocks for the luma component or coded blocks for the two chroma components, while a CU in a P-slice or B-slice can consist of coded blocks for all three color components.

[0130] In some other implementations, the QTBT scheme can be supplemented by the aforementioned ternary scheme. Such an implementation can be called a Multi-Type Tree (MTT) structure. For example, in addition to binary splitting of nodes, one can also choose... Figure 13This is one of the ternary partitioning patterns. In some implementations, only square nodes may undergo ternary partitioning. Additional flags can be used to indicate whether the ternary partition is horizontal or vertical.

[0131] The primary motivation for designing two- or multi-level trees (e.g., QTBT implementations and QTBT implementations supplemented by ternary partitioning) is likely to reduce complexity. Theoretically, the complexity of traversing a tree is T0. D Where T represents the number of split types and D is the depth of the tree. A tradeoff can be made by using multiple types (T) while simultaneously reducing the depth (D).

[0132] In some implementations, the CB can be further subdivided. For example, for the purpose of intra-frame or inter-frame prediction during encoding and decoding, the CB can be further subdivided into multiple prediction blocks (PBs). In other words, the CB can be further divided into different sub-partitions in which separate prediction decisions / configurations can be made. In parallel, for the purpose of defining the levels at which transforms or inverse transforms of the video data are performed, the CB can be further subdivided into multiple transform blocks (TBs). The schemes for dividing the CB into PBs and TBs can be the same or different. For example, each partitioning scheme can be performed using its own process based on, for example, various features of the video data. In some example implementations, the PB partitioning scheme and the TB partitioning scheme can be independent. In some other example implementations, the PB partitioning scheme and the TB partitioning scheme can be associated with boundaries. In some implementations, for example, TBs can be partitioned after PB partitioning, and in particular, each PB, after being determined according to the partitioning of the coding blocks, can then be further subdivided into one or more TBs. For example, in some implementations, a PB can be split into one, two, four, or other numbers of TBs.

[0133] In some implementations, the luma and chroma channels can be treated differently to divide a basic block into coded blocks and further into prediction and / or transform blocks. For example, in some implementations, dividing a coded block into prediction and / or transform blocks may be permissible for the luma channel, while such division may not be permissible for the chroma channel. In such implementations, the transform and / or prediction of the luma block can therefore be performed only at the coded block level. For another example, the minimum transform block size for the luma and chroma channels can be different; for example, the coded block for the luma channel may be allowed to be divided into smaller transform and / or predict blocks compared to the chroma channel. For yet another example, the maximum depth of dividing a coded block into transform and / or predict blocks can be different between the luma and chroma channels; for example, the coded block for the luma channel may be allowed to be divided into deeper transform and / or predict blocks compared to the chroma channel. For a specific example, the luma coding block can be divided into transform blocks of various sizes, which can be represented by recursive partitioning down to two levels, and can allow transform block shapes such as square, 2:1 / 1:2, and 4:1 / 1:4, as well as transform block sizes from 4×4 to 64×64. However, for the chroma block, only the largest possible transform block specified for the luma block may be allowed.

[0134] In some example implementations of dividing a coded block into a PB, the depth, shape, and / or other characteristics of the PB division may depend on whether the PB is intra-coded or inter-coded.

[0135] Dividing a coded block (or prediction block) into transform blocks can be implemented using various example schemes, including but not limited to recursive or non-recursive quadtree splitting and predefined pattern splitting, with additional consideration given to transform blocks at the boundaries of the coded or prediction blocks. Typically, the resulting transform blocks can be at different splitting levels, can have different sizes, and do not need to be square (e.g., the resulting transform blocks can be rectangles with certain allowed sizes and aspect ratios). The following discusses... Figure 15 , Figure 16 and Figure 17 Further details on other examples are provided.

[0136] However, in some other implementations, the CB obtained through any of the above partitioning schemes can be used as the basic or minimum coding block for prediction and / or transform. In other words, no further splitting is performed for inter-frame / intra-frame prediction and / or transform purposes. For example, the CB obtained from the above QTBT scheme can be directly used as the unit for performing prediction. Specifically, such a QTBT structure eliminates the concept of multiple partitioning types, i.e., it eliminates the separation of CU, PU, ​​and TU, and supports greater flexibility in the CU / CB partitioning shape as described above. In such a QTBT block structure, CU / CB can have a square or rectangular shape. The leaf nodes of such a QTBT are used as units for prediction and transform processing without any further partitioning. This means that in such an example QTBT coding block structure, CU, PU, ​​and TU have the same block size.

[0137] The various CB partitioning schemes described above, as well as further partitioning CB into PB and / or TB (including no PB / TB partitioning), can be combined in any way. The following specific implementations are provided as non-limiting examples.

[0138] The following describes a specific example implementation of the partitioning of coded blocks and transform blocks. In such an example implementation, the recursive quadtree splitting described above or a predefined splitting pattern (such as...) can be used. Figure 9 and Figure 10 The basic block (CB) is split into coding blocks. At each level, whether further quadtree splitting of a particular partition should be performed can be determined by the characteristics of the local video data. The resulting CBs can be at different quadtree splitting levels and can have different sizes. A decision can be made at the CB level (or CU level, for all three color channels) regarding whether to use inter-frame picture (temporal) prediction or intra-frame picture (spatial) prediction to encode the picture region. Each CB can be further split into one, two, four, or other numbers of PBs according to a predefined PB splitting type. Within a PB, the same prediction process can be applied, and relevant information can be transmitted to the decoder based on the PB. After obtaining the residual block by applying the prediction process based on the PB splitting type, the CB can be divided into TBs according to another quadtree structure similar to the coding tree used for the CB. In this particular implementation, the CB or TB can be square in shape, but is not limited to square shapes. Furthermore, in this particular example, for inter-frame prediction, the PB can be square or rectangular in shape, but for intra-frame prediction, the PB can be only square. The coding block can be split into, for example, four square-shaped TBs. Each TB can be further recursively split into smaller TBs (using quadtree splitting), which are called residual quadtrees (RQTs).

[0139] The following section further describes another example implementation of dividing basic blocks into CBs, PBs, and / or TBs. For example, instead of using multi-partitioning unit types such as... Figure 9 or Figure 10 The multi-partitioning unit type shown can utilize a quadtree with nested multi-type trees using binary and ternary splitting structures (e.g., QTBT or QTBT with the aforementioned ternary splitting). Separation of CB, PB, and TB (i.e., splitting a CB into PB and / or TB, and splitting a PB into TB) can be omitted unless further splitting is needed for CBs whose size is too large for the maximum transform length. This example partitioning scheme can be designed to support greater flexibility in CB partitioning shapes, allowing both prediction and transformation to be performed at the CB level without further partitioning. In such a coding tree structure, the CB can have a square or rectangular shape. Specifically, the coding tree block (CTB) can first be partitioned by a quadtree structure. Then, the quadtree leaf nodes can be further partitioned by nested multi-type tree structures. Examples of nested multi-type tree structures using binary or ternary splitting are shown in [the original text]. Figure 11 shown in . Specifically, Figure 11 The example multi-type tree structure includes four splitting types, referred to as vertical binary split (SPLIT_BT_VER) (1102), horizontal binary split (SPLIT_BT_HOR) (1104), vertical ternary split (SPLIT_TT_VER) (1106), and horizontal ternary split (SPLIT_TT_HOR) (1108). CB then corresponds to the leaves of the multi-type tree. In this example implementation, unless CB is too large for the maximum transform length, the split is used for both prediction and transform processing without any further partitioning. This means that, in most cases, CB, PB, and TB have the same block size in a quadtree with nested multi-type tree coding block structures. Exceptions occur when the maximum supported transform length is less than the width or height of the color component of CB. In some implementations, in addition to binary or ternary splits, Figure 11 Nested patterns can also include quadtree splitting.

[0140] Figure 12 This illustrates a concrete example of a quadtree with a nested, multi-type tree-coded block structure, including quadtree, binary, and ternary splitting options, for the block partitioning of a basic block. More detailed... Figure 12 The diagram shows that basic block 1200 is split into four square partitions 1202, 1204, 1206, and 1208 by a quadtree. Further uses are then made for each of these quadtree partitions. Figure 11 The decision to further split the multi-type tree structure and quadtree. Figure 12In the example, partition 1204 was not further split. Partitions 1202 and 1208 were each split using a different quadtree. For partition 1202, the second-level quadtree splits of the top-left, top-right, bottom-left, and bottom-right partitions were respectively split using the third-level quadtree splits. Figure 11 Horizontal binary split 1104, no split and Figure 11 The horizontal ternary partition is 1108. Partition 1208 uses another quadtree partition, and the second-level quadtree partitions the top-left, top-right, bottom-left, and bottom-right partitions respectively use... Figure 11 Vertical ternary splitting 1106, third-level splitting, no splitting, no splitting and Figure 11 The horizontal binary split 1104.1208's third-level top-left partition's two subpartitions are respectively based on... Figure 11 The horizontal binary partition 1104 and the horizontal ternary partition 1108 were further subdivided. Partition 1206 followed... Figure 11 The vertical binary partitioning 1102 is divided into two partitions based on the second-level partitioning pattern. Figure 11 The horizontal ternary partition 1108 and the vertical binary partition 1102 are further partitioned at the third level. According to Figure 11 The horizontal binary split 1104 is further applied to one of them by applying a fourth-level split.

[0141] For the specific example above, the maximum luminance transformation size can be 64×64, and the maximum supported chrominance transformation size can be different from the luminance, for example, it can be 32×32. Despite the above... Figure 12 The example CB in the example is not usually further split into smaller PB and / or TB, but when the width or height of the luma-coded block or chroma-coded block is greater than the maximum transform width or height, the luma-coded block or chroma-coded block can be automatically split in the horizontal and / or vertical directions to meet the transform size limit in that direction.

[0142] In the specific example of dividing a basic block into CBs described above, and as mentioned above, the coding tree scheme can support the ability for luma and chroma to have separate block tree structures. For example, for P-slices and B-slices, the luma CTB and chroma CTB in a CTU can share the same coding tree structure. For example, for I-slices, luma and chroma can have separate coding block tree structures. When applying separate block tree structures, the luma CTB can be divided into luma CBs using one coding tree structure, and the chroma CTB into chroma CBs using another coding tree structure. This means that a CU in an I-slice can include coding blocks for the luma component or coding blocks for both chroma components, and unless the video is monochrome, a CU in a P-slice or B-slice always includes coding blocks for all three color components.

[0143] When a coded block is further divided into multiple transform blocks, these transform blocks can be ordered in the bitstream in various sequences or scanning methods. The following details example implementations of dividing a coded block or prediction block into transform blocks and the encoding order of these transform blocks. In some example implementations, as mentioned above, transform partitioning can support transform blocks of various shapes, such as 1:1 (square), 1:2 / 2:1, and 1:4 / 4:1, with transform block sizes ranging from, for example, 4×4 to 64×64. In some implementations, if the coded block is less than or equal to 64×64, then transform block partitioning can be applied only to the luma component, such that for the chroma block, the transform block size is the same as the coded block size. Additionally, if the coded block width or height is greater than 64, then the luma and chroma coded blocks can be implicitly split into multiple min(W,64)×min(H,64) and min(W,32)×min(H,32) transform blocks, respectively.

[0144] In some example implementations of transform block partitioning, for both intra-frame and inter-frame coding blocks, the coding block can be further divided into multiple transform blocks, with the partitioning depth reaching a predefined number of levels (e.g., 2 levels). The transform block partitioning depth and size can be related. For some example implementations, Table 1 below shows the mapping from the transform size at the current depth to the transform size at the next depth.

[0145] Table 1: Settings for changing partition size

[0146]

[0147] Based on the example mapping in Table 1, for a 1:1 square block, the next-level transform split can create four 1:1 square sub-transform blocks. The transform split can stop at, for example, 4×4. Therefore, the transform size of 4×4 at the current depth corresponds to the same size of 4×4 at the next depth. In the example in Table 1, for a 1:2 / 2:1 non-square block, the next-level transform split can create two 1:1 square sub-transform blocks, while for a 1:4 / 4:1 non-square block, the next-level transform split can create two 1:2 / 2:1 sub-transform blocks.

[0148] In some example implementations, additional constraints can be applied to the transform block partitioning for the luma component of an intra-coded block. For instance, for each level of transform partitioning, all sub-transform blocks can be constrained to have the same size. For example, for a 32×16 coded block, a level 1 transform split creates two 16×16 sub-transform blocks, and a level 2 transform split creates eight 8×8 sub-transform blocks. In other words, the second-level split must be applied to all first-level sub-blocks to maintain equal transform unit sizes. Figure 15Examples of transform block partitioning for intra-frame coded square blocks according to Table 1 are shown, along with the coding order indicated by arrows. Specifically, 1502 shows a square coded block. 1504 shows a first-level partitioning into four equal-sized transform blocks according to Table 1, along with the coding order indicated by arrows. 1506 shows a second-level partitioning into 16 equal-sized transform blocks according to Table 1, along with the coding order indicated by arrows.

[0149] In some example implementations, the aforementioned restrictions on intra-frame coding may not apply to the luma component of inter-frame coded blocks. For instance, after the first level of transform splitting, any one of the sub-transform blocks can be further split independently at more than one level. Therefore, the resulting transform blocks may have the same size or may not have the same size. Figure 16 The example of splitting inter-coded blocks into transform blocks and their encoding order is shown in the figure. Figure 16 In the example, according to Table 1, the inter-coded block 1602 is split into transform blocks at two levels. At the first level, the inter-coded block is split into four transform blocks of equal size. Then, only one of the four transform blocks (not all four) is further split into four sub-transform blocks, resulting in a total of seven transform blocks with two different sizes, as shown in 1604. The example coding order of these seven transform blocks is as follows: Figure 16 The arrow in 1604 is shown.

[0150] In some example implementations, additional constraints on the transform block can be applied to the chroma component. For example, for the chroma component, the transform block size can be the same as the coding block size, but not smaller than a predefined size such as 8×8.

[0151] In some other example implementations, for coded blocks whose width (W) or height (H) is greater than 64, both the luma coded block and the chroma coded block can be implicitly split into multiple min(W,64)×min(H,64) and min(W,32)×min(H,32) transform units, respectively. Here, in this disclosure, "min(a,b)" can return the smaller value between a and b.

[0152] Figure 17 Another alternative example scheme for dividing a coded block or prediction block into transform blocks is further illustrated. For example... Figure 17 As shown, instead of using recursive transformation partitioning, a predefined set of partitioning types can be applied to the coded block based on the transformation type of the coded block. Figure 17In the specific example shown, one of the six example partitioning types can be applied to split the coded block into various numbers of transform blocks. Such a scheme for generating transform block partitions can be applied to either coded blocks or prediction blocks.

[0153] In more detail, Figure 17 The partitioning scheme provides up to six example partition types for any given transform type (transform type refers to, for example, the type of main transform such as ADST and others). In this scheme, a transform partition type can be assigned to each coding block or prediction block based on, for example, rate distortion cost. In the examples, the transform partition type assigned to a coding block or prediction block can be determined based on the transform type of the coding block or prediction block. A specific transform partition type can correspond to a transform block split size and mode, such as... Figure 17 The six transform partition types described herein are illustrated. The correspondence between various transform types and various transform partition types can be predefined. An example is shown below, where uppercase labels indicate the transform partition type that can be assigned to a coding block or prediction block based on rate distortion cost:

[0154] • PARTITION_NONE: Allocates a transformation size equal to the block size.

[0155] • PARTITION_SPLIT: A transform size allocated to a width of 1 / 2 the block size and a height of 1 / 2 the block size.

[0156] • PARTITION_HORZ: Allocates a transform size with the same width as the block size and a height that is half the block size.

[0157] • PARTITION_VERT: Allocates a transform size with a width of half the block size and a height equal to the block size.

[0158] • PARTITION_HORZ4: Allocates a transform size with the same width as the block size and a height that is 1 / 4 of the block size.

[0159] • PARTITION_VERT4: Allocates a transform size with a width of 1 / 4 of the block size and a height equal to the block size.

[0160] In the example above, Figure 17 The transform partition types shown all contain uniform transform sizes for the transform blocks used in the partitioning. This is merely an example, not a limitation. In some other implementations, mixed transform block sizes may be used for transform blocks partitioned for a specific partition type (or pattern).

[0161] The PB (or CB, also called PB when not further divided into prediction blocks) obtained from any of the above partitioning schemes can then become individual blocks encoded by intra-frame prediction or inter-frame prediction. For inter-frame prediction of the current PB, a residual between the current block and the prediction block can be generated, encoded, and included in the encoded bitstream.

[0162] Turning to some example implementations of signaling for specific types of coding blocks / units, for each intra-coding unit and inter-coding unit, a signaling flag, namely the skip_txfm flag, can be used in the coded bitstream, as shown in the example syntax in Tables 2 through 4 and represented by the read_skip() function used to retrieve these flags from the bitstream. This flag can indicate whether all transform coefficients in the current coding unit are zero. In some example implementations, if the flag is signaled with, for example, a value of 1, then another transform coefficient-related syntax, such as EOB (End of Block), does not need to be signaled for any color-coded block within the coding unit, and a predefined value or data structure associated with zero transform coefficient blocks can be derived. For inter-coding blocks, as shown in the examples in Tables 2 through 4, the flag can be signaled after the skip_mode flag, which indicates that the coding unit can be skipped for various reasons. When skip_mode is true, the coding unit should be skipped, and no skip_txfm flag needs to be signaled, and the skip_texfm flag is inferred to be 1. Otherwise, if skip_mode is false, then more information about the coding unit will be included in the bitstream, and the skip_txfm flag will be additionally signaled to indicate whether the coding unit is all zero.

[0163] Table 2: Skip Modes and Skip Syntax: Intra-Frame Mode Information Syntax

[0164]

[0165] Table 3: Skip Modes and Skip Syntax: Intra-Frame Mode Information Syntax

[0166]

[0167] Table 4: Skip Modes and Skip Syntax: Skip Syntax

[0168]

[0169] In the following exemplary implementations, the term chroma channel can generally refer to both the Cb and Cr color components (or channels), or both the U and V color components (or channels). The term luminance channel can include a luminance component or a Y component. A luminance component or channel can be referred to as a luminance color component or channel. Y, U, and V are used below to represent the three color components. Furthermore, the terms "coded block" and "coded" block are used interchangeably to refer to a block to be encoded or a block that has already been encoded. They can be blocks of any of the three color components. Three corresponding color-coded / coded blocks can be used for coded / coded units.

[0170] Moving on to the encoding and decoding (entropy encoding) of the transform coefficients of the residuals in each color component, for each transform block, the transform coefficient encoding can begin with a signal indicating the skipped symbol, followed by the transform kernel type and the end-of-block (EOB) position when the skipped symbol is zero (indicating the presence of non-zero coefficients). Each coefficient value is then mapped to multiple level maps (amplitude maps) and symbols.

[0171] After the EOB position is encoded, the low-level and mid-level plots can be encoded in reverse scan order. The former indicates whether the coefficient amplitude is within the low-level range (e.g., between 0 and 2), while the latter indicates whether the range is within the mid-level range (e.g., between 3 and 14). The next step is to encode the sign of the coefficients and the residual values ​​of coefficients greater than the high-level (e.g., 14) using, for example, Exp-Golomb encoding, in forward scan order.

[0172] Regarding the use of context modeling, low-level graph coding can include transform magnitude and direction, as well as information on up to five neighboring coefficients. On the other hand, mid-level graph coding can follow a similar approach to low-level magnitude coding, except that the number of neighboring coefficients is reduced to a smaller number (e.g., two). Example Exp-Golomb coding for residual levels, where the sign of AC coefficients is encoded without any context model, and the sign of DC coefficients is encoded using the dc sign of their neighboring transform blocks.

[0173] In some exemplary implementations, chroma residuals can be jointly encoded. Such encoding schemes can be based on some statistical correlation between chroma channels. For example, in many cases, Cr and Cb chroma coefficients may be similar in amplitude and opposite in sign, so at the transform block level, for example, where the transform coefficients are signaled, joint encoding can be performed to improve encoding efficiency by introducing only small color distortion. The use (activation) of the joint chroma encoding mode can be indicated, for example, by a joint chroma encoding flag (e.g., the TU-level flag tu_joint_cbcr_residual_flag), and the selected joint mode can be implicitly indicated by the chroma-encoded / encoded block flag (CBF).

[0174] Specifically, the flag `tu_joint_cbcr_residual_flag` may appear if any one or both chroma CBFs of the TU (transform block) are equal to 1. In the PPS and slice headers, the chroma quantization parameter (QP) offset values ​​are signaled for the joint chroma residual coding mode, distinguishing them from the chroma QP offset values ​​signaled for the regular chroma residual coding mode. These chroma QP offset values ​​can be used to derive the chroma QP values ​​for blocks encoded using the joint chroma residual coding mode. When the corresponding joint chroma coding mode (Mode 2 in Table 5) is activated in the TU, this chroma QP offset can be added to the applied luminance-derived chroma QP during the quantization and decoding process of that TU. For other modes (Modes 1 and 3 in Table 5), the chroma QP can be derived in the same way as traditional Cb or Cr blocks. Table 5 describes the process of reconstructing the chroma residuals (resCb and resCr) from the transmitted transform block. When this mode is activated (modes 1-3), a single joint chroma residual block (resJointC[x][y] in Table 5) can be signaled, and the residual blocks of Cb (resCb) and Cr (resCr) can be derived taking into account information such as tu_cbf_cb, tu_cbf_cr, and CSign, where CSign is a sign value specified, for example, in the slice header rather than at the transform block level. In some implementations, CSign may be -1 most of the time. In Table 5, the value CSign is a sign value (+1 or -1) that can be specified in the slice header, while resJointC[][] is the transmitted residual.

[0175] The three exemplary joint chroma coding modes described above can be supported only in intra-frame coding CUs. In inter-frame coding CUs, only mode 2 can be supported. Therefore, for inter-frame coding CUs, the syntax element tu_joint_cbcr_residual_flag appears only when both chroma CBFs are 1.

[0176] Table 5. Reconstruction of chromaticity residuals

[0177]

[0178] The joint chroma coding scheme above assumes a certain correlation between the transform coefficients between the juxtaposed Cr and Cb transform blocks. These assumptions are generally statistical and can therefore introduce distortion in some cases. In particular, when one of the color coefficients in the transform block is non-zero while the coefficient of the other color component is zero, some of the assumptions made in the joint chroma coding scheme will certainly not hold, and such coding will not save any coding bits (because one chroma coefficient is zero anyway).

[0179] Among the various exemplary implementations described below, a coefficient-level (i.e., transform-coefficient-by-coefficient) cross-component coding scheme is described that leverages some correlation between the juxtaposed transform coefficients of color components (juxtaposed in the frequency domain). Such a scheme is particularly useful for transform blocks (or units) where one color component has zero coefficients and the corresponding transform coefficients of another color component are non-zero. For those zero and non-zero color coefficient pairs, before or after dequantization, the non-zero color coefficients can be used to estimate the original small value of the zero-coded coefficient of the other color component (which may have initially been non-zero, albeit small, before quantization during encoding), potentially recovering some information lost, for example, during quantization. Due to the statistically existing inter-color correlations, some information lost during zero-quantization may be recovered. Such cross-component coding will recover lost information to some extent without significant (zero-coefficient) coding costs.

[0180] Specifically, a cross-component coefficient sign encoding method can be implemented, which uses the coefficient sign value of the first color component to encode the coefficient sign of the second color component. In a more specific example, the sign value of the Cb transform coefficient can be used as a background for encoding the signs of other Cr transform coefficients. Such cross-component encoding can be implemented on pairs of transform coefficients of the color components, coefficient by coefficient. The basic principle of such implementation and other implementations described in further detail below is not limited to the Cr and Cb components. They are applicable between any two of the three color components. In this regard, the luminance channel is considered one of the color components.

[0181] In some implementations, cross-component coefficient sign encoding methods may include encoding the coefficient signs of the second color component using the coefficient sign values ​​of the first color component. As a non-limiting example, the sign values ​​of the Cb transform coefficients can be used as a background for encoding the sign values ​​of the Cr transform coefficients.

[0182] In some implementations, the method may include deriving an offset value using the level value and / or sign value of the transform coefficients of the first color component, and adding that offset value to the level value of the transform coefficients of the second color component. This method is called Cross-Component Level Reconstruction (CCLR).

[0183] The exemplary implementations below can be used individually or in any order. The term "block size" can refer to the width or height of the block, or the maximum of the width and height, or the minimum of the width and height, or the area size (width * height), or the aspect ratio of the block (width:height, or height:width). The term "level value" or "level" can refer to the magnitude of the transformation coefficient value.

[0184] There may be some issues / challenges associated with certain implementations. For example, in cross-component coefficient symbol encoding methods for more efficient coefficient symbol encoding, some implementations of cross-component coefficient symbol encoding may be limited to cases where there is a pair of non-zero coefficients for multiple color components at the same frequency, since symbol encoding is not required for zero coefficients.

[0185] In some implementations—where only one of the Cb and Cr transform coefficients is non-zero—the method derives an offset value using the level value and / or sign value of the transform coefficient of the first color component, and adds that offset value to the level value of the transform coefficient of the second color component. Various embodiments described in this disclosure depict methods for deriving the offset value.

[0186] Figure 18 A flowchart 1800 illustrates an example method for deriving offset values ​​in cross-component transform coefficient level reconstruction, following the basic principles of the implementation described above. This example decoding method flow begins at S1801 and may include some or all of the following steps: S1810, receiving an encoded video bitstream; S1820, extracting a first transform coefficient for a first color component of the current video block from the encoded video bitstream; S1830, determining a second transform coefficient for a second color component of the current video block; S1840, deriving an offset value based on at least one of: a quantization step size, or the first transform coefficient; S1850, adding the offset value to the second transform coefficient to generate a modified second transform coefficient for the second color component; and / or S1860, decoding the current video block based on the first transform coefficient for the first color component and the modified second transform coefficient for the second color component. The example method terminates at S1899.

[0187] In some implementations, Figure 18 S1840 in the code may include deriving offset values ​​based on the quantization step size, wherein the magnitude of the offset value monotonically depends on the quantization step size. The offset values ​​of the transform coefficients of the second color component can be derived based on the quantization step size.

[0188] For a non-limiting example, the quantization step size can be the same for the first color component and the second color component (e.g., Cr and Cb). For another non-limiting example, the quantization step size can be different for the first color component and the second color component (e.g., a chroma channel and a luminance channel), and the quantization step size for the second color component can be chosen to determine the offset value of the second color component.

[0189] In some implementations, the magnitude of the offset value has a monotonic relationship with the quantization step size, which can be expressed by a monotonic function used to derive the offset value. In one example, the magnitude of the offset value has a linear relationship with the quantization step size (e.g., offset = k * quantization_step_size + offset_zero). In another example, the magnitude of the offset value has a polynomial relationship with the quantization step size. In some implementations, this relationship (or the function expressing it) can be chosen based on the sign or magnitude of the first transform coefficients.

[0190] In some implementations, Figure 18 S1840 can include deriving offset values ​​based on the sign or amplitude values ​​of the quantization step size and the first transform coefficient.

[0191] In some implementations, Figure 18 S1840 may include deriving an offset value based on a quantization step size corresponding to a DC coefficient in response to the offset value corresponding to a DC coefficient; and / or deriving an offset value based on a quantization step size corresponding to an AC coefficient in response to the offset value corresponding to an AC coefficient. For a non-limiting example, the quantization step size may be different for the DC and AC coefficients, and the offset values ​​for the DC and AC coefficients may be derived based on the quantization step sizes for the DC and AC coefficients, respectively. When using a monotonic relationship, a larger quantization step size results in a larger derived offset value.

[0192] In some implementations, Figure 18 S1840 may include mapping an index in a lookup table (LUT) based on a quantization step size; and obtaining the element in the LUT indicated by that index as an offset value.

[0193] In some implementations, the LUT includes one of the following: a fixed table, a dynamic table signaled in the encoded video bitstream, and / or a dynamic table derived from the encoded video bitstream.

[0194] For a non-restrictive example, the magnitude of the offset value is obtained from a lookup table (LUT), and the quantization step size is mapped to an index in the lookup table, using the associated offset. In some cases, the LUT may be a hard-coded fixed table; in others, it may be a dynamic table explicitly signaled; and / or in still others, it may be a dynamic table implicitly derived using encoded information.

[0195] In some implementations, the LUT is selected based on the sign or magnitude of the first transform coefficient. A separate monotonic function and / or LUT is used based on the sign and / or magnitude of the transform coefficients of the first color component.

[0196] In some implementations, the LUT is derived from a high-level syntax that includes at least one of the following: Video Parameter Set (VPS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), Picture header, Frame header, Slice header, Patch header, or Code Tree Unit (CTU) header.

[0197] In some implementations, Figure 18 S1840 may include an offset value derived based on the bit depth and quantization step size of the encoded video bitstream, wherein the bit depth of the encoded video bitstream includes one of 8 bits, 10 bits, or 12 bits; and the offset value monotonically depends on the bit depth of the encoded video bitstream. For example, the larger the bit depth of the encoded video bitstream, the larger the derived offset value.

[0198] In the embodiments and implementations of this disclosure, any steps and / or operations may be combined or arranged in any quantity or order as needed. Two or more steps and / or operations may be performed in parallel. The embodiments and implementations of this disclosure may be used individually or in combination in any order. Furthermore, each of the method (or implementation), encoder, and decoder may be implemented by a processing circuitry system (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium. The embodiments of this disclosure may be applied to luma blocks or chroma blocks. The term "block" may be interpreted as a prediction block, coding block, or coding unit, i.e., CU. The term "block" may also be used to refer to a transform block. In the following items, when referring to block size, it may refer to block width or block height, or the maximum value of width and height, or the minimum value of width and height, or area size (width * height), or block aspect ratio (width:height, or height:width).

[0199] The above-described techniques can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 19 A computer system (1900) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0200] Computer software can be encoded using any suitable machine code or computer language. The machine code or computer language can be assembled, compiled, linked, or otherwise used to create code containing instructions. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), or through interpretation, microcode execution, or other means.

[0201] The instructions can be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0202] Figure 19 The components shown for the computer system (1900) are exemplary in nature and are not intended to impose any limitation on the scope or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any one or a combination of the components shown in the exemplary embodiments of the computer system (1900).

[0203] The computer system (1900) may include certain human-computer interface input devices. Such human-computer interface input devices may respond to input from one or more human users via, for example, tactile input (e.g., keystrokes, swiping, movement of a data glove), audio input (e.g., speech, tapping), visual input (e.g., gestures), and olfactory input (not depicted). The human-computer interface device may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from still image capturing devices), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0204] Input human-machine interface devices may include one or more of the following (only one of each depicted): keyboard (1901), mouse (1902), touchpad (1903), touch screen (1910), data glove (not shown), joystick (1905), microphone (1906), scanner (1907), and camera device (1908).

[0205] The computer system (1900) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (e.g., tactile feedback via a touchscreen (1910), data gloves (not shown), or joystick (1905), but tactile feedback devices that are not used as input devices may also exist); audio output devices (e.g., speakers (1909), headphones (not depicted)); visual output devices (e.g., screens (1910), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capability, each with or without tactile feedback capability—some of which may be able to output two-dimensional or more than three-dimensional visual output in a manner such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and smoke generators (not depicted)); and printers (not depicted).

[0206] Computer systems (1900) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1920) with media such as CD / DVD (1921), thumb drives (1922), removable hard disk drives or solid-state drives (1923), legacy magnetic media such as magnetic tapes and floppy disks (not depicted), devices based on dedicated ROM / ASIC / PLD such as security dongles (not depicted), etc.

[0207] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0208] The computer system (1900) may also include interfaces (1954) to one or more communication networks (1955). Networks may be, for example, wireless networks, wired networks, or optical networks. Networks may also be local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), vehicular and industrial networks, real-time networks, latency-tolerant networks, etc. Examples of networks include: LANs (e.g., Ethernet, wireless LANs), cellular networks including GSM, 3G, 4G, 5G, LTE, etc., cable or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicular and industrial networks including CAN buses, etc. Some networks typically require external network interface adapters attached to certain general-purpose data ports or peripheral buses (1949) (e.g., the USB port of the computer system (1900)); others are typically integrated into the core of the computer system (1900) by attaching to the system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1900) can communicate with other entities. Such communication can be one-way receiving (e.g., broadcast television), one-way transmitting (e.g., a CAN bus to some CAN bus device), or bidirectional (e.g., using a local area digital network or a wide area digital network to other computer systems). Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0209] The aforementioned human-computer interface devices, human-accessible storage devices, and network interfaces could be attached to the core (1940) of a computer system (1900).

[0210] The core (1940) may include one or more central processing units (CPUs) (1941), graphics processing units (GPUs) (1942), dedicated programmable processing units in the form of field-programmable gate areas (FPGAs) (1943), hardware accelerators for certain tasks (1944), graphics adapters (1950), etc. These devices, along with read-only memory (ROM) (1945), random access memory (1946), and internal mass storage devices (e.g., internal non-user-accessible hard disk drives, SSDs, etc.) (1947), can be connected via a system bus (1948). In some computer systems, the system bus (1948) may be accessed as one or more physical plugs to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly or via a peripheral bus (1949) to the core's system bus (1948). In the example, a screen (1910) may be connected to a graphics adapter (1950). Peripheral bus architectures include PCI, USB, etc.

[0211] CPUs (1941), GPUs (1942), FPGAs (1943), and accelerators (1944) can execute certain instructions, which, when combined, constitute the computer code mentioned above. This computer code can be stored in ROM (1945) or RAM (1946). Temporary data can also be stored in RAM (1946), while permanent data can be stored, for example, in an internal mass storage device (1947). Fast storage and retrieval of any memory device within the memory apparatus can be achieved using a cache memory, which can be closely associated with one or more CPUs (1941), GPUs (1942), mass storage devices (1947), ROMs (1945), RAMs (1946), etc.

[0212] Computer-readable media may have computer code thereon for performing operations of various computer implementations. These media and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of a type well-known and available to those skilled in the art of computer software.

[0213] As a non-limiting example, a computer system with an architecture (1900), and particularly a core (1940), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software implemented in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage device as described above, and certain storage devices of the core (1940) having non-transitory characteristics, such as an internal mass storage device (1947) or ROM (1945) within the core. Software implementing various embodiments of this disclosure can be stored in such a device and executed by the core (1940). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can cause the core (1940), and particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (1946) and modifying such data structures according to the processes defined by the software. Alternatively or alternatively, the computer system may provide functionality as a result of hard-wired logic or otherwise implemented in circuitry (e.g., an accelerator (1944)), which may operate in place of or with the software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, reference to software may cover logic, and conversely, reference to logic may cover software. Where appropriate, reference to a computer-readable medium may include circuitry storing software for execution (e.g., an integrated circuit (IC)), circuitry embodying logic for execution, or both. This disclosure covers any suitable combination of hardware and software.

[0214] While several exemplary embodiments have been described in this disclosure, there are variations, substitutions, and various alternatives that fall within the scope of this disclosure. Therefore, it should be recognized that those skilled in the art will be able to conceive of many systems and methods that, while not expressly shown or described herein, implement the principles of this disclosure and are therefore within its spirit and scope.

[0215] Appendix A: Abbreviations

[0216] JEM: Joint Development Model

[0217] VVC: Universal Video Coding

[0218] BMS: Benchmark Set

[0219] MV: Motion Vector

[0220] HEVC: High-Efficiency Video Coding

[0221] SEI: Supplemental Enhancement Information

[0222] VUI: Video Availability Information

[0223] GOP: Image Group

[0224] TU: Transformation Unit

[0225] PU: Prediction Unit

[0226] CTU: Coding Tree Unit

[0227] CTB: Coded Tree Block

[0228] PB: Prediction Block

[0229] HRD: Hypothetical Reference Decoder

[0230] SNR: Signal-to-noise ratio

[0231] CPU: Central Processing Unit

[0232] GPU: Graphics Processing Unit

[0233] CRT: Cathode Ray Tube

[0234] LCD: Liquid Crystal Display

[0235] OLED: Organic Light Emitting Diode

[0236] CD: Compact Disc

[0237] DVD: Digital Video Disc

[0238] ROM: Read-Only Memory

[0239] RAM: Random Access Memory

[0240] ASIC: Application-Specific Integrated Circuit

[0241] PLD: Programmable Logic Device

[0242] LAN: Local Area Network

[0243] GSM: Global System for Mobile Communications

[0244] LTE: Long Term Evolution

[0245] CANBus: Controller Area Network Bus

[0246] USB: Universal Serial Bus

[0247] PCI: Peripheral Component Interconnect

[0248] FPGA: Field Programmable Gate Domain

[0249] SSD: Solid State Drive

[0250] IC: Integrated Circuit

[0251] HDR: High Dynamic Range

[0252] SDR: Standard Dynamic Range

[0253] JVET: Joint Video Experts Group

[0254] MPM: Most Likely Pattern

[0255] WAIP: Wide-angle Intra-frame Prediction

[0256] CU: Encoding Unit

[0257] PU: Prediction Unit

[0258] TU: Transformation Unit

[0259] CTU: Coding Tree Unit

[0260] PDPC: Location-Related Prediction Combination

[0261] ISP: Intra-Frame Sub-Partition

[0262] SPS: Sequence Parameter Settings

[0263] PPS: Image Parameter Set

[0264] APS: Adaptive Parameter Set

[0265] VPS: Video Parameter Set

[0266] DPS: Decoding Parameter Set

[0267] ALF: Adaptive Loop Filter

[0268] SAO: Sample Adaptive Offset; CC-ALF: Cross Component Adaptive Loop Filter; CDEF: Constraint Direction Enhancement Filter

[0269] CCSO: Cross Component Sample Offset

[0270] LSO: Local Sample Offset

[0271] LR: Loop Recovery Filter

[0272] AV1: AOMedia Video 1

[0273] AV2: AOMedia Video 2

[0274] MVD: Motion Vector Difference

[0275] CfL: Chromaticity derived from luminance

[0276] SDT: Semi-decoupled tree

[0277] SDP: Semi-decoupling partitioning

[0278] SST: Semi-separated tree

[0279] SB: Superblock

[0280] IBC (or IntraBC): Intra-block copying

[0281] CDF: Cumulative Density Function

[0282] SCC: Screen Content Encoding

[0283] GBI: Generalized Two-Way Forecasting

[0284] BCW: Bidirectional prediction with CU-level weights

[0285] CIIP: Combined Intra-Frame-Inter-Frame Prediction

[0286] POC: Image Sequential Counting

[0287] RPS: Reference Image Gallery

[0288] DPB: Decoding Image Buffer

[0289] CBF: Coded Block Flag

[0290] CCLR: Cross-component level reconstruction

Claims

1. A method for decoding a current video block of a video bitstream, characterized in that, include: The device receives encoded video bitstreams, the device including a memory for storing instructions and a processor in communication with the memory; The device extracts the first transform coefficient of the first color component of the current video block from the encoded video bitstream, and further extracts quantizer parameters, the quantizer parameters including the quantization step size for encoding the encoded video bitstream; The device determines the second transform coefficient of the second color component of the current video block, wherein the first transform coefficient and the second transform coefficient are jointly encoded into the encoded video bitstream, and the first transform coefficient and the second transform coefficient are co-located in the frequency domain; The device derives an offset value based on the quantization step size, wherein the magnitude of the offset value depends on the quantization step size according to a monotonic function, the monotonic function being selected based on the sign or magnitude of the first transform coefficient; The device adds the offset value to the second transformation coefficient to generate a modified second transformation coefficient for the second color component; and The device decodes the current video block based on a first transform coefficient of the first color component and a modified second transform coefficient of the second color component.

2. The method according to claim 1, characterized in that, Deriving the offset value includes: The offset value is derived based on the quantization step size and the sign or amplitude value of the first transform coefficient.

3. The method according to claim 1, characterized in that, Deriving the offset value includes: In response to the offset value corresponding to a DC coefficient, the offset value is derived based on the quantization step size corresponding to the DC coefficient; and In response to the offset value corresponding to the AC coefficient, the offset value is derived based on the quantization step size corresponding to the AC coefficient.

4. The method according to claim 1, characterized in that, Deriving the offset value includes: Based on the index in the quantization step size mapping lookup table (LUT); and The element in the LUT indicated by the index is obtained as the offset value.

5. The method according to claim 4, characterized in that: The LUT includes one of the following: Fixed table, The dynamic table notified by signals in the encoded video bitstream, or A dynamic table derived from the encoded video bitstream.

6. The method according to claim 4, characterized in that: The LUT is selected based on the sign or magnitude of the first transform coefficient.

7. The method according to claim 4, characterized in that: The LUT is obtained from a high-level syntax, which includes at least one of the following: Video Parameter Set (VPS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), Picture Header, Frame Header, Slice Header, Patch Header, or Code Tree Unit (CTU) Header.

8. The method according to claim 1, characterized in that, Deriving the offset value includes: The offset value is derived based on the bit depth of the encoded video bitstream and the quantization step size.

9. The method according to claim 8, characterized in that: The bit depth of the encoded video bitstream includes one of 8 bits, 10 bits, or 12 bits; and The offset value depends monotonically on the bit depth of the encoded video bitstream.

10. A method for encoding a current video block of a video bitstream, characterized in that, include: The device acquires the current video block, the device including a memory for storing instructions and a processor in communication with the memory; The device obtains the first transform coefficient of the first color component of the current video block and obtains quantizer parameters, the quantizer parameters including the quantization step size for encoding the current video block; The device determines the second transformation coefficient of the second color component of the current video block, wherein the first transformation coefficient and the second transformation coefficient are co-located in the frequency domain. The device derives an offset value based on the quantization step size, wherein the magnitude of the offset value depends on the quantization step size according to a monotonic function, the monotonic function being selected based on the sign or magnitude of the first transform coefficient; The device adds the offset value to the second transformation coefficient to generate a modified second transformation coefficient for the second color component; and The device encodes the current video block based on a first transform coefficient of the first color component and a modified second transform coefficient of the second color component to generate an encoded video bitstream, wherein the first transform coefficient and the second transform coefficient are jointly encoded into the encoded video bitstream.

11. An apparatus for decoding a current video block of a video bitstream, characterized in that, The device includes: Memory for storing instructions; and A processor communicating with the memory, wherein when the processor executes the instructions, the processor is configured to cause the device to perform the method according to any one of claims 1 to 9.

12. An apparatus for encoding a current video block of a video bitstream, characterized in that, The device includes: Memory for storing instructions; and A processor communicating with the memory, wherein when the processor executes the instructions, the processor is configured to cause the device to perform the method according to claim 10.

13. An electronic device, characterized in that, include: Memory that stores instructions; as well as A processor communicating with the memory, wherein when the processor executes the instructions, the processor is configured to cause the device to perform the method according to any one of claims 1 to 10.

14. A non-transitory computer-readable storage medium for storing instructions, characterized in that, When the instructions are executed by the processor, the instructions are configured to cause the processor to perform the method according to any one of claims 1 to 10.

15. A method for storing a bit stream, characterized in that, The method of claim 10 is performed to generate a bit stream; and the bit stream is stored.

16. A method for transmitting a bit stream, characterized in that, The method of claim 10 is used to generate a bit stream; and the bit stream is transmitted.

17. A computer-readable storage medium storing a computer program / instructions and a bit stream thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 10 to generate the bit stream.

Images