Use-case driven context model selection for hybrid video coding tools
By employing multiple CABAC contexts tailored to specific application scenarios, the method optimizes encoding tool flag transmission, addressing complexity and efficiency tradeoffs in video encoding, particularly for blocks with varying aspect ratios.
Patent Information
- Application Number
- JP2024153760
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2019-03-05
- Filing Date
- 2024-09-06
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-03-04
AI Technical Summary
The evolution of video encoding methods has led to diverse block shapes and increased algorithmic complexity, necessitating a better complexity versus compression efficiency tradeoff, particularly in handling encoding tool flags for blocks with varying aspect ratios.
The solution involves assigning multiple CABAC contexts based on specific application scenarios, such as block aspect ratios, to optimize the transmission of tool enable flags, reducing unnecessary signaling overhead and maintaining encoding efficiency.
This approach ensures efficient encoding by adaptively selecting CABAC contexts for different scenarios, minimizing bitrates and maintaining high coding efficiency without penalizing compression performance.
Smart Images

Figure 2024164311000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to the field of picture, image or video encoding / decoding, and more particularly to the encoding of one or more encoding tools, such as affine linear weighted intra prediction LWIP or matrix-based intra prediction MIP, of the versatile video coding VVC standard, using a context or a context mode of a context-adaptive binary arithmetic coding CABAC engine. An embodiment relates to the encoding of a flag, such as intra_mip_flag, indicating the applicability of the LWIP or MIP of the VVC standard based on a context model selected depending on the aspect ratio of a block of image or video data to be processed. [Background technology]
[0002] In state-of-the-art video coding standards such as ITU-T H.265|MPEG H HEVC[1], a picture is divided into fixed square-sized CodingTreeUnits, which can be further subdivided into smaller blocks. The reconstructed signal of such a block is usually a superposition of a prediction signal and a residual signal. The prediction signal is obtained either by extrapolating adjacent neighboring samples to the current block (intra prediction) or by copying filtered or unfiltered representations of samples from one or two reference pictures (inter prediction). A reference picture is a picture that has already been reconstructed from the bitstream and stored in a picture buffer for reference. The residual signal is obtained by inverse transforming the dequantized transform coefficients read from the bitstream. After the block reconstruction process, an in-loop filter is applied to enhance the signal of the reconstructed block and obtain the reconstructed picture.
[0003] The entropy decoding process, which reads symbols such as transform coefficients, delta QP, intra prediction modes, motion vector differences, etc. from the bitstream, is performed by a parser that converts the bits read from the bitstream into binary decisions (bins) using a context-adaptive binary arithmetic coding (CABAC) engine. The parser converts or combines these bins into symbols or syntax elements. The adaptability of the entropy coding process is achieved by using CABAC contexts (CCs). Each context represents an adaptive probability model that models the entropy of a particular symbol or set of symbols. The term adaptation denotes a permanent update of the model towards the current coding state. Thus, the model adapts to the local statistics of the corresponding symbol. The update step is usually embedded in the arithmetic coding operation. First, the current state of the CC is used to parameterize the arithmetic coding process. Then, once the decoded symbol is derived, it is used to update the CC with a given step size relative to the current decoded probability.
[0004] Various improvements on the arithmetic coding process have been evaluated and adopted in the JEM-Software [2] as well as in the upcoming VVC standard [3]. The arithmetic coding engine has been modified and the CC initialization and update process has been improved. The dynamic range of the probability representation as well as the behavior of the CC update process have been improved. Each CC has an individual two-level update step that controls the adaptation strength of the CC to the current probability. This improvement helps to customize the CC update process according to the expected CC usage statistics.
[0005] Due to the large number of binary decisions required to transmit syntax elements, and due to the number of such syntax elements, the binary decisions must be grouped using the same CC in order to reach a practical number or amount of CCs that can be processed by the decoder. Furthermore, the grouping helps the update process to exploit local statistics and improves the stability of the underlying probability model.
[0006] Binary decisions with statistically similar probabilities of belonging to the same syntax element are usually grouped into one CC. Exceptions to this kind of grouping are made in cases where the binary decisions are likely to have different probabilities that they can be predicted from already decoded symbols in the immediate neighborhood. In this case, the selected CC is predicted from already decoded symbols in the immediate neighborhood. Such a procedure is usually applied to symbols that are transmitted quite frequently in the bitstream.
[0007] Besides context-controlled arithmetic coding, there is a bypass mode with a fixed probability of 0.5. This bypass mode, built into the arithmetic coder, is a low-complexity mode for high throughput. The bypass mode is widely used, for example, in transform coding. Summary of the Invention [Problem to be solved by the invention]
[0008] The evolution of video coding methods shows more and more diverse block shapes and more and more coding tools, which leads to a significant amount of algorithmic complexity in the encoder to find a good coding representation. Therefore, it may be beneficial for the encoder to skip the evaluation (i.e., switch off) of coding tools in certain contexts to achieve a better complexity vs. compression efficiency tradeoff. The use of coding tools for a block is usually conveyed to the decoder by sending a context modeling tool enable flag in the bitstream.
[0009] Ideally, the decoder has minimal constraints on whether tool enable flags, i.e., flags that determine whether a tool, such as an encoding mode or a prediction mode, is applied to a particular block, are transmitted in the bitstream. The reason is that disabling a tool in certain cases may have a worse impact on compression performance in some scenarios, even if these scenarios are rather unlikely. In fact, one of the main reasons for the efficiency of hybrid video codecs is that a very large variety of competing encoding tools is always possible, and only one of these tools is selected in a given case.
[0010] For example, allowing tools only for small block sizes, and therefore the constraint on sending tool enable flags, could potentially reduce the coding efficiency of future applications that have very high resolutions that typically involve only small portions of small blocks.
[0011] On the other hand, sending tool enable flags for all possible cases is inefficient in application scenarios where the fast encoder search strategy does not test the tool for some cases, either because the tool is unlikely to be selected in these cases because it is too expensive in terms of execution time, or because the impact of using the tool in these cases on the overall coding efficiency is rather small. In such a setting, not testing the tool in a particular case leads to a faster encoder, but at the cost of some loss in coding efficiency. In a particular case, the tool enable flag is sent in the bitstream, but the tool is not used for that case in a given scenario. Thus, in this scenario, the coding efficiency is higher if the encoder search constraint is also expressed by a constraint on sending the tool enable flag in the bitstream. [Means for solving the problem]
[0012] From the prior art as described above, there may be a need for improvements or enhancements to the encoding of one or more encoding tools used to encode / decode pictures, images or videos. Embodiments of the invention will now be described in further detail with reference to the accompanying drawings, in which: [Brief description of the drawings]
[0013] [Figure 1] FIG. 1 illustrates an apparatus for encoding image or video data according to an embodiment of the present invention; [Diagram 2] 1 shows an apparatus for decoding encoded image or video data according to an embodiment of the present invention; [Diagram 3] FIG. 2 illustrates an apparatus for encoding image or video data according to an embodiment of the present invention, which introduces a separate additional CABAC context for transmitting flags for blocks with an aspect ratio greater than 2; [Figure 4] FIG. 4 shows an apparatus for decoding image or video data according to an embodiment of the present invention, which introduces an individual additional CABAC context for flags of blocks with aspect ratios greater than 2, encoded using the apparatus of FIG. [Diagram 5] 1 illustrates an example of a computer system in which the units or modules and method steps described according to the present technique may be executed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings, in which the same or similar elements are assigned the same reference numerals.
[0015] As mentioned above, in the design of how to group binary decisions into context models, previous standards only consider aspects that lower the overall entropy relative to the number of context models used. In contrast to this approach, the present invention presents a new aspect for the design of how to group binary decisions into context models. This approach takes into account the increasing complexity of the algorithm itself. The present invention adapts the context modeling, for example by inserting a context, to align the coding context with a context that is suitable for turning off a particular coding tool. This allows the encoder to choose operating points with different algorithmic complexity while avoiding a loss of compression efficiency.
[0016] The technique of the present invention is illustrated using the following example. Assume that a probability modeled binary decision is to switch between a first tool represented by Algorithm 1 and a second tool represented by Algorithm 2, where the second tool is considered to be a baseline tool and the first tool is considered to be a more specialized tool. According to this assumption, the second tool is more likely to be selected overall than the first tool. As an example, assume that the overall probability that the first tool is preferred over the second tool is 0.3.
[0017] Now suppose two application scenarios are given. In a first application A, there are N cases where both tools are tested and the best performing tool is selected. In a second application B, for some reason, both tools are tested only for a determined part of all N cases and for the remaining cases only the baseline tool, i.e. the second tool, is selected. For both applications, the decisions for all N cases need to be context modeled and transmitted in the bitstream. As an example, suppose the number of cases where both tools are tested in application B is equal to N / 2. For the other N / 2 cases, the tool is tested in application A but not in application B.
[0018] If the tool flag is context coded with a single CC, then for a first application A, the probability of algorithm 1 has a stable value of 0.3, but for a second application B, its average probability drops to 0.15, introducing a fixed probability penalty of ±0.15 for all N cases. If the tool is tested with application B, the true probability will be 0.3 instead of 0.15, and if the tool is not tested with application B, the true probability will be 0. In other words, for application B, using a single CC, the probability of the actual decision is modeled in a non-optimal way that results in a higher bitrate when transmitted in the bitstream.
[0019] In the inventive approach, the drawbacks of such non-optimal modeling of decision probabilities are overcome as follows: Instead of using one probability model for all decisions, two (or more) probability models are used respectively. CABAC contexts are assigned to the same decisions. For each of the N cases, a selection index is used to select which probability model to use.
[0020] Considering the above example again, the selection index is selected to distinguish between the determined portion where algorithm 1 and algorithm 2 are tested and the remaining portion where only algorithm 2 is tested with a second application B. In other words, the two different cases are clustered and represented by different values of the selection index.
[0021] If we use this selection index with two probability models in application A, the selection index switches between the two probability models, but several different CCs can still model the statistics of the tool flags. In both models, the tool represented by Algorithm 1 has a probability of 0.3. This results in a modeling equivalent to the original case modeled with only one probability model.
[0022] However, using the aforementioned selection index with the two probability models in the second application B also leads to optimal modeling in this case: for a determined portion of all N / 2 cases where both algorithms are tested, the probability of algorithm 1 is 0.3, and for the remaining portion where only baseline algorithm 2 is tested, the probability of algorithm 1 is 0.0. Both probabilities are captured in a well-differentiated manner in the two probability models, thus resulting in modeling without modeling penalty and, consequently, a low bit rate when transmitted in the bitstream.
[0023] Therefore, an embodiment of the inventive approach is based on introducing an additional CABAC context for sending a tool enable flag. This additional context is applied only if a condition is met. Otherwise, the CABAC context is selected as usual. According to one embodiment, the condition may be that the size of the current block belongs to a predefined subset of block sizes that may be skipped by a fast encoder search strategy but may be beneficial for applications that require high coding efficiency. According to another embodiment, the condition may be that the aspect ratio of the current block is above a certain value such as 2 that may be skipped by a fast encoder search strategy but may be beneficial for applications that require high coding efficiency and may be controlled at the block level rather than the picture or slice level.
[0024] On the other hand, if, due to the probability adaptation of the CABAC context, in an application scenario, the tools are never tested for certain cases defined by the conditions, the signaling overhead for sending the tool flags for these cases becomes very small, and thus the coding efficiency is almost as good as if the tool flags were not sent in these cases.
[0025] On the other hand, if, due to probability adaptation of the CABAC context, the tool is also tested by the encoder for cases determined by conditions in different application scenarios, the coding efficiency for transmitting the tool flags does not decrease significantly if a separate CABAC context is used for the cases determined by conditions.
[0026] Thus, in contrast to state-of-the-art approaches, the assignment of different CABAC contexts as proposed by the present invention is not guided by trying to model the entire conditional probability distribution of tool flags. Instead, as explained in the above example, the assignment of different CABAC contexts corresponds to different application scenarios of the tools. Here, each application scenario is defined as a particular condition under which the execution of a given tool is in principle possible, but which is never tested by the encoder in a given scenario.
[0027] The reasons for excluding each tool and algorithm under certain conditions are various, and below, some embodiments of the cases determined by the conditions are given, but the present invention is not limited to these embodiments. First, the excluded algorithms may be too complex for these cases. Second, the algorithms may not be implemented or even impossible to implement in these cases, for example due to hardware or resource limitations. Third, there are scenarios where the algorithms used for these cases may only slightly improve the compression performance. Fourth, using the underlying algorithms for these cases basically always gives very limited compression benefits and is therefore only feasible when maximum compression performance is targeted. Fifth, these cases do not include the core application field of the respective algorithms, for which the tools were originally designed. Two or more context divisions for tools
[0028] An embodiment of the present invention also incorporates a splitting of the context into various contexts, each of which corresponds to a different use case scenario of the underlying tool. An embodiment herein may be as follows: The original or traditional tool enable flag is modeled by a single context. According to an embodiment, the inventive approach uses three contexts instead, where the selection index is controlled by, for example, a quantized version of the block region. Here, the selection index may be assigned as follows:
[0029] 0|(block_area>=max_block_area / 4), SelectionIndex=1|(block_area>=max_block_area / 16)AND(block_area <max_block_area / 4)、 2|Otherwise According to another embodiment, the inventive technique uses four contexts or context models for the binarization of the syntax element intra_mip_flag, which represents the Affine Linear Weighted Intra Prediction LWIP or Matrix-based Intra Prediction MIP of the Generic Video Coding VVC standard. The selection index is controlled by the aspect ratio (width / height or height / width) of the current block: 0 | else 1|Otherwise, SelectionIndex=2 | else 3|If the aspect ratio of the current block is greater than 2
[0030] FIG. 1 illustrates an apparatus 100 for encoding image or video data according to an embodiment of the present invention. The apparatus 100 includes an encoder 102. The encoder 102 receives image or video data 104 and encodes the received image or video data 104 to provide a bitstream 106 representing the encoded image or video data. The encoder 102 includes a CABAC encoder 108. The CABAC encoder 108 receives a binary-valued syntax element 110 associated with a particular data block of the image or video data to be encoded and encodes the binary-valued syntax element into coding bits 112 for the bitstream using a selected context model. The binary-valued syntax element includes a tool flag indicating whether or not a particular encoding tool is employed when encoding the image or video data. A first group of context models for encoding the tool flag is selected for one or more first parts of the particular data block where the encoding tool is always applicable, independent of the application. A second group of context models for encoding the tool flag is selected for one or more second parts of the particular data block where the encoding tool is applicable or not, depending on the application. According to an embodiment, as also described below, the CABAC encoder 108 selects the first context model or the second context model for the currently processed portion of the particular data block in response to the selection indicator, the selection indicator having a first value indicating that the currently processed portion of the particular data block is the first portion, and the selection indicator having a second value indicating that the currently processed portion of the particular data block is the second portion.
[0031] FIG. 2 illustrates an apparatus 200 for decoding encoded image or video data according to an embodiment of the present invention. The apparatus 200 includes a decoder 202, which receives a bitstream 106, similar to the bitstream provided by the encoder 102 of FIG. 1. The bitstream 106 includes encoded image or video data, and the decoder 202 decodes the encoded image or video data from the received bitstream to provide decoded image or video data 204. The decoder includes a CABAC decoder 206, which decodes, from the bitstream 106, a binary-valued syntax element 110 associated with a particular data block of the encoded image or video data using a selected context model. The binary-valued syntax element includes a tool flag indicating whether or not to employ a particular encoding tool when encoding the image or video data. A first group of context models for decoding the tool flag is selected for parts of the particular data block for which the encoding tool is always applicable, independent of the application, and a second group of context models for decoding the tool flag is selected for parts of the particular data block for which the encoding tool is applicable or not, depending on the application. According to an embodiment, as also described below, the CABAC decoder 206 selects the first context model or the second context model for the currently processed portion of the particular data block in response to the selection index, the selection index having a first value indicating that the currently processed portion of the particular data block is the first portion, and the selection index having a second value indicating that the currently processed portion of the particular data block is the second portion.
[0032] According to an embodiment, the group of first context models includes one first context model or various first context models, and the group of second context models includes one second context model or various second context models.
[0033] Combined with original context indexing As mentioned above, in many cases, the tool enable flag is modeled by more than one context model when the binary decision is likely to have different probabilities that can be predicted from the already decoded symbols in the adjacent neighborhood. The technique of the present invention can also be applied as a combination of such entropy-driven context selection and the selection index of the present invention. The motivation for such a combination is clear, since the original entropy-driven context indexing may also be appropriate after separation.
[0034] An embodiment in which the context model of a tool flag is selected by both the entropy-driven context selection and the technique of the present invention is as follows. 0 | The tool is not used in blocks above or to the left of the current block EntropyIndex=1 | The tool is only used in one of the blocks above or to the left of the current block 2|The tool is used in the blocks above and to the left of the current block. SelectionIndex=0|(block_area>=max_block_area / 16), 1|Otherwise CombinedIndex=EntropyIndex+3*SelectionIndex.
[0035] Thus, in this embodiment, a purely entropy-driven context selection would result in three possible context models for a given tool flag, but in combination with the inventive context model selection results in six possible context models, indexed by the index CombinedIndex.
[0036] Bypass mode encoding enable flag replacement The inventive approach is also applicable to flags that are conventionally coded using a bypass mode. In such cases, one or two additional context models are needed to achieve a use case-driven implementation. If only one additional context model is used, the selection indicator distinguishes between the use of the bypass mode for coding the unaffected parts of the case and the use of one context model for all other cases where the tool can be switched on and off.
[0037] When two context models are used, the bypass mode is completely replaced by context-modeled arithmetic coding, and a selection index distinguishes between the two context models. Note that with improved update techniques in the upcoming VVC standard, one context modeling the non-switchable part of the cases can be updated with a small update strength to achieve a quasi-stationary model.
[0038] It should also be noted that the additional model modeling the switchable part of the case is indeed used with a very strong update strength to achieve fast adaptation of the context model to either the tool on or tool off probability.
[0039] Alternative Method for Signaling Partial Tool Enablement in a Parameter Set The aforementioned partial tool enabling operation can also be signaled in a parameter set signaled in the bitstream for each predefined part of one or more frames, for example for each slice. In this case, no context separation is required, since the flags transmitted from the parameter set contain all necessary information. However, a drawback of this kind of signaling compared to a use case driven context model selection is that in the former case, the tool can only be fully enabled or disabled only for the cases corresponding to the application scenario throughout all parts of the video sequence to which the parameter set applies, whereas in the latter case, it can also be disabled within any variable part of the video sequence, which does not have to be determined in advance, respectively, known to the decoder. The reason is that in the latter case, if a special context model for the tool flag is assigned to all cases where it is sometimes feasible to disable the tool for a particular application scenario, starting from any of the sequential encodings, the encoder corresponding to the particular application scenario cannot test the tool for the corresponding cases up to a flexible point of sequential encoding with only a small signal overhead, compared to the situation where the tool is completely prohibited for all cases corresponding to the application scenario of this particular part of the video sequence.
[0040] Applications using any coding tool The context splitting of the present invention can be applied to any coding tool controlled by an enable flag. The current candidate tools occurring in future VVC standards that can be used with the present technique are listed below. However, the application of the present technique is not limited to these tools. The candidate tools are DMVR, OBMC, BIO, FRUC, LIC, ISP, ALF, SAO, Inter or Intra MTS, 65-directional intra mode, MRL, and splitting tools such as QTBT, MTT, or QTBT+TT.
[0041] The inventive approach can also be applied to tools with different configurations represented by index values. Here, the inventive CC allocation is determined by the fact that in some application scenarios, only a subset of all configurations of the tool are feasible in a specific case or in general. The inventive CC allocation takes these different application scenarios into account by allocating additional CCs to configurations that the tool is not feasible for the scenario. One embodiment of this aspect of the invention is a tool that applies one of n transforms for the inverse transformation of the prediction residual, and an index of the transform is transmitted in the bitstream.
[0042] Using the inventive approach, many of the tools can have some context splitting capable tools only under certain circumstances, where the inventive context splitting is subject to certain cases where certain tools may be infeasible in certain application scenarios. These specific cases depend on the important characteristics of the tools. The list of characteristics that can be evaluated or combined for certain circumstances, which is not limited to the inventive use case, is: block size, block shape, block aspect ratio, temporal level, QP, picture type, picture resolution, picture dynamic range, reference picture, first picture of GOP. A particular situation may be a combination of these aforementioned characteristics.
[0043] EMBODIMENTS FOR APPLYING THE TECHNIQUE OF THE PRESENT EMBODIMENT FOR CONTEXT ALLOCATION Affine Linear Weighted Intra Prediction (LWIP) [4] is a new intra prediction technique. Similar to conventional intra prediction, LWIP consists of a set of prediction modes. Given the left and top reconstructed (reference) samples, each of the signaled prediction modes corresponds to a prediction function that produces a different prediction signal.
[0044] For both conventional and LWIP modes available, the encoder compares the rate-distortion costs of the conventional and LWIP modes and determines the mode with the lowest overall cost. Then, the selected prediction mode is transmitted to the decoder in the bitstream and is used to select the corresponding prediction function for predicting the block. The syntax for signaling the prediction mode is as follows: first, a flag is transmitted indicating whether the block is predicted in conventional or LWIP mode. If conventional prediction is selected, the conventional prediction mode is read from the bitstream according to the intra prediction signal. Otherwise, if LWIP prediction is selected, the flag is followed by a mode indicator in the set of available LWIP modes. Since both conventional and LWIP modes are available for all block sizes commonly supported by the codec for intra prediction, the flag must be transmitted for each block.
[0045] The VVC coding standard currently covers the following:
number
[0046] The core forecast for LWIP is:
number
number
number
[0047] This effect can be used to reduce encoder complexity by restricting LWIP mode to blocks with aspect ratios of 2 or less and not testing it for blocks with higher aspect ratios, although this also reduces coding efficiency somewhat. Not sending a flag for blocks with aspect ratios greater than 2 reduces the loss in coding efficiency, but it also makes it impossible to implement an encoder that improves overall coding efficiency by testing LWIP mode for blocks with aspect ratios greater than 2 that may be required for different applications.
[0048] A solution that supports both fast and high compression efficiency encoders is to introduce a separate additional CABAC context for sending flags for blocks with aspect ratios larger than 2. Now, if LWIP mode is not tested in the encoder for those blocks, the flag is always 0 (=blocks not predicted in LWIP mode) and sending the flag causes almost no overhead (only to adapt the context to 0 probability to send 1), which means that the coding efficiency is very close to the solution of not sending flags for those blocks. If LWIP blocks are tested in the encoder for those blocks, the flag is 1 with a certain probability (=block predicted in LWIP mode) and sending the flag causes almost no overhead, which means that the coding efficiency is very close to the solution of sending flags in the same context for all block sizes.
[0049] In other words, as mentioned above, according to an embodiment, the inventive approach uses four contexts or context models for the binarization of the syntax element intra_mip_flag representing the affine linear weighted intra prediction LWIP or matrix-based intra prediction MIP of the generic video coding VVC standard. The selection index is controlled by the aspect ratio (width / height or height / width) of the current block: 0 | else 1|Otherwise, SelectionIndex=2 | else 3|If the aspect ratio of the current block is greater than 2
[0050] Fig. 3 shows an apparatus for encoding image or video data according to an embodiment of the present invention, for example an apparatus similar to that of Fig. 1, which introduces an individual additional CABAC context for transmitting flags for blocks with an aspect ratio greater than 2. An encoder 102 receives image or video data 104 and encodes the received image or video data 104 to provide a bitstream 106 representing the encoded image or video data. A CABAC encoder 108 receives a tool flag 110 indicating whether a particular encoding tool, such as affine linear weighted intra prediction, LWIP, is used when encoding a block of image or video data. Affine linear weighted intra prediction LWIP is also called matrix-based intra prediction MIP in the generic video coding VVC standard, and the tool flag is also called intra_mip_flag indicating the applicability of affine LWIP or MIP in the VVC standard. For a block 300 of image or video data having an aspect ratio greater than 2 and for which a particular encoding tool is applicable, a first context model for encoding the tool flag is selected from a group of one or more first context models and provided to CABAC 108. For a block 302 of image or video data having an aspect ratio less than or equal to 2 and for which a particular encoding tool is applicable, a second context model for encoding the tool flag is selected from a group of one or more second context models and provided to CABAC 108. For example, if the LWIP mode is not tested for a block 300 with an aspect ratio greater than 2, the flag is always 0 and the additional CABAC context adapts to have a zero probability of transmitting 1, and if the LWIP mode is tested for a block 300 with an aspect ratio greater than 2, the flag is 1 with a certain probability. According to an embodiment, the CABAC encoder 108 can select the first and second context models for the currently processed block in response to the selection indicator. The selection indicator indicates that the currently processed block has an aspect ratio greater than 2 or an aspect ratio less than or equal to 2.
[0051] Figure 4 illustrates an apparatus for decoding image or video data according to an embodiment of the present invention, e.g. an apparatus 100 similar to the apparatus of Figure 2, which introduces an individual additional CABAC context for flags of blocks with an aspect ratio greater than 2, encoded using the apparatus of Figure 3. For a block 300 of image or video data with an aspect ratio greater than 2 and for which a particular encoding tool is applicable, a first context model for encoding the tool flags is selected from a group of one or more first context models and provided to CABAC 206. For a block 302 of image or video data with an aspect ratio equal to or less than 2 and for which a particular encoding tool is applicable, a second context model for encoding the tool flags is selected from a group of one or more second context models and provided to CABAC 206.
[0052] For example, the binarization of the syntax element intra_mip_flag can use a total of four context models with context indices {0,1,2,3} as follows: If the aspect ratio (width / height or height / width) of the current block is greater than 2, use the context model with index 3;
[0053] Otherwise, one of context models 0, 1, or 2 is used, and the choice may depend, for example, on the intra_mip_flag of the blocks to the left and above the current block, as known and used for some other syntax elements.
[0054] In the VVC specification, the coding unit syntax can be as follows (see, for example, 7.3.10.5 - Coding unit syntax in [5]): [Table 1]
[0055] intra_mip_flag[x0][y0] equal to 1 specifies that the intra prediction type of the luma sample is matrix-based intra prediction. intra_mip_flag[x0][y0] equal to 0 specifies that the intra prediction type of the luma sample is not matrix-based intra prediction (see, e.g., 7.4.11.5 - Coding Unit Semantics in [5]).
[0056] The binarization of the syntax element intra_mip_flag can be as follows (see, for example, 9.3.4.2 of [5] - the derivation process of ctxTable, ctxIdx, and bypassFlag):
[0057] [Table 2]
[0058] Although some aspects of the described concepts have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, with a block or apparatus corresponding to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
[0059] Various elements and features of the invention may be implemented in hardware using analog and / or digital circuitry, software, through execution of instructions by one or more general-purpose or special-purpose processors, or as a combination of hardware and software. For example, embodiments of the invention may be implemented in the environment of a computer system or another processing system. FIG. 5 shows an example of a computer system 400. The units or modules, as well as the steps of the methods performed by these units, may be executed on one or more computer systems 400. The computer system 400 includes one or more processors 402, such as a special-purpose or general-purpose digital signal processor. The processors 402 are connected to a communication infrastructure 404, such as a bus or a network. The computer system 400 includes a main memory 406, such as a random access memory (RAM), and a secondary memory 408, such as a hard disk drive and / or a removable storage drive. The secondary memory 408 may allow computer programs or other instructions to be loaded into the computer system 400. The computer system 400 may further include a communication interface 410, which allows software and data to be transferred between the computer system 400 and external devices. The communications may be from electronic, electromagnetic, optical, or other signals capable of being processed by the communications interface. The communications may use wire or cable, fiber optics, phone lines, cellular phone links, RF links, and other communications channels 412.
[0060] The terms "computer program medium" and "computer readable medium" are generally used to refer to tangible storage media, such as a hard disk installed in a removable storage unit or a hard disk drive. These computer program products are means for providing software to the computer system 400. Computer programs, also called computer control logic, are stored in the main memory 406 and / or the secondary memory 408. Computer programs may also be received via the communication interface 410. The computer programs, when executed, enable the computer system 400 to perform the present invention. In particular, the computer programs, when executed, enable the processor 402 to perform the processes of the present invention, such as any of the methods described herein. Such computer programs may therefore represent the controller of the computer system 400. If the present disclosure is implemented using software, the software may be stored in a computer program product and loaded into the computer system 400 using an interface, such as a removable storage drive, the communication interface 410.
[0061] The hardware or software embodiments may be implemented using a digital storage medium, such as, for example, cloud storage, floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM or flash memory, having electronically readable control signals stored therein and cooperating (or capable of cooperating) with a programmable computer system to perform the respective methods. Thus, the digital storage medium may be computer readable.
[0062] Some embodiments according to the invention comprise a data carrier having electronically readable control signals which cooperate with a programmable computer system to perform one of the methods described herein.
[0063] In general, embodiments of the present invention may be implemented as a computer program product having a program code operative to perform one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
[0064] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier. In other words, an embodiment of the inventive method is a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0065] A further embodiment of the inventive method is therefore a data carrier (or a digital storage medium or a computer readable medium) comprising, and having recorded thereon, a computer program for performing one of the methods described herein. A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can for example be arranged to be transferred via a data communication connection, for example the Internet. A further embodiment comprises a processing means, for example a computer or a programmable logic device, adapted to or adapted for performing one of the methods described herein. A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
[0066] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.
[0067] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the configurations and details described herein will be apparent to others skilled in the art. It is therefore intended to be limited only by the scope of the appended claims and not by the specific details shown by the description and illustration of the embodiments herein.
[0068] References [1] ISO / IEC, ITU-T. High efficiency video coding. ITU-T Recommendation H.265 | ISO / IEC 23008 10 (HEVC), edition 1, 2013; edition 2, 2014.
[0069] [2] JEM reference software, https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / .
[0070] [3] B. Bross, J. Chen, Shan Liu, “Versatile Video Coding (Draft 4)”, JVET-M1001-v5, February 2019, Marrakesh, Morocco
[0071] [4] J. Pfaff, B. Stallenberger, M. Schaefer, P. Merkle, P. Helle, R. Rische, H. Schwarz, D. Marpe, T. Wiegand, “Affine Linear Weighted Intra Prediction”, JVET-M0043, February 2019, Marrakesh, Morocco
[0072] [5] B. Bross, J. Chen, Shan Liu, “Versatile Video Coding (Draft 8)”, JVET-Q2001-vD, February 2020, Brussels, Belgium
Claims
1. 1. A decoder for decoding an image, the decoder comprising: receiving a bitstream including encoded data representing the image; determining, for a block of the image, whether a first dimension of the block is twice as large as a second dimension of the block; selecting a context based on said determining, the context is a first context when the first size of the block is twice as large as the second size of the block; the selecting, when the first size of the block is not twice as large as the second size of the block, the context being a second context different from a first context; decoding, from the bitstream using the selected context, a flag associated with the block, the flag indicating whether or not a Linear Weighted Intra Prediction (LWIP) coding tool is used to decode the block; decrypting the block based in part on the flag; A decoder configured to:
2. the first dimension of the block is a height of the block and the second dimension is a width of the block; or the first dimension of the block is a width of the block and the second dimension is a height of the block; A decoder according to claim 1.
3. determining, in response to the flag representing a first value, to use the LWIP encoding tool to decode the block; determining not to use the LWIP encoding tool to decode the block in response to the flag indicating a second value different from the first value; A decoder according to claim 1.
4. the first value is 0; the second value is 0; A decoder according to claim 3.
5. The decoder comprises: and after determining to use the LWIP encoding tool to decode the block, decoding a mode index for a set of available LWIP modes. A decoder according to claim 3.
6. The LWIP coding tool supports matrix-based intra prediction; A decoder according to claim 1.
7. the first size of the block being twice as large as the second size of the block indicates that the aspect ratio of the block is greater than 2; A decoder according to claim 1.
8. 1. A method for decoding an image, the method comprising: receiving a bitstream including encoded data representing the image; determining, for a block of the image, whether a first dimension of the block is twice as large as a second dimension of the block; selecting a context based on said determining, the context is a first context when the first size of the block is twice as large as the second size of the block; the selecting, when the first size of the block is not twice as large as the second size of the block, the context being a second context different from a first context; decoding, from the bitstream using the selected context, a flag associated with the block, the flag indicating whether or not a Linear Weighted Intra Prediction (LWIP) coding tool is used to decode the block; decrypting the block based in part on the flag; A method comprising:
9. the first dimension of the block is a height of the block and the second dimension is a width of the block; or the first dimension of the block is a width of the block and the second dimension is a height of the block; The method according to claim 8.
10. determining, in response to the flag representing a first value, to use the LWIP encoding tool to decode the block; determining not to use the LWIP encoding tool to decode the block in response to the flag indicating a second value different from the first value; The method of claim 8 , further comprising:
11. the first value is 0; the second value is 0; The method of claim 10.
12. after determining to use the LWIP encoding tool to decode the block, decoding a mode index for a set of available LWIP modes; The method of claim 10 further comprising:
13. The LWIP coding tool supports matrix-based intra prediction; The method according to claim 8.
14. the first size of the block being twice as large as the second size of the block indicates that the aspect ratio of the block is greater than 2; The method according to claim 8.
15. When executed, on at least one processor, Receiving a bitstream including coded data representing an image; determining, for a block of the image, whether a first dimension of the block is twice as large as a second dimension of the block; selecting a context based on said determining, the context is a first context when the first size of the block is twice as large as the second size of the block; the selecting, when the first size of the block is not twice as large as the second size of the block, the context being a second context different from a first context; decoding, from the bitstream using the selected context, a flag associated with the block, the flag indicating whether or not a Linear Weighted Intra Prediction (LWIP) coding tool is used to decode the block; decrypting the block based in part on the flag; A non-transitory computer-readable medium comprising instructions to cause a
16. the first dimension of the block is a height of the block and the second dimension is a width of the block; or the first dimension of the block is a width of the block and the second dimension is a height of the block; 16. The non-transitory computer readable medium of claim 15.
17. In response to the flag representing a first value, the instructions, when executed, cause the at least one processor to determine to use the LWIP encoding tool to decode the block; In response to the flag indicating a second value different from the first value, the instructions, when executed, cause the at least one processor to determine not to use the LWIP encoding tool to decode the block.
16. The non-transitory computer readable medium of claim 15.
18. the first value is 0; the second value is 0; 20. The non-transitory computer readable medium of claim 17.
19. after determining to use the LWIP encoding tool to decode the block, the instructions, when executed, cause decoding, on the at least one processor, a mode index for a set of available LWIP modes.
20. The non-transitory computer readable medium of claim 17.
20. said LWIP coding tool supports matrix-based intra prediction; the first size of the block being twice as large as the second size of the block indicates that the aspect ratio of the block is greater than 2; 16. The non-transitory computer readable medium of claim 15.
21. 1. An encoder for encoding an image into a bitstream, the encoder comprising: determining, for a block of the image, whether a first dimension of the block is twice as large as a second dimension of the block; selecting a context based on said determining, the context is a first context when the first size of the block is twice as large as the second size of the block; the selecting, when the first size of the block is not twice as large as the second size of the block, the context being a second context different from a first context; encoding a flag associated with the block into the bitstream using the selected context, the flag indicating whether or not to use a Linear Weighted Intra Prediction (LWIP) encoding tool to decode the block; encoding the block based in part on the flag; The encoder is configured to:
22. 1. A method for encoding an image into a bitstream, the method comprising: determining, for a block of the image, whether a first dimension of the block is twice as large as a second dimension of the block; selecting a context based on said determining, the context is a first context when the first size of the block is twice as large as the second size of the block; the selecting, when the first size of the block is not twice as large as the second size of the block, the context being a second context different from a first context; encoding a flag associated with the block into the bitstream using the selected context, the flag indicating whether or not to use a Linear Weighted Intra Prediction (LWIP) encoding tool to decode the block; encoding the block based in part on the flag; A method comprising:
23. When executed, on at least one processor, determining, for a block of an image, whether a first dimension of the block is twice as large as a second dimension of the block; selecting a context based on said determining, the context is a first context when the first size of the block is twice as large as the second size of the block; the selecting, when the first size of the block is not twice as large as the second size of the block, the context being a second context different from a first context; encoding a flag associated with the block into a bitstream using the selected context, the flag indicating whether or not to use a Linear Weighted Intra Prediction (LWIP) encoding tool to decode the block; encoding the block based in part on the flag; A non-transitory computer-readable medium comprising instructions to cause
Citation Information
Patent Citations
Side information signaling method for inter prediction using geometric partitioning
JP2022506152A
Prediction signal filtering in affine linear weighted intra prediction
JP2022531334A
Use-case-driven context model selection for hybrid video coding tools
JP7377878B2
Use-case-driven context model selection for hybrid video encoding tools
JP7554335B2
Affine linear weighted intra prediction in video coding
US20200359033A1
