Managing codec tool combinations and restrictions

Abstract

Video coding tools may be controlled by including syntax in the video bitstream that better utilizes video decoding resources. An encoder inserts syntax into a video bitstream to enable a decoder to parse the bitstream and readily control which combinations of tools are enabled, which combinations are not allowed, and which tools are activated for various components in a multi-component bitstream, resulting in potential parallelization of bitstream decoding.

Description

[0001] This divisional application is a divisional application with an application date of December 9, 2019, application number 201980081202.4, and invention name “Managing Codec Tool Combinations and Restrictions”. Technical Field

[0002] At least one of the present embodiments generally relates to a method or apparatus for video encoding or decoding, compression or decompression. Background Art

[0003] In order to achieve high compression efficiency, image and video coding schemes usually use prediction (including motion vector prediction) and transformation to leverage spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original image and the predicted image (usually expressed as a prediction error or prediction residual) is transformed, quantized, and entropy encoded and decoded. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy encoding and decoding, quantization, transformation, and prediction. Summary of the invention

[0004] At least one of the present embodiments generally relates to a method or apparatus for video encoding or decoding, and more particularly, to a method or apparatus for simplifying a coding or decoding mode based on a parameter model that depends on neighboring samples.

[0005] According to a first aspect, a method is provided. The method comprises the following steps: inserting a high-level syntax associated with at least one video codec tool into a video bitstream; conditionally inserting one or more low-level controls for one or more video codec tools into the bitstream based on the high-level syntax; activating one or more video codec tools corresponding to one or more video components based on the low-level controls; and encoding the video bitstream using the activated video codec tools.

[0006] According to a second aspect, a method is provided. The method comprises the following steps: parsing a video bitstream to obtain a high-level syntax associated with at least one video codec tool; based on the high-level syntax, determining one or more low-level controls for one or more video codec tools; based on the determination, activating one or more video codec tools corresponding to one or more video components; and decoding the video bitstream using the activated video codec tools.

[0007] According to another aspect, a device is provided. The device includes a processor. The processor can be configured to encode a video block or decode a bitstream by executing any one of the aforementioned methods.

[0008] According to another general aspect of at least one embodiment, there is provided an apparatus comprising an apparatus according to any one of the decoding embodiments; and at least one of the following items: (i) an antenna configured to receive a signal, the signal comprising a video block; (ii) a band limiter configured to limit the received signal to a frequency band comprising the video block; or (iii) a display configured to display an output representing the video block.

[0009] According to another general aspect of at least one embodiment, there is provided a non-transitory computer-readable medium comprising data content generated according to any of the encoding embodiments or variations described.

[0010] According to another general aspect of at least one embodiment, there is provided a signal comprising video data generated according to any of the encoding embodiments or variations described.

[0011] According to another general aspect of at least one embodiment, a bitstream is formatted to include data content generated according to any of the described encoding embodiments or variations.

[0012] According to another general aspect of at least one embodiment, there is provided a computer program product comprising instructions that, when the program is run by a computer, cause the computer to perform any one of the decoding embodiments or variants described.

[0013] These and other aspects, features and advantages of the various general aspects will become apparent from the following detailed description of illustrative embodiments which is to be read in connection with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A standard general purpose video compression scheme is shown.

[0015] Figure 2 A standard general purpose video decompression scheme is shown.

[0016] Figure 3 A general decoding process according to one embodiment is shown.

[0017] Figure 4 The general decoding process for Case 1 is shown.

[0018] Figure 5 A first proposed general decoding process for Case 2 is shown.

[0019] Figure 6 A second proposed general decoding process for Case 2 is shown.

[0020] Figure 7 The general decoding process for Case 3 is shown.

[0021] Figure 8 An example decoding process based on picture partition restriction signaling is shown.

[0022] Fig. 9 An alternative decoding process based on picture partition restriction signaling is shown.

[0023] Fig.10 An example decoding process based on multi-hypothesis prediction enable flag signaling is shown.

[0024] Fig.11 An example decoding process based on MTS (Multiple Transform Set) enable flag signaling is shown.

[0025] Fig.12 An example alternative decoding process based on MTS enable flag signaling is shown.

[0026] Fig.13 An example decoding process for dependency quantization and sign data hiding enable flag signaling is shown.

[0027] Fig.14 A codec tree unit and codec tree concept for representing a compressed HEVC picture are shown.

[0028] Fig.15 An example partitioning of a codec tree unit into codec units, prediction units, and transform units is shown.

[0029] Fig.16 An example quadtree plus binary tree (QTBT) CTU representation is shown.

[0030] Fig.17 A processor-based system for encoding / decoding is shown under the general aspects of the description.

[0031] Fig.18 A simplified block diagram of a decoding process according to a first embodiment is shown.

[0032] Fig.19 One embodiment of a method under the general aspects described is shown.

[0033] Fig. 20 Another embodiment of the method under the general aspects described is shown.

[0034] Fig.21 An example apparatus is shown under the general aspects of the description. DETAILED DESCRIPTION

[0035] The embodiments described herein belong to the field of video compression and generally relate to video compression and video encoding and decoding.The general aspects described aim to provide mechanisms for manipulating restrictions on high-level video codec syntax or video codec semantics to constrain the possible set of tool combinations.

[0036] In order to achieve high compression efficiency, image and video coding schemes usually use prediction (including motion vector prediction) and transformation to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original image and the predicted image (usually expressed as a prediction error or prediction residual) is transformed, quantized, and entropy encoded and decoded. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy encoding and decoding, quantization, transformation, and prediction.

[0037] In the HEVC (High Efficiency Video Codec, ISO / IEC 23008-2, ITU-T H.265) video compression standard, motion compensated temporal prediction is used to exploit the redundancy between consecutive pictures of a video.

[0038] To do this, a motion vector is associated with each prediction unit (PU). Each codec tree unit (CTU) is represented in the compressed domain by a codec tree. This is a quadtree partitioning of the CTU, where each leaf is called a codec unit (CU).

[0039] Each CU is then given some intra or inter prediction parameters (prediction information). For this purpose, it is spatially partitioned into one or more prediction units (PUs), each of which is assigned some prediction information. Intra or inter coding modes are assigned at the CU level.

[0040] In a new video compression standard called Joint Exploration Model (JEM) proposed by JVET (Joint Video Exploration Team), it has been proposed to accept a quadtree-binary tree (QTBT) block partition structure due to high compression performance. A block in a binary tree (BT) can be divided into two sub-blocks of equal size by dividing it horizontally or vertically from the middle. Therefore, a BT block can have a rectangular shape of unequal width and height, unlike a square-shaped block in QT that always has equal width and height. In HEVC, the angular intra-frame prediction direction is defined to span an angle of 180 degrees from 45 degrees to -135 degrees, and it has been maintained in JEM, which makes the definition of the angular direction independent of the target block shape.

[0041] To encode these blocks, intra prediction is used to provide an estimated version of the block using previously reconstructed neighboring samples. The difference between the source block and the prediction is then encoded. In the classical codecs described above, a single line of reference samples is used to the left and top of the current block.

[0042] In HEVC (High Efficiency Video Codec, H.265), the encoding of frames of a video sequence is based on a quadtree (QT) block partition structure. Frames are divided into square codec tree units (CTUs), which are each partitioned into multiple codec units (CUs) based on a quadtree based partitioning based on a rate-distortion (RD) criterion. Each CU is either intra-predicted (i.e., it is spatially predicted from causally neighboring CUs) or inter-predicted (i.e., it is temporally predicted from an already decoded reference frame). In I slices, all CUs are intra-predicted, while in P and B slices, CUs can be intra- or inter-predicted. For intra prediction, HEVC defines 35 prediction modes, including one planar mode (indexed as mode 0), one DC mode (indexed as mode 1), and 33 angular modes (indexed as modes 2–34). Angular modes are associated with prediction directions ranging from 45 degrees to -135 degrees in the clockwise direction. Since HEVC supports a quadtree (QT) block partition structure, all prediction units (PUs) have a square shape. Therefore, from the perspective of the PU (prediction unit) shape, the definition of prediction angles from 45 degrees to -135 degrees is reasonable. For a target prediction unit of size NxN pixels, the top reference array and the left reference array are both 2N+1 samples in size, which is required to cover the aforementioned angle range of all target pixels. Considering that the height and width of the PU are equal, it also makes sense that the lengths of the two reference arrays are equal.

[0043] The present invention belongs to the field of video compression. Compared with existing video compression systems, it aims to improve the bidirectional prediction in inter-frame codec blocks. The present invention also proposes to separate the luminance and chrominance codec trees for inter-frame strips.

[0044] In the HEVC video compression standard, a picture is divided into so-called codec tree units (CTUs), which are usually 64x64, 128x128 or 256x256 pixels in size. Each CTU is represented in the compressed domain by a codec tree. This is a quadtree partitioning of the CTU, where each leaf is called a codec unit (CU), see Fig.14 .

[0045] Each CU is then given some intra or inter prediction parameters (prediction information). For this purpose, it is spatially partitioned into one or more prediction units (PUs), each of which is assigned some prediction information. Intra or inter coding modes are assigned at CU level, see Fig.15 .

[0046] In order to represent picture data in a more flexible way in the compressed domain, an emerging video compression tool is proposed that includes a codec tree unit representation in the compressed domain. The advantage of this more flexible representation of the codec tree is that it provides increased compression efficiency compared to the CU / PU / TU arrangement of the HEVC standard.

[0047] The Quadtree plus Binary Tree (QTBT) codec tool provides this enhanced flexibility. It consists of a codec tree in which the coding units can be divided in both quadtree and binary tree ways. This codec tree representation of the codec tree unit is as follows: Fig.16 shown.

[0048] The division of the codec units is decided at the encoder side by a rate-distortion optimization process, which consists in determining the QTBT representation of the CTU with the minimum rate-distortion cost.

[0049] In QTBT technology, the CU has a square or rectangular shape. The size of the codec unit is always a power of 2 and is usually from 4 to 128.

[0050] In addition to this diversity in the rectangular shape of the codec unit, this new CTU representation has the following different characteristics compared to HEVC.

[0051] The QTBT decomposition of a CTU consists of two stages: first, the CTU is partitioned in a quadtree fashion, and then each quadtree leaf can be further partitioned in a binary fashion. Fig.16 The right side shows that, Fig.16 In FIG. 5 , the solid line represents the quadtree decomposition stage, and the dashed line represents the binary decomposition spatially embedded in the quadtree leaves.

[0052] The general aspects described herein belong to the field of video compression. Video codecs consist of a combination of multiple encoding and decoding tools. It is common practice to standardize the decoder side (syntax and decoding process).

[0053] The documents JVET-L0044 "AHG 15: Proposed interoperability point syntax" and JVET-L0696 "Proposed starting point for interoperability point syntax" specify a large number of high-level syntax elements (in the SPS, or in the profile_tier_level section) that are intended to control the activation of codec tools in the VVC decoder. The goal of this signaling mechanism is to achieve bitstream interoperability points defined by parties other than MPEG or ITU (such as DVB, ATSC, 3GPP). An example of syntax is proposed in Table 1, where the syntax is inserted at the beginning of the SPS to indicate properties that cannot be violated throughout the bitstream. JVET plans to extend this table for most codec tools that will be added to the VVC specification. In principle, JVET has no responsibility to specify profiles or sub-profiles based on these flags. JVET only defines tools and the relationship between constraint flags / parameters and tools.

[0054] Table 1. Syntax elements of SPS used to provide interoperability point syntax

[0055]

[0056] When the constraint flag is set in an SPS (or similar) syntax structure, the decoder can safely assume that the tool will not be used in the bitstream. When the constraint flag is set to 1, the tool can be activated in the associated bitstream.

[0057] Several contributions were also proposed to specify similar signaling mechanisms, specifically JVET-K0311, which provided an initial high-level tool signaling concept, JVET-L0042, which grouped tools by category, and JVET-L0043, which proposed layered signaling (which JVET considered might be too complex to parse).

[0058] Since a codec includes multiple codec tools, this leads to a large number of possible tool combinations, and in current designs, the impact of tool interactions is not considered. In addition, for some cases, simply deactivating a tool is not feasible. Some tools require a fall-back mode. The present invention aims to solve this problem by inserting a process for limiting the possible tool activation / deactivation at a syntactic or semantic level.

[0059] As described above, the approach adopted by JVET for controlling codec tools at a high level with fine granularity results in a large number of tool combinations, some of which are not practical in terms of codec efficiency or implementability.

[0060] There are currently no tool combination restrictions at a high level in the proposed syntax.The main solutions offered in the prior art were discussed in the previous paragraphs (documents JVET-L0042 and JVET-L0043) and have not been considered by JVET.

[0061] Considering the interaction problems that may arise when some tools are combined together and other tools are deactivated, the described aspects propose inserting syntax changes or semantic changes to specify tool combination restrictions.

[0062] Table 2 gives a list of the tools included in VTM3 plus some additional tools being explored and added to VTM later. The tools are categorized by category. A rough estimate of the PSNR-Y BD-rate performance of the tool is provided. A negative number (–x) indicates an average bitrate savings of x%. Interdependencies with other tools are also indicated.

[0063] Several SPSs may be signaled in the bitstream. These SPSs may include flags for controlling tools at the sequence or scalability level.

[0064] In addition to the SPS, additional parameter sets at lower levels in the syntax may be specified (such as PPS, slice header, tile group header, tile header). They may include additional flags for controlling tools at a level lower than the sequence level.

[0065] In the following, syntax elements defined at the highest (SPS) level will use "hl_" as a prefix. When the prefix "ll_" is used, this indicates that the level of the syntax element is lower than the SPS to control the tool more locally.

[0066] Table 2. List of tools considered in VTM3 and used for further VTM

[0067]

[0068]

[0069]

[0070] Based on the analysis of the tool, the following characteristics can be observed:

[0071] - Case 1 – Several tools are designed based on the same concept, which leads to controlling them together

[0072] - Case 2 – Several tools are constrained by other tools, either in terms of performance or technological dependencies, which leads to controlling them based on the control of the tools they are constrained by

[0073] - Case 3 – Several tools are mutually exclusive and cannot be activated together

[0074] The following definitions are used in the remainder of this document:

[0075] - Tools considered – this relates to the set of tools controlled by the proposed treatment

[0076] - Master tools and slave tools – When the tools under consideration belong to the case 2 category, they are divided into two sets, master tools and slave tools, and the activation / deactivation of the slave tools depends on the activation / deactivation of the master tools

[0077] - Dedicated tools – when the tools under consideration are mutually exclusive (case 3), they are grouped into “dedicated tool set 1” and “dedicated tool set 2”, where when a tool in set 1 is activated, a tool in set 2 is deactivated and vice versa

[0078] Figure 3 A general block diagram of the decoding process covering these three cases is provided in . For simplicity, it is considered that two tools are being processed. This concept can be extended to more than two tools. The block diagram basically consists of three branches, depending on the type of tool under consideration checked in steps 901 and 902.

[0079] The branch corresponding to case 1 consists of two main steps.

[0080] - In step 903, at least one high-level syntax element HL_SE is decoded from the bitstream.

[0081] - In step 904, depending on the value of HL_SE, the low-level syntax elements controlling the tool under consideration are:

[0082] o inferred as a given value that results in generally deactivating (or activating) all considered instruments,

[0083] o Or decode from the bitstream (enabling independent control of each of the tools considered).

[0084] The branch corresponding to case 3 consists of four main steps.

[0085] - In step 905, at least two high-level syntax elements HL_SE1 and HL_SE2 related to the control of dedicated tools 1 and 2 are decoded from the bitstream.

[0086] - In step 906, depending on the value of HL_SE1, at least one low-level syntax element LL_SE1 of the control specific tool 1 is:

[0087] ○ Decode from the bitstream,

[0088] o Or inferred as a given value which normally results in deactivation of the special tool 1 .

[0089] - In step 907, depending on the values ​​of HL_SE2 and LL_SE1, at least one low-level syntax element LL_SE2 of the control specific tool 2 is:

[0090] ○ Decode from the bitstream,

[0091] o Or inferred as a given value which normally results in deactivation of the special tool 2 .

[0092] Step 907 results in:

[0093] o 1) If Specific Tool 1 is enabled (according to LL_SE1), the LL_SE2 value causes Specific Tool 2 to be disabled, and

[0094] o 2) If Dedicated Tool 1 is disabled (according to LL_SE1), Dedicated Tool 2 can be enabled.

[0095] The branch corresponding to case 2 consists of four main steps.

[0096] - In step 908, at least one high-level syntax element HL_SE_main is decoded from the bitstream.

[0097] - In step 909, depending on the value of HL_SE_main, at least one high-level syntax element HL_SE_dependent that controls the dependent tool is:

[0098] ○ Decode from the bitstream,

[0099] o Or inferred as a given value which normally results in deactivation of the slave tool.

[0100] Step 909 causes the HL_SE_dependent value to disable the slave tool if the HL_SE_main value causes the master tool to be disabled.

[0101] - In step 910, depending on the value of HL_SE_main, at least one low-level syntax element LL_SE_main of the control main tool is:

[0102] ○ Decode from the bitstream,

[0103] o Or inferred to a given value that would normally result in deactivation of the master tool.

[0104] - In step 911, depending on the values ​​of HL_SE_dependent and LL_SE_main, at least one low-level syntax element LL_SE_dependent of the control slave tool is:

[0105] ○ Decode from the bitstream,

[0106] o Or inferred as a given value which normally results in deactivation of the slave tool.

[0107] Step 911 causes LL_SE_dependent to disable a slave tool if HL_SE_dependent causes the slave tool to be disabled, or if LL_SE_main causes the master tool to be disabled.

[0108] The final step 912 corresponds to applying the decoder processing with activation / deactivation of tools controlled by the low-level syntax elements derived from the previous steps.

[0109] The following sections correspond to exemplary embodiments for different considered tool situations.

[0110] General solution for case 1 (tools designed based on the same concept)

[0111] The solution for case 1 is to define a high-level syntax element that controls several tools. Additional low-level syntax elements related to the activation / deactivation of these tools are also defined. Depending on the value of the high-level syntax element, the additional low-level syntax element is set to a given predefined value or is read from the bitstream. The decoding process is then performed by activating or deactivating these tools according to the values ​​of the low-level syntax elements to which they correspond.

[0112] The treatment is summarized as follows (also as Figure 4 as depicted in ):

[0113] -Decode a high level syntax element high_level_SE (1001) from the bitstream

[0114] -If high_level_SE is equal to the given value V1(1002)

[0115] ○ Based on the value of high_level_SE, infer the low-level syntax elements (low_level_SE) related to tool activation / deactivation (1003)

[0116] -Otherwise (1002)

[0117] ○Decode low_level_SE from the bitstream (1004)

[0118] - Apply the decoding process (1005) by activating / deactivating the tool according to the value of its low-level syntax element low_level_SE

[0119] General solution for case 2 (tools depend on other tools)

[0120] The solution to case 2 involves that the activation of some tools (let's call them slave tools) can be constrained by the activation of other tools (let's call them master tools).

[0121] As in case 1, these tools can be controlled by a single high-level flag that controls the activation of both the master tool and the slave tools. The low-level syntax elements of the slave tools are then constrained by the values ​​of the low-level syntax elements that control the master tool.

[0122] The treatment is summarized as follows (also as Figure 5 as depicted in ):

[0123] -Decode a high level syntax element high_level_SE (1101) from the bitstream

[0124] -If high_level_SE is equal to the given value V1(1102)

[0125] ○ According to the value of high_level_SE, infer the low-level syntax elements related to the activation / deactivation of the main tool and the dependent tool (low_level_main_SE and low_level_dependent_SE) (1103)

[0126] -Otherwise (1102)

[0127] ○ Decode low-level syntax elements related to main tool activation / deactivation (low_level_main_SE) from the bitstream (1104)

[0128] ○ If low_level_main_SE is equal to the given value V2 (1105)

[0129] ■Based on the value of the low-level syntax element low_level_main_SE related to main tool activation / deactivation, infer the low-level syntax element related to dependent tool activation / deactivation (low_level_dependent_SE) (1106)

[0130] ○Otherwise (1105)

[0131] ■Decode low-level syntax elements related to dependent tool activation / deactivation from the bitstream (low_level_dependent_SE) (1107)

[0132] - Apply the decoding process (1108) by activating / deactivating tools according to the values ​​of their low-level syntax elements (low_level_main_SE and low_level_dependent_SE)

[0133] Alternatively, at least two high level flags are used, one for controlling master tool activation and one for controlling slave tool activation, the second being constrained by the value of the first.

[0134] The treatment is summarized as follows (also as Figure 6 as depicted in ):

[0135] -Decoding a high-level syntax element (1201) associated with master tool activation / deactivation from the bitstream

[0136] - If the high-level syntax element associated with master tool activation / deactivation has a given value V1 (1202)

[0137] ○ Inferring high-level syntax elements related to activation / deactivation of slave tools based on the values ​​of high-level syntax elements related to activation / deactivation of the master tool (1203)

[0138] -Otherwise (1202)

[0139] ○ Decode high-level syntax elements related to slave tool activation / deactivation from the bitstream (1204)

[0140] - For a master tool or a slave tool (1205), the following applies:

[0141] o If the high-level syntax element has a given value V3 (which may be different for the master tool and the slave tool) (1206)

[0142] ■ Inferring low-level syntax elements related to tool activation / deactivation based on the value of high-level syntax elements (1207)

[0143] ○Otherwise (1206)

[0144] ■Decode low-level syntax elements related to tool activation / deactivation from the bitstream (1208)

[0145] - Apply the decoding process using the activation / deactivation of tools according to the value of their lower-level syntax elements General solution for case 3 (tool mutual exclusion)

[0146] The solution to case 3 is to define at least one high-level syntax element for controlling a first tool or a first tool set, and parameters for controlling a second tool or a second tool set that is mutually exclusive with the first tool or the first tool set.

[0147] The treatment is summarized as follows (also as Figure 7 as depicted in ):

[0148] - Decoding two high-level syntax elements high_level_SE1 and high_level_SE2 associated with a first toolset and a second toolset from a bitstream (1301)

[0149] - If high_level_SE1 is equal to the given value V1 (1302)

[0150] ○ Based on the value of high_level_SE1, infer the low-level syntax element (low_level_SE1) related to the first toolset activation / deactivation (1303)

[0151] -Otherwise (1302)

[0152] ○Decode low_level_SE1 (1304) from the bitstream

[0153] - If high_level_SE2 is equal to a given value V2, and low_level_SE1 is equal to a given value V3 (1305)

[0154] ○ Based on the value of highLevelParam2, infer the low-level syntax element (low_level_SE2) related to the second toolset activation / deactivation (1306)

[0155] -Otherwise (1305)

[0156] ○Decode low_level_SE2 (1307) from the bitstream

[0157] - Apply the decoding process (1308) by activating / deactivating the tool according to the values ​​of its low-level syntax elements low_level_SE1 and low_level_SE2

[0158] Advantageously, the invention enables to reduce the complexity of encoders and decoders by limiting the number of possible combinations of tools. It also allows to reduce the risk of implementation problems due to the need to support tool combinations involving increased complexity problems and increased tool incompatibility situations. It ultimately reduces interoperability problems that may be involved by enabling too many possible tool combinations.

[0159] For these different situations, examples are provided for the tools listed in Table 2 (the situations are indicated in brackets in the example title).

[0160] Example 1 - Joint Control of Partition Restrictions (Case 1)

[0161] Main Examples

[0162] In an embodiment, multiple partition restrictions associated with the partitioning (or codec structure) of a CTU or CU with at least one of the vertical or horizontal dimensions greater than a given limit (typically 64) are controlled by a single high-level syntax element (e.g., hl_picture_partitioning_restrictions_flag) inserted in an advanced parameter set (e.g., SPS).

[0163]

[0164] Furthermore, in a parameter set (eg, PPS, slice header, block group header, block header) at a lower level than a high-level parameter set, syntax elements related to a specific partition restriction are inserted.

[0165] As an example, the specific partition restrictions may be the following tools described in Table 2: C_TT64, C_QT128, C_BT, which are respectively controlled by the following low-level flags: ll_split_tt128_restriction_flag, ll_split_qt128_restriction_flag, ll_split_bt128_restriction_flag.

[0166] As an example, the following specific syntax elements are inserted in the chunk group header:

[0167]

[0168] Figure 8 A simplified decoding process block diagram is depicted. In step 301, a high-level partition restriction flag hl_picture_partitioning_restrictions_flag is decoded from the bitstream. Its value is checked in step 302. If hl_picture_partitioning_restrictions_flag is true, then a specific low-level syntax element associated with a specific partition restriction is set to true (step 303). If the flag is false, a specific low-level syntax element associated with a specific partition restriction is decoded in step 304. In step 305, a decoding process for a CTU or CU is applied using the specific partition restriction controlled by the low-level syntax element set in step 303 or decoded in step 304.

[0169] The corresponding semantics is an example:

[0170] hl_picture_partitioning_restrictions_flag equal to 1 specifies that the syntax elements 11_split_tt128_restriction_flag, 11_split_qt128_restriction_flag and 11_split_bt128_restriction_flag are not present in the tile group header of the picture in the CVS. hl_picture_partitioning_restrictions_flag equal to 0 specifies that the syntax elements 11_split_tt128_restriction_flag, 11_split_qt128_restriction_flag and 11_split_bt128_restriction_flag are present in the tile group header of the picture in the CVS.

[0171] ll_split_tt128_restriction_flag equal to 1 specifies that ternary tree partitioning is disabled when the CU size is greater than 64x64. ll_split_tt128_restriction_flag equal to 0 specifies that ternary tree partitioning is enabled when the CU size is greater than 64x64. When ll_split_tt128_restriction_flag is not present, it is set to 1.

[0172] ll_split_qt128_restriction_flag equal to 1 specifies that a CU of size 128x128 is forced to be split into four CUs of size 64x64. ll_split_qt128_restriction_flag equal to 0 specifies that a CU of size 128x128 is not forced to be split into four CUs of size 64x64. When ll_split_qt128_restriction_flag is not present, it is set to 1.

[0173] ll_split_bt128_restriction_flag equal to 1 specifies that a CU of size 64x128 is forced to be split into two CUs of size 64x64. ll_split_bt128_restriction_flag equal to 0 specifies that a CU of size 64x128 is not forced to be split into two CUs of size 64x64. When ll_split_bt128_restriction_flag is not present, it is set to 1.

[0174] Alternative 1

[0175] In the above solution, high-level control of partitioning restrictions facilitates activation of restrictions because hl_picture_partitioning_restrictions_flag equal to true involves that the specific partitioning restrictions flag is also set to true.

[0176] In the following embodiments, the method is reversed. When hl_picture_partitioning_restrictions_flag is equal to false, the specific partition restriction flags are also set to false. Otherwise, these specific partition restriction flags are read from the bitstream. This is done in Fig. 9 302a checks whether hl_picture_partitioning_restrictions_flag is equal to false. If this condition is true, then in step 303a, a specific low-level syntax element related to a specific partition restriction is set to false. The other steps are the same as in the previous embodiment.

[0177] Example 2

[0178] In an embodiment, there are multiple high-level partition restriction flags (N) in the syntax, and the semantics enable them to be controlled sequentially, as described below. In this example, the flag is generally named "hl_picture_partitioning_restrictionk_flag", where k is an integer value. And a low-level flag is additionally inserted at a lower level in the syntax (e.g., block group header), which is named "ll_picture_partitioning_restrictionk_kk_flag", where kk is an integer value. For a given restriction k, the high-level flag "hl_picture_partitioning_restrictionk_flag" controls the Mk low-level restriction, which is controlled by the flag "ll_picture_partitioning_restrictionk_kk_flag", kk=1 to Mk.

[0179]

[0180]

[0181] The corresponding semantics are examples:

[0182] hl_picture_partitioning_restrictionk_flag equal to 1 specifies that the syntax elements 11_picture_partitioning_restrictionka_flag, 11_picture_partitioning_restrictionkb_flag are not present in the block group header of the picture in the CVS. hl_picture_partitioning_restriction1_flag equal to 0 specifies that the syntax elements 11_picture_partitioning_restriction1a_flag, 11_picture_partitioning_restriction1b_flag are present in the block group header of the picture in the CVS.

[0183] In this example, the same semantics apply for hl_picture_partitioning_restrictionk_flag for any k from 1 to N.

[0184] ll_picture_partitioning_restrictionkx_flag equal to 1 specifies that picture partitioning restriction kx is enabled. ll_picture_partitioning_restrictionkx_flag equal to 0 specifies that picture partitioning restriction kx is disabled. When ll_picture_partitioning_restrictionkx_flag is not present, it is set to 1.

[0185] In this example, the same semantics apply for ll_picture_partitioning_restrictionkx_flag for any kk from 1 to Mk.

[0186] This may more specifically relate to the partition limits identified as C_TT64, C_QT128, C_BT in Table 2.

[0187] Example 3 - Joint control of other tools related to partitioning (Case 1)

[0188] The same approach can be applied to tools related to the following partition-related tools described in Table 2: C_PR, C_FBP.

[0189] The detailed syntax and semantics can be completed as above.

[0190] Example 4 - Control of Multi-hypothesis Forecasting Tool (Case 1)

[0191] Main Examples

[0192] In an embodiment, the inter prediction tool based on multi-hypothesis prediction is controlled by a high-level single syntax element (e.g., hl_multi_hypothesis_enable_flag) inserted in a high-level parameter set (e.g., SPS). In the current document, the multi-hypothesis prediction tool involves weighted mixing of samples from several predictions with spatially varying weights.

[0193]

[0194] Furthermore, in a parameter set (eg, PPS, slice header, block group header, block header) at a lower level than a high-level parameter set, syntax elements related to a specific multi-hypothesis prediction tool are inserted.

[0195] As an example, the specific multi-hypothesis prediction tools may be the following tools described in Table 2: R_GBI, R_TRI, R_MH, which are respectively controlled by the following low-level flags: ll_generalized_bipred_enable_flag, ll_triangular_partition_enable_flag, ll_multi_hypothesis_intra_inter_enable_flag. In addition, R_OBMC may also be included in this toolset, controlled by a specific flag (e.g., ll_obmc_enable_flag).

[0196] As an example, the following specific syntax elements may be inserted into the chunk group header:

[0197]

[0198] Fig.10 A simplified decoding process block diagram is depicted. In step 401, a flag hl_multi_hypothesis_enable_flag is decoded from the bitstream. Its value is checked in step 402. If the flag is true, a specific syntax element related to a specific multi-hypothesis prediction tool is set to true (step 403). If hl_multi_hypothesis_enable_flag is true, a specific low-level syntax element related to a specific multi-hypothesis prediction tool is decoded in step 404. In step 405, the decoding process for the CTU or CU is applied using the specific multi-hypothesis prediction tool controlled by the low-level syntax element set in step 403 or decoded in step 404.

[0199] The corresponding semantics is an example:

[0200] hl_multi_hypothesis_enable_flag equal to 1 specifies that the syntax elements ll_generalized_bipred_enable_flag, ll_triangular_partition_enable_flag, ll_multi_hypothesis_intra_inter_enable_flag, ll_obmc_enable_flag are present in the block group header of the picture in the CVS. hl_multi_hypothesis_enable_flag equal to 0 specifies that the syntax elements ll_generalized_bipred_enable_flag, ll_triangular_partition_enable_flag, ll_multi_protection_intra_inter_enable_flag, ll_obmc_enable_flag are not present in the block group header of the picture in the CVS.

[0201] ll_generalized_bipred_enable_flag equal to 1 specifies that the generalized bi-prediction mode is enabled. ll_generalized_bipred_enable_flag equal to 0 specifies that the generalized bi-prediction mode is disabled. When ll_generalized_bipred_enable_flag is not present, it is set to 0.

[0202] Equivalent semantics apply to the syntax elements ll_triangular_partition_enable_flag, ll_multi_protection_intra_inter_enable_flag, ll_obmc_enable_flag.

[0203] Alternatives

[0204] As in the previous section, in an alternative solution, there are multiple high-level multi-hypothesis prediction enabling flags in the syntax, and the semantics enable controlling them together.

[0205] The detailed syntax & semantics can be done as in the previous section.

[0206] Example 5 - Control within and between MTS frames (Case 2)

[0207] In an embodiment, one or two flags are inserted at a high level to control MTS (Multiple Transform Set) activation. This is related to the following tools described in Table 2: T_MTS_intra, T_MTS_intra. It is also possible that T_ST can be controlled by the same mechanism.

[0208] Solution 1

[0209] In the first example, a single high-level flag hl_mts_enable_flag is added as shown in the following table.

[0210]

[0211] Depending on the value of the flag hl_mts_enable_flag, one or two additional flags are inserted at a lower level to control MTS activation for intra-frame or inter-frame regions (such as slices, tile groups or tiles), as shown in the following table.

[0212]

[0213] The decoding process is conceptually similar to that described for controlling the multiple hypothesis prediction tool.

[0214] The corresponding semantics are proposed.

[0215] hl_mts_enable_flag equal to 1 specifies that the syntax elements ll_mts_intra_enable_flag and ll_mts_inter_enable_flag are present in the tile group header of a picture in the CVS. hl_mts_enable_flag equal to 0 specifies that the syntax elements ll_mts_intra_enable_flag and ll_mts_inter_enable_flag are not present in the tile group header of a picture in the CVS.

[0216] ll_mts_intra_enable_flag equal to 1 specifies that multiple transform set mode is enabled for CUs coded in intra mode. ll_mts_intra_enable_flag equal to 0 specifies that multiple transform set mode is disabled for CUs coded in intra mode. When ll_mts_intra_enable_flag is not present, it is set to 0.

[0217] ll_mts_inter_enable_flag equal to 1 specifies that multiple transform set mode is enabled for CUs coded in inter mode. ll_mts_inter_enable_flag equal to 0 specifies that multiple transform set mode is disabled for CUs coded in inter mode. When ll_mts_inter_enable_flag is not present, it is set to 0.

[0218] Solution 2

[0219] The second example is based on the first example. In addition, the low-level flag controlling the MTS activation between frames is subject to the MTS activation within a frame. This can be expressed as follows.

[0220]

[0221] Alternatively, similar behavior can be achieved by two high-level flags, as shown in the following table.

[0222]

[0223] The corresponding semantics is an example:

[0224] hl_mts_intra_enable_flag equal to 1 specifies that the syntax element 11_mts_intra_enable_flag is present in the tile group header of the picture in the CVS. hl_mts_intra_enable_flag equal to 0 specifies that the syntax element 11_mts_intra_enable_flag is not present in the tile group header of the picture in the CVS.

[0225] hl_mts_inter_enable_flag equal to 1 specifies that the syntax element 11_mts_inter_enable_flag is present in the tile group header of the picture in the CVS. hl_mts_inter_enable_flag equal to 0 specifies that the syntax element 11_mts_inter_enable_flag is not present in the tile group header of the picture in the CVS. When hl_mts_inter_enable_flag is not present, it is set to 0.

[0226] ll_mts_intra_enable_flag equal to 1 specifies that multiple transform set mode is enabled for CUs coded in intra mode. ll_mts_intra_enable_flag equal to 0 specifies that multiple transform set mode is disabled for CUs coded in intra mode. When ll_mts_intra_enable_flag is not present, it is set to 0.

[0227] ll_mts_inter_enable_flag equal to 1 specifies that multiple transform set mode is enabled for CUs coded in inter mode. ll_mts_inter_enable_flag equal to 0 specifies that multiple transform set mode is disabled for CUs coded in inter mode. When ll_mts_inter_enable_flag is not present, it is set to 0.

[0228] Fig.12 A corresponding decoding process block diagram is depicted.

[0229] Secondary transformation activation dependent on MTS activation

[0230] In an embodiment, activation of the secondary transform mode (T_ST) is subject to activation of the MTS mode.

[0231] This can be expressed as follows.

[0232]

[0233] Similarly, activation of the recursive transform tree mode (T_RTT) is subject to activation of the MTS mode.

[0234] Example 6 - A_MDLM control based on A_CCLM activation (Case 2)

[0235] In an embodiment, the mechanism described for the MTS case is applicable to the cases of the tools A_CCLM and A_MDLM described in Table 2, and the activation / deactivation of A_MDLM is subject to the activation / deactivation of A_CCLM.

[0236] The detailed syntax and semantics can be completed as above.

[0237] Example 7 - ATMVP control based on ATMVP activation (Case 2)

[0238] In an embodiment, the mechanism described for the MTS case is applicable to the cases of the tools R_TMVP and R_ATMPV described in Table 2, and the activation / deactivation of R_ATMPV is subject to the activation / deactivation of R_TMVP. The detailed syntax & semantics can be completed as above.

[0239] Embodiment 8 - Inter-frame tool control based on high-precision MV codec activation (case 2)

[0240] In an embodiment, activation / deactivation of R_AFF is subject to activation / deactivation of R_HPMV, which is implemented in a manner similar to that described for the MTS case.

[0241] The detailed syntax and semantics can be completed as above.

[0242] In another embodiment, the activation / deactivation of R_MMVD, R_DMVR is subject to the activation / deactivation of R_HPMV, which is implemented in a manner similar to that described for the MTS case.

[0243] The detailed syntax and semantics can be completed as above.

[0244] In another embodiment, the activation / deactivation of R_PAMC is subject to the activation / deactivation of R_HPMV, which is implemented in a manner similar to that described for the MTS case. The detailed syntax & semantics can be completed as above.

[0245] Example 9 - Dependency of palette mode and current picture reference on activation of separate luma / chroma trees (Case 2)

[0246] It is observed that the palette mode highly benefits from the activation of the use of separate luma / chroma trees.

[0247] In an embodiment, the activation / deactivation of A_PM and A_CPR is subject to the activation / deactivation of C_CST, which is implemented in a manner similar to that described for the MTS case. The detailed syntax & semantics can be completed as above.

[0248] Example 10 - Joint Control of Dependency Quantification and Symbolic Data Hiding Tools (Case 3)

[0249] It is observed that tool-dependent quantization (Q_DQ) and symbolic data hiding (C_SDH) are difficult to work together.

[0250] In an embodiment, high-level syntax control of Q_DQ and C_SDH is designed to avoid common usage of these two tools.

[0251] Two high-level flags hl_dependent_quant_enable_flag and hl_sign_data_hiding_flag are added as shown in the following table.

[0252]

[0253] The low-level activation syntax elements associated with these two tools are then set based on the values ​​of hl_dependent_quant_enable_flag and hl_sign_data_hiding_flag. However, one of the flags controlling a given tool is only read if the other flag indicates that the other tool is deactivated. The following table illustrates this.

[0254]

[0255]

[0256] The corresponding semantics is an example.

[0257] hl_dependent_quant_enable_flag equal to 1 specifies that the syntax element ll_dependent_quant_enable_flag is present in the block group header of the picture in the CVS. hl_dependent_quant_enable_flag equal to 0 specifies that the syntax element ll_dependent_quant_enable_flag is not present in the block group header of the picture in the CVS.

[0258] Similar semantics are used for hl_sign_data_hiding_flag.

[0259] ll_dependent_quant_enable_flag equal to 1 specifies that dependent quantization mode is enabled. ll_dependent_quant_enable_flag equal to 0 specifies that dependent quantization mode is disabled. When ll_dependent_quant_enable_flag is not present, it is set to 0.

[0260] ll_sign_data_hiding_enable_flag equal to 1 specifies that the sign data hiding mode is enabled. ll_sign_data_hiding_enable_flag equal to 0 specifies that the sign data hiding mode is disabled. When ll_sign_data_hiding_enable_flag is not present, it is set to 0.

[0261] Fig.13 A simplified block diagram of the decoding process is depicted.

[0262] Example 11 - Joint control of separate luminance / chrominance trees and cross-component linear model / multi-directional LM tools (case 3)

[0263] In an embodiment, high-level syntax control of C_CST, A_CCLM, and A_DMLM is such that C_CST activation requires deactivation of A_CCLM and A_DMLM, and conversely, C_CST deactivation requires activation of A_CCLM and A_DMLM.

[0264] Similar designs to those described in the previous subsections above may be used.

[0265] The detailed syntax and semantics can be completed as above.

[0266] Summary of tool categories

[0267] This section summarizes the classification of tools in three considered situations.

[0268] ·Situation 1

[0269] o C_TT64, C_QT128, C_BT can be grouped into a single flag associated with the split constraint used to make the split VPDU friendly

[0270] o C_PR and C_FBP can be grouped as tools related to partition control

[0271] o R_GBI, R_TRI, R_MH can be grouped as tools based on the same concept as multiple hypotheses with weighted samples from 2 predictions, and logically should be activated / deactivated together; R_OBMC can also be considered conceptually in the same set as multiple hypotheses with weighted samples

[0272] ·Case 2

[0273] o A_CCLM being deactivated means A_MDLM being deactivated

[0274] o R_TMVP being deactivated means R_ATMVP being deactivated

[0275] o T_MTS_intra is deactivated which means T_MTS_inter is deactivated

[0276] o T_RTT performance potentially depends on T_MTS, so deactivation of T_MTS means T_RTT is deactivated

[0277] o R_HPMV disabling may affect R_AFF, R_MMVD, R_DMVR,

[0278] R_PAMC codec efficiency, so its deactivation means that some or all of the following tools are also deactivated: R_AFF, R_MMVD, R_DMVR, R_PAMCo C_CST may have a strong impact on A_PM for screen content due to the heavy use of palette modes that favor separate luma / chroma trees, so deactivation of C_CST means that A_PM is also deactivated

[0279] o C_CST being deactivated means A_CPR being deactivated

[0280] Case 3

[0281] ○ When C_SDH is activated, Q_DQ should be deactivated, and vice versa

[0282] ○ When A_CCLM and A_MDLM are activated, C_CST should be deactivated, and vice versa

[0283] The described aspects and embodiments may provide codec gains, i.e., for the same bitrate, improved quality, or for the same quality, reduced bitrate. Furthermore, by limiting the possibilities of tool combinations, it may greatly simplify the implementation of codecs. This is a key point for interoperability.

[0284] Another aspect of the toolkit is related to the syntax of multi-component video. One problem that arises with the new tools in VVC is that some of these tools may have a negative impact on performance, or may simply not be properly designed to work together when applied to individual components of a multi-component picture. A typical example is the way of handling chroma separation trees. In a typical approach, the picture to be encoded and decoded may include a luma component (e.g., Y') and 2 chroma components (typically Cb, Cr) or 3 components (e.g., R, G, B). In the case of a separate CU-tree, the luma component and the 2 chroma components are encoded and decoded independently. When dealing with such individually encoded multiple components, the possible relationships between the components are not defined.

[0285] Table 3 Example of multi-component syntax (new elements are in italics)

[0286]

[0287]

[0288] The advantage of encoding and decoding independent (separate) components in the same bitstream is that the GOP structure is the same for all components: reference frames, IDR frames, etc. are the same. Another advantage is parallelization (component level encoding and decoding pipelines).

[0289] The general aspects described next are intended to set up tool interactions between components for multi-channel formats.

[0290] The syntax and behavior between codec components need to be created for the candidate tools proposed in VVC that will introduce dependencies between components.

[0291] If multi-channel formats are used, this problem has yet to be solved in JVET. Note that even if multi-channel formats are not used, the same problem needs to be solved for RGB formats or other image formats (image+depth or auxiliary images...).

[0292] The general aspects described next propose to provide a process that specifies tool interactions between channels. This can be performed by inserting syntax changes and / or semantic changes. The invention describes several variations depending on the relationship between components.

[0293] List of tools considered

[0294] Table 2 gives a list of the tools included in VTM3 and some additional tools that are being explored and may be added to VTM in the future. The tools are organized into categories.

[0295] Table 2. List of tools considered in VTM3 and used for further VTM

[0296]

[0297]

[0298] Several SPSs may be signaled in the bitstream. These SPSs may include flags for controlling tools at the sequence level.

[0299] In addition to the SPS, additional parameter sets at lower levels in the syntax may be specified (such as PPS, slice header, chunk group header, chunk header). This may include additional flags for controlling tools at a level lower than the sequence level.

[0300] Some terms and concepts that are helpful in understanding the present invention are as follows:

[0301] Quantity : Same as defined in the H.26x standard (e.g., in clause 3.37 of the February 2018 version of HEVC)

[0302] Main Components : The component on which the secondary component depends. Typically, the primary component does not depend on any other component during the decoding process.

[0303] Note 1: The main component may have a dependency on an entropy codec (e.g. CABAC)

[0304] Note 2: Typically, the primary component is the luminance component of a video signal that includes luminance and two chrominance channels.

[0305] Secondary Components : Components that depend on the primary component during the decoding process.

[0306] Note 3: Typically, the secondary components are the luminance and 2 chrominance channels of a video signal.

[0307] (Non)Separation Components : Components that have (in)dependencies during the decoding process.

[0308] Component Set : A set of independent components.

[0309] Note 4: A component set usually includes dependent major and minor components.

[0310] Note 5: An example of two component sets would be a first component set including Y' (primary component), Cb (secondary component), Cr (secondary component) components, and a second component set including mask (primary component) and depth (secondary component) channels

[0311] Note 6: Components may be referred to as "color planes"

[0312] Example 12: Signaling and logic for separated components

[0313] In one embodiment, the components are signaled as separate components or as non-separate components.By means of parsing the structure sps_active_tools_set, a method for describing the state of active tools associated with the encoding and decoding of separate or non-separate components is applied.

[0314] One advantage of the approach is that some codec tools may be deactivated when they are known to be inefficient for a given component type (eg this may be constrained in a configuration file). The following table provides syntax examples, where syntax changes are highlighted in grey.

[0315]

[0316] By design, if a non-primary component is not marked as separate, it is a secondary component (e.g., Y). As described above, the toolsets activated for primary and secondary components are indicated at the first occurrence of sps_active_tools_set(). A typical example is a "Y'CbCr+depth" video signal, where the Y', Cb, and Cr components are non-separated components, but the depth component is a separate component.

[0317] Fig.18 A simplified block diagram of a decoding process according to an embodiment is depicted. In step 1001, syntax elements related to tool activation are decoded. They apply at least to the main component. In step 1002, a loop is performed on the remaining components (indexed i, i greater than 1, i=0 corresponds to the main component). In step 1003, the flag separate_cmp_flag[i] is decoded. The value of the flag is checked in step 1004. If the flag is equal to true, step 1005 is applied. Step 1005 performs decoding of syntax elements related to tool activation for component i. If the flag is equal to false, the syntax elements related to tool activation decoded in step 1001 apply to component i. The final step 1006 corresponds to performing a decoding process using the decoded syntax elements and possibly additional low-level syntax elements that control the tool at a level lower than the sequence level to activate or deactivate the tool.

[0318] Example of sps_active_tools_set()

[0319] A typical example definition of sps_active_tools_set() is as follows:

[0320]

[0321] Note that the syntax elements described in the above table may be indexed by component index.

[0322] There are specific tools that involve inter-component dependencies, meaning that the decoding process of at least one component involves accessing samples from another component. One example is the Cross-Component Linear Model (CCLM) tool, which is used to predict chroma samples from luma samples.

[0323] Example 13: Signaling and logic in the case of correlated components

[0324] When some components are non-separable, their corresponding high-level syntax elements (eg, present in SPS, PPS, ...) for controlling tool activation / deactivation may be grouped in only one set.

[0325] The primary and secondary components have dependencies during the decoding process; for example:

[0326] - For an inter slice or inter CTU: all components are considered non-separable (eg, motion compensation is signaled and done during the same phase for all components).

[0327] - For intra slices: components are considered non-separated when C_CST (chroma separation tree) or any other inter-component tool is disabled, or separated (some relationship still exists from Y to UV) when C_CST or any other inter-component tool is enabled. This is especially true for tools that use prediction of chroma data from co-located luma data.

[0328] To describe these relationships, the following syntax is proposed:

[0329]

[0330] When the value of primary_cmp_id_plus1[i] is equal to 0, the component of index i is considered to be the primary component. By default, the first component (index 0) is the primary component. If the value is not 0, it indicates that the primary component is used by the secondary component. The syntax elements that signal the activation status of the codec tools for the components are stored in sps_active_tools_set(). If necessary, use decode_inter_components_tools_flags(i) to parse the tools related to the relationship between components.

[0331] A typical example is the YUV format, where Y is the primary component and UV is the secondary component.

[0332] number_of_components_group_minus[i] specifies the number of components in the component set to which primary component i belongs. If the associated component group has more than one component, a flag indicating inter-component tool dependency is used.

[0333] For example, in the case of a 4:2:0 (non-separated) YUV format, the following values ​​are obtained for the considered syntax elements:

[0334]

[0335] Another example of values ​​for a multi-channel bitstream containing YUV+Depth is given by the following table:

[0336]

[0337]

[0338] The following table gives examples of the decode_inter_components_tools_flags syntax:

[0339]

[0340] The flags in decode_inter_components_tools_flags enable or disable the mode of using data from the primary components.

[0341] For example, lm_chroma_mode allows using CCLM tools to predict components from the primary component. The flag separate_chroma_tree allows having separate trees in intra slices for encoding and decoding the current component. When the flag separate_chroma_tree is true, it is assumed that all components except the primary component are grouped together (i.e., share the same tree). For other cases, the secondary components use the same signaling as the primary component.

[0342] Additional Secondary Component Syntax

[0343] Table 4. Additional minor component syntax

[0344]

[0345] In a variation, the syntax described in Table 4 is added to the syntax.

[0346] The first flag CABAC_dependency signals whether the primary and secondary components share some dependencies during the CABAC entropy decoding process. If no, it means that the entire primary component can be parsed independently of the secondary component.

[0347] The flag chroma_separate_tree_inter signals whether inter slices / CTUs use a separation tree similar to the chroma_separate_tree currently used for intra slices / CTUs. If this flag is true, an additional flag inter_direct_mode indicates whether the secondary component can derive some inter data (usually co-located motion information) from the primary component.

[0348] The flag reduced_tools_set_chroma indicates whether the secondary component uses a reduced toolset. For example, in the current version, the reduced toolset is:

[0349] - Fewer intra-frame directions

[0350] - Smaller number of possible transform types (MTS)

[0351] -etc.

[0352] Example 14: Finer Control of Secondary Component Tools

[0353] In a component set, the relationship between the components and the tool state may need to be described more precisely and / or may depend on the component. In this case, an additional flag separate_tools_flag assigned to each secondary component is read. This flag indicates whether a specific tool control syntax element is inserted for this secondary component. If this flag is true, the tool enable / disable flags are parsed for this secondary component (using a call to decode_tools_flags()).

[0354] The associated syntax is given below:

[0355]

[0356]

[0357] In this case, the secondary components are grouped together and share the same flags of the inter-component tool.

[0358] Secondary Component Relationship

[0359] A more general approach is to fully describe the inter-component relationships of a tool. This has the advantage of allowing to have non-separated component groups, where both regular (non-inter-component) tools and inter-component tool states (activation / deactivation) are customized for each component.

[0360] An example of the associated syntax is given below:

[0361]

[0362]

[0363] In this variant, the decoding of component 0 parameters has been placed within the component loop for simplicity.

[0364] For a given component, when separate_tools_flag is true, the inter-component tools syntax elements for the component are decoded (calling decode_inter_components_tools_flags_2(i)). In addition, when the flag custom_decode_flag is true, there may be specific syntax elements that control non-inter-component tools for this component.

[0365] The decode_inter_components_tools_flags_2 flag performs the following operations:

[0366]

[0367] For separate components, if the inter-tool component is activated, the component to inherit is read in the bitstream (syntax element component_to_inherit_from). Inheritance of a component means:

[0368] - If chroma_separate_tree is false, the component's tree is shared with the component specified by component_to_inherit_from

[0369] - For other tools (CPR, LM, etc.), if the flag associated with the mode is true, the data (motion vector predictor, pixel samples, etc.) is inherited from the component specified by component_to_inherit_from

[0370] When separate_tools_flag is false, inter-component tools are set to true, except for the separate chroma_separate_tree which is set to false.

[0371] Reads an additional flag for customizing non-inter-component tools to determine whether the component can customize each tool individually. If the flag is false, the flag of the associated primary component is used.

[0372] The invention may provide codec gains, ie improved quality for the same bit rate, or reduced bit rate for the same quality, because it allows for an accurate description of the relationship between components of the same temporal data.

[0373] Fig.19 One embodiment of a method 1900 under the general aspects described herein is shown in FIG. 19. The method begins at start block 1901, and control proceeds to block 1910 to insert syntax associated with at least one codec tool into a video bitstream. Control proceeds from block 1910 to block 1920 to conditionally insert one or more controls for the codec tools into the bitstream based on the high-level syntax inserted into the bitstream. Control proceeds from block 1920 to block 1930 to activate one or more codec tools corresponding to a video component based on the one or more controls. Control proceeds from block 1930 to block 1940 to encode the video bitstream using the activated codec tools.

[0374] Fig. 20 One embodiment of a method 2000 under the general aspects described herein is shown. The method begins at start block 2001, and control proceeds to block 2010 to parse a video bitstream to obtain syntax associated with at least one video codec. Control proceeds from block 2010 to block 2020 to determine one or more controls for a codec based on the video bitstream syntax. Control proceeds from block 2020 to block 2030 to activate one or more codecs corresponding to a video component based on the determination of the controls. Control proceeds from block 2030 to block 2040 to decode the video bitstream using the activated codec.

[0375] Fig.21 An embodiment of an apparatus 2100 is shown for encoding, decoding, compressing or decompressing video data using a simplified codec mode based on a parameter model dependent on neighboring samples. The apparatus includes a processor 2110 and may be interconnected to a memory 2120 via at least one port. Both the processor 2110 and the memory 2120 may also have one or more additional interconnections to external connections.

[0376] The processor 2110 is also configured to insert or receive information in a bitstream and compress, encode or decode using any of the described aspects.

[0377] The present application describes various aspects, including tools, features, embodiments, models, methods, etc. Many aspects in these aspects are specifically described, and are usually described in a manner that may sound restrictive at least to show their respective characteristics. However, this is for the purpose of describing clearly, and does not limit the application or scope of those aspects. In fact, all different aspects can be combined and interchanged to provide further aspects. In addition, these aspects can also be combined and interchanged with the aspects described in earlier files.

[0378] The aspects described and contemplated in this application can be implemented in a variety of different forms. Figure 1 , 2 and 17 provide some embodiments, but other embodiments are contemplated, and Figure 1 , 2 The discussion of 17 does not limit the breadth of implementations. At least one of these aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the methods.

[0379] In this application, the terms "reconstruction" and "decoding" can be used interchangeably, the terms "pixel" and "sample" can be used interchangeably, and the terms "image", "picture" and "frame" can be used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, and "decoding" is used on the decoder side.

[0380] Various methods are described herein, and each of these methods includes one or more steps or actions for implementing the method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0381] Various methods and other aspects described in this application can be used to modify modules, such as Figure 1 and Figure 2 Intra-frame prediction, entropy coding and / or decoding modules (160, 360, 145, 330) of the video encoder 100 and decoder 200 shown. In addition, the present aspects are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations (whether existing or developed in the future), and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise stated, or technically excluded, the various aspects described in this application can be used alone or in combination.

[0382] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes, and the described aspects are not limited to these specific values.

[0383] Figure 1 An encoder 100 is shown. Variations of the encoder 100 are contemplated, but for clarity, the encoder 100 is described below without describing all contemplated variations.

[0384] Before being encoded, the video sequence may undergo a pre-encoding process (101), for example, applying a color transform to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0385] In the encoder 100, a picture is encoded by an encoder element as described below. The picture to be encoded is partitioned (102) and processed, for example, in units of CUs. Each unit is encoded using, for example, an intra-frame or inter-frame mode. When the unit is encoded in intra-frame mode, it performs intra-frame prediction (160). In inter-frame mode, motion estimation (175) and compensation (170) are performed. The encoder determines (105) which of the intra-frame mode or inter-frame mode is used to encode the unit, and indicates the intra-frame / inter-frame determination by, for example, a prediction mode flag. For example, a prediction residual is calculated by subtracting (110) the prediction block from the original image block.

[0386] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy encoded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., encode and decode the residual directly without applying the transform or quantization process.

[0387] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are inversely quantized (140) and inversely transformed (150) to decode the prediction residual. The image block is reconstructed by combining (155) the decoded prediction residual and the prediction block. An in-loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0388] Figure 2A block diagram of a video decoder 200 is shown. In the decoder 200, a bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs the same Figure 1 The decoding process is the reciprocal of the encoding process described in . The encoder 100 also typically performs video decoding as part of encoding the video data.

[0389] Specifically, the input to the decoder includes a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other codec information. The picture partition information indicates how the picture is partitioned. Thus, the decoder may partition the picture according to the decoded picture partition information (235). The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The image block is reconstructed by combining (255) the decoded prediction residual and the prediction block. The prediction block may be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0390] The decoded image may be further subjected to post-decoding processing (285), such as an inverse color transform (e.g., from YCbCr 4:2:0 to RGB 4:4:4), or inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived from the pre-encoding process and signaled in the bitstream.

[0391] Fig.17 A block diagram of an example of a system in which various aspects and embodiments are implemented is shown. System 1000 can be implemented as a device including various components described below, and is configured to perform one or more of the various aspects described in this document. Examples of such equipment include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, and servers. The elements of system 1000 can be implemented in a single integrated circuit (IC), multiple ICs, and / or discrete components, either individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed on multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or by dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more of the various aspects described in this document.

[0392] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing, for example, various aspects described in this document. The processor 1010 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a magnetic disk drive, and / or an optical disk drive. As non-limiting examples, the storage device 1040 may include an internal storage device, an additional storage device (including removable and non-removable storage devices), and / or a network accessible storage device.

[0393] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a (multiple) module that can be included in a device to perform encoding and / or decoding functions. As known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000, or may be incorporated into the processor 1010 as a combination of hardware and software known to those skilled in the art.

[0394] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described in this document may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operation logic.

[0395] In some embodiments, memory internal to the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) may be used for one or more of these functions. The external memory may be memory 1020 and / or a storage device 1040, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, an external non-volatile flash memory is used to store, for example, an operating system for a television. In at least one embodiment, fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, while 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Codec, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Codec, a new standard being developed by the Joint Video Experts Group JVET)).

[0396] As shown in block 1130, input to the elements of system 1000 may be provided through various input devices. Such input devices include, but are not limited to: (i) a radio frequency (RF) component that receives a radio frequency (RF) signal transmitted over the air, such as by a broadcast station, (ii) a component (COMP) input (or a collection of COMP inputs), (iii) a universal serial bus (USB) input, and / or (iv) a high-definition multimedia interface (HDMI) input. Fig.17 Other examples not shown include composite video.

[0397] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF component can be associated with elements suitable for the following operations: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select (for example) a signal frequency band (which may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a stream of desired data packets. The RF components of various embodiments include one or more elements that perform these functions, such as a frequency selector, a signal selector, a frequency band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF component can include a tuner that performs a variety of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband. In a set-top box embodiment, the RF components and the associated input processing element thereof receive the RF signal transmitted by wired (for example, cable) medium, and perform frequency selection by filtering, down-conversion and filtering to the desired frequency band again. Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements of performing similar or different functions. Adding element can include inserting element between existing elements, for example, inserting amplifier and analog-to-digital converter. In various embodiments, the RF components include antenna.

[0398] In addition, USB and / or HDMI terminals may include corresponding interface processors for connecting system 1000 to other electronic devices via USB and / or HDMI. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, can be implemented as needed in, for example, a separate input processing IC or in processor 1010. Similarly, various aspects of USB or HDMI interface processing can be implemented as needed in a separate interface IC or in processor 1010. The stream through demodulation, error correction and demultiplexing is provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operates in combination with memory and storage element to process data streams as needed for presentation on output devices.

[0399] The various components of system 1000 may be disposed within an integrated housing. Within the integrated housing, the various components may be interconnected and transfer data between using a suitable connection arrangement 1140 (e.g., an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and printed circuit boards).

[0400] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data through the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented in, for example, a wired and / or wireless medium.

[0401] In various embodiments, data is streamed or otherwise provided to the system 1000 using a wireless network such as a Wi-Fi network (e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)). The Wi-Fi signal of these embodiments is received through a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 of these embodiments is typically connected to an access point or router that provides access to an external network including the Internet, for allowing streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to the system 1000, and the set-top box transmits data through the HDMI connection of the input box 1130. Still other embodiments use the RF connection of the input box 1130 to provide streaming data to the system 1000. As described above, various embodiments provide data in a non-transmission manner. In addition, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0402] The system 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be used for a television, a tablet computer, a laptop computer, a cellular phone (mobile phone), or other devices. The display 1100 can also be integrated with other components (for example, as in a smart phone), or separated (for example, an external monitor of a laptop computer). In various examples of embodiments, other peripheral devices 1120 include one or more of a stand-alone digital video disk (or digital versatile disk) (DVR for both terms), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functions based on the output of the system 1000. For example, a disk player performs the function of playing the output of the system 1000.

[0403] In various embodiments, control signals are transmitted between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 using signaling such as AV.Link, consumer electronics control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicatively coupled to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to the system 1000 via communication interface 1050 using communication channel 1060. In an electronic device (e.g., a television), the display 1100 and speaker 1110 may be integrated into a single unit with other components of the system 1000. In various embodiments, the display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0404] Display 1100 and speaker 1110 may alternatively be separate from one or more of the other components, such as if the RF components of input 1130 are part of a separate set-top box. In various embodiments where display 1100 and speaker 1110 are external components, output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0405] These embodiments may be implemented by computer software implemented by the processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. As a non-limiting example, the memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as an optical storage device, a magnetic storage device, a semiconductor-based storage device, a fixed memory, and a removable memory. As a non-limiting example, the processor 1010 may be of any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0406] Various implementations include decoding. "Decoding" as used in this application may include, for example, all or part of the processing performed on the received coded sequence to produce a final output suitable for display. In various embodiments, such processing includes one or more of the processing typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processing also includes or may alternatively include processing performed by a decoder of the various implementations described in this application.

[0407] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to refer generally to a broader decoding process will be clear based on the context of the specific description and is considered to be fully understood by those skilled in the art.

[0408] Various implementations involve encoding. In a manner similar to the discussion above about "decoding", "encoding" as used in this application may include, for example, all or part of the processing performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processing includes one or more of the processing typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processing also includes or alternatively includes processing performed by an encoder of the various implementations described in this application.

[0409] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to refer generally to a broader encoding process will be clear based on the context of the specific description and is considered to be fully understood by those skilled in the art.

[0410] Note that the syntax elements used herein are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0411] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0412] Various embodiments may refer to parameter models or rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is considered, usually given the constraints of computational complexity. It can be measured by rate-distortion optimization (RDO), or by least mean square (LMS), mean absolute error (MAE), or other such measurements. Rate-distortion optimization is usually expressed as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods to solve the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all coding options (including all considered modes or coding parameter values), as well as a complete evaluation of their coding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save coding complexity, specifically to calculate approximate distortion based on prediction or prediction residual signals rather than reconstructed signals. A mixture of these two methods can also be used, such as by using approximate distortion for only some of the possible coding options and using full distortion for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many approaches employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete assessment of both coding cost and associated distortion.

[0413] The implementation and various aspects described herein can be implemented, for example, with methods or processing, equipment, software programs, data streams, or signals. Even if only discussed in the context of a single form of implementation (e.g., only discussed as a method), the implementation of the discussed feature can also be implemented in other forms (e.g., equipment or programs). Equipment can be implemented, for example, with appropriate hardware, software, and firmware. The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as, for example, a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that contribute to the information communication between end users.

[0414] References to "one embodiment" or "an embodiment" or "an implementation" or "an implementation", and other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation", and any other variations thereof, appearing in various places throughout this application do not necessarily all refer to the same embodiment.

[0415] Additionally, this application may refer to “determining” various pieces of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0416] Additionally, this application may refer to "accessing" various pieces of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0417] Furthermore, the present application may refer to "receiving" various pieces of information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information, or retrieving information (e.g., from a memory). Furthermore, "receiving" is often referred to in one way or another during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0418] It will be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," use of any of the following " / ," "and / or," and "at least one of..." is intended to include selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to multiple items listed, as will be apparent to one of ordinary skill in this and related arts.

[0419] In addition, as used herein, the term "signal" refers to, among other things, indicating something to a corresponding decoder. For example, in some embodiments, the encoder signals a specific one of a plurality of transforms, coding modes or flags. In this way, in an embodiment, the same transform, parameter or mode is used on both the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It will be understood that signaling can be done in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal" is mentioned above, the word "signal" can also be used as a noun in this article.

[0420] It will be apparent to one of ordinary skill in the art that various implementations can generate various signals that are formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for executing a method, or data generated by one of the implementations. For example, a signal can be formatted to carry a bitstream of the embodiment. Such a signal can be formatted as, for example, an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. As is known, the signal can be transmitted by various wired or wireless links. The signal can be stored on a processor readable medium.

[0421] We have described multiple embodiments across various claim categories and types. Features of these embodiments may be provided individually or in any combination. In addition, embodiments may include one or more of the following features, devices, or aspects across various claim categories and types, individually or in any combination:

[0422] · Restrict the processes or devices that activate or deactivate tools in encoders or decoders.

[0423] • Processes or devices that operate on high-level syntax or semantics constraints in encoders or decoders, constraining the possible set of tool combinations.

[0424] ●A process or device that includes the syntax or semantics for specifying tool combination constraints.

[0425] ●Use or insert syntax in the bitstream to control the partitioning process in the encoder or decoder.

[0426] ●Use or insert syntax in the bitstream to control partition-related processing in the encoder or decoder.

[0427] ●Use or insert syntax in the bitstream to control the multi-hypothesis prediction tools in the encoder or decoder.

[0428] ●Use or insert syntax in the bitstream to control the MTS (Multi-core Transform) processing in the encoder or decoder.

[0429] ●Use or insert syntax in the bitstream to control the secondary transform mode in the encoder or decoder.

[0430] ●Use or insert syntax in the bitstream to control the recursive transform tree mode in the encoder or decoder.

[0431] ●Use or insert syntax in the bitstream to control the cross-component linear model or multi-directional linear model mode in the encoder or decoder.

[0432] ●Use or insert syntax in the bitstream to control the ATMVP (Alternate Temporal Motion Vector Prediction) mode in the encoder or decoder.

[0433] ●Use or insert syntax in the bitstream to control high-precision motion vector encoding tools in the encoder or decoder.

[0434] ●Use or insert syntax in the bitstream to control the palette mode, current picture reference, or separate luma / chroma trees in the encoder or decoder.

[0435] ●Use or insert syntax in the bitstream to control dependency quantization or symbolic data hiding tools in the encoder or decoder.

[0436] • Use or insert syntax in the bitstream to jointly control the separate luma / chroma trees and cross-component linear models / multi-directional linear models in the encoder or decoder.

[0437] • A bitstream or signal comprising one or more of the described syntax elements or variants thereof.

[0438] • A bitstream or signal comprising syntax conveying information generated according to any of the described embodiments.

[0439] Creation and / or transmission and / or reception and / or decoding according to any one of the described embodiments.

[0440] A method, process, apparatus, medium storing instructions, medium storing data, or signal according to any one of the described embodiments.

[0441] Syntax elements are inserted in the signaling that enable the decoder to determine the tools in a way that corresponds to the way used by the encoder.

[0442] Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variations thereof.

[0443] A TV, set-top box, mobile phone, tablet computer or other electronic device that performs the transformation method(s) according to any one of the described embodiments.

[0444] A TV, set-top box, mobile phone, tablet computer, or other electronic device that performs the transformation method(s) determined according to any one of the described embodiments and displays (e.g., using a monitor, screen, or other type of display) the resulting image.

[0445] A TV, set-top box, mobile phone, tablet computer or other electronic device that selects, limits or tunes (e.g., using a tuner) a channel to receive a signal including an encoded image, and performs a transformation method (or methods) according to any one of the described embodiments. A TV, set-top box, mobile phone, tablet computer or other electronic device that receives (e.g., using an antenna) a signal including an encoded image over an air interface, and performs a transformation method (or methods).

Claims

1. A method comprising: inserting high-level syntax elements associated with at least one video codec tool into a high-level parameter set of a video bitstream; Based on the high-level syntax element, control a plurality of partition restrictions of the coding unit; inserting syntax elements related to the plurality of partition restrictions in a parameter set at a lower level than the high-level parameter set; as well as The video bitstream is encoded using at least one partition constraint.

2. A device comprising: The processor is configured as: inserting high-level syntax elements associated with at least one video codec tool into a high-level parameter set of a video bitstream; Based on the high-level syntax element, control a plurality of partition restrictions of the coding unit; inserting syntax elements related to the plurality of partition restrictions in a parameter set at a lower level than the high-level parameter set; as well as The video bitstream is encoded using at least one partition constraint.

3. A method comprising: Parsing the video bitstream to obtain syntax elements in a high-level parameter set; Conditionally determining a plurality of partition limits of a coding unit based on the syntax element; Parsing the video bitstream to obtain syntax elements related to the plurality of partition restrictions in a parameter set lower than the high-level parameter set; as well as The video bitstream is decoded using at least one of the plurality of partition restrictions.

4. A device comprising: The processor is configured as: Parsing the video bitstream to obtain syntax elements in a high-level parameter set; Conditionally determining a plurality of partition limits of a coding unit based on the syntax element; Parsing the video bitstream to obtain syntax elements related to the plurality of partition restrictions in a parameter set lower than the high-level parameter set; as well as The video bitstream is decoded using at least one of the plurality of partition restrictions.

5. The method of claim 1 or claim 3, or the apparatus of claim 2 or claim 4, wherein the syntax indicates information of activated video codec tools associated with separate or non-separate components of a multi-component video bitstream.

6. The method or apparatus of claim 5, wherein an index is used to indicate to which component a video codec tool is to be applied.

7. The method of claim 1 or claim 3, or the apparatus of claim 2 or claim 4, wherein high-level syntax for non-separable components in a multi-component video bitstream is grouped in one syntax set.

8. The method of claim 1 or claim 3, or the apparatus of claim 2 or claim 4, wherein the syntax for the multi-component video bitstream indicates whether a specific video codec tool control is inserted for a bitstream component.

9. The method of claim 1 or claim 3, or the apparatus of claim 2 or claim 4, wherein the plurality of partition constraints are controlled by high-level syntax elements inserted into a high-level parameter set.

10. The method according to claim 1 or claim 3, or the apparatus according to claim 2 or claim 4, wherein a plurality of high-level partition restriction flags are controlled sequentially, and wherein at least one high-level restriction flag controls a low-level restriction.

11. The method of claim 1 or claim 3, or the apparatus of claim 2 or claim 4, wherein the inter prediction tool based on multi-hypothesis prediction is controlled by a high-level syntax element.

12. An apparatus for processing a video bitstream, comprising: The device according to any one of claims 4 to 11; as well as At least one of the following: i. an antenna configured to receive a signal, the signal comprising a video block; ii. a band limiter configured to limit the received signal to a frequency band including the video block; and iii. A display configured to display an output representing the video block.

13. A non-transitory computer readable medium having stored thereon instructions which, when executed, cause a processor to perform the method of any one of claims 1, 3, and 5 to 11.

14. A computer program product comprising instructions which, when said program is run by a computer, cause said computer to perform the method of any one of claims 1, 3 and 5 to 11.

Images