Conversion selection for implicit multiple conversion selection

Abstract

To provide method and apparatus for video encoding or decoding.SOLUTION: An encoding method includes a step of setting syntax information indicating use of a multiple conversion selection mode when an implicit conversion selection mode is used to encode at least one video block in a bitstream, a step of encoding the at least one video block by using at least one conversion corresponding to the syntax information, and a step of storing, communicating, or transmitting a bitstream including the encoded at least one video block. A decoding method includes a step of analyzing the bitstream for syntax information indicating use of a multiple conversion selection mode when an implicit conversion selection mode is used to decode a video block, and a step of decoding at least one video block in the bitstream by using at least one conversion of the multiple conversion selection mode corresponding to the information when the implicit transform selection mode is used.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] Technical Field At least one of the present embodiments generally relates to a method or apparatus for video encoding or decoding. [Background technology]

[0002] background To achieve high compression efficiency, image and video coding schemes usually employ prediction (including spatial and / or motion vector prediction) and transformation to exploit spatial and temporal redundancy in video content. In general, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, and then the difference between the original image and the predicted image (often represented as prediction error or prediction residual) is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to entropy coding, quantization, transformation, and prediction. Many coding tools can be used in the process of coding and decoding (including transformation and inverse transformation). Summary of the Invention

[0003] overview The shortcomings and disadvantages of the prior art may be addressed by the general aspects described herein, which are directed towards block shape adaptive intra prediction during encoding and decoding.

[0004] According to a first aspect, a method is provided, the method including: setting syntax information indicating use of a multiple transform selection mode when an implicit transform selection mode is used to encode at least one video block in a bitstream; encoding the at least one video block by using at least one transform corresponding to the syntax information; and storing, communicating, or transmitting the bitstream including the encoded at least one video block.

[0005] According to a second aspect, a method is provided, the method including: parsing a bitstream for syntax information indicating use of a multiple transform selection mode when an implicit transform selection mode is used to decode a video block; and, when an implicit transform selection mode is used, decoding at least one video block in the bitstream by using at least one transform of a multiple transform selection mode corresponding to the syntax information.

[0006] According to another aspect, an apparatus is provided, the apparatus including a processor, the processor may be configured to encode blocks of a video or to decode a bitstream by performing any of the methods described above.

[0007] According to another general aspect of at least one embodiment, there is provided a device including an apparatus according to any of the decoding embodiments; and at least one of (i) an antenna configured to receive a signal including the video block, (ii) a band limiter configured to limit the received signal to a band of frequencies including the video block, or (iii) a display configured to display an output representing the video block.

[0008] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided that includes data content generated according to any of the described encoding embodiments or variations.

[0009] According to another general aspect of at least one embodiment there is provided a signal including video data generated according to any of the described encoding embodiments or variants.

[0010] According to another general aspect of at least one embodiment, a bitstream is formatted to include data content generated according to any of the described encoding embodiments or variants.

[0011] According to another general aspect of at least one embodiment, there is provided a computer program product including instructions that, when executed by a computer, cause the computer to perform any of the above-described coding embodiments or variations.

[0012] These and other aspects, features and advantages of the general aspects will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings. [Brief description of the drawings]

[0013] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] 1 shows a standard and common video compression scheme. [Diagram 2] 1 shows a standard and common video compression scheme. [Diagram 3] 1 shows an exemplary flowchart of a proposed method for selecting a transform pair depending on an LFNST flag. [Figure 4] 1 shows an exemplary flowchart of a proposed method for selecting a transform pair depending on the LFNST flag and the MIP flag. [Diagram 5] 13 shows an exemplary flowchart of a proposed method for selecting a transform pair depending on the LFNST CU flag and the MIP CU flag. [Figure 6] 1 shows an exemplary flowchart of a proposed method for selecting a transform pair depending on an LFNST CU flag. [Figure 7] 1 illustrates one embodiment of an encoding method under the general aspects described. [Figure 8] 1 illustrates one embodiment of a decoding method under the described general aspects. [Figure 9] 1 illustrates an embodiment of an apparatus for encoding or decoding using intra-prediction mode extensions. [Figure 10] 2 illustrates an exemplary processor arrangement in which the described embodiments may be implemented; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Detailed Description The following general aspects are in the field of video compression. They aim to improve the compression efficiency of an upcoming video compression standard: Versatile Video Coding (VVC). The Versatile Video Coding test model (VTM) is a standardization effort to improve video compression. Specifically, these aspects affect the transform design of this standard, where compression defects are newly introduced by reason of the adoption of two tools, namely the Low-frequency Non-Separable Transform (LFNST) and Matrix-based intra prediction (MIP).

[0015] In addition to DCT2, the VVC transform design includes other transforms: DCT8 and DST7 (collectively known as multiple transform selection (MTS)). The encoder may select one combination of the defined transforms (DCT2, DST7 and DCT8). There are the following possible pairs of horizontal and vertical transforms: - {DCT2,DCT2} - {DST7,DCT7} - {DST7,DCT8} - {DCT8,DST7} - {DCT8,DCT8}

[0016] The MTS is controlled by a SPS (Sequence Parameter Set) flag that can deactivate the encoder and only the pair {DCT2, DCT2} is considered.

[0017] When the MTS flag is enabled, about 1.7% of coding gain over the MTS off case is achieved, while longer coding time is needed due to RDO (Rate Distortion Optimization) competition between multiple candidates, which is time consuming. Alternatively, another mode of transform signaling, named MTSImplicit, has been recently adopted in VVC. In this mode, the transform selection is predefined (implicit) depending on the dimensions of the transform block. In the VVC specification, the selection of transform for luma blocks by MTSImplicit is as follows: trTypeHor=(Width>=4 && Width<=16)?DST7:DCT2 trTypeVer=(Height>=4 && Height<=16)?DST7:DCT2

[0018] The same selection mechanism is used for intra-subpartioning (ISP). This type of selection generates new pairs that are combinations of DCT2 and DST7. The following table summarizes the cases in which new pairs arise.

[0019] [Table 1]

[0020] Recently, two new coding tools have been adopted into the JVET (Joint Video Experts Team) standard. The first one is LFNST (low-frequency non-separable transform), previously known as NSST (non-separable secondary transform). It is an efficient tool that applies "another" non-separable transform to account for redundancy in the transform domain and produce a sparse representation of the residual to be quantized and coded. In the version of the VVC test model VTM-5.0, two LFNST transform sets are defined. One transform set is for large residual blocks, and the other is for small residual blocks. For each set, four groups of transforms are defined, each group containing two transform matrices. The selection of the group depends on the intra prediction mode, and the transform index is coded in the bitstream.

[0021] Unlike the linear transforms that are DCT2, DST7 and DCT8, LFSNT is a trained transform. In other words, the basic functions of LFNST are obtained by learning from some large dataset of residual data. The nature of the dataset and the training method are not explained in any public document.

[0022] The second tool is Matrix based Intra Prediction (MIP). The second tool is a trained tool for predicting a block from its neighboring reference samples. The second tool can be considered as an extension to existing intra prediction, where the prediction signal is generated by multiplying the reference samples with a matrix, rather than a simple duplication along the prediction direction. As with LFNST, the matrices used for multiplication are trained matrices with no description of the dataset used to obtain them.

[0023] To understand the interaction of these tools with the MTSImplicit transform design, simulations were performed using VTM-5.0 with common test conditions. For simplicity, simulation results with one frame intra coding are presented. The anchor is VTM-5.0 with MTS disabled (--MTS=0) and the test is the implicit transform design (--MTS=0 -MTSImplicit=1). The results are shown in the table below.

[0024] [Table 2]

[0025] As can be seen, instead of the expected gain of the implicit conversion design there is a loss of 0.3%. Once the two new tools are deactivated another test is performed to verify the coding gain, i.e. the anchors and tests are the same but LFNST and MIP are set to zero. The results are provided in the table below.

[0026] [Table 3]

[0027] Therefore, it can be concluded that the predicted gain is retained when the new tool is switched off.

[0028] To understand the impact of each of the tools, the two tools are deactivated individually. The table below presents the results of switching only LFNST to zero and leaving MIP activated.

[0029] [Table 4]

[0030] And the table below is for when MIP is off and LFNST is off.

[0031] [Table 5]

[0032] From the simulation results, it can be observed that MIP and / or LFNST are not compatible with the implicit transformation design. However, the impact of LFNST is much more severe than MIP. The reason for the incompatibility is that those tools are trained by residuals (which do not have the new pair transformations defined in the MTSImplicit tool). In fact, MTSImplicit entails a loss of coding efficiency when used in combination with these two tools.

[0033] The described general aspect solves the problem of MTSImplicit loss and improves the transformation design of ImplicitMTS by considering the interaction between the tools LFNST and MIP. The MTSImplicit design does not fit the new added tools of LFNST and MIP. To solve this problem, the simplest way is to deactivate the implicit selection when these tools are activated.

[0034] Embodiment 1: Implicit MTS with DCT2 when LFNST and / or MIP are activated In VTM5, the choice of MTSImplicit transformation is described as follows: trTypeHor=(Width>=4 && Width<=16)?DST7:DCT2 trTypeVer=(Height>=4 && Height<=16)?DST7:DCT2

[0035] To resolve LFNST and MIP interaction issues, the MIP's SPS flag (sps_mip_flag) and / or the LFNST's SPS flag (lfnst_enabled_flag) need to be checked: if(lfnst_enabled_flag) { trTypeHor=DCT2 trTypeVer=DCT2 } { trTypeHor=(Width>=4 && Width<=16)?DST7:DCT2 trTypeVer=(Height>=4 && Height<=16)?DST7:DCT2 }

[0036] The corresponding flow chart is shown in FIG.

[0037] To show the advantages of this method, a simulation was performed with VTM5.0 (with the new method as a test (both with the following options: --MTS=0 and -MTSImplicit=1)) with the same conditions as those described in Table 2 (one frame analysis). The following results are obtained:

[0038] [Table 6]

[0039] An additional condition was also tested: if LFNST is deactivated, the following results are obtained:

[0040] [Table 7]

[0041] Finally, when the MIP is deactivated, the following results are obtained:

[0042] [Table 8]

[0043] Another way to implement the method is to consider both the LFNST and MIP flags: if(lfnst_enabled_flag && sps_mip_flag) { trTypeHor=DCT2 trTypeVer=DCT2 } { trTypeHor=(Width>=4 && Width<=16)?DST7:DCT2 trTypeVer=(Height>=4 && Height<=16)?DST7:DCT2 }

[0044] The corresponding flow chart is shown in FIG.

[0045] In a variation of this embodiment, if LFNST or MIP is activated for the current CU, then the implicit MTS becomes DCT2, DCT2.

[0046] Embodiment 2: Implicit MTS with DCT2 when LFNST and / or MIP are used Besides checking the SPS flag, we can also look at the current coding unit (CU) and see if MIP or LFNST is used. For this case, the implicit mts selection is DCT2. In VTM5.0, lfnstIdx is the index of the LFNST with values ​​from 0 to 2. Zero indicates no LFNST. For MIP, the flag mipFlag indicates whether MIP is used or not. So the modification is as follows: if(CU::lfnstIdx || cu::mipFlag) { trTypeHor=DCT2 trTypeVer=DCT2 } { trTypeHor=(Width>=4 && Width<=16)?DST7:DCT2 trTypeVer=(Height>=4 && Height<=16)?DST7:DCT2 }

[0047] The corresponding flow chart is shown in FIG.

[0048] The difficulty with this method is that the transform selection depends on the prediction mode selected. In general, it would be preferable to not have this dependency. However, since the current LFNST is intra-mode dependent, this method is acceptable.

[0049] Similarly, we can consider only the cases where LFNST is used: if(CU::lfnstIdx) { trTypeHor=DCT2 trTypeVer=DCT2 } { trTypeHor=(Width>=4 && Width<=16)?DST7:DCT2 trTypeVer=(Height>=4 && Height<=16)?DST7:DCT2 }

[0050] The corresponding flow chart is shown in FIG.

[0051] In a variation of this embodiment, if LFNST or MIP is activated for the current CU, the implicit MTS becomes DCT2, DCT2.

[0052] A one-frame analysis is also done here, the anchor is VTM5.0 (with the new method as test (both with the following options: --MTS=0 and -MTSImplicit=1)), giving the following results:

[0053] [Table 9]

[0054] To show the advantages over the method of embodiment 1, one frame analysis was performed with embodiment 1 as anchor and the current method as test. The following results are obtained:

[0055] [Table 10]

[0056] One embodiment of a method 700 under the general aspects described herein is shown in Figure 7. The method begins at start block 701 and control passes to block 710 for setting syntax information indicating the use of a multiple transform selection mode if an implicit transform selection mode is used for encoding at least one video block in a bitstream. Control passes from block 710 to block 720 for encoding the at least one video block by using at least one transform corresponding to the syntax information. Control passes from block 720 to block 730 for storing, communicating or transmitting a bitstream including the encoded at least one video block.

[0057] One embodiment of a method 800 under the general aspects described herein is shown in Figure 8. The method begins at start block 801 and control passes to block 810 for parsing a bitstream for syntax information indicating the use of a multiple transform selection mode when an implicit transform selection mode is used to decode a video block. Control passes from block 810 to block 820 for decoding at least one video block in the bitstream by using at least one transform of a multiple transform selection mode that corresponds to the syntax information when an implicit transform selection mode is used.

[0058] 9 shows one embodiment of an apparatus 900 for compressing, encoding or decoding video by using an encoding or decoding tool. The apparatus includes a processor 910 and may be interconnected via at least one port to a memory 920. Both the processor 910 and the memory 920 may also have one or more additional interconnects to external connections.

[0059] The processor 910 is also configured to insert information into the bitstream or receive information in the bitstream by either compressing, encoding or decoding it using various coding tools.

[0060] This application describes various aspects, including tools, features, embodiments, models, methods, and the like. Many of these aspects are described with specificity, but in a manner that may at times sound limiting, at least to illustrate individual features. However, this is for purposes of clarity in the description and thus does not limit the applicability or scope of these aspects. In fact, all of the various aspects may be combined or interchanged to provide further aspects. Additionally, the various aspects may be combined or interchanged with aspects described in previous applications.

[0061] The aspects described and contemplated in this application may be implemented in many different forms. While Figures 1, 2, and 10 provide some embodiments, other embodiments are contemplated, and thus the discussion of Figures 1, 2, and 10 does not limit the scope of the embodiments. At least one of the aspects relates generally to video encoding and decoding, and at least one other aspect relates generally to transmitting a bitstream that is generated or encoded. These and other aspects may be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0062] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "picture" and "frame" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side while the term "decoded" is used on the decoder side.

[0063] Various methods are described herein, each of which includes one or more steps or acts for carrying out the described method. Unless a specific order of steps or acts is necessary for the proper operation of the method, the order and / or use of specific steps and / or acts may be modified or combined.

[0064] Various methods and other aspects described in the present application may be used to modify modules, such as the intra prediction, entropy coding, and / or decoding modules (160, 360, 145, 330) of the video encoder 100 and decoder 200 as shown in FIGS. 1 and 2. Furthermore, the aspects are not limited to VVC or HEVC and thus may be applied, for example, to other standards and recommendations (whether existing or developed in the future) and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically precluded, the aspects described in the present application may be used individually or in combination.

[0065] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes, and therefore the described aspects are not limited to these specific values.

[0066] 1 shows an encoder 100. Although variations of this encoder 100 are contemplated, encoder 100 is described below for clarity without describing all anticipated variations.

[0067] Before being encoded, a video sequence may go through a pre-encoding process (101): for example, applying a color transformation (e.g., from RGB 4:4:4 to YCbCr 4:2:0) to the input color picture, or remapping the input picture components to obtain a signal distribution that is more resistant to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0068] In the encoder 100, a picture is coded by the encoder elements as described below. The picture to be coded is divided (102) and processed, for example, in units of CUs. Each unit is coded, for example, using either intra mode or inter mode. If a unit is coded in intra mode, the encoder 100 performs intra prediction (160). In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder (105) decides whether to use intra mode or inter mode to code the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. A prediction residual is calculated, for example, by subtracting (110) the prediction block from the source image block.

[0069] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients as well as the motion vectors and other syntax elements are entropy coded (145) to output a bitstream. The encoder may skip this transform and apply quantization directly to the untransformed residual signal. The encoder may bypass both the transform and quantization, i.e., the residual is coded directly without application of the transform or quantization processes.

[0070] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. An image block is reconstructed by combining (155) the decoded prediction residual with the predicted block. An in-loop filter (165) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0071] 2 shows a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by decoder elements as described below. The video decoder 200 typically performs a reciprocal decoding path to the encoding path described in FIG. 1, and the encoder 100 also typically performs video decoding as part of the encoding of the video data.

[0072] In particular, the decoder input includes a video bitstream, which may be generated by video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coded information. Picture partition information indicates how the image is partitioned. Thus, the decoder may partition the picture according to the decoded picture partition information (235). The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. An image block is reconstructed by combining (255) the decoded prediction residual with a prediction block. The prediction block may be obtained from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (275) (270). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0073] The decoded picture may further go through a post-decoding process (285) (such as an inverse color transformation (e.g., from YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping that reverses the remapping process performed in the pre-encoding process (101)). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0074] FIG. 10 illustrates a block diagram of an example system in which various aspects and embodiments may be implemented. System 1000 may be embodied as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include various electronic devices, such as, but not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, servers, etc. Elements of system 1000 (alone or in combination) may be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, system 1000 is configured to perform one or more of the aspects described herein.

[0075] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein, for example to perform various aspects described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. The system 1000 includes at least one memory 1020 (e.g., volatile and / or non-volatile memory devices). The system 1000 includes a storage device 1040, which may include non-volatile and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read Only Memory (EEPROM), Read Only Memory (ROM), Programmable Read Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash, magnetic disk drives, and / or optical disk drives. The storage device 1040 may include, by way of non-limiting example, an internal storage device, an attached storage device (including a removable or non-removable storage device), and / or a network-accessible storage device.

[0076] The system 1000 includes an encoder / decoder module 1030 configured to process data, for example to provide encoded or decoded video. The encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Additionally, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated within the processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0077] Program code to be loaded into the processor 1010 or the encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and then loaded into the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more various items during performance of the processes described herein. Such stored items include, but are not limited to, input video, decoded video, or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0078] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and to provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory can be the memory 1020 and / or the storage device 1040 (e.g., dynamic volatile memory and / or non-volatile flash memory). In some embodiments, the external non-volatile flash memory is used to store the operating system of the television (e.g.,). In at least one embodiment, a high-speed external dynamic volatile memory such as a RAM is used as working memory for video coding and decoding operations such as MPEG2 (MPEG stands for Moving Picture Expert Group, MPEG2 is also known as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard being developed by JVET (Joint Video Experts Team)).

[0079] Inputs to the elements of system 1000 may be provided via various input devices, as shown in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) section, for example for receiving RF signals transmitted wirelessly by a broadcaster, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples not shown in FIG. 10 include composite video.

[0080] In various embodiments, the input devices of block 1130 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or bandlimiting a signal to a band of frequencies), (ii) downconverting the selected signal, (iii) bandlimiting again to a narrow band of frequencies to select (for example) a signal frequency band, which in certain embodiments may be referred to as a channel, (iv) demodulating the downconverted and bandlimited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various embodiments includes one or more elements (e.g., a frequency selector, a signal selector, a bandlimiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer) to perform these functions. The RF section may include, for example, a tuner to perform various of these functions, including downconverting a received signal to a lower frequency (e.g., an intermediate frequency or a near baseband frequency) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements (e.g., inserting amplifiers and analog-to-digital converters, etc.). In various embodiments, the RF section includes an antenna.

[0081] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting the system 1000 to other electronic devices over USB and / or HDMI connections. It should be understood that various aspects of the input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within the processor 1010 as desired. Similarly, aspects of the USB or HDMI interface processing may be realized within a separate interface IC or within the processor 1010 as desired. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements (including, for example, the processor 1010 and an encoder / decoder 1030 operating in combination with memory and storage elements) to process the data stream as needed for presentation on an output device.

[0082] The various elements of system 1000 may be provided within a unified housing, where the various elements may be interconnected and data may be transmitted therebetween using suitable connection arrangements (e.g., internal buses known in the art, including Inter-IC (I2C) bus, wires, and printed circuit boards).

[0083] The system 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented within a wired and / or wireless medium, for example.

[0084] Data is streamed or otherwise provided to the system 1000 in various embodiments by using a wireless network such as a Wi-Fi network (e.g., IEEE 802.11: IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received over a communication channel 1060 and communication interface 1050 adapted for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, that enable streaming applications and other over-the-top communications. Other embodiments provide stream data to the system 1000 by using a set-top box that delivers data over an HDMI connection of the input block 1130. Still other embodiments provide stream data to the system 1000 by using an RF connection of the input block 1130. As indicated above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use wireless networks other than Wi-Fi (e.g., cellular networks, Bluetooth networks).

[0085] The system 1000 may provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The display 1100 in various embodiments includes one or more of a touch screen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 may be for a television, a tablet, a laptop, a cellular (mobile) phone, or other device. The display 1100 may also be integrated with other components (e.g., components in a smartphone) or may be separate (e.g., an external monitor for a laptop). The other peripheral devices 1120 in various example embodiments include one or more of a stand-alone digital video disc (or digital versatile disc) (both referred to as DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 to provide functionality based on the output of the system 1000. For example, a disc player performs the function of playing the output of the system 1000 .

[0086] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 by using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that allow device-to-device control with or without user intervention. The output devices may be communicatively coupled to the system 1000 through dedicated connections via respective interfaces 1070, 1080, 1090. Alternatively, the output devices may be connected to the system 1000 by using a communication channel 1060 via the communication interface 1050. The display 1100 and speaker 1110 may be integrated into a single unit with other components of the system 1000 in an electronic device such as a television. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0087] The display 1100 and speakers 1110 may alternatively be separate from one or more of the other components, for example if the RF portion of the input block 1130 is part of a separate set-top box. In various embodiments where the display 1100 and speakers 1110 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0088] Some embodiments may be implemented by the processor 1010, or by computer software implemented in hardware, or by a combination of hardware and software. As a non-limiting example, embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories. The processor 1010 may be of any type suitable for the technical environment and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0089] Various implementations involve decoding. "Decoding" as used herein may encompass all or part of the processing performed on a received encoded sequence, e.g., to generate a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder (e.g., entropy decoding, inverse quantization, inverse transform, and differential decoding). In various embodiments, such processes also or alternatively include processes performed by the decoders of the various embodiments described herein.

[0090] As another example, in one embodiment "decoding" refers to entropy decoding only, in another embodiment "decoding" refers to differential decoding only, and in another embodiment "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer to a subset of operations in particular or to the broader decoding process in general will be clear based on the context of the particular description and thus will be well understood by one of ordinary skill in the art.

[0091] Various implementations involve encoding. In a similar manner to the above discussion of "decoding," "encoding" as used in this application may encompass all or some of the processes performed, for example, on an input video sequence to generate an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder (e.g., partitioning, differential encoding, transforming, quantizing, and entropy encoding). In various embodiments, such processes also or alternatively include processes performed by an encoder in various implementations described herein.

[0092] As another example, in one embodiment "encoding" refers to entropy encoding only, in another embodiment "encoding" refers to differential encoding only, and in another embodiment "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to refer to a subset of operations in particular or to the broader encoding process in general will be clear based on the context of the particular description and thus will be well understood by one of ordinary skill in the art.

[0093] It should be noted that syntax elements as used herein are descriptive terms, and therefore syntax elements as used herein do not preclude the use of other syntax element names.

[0094] Where a diagram is presented as a flow diagram, it should be understood that the diagram also provides a block diagram of the corresponding apparatus. Similarly, where a diagram is presented as a block diagram, it should be understood that the diagram also provides a flow diagram of the corresponding method / process.

[0095] Various embodiments may refer to parametric models or rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, often given a computational complexity constraint. This may be measured via a Rate Distortion Optimization (RDO) metric, or via Least Mean Square (LMS), Mean of Absolute Errors (MAE), or other such measures. Rate-distortion optimization is usually formulated as minimizing a rate-distortion function that is a weighted sum of rate and distortion. There are various approaches to solving the rate-distortion optimization problem. For example, some approaches may be based on an extensive test of all coding alternatives (including all modes or coding parameter values ​​considered) with a thorough evaluation of their coding costs and associated distortions of the reconstructed signal after coding and decoding. Faster approaches may also be used to reduce the encoding complexity, especially by calculation of approximate distortions based on prediction or prediction residual signals (not the reconstructed signal). A mixture of these two approaches may also be used, such as by using approximate distortion for only some of the possible encoding choices and the full distortion for others. Other approaches only evaluate a subset of the possible encoding choices. More generally, many approaches employ any of a variety of techniques to perform an optimization that is not necessarily a complete evaluation of both the coding cost and the associated distortion.

[0096] The implementations and aspects described herein may be embodied in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed in the context of only one type of implementation (e.g., discussed only as a method), the implementation of the discussed features may also be embodied in other forms (e.g., in an apparatus or program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The method may be implemented in, for example, a processor, which refers to processing devices in general (including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device). Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants (PDAs), and other devices that facilitate the transfer of information between end users.

[0097] References to "one embodiment" or "embodiment" or "one implementation" or "implementation" and other variations mean that the particular features, structures, characteristics, etc. described in connection with the embodiment are included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" in various places throughout this application, and in any other variations, are not necessarily referring to the same embodiment.

[0098] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0099] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, replicating information, computing information, determining information, predicting information, or estimating information.

[0100] Additionally, the application may refer to "receiving" various information. Receiving is intended to be broad, similar to "accessing." Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" typically involves in some manner an operation, such as, for example, storing information, processing information, transmitting information, moving information, duplicating information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0101] For example, in the cases of "A / B," "A and / or B," and "at least one of A and B," it should be understood that the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As another example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A and B and C). This may be extended to as many items as are listed, as will be apparent to those skilled in the art.

[0102] Also, as used herein, the word "signaling" refers to, among other things, indicating something to a corresponding decoder. For example, in some embodiments, an encoder signals a particular one of multiple transforms, coding modes, or flags. In this way, in one embodiment, the same transform, parameter, or mode is used at both the encoder side and the decoder side. Thus, for example, an encoder may transmit a particular parameter to a decoder so that the decoder may use the same parameter (explicit signaling). Conversely, if the decoder already has the particular parameter and also other parameters, signaling may be used without transmitting the particular parameter (implicit signaling) simply to allow the decoder to know and select it. By avoiding transmitting any actual function, bit savings are realized in various embodiments. It should be recognized that signaling may be achieved in various ways. For example, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder in various embodiments. Although the above relates to the verb form of the word "signal," the word "signal" may also be used herein as a noun.

[0103] As will be apparent to one of ordinary skill in the art, some implementations may generate a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described embodiments. For example, the signal may be formatted to carry a bit stream of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a high frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information the signal carries may be, for example, analog or digital information. The signal may be transmitted over a wide variety of wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[0104] We have described a number of embodiments across various claim categories and types. The features of these embodiments may be provided alone or in any combination. Further, some embodiments may include one or more of the following features, devices or aspects across various claim categories and types, alone or in any combination: A process or device for encoding or decoding video data by using a low frequency non-separable transform with multiple transform selection. A process or device for encoding or decoding video data by using low matrix based intra prediction with multiple transform selection. A process or device for encoding or decoding video data by using a low-frequency non-separable transform with implicit multiple transform selection. A process or device for encoding or decoding video data by using low matrix based intra prediction with implicit multiple transform selection. A bitstream or signal including one or more of the described syntax elements or variations thereof. A bitstream or signal comprising a syntax conveying information generated according to any of the described embodiments. - Generating and / or transmitting and / or receiving and / or decoding according to any of the described embodiments. - A medium storing a method, process, apparatus, instructions, data or signals according to any of the described embodiments. Inserting signaling syntax elements that allow the decoder to determine the coding mode in a manner corresponding to the one used by the encoder. ● Generating and / or transmitting and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variations thereof. A TV, set-top box, cellular phone, tablet or other electronic device performing the conversion method according to any of the described embodiments. ● A TV, set-top box, cellular phone, tablet or other electronic device that performs the conversion method determination according to any of the described embodiments and displays the resulting image (e.g., using a monitor, screen or other type of display). ●A TV, set-top box, cellular phone, tablet or other electronic device that selects, band-limits or tunes a channel (e.g., by using a tuner) and performs a conversion method to receive a signal containing an encoded image according to any of the described embodiments. • A TV, set-top box, cellular phone, tablet or other electronic device that receives the signal containing the encoded image wirelessly (eg, by using an antenna) and performs the conversion method.

Claims

1. configuring syntax information indicating use of a multiple transform selection mode when an implicit transform selection mode is used to encode at least one video block in the bitstream; encoding the at least one video block using at least one transform corresponding to the syntactic information; storing, transmitting, or transmitting the bitstream including the encoded at least one video block; The method includes:

2. configuring syntax information indicating use of a multiple transform selection mode when an implicit transform selection mode is used to encode at least one video block in the bitstream; encoding the at least one video block using at least one transform corresponding to the syntactic information; and storing, communicating, or transmitting the bitstream including the encoded at least one video block.

3. parsing the bitstream for syntax information indicating use of a multiple transform selection mode when an implicit transform selection mode is used to decode the video block; if an implicit transform selection mode is used, decoding at least one video block in the bitstream by using at least one transform of a multiple transform selection mode corresponding to the syntax information; The method includes:

4. parsing the bitstream for syntax information indicating use of a multiple transform selection mode when an implicit transform selection mode is used to decode the video block; and when an implicit transform selection mode is used, decoding at least one video block in the bitstream by using at least one transform of a multiple transform selection mode corresponding to the syntax information.

5. The method according to claim 1 or claim 3, wherein the syntax information comprises at least one flag.

5. Apparatus according to claim 2 or claim 4.

6. The method according to claim 1 or 3, wherein the syntax information includes at least one flag indicating a multiple transform selection mode.

5. Apparatus according to claim 2 or claim 4.

7. The method of claim 1 or claim 3, wherein the syntax information includes at least one flag indicating a multiple transform selection implicit mode.

5. Apparatus according to claim 2 or claim 4.

8. The method according to claim 1 or 3, wherein the syntax information comprises at least one flag indicating a low-frequency non-separable transform mode.

5. Apparatus according to claim 2 or claim 4.

9. The method of claim 1 or 3, wherein the syntax information comprises at least one flag indicating a matrix-based intra-prediction mode.

5. Apparatus according to claim 2 or claim 4.

10. The method according to claim 1 or claim 3, in which the multiple transform selection mode is conditioned by encoding the unit size.

5. Apparatus according to claim 2 or claim 4.

11. The method according to claim 1 or claim 3, wherein the transformation is composed of horizontal and vertical components.

5. Apparatus according to claim 2 or claim 4.

12. An apparatus according to any one of claims 4 to 11 and A device comprising at least one of: (i) an antenna configured to receive a signal including the video block; (ii) a band limiter configured to limit the received signal to a band of frequencies including the video block; and (iii) a display configured to display an output representing the video block.

13. A non-transitory computer readable medium containing data content generated according to the method of any one of claims 1 and 5 to 11 or generated by the apparatus of any one of claims 2 and 5 to 11 for reproduction by using a processor.

14. A signal containing video data generated according to the method of any one of claims 1 and 5 to 11 or generated by the apparatus of any one of claims 2 and 5 to 11 for reproduction by using a processor.

15. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method of claims 1 and 3 and any one of claims 5 to 11.

Images