Hybrid explicit / implicit LFNST / NSPT

Abstract

A method and apparatus for encoding or decoding an image or video are provided, in which an intra prediction mode is obtained for a block of the image or video. Based on the intra prediction mode obtained for the block, it is determined whether an index of a non-separable transform is determined in an explicit mode or in an implicit mode, in which in the explicit mode the index of the non-separable transform is explicitly signaled in a bitstream and in the implicit mode the index of the non-separable transform is derived from other data signaled in the bitstream. The block is encoded or decoded using a non-separable transform selected based on the index from a set of non-separable transforms.

Description

[0001] This application claims priority to European Application No. 24305439.2, filed on 25 March 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This document generally relates to video compression. Specifically, it relates to methods and apparatus for encoding or decoding images or videos. More specifically, it relates to transform coding in improved video compression systems. Background Technology

[0003] To achieve high compression efficiency, image and video coding schemes typically employ prediction and transform to utilize spatial and temporal redundancy in video content. Intra-frame or inter-frame prediction is usually used to develop intra-frame or inter-frame image correlations, followed by transforming, quantizing, and entropy coding of the difference between the original block and the predicted block (typically represented as prediction error or prediction residual). In inter-frame prediction, the motion vectors used for motion compensation are typically predicted from a motion vector predictor. To reconstruct the video, the compressed data is decoded through the inverse process corresponding to entropy coding, quantization, transform, and prediction. Summary of the Invention

[0004] According to one aspect, a method for encoding an image or video is provided. The method includes: obtaining an intra-prediction mode for a block of the image or video; selecting, based on the obtained intra-prediction mode for the block, an inseparable transform for the block from a first set of inseparable transforms; and encoding the block using the selected inseparable transform.

[0005] According to another aspect, an apparatus for encoding an image or video is provided. The apparatus includes one or more processors operable to: obtain an intra-prediction mode for a block of an image or video; based on the obtained intra-prediction mode for the block, select an inseparable transform for the block from a first set of inseparable transforms; and encode the block using the selected inseparable transform.

[0006] According to one aspect, a method for decoding an image or video is provided. The method includes: obtaining an intra-prediction mode for a block of the image or video; selecting, based on the obtained intra-prediction mode for the block, an inseparable transform for the block from a first set of inseparable transforms; and decoding the block using the selected inseparable transform.

[0007] According to another aspect, an apparatus for decoding an image or video is provided. The apparatus includes one or more processors operable to: obtain intra-frame prediction modes for a block of an image or video; based on the obtained intra-frame prediction modes for the block, select an inseparable transform for the block from a first set of inseparable transforms; and decode the block using the selected inseparable transform.

[0008] According to another aspect, a method for encoding an image or video is provided. The method includes: obtaining an intra-prediction mode for a block of the image or video; determining, based on the obtained intra-prediction mode for the block, whether an inseparable transform index for selecting the inseparable transform for the block from a first set of inseparable transforms is determined in an explicit mode or an implicit mode, wherein in an explicit mode, the inseparable transform index is explicitly signaled in the bitstream, and in an implicit mode, the inseparable transform index is derived from other data signaled in the bitstream; and encoding the block using the determined inseparable transform.

[0009] According to another aspect, an apparatus for encoding an image or video is provided. The apparatus includes one or more processors operable to: obtain an intra-prediction mode for a block of an image or video; based on the obtained intra-prediction mode for the block, determine whether an irreparable transform index for selecting the block from a first set of irreparable transforms is determined in an explicit mode or an implicit mode, wherein in an explicit mode, the irreparable transform index is explicitly signaled in the bitstream, and in an implicit mode, the irreparable transform index is derived from other data signaled in the bitstream; and encode the block using the determined irreparable transform.

[0010] According to one aspect, a method for decoding an image or video is provided. The method includes: obtaining an intra-prediction mode for a block of the image or video; determining, based on the obtained intra-prediction mode for the block, whether an inseparable transform index for selecting the inseparable transform for the block from a first set of inseparable transforms is determined in an explicit mode or an implicit mode, wherein in an explicit mode, the inseparable transform index is explicitly signaled in the bitstream, and in an implicit mode, the inseparable transform index is derived from other data signaled in the bitstream; and decoding the block using the determined inseparable transform.

[0011] According to another aspect, an apparatus for decoding an image or video is provided. The apparatus includes one or more processors operable to: obtain an intra-prediction mode for a block of an image or video; based on the obtained intra-prediction mode for the block, determine whether an irreparable transform index for selecting the block from a first set of irreparable transforms is determined in an explicit mode or an implicit mode, wherein in an explicit mode, the irreparable transform index is explicitly signaled in the bitstream, and in an implicit mode, the irreparable transform index is derived from other data signaled in the bitstream; and decode the block using the determined irreparable transform.

[0012] According to another aspect, a method for encoding an image or video is provided. The method includes: obtaining an intra-prediction mode for a block of the image or video; based on the obtained intra-prediction mode for the block, determining whether a primary transform to be applied to the block is a single separable transform or a separable transform selected from multiple separable transforms; and encoding the block using the determined separable transform.

[0013] According to another aspect, an apparatus for encoding an image or video is provided. The apparatus includes one or more processors operable to: obtain an intra-prediction mode for a block of an image or video; based on the obtained intra-prediction mode for the block, determine whether a primary transform to be applied to the block is a single separable transform or a separable transform selected from a plurality of separable transforms; and encode the block using the determined separable transform.

[0014] According to one aspect, a method for decoding an image or video is provided. The method includes: obtaining an intra-prediction mode for a block of the image or video; based on the obtained intra-prediction mode for the block, determining whether a primary transform to be applied to the block is a single separable transform or a separable transform selected from multiple separable transforms; and decoding the block using the determined separable transform.

[0015] According to another aspect, an apparatus for decoding an image or video is provided. The apparatus includes one or more processors operable to: obtain an intra-prediction mode for a block of an image or video; based on the obtained intra-prediction mode for the block, determine whether a primary transform to be applied to the block is a single separable transform or a separable transform selected from a plurality of separable transforms; and decode the block using the determined separable transform.

[0016] One or more embodiments also provide a computer program including instructions that, when executed by one or more processors, cause the one or more processors to perform any method for encoding or decoding images or videos according to any of the embodiments described herein. One or more of the embodiments herein also provide a non-transitory computer-readable medium and / or a computer-readable storage medium storing instructions thereon for encoding or decoding images or videos according to the methods described herein.

[0017] One or more embodiments also provide a computer-readable storage medium having a bit stream generated according to the method described herein.

[0018] One or more embodiments also provide a method and apparatus for transmitting or receiving a bit stream generated according to the method described herein.

[0019] According to one aspect, an apparatus is provided, comprising: means for decoding an image or video according to any embodiment described herein; and at least one of: (i) an antenna configured to receive or transmit a signal including data representing the image or video, (ii) a band limiter configured to limit the signal to a band including data representing the image or video, or (iii) a display configured to display the image or video.

[0020] In one variant, the device includes at least one of a television, a cellular phone, a tablet computer, and a set-top box. Attached Figure Description

[0021] Figure 1A The illustration shows a block diagram of a system in which aspects of the embodiments herein may be implemented, according to one embodiment.

[0022] Figure 1B A block diagram of a system in which aspects of the embodiments herein may be implemented, according to another embodiment, is illustrated.

[0023] Figure 1C A block diagram of a system in which aspects of the embodiments herein may be implemented, according to another embodiment, is illustrated.

[0024] Figure 2 A block diagram illustrating an embodiment of a video encoder in which various aspects of the embodiments described herein may be implemented.

[0025] Figure 3 A block diagram illustrating an embodiment of a video decoder in which various aspects of the embodiments described herein may be implemented.

[0026] Figure 4 An example of explicit MTS set selection in ECM is shown.

[0027] Figure 5 An example of a region of interest for the LFNST16 core is shown.

[0028] Figure 6 An example of a region of interest for the LFNST8 core is shown.

[0029] Figure 7 An example of mapping intra-prediction modes to LFNST set indices is shown.

[0030] Figure 8 Examples of applying the NSPT transformation to the small block and applying LFNST to the remaining block size are shown.

[0031] Figure 9 An example of a method for encoding blocks using the LFNST or NSPT inseparable transformation is shown.

[0032] Figure 10 An example of a method for decoding blocks using the LFNST or NSPT non-separable transform is shown.

[0033] Figure 11 An example of a method for encoding video blocks according to one embodiment is illustrated.

[0034] Figure 12 An example of a method for decoding video blocks according to one embodiment is illustrated.

[0035] Figure 13 An example of a method for encoding video blocks according to another embodiment is illustrated.

[0036] Figure 14 An example of a method for decoding video blocks according to another embodiment is illustrated.

[0037] Figure 15 The diagram illustrates the following: Figure 13 An example of a variant of the method used to encode video blocks.

[0038] Figure 16 The diagram illustrates the following: Figure 14 An example of a variant of the method used for decoding video blocks.

[0039] Figure 17 An example of a method for encoding video blocks according to another embodiment is illustrated.

[0040] Figure 18 An example of a method for decoding video blocks according to another embodiment is illustrated.

[0041] Figure 19The diagram illustrates the following: Figure 17 An example of a variant of the method used to encode video blocks.

[0042] Figure 20 The diagram illustrates the following: Figure 18 An example of a variant of the method used for decoding video blocks.

[0043] Figure 21 An example of a method for encoding video blocks according to another embodiment is illustrated.

[0044] Figure 22 An example of a method for decoding video blocks according to another embodiment is illustrated.

[0045] Figure 23 This illustrates an example of two remote devices communicating via a communication network, based on the principles outlined in this paper.

[0046] Figure 24 The diagram illustrates an example of signal syntax based on the principles described in this paper. Detailed Implementation

[0047] This application describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described in detail and, at least to illustrate individual characteristics, are generally described in a manner that may sound restrictive. However, this is for the purpose of clarity and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in previous documents.

[0048] The aspects described and envisioned in this application can be realized in many different forms. The following... Figure 1A , 1B 1C, 2, and 3 provide some embodiments, but other embodiments are conceivable, and Figure 1A , 1B The discussions in 1C, 2, and 3 do not limit the breadth of implementations. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to the transmission of generated or encoded bitstreams. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having a bitstream generated according to any of the described methods stored thereon.

[0049] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0050] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Additionally, terms such as "first," "second," etc., may be used in various embodiments to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of these terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding need not be performed before the second decoding, and may occur, for example, before, during, or in a time period overlapping with the second decoding.

[0051] This aspect is not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, whether pre-existing or developed in the future, as well as extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise stated or technically excluded, the aspects described in this application may be used individually or in combination.

[0052] Figure 1A-1C The illustration shows a block diagram of an example system in which various aspects and embodiments may be implemented. Any of systems 100A, 100B, or 100C may be embodied as a device including the various components described below and configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. In various embodiments, systems 100A, 100B, or 100C are communicatively coupled to other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, systems 100A, 100B, or 100C are configured to implement one or more aspects described in this application.

[0053] Figure 1AThe diagram illustrates an example of a system in which various aspects and embodiments can be implemented. System 100A includes at least one processor 110 configured to execute instructions loaded therein to implement, for example, the various aspects described herein. Processor 110 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 100A includes at least one memory 120, such as a volatile memory device and / or a non-volatile memory device, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drive, and / or optical disk drive. Memory 120 may include internal storage devices, attached storage devices, and / or network-accessible storage devices, as a non-limiting example. Processor 110 may be interconnected with memory 120 via interconnect bus 115.

[0054] Program code to be loaded onto processor 110 to execute the various aspects described in this application is then loaded into memory 120 for execution by processor 110. In some embodiments, memory within processor 110 is used to store program code instructions and provide working memory for processing during encoding or decoding. Input to the components of system 100A may be provided through various input devices (not shown). Processor 110 and memory 120 may also have one or more additional interconnects for external connections.

[0055] Figure 1B A block diagram illustrating an example of a system 100B in which various aspects and embodiments may be implemented. System 100B includes, as described above... Figure 1A The processor 110 and memory 120 are described. Inputs to the components of system 100B can be provided via various input devices indicated in block 105, which will be discussed below. Figure 1C Further description. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives RF signals transmitted over the air, for example by a broadcasting device; (ii) a component (COMP) input terminal (or a set of COMP input terminals); (iii) a universal serial bus (USB) input terminal; and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 1B Other examples not shown include composite video.

[0056] Various components can be interconnected and transmit data therebetween using suitable connection arrangements 115 (e.g., internal buses known in the art, including I2C buses, wiring and printed circuit boards).

[0057] System 100B includes a communication interface 150, which enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, within a wired and / or wireless medium.

[0058] System 100B can provide output signals to various output devices, including displays, speakers, and other peripheral devices. Output devices can be communicatively coupled to system 100B via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices can be connected to system 100B via communication interface 150 using communication channel 190.

[0059] Figure 1C The illustration shows an example block diagram of a system 100C according to another embodiment, in which various aspects and embodiments may be implemented. The elements of the system 100C may be embodied individually or in combination in a single integrated circuit, multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of the system 100C are distributed across multiple ICs and / or discrete components.

[0060] System 100C includes, for example, regarding Figure 1A The processor 110 and memory 120 are described in 1B.

[0061] System 100C includes a storage device 140, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drive, and / or optical disk drive. As a non-limiting example, storage device 140 may include internal storage, attached storage, and / or network-accessible storage.

[0062] System 100C includes an encoder / decoder module 130, which is configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 130 may include its own processor and memory. Encoder / decoder module 130 represents one or more modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Furthermore, encoder / decoder module 130 may be implemented as a separate component of system 100C, or it may be incorporated into processor 110 as a combination of hardware and software known to those skilled in the art.

[0063] Program code to be loaded into processor 110 or encoder / decoder 130 to execute the various aspects described in this application may be stored in storage device 140 and subsequently loaded onto memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more different items during execution of the processes described in this application. These stored items may include, but are not limited to, input data (images, videos, volumetric content), decoded data (images, videos, volumetric content) or portions of decoded data, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and arithmetic logic processing.

[0064] In some embodiments, memory within processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., processor 110 or encoder / decoder module 130) is used for one or more of these functions. External memory may be memory 120 and / or storage device 140, such as dynamically volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, fast external dynamically volatile memory (such as RAM) is used as working memory for data encoding and decoding operations, such as for MPEG-2, HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Various Video Coding, also known as H.266, a standard developed by JVET (Joint Video Experts Group)).

[0065] The inputs to the various components of system 100C can be provided through the various input devices indicated in block 105, and also... Figure 1B As mentioned above, such input devices for System 100B or 100C include, but are not limited to: (i) an RF section that receives radio frequency (RF) signals, for example, wirelessly transmitted by a broadcasting company; (ii) a component (COMP) input terminal (or a set of COMP input terminals); (iii) a universal serial bus (USB) input terminal; and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 1B Other examples not shown in 1C include composite video.

[0066] In various embodiments, the input device of block 105 of system 100B or 100C has been associated with corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal or limiting a signal band to a certain band); (ii) downconverting the selected signal; (iii) further limiting the band to a narrower band to select, for example, a signal band (which may be referred to as a channel in some embodiments); (iv) demodulating the downconverted and band-limited signal; (v) performing error correction; and (vi) performing demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner that performs various of these functions, including, for example, downconverting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or downconverting it to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various embodiments may rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0067] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 100B or 100C to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 110. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processor 110. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 110 and encoder / decoder 130, which cooperate with memory and storage elements to process the data stream as needed for presentation on the output device.

[0068] The individual components of system 100A, 100B, or 100C can be provided within an integrated housing. Within this integrated housing, the individual components can be interconnected and transmit data therebetween using suitable connection means 115, such as internal buses known in the art, including I2C buses, lines, and printed circuit boards.

[0069] Similarly, regarding Figure 1B System 100B and System 100C include a communication interface 150, which enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, in a wired and / or wireless medium.

[0070] In various embodiments, data is streamed to system 100B or 100C using a Wi-Fi network (such as IEEE 802.11, IEEE referring to the Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication channel 190 and a communication interface 150 suitable for Wi-Fi communication. The communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks (including the Internet) to allow streaming applications and other OTT (over-the-top) communications. Other embodiments use a set-top box to provide streaming data to system 100B or 100C, delivering data via an HDMI connection of input block 105. Still other embodiments use an RF connection of input block 105 to provide streaming data to system 100B or 100C. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0071] System 100C can provide output signals to various output devices, including a display 165, a speaker 175, and other peripheral devices 185. The display 165 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. The display 165 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. The display 165 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., as in an external monitor for a laptop). In various examples of embodiments, other peripheral devices 185 include one or more of a standalone digital video disc (or digital multifunction disc) (these two terms are collectively referred to as DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments utilize one or more peripheral devices 185 that provide functionality based on the output of system 100C. For example, a disc player performs the function of playing the output of system 100C.

[0072] In various embodiments, signal transmission, such as AV.link, CEC, or other communication protocols that enable device-to-device control with or without user intervention, is used to transmit control signals between system 100C and display 165, speaker 175, or other peripheral devices 185. Output devices may be communicatively coupled to system 100C via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices may be connected to system 100C via communication interface 150 using communication channel 190. Display 165 and speaker 175 may be integrated into a single unit along with other components of system 100C in an electronic device, such as a television. In various embodiments, display interface 160 includes a display driver, such as a timing controller (TCon) chip.

[0073] Alternatively, the display 165 and speaker 175 may be separate from one or more other components, for example, in the case where the RF portion of input 105 is part of a separate set-top box. In various embodiments where the display 165 and speaker 175 are external components, the output signal may be provided via a dedicated output connection, which may include, for example, an HDMI port, a USB port, or a COMP output.

[0074] In any of systems 100A, 100B, or 100C, the embodiments may be implemented by a computer program product comprising code instructions that implement any of the embodiments described herein. The computer program product may be implemented by computer software implemented by processor 110, or by hardware, or a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 120 of any of systems 100A, 100B, or 100C may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. As a non-limiting example, the processor 110 of any of systems 100A, 100B, or 100C may be of any type suitable for the technical environment and may include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0075] Figure 2 An example of a block-based hybrid video encoder 200 is illustrated. Variations of this encoder 200 are envisioned, but for clarity, encoder 200 is described below without describing all anticipated variations.

[0076] In some embodiments, Figure 2 The illustration also shows encoders that improve upon the HEVC or VVC standards (General Video Coding, standard ITU-T H.266, ISO / IEC 23090-3, 2020), or encoders that employ technologies similar to HEVC or VVC, such as the ECM (Enhanced Compression Model) encoder being developed by JVET (Joint Video Exploration Group).

[0077] Before being encoded, the video sequence may undergo pre-coding processing (201), such as applying color transformations to the input color image (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), performing remapping of the input image components to obtain a signal distribution more resilient to compression (e.g., using histogram equalization of the color components), or resizing the image (e.g., downsizing). Metadata may be associated with pre-processing and attached to the bitstream.

[0078] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is segmented (202) and processed in units such as CUs (coding units) or blocks. In this disclosure, different expressions may be used to refer to such units or blocks resulting from the segmentation of the image. This terminology may be encoding unit or CU, encoding block or CB, luminance CB, or block. A CTU (coding tree unit) refers to a set of blocks or a set of units or a set of encoding units (CUs). In some embodiments, a CTU may be considered a block or a unit itself.

[0079] Each unit is encoded using, for example, an intra-frame or inter-frame mode. When a unit is encoded in an intra-frame mode, it performs intra-frame prediction (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. Intra-frame and / or inter-frame modes may include several different sub-modes. For example, intra-frame modes may include directional intra-frame prediction, template-based intra-frame mode derivation prediction, intra-block copy prediction, or other modes that spatially predict the sample values ​​of the unit. Inter-frame modes may include skip modes, merging modes that derive motion information from a motion candidate list and do not encode motion vector prediction residuals, inter-frame modes that derive motion information from a motion candidate list and further refine motion information by encoding motion vector prediction residuals or by template matching performed at both the encoder and decoder, and other inter-frame modes are also possible. The encoder decides (205) which of the intra-frame or inter-frame modes to use to encode the unit. When different intra-frame and / or inter-frame modes are possible, the encoder decides (205) which intra-frame or inter-frame mode to use. The encoder indicates intra-frame / inter-frame decisions through, for example, one or more syntax elements that signal prediction modes. The encoder can also mix (205) intra-frame and inter-frame prediction results, or mix results from different intra-frame / inter-frame prediction methods. The prediction residual is calculated, for example, by subtracting (210) the prediction block from the original image block.

[0080] The motion refinement module (272) uses an already available reference image to refine the motion field of a block without referencing the original block. The motion field of a region can be considered as the set of motion vectors of all pixels within that region. If the motion vectors are based on sub-blocks, the motion field can also be represented as the set of motion vectors of all sub-blocks within that region (all pixels within a sub-block have the same motion vector, and the motion vectors can vary between sub-blocks). If a single motion vector is used for the region, the motion field of that region can also be represented by a single motion vector (the same motion vector of all pixels in that region).

[0081] The predicted residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with the motion vector and other syntax elements, are entropy-encoded (245) to output a bitstream. The encoder can skip the transform and apply the quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process.

[0082] The encoder decodes (reconstructs) the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. A loop filter (265) is applied to the reconstructed image to perform one or more of the following: deblocking filtering, SAO (Sample Adaptive Shift) filtering, or ALF (Adaptive Loop Filter) filtering to reduce coding artifacts. The filtered image is stored at a reference image buffer (280), which is also referred to below as the reference image.

[0083] Figure 3 A block diagram of a video decoder 300 is shown. In decoder 300, the bitstream is decoded by decoder elements, as described below. Video decoder 300 typically performs operations similar to... Figure 2 The decoding rounds are the inverse of the encoding rounds described herein. Encoder 200 typically also performs video decoding as part of the encoded video data.

[0084] Specifically, the input to the decoder includes a video bitstream that can be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, and other encoded information. Image segmentation information indicates how the image is segmented. Therefore, the decoder can segment (335) the image based on the decoded image segmentation information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct image blocks. Prediction blocks (370) can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). A loop filter (365) is applied to the reconstructed image. The filtered image is stored at a reference image buffer (380).

[0085] The prediction block can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375) (370). In a similar manner to that in the encoder, intra-frame prediction and / or inter-frame prediction can include several different sub-modes. The decoder obtains (370) the predictor block based on one or more syntax elements of the prediction modes in the available intra-frame and inter-frame modes, transmitted by signaling. The decoder can mix (370) the intra-frame prediction results and the inter-frame prediction results, or mix the results from multiple intra-frame / inter-frame prediction methods. Before motion compensation, the motion field can be refined (372) using an already available reference picture. A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380). Note that for a given picture, the contents of the reference picture buffer 380 on the decoder 300 side are the same as the contents of the reference picture buffer 280 on the encoder 200 side for the same picture.

[0086] The decoded image can also undergo post-decoding processing (385), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the reverse of the remapping process performed in the pre-encoding process (201), or resizing the reconstructed image (e.g., enlarging). Post-decoding processing can use metadata derived in the pre-encoding process and signaled in the bitstream.

[0087] Some of the embodiments described herein relate to encoding or decoding video, where transform coding uses an inseparable transform as a primary or secondary transform.

[0088] Some implementations focus on improving the transform coding portion of video compression systems, particularly the Low Frequency Non-Separable Transform (LFNST) and Non-Separable Primary Transform (NSPT) tools. In ECM, the optimal non-separable kernel is selected by performing rate-distortion optimization among three transform candidates, whether for LFNST or NSPT. While maximizing performance, this approach increases complexity because additional checking operations are performed during the rate-distortion optimization loop.

[0089] Some embodiments provided herein offer implicit LFNST / NSPT methods and apparatuses in which the optimal kernel from a set of transforms is selected from information known on the decoder side.

[0090] Other embodiments provided herein offer hybrid methods in which implicit or explicit LFNST / NSPT is performed based on the intra-frame mode.

[0091] These embodiments allow video encoders to achieve different rate-distortion and complexity trade-offs.

[0092] The embodiments described herein can be implemented, for example, in the transformation module of a video encoder or video decoder, such as... Figure 2 The mid-video encoder 200 includes a transformation module 225 and an inverse transformation module 250, or... Figure 3 The inverse transformation module 350 of the medium video decoder 300. Some embodiments described herein can also be implemented in the signal transmission module of the video encoder or the parsing module of the video decoder, for example... Figure 2 The entropy encoding module 245 of the medium video encoder 200, or Figure 3 The entropy decoding module 330 of the medium video decoder 300.

[0093] In VVC or ECM, a two-step transform can be applied to the residual block of the coding unit. The first transform applied in this two-step method is usually called the primary transform, while the second transform applied in this two-step method is called the secondary transform.

[0094] It should be understood that the first and second transformations mentioned here refer to the transformation order on the encoding side, and the order is reversed on the decoding side.

[0095] In VVC, the forward LFNST (Low Frequency Inseparable Transform) is applied after the DCT2 transform for the intra-coded block. Therefore, LFNST is a secondary transform, while DCT2 is the primary transform. On the decoder side, the inverse LFNST transform is applied before the inverse DCT2 transform.

[0096] In ECM (Enhanced Compression Model), an additional primary transform is added, called the Inseparable Secondary Transform (NSPT). This primary transform replaces the two-step DCT2-LFNST stage and uses a single inseparable transform. This single NSPT is only permitted for small blocks because using inseparable transforms for large blocks requires enormous memory and computational complexity. It has been shown that this transform stage is associated with high coding gain.

[0097] For non-separable transforms, whether LFNST or NSPT, the optimal kernel is selected through rate-distortion optimization performed among the three transform candidates, and the transform index is passed to the decoder. The selection between NSPT and LFNST is based on block size. For small blocks, the encoder performs rate-distortion optimization among the three NSPT kernels. For larger blocks, RDO is performed among the three LFNST kernels.

[0098] However, by performing LFNST / NSPT searches only for some intra-frame modes, for example, the trade-off between bit rate savings and encoding time and signal transmission costs can be improved.

[0099] This document provides methods and apparatus for improving compression trade-offs when using inseparable transforms. The embodiments provided herein can be applied, for example, to any inseparable transform used in ECM, such as LFNST or NSPT. It should be understood that the embodiments provided herein can also be applied to any other inseparable or separable transform that may be considered in a video compression system. In some embodiments, implicit inseparable transform (e.g., for LFNST / NSPT) methods are provided. In other embodiments, a hybrid scenario utilizing both implicit and explicit inseparable transforms is provided. In still other embodiments, implicit selection of primary transforms among a plurality of primary transforms (before secondary transforms) is provided.

[0100] MTS (Multiple Transform Selection) is used in both VVC and ECM as follows. In VVC design (Universal Video Coding, standard ITU-T H.266, ISO / IEC 23090-3, 2020), in addition to DCT2, DST7 and DCT8 transform cores are used. These cores are used for intra-frame and inter-frame coding. Transform skipping can also be used.

[0101] In ECM, additional primary transformations, including DCT5, DST4, DST1, and identity transformation (IDT), are used to design the MTS set.

[0102] It should be noted that in the current implementations of ECM and VVC, the secondary transformation LFNST is applied only when the primary transform is DCT2. Therefore, although MTS is called the primary transform, in these implementations, the secondary transform is not followed by the MTS transform.

[0103] Furthermore, in ECM, the MTS set depends on the TU (Transform Unit) size and intra-frame mode information. For blocks predicted via IntraTMP (Intra-Template Matching Prediction Mode), the DIMD (Decoder-Side Intra-Frame Mode Derivation) procedure is used to derive the intra-frame mode for transform selection. Specifically, the horizontal and vertical gradients are calculated for each predicted sample to construct a gradient histogram (HoG). The intra-frame prediction mode with the largest histogram magnitude value (called the Virtual Intra-Frame Prediction Mode (VIPM)) is then used to determine the MTS transform set.

[0104] In general, 16 different TU sizes are considered, and for each TU size, 5 different categories are considered based on intra-frame mode information. For each category, 1, 4, or 6 different transform pairs are considered. The number of intra-frame MTS candidates is adaptively selected based on the sum of the absolute values ​​of the transform coefficients (between 1, 4, and 6 MTS candidates). This sum is compared with two fixed thresholds to determine the total number of allowed MTS candidates: One candidate: sum <= th0 Four candidates: th0 <= sum <= th1 6 candidates: sum >= th1 Although a total of 80 distinct categories (16 transform dimensions multiplied by 5 categories) were considered, some of these categories may share the same set of transforms. In the ECM, the resulting lookup table (LUT) contains 58 unique entries (fewer than 80), an example of which is shown in... Figure 4 The diagram is shown in the image.

[0105] For angular modes, joint symmetry of TU shape and intra-prediction is considered. Therefore, mode i (i>34) with TU shape A×B will be mapped to the same category as mode j = (68 - i) with TU shape B×A. However, for each transform pair, the order of the horizontal and vertical transform kernels is swapped. For example, a 16x4 block with mode 18 (horizontal prediction) and a 4x16 block with mode 50 (vertical prediction) are mapped to the same category. However, the vertical and horizontal transform kernels are swapped. For wide-angle modes, the closest regular angular mode is used to determine the transform set. For example, mode 2 is used for all modes between -2 and -14. Similarly, mode 66 is used for modes 67 through 80. Furthermore, a dedicated mode index is assigned to MIP (Matrix Intra-Prediction), resulting in a total of 36 possible mode entries in the LUT.

[0106] An example of selecting a set of transforms in a LUT for a given TU size and intra-frame mode is shown in Figure 4 The diagram shows that `modelIdx` is an index calculated based on the intra-frame mode of the block, and `sizeIdx` is calculated based on the size of the block. For example, a 4x4 block has `sizeIdx=0`, a 4x8 block has `sizeIdx=1`, and so on.

[0107] IDT is applied to blocks that are 16x16 or smaller and whose intra-frame mode is near the horizontal and vertical intra-frame orientations, where this proximity is defined by a threshold that depends on the block size. If the transform index is equal to 3 and the block satisfies the above conditions, then the horizontal and / or vertical identity transform is applied.

[0108] Implicit MTS is a tool in VVC that provides a faster version of explicit MTS, where a single pair of horizontal and vertical transforms is tested instead of multiple pairs. When using implicit MTS, no index is needed to signal the MTS pairs in the set, as only a single transform pair is allowed. The advantages of implicit MTS mode are that it is faster on the encoder side compared to explicit MTS mode and has higher compression gain compared to the case of MTS off.

[0109] In ECM, implicit MTS is implemented by considering only DCT2 and DST7 transformations, as in VVC. In implicit MTS, indexes are not transmitted; transformations are selected based on block size.

[0110] Specifically, if the block height nTbH or width nTbW is between 4 and 16, then the vertical or horizontal transformation is set to DST7 respectively. Otherwise, select DCT2. trTypeHor = (nTbW>= 4&&nTbW<= 16) DST7 : DCT2 trTypeVer = (nTbH>= 4&&nTbH<= 16) DST7 : DCT2 The LFNST design in ECM is an extension of the LFNST design in VVC as follows. The number of LFNST sets (S) and the number of candidates (C) are expanded to S = 35 and C = 3, and the LFNST set (lfnstTrSetIdx) for a given intra-frame mode (predModeIntra) is derived according to the following formula: For predModeIntra < 0, lfnstTrSetIdx equals 2. For predModeIntra in [0,34], lfnstTrSetIdx = predModeIntra For predModeIntra in [35,66], lfnstTrSetIdx = 68 - predModeIntra Three different kernels, namely LFNST4, LFNST8 and LFNST16, are defined to indicate the sets of LFNST kernels applied to the following block sizes: 4xN / Nx4 (N≥4), 8xN / Nx8 (N≥8) and MxN (M, N≥16).

[0111] The kernel dimensions are specified as: (LFSNT4, LFNST8, LFNST16) = (16×16, 32×64, 32×96).

[0112] Forward LFNST is applied to the upper left low-frequency region, which is called the region of interest (ROI). When LFNST is applied, the primary transform coefficients in regions outside the ROI are set to zero, which is unchanged from the VVC standard.

[0113] The ROI of LFNST16 is Figure 5 It is depicted in the diagram. It consists of six 4x4 sub-blocks that are consecutive in the scan order. Since the number of input samples is 96, the transformation matrix of the forward LFNST16 can be Rx96. In the ECM, R is chosen to be 32, so 32 coefficients (two 4x4 sub-blocks) are generated from the forward LFNST16, which are placed in the coefficient scan order.

[0114] The ROI of LFNST8 is as follows: Figure 6 The forward LFNST8 matrix can be Rx64, and R is chosen to be 32. The generated coefficients are positioned in the same way as LFNST16.

[0115] The mapping from intra-prediction modes to these sets is in Figure 7 The table shown illustrates the mapping from intra-frame prediction modes to LFNST set indices.

[0116] The NSPT (Inseparable Primary Transform) used in ECM replaces the two-stage transform (DCT2-LFNST) with a single inseparable transform. In ECM, this is only permitted for small blocks, such as... Figure 8 As indicated in the document, Figure 8 The transformations that can be used depending on the block or CU size are shown.

[0117] All NSPTs consist of 35 sets and 3 candidates (similar to the current LFNST). The kernel of an NSPT has the following shape: NSPT4x4: 16x16 NSPT4x8 / NSPT8x4: 32x20 NSPT8x8: 64x32 NSPT4x16 / NSPT16x4: 64x24 NSPT8x16 / NSPT16x8: 128x40 Therefore, NSPT4x8 / NSPT8x4, NSPT8x8, NSPT4x16 / NSPT16x4, and NSPT8x16 / NSPT16x8 are used to set 12, 32, 40, and 88 coefficients to zero, respectively.

[0118] The encoder and decoder processes of LFNST / NSPT are respectively in Figure 9 and Figure 10 As described in the text.

[0119] Figure 9 The illustration shows an example of method 900 used to encode the current block of an image or video.

[0120] On the encoder side, the residual of the current block is obtained, for example, by subtracting the prediction from the original block (910). At 920, it is checked whether NSPT is allowed for the block. If not, the primary transform coefficients are obtained (930) after applying the DCT2 transform to the residual luminance block. If NSPT is allowed for the current block (yes at 920), this step is skipped. Then, an explicit LFNST or NSPT is performed (940); if DCT2 has been applied, LFNST is performed; or if DCT2 has been skipped, NSPT is performed. For example, a rate-distortion search is performed to select the best kernel in the transform set for LFNST or NSPT, and the index (lfinstIdx) of the selected transform in the set is signaled in the bitstream. It can be seen that the same index, lfinstIdx, is used, which refers to the transform selected in the set for LFNST or the transform selected in the set for NSPT, depending on whether LFNST or NSPT is used.

[0121] The transform coefficients are then quantized (950) and entropy encoded (960) to be included in the bitstream (970).

[0122] Figure 10An example of method 1000 for decoding the current block of an image or video is illustrated. On the decoder side, entropy decoding (1020) is performed on the bitstream (1010), and inverse quantization (1030) is applied to obtain transform coefficients. At 1040, the secondary transform LFNST or the primary transform NSPT is applied, depending on whether NSPT is allowed. Using the obtained intra-frame mode for the current block, a transform set is selected from the available transform sets defined above for LFNST or NSPT. Each transform LFNST and NSPT has its own transform set, but the process for selecting a transform set for the current block based on the intra-frame mode is the same; the transform set is either for LFNST or for NSPT.

[0123] The LFNST / NSPT index (lfinstIdx) is decoded and used to select a transform kernel from the chosen transform set. Using the selected kernel, an inverse LFNST or NSPT transform (1040) is applied to the transform coefficients. If NSPT is not allowed (no at 1050), an inverse DCT2 transform (1060) is further applied to obtain the reconstructed residuals; otherwise, the inverse DCT2 transform is skipped (yes at 1050). The reconstructed luma block (1070) is then obtained by adding predictions.

[0124] exist Figure 9 and Figure 10 In the example, it is assumed that the MTS transform is disabled because, according to the current implementation of ECM or VVC, MTS is not allowed as a primary transform when using LFNST.

[0125] In one embodiment, an implicit mode is provided for determining the inseparable transforms (LFNST or NSPT) within a transform set. In this embodiment, instead of using an index sent by a signal in the bitstream to select the inseparable transform from the transform set, a default inseparable transform is selected based on other information obtained from the bitstream, such as an intra-frame mode determined for the block to be encoded / decoded. This embodiment can be applied to inseparable transforms (NSPT) used as primary transforms or inseparable transforms (LFNST) used as secondary transforms.

[0126] By employing implicitly non-separable transforms LFNST or NSPT, it is possible to reduce coding time while maintaining good compression performance compared to a full search of the transform set. This approach infers the optimal transform type for a given TU from information known on the decoder side, such as intra-prediction mode and TU size.

[0127] This embodiment proposes alternating different transform types for consecutive intra-prediction modes during rate-distortion optimization (RDO). The LFNST or NSPT lookup table can then be fully utilized by distributing individual transforms from the same set across consecutive intra-prediction modes. For example, the index (lfinstIdx) of the corresponding transform pair in the set is computed as the modulus of the intra-prediction mode.

[0128] In one variant, if implicit LFNST / NSPT is enabled, then lfinstIdx is calculated as the intra-prediction mode modulo N: lfinstIdx = intramode % N, (1) Where N is a fixed integer between 1 and the maximum number of transform candidates in the set (3 in ECM). This value determines the maximum number of candidates to be distributed in each transform set. One benefit of this variant is that the encoder checks one transform candidate for each intra-frame mode, instead of all 3 candidates. This reduces encoder complexity while maintaining high performance. However, because consecutive intra-frame modes share the same transform set, this variant avoids always using the same transform from the shared transform set.

[0129] In this variant, when using Figure 7 When mapping intra-prediction modes to LFNST set indices, the selected transform set depends on the intra-prediction mode, and the inseparable transforms selected from the selected transform set also depend on the intra-prediction mode. When the selected transform set is shared among several intra-prediction modes, the default transform selected from said set in implicit mode (without using the transform indices in the signal transmission set) also depends on the intra-prediction mode. Therefore, this allows for the preservation of diversity of inseparable transforms even in implicit mode.

[0130] In ECM, some intra modes compute the VIPM to determine which LFNST / NSPT transform set is used for non-angular (non-directional) intra prediction modes (e.g., MIP or IntraTMP). In one variant, for these modes, the default index lfnstIdx in the implicit mode is set to a fixed 0, 1, or 2 in the transform set. Other intra modes use (1) to compute lfnstIdx.

[0131] In one variant, if LFNST or NSPT is used for IBC or inter-block, then for that mode, lfnstIdx is fixed to a default value in the transform set, which can be 0, 1, or 2. In this case, the transform set can be determined by deriving the intra-mode using DIMD. It can also be a fixed transform set, such as a planar transform set (the transform set indexed by intramode = 0).

[0132] In one variant, a new LUT is constructed where each intra-frame mode is associated with a single LFNST or NSPT kernel. For example, this LUT can be constructed offline by selecting the most frequently used lfnstIdx for each intra-frame mode. In this case, LFNST or NSPT is directly inferred from the LUT for the intra-frame modes with respect to the CU on both the encoder and decoder sides, without using indexes.

[0133] Figure 11 An example of method 1100 for encoding a current block of an image or video according to this embodiment is illustrated. At 1110, an intra-prediction mode is obtained for the current block. This can be an intra-prediction mode for predicting the current block, or a virtual intra-prediction mode determined for the block as described above. At 1120, based on the intra-prediction mode obtained for the current block at 1110, an inseparable transform for the block is selected from a given set of inseparable transforms. The given set of inseparable transforms can be a set of transforms selected from multiple sets of inseparable transforms when multiple sets of inseparable transforms are available for the current block. This selection can be made, for example, using an intra-prediction mode. In another variation, the given set of inseparable transforms can be a LUT constructed as described above by selecting the most commonly used inseparable transform for each intra-mode. At 1130, the current block is encoded using the selected inseparable transform.

[0134] Figure 12 An example of method 1200 for decoding a current block of an image or video according to this embodiment is illustrated. At 1210, an intra-prediction mode is obtained for the current block. This can be an intra-prediction mode for predicting the current block, or a virtual intra-prediction mode determined for the block as described above. At 1220, based on the intra-prediction mode obtained for the current block at 1210, an inseparable transform for the block is selected from a given set of inseparable transforms. At 1230, the current block is decoded using the selected inseparable transform.

[0135] Another embodiment is provided, in which an implicit choice is made between implicit / explicitly inseparable transformations (such as LFNST or NSPT).

[0136] In this embodiment, the intra-prediction mode of the current block is used to select between implicit or explicit LFNST / NSPT for the current TU. This approach allows the encoder to achieve a different performance / complexity tradeoff compared to enabling implicit or explicit LFNST / NSPT using flags signaled in the high-level syntax.

[0137] It should be noted that, for example in ECM, some coding tools affect certain intra-frame modes (e.g., DIMD, TIMD), which may be unavailable when processing transform indices, thus preventing the decoder from choosing between explicit or implicit non-separable transforms based on the intra-frame prediction mode. Therefore, implicit selection is limited to intra-frame prediction modes known during the parsing stage, such as plane and MIP in the case of ECM implementations.

[0138] Therefore, the implicit selection implementation based on intra-frame mode is only effective when a given intra-frame prediction mode is available during the parsing phase. Explicit modes for inseparable transforms (e.g., LFNST or NSPT) are defined as modes in which an index selects an inseparable transform from the set of inseparable transforms, while implicit modes are defined as modes in which the selection of inseparable transforms does not use an index explicitly sent with a signal. For example, in implicit modes, additional data from the bitstream can be used. For example, the implementation described above that uses an intra-frame prediction mode to select inseparable transforms from the set of inseparable transforms can be used.

[0139] It can be seen that the choice between implicit and explicit modes affects whether the inseparable transform index is parsed, therefore the decision between implicit and explicit modes must be made using the information available during bitstream parsing.

[0140] Therefore, the implicit selection between implicit and explicit modes for non-separable transforms should use intra-modes known during the resolution phase, i.e., intra-modes not derived from other modes unknown during the resolution phase. For example, intra-prediction modes obtained through the DIMD process or IntraTMP are not available during the resolution phase, while MIP or planar intra-prediction modes are available because they are explicitly signaled in the bitstream.

[0141] Figure 13An example of method 1300 for encoding a current block of an image or video according to this embodiment is illustrated. At 1310, an intra-prediction mode is obtained for encoding the current block. At 1320, based on the intra-prediction mode obtained for the block, it is determined whether the index of the indivisible transform selected from the set of indivisible transforms is determined according to an explicit mode or an implicit mode. In one variation, the determination at 1320 is based on whether the intra-prediction mode obtained for the block is available during the parsing stage. In another variation, the determination at 1320 is based on whether the intra-prediction mode belongs to a given set of intra-prediction modes. The given set of intra-prediction modes includes, for example, one or more intra-prediction modes available during the parsing stage. In another variation, all intra-prediction modes within the given set of intra-prediction modes are intra-prediction modes available during the parsing stage or can be known during the parsing stage of the bitstream. In yet another variation, the given set of intra-prediction modes includes at least one of MIP mode, IBC mode, intra-template matching mode, or planar mode.

[0142] For example, if the intra-prediction mode is in the given set of intra-prediction modes, the implicit mode is used; otherwise, the explicit mode is used. In another example, if the intra-prediction mode is in the given set of intra-prediction modes, the explicit mode is used; otherwise, the implicit mode is used.

[0143] At 1330, the inseparable transform is determined in either implicit or explicit mode, and the block is encoded using the determined inseparable transform. In implicit mode, the inseparable transform is not determined using the index of the signal explicitly sent in the bitstream. In one variant, it can be determined based on the above regarding... Figure 11 and 12 The described embodiments determine the inseparable transform. In another variation, the inseparable transform can be determined using additional data obtained from the bitstream or from the reconstructed data. In yet another variation, the inseparable transform can be a default inseparable transform known between the encoder and decoder. In explicit mode, rate-distortion optimization is performed to determine which inseparable transform in the set provides the best rate / distortion cost, and the index of the determined inseparable transform in the set is signaled in the bitstream.

[0144] Figure 14 An example of method 1400 for decoding the current block according to this embodiment is illustrated. At 1410, an intra-prediction mode is obtained for the current block. If the intra-prediction mode is a mode available during the parsing stage, it can be decoded from the bitstream. Otherwise, the intra-prediction mode can be set to unavailable at this stage.

[0145] At 1420, based on the intra-prediction mode obtained for this block, it is determined whether the index of the inseparable transform is selected from the set of inseparable transforms according to the explicit or implicit mode. Here, the encoding method is performed ( Figure 13 The same process as in 1320. At 1430, the inseparable transform is determined in either implicit or explicit mode, and the block is decoded using the determined inseparable transform. In implicit mode, for example, according to the above regarding... Figure 11 The embodiments described in 12 determine the inseparable transformations. In explicit mode, decoding is used to select the index of the inseparable transformation from the set.

[0146] In one variant of this embodiment, such as in the current implementation of ECM, only MIP and planar intra-prediction modes are intra-modes available during the resolution phase. Therefore, in this variant, only MIP and planar intra-prediction modes are used to determine between implicit and explicit modes for the non-separable transform LFNST / NSPT. However, other implementations are conceivable where more and / or other modes are available during the resolution phase and can be used for implicit selection.

[0147] Figure 15 Figures 1 and 16 illustrate the encoder and decoder processes for this variant, respectively. Figure 15 An example of method 1500 for encoding the current block of an image or video according to this variant is illustrated. On the encoder side, the residual of the current block is obtained (1510), for example, by subtracting the prediction from the original block. At 1520, it is checked whether NSPT is allowed for the block. If not, the primary transform coefficients are obtained (1530) after applying the DCT2 transform to the residual luma block. If NSPT is allowed for the current block (yes at 1520), this step is skipped. At 1531, it is determined whether implicit or explicit mode is used for the non-separable transform (LFNST or NSPT). If the intra-prediction mode of the current block is planar and / or MIP, explicit LFNST or NSPT is performed at 1540. If DCT2 has already been applied, LFNST is performed; or if DCT2 has been skipped, NSPT is performed. For example, a rate-distortion search is performed to select the best LFNST or NSPT kernel from the transform set, and its index (lfinstIdx) is signaled in the bitstream. For other intra-frame modes (not at 1531), an implicit LFNST or NSPT is performed at 1532. Here, no syntax element is used to signal the LFNST or NSPT used; the transform kernel index is derived from information known on the decoder side, such as using the information above regarding... Figure 11Or the embodiment described in 12. The transform coefficients are then quantized (1550) and entropy encoded (1560) to be included in the bitstream (1570).

[0148] Figure 16 An example of method 1600 for decoding the current block according to this variant is illustrated. On the decoder side, entropy decoding (1620) is performed on the bitstream (1610), and inverse quantization (1630) is applied to obtain the transform coefficients. At 1631, it is determined whether implicit or explicit mode is used for the non-separable transform (LFNST or NSPT). If the intra-prediction mode of the current block is planar and / or MIP, then at 1640, explicit mode of LFNST / NSPT is enabled for the current block (i.e., inferred from the use of planar or MIP intra-prediction mode), and the LFNST / NSPT kernel is selected in the transform set using the decoded LFNST / NSPT index (lfinstIdx). It should be noted that in this embodiment, the determination at 1631 is also performed during the entropy decoding (1620) of the bitstream to determine whether the LFNST / NSPT index (lfinstIdx) must be decoded.

[0149] Otherwise (No at 1631), implicit LFNST / NSPT is enabled (1632), and the transform kernel for the non-separable transform is derived from information known on the decoder side, such as using information about Figure 11 The embodiments described in 12 are as follows. The inverse LFNST / NSPT transform is applied using the selected check transform coefficients.

[0150] At 1640 and 1632, the inverse transform LFNST or the inverse transform NSPT is applied, depending on whether NSPT is allowed. If NSPT is not allowed (no at 1650), the inverse DCT2 transform (1660) is further applied to obtain the reconstruction residual; otherwise, the inverse DCT2 transform is skipped (yes at 1650). The reconstructed luma block (1670) is then obtained by adding prediction.

[0151] exist Figure 15 and Figure 16 In another variant, implicit LFNST / NSPT is enabled for MIP and / or planes, and explicit LFNST / NSPT is enabled for other modes (i.e., in...). Figure 15 Middle blocks 1540 and 1532 were swapped. Figure 16 Middle blocks 1640 and 1632 were swapped.

[0152] Another embodiment is provided, in which implicit selection between primary transforms (before secondary transforms) is provided. For example, in the case of an ECM implementation, this embodiment provides a hybrid implicit / explicit MTS for LFNST. In other words, this embodiment provides implicit selection between DCT2 or MTS before applying secondary transforms such as LFNST.

[0153] For example, in a manner similar to the embodiments described above, the intra-frame mode of the current block can be used to select whether to apply DCT2 or Multi-Transform Selection (MTS) before the LFNST for the current TU.

[0154] As mentioned above, using intra-frame modes is only effective when a given intra-frame prediction mode is available during the resolution phase, i.e., the intra-frame prediction mode does not require reconstructed data for its derivation.

[0155] Therefore, for example, in current ECM implementations, only MIP and planar intra-prediction modes can be used for this purpose. However, other implementations can be considered, where more and / or other modes are available during the resolution phase and can be used for implicit selection.

[0156] In this embodiment, for certain intra-prediction modes, MTS is then allowed as a primary transform before a secondary transform such as LFNST. The complexity of evaluating all combinations of intra-prediction modes and possible transforms at the encoder does not increase much, since MTS is only allowed after LFNST for some intra-prediction modes, and the cost of signaling is also limited, since no additional flag is needed to signal that MTS is enabled for the current block.

[0157] Figure 17 An example of method 1700 for encoding a current block of an image or video according to this embodiment is illustrated. At 1710, an intra-prediction mode for encoding the block is obtained. At 1720, based on the intra-prediction mode obtained for the block, it is determined whether the primary transform to be applied to the block is a single separable transform or a separable transform selected from multiple separable transforms. At 1720, previously regarding... Figure 13The same variations of the embodiment described in 14 can be applied here. In one variation, the determination at 1720 is based on whether the intra-prediction mode obtained for the block is available during the parsing phase. In another variation, the determination at 1720 is based on whether the intra-prediction mode belongs to a given set of intra-prediction modes. This given set of intra-prediction modes includes, for example, one or more intra-prediction modes available during the parsing phase. In another variation, all intra-prediction modes within the given set of intra-prediction modes are intra-prediction modes available during the parsing phase, i.e., intra-prediction modes that can be known during the bitstream parsing phase. In yet another variation, the given set of intra-prediction modes includes at least one of MIP mode, IBC mode, intra-template matching mode, or planar mode.

[0158] For example, if the intra-prediction mode is in the given set of intra-prediction modes, a single separable transform (e.g., DCT2) is used; otherwise, a separable transform selected from multiple separable transforms is used (e.g., as in the MTS tool). In another example, if the intra-prediction mode is in the given set of intra-prediction modes, a separable transform (MTS) selected from multiple separable transforms is used; otherwise, a single separable transform (e.g., DCT2) is used.

[0159] At 1730, the block is encoded using the determined separable transform. When obtaining the primary transform using multiple separable transforms (MTS), the same mechanism described for the MTS can be used. In one variant, the index of the transform selected from the MTS set can be explicitly signaled. In another variant, the MTS can be used in implicit mode, where the pair of separable transforms is selected from the MTS set, for example, with respect to... Figure 11 Similar to the embodiments related to the implicit non-separable transform described in 12, the intra-prediction mode obtained for the current block is used.

[0160] In another variant of 1730, the coefficients obtained from the block are further transformed using a secondary transform, which is transmitted as a signal in either implicit or explicit mode, using the determined separable transform. For example, the secondary transform is LFNST.

[0161] Figure 18 The illustration shows an example of method 1800 for decoding the current block according to this variant. At 1810, the intra-prediction mode is obtained for decoding the block. If the intra-prediction mode is available during the parsing stage, it can be decoded from the bitstream. Otherwise, the intra-prediction mode can be set to unavailable at this stage.

[0162] At 1820, based on the intra-prediction mode obtained for this block, it is determined whether the primary transform to be applied to this block is a single separable transform or a separable transform selected from multiple separable transforms. At 1820, the application is... Figure 17 The same process as in 1720.

[0163] At 1830, the block is reconstructed using the determined separable transform. When it is determined at 1820 that the primary transform was obtained using multiple separable transforms (MTS), the same mechanism described for MTS can be used. The separable transform pairs used for MTS can be signaled in implicit or explicit mode. In another variant of 1830, before applying the separable transform, the decoded coefficients can first be transformed using a secondary inverse transform, which can be signaled in implicit or explicit mode. The inversely transformed coefficients are then further inversely transformed using the determined separable transform to reconstruct the current block.

[0164] In the VVC standard or current ECM implementations, the primary transform of MTS is not allowed when using the secondary transform LFNST. This is discussed in the section on... Figure 17-20 The embodiments described in the variants shown provide the ability to enable MTS even when LFNST is used. For example, the use of LFNST can be determined by decoding lfnstIdx (e.g., lfnstIdx > 0 indicating the use of LFNST, or a given flag indicating the use of LFNST for the current block). Then, at 1820, in response to determining that LFNST is used for the current block, based on the intra-prediction mode obtained for the current block, the same method used for the above is applied. Figure 17 The same variant as described in 1720 determines whether MTS is allowed for the current block. When MTS is allowed, the MTS index is determined implicitly or explicitly. When MTS is not allowed, DCT2 is used, for example.

[0165] Figure 19 Figures 20 and 21 respectively illustrate the encoder and decoder processes of a variant of this embodiment when applied to the current implementation of the ECM. The encoding and decoding steps are related to... Figure 9 and Figure 10 The steps described are similar, except for the following changes, which will be described below.

[0166] On the encoder side, in the absence of NSPT, the intra-prediction mode of the block is used to determine (at 1931) whether to apply DCT2 or MTS to the residual luminance block to obtain the primary transform coefficients. If the intra-prediction mode of the current block is planar and / or MIP, MTS (1933) is performed. In this variant, rate-distortion search is performed to select the optimal primary transform pair, and the index is sent in the bitstream (mts_idx). For other intra-prediction modes (not at 1931), DCT2 (1932) is performed. LFNST / NSPT is then applied to obtain the transform coefficients. Finally, the transform coefficients are quantized and entropy encoded.

[0167] On the decoder side, the bitstream is first entropy decoded, and inverse quantization is applied to obtain the transform coefficients. First, if allowed, LFNST or NSPT is applied to the transform coefficients. If NSPT is not allowed, the intra-prediction mode of the block is used to determine (2031) which inverse transform to further apply to the transform coefficients. If the intra-prediction mode of the current block is planar and / or MIP (yes at 2031), the inverse primary transform pair selected by the MTS index (2033) is applied to the transform coefficients to obtain the reconstructed luma block. Otherwise (no at 2031), inverse DCT2 (2032) is used.

[0168] In one variant of this embodiment, DCT2 is enabled for MIP and planar intra-prediction modes, and MTS for LFNST is enabled for other intra-prediction modes (i.e., in...). Figure 19 Middle blocks 1932 and 1933 were swapped, and... Figure 20 Middle blocks 2032 and 2033 were swapped.

[0169] exist Figure 19 and Figure 20 In the variant shown, explicit LFNST / NSPT is used. In another variant, explicit LFNST / NSPT can be replaced with implicit LFNST / NSPT (i.e., Figure 19 Block 1934 and Figure 20 Block 2034 in the middle was Figure 15 Implicit LFNST / NSPT 1532 and Figure 16 Replace 1632 in the code, do not use lfnstIdx as... Figure 20 (Input for 2034).

[0170] exist Figure 19 and Figure 20In the variant shown, an explicit MTS is used. In another variant, the MTS for LFNST is replaced by an implicit MTS, and mts_idx is not transmitted (i.e., Figure 19 1933 and Figure 20 2033 in the original code is replaced by MTS in implicit mode, where mts_idx is not used as... Figure 20 (Input for 2033).

[0171] In another variant, DCT2 is replaced by MTS in implicit mode, and mts_idx is not transmitted (i.e., Figure 19 1932 and Figure 20 The 2032 in the original code is replaced by MTS in implicit mode. mts_idx is not used as... Figure 20 (Input for 2032).

[0172] In another variant, explicit LFNST / NSPT can be as follows: Figure 15 Conditioning is performed as in the embodiment described in 16. This variant provides a hybrid implicit / explicit MTS prior to the hybrid implicit / explicit LFNST. In this variant, regarding Figure 15 and Figure 16 The described embodiments are similar to those previously discussed. Figure 19 and Figure 20 The described embodiment combination. The intra-prediction mode of the current block is used to first select between DCT2 and MTS, and then between implicit or explicit LFNST / NSPT for the current TU. The use of this variant allows the encoder to achieve a different performance / complexity tradeoff than that proposed in previous embodiments. As previously stated, this variant is only effective if a given intra-prediction mode is available during the resolution phase. Therefore, using the current ECM implementation, only MIP and planar intra-prediction modes are used to decide between implicit and explicit LFNST / NSPT. Figure 21 and Figure 22 The encoder and decoder processes for this variant are illustrated separately.

[0173] On the encoder side, if NSPT is not allowed, the intra-prediction mode of the block is used to determine (1931) whether to apply DCT2 or MTS to the residual luminance block to obtain the primary transform coefficients. If the intra-prediction mode of the current block is planar and / or MIP, MTS (1933) is performed. In this variant, a rate-distortion search is performed to select the best primary transform pair, and its index (mts_idx) is signaled in the bitstream. For other intra-prediction modes (not at 1931), DCT2 (1932) is performed. Then, if the intra-prediction mode of the current block is planar and / or MIP, explicit LFNST / NSPT (1540) is performed. In this variant, a rate-distortion search is performed to select the best LFNST / NSPT kernel in the transform set, and its index (lfinstIdx) is signaled in the bitstream. For other intra-prediction modes (not at 1531), implicit LFNST / NSPT (1532) is performed. Here, no syntax elements are used for LFNST / NSPT. The transform kernel index can be derived, for example, from information known on the decoder side, such as using information about... Figure 11 The embodiments described in 12 are followed. Finally, the transform coefficients are quantized and entropy encoded.

[0174] On the decoder side, entropy decoding of the bitstream is first performed, and inverse quantization is applied to obtain the transform coefficients. If explicit LFNST / NSPT is enabled for the current block (i.e., using planar or MIP intra-prediction mode 1631), the LFNST / NSPT kernel is selected from the transform set using the LFNST / NSPT index (lfinstIdx) (1640). Otherwise (no at 1631), implicit LFNST / NSPT is enabled (1632), and the transform kernel is derived from information known on the decoder side, such as information about... Figure 11 The embodiment described in section 12 is then applied to the transform coefficients using the selected kernel. If NSPT is not allowed, and the intra-prediction mode of the current block is planar and / or MIP (2031), the inverse primary transform pair selected by the MTS (2033) is applied to the transform coefficients to obtain the reconstructed luma block. Otherwise (no at 2031), inverse DCT2 (2032) is used.

[0175] In one variant, DCT2 is enabled for MIP and planar intra-prediction modes, and MTS for LFNST is enabled for other intra-prediction modes (i.e., in...). Figure 21 The two were exchanged in 1932 and 1933, and in Figure 22 (The 2032 and 2033 were swapped).

[0176] In one variant, implicit LFNST / NSPT is enabled for MIP and / or planes, and explicit LFNST / NSPT is enabled for other modes (i.e., in...). Figure 21 In 1540 and 1532, they were exchanged, and in Figure 22 (In 1640 and 1632, they were exchanged).

[0177] In another variant, the MTS for LFNST is replaced by an implicit MTS, and mts_idx is not transmitted (i.e., Figure 21 1933 and Figure 22 The 2033 in the code is replaced by implicit MTS. mts_idx is not used as... Figure 22 (Input for 2033).

[0178] In another variant, DCT2 is replaced by implicit MTS, and mts_idx is not transmitted (i.e., in...). Figure 21 and Figure 22 In versions 1932 and 2032, implicit MTS is used instead. `mts_idx` is not used as... Figure 22 (Input for 2032).

[0179] In one embodiment, such as Figure 23 As shown, in a transmission scenario between two remote devices A and B via a communication network NET, device A includes processors associated with RAM and ROM, configured to implement methods for encoding images or video according to any embodiment described herein, and device B includes processors associated with RAM and ROM, configured to implement methods for decoding images or video according to any embodiment described herein. According to the example, the network is a broadcast network adapted to broadcast / transmit encoded video from device A to a decoding device including device B.

[0180] Figure 24 An example of the syntax for signals transmitted via a packet-based transport protocol is shown. Each transmitted data packet P includes a header H and a payload PAYLOAD. In some embodiments, the payload PAYLOAD may include data representing at least a portion of an image or video encoded according to any of the embodiments described above. The payload may also include any signal transmissions as described above.

[0181] Various implementations involve decoding. As used herein, “decoding” can encompass, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process also includes, or alternatively includes, processes performed by a decoder of the various implementations described herein, such as entropy decoding of a sequence of binary symbols to reconstruct image or video data.

[0182] As further examples, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding; and in yet another embodiment, "decoding" refers to the entire image reconstruction process including entropy decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process will be clear based on the specific context of the description and is considered well understood by those skilled in the art.

[0183] Various implementations involve encoding. In a manner similar to the discussion of “decoding” above, as used herein, “encoding” can encompass all or part of the process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such a process includes one or more processes typically performed by an encoder, such as segmentation, differential coding, transform, quantization, and entropy coding. In various embodiments, such a process also includes, or alternatively includes, processes performed by an encoder of the various implementations described herein, such as determining resampling filter coefficients and resampling the decoded image.

[0184] As a further example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process will be clear based on the specific context of the description and is considered well understood by those skilled in the art.

[0185] Note that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0186] This disclosure describes various types of information, such as, for example, transmissible or storable information, including syntax. This information can be encapsulated or arranged in various ways, including those common in video standards, such as placing the information in SPS, PPS, NAL units, headers (e.g., NAL unit headers, picture headers, or slice headers), or SEI messages. Other methods are also available, including those common in system-level or application-level standards, such as placing the information in one or more of the following: a. SDP (Session Description Protocol), a format for describing multimedia communication sessions, used for session announcements and invitations, such as those described in RFCs and used in conjunction with RTP (Real-Time Transport Protocol) transmission. b. DASH MPD (Media Presentation Description) descriptors, such as those used in DASH and transmitted via HTTP. A descriptor is associated with one or more representations to provide additional characteristics for content representation. c. RTP header extensions, for example, used during RTP streaming. d. ISO-based media file formats, such as those used in OMAF, and using boxes as object-oriented building blocks (also referred to as “atoms” in some specifications) defined by unique type identifiers and lengths. e. An HLS (HTTP Live Streaming) manifest transmitted over HTTP. The manifest can be associated with, for example, versions or sets of versions of content to provide the characteristics of that version or set of versions.

[0187] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding apparatus. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0188] Some embodiments involve rate distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, given computational complexity constraints. Rate distortion optimization is generally planned to minimize a rate distortion function, which is a weighted sum of rate and distortion. Different approaches exist to address the rate distortion optimization problem. For example, these approaches can be based on extensive testing of all encoding options, including all considered modes or encoding parameter values, where their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding are fully evaluated. Faster methods can also be used to save encoding complexity, particularly in the case of computing approximate distortion based on predicting or predicting the residual signal rather than the reconstructed signal. A hybrid of these approaches can also be used, such as by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both encoding costs and associated distortion.

[0189] The implementations and aspects described herein can be implemented, for example, as methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the features in question can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, as appropriate hardware, software, and firmware. Methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants (PDAs), and other devices that facilitate information communication between end users.

[0190] The references to "an embodiment," "an embodiment," "an implementation," or "an implementation," and other variations thereof, refer to the inclusion of a specific feature, structure, characteristic, etc., described in relation to that embodiment in at least one embodiment. Therefore, the phrases "in an embodiment," "in an embodiment," "in an implementation," or "in an implementation," and any other variations appearing in various places throughout this application, do not necessarily refer to the same embodiment.

[0191] Furthermore, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of the following: estimated information, calculated information, predicted information, or information retrieved from memory.

[0192] Furthermore, this application may involve "accessing" various types of information. Accessing information may include, for example, one or more of the following: receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0193] Furthermore, this application may involve "receiving" various types of information. "Receiving," like "accessing," is a broad term. Receiving information may include, for example, one or more of the following: accessing information or retrieving information (e.g., from memory). Additionally, "receiving" is generally referred to in one or more ways during operations such as: storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0194] It should be understood that the use of any of the following “ / ”, “and / or” and “at least one” (e.g., in the cases of “A / B”, “A and / or B”, and “at least one of A and B”) is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such expressions are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). It will be clear to those skilled in the art and related fields that this can be extended to all listed items.

[0195] Furthermore, as used herein, the word "signal" refers, among other meanings, to instructing the corresponding decoder to do something. Thus, in embodiments, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can transmit (explicitly signal) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already possesses the specific parameter and other information, signaling can be used to inform the decoder of and select the specific parameter without transmission (implicit signaling). Bit savings are achieved in various embodiments by avoiding the transmission of any actual function. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word "signal" has been used above, the word "signal" can also be used as a noun herein.

[0196] It will be apparent to those skilled in the art that various implementations can generate various signals, which are formatted to carry, for example, storable or transmissible information. This information may include, for example, instructions for performing a method or data generated by one of the implementations. For example, the signal may be formatted to carry a bitstream of the embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding the data stream and using the encoded data stream to modulate a carrier wave. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0197] Numerous embodiments have been described herein. Features of these embodiments may be provided individually or in any combination across a variety of claim classes and types, for example: A method includes: obtaining an intra-prediction mode for a block of an image; determining, based on the intra-prediction mode obtained for the block, whether an inseparable transform index for selecting the inseparable transform for the block from a first set of inseparable transforms is determined in an explicit mode or an implicit mode, wherein in an explicit mode the inseparable transform index is explicitly signaled in a bitstream, and in an implicit mode the inseparable transform index is derived from other data signaled in a bitstream; and encoding the block using the determined inseparable transform.

[0198] Variants of the embodiments that can be used alone or in combination may include at least one of the following: - An inseparable transformation is either the primary transformation to be applied to the block, or a secondary transformation to be applied to a version of the block obtained after applying the primary transformation. - When the inseparable transform index is implicitly derived from other data sent by signaling in the bitstream, the inseparable transform index is obtained based on the intra-prediction mode obtained for that block.

[0199] A method includes: obtaining an intra-prediction mode for a block of an image; determining, based on the intra-prediction mode obtained for the block, whether a primary transform to be applied to the block is a single separable transform or a separable transform selected from a plurality of separable transforms; and encoding the block using the determined separable transform.

[0200] Variants that can be used alone or in combination may include at least one of the following: - Determining whether the primary transform to be applied to the block, based on the intra-prediction mode obtained for that block, is a single separable transform or a separable transform selected from multiple separable transforms, depends on whether the intra-prediction mode belongs to a given set of intra-prediction modes. - All intra prediction modes within the given set of intra prediction modes are intra prediction modes that are available or known during the bitstream parsing stage. - If the intra-prediction mode is within the given set of intra-prediction modes, then the primary transform is a single separable transform; otherwise, the primary transform is selected from multiple separable transforms. - If the intra-prediction mode is not in the given set of intra-prediction modes, the primary transform is a single separable transform; otherwise, the primary transform is selected from multiple separable transforms. - The given set of intra prediction modes includes at least one of MIP, IBC, intra template matching, or planar mode. - After applying the primary transformation to this block, a secondary transformation is applied. This secondary transformation is an inseparable transformation. Secondary transformations can be sent implicitly or explicitly as signals within the bitstream. - When the primary transform is a separable transform selected from a plurality of separable transforms, the separable transform is transmitted in explicit mode by signaling, according to which the index of the separable transform among the plurality is transmitted in the bitstream by signaling. - When the primary transform is a separable transform selected from a plurality of separable transforms, the separable transform is signaled in an implicit mode, according to which the separable transform is derived from the plurality using data other than the separable transform index that is signaled in the bitstream.

Claims

1. A method comprising: Obtain the intra-frame prediction mode for a block of image. Based on the intra-prediction mode obtained for the block, it is determined whether the index of the inseparable transform is determined in explicit mode or implicit mode, wherein in explicit mode, the index of the inseparable transform is explicitly signaled in the bitstream, and in implicit mode, the index of the inseparable transform is derived from other data signaled in the bitstream. The block is encoded using an indivisible transformation selected from the set of indivisible transformations based on the index.

2. An apparatus comprising one or more processors, operating as follows: Obtain the intra-frame prediction mode for a block of image. Based on the intra-prediction mode obtained for the block, it is determined whether the index of the inseparable transform is determined in explicit mode or implicit mode, wherein in explicit mode, the index of the inseparable transform is explicitly signaled in the bitstream, and in implicit mode, the index of the inseparable transform is derived from other data signaled in the bitstream. The block is encoded using an indivisible transformation selected from the set of indivisible transformations based on the index.

3. A method comprising: Obtain the intra-frame prediction mode for a block of image. Based on the intra-prediction mode obtained for the block, it is determined whether the index of the inseparable transform is determined in explicit mode or implicit mode, wherein in explicit mode, the index of the inseparable transform is explicitly signaled in the bitstream, and in implicit mode, the index of the inseparable transform is derived from other data signaled in the bitstream. The block is decoded using an inseparable transform selected from the set of inseparable transforms based on the index.

4. An apparatus comprising one or more processors, operating as follows: Obtain the intra-frame prediction mode for a block of image. Based on the intra-prediction mode obtained for the block, it is determined whether the index of the inseparable transform is determined in explicit mode or implicit mode, wherein in explicit mode, the index of the inseparable transform is explicitly signaled in the bitstream, and in implicit mode, the index of the inseparable transform is derived from other data signaled in the bitstream. The block is decoded using an inseparable transform selected from the set of inseparable transforms based on the index.

5. The method according to any one of claims 1 or 3, or the apparatus according to any one of claims 2 or 4, wherein, Whether the index of the inseparable transform is determined based on the intra-prediction mode obtained for the block is determined in explicit or implicit mode, depends on whether the intra-prediction mode belongs to a given set of intra-prediction modes available during the parsing phase of the bitstream.

6. The method according to any one of claims 1, 3, or 5, or the apparatus according to any one of claims 2, 4, or 5, wherein, When the index is determined in the implicit mode, the inseparable transform is selected from the set of inseparable transforms based on the intra-prediction mode.

7. The method according to any one of claims 1, 3, or 5-6, or the apparatus according to any one of claims 2, 4-6, wherein, The inseparable transformation is either a primary transformation to be applied to the block or a secondary transformation to be applied to a version of the block obtained after applying the primary transformation.

8. The method according to any one of claims 1, 3, or 5-7, or the apparatus according to any one of claims 2, 4-7, wherein, The index is obtained by taking the modulo of a given integer as the index of the intra-frame prediction mode.

9. The method or apparatus according to claim 8, wherein, The given integer is between 1 and the number of inseparable transformations in the set of inseparable transformations.

10. The method according to any one of claims 1, 3, or 5-9, or the apparatus according to any one of claims 2, 4-9, wherein, Obtaining the intra-prediction mode for the block from the set of intra-prediction modes, and selecting the inseparable transform for the block from the set of inseparable transforms, includes: A plurality of sets of inseparable transforms are obtained, wherein each intra-prediction mode in the set of intra-prediction modes is associated with one of the sets of inseparable transforms. At least one of the multiple sets of inseparable transforms is shared among at least two intra-prediction modes in the set of intra-prediction modes.

11. The method of any one of claims 1, 3, or 5-10, or the apparatus of any one of claims 2, 4-10, wherein in response to determining that the block is encoded using a prediction mode other than a directional intra-prediction mode, the index of the intra-prediction mode obtained for the block is set to a default given value.

12. The method according to any one of claims 1, 3, or 5-11, or the apparatus according to any one of claims 2, 4-11, wherein, The non-separable transform is a secondary transform to be applied to the block version obtained after the application of the primary transform, and wherein the method further includes, or the one or more processors further operate, to: determine, based on the intra-prediction mode obtained for the block, whether the primary transform to be applied to the block is a single separable transform or a separable transform selected from a plurality of separable transforms.

13. The method or apparatus according to claim 12, wherein, Determining whether the primary transform to be applied to the block is a single separable transform or a separable transform selected from multiple separable transforms based on the intra-prediction mode obtained for the block depends on whether the intra-prediction mode belongs to a given set of intra-prediction modes available during the parsing phase of the bitstream.

Images