Intra sub-partitioning for video encoding and decoding in combination with multiple transform selections, matrix-weighted intra prediction or multiple reference line intra prediction

Abstract

In an encoding device or a decoding device, the encoding method or the decoding method combines various encoding tools with an intra-frame sub-partition mode, in which the coding unit is divided into multiple sub-partitions. At least one embodiment is particularly intended to select a primary transform from a set of multiple transforms for video encoding or decoding of the intra-frame sub-block partition. At least one embodiment is particularly intended to use a matrix-based intra-frame prediction mode combined with the intra-frame sub-partition mode for intra-frame coding. At least one embodiment is particularly intended to use a multi-reference line intra-frame prediction mode combined with the intra-frame sub-partition mode for intra-frame coding.

Description

Technical Field

[0001] At least one of the embodiments of the present invention generally relates to the field of video compression. The embodiments are intended to adapt some coding techniques to an intra-frame sub-partition mode in which a coding unit is divided into multiple sub-partitions. At least one embodiment is particularly intended to select a primary transform from a set of multiple transforms for video encoding or decoding of an intra-frame sub-block partition. At least one embodiment is particularly intended to use a matrix-based intra-frame prediction mode combined with the intra-frame sub-partition mode for intra-frame coding. At least one embodiment is particularly intended to use a multi-reference line intra-frame prediction mode combined with the intra-frame sub-partition mode for intra-frame coding. Background Art

[0002] To achieve high compression efficiency, image and video coding schemes usually use prediction and transform to exploit spatial and temporal redundancy in video content. Generally speaking, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original block and the predicted block (usually expressed as prediction error or prediction residual) is transformed, quantized, and entropy encoded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy coding, quantization, conversion, and prediction. Summary of the invention

[0003] One or more of the embodiments herein relate to selecting transform pairs for transform units in a coding unit encoded in an intra sub-partition mode, whereby the coding unit is partitioned into a plurality of sub-partitions, and also to signaling of the selected transform pairs.

[0004] One or more embodiments of the present embodiment relate to using matrix-based intra prediction for blocks encoded using intra sub-partitioning mode. This allows for improved compression efficiency of video codecs. In other words, it allows encoding, signaling, and decoding of blocks for which intra prediction divides the block into multiple sub-blocks, and where prediction of the sub-blocks uses matrix-based intra prediction, where a 2D matrix and a 1D vector are applied to the vector based on a column of reconstructed reference samples on the left side of the block and a line of reconstructed reference samples on the top of the current block.

[0005] Various embodiments relate to a video encoding system including an intra prediction mode, wherein a block of an image of the video is partitioned into a plurality of sub-partitions, and samples of the sub-partitions are predicted based on a reference line including reference samples, the reference line being selected from a plurality of reference lines.

[0006] One or more embodiments of the present invention relate to using multiple reference line intra prediction for blocks encoded using intra sub-partition mode. This improves the compression efficiency of the video codec. In other words, it allows encoding, signaling and decoding of blocks where intra prediction divides the block into multiple equal-sized sub-blocks, while allowing each sub-block to be predicted based on any one of the available multiple reference lines.

[0007] According to a first aspect of at least one embodiment, a video encoding method includes: dividing an obtained video block into sub-partitions; applying a coding tool on samples of the sub-partitions; and encoding blocks and signaling information, wherein the signaling information includes at least information representing the coding tool and associated parameters, and wherein the coding tool applies a transform to the sub-partition based on a set of transforms, or performs intra-frame prediction by applying a 2D matrix and a 1D vector to a reconstructed neighbor of the block, or performs intra-frame prediction by predicting samples of the sub-partition based on a reference line including a reference sample, and the reference line is selected from multiple reference lines.

[0008] According to a second aspect of at least one embodiment, a video encoding method includes: obtaining a video block and signaling information representing a coding tool and associated parameters, partitioning the obtained video block into sub-partitions, applying the coding tool on samples of the partitions, and decoding the block, wherein the coding tool applies an inverse transform to the sub-partition based on a set of transforms, or performs intra-frame prediction by applying a 2D matrix and a 1D vector to a reconstructed neighbor of the block, or performs intra-frame prediction by predicting samples of the sub-partition based on a reference line including a reference sample, and the reference line is selected from multiple reference lines.

[0009] According to a third aspect of at least one embodiment, a video encoding device includes an encoder configured to partition an obtained video block into sub-partitions; apply a coding tool on samples of the sub-partitions; and a coded block and signaling information, wherein the signaling information includes at least information representing the coding tool and associated parameters, wherein the coding tool applies a transform to the sub-partition based on a set of transforms, or performs intra-frame prediction by applying a 2D matrix and a 1D vector to a reconstructed neighbor of the block, or performs intra-frame prediction by predicting samples of the sub-partition based on a reference line including a reference sample, wherein the reference line is selected from multiple reference lines.

[0010] According to a fourth aspect of at least one embodiment, a video decoding device includes a decoder, which is configured to obtain a video block and signaling information representing a coding tool and associated parameters, divide the obtained video block into sub-partitions, apply an inverse coding tool on samples of the partitions, and decode the block, wherein the coding tool applies a transform to the sub-partition based on a set of transforms, or performs intra-frame prediction by applying a 2D matrix and a 1D vector to a reconstructed neighbor of the block, or performs intra-frame prediction by predicting samples of the sub-partition based on a reference line including a reference sample, and the reference line is selected from multiple reference lines.

[0011] According to a fifth aspect of at least one embodiment, a signal includes an encoded video block, the video block including information representing a coding tool and associated parameters, wherein the coding tool applies a transform to a subpartition based on a set of transforms, or performs intra-frame prediction by applying a 2D matrix and a 1D vector to a reconstructed neighbor of the block, or performs intra-frame prediction by predicting samples of the subpartition based on a reference line including reference samples, wherein the reference line is selected from multiple reference lines.

[0012] According to a sixth aspect of at least one embodiment, there is presented a computer program comprising program code instructions executable by a processor, the computer program implementing the steps of the method according to at least the first aspect or the second aspect.

[0013] According to a seventh aspect of at least one embodiment, a computer program product stored on a non-transitory computer readable medium and comprising program code instructions executable by a processor is presented, which implements the steps of the method according to at least the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A block diagram of an example of a video encoder 100 is illustrated.

[0015] Figure 2 A block diagram of an example of a video decoder 200 is illustrated.

[0016] Figure 3 A block diagram illustrating an example of a system in which various aspects and embodiments may be implemented is shown.

[0017] Figure 4 Examples of possible intra-frame sub-partitioning (ISP) modes are illustrated.

[0018] Figure 5 Examples of predefined transformation pairs used in ISP mode are illustrated.

[0019] Figure 6 An example of a parsing and decoding process for residual data of a CU of an intra-coding unit encoded in ISP mode is illustrated.

[0020] Figure 7 An exemplary implementation of the parsing and decoding process is illustrated, where the major transforms of a TU are explicitly signaled at the CU level.

[0021] Figure 8 An exemplary implementation of the parsing and decoding process is illustrated, where the major transforms of a TU are explicitly signaled at the TU level.

[0022] Fig.9A An example of an encoding process according to at least one embodiment is illustrated.

[0023] Fig. 9B An example of a decoding process according to at least one embodiment is illustrated.

[0024] Fig.10 A first example of a transformation process in accordance with at least one embodiment is illustrated.

[0025] Fig.11 A second example of a transformation process in accordance with at least one embodiment is illustrated.

[0026] Fig.12 and 13 An example of the overall process of matrix-based intra prediction is illustrated.

[0027] Fig.14 A first exemplary implementation of a coding unit syntax that implements a combination of MIP and ISP is illustrated.

[0028] Fig.15 A second exemplary implementation of a coding unit syntax that implements a combination of MIP and ISP is illustrated.

[0029] Fig.16 A third exemplary implementation of a coding unit syntax that implements a combination of MIP and ISP is illustrated.

[0030] Fig.17 A fourth exemplary implementation of a coding unit syntax that implements a combination of MIP and ISP is illustrated.

[0031] Fig.18 The transformation process for deriving the implicitMtsEnabled flag for an implementation using the second transformation selection method is illustrated.

[0032] Fig.19 An excerpt of a coding unit syntax for an implementation using the third transform selection method is illustrated.

[0033] Fig. 20 The transformation process for deriving the implicitMtsEnabled flag for an implementation using the fourth transformation selection method is illustrated.

[0034] Fig.21 The corresponding syntax changes for an implementation using a low-frequency non-separable transform are illustrated.

[0035] Fig. 22 An excerpt of the coding unit syntax for an implementation that removes the MIP transposition flag is illustrated.

[0036] Fig.23 An excerpt of a coding unit syntax of one embodiment is illustrated, where the ISP partition direction is derived when MIP is combined with ISP.

[0037] Fig.24 An excerpt of the coding unit syntax illustrating an implementation in which the MIP mode is implicitly selected if the MIP-ISP combination is used.

[0038] Fig.25A An exemplary implementation of an encoding process that implements a combination of a matrix-based intra prediction mode and an intra sub-partitioning mode is illustrated.

[0039] Fig.25B An exemplary implementation of a decoding process that implements a combination of a matrix-based intra prediction mode and an intra sub-partitioning mode is illustrated.

[0040] Fig.26 Examples of reference lines that can be used for multiple reference line intra prediction are illustrated.

[0041] Fig. 27 An example of a decoding process for multi-reference line intra prediction is illustrated.

[0042] Fig.28A An example of an encoding process according to at least one embodiment is illustrated.

[0043] Fig.28B An example of a decoding process according to at least one embodiment is illustrated.

[0044] Fig.29 An exemplary implementation of a coding unit grammar that implements a combination of MRL and ISP is illustrated.

[0045] Fig. 30A An exemplary implementation of a multi-reference line intra prediction system in a coding unit of intra subpartition encoding is illustrated.

[0046] Fig. 30B A second exemplary implementation of a multi-reference line intra prediction system in a coding unit of intra subpartition encoding is illustrated.

[0047] Fig.31

[0066] Example syntax for an implementation of using a multiple reference line intra prediction system in coding units encoded in intra subpartitions using matrix-based intra prediction is illustrated.

[0048] Fig.32 The derivation process of determining the implicitMTS variable according to the second method is illustrated.

[0049] Fig.33 An example of encoding according to the third method is illustrated.

[0050] Fig.34 The derivation process of determining the implicitMTS variable according to the third method is illustrated.

[0051] Fig.35 A first example of a transformation process in accordance with at least one embodiment is illustrated.

[0052] Fig.36 A second example of a transformation process in accordance with at least one embodiment is illustrated.

[0053] Fig.37A An example of an encoding process combining various embodiments is illustrated.

[0054] Fig.37B An example of a decoding process combining various embodiments is illustrated. DETAILED DESCRIPTION

[0055] Various embodiments are directed to using multiple transform selections for video encoding or decoding of intra-frame sub-block partitions. Various methods and other aspects described herein can be used to signal and select a transform to use based on various parameters.

[0056] Furthermore, although principles related to a particular draft of the VVC (e.g., Versatile Video Coding according to Draft 7) or HEVC (High Efficiency Video Coding) specification are described, aspects of the present invention are not limited to VVC or HEVC and may be applied, for example, to other standards and recommendations (whether pre-existing or developed in the future) and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise specified or technically excluded, the aspects described in this application may be used alone or in combination.

[0057] Figure 1 A block diagram illustrating an example of a video encoder 100 , such as a HEVC encoder, is illustrated. Figure 1 Encoders in which the HEVC standard is improved or which employ techniques similar to HEVC may also be shown, such as the JEM (Joint Exploration Model) encoder being developed by the JVET (Joint Video Exploration Team) for VVC according to draft 7, for example.

[0058] Before being encoded, a video sequence may undergo a pre-encoding process (101). This may be done, for example, by applying a color transform to an input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0059] In HEVC, to encode a video sequence having one or more pictures, the pictures are partitioned (102) into one or more slices, where each slice may include one or more slice segments. The slice segments are organized into coding units, prediction units, and transform units. The HEVC specification distinguishes between "blocks" and "units", where a "block" deals with a specific area in a sample array (e.g., luma, Y), and a "unit" includes all coded color components (Y, Cb, Cr, or monochrome), syntax elements, and a collocated block of prediction data (e.g., motion vectors) associated with the block.

[0060] For encoding in HEVC, a picture is partitioned into coding tree blocks (CTBs) of a square shape with a configurable size, and a continuous set of coding tree blocks is grouped into slices. A coding tree unit (CTU) contains the CTB of the coded color component. The CTB is the root of a quadtree partitioned into coding blocks (CBs), and a coding block can be partitioned into one or more prediction blocks (PBs) and forms the root of a quadtree partitioned into transform blocks (TBs). Corresponding to coding blocks, prediction blocks, and transform blocks, a coding unit (CU) includes a prediction unit (PU) and a transform unit (TU) of a tree structure set, the PU includes prediction information for all color components, and the TU includes a residual coding syntax structure for each color component. The size of the CB, PB, and TB of the luminance component is applicable to the corresponding CU, PU, ​​and TU. In the present application, the term "block" may be used to refer to any of, for example, CTU, CU, PU, ​​TU, CB, PB, and TB. In addition, "block" may also be used to refer to macroblocks and partitions specified in H.264 / AVC or other video coding standards, and more generally to data arrays of various sizes.

[0061] In the example of encoder 100, a picture is encoded by an encoder element as described below. The picture to be encoded is processed in units of CUs. Each CU is encoded using intra mode or inter mode. When a CU is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode is used to encode the CU, and indicates the intra / inter decision by a prediction mode flag. The prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0062] A CU in intra mode is predicted based on reconstructed neighboring samples within the same slice. A set of 35 intra prediction modes are available in HEVC, including DC, planar, and 33 angular prediction modes. The intra prediction reference is reconstructed based on rows and columns adjacent to the current block. The reference is extended horizontally and vertically by more than twice the block size using available samples from previously reconstructed blocks. When intra prediction is performed using angular prediction mode, reference samples may be copied along the direction indicated by the angular prediction mode.

[0063] The applicable luma intra prediction mode for the current block can be encoded using two different options. If the applicable mode is contained in a built list of six most probable modes (MPMs), the mode is signaled by an index in the MPM list. Otherwise, the mode is signaled by a fixed-length binarization of the mode index. The six most probable modes are derived from the intra prediction modes of the top and left neighboring blocks (see Table 1 below).

[0064]

[0065] Table 1

[0066] For an inter CU, motion information (eg, motion vector and reference picture index) may be signaled in a variety of methods, such as “merge mode” or “advanced motion vector prediction (AMVP)”.

[0067] In merge mode, the video encoder or decoder assembles a candidate list based on already coded blocks, and the video encoder signals an index for one of the candidates in the candidate list. At the decoder side, the motion vector (MV) and reference picture index are reconstructed based on the signaled candidate.

[0068] In AMVP, a video encoder or decoder assembles a candidate list based on motion vectors determined from already coded blocks. The video encoder then signals an index into the candidate list to identify the motion vector predictor (MVP) and signals the motion vector difference (MVD). At the decoder end, the motion vector (MV) is reconstructed as MVP+MVD. The applicable reference picture index is also explicitly coded in the CU syntax for AMVP.

[0069] For each coding unit, the video encoder performs a rate-distortion optimization step that exhaustively measures the coding performance of different modes and selects one of the modes (usually the mode that provides the best coding performance for the coding unit).

[0070] The prediction residual is then transformed (125) and quantized (130), including at least one implementation for adjusting the chroma quantization parameters described below. The transform is typically based on a separable transform. For example, a DCT transform is first applied in the horizontal direction and then in the vertical direction. In recent codecs such as JEM, the transforms used in the two directions can be different (e.g., DCT in one direction, DST in the other), which results in a variety of 2D transforms, whereas in previous codecs, the variety of 2D transforms was typically limited to a given block size.

[0071] The quantized transform coefficients are entropy encoded (145) along with motion vectors and other syntax elements to output a bitstream. The encoder may also skip the transform and apply quantization directly to the untransformed residual signal based on the 4x4 TU. The encoder may also bypass both the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process. In direct PCM encoding, prediction is not applied and the coding unit samples are encoded directly into the bitstream.

[0072] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. A loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0073] Figure 2 1 illustrates a block diagram of an example of a video decoder 200, such as a HEVC decoder. In the example of the decoder 200, a bitstream is decoded by decoder elements, as described below. The video decoder 200 generally performs the same operations as described above. Figure 1 The encoding pass described is the reciprocal of the decoding pass, which performs video decoding as part of encoding the video data. Figure 2 It is also possible to show decoders in which the HEVC standard is improved or decoders that employ techniques similar to HEVC, such as the JEM decoder.

[0074] Specifically, the input to the decoder includes a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, picture partition information and other coded information. The picture partition information indicates the size of the CTU and the manner in which the CTU is partitioned into CUs (and possibly PUs when applicable). Thus, the decoder may partition (235) the picture into CTUs and each CTU into CUs based on the decoded picture partition information. The transform coefficients are dequantized (240), including at least one implementation for adjusting the chroma quantization parameters described below, and an inverse transform (250) is performed to decode the prediction residual.

[0075] The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. The prediction block may be obtained (270) from intra prediction (260) or motion compensated prediction (i.e., inter prediction) (275). As described above, AMVP and merge mode techniques may be used to derive motion compensated motion vectors that may use an interpolation filter to calculate interpolated values ​​of sub-integer samples of a reference block. A loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0076] The decoded picture may also undergo post-decoding processing (285), such as an inverse color transform (e.g., a transform from YCbCr 4:2:0 to RGB 4:4:4) or performing an inverse remapping of the remapping process performed in the pre-encoding process (101). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0077] Figure 3Illustrated is a block diagram of an example of a system in which various aspects and embodiments are implemented. System 300 may be embodied as a device including various components described below, and is configured to perform one or more aspects described in this patent application. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, encoders, transcoders and servers. The elements of system 300 may be embodied in a single integrated circuit, multiple ICs and / or discrete components individually or in combination. For example, in at least one embodiment, the processing elements and encoder / decoder elements of system 300 are distributed over multiple ICs and / or discrete components. In various embodiments, the elements of system 300 are coupled communicatively via an internal bus 310. In various embodiments, system 300 is coupled communicatively to other similar systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port. In various embodiments, system 300 is configured to implement one or more aspects of the aspects described in this document, such as the above-mentioned video encoder 100 and video decoder 200 and modified as described below.

[0078] The system 300 includes at least one processor 301 configured to execute instructions loaded therein for implementing, for example, various aspects described in this document. The processor 301 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 300 includes at least one memory 302 (e.g., a volatile memory device and / or a non-volatile memory device). The system 300 includes a storage device 304, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, magnetic disk drive, and / or optical disk drive. As non-limiting examples, the storage device 304 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0079] The system 300 includes an encoder / decoder module 303, which is configured to process data, for example, to provide encoded video or decoded video, and the encoder / decoder module 303 may include its own processor and memory. The encoder / decoder module 303 represents a module that can be included in a device to perform encoding and / or decoding functions. As is well known, a device may include one or both of an encoding module and a decoding module. In addition, the encoder / decoder module 303 may be implemented as an independent element of the system 300, or may be combined in the processor 301 as a combination of hardware and software known to those skilled in the art.

[0080] Program code to be loaded onto the processor 301 or encoder / decoder 303 to perform various aspects described in this document may be stored in the storage device 304 and subsequently loaded onto the memory 302 for execution by the processor 301. According to various embodiments, one or more of the processor 301, memory 302, storage device 304, and encoder / decoder module 303 may store one or more of the various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or partially decoded video, bitstreams, matrices, variables, and intermediate or final results of processing equations, formulas, operations, and operation logic.

[0081] In several embodiments, memory inside the processor 301 and / or the encoder / decoder module 303 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device can be the processor 301 or the encoder / decoder module 303) is used for one or more of these functions. The external memory can be a memory 302 and / or a storage device 304, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2, HEVC or VVC.

[0082] Input to the elements of system 300 may be provided through various input devices as shown in block 309. Such input devices include, but are not limited to: (i) an RF portion that receives an RF signal transmitted over the air, such as by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0083] In various embodiments, the input device of block 309 has associated corresponding input processing elements as known in the art. For example, the RF section may be associated with the following required elements: (i) selecting the desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired packet stream. The RF section of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or near-baseband frequency) or to baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal transmitted by wired (for example, cable) medium, and filter to required frequency band again by filtering, down-conversion and perform frequency selection.Various embodiments rearrange the order of above-mentioned (and other) elements, remove some elements in these elements, and / or add other elements of similar or different functions.Adding element can include inserting element between existing element, for example, inserting amplifier and analog-to-digital converter.In various embodiments, the RF part comprises antenna.

[0084] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 300 to other electronic devices across the USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, in a separate input processing IC or in the processor 301. Similarly, aspects of USB or HDMI interface processing may be implemented as needed, in a separate interface IC or in the processor 301. The demodulated stream, error correction stream, and demultiplexed stream are provided to various processing elements, including, for example, the processor 301 and the encoder / decoder 303, which operate in conjunction with the memory and storage elements to process the data stream as needed for presentation on the output device.

[0085] The various elements of system 300 may be disposed within an integrated housing. Within the integrated housing, the various elements may interconnect and transmit data therebetween using a suitable connection arrangement (eg, an internal bus as known in the art, including an Inter-IC (I2C) bus, wiring, and printed circuit boards).

[0086] The system 300 includes a communication interface 305 capable of communicating with other devices via a communication channel 320. The communication interface 305 may include, but is not limited to, a transceiver configured to transmit and receive data through the communication channel 320. The communication interface 305 may include, but is not limited to, a modem or a network card, and the communication channel 320 may be implemented, for example, within a wired and / or wireless medium.

[0087] In various embodiments, data is streamed to the system 300 using a Wi-Fi network such as IEEE 802.11. The Wi-Fi signals of these embodiments are received via a communication channel 320 and a communication interface 305 suitable for Wi-Fi communication. The communication channel 320 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet to allow streaming applications and other cloud communications. Other embodiments provide streaming data to the system 300 using a set-top box that delivers data via an HDMI connection of an input block 309. Still other embodiments provide streaming data to the system 300 using an RF connection of an input block 309.

[0088] The system 300 can provide output signals to various output devices, including a display 330, a speaker 340, and other peripherals 350. In various examples of embodiments, the other peripherals 350 include one or more of the following: a stand-alone DVR, a disk player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system 300. In various embodiments, control signals are transmitted between the system 300 and the display 330, the speaker 340, or other peripherals 350 using signaling (such as AV.Link, CEC, or other communication protocols) that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 300 via dedicated connections through respective interfaces 306, 307, and 308. Alternatively, the output devices can be connected to the system 300 via a communication interface 305 using a communication channel 320. The display 330 and the speaker 340 can be integrated in a single unit with other components of the system 300 in an electronic device (such as, for example, a television). In various implementations, the display interface 306 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0089] Alternatively, if the RF portion of input 309 is part of a separate set-top box, the display 330 and the speaker 340 can be separated from one or more components in other components. In various embodiments where the display 330 and the speaker 340 are external components, the output signal can be provided via a dedicated output connection (including, for example, an HDMI port, a USB port, or a COMP output). The specific implementation described herein can be implemented in, for example, a method or process, a device, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of specific implementation (for example, discussed only as a method), the specific implementation of the discussed features can also be implemented in other forms (for example, a device or a program). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device (such as, for example, a processor) that generally refers to a processing device, which includes, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as, for example, a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0090] Figure 4 An example of intra sub-partitioning (ISP) mode is illustrated. ISP mode is used for intra prediction and consists of splitting a coding unit into 2 or 4 transform units of the same size. The splitting can be done horizontally or vertically. The splitting depends on the block size, as shown in Table 2. Basically, a 4×4 block cannot be split. A CU of size 4×8 or 8×4 is split into 2 TUs. Other CUs are split into 4 TUs.

[0091]

[0092] Table 2

[0093] Within a CU encoded using ISP mode, TUs are decoded sequentially and intra prediction is performed from TU to TU using the same intra prediction mode, which is signaled at the CU level. Finally, the residual coding is also adjusted according to the intra sub-partition size. In practice, the size of the sub-partition can be 1×N, N×1, 2×N, or N×2, and coding groups of size 1×16, 16×1, 2×8, or 8×2 are used in these corresponding cases.

[0094] The ISP encoding mode can be signaled, for example, by two consecutive flags:

[0095] - intra_subpartitions_mode_flag indicates that the given intra CU uses ISP mode. A value of 0 indicates that the CU is not partitioned, and a value of 1 indicates that the CU is partitioned.

[0096] - If intra_subpartitions_mode_flag is 1, intra_subpartitions_split_flag indicates the orientation of the partition into subpartitions, where 0 indicates horizontal partition and 1 indicates vertical partition. In this case, the partition direction cannot be directly inferred from the CU size.

[0097] Multiple transform selection (MTS) uses 2D transform pairs to be applied to transform units and can be signaled in explicit MTS or implicit MTS mode. With explicit MTS, the 2D transform pairs to be used for the considered block are signaled on the block level. With implicit MTS, the 2D transform pairs to be applied to the considered block are signaled based on some given available information available at the stage where the considered block is decoded, e.g. Figure 5 For the block sizes shown, a 2D transform pair is derived for the block under consideration.

[0098] Figure 5 An example of a predefined transform pair used in ISP mode is illustrated. In fact, when ISP is used for a given intra-frame CU, a predefined transform pair is used to encode and decode each TU in the (2 or 4) TUs contained in the CU. The transform pair is selected as a combination between DCT2 and DST7 and applied horizontally or vertically according to the size of the transform block. This results in a set of 4 different pairs, which are different combinations between DCT2 and DST7. The DST7 horizontal transform is used when the TU width is between 4 and 16, otherwise DCT2 is used as a horizontal transform. Similarly, a vertical DST7 transform is used when the TU height is between 4 and 16, otherwise DCT2 is used. In other cases, DST7×DCT2 or DCT2×DST7 2D separable main transforms are used as shown.

[0099] Figure 6 The parsing and decoding process of residual data for a CU of an intra coding unit encoded in ISP mode is illustrated. The process comprises entropy decoding the transform coefficients contained in each TU of the CU in step 605, followed by parsing the LNST index associated with the ISP CU when applied in step 620. In steps 650 to 680, the parsing phase is followed by a decoding phase that involves a loop over each TU transform unit. For each TU, an inverse LFNST is applied to the TU in step 655, where the LFNST index was previously decoded at the CU level (in steps 615 or 620). Next, in step 660, the inverse LFNST is applied to the TU according to the method described in the previous paragraph and in step 670. Figure 5 The implicit MTS mechanism used in the ISP coding unit shown in Figure 6 is used to select the 2D main transform for the current TU. Next, in step 670, the derived 2D main inverse transform is applied to provide a residual block associated with the considered TU. When the loop on the TU is completed, Figure 6 The process ends (test in step 680).

[0100] Fig.12 and 13 An example of the overall process of matrix-based intra prediction is illustrated. Matrix-based intra prediction (MIP) is an intra prediction mode that includes prediction of a block encoded based on columns of reconstructed neighbors, such as reference samples to the left of the block (701) and a line of reconstructed reference samples at the top of the current block (702). The basic principle of MIP is to predict the current block by placing the top and left references into a vector. This vector (703) in a sub-sampled version is then multiplied by a 2D matrix and a 1D vector (704) is added, providing a sub-sampled version of the desired intra-predicted block (705), which is up-sampled to generate the final intra-predicted block (706).

[0101] In more detail, the following 3 steps can be used to perform Fig.12 and Fig.13 The MIP described in:

[0102] - Step 1: Average the reference samples. This first step is performed in order to reduce the number of coefficients to be multiplied. The main motivation for doing this is to reduce the storage and computation requirements by reducing the number of multiplications to be performed and the memory required to store all the matrices A_k and vectors v_k. The averaging step is performed as follows. A certain set of consecutive reference samples on the top and left of the current block are averaged, resulting in a total of 4 averaged reference samples in the case of a 4×4 current block, or a total of 8 averaged samples in the case of other block sizes. The two simplified vectors are averaged. and , which are concatenated together to form a 1D vector , if it is a 4×4 CU, it consists of 4 elements, otherwise it consists of 8 elements.

[0103] -Step 2: Matrix-vector multiplication. In this operation, the simplified reference sample vector A low-resolution version of the preselected block matrix is ​​generated by multiplying it with a specific matrix depending on the signaled intra mode. Thus, the low-resolution prediction signal is obtained as follows :

[0104]

[0105] Low-resolution prediction signal The 2D size in width and height is equal to and ,in:

[0106]

[0107]

[0108] Where (W,H) is the size of the processed CU. If W=H=4, then the matrix A has rows and 4 columns, otherwise it has 8 columns. The matrix A and vector b are taken from the set ,in

[0109]

[0110] gather Composed of 18 matrices Each matrix has 16 rows and 4 columns and consists of 18 offset vectors Each offset vector is of size 16. This set of matrices and offset vectors is used for blocks of size 4×4. From 10 matrices Each matrix has rows and 8 columns, and consists of 10 offset vectors Each offset vector is of size 16. This set of matrices and offset vectors can be used to generate , and Finally, the set The 6 matrices Each matrix has 64 rows and 8 columns and consists of 6 offset vectors of size 64. The matrices and offset vectors of this set or parts of these matrices and offset vectors can be used for all other block shapes.

[0111] - Step 3: Interpolation: In this last operation, an interpolation operation is performed to expand the prediction signal to fill the prediction buffer. It simply consists of linear interpolation in vertical and horizontal directions between the sub-sampled prediction samples to generate the remaining samples and provide the full resolution prediction block.

[0112] MIP mode Fig.12 and Fig.13 The above is a pattern K , and signaled as follows. For each intra CU, a flag indicates whether MIP is used for intra prediction of the considered CU. If the MIP mode is to be applied, the MIP mode of the considered CU is encoded using a truncated binary code.

[0113] The number of supported MIP modes depends on the block size. In one approach, 35 modes are available for blocks with max(W,H)<=8&&W*H<32. And 19 and 11 modes are used for max(W,H)=8 and max(W,H)>8 respectively. In addition, both modes share the same matrix and offset vector to reduce memory requirements, as shown below:

[0114]

[0115] Similar to the conventional intra prediction mode, MIP prediction residuals are allowed to adopt multiple transform selection (MTS) and low-frequency non-separable transform (LFNST). However, the combination of MIP and ISP is not allowed. In other words, if the CU is partitioned with ISP, only conventional prediction is allowed and MIP cannot be used.

[0116] Fig.26 An example of reference lines that can be used for multiple reference line intra prediction is illustrated. For this prediction mode, the angular intra prediction of the luma block is based on the reconstructed reference samples belonging to a reference line and column above and to the left of the current CU selected from a set of reference lines, respectively. Each reference line can be identified by its index as 0, 1, 2, and 3, as shown in Fig.26 In one example method, only three reference lines are used: lines 0, 1, and 3. The reference lines used for intra prediction can be specified by the syntax element ' intra_luma_ref_idx' Signaled in the bitstream.' intra_luma_ref_idx' The marker can be encoded as follows. It can be binarized as a truncated Rice bin string, which means that it is encoded by a series of regular CABAC bins equal to 1, ending with a regular CABAC bin equal to 0. In total, up to 3 bins can be signaled. In one example method, since only three reference lines are used, up to two bins are signaled.

[0117] Fig. 27 An example of the decoding process for multiple reference line intra prediction is illustrated. The figure shows the general process involving up to 4 reference lines. lineIDx[.] It consists of 4 line reference indexes, the value ' MaxNumRefLines' Indicates the maximum number of reference lines allowed for intra prediction. In one example approach, this is equal to 3, and the array lineIdx It is composed of the following elements:

[0118] lineIdx={0,1,3}

[0119] Syntax elements' intra_luma_ref_idx' The decoding process of is as follows. The output of the process is multiRefIdxThe decoded value of indicates which reference line should be used. First, in step 810, the output value MultiRefIdx is initialized to 0. Next, in step 820, if more than 1 reference line is not allowed, the process ends and returns 0 as the reference line to be used. In step 830, a CABAC regular bin is parsed using a single CABAC context with index 0. In step 835, if bin is equal to 1, multiRefIdx is set equal to lineIdx[1]. Otherwise, the multiRefIdx value is unchanged and the process will terminate in step 840. In addition, in step 840, if MaxNumRefLines is not strictly higher than 2, the process terminates. In addition, in step 850, a second regular CABAC bin is decoded using a single context with index 1. In step 855, if bin is equal to 1, multiRefIdx is set equal to lineIdx[2]. Otherwise, the multiRefIdx value is unchanged and the process will terminate in step 860. Otherwise, in step 860, if MaxNumRefLines is not strictly above 3, the process terminates. Otherwise, in step 870, the third regular CABAC bin is decoded using a single context with index 2. In step 875, if bin is equal to 1, multiRefIdx is set equal to lineIdx[3]. Otherwise, the MultiRefIdx value remains unchanged and holds the value lineIdx[2].

[0120] This document describes several embodiments conceived with the foregoing in mind.

[0121] The various methods and other aspects described herein may be implemented by a video encoder 100 ( Figure 1 ) or video decoder 200 ( Figure 2 ) device 300( Figure 3 ) is executed by the processor 301. Therefore, Figure 1 Encoder 100, Figure 2 The decoder 200 and Figure 3 The system 1000 is configured to implement at least one of the embodiments described below. In other words, its encoding and decoding processes are configured to process the new syntax described below. The encoder generates a bit stream signal that conforms to the embodiments described below.

[0122] The following embodiments relate to selecting transform pairs for TUs in a CU encoded in ISP mode and corresponding signaling in the bitstream. Figure 5 Such an implementation allows improving the compression efficiency of the video codec by providing the ability to have different transforms on TUs, compared to the case of TUs.

[0123] In at least one first embodiment, the primary transforms of all TUs of an intra CU encoded in ISP mode are explicitly signaled. Such signaling is done at the CU level and uses the MTS index mts_idx , the MTS index is used to encode a unit that is not encoded in ISP mode. The same conditions for encoding the MTS index as for encoding a coding unit encoded in another mode are applied to the coding unit encoded in ISP mode.

[0124] In particular, for non-ISP intra CUs, the MTS index mts_idx The signaling of depends on the position of the last significant quantized coefficient contained in the considered coding unit: if the last position is at least 16 in x or y coordinate in the luma CB of the CU, the MTS index of the CU is inferred to be zero, which means that DCT2_DCT2 is used as the primary 2D transform for each TU of the CU. If the MTS index is equal to zero, DCT2_DCT2 is used as the primary 2D transform for each TU of the CU. mts_idx If the 2D transform corresponding to the MTS index is different from 0, then the 2D transform corresponding to the MTS index is used for each TU in the luma component of the considered CU. This includes 2 or 4 luma transform blocks contained in the considered CU. This first implementation is as follows Figure 7 shown.

[0125] Figure 7 An exemplary implementation of the parsing and decoding process is illustrated, in which the major transforms of the TU are explicitly signaled at the CU level. This decoding process is Figure 1 Encoder 100, Figure 2 The decoder 200 and Figure 3 According to the proposed embodiment, the decoding process 700 of the intra CU in the ISP mode includes parsing the MTS index in step 726 if MTS is allowed for the current CU. mts_idx The latter condition is tested in step 725. According to the first embodiment, the MTS index of the intra CU is parsed in ISP mode mts_idx The conditions can be, for example, as follows:

[0126] - the intra CU is not in the chroma component of a slice coded in dual-tree mode,

[0127] - The CU size width and height in the considered component is at most 32,

[0128] - the LFNST index associated with the current intra ISP CU is equal to zero,

[0129] - the tu_cbf_luma value associated with the coding unit is not equal to zero, meaning that at least one luma TB in the CU has a non-zero residual, or

[0130] - For at least one TU in the CU, the condition of the last significant position being at most equal to 15 in x and y coordinates is met.

[0131] When parsing is complete, the MTS index identifies the transform pair to be used for all TUs of the current CU in step 770. The other elements of process 700 are similar to Figure 6 The first embodiment can be implemented by the coding unit syntax shown in Table 3.

[0132]

[0133] Table 3

[0134] This syntax authorizes encoding of explicit MTS and insertion of MTS indexes mts_idx , independent of the intra-frame sub-partition flag ISP_NO_SPLIT Therefore, for CUs using ISP mode, there is also explicit MTS encoding.

[0135] In variant implementations of the first implementation, different conditions need to be met so that explicit MTS can be used for all TUs of a CU. Except when incompatible, these variants can be combined to generate other variants including more or fewer conditions to implement implicit MTS.

[0136] In a variation of the first implementation, the condition for using explicit MTS is related to the last significant coefficient position in the luma TB in the considered CU, which must be at most equal to 15 in x and y coordinates for each TU in the CU. This is more in line with the overall coding strategy for non-ISP intra CUs.

[0137] In another variation of the first embodiment, the condition for using explicit MTS is to simply ignore the position of the last significant coefficient. This allows for greater freedom and thus provides potentially better coding performance. In an exemplary embodiment, this flag is referred to as MtsZeroOutSigCoeffFlag , and the corresponding syntax is shown in Table 4.

[0138]

[0139] Table 4

[0140] In several variations of the first embodiment, the conditions for using explicit MTS depend on rules related to the block size of the TUs resulting from ISP segmentation, for example:

[0141] The width and height of TU are no greater than 32.

[0142] The width and height of TU are no greater than 16.

[0143] The TU size must be at least 4 in width and height.

[0144] The TU size must be at least 4 in width or height.

[0145] The TU size must be at least 8 in width and height.

[0146] The TU size must be at least 8 in width or height.

[0147] According to a variant implementation, if a combination of MTS / ISP is allowed for certain TU sizes, then for other TU sizes that can be generated by ISP segmentation, the ISP coding unit can be used. Figure 5 Implicit MTS mechanism.

[0148] In at least one second embodiment, the primary transform of all TUs of an intra CU encoded in ISP mode is explicitly signaled for each TU. It uses the MTS index mts_idx , which is the MTS index used to encode units that are not encoded in ISP mode. Such signaling is done at the TU level, so one MTS index is signaled for each TU mts_idx The same MTS index encoding rules for coding units coded in ISP mode as for coding units coded in another mode but at TU level are applied. mts_idx conditions.

[0149] Figure 8 An exemplary implementation of the parsing and decoding process is illustrated, in which the main transform of the TU is explicitly signaled at the TU level. In steps 850 to 880, the loop over the TUs contained in the CU includes parsing the MTS index for each TU in step 851. Figure 7 Compared to process 700 of , the other elements of process 800 are unchanged. With this second embodiment, different main inverse transforms can be used for different TUs of the CU. This improves coding efficiency compared to conventional methods, but also compared to the first embodiment. In fact, some flexibility is introduced when assigning the MTS index to each TU within a coding unit encoded in ISP mode. Therefore, the encoder has more freedom for rate-distortion optimization, where it determines the optimal coding mode for intra-CUs in ISP mode. Note that at TU level, the MTS index is signaled only when the secondary transform is not used (if the CU-level LFNST index is equal to zero). Otherwise, the DCT2_DCT2 main inverse transform is used for all TUs.

[0150] ISP may result in smaller TU sizes, such as as small as 2×N, N×2, N×1, and 1×N. However, the syntax of some conventional video coding systems does not support MTS transforms (DST7, DCT8) for blocks smaller than 4×4, and only supports DCT2 transforms for TU sizes up to 2×2. Therefore, for smaller TUs of the above sizes, compliant decoders will not know how to interpret the MTS index. mts-idx This can be solved by at least one of the following techniques:

[0151] - explicit MTS for TU sizes of 2×N, N×2, N×1, and 1×N are not allowed,

[0152] - Allows explicit MTS for TU sizes of 2×N and N×2 using DCT2 and DST7 / DCT8 combinations,

[0153] - Allow explicit MTS for TU sizes of 1×N and N×1 DST7 / DCT8,

[0154] - Added 2X2 DST7 and DCT 8.

[0155] The first technique is to prevent explicit MTS for TU sizes of 2×N, N×2, N×1, and 1×N. This can be done by adding additional size constraints to allow insertion of the MTS index mts_idx In the exemplary embodiment shown in Table 5, the coding unit syntax allows implementation of this first technique in the example of syntax applied to VVC by requiring a minimum size greater than or equal to 4 to enable explicit MTS.

[0156]

[0157] Table 5

[0158] The second technique is to allow explicit MTS for TU sizes of 2×N and N×2 with a combination of DCT2 and DST7 / DCT8. Using this technique, longer sizes can use DST7 or DCT8, while shorter sizes use DCT2. This is because DCT2 of 2×2 size and DST7 / DCT8 of 4×4 and larger sizes are traditionally supported. In an example application, the following syntax shown in Table 6 allows the implementation of this first technique in an example applied to VVC by requiring a minimum size greater than or equal to 2 to enable explicit MTS.

[0159]

[0160] Table 6

[0161] In other words, MTS is allowed only when the TU height or width is at least 2. It should be noted that the TU obtained from the ISP has at least 16 coefficients. That is, 2×N and N×2 TUs have N>=8. Therefore, there is no need to add another condition to check whether N is greater than or equal to 4 to enable MTS.

[0162] However, the interpretation of the MTS index needs to be adjusted. mts_idx The conventional interpretation of is shown in Table 7, where TrTypeHor and TrTypeVer are the transform types for the horizontal and vertical directions. In this table, DCT2, DST7, and DCT8 are represented by transform type values ​​0, 1, and 2, respectively.

[0163]

[0164] Table 7

[0165] In the 2×N case, the MTS index mts_idx The explanation is shown in Table 8. In practice, since the vertical dimension is 2, only DCT2 is allowed and should be supported.

[0166]

[0167] Table 8

[0168] Similarly, in the N×2 case, the MTS index mts_idx The explanation is shown in Table 9.

[0169]

[0170] Table 9

[0171] The third technique is to allow explicit MTS for TU sizes of 1×N and N×1 DST7 / DCT8. In both cases, the 1D transform is applied only to the longer size, where N>=16 to have at least 16 coefficients. At this size, DCT2 is allowed (if the MTS index is zero), otherwise DST7 or DCT8 is allowed. The corresponding specification changes are the same as in the first implementation. However, the MTS index is mts_idx The interpretation of must be further adjusted by adding Tables 10 and 11 for the cases with sizes 1×N and N×1 to Tables 8 and 9, respectively.

[0172]

[0173] Table 10

[0174]

[0175] Table 11

[0176] The fourth technique is to add 2×2 DST7 and DCT8. To allow all combinations of MTS up to 2×N and N×2, 2×2 DST7 and DCT8 need to be added. This requires 8 bytes because each transform requires 4 elements (2×2 bytes). By doing this, N×2 and 2×N can use any transform combination {DCT2,DCT2}, {DST7,DST7}, {DST7,DCT8}, {DCT8,DST7}, or {DCT8,DCT8}.

[0177] Regarding the high-level syntax, MTS can be controlled by 3 flags present in the Sequence Parameter Set (SPS): sps_mts_ enabled_flag , sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_ enabled_flag . When set to zero, sps_mts_enabled_flag Set all transforms to DCT2, otherwise non-DCT2 transforms (DST7 and / or DCT8) can be used. sps_mts_enabled_flag When 1, the following configuration is defined:

[0178] 1-When sps_explicit_mts_intra_enabled_flag When equal to zero: use implicit transform selection for all intra blocks (including ISP blocks) and sub-block transform (SBT) blocks, where SBT is a transform tiling tool for inter coding units.

[0179] 2-When sps_explicit_mts_intra_enabled_flag When equal to 1: use explicit transform selection for all intra blocks, while SBT and ISP use implicit transform selection,

[0180] 3- When sps_explicit_mts_inter_enabled_flag When equal to zero: DCT2 is used for all inter-frame blocks (excluding SBT),

[0181] 4-When sps_explicit_mts_inter_enabled_flag When equal to 1: use explicit transform selection for all inter blocks (excluding SBT).

[0182] Fig.9A An example of an encoding process according to at least one embodiment is illustrated. In this example, the encoding process encodes selected transform pairs of TUs in a CU encoded in ISP mode. In step 910, a video block is obtained. In step 920, the block is divided into sub-partitions. This can be performed using the above-mentioned intra-frame sub-partition mode. Then, in step 930, for each partition in the partition, the samples of each sub-partition are transformed using a selected set of transforms. This can be performed using the above-mentioned set of transforms. In step 940, the block is then encoded as described above.

[0183] Fig. 9B An example of a decoding process according to at least one embodiment is illustrated. In this exemplary embodiment, the decoding process decodes a CU encoded in ISP mode, where a set of transforms is used to transform the sub-partitions. In step 950, a coded block and related signaling information are obtained, and the related signaling information includes information representing a set of transforms used for encoding. In step 960, the block is divided into sub-partitions. This can be performed using the above-mentioned intra-frame sub-partition mode. Then, in step 970, an inverse transform using the transform set used for encoding is applied to the samples of the sub-partitions. Then, in step 980, the block is decoded as described above.

[0184] Fig.10 A first example of a transformation process according to at least one embodiment is illustrated. In this example, the embodiment is applied to a transformation process of a VVC. sps_explicit_mts_intra_enabled_flag=1 When , explicit MTS for ISP is enabled. Otherwise, when this flag is set to zero, the default implicit selection is used or DCT2 is used.

[0185] Fig.11 A second example of a transformation process according to at least one embodiment is illustrated. In this example, the embodiment is applied to the transformation process of VVC. An additional flag selected for enabling explicit transformation for ISP is used. That is, for example, in the encoding of SPS, sps_explicit_mts_isp_enabled_flag , as shown in Table 12. Added SPS level marker sps_explicit_mts_isp_enabled_flag The meaning of is as follows: If it is equal to 1, explicit MTS signaling is allowed in CUs coded in ISP mode. Otherwise, if it is equal to zero, explicit signaling of mts_idx for intra CUs coded in ISP mode is disabled.

[0186]

[0187] Table 12

[0188] However, explicit signaling of MTS comes at a cost. In this paper, different approaches are proposed to limit the excess cost.

[0189] A first approach to limiting the excess cost of explicit signaling of the MTS is to limit the MTS index to values ​​less than 4. In practice, the current MTS index can be as high as 4 to cover all 4 combinations of DST7 and DCT8 as well as DCT2. However, it seems that not all combinations are actually needed. The maximum index can be set to 3, 2, or 1 to reduce the number of available transform combinations and thus reduce the signaling excess cost. The transform selection remains the same as in Table 13.

[0190]

[0191] Table 13

[0192] For example, if the maximum index is 1, the ISP transform selection can be switched between DCT2, DCT2 and DST7, DST7. In such a case, the above Table 13 can take the following shortened form of Table 14.

[0193]

[0194] Table 14

[0195] According to a variation of this embodiment, the implicit MTS mechanism associated with the ISP can be included as a specific transformation type in the explicit MTS system proposed for the ISP. Figure 5 The implicit MTS mode of the ISP described in the table is associated with a specific transform index value. This usually results in, for example, Figure 5 Table 15 shows the correspondence between the MTS index and the transform type for the CU, where the mts_idx value 2 indicates implicit selection.

[0196]

[0197] Table 15

[0198] According to a variant of this embodiment, an implicit MTS mechanism associated with the ISP may be associated with a value of mts_idx equal to 1. In this variant, DCT2, DCT2 transform pairs or implicit transform types derived according to Table 5 may be used. This typically results in, for example, Figure 5 Table 16 shows the correspondence between the MTS index and the transform type for the CU, where the mts_idx value 1 indicates implicit selection.

[0199]

[0200] Table 16

[0201] Finally, according to another variant, the activation of one of the above variants may depend on the block size, typically the TU size resulting from the CU partitioning according to the ISP sub-partitioning.

[0202] The second method to limit the excess cost of explicit signaling of MTS is to distribute the MTS indexes between intra prediction modes. In this method, transform pairs are distributed between prediction modes to reduce the length of the MTS index. Specifically, odd intra prediction modes can use MTS indexes 0, 1, and 2, while even modes use 3 and 4 (and vice versa). In addition, if the maximum index is 3, odd modes can use MTS indexes of 0 and 1, while even modes use MTS indexes of 2 and 3, thereby reducing the cost of signaling.

[0203] The following embodiments relate to the use of matrix-based intra prediction for blocks encoded using intra sub-partitioning mode. This allows for improved compression efficiency of video codecs. In other words, it allows encoding, signaling and decoding of blocks for which intra prediction divides the block into multiple sub-blocks, and wherein prediction of the sub-blocks uses matrix-based intra prediction, where a 2D matrix and a 1D vector are applied to the vector based on a column of reconstructed reference samples on the left side of the block and a line of reconstructed reference samples at the top of the current block. The following embodiments describe the syntax and process required to enable a combination of matrix-based intra prediction mode and intra sub-partitioning mode for a given block when using VVC Draft 7. Other video coding systems will use another syntax to combine matrix-based intra prediction with sub-partitioning of blocks into transform units.

[0204] Fig.14 A first exemplary implementation of a coding unit syntax that implements a combination of MIP and ISP is illustrated. In this context, independent of intra_mip_flag Signaling intra_subpartitions_mode_flag and intra_ subpartition_split_flag A combination of MIP and ISP is allowed. In fact, regardless of the intra_mip_ flag What is the value of the syntax element, in section 980, signals the use of the ISP, thereby allowing a combination of MIP and ISP for a given intra-coding unit.

[0205] Fig.15 A second exemplary implementation of a coding unit syntax that enables a combination of MIP and ISP is illustrated. In this context, after the MRL and before the ISP-related elements in section 1080, the MIP-related elements in section 1040 are signaled to allow a combination of MIP and ISP.

[0206] Fig.16A third exemplary implementation of a coding unit syntax that implements a combination of MIP and ISP is illustrated. In this implementation, the ISP signaling in section 1180 is inserted before the MRL and MIP are signaled in section 1140. In section 1110, if ISP is not applied, the MRL index is encoded. This is to keep the current design as simple as possible without allowing a combination of MRL and ISP. However, more complex encoders may allow this type of combination, such as Fig.17 shown.

[0207] Fig.17 A fourth exemplary implementation of a coding unit syntax that implements a combination of MIP and ISP is illustrated. In this implementation, ISP signaling in section 1280 is inserted before signaling MRL and MIP in section 1240.

[0208] The combination of MIP and ISP further affects the choice of transform to be used. In fact, on the one hand, in the current ISP design, the TU transform pairs generated by ISP partitioning implicitly choose between DST7 and DCT2, as mentioned before, and as Figure 5 As shown. On the other hand, explicit transform selection is allowed for the MIP prediction residual. That is, DCT2 transform or a combination of DST7 and DCT8 transforms can be used as normal intra-frame prediction residuals. Therefore, when combining MIP and ISP, a transform selection process needs to be defined. To this end, at least one of the following methods can be used:

[0209] -Method 1: Implicit transformation of ISP selection

[0210] -Method 2: MIP-ISP residual implicitly selects DCT2 transform pairs

[0211] -Method 3: MIP-ISP residual explicitly selects the transformation pair of MTS

[0212] - Method 4: Advanced syntax for switching between explicit and implicit transform selection

[0213] In at least one embodiment, transform selection uses a first method of implicit transform selection based on an ISP from a set of predetermined transform pairs, the selection being performed based on block size, such as Figure 5 Detailed description is given in . Therefore, in this embodiment, the same implicit transform selection is used for normal ISP mode and in coding units using MIP-ISP combination. According to a variant, a CU level flag indicates the use of this implicit transform selection for the normal ISP case.

[0214] Additionally, this implementation means that when ISP is used in combination with MIP, the explicit transformation mechanism previously used with MIP is no longer allowed. This implementation does not result in any changes to the coding unit level syntax.

[0215] In at least one embodiment, transform selection uses a second method based on implicitly selecting a DCT2 transform pair. This is an example method of transform selection. According to one method, if implicit transform selection is used for all intra blocks, then the MIP block will select DCT2. Therefore, the DCT2 transform is an appropriate choice for the MIP-ISP residual. Fig.18 The following example illustrates an implementation scheme for using the second transform selection method. implicitMtsEnabled With this process, when MIP is applied, the implicit ISP transform selection is disabled and DCT2 will be selected.

[0216] In at least one embodiment, transform selection uses a third method based on explicitly selecting transform pairs from MTS. Since the current design allows MTS with MIP, MIP-ISP residual can still use MTS. However, MTS with block sizes from 4×4 to 32×32 is allowed, while ISP may result in TUs with sizes greater than 32 or less than 4. Therefore, the TU size needs to be checked before encoding the MTS index. Fig.19 An excerpt of the coding unit syntax for an implementation using the third transform selection method is illustrated. The specification of the conversion process remains the same as Fig.18 Same specifications as in.

[0217] In at least one embodiment, transform selection uses a fourth method based on a high-level syntax for switching between explicit and implicit transform selection. In the current specification, MTS is controlled by 3 SPS tags: sps_mts_enabled_flag , sps_mts_enabled_flag and sps_explicit_mts_inter_enabled_flag .when sps_mts_enabled_ flag When equal to zero, all transforms use DCT2. When this flag is set to 1, the following configuration is defined. sps_explicit_ mts_intra_enabled_flag When equal to zero: implicit transform selection is used for all intra blocks (including ISP blocks). When the flag is set to 1: explicit transform selection is used for all intra blocks, while ISP uses implicit transform selection. sps_ explicit_mts_inter_enabled_flag When equal to zero: DCT2 is used for all inter blocks (excluding SBT). When this flag is set to zero: explicit transform selection is used for all inter blocks (excluding SBT).

[0218] In the fourth method for transform selection, if sps_explicit_mts_intra_enabled_flag is 1, then the explicit MTS of MIP-ISP is used as described in the third method. sps_explicit_mts_intra_ enabled_flag is zero or sps_mts_enabled_flag is zero, DCT2 is used as described in the second method. Fig. 20 Illustrates the derivation of the transform process for the implementation of the fourth transform selection method implicitMtsEnabled The transform process of the flag. Basically, only when the high-level syntax element sps_explicit_mts_intra_enabled_flag indicates that no explicit transform is used for the sequence under consideration, the implicit transform type of the current intra-CU encoded in ISP mode is used. Therefore, when the SPS-level syntax element sps_explicit_mts_intra_enabled_flag indicates the use of explicit MTS in the considered VVC bitstream, the implicit transform mechanism of ISP is deactivated.

[0219] In addition, if both the TU width and height are greater than or equal to 16, the current LFNST can be applied to the MIP prediction residual. On the other hand, LFNST is applied to the ISP residual with the same conditions as the normal block (both the TU width and height are greater than or equal to 4). To explain this, in at least one implementation, LFNST uses the same rules as ISP for the MIP-ISP combination. That is, if the TU of the MIP-ISP residual is greater than 4×4, LFNST can be used. Fig.21 Illustrates the corresponding syntax of the implementation using the low-frequency non-separable transform.

[0220] The following implementations are intended to relate to methods for reducing signaling overhead and limiting the RD search.

[0221] The first method for reducing signaling overhead is based on removing the MIP transpose flag. This allows reducing the signaling by one bin. If the ISP-MIP combination is adopted, the MIP transpose flag is inferred to be zero, where no transpose operation of the reference sample is performed. To perform this operation, it is necessary to signal ISP before MIP, for example, as Fig.15 described. Fig. 22 Illustrates an excerpt of the coding unit syntax of the implementation for removing the MIP transpose flag. In this syntax, intra_mip_transposed the flag is only encoded when ISP is not used.

[0222] The second method for reducing signaling overhead is based on implicitly deriving the ISP split direction according to the MIP mode. In other words, when the MIP-ISP combination is used, instead of signaling intra_subpartitions_split_flag , it is derived. For example, for an even-mode MIP, this can be achieved by setting the ISP direction to horizontal, and vice versa. Fig.23 Illustrates an excerpt of the coding unit syntax of an implementation where the ISP split direction is derived when MIP is combined with ISP.

[0223] In one variation, a partitioning is used that results in a thinner TU, resulting in a horizontal partitioning if the height is smaller than the width, and vice versa.

[0224] The third method for reducing signaling overhead is based on implicitly selecting MIP mode if a MIP-ISP combination is used. In other words, no signaling is performed when an ISP is used. intra_mip_mode It can be set to zero (often in the first MIP mode when using ISP) or to some other mode, depending on the block size. In any case, the ISP needs to be signaled before the MIP, e.g. Fig.15 In the example. Fig.24 An excerpt of the coding unit syntax illustrating an implementation in which the MIP mode is implicitly selected if the MIP-ISP combination is used.

[0225] Fig.25A An exemplary embodiment of an encoding process that implements a combination of a matrix-based intra prediction mode and an intra sub-partitioning mode is illustrated. In step 2010, a video block is obtained. In step 2020, the block is divided into sub-partitions. This can be performed using the intra sub-partitioning mode described above. Then, in step 2030, for each of the partitions, the samples of each sub-partition are predicted using matrix-based intra prediction, as described above. In step 2040, the block is then encoded as described above.

[0226] Fig.25B An exemplary implementation of a decoding process that implements a combination of a matrix-based intra prediction mode and an intra sub-partition mode is illustrated. In step 2050, a coding block and related signaling information are obtained, the related signaling information including information representing a matrix-based intra prediction mode and parameters for encoding. In step 2060, the block is divided into sub-partitions. This can be performed using the above-mentioned intra sub-partition mode. Then, in step 2070, samples are reconstructed from reconstructed neighbor samples using matrix-based intra prediction, as described above. Then, in step 2080, the block is decoded as described above.

[0227] Now, the following embodiments relate to intra prediction using multiple reference lines for blocks encoded using intra sub-partitioning mode. This allows for improved compression efficiency of video codecs. In other words, it allows encoding, signaling and decoding of blocks where intra prediction can partition a block into multiple equal-sized sub-blocks, while allowing each sub-block to be predicted based on any one of the available multiple reference lines. The prediction in each sub-block is performed using the prediction mode of the parent block.

[0228] Fig.28AAn example of an encoding process according to at least one embodiment is illustrated. In such an embodiment, a multiple reference line intra prediction mode is combined with an intra sub-partition mode. In step 910, the block is divided into sub-partitions. This can be performed using the above-mentioned intra sub-partition mode. Then, in step 920, for each partition in the partition, the samples of each sub-partition are predicted based on one of the available multiple reference lines. This can be performed using the above-mentioned multiple reference line mode. In step 930, the block is then encoded as described above.

[0229] Fig.28B An example of a decoding process according to at least one embodiment is illustrated. In such an embodiment, a multiple reference line intra prediction mode is combined with an intra sub-partition mode. In step 940, the block is divided into sub-partitions. This can be performed using the above-mentioned intra sub-partition mode. Then, in step 950, for each partition in the partition, a prediction based on one of the available multiple reference lines is used to reconstruct the samples of each sub-partition. This can be performed using the above-mentioned multiple reference line mode. Then, in step 950, the block is decoded as described above.

[0230] The following implementation describes the syntax and procedures required to implement the combination of multi-reference line intra prediction mode and intra sub-partitioning mode for a given block in an example application of the syntax and procedures of VVC draft 7. Other video coding systems will adopt another syntax to combine matrix-based intra prediction (MIP) with sub-partitioning of blocks into transform units.

[0231] Fig.29 An exemplary implementation of a coding unit syntax that implements a combination of MRL and ISP is illustrated.Other video coding systems may adopt another syntax to combine the intra sub-partitioning mechanism with prediction using multiple reference lines.

[0232] Fig. 30A An exemplary embodiment of a multi-reference line intra prediction system in a coding unit coded in an intra sub-partition is illustrated. According to this embodiment, the MRL tool is used for each TU contained in the considered CU. For example, TUs 1110, 1120, 1130, and 1140 use multiple reference lines 1115, 1125, 1135, and 1145, respectively, to perform their intra prediction. The reference line index used is the same for each TU and is, for example, determined by intra_luma_ref_idx Syntax elements are signaled on CU level with non-ISP CUs. Fig. 30A Horizontal partitioning is illustrated. In this case, TUs are decoded sequentially from top to bottom. The reference line after the second TU consists of some decoded samples in the previous TU. intra_luma_ref_idx The value of selects the reference line to be used for prediction for each TU.

[0233] The same principle similarly applies to vertical partitioning. In this case, the TU is partitioned vertically so that each TU benefits from the availability of multiple reference lines. In addition, based on intra_luma_ref_idx The value of selects the reference line to be used for prediction for each vertical TU.

[0234] Fig. 30B A second exemplary embodiment of a multi-reference line intra prediction system in a coding unit encoded in an intra sub-partition is illustrated. According to this embodiment, prediction uses intra_luma_ref_idx The reference samples of the reference line and column identified by the syntax element are used for intra prediction of the first TU of the 2 or 4 TUs issued from the ISP partition. Next, for the other TUs, a hybrid approach is used. Reference samples of TU boundaries located on closed CU boundaries are obtained on the reference line indicated by the MRL index. For other reference samples inside the considered CU, reference samples on the reference line closest to the considered TU are used for intra prediction of the considered TU.

[0235] Fig.29 A disadvantage of the implementation shown in is that for each reference line, the encoder must check the rate-distortion performance with and without CU partitioning, which is allowed by the ISP. This would make the encoder too complex. Therefore, in at least one implementation, the rate-distortion optimization step is improved by forcing one type of CU partitioning whenever the reference line index is not equal to 0. The partitioning type may be determined based on the directionality of the intra prediction mode. In this case, intra_ subpartitions_mode_flag and intra_subpartition_split_flag There is no need to send any signal notification, the segmentation type will be automatically derived.

[0236] In another embodiment, segmentation is always performed whenever the reference line index is non-zero. However, instead of deriving the segmentation type from the prediction mode, the segmentation type is mapped to the reference line used. For example, in one version, if multiRefIdx is equal to 1, then we can perform horizontal CU segmentation if multiRefIdx If t is equal to 3, vertical CU splitting can be performed. In another version, we can map the splitting types in the opposite way.

[0237] Fig.31 An example syntax for an implementation using a multiple reference line intra prediction system in coding units encoded in intra sub-partitions using matrix-based intra prediction is illustrated. In this implementation, Fig.31 The syntax combines the use of these three modes.

[0238] In one approach, two transforms are specified for any TU, one applied to the left side of the prediction residual (vertical transform) and the other applied to the right side (horizontal transform). When ISP is used, the transform pair of the TU obtained from the ISP partition is implicitly selected between DST7 and DCT2. If the TU width is between 4 and 16, the DST7 horizontal transform is used, otherwise DCT2 is used as the horizontal transform. Similarly, the vertical DST7 transform is used when the TU height is between 4 and 16, otherwise DCT2 is used. In any case, DST7×DCT2 or DCT2×DST7 2D separable primary transforms are used. On the other hand, explicit transform selection is allowed for MRL prediction residuals. That is, DCT2 transform or a combination of DST7 and DCT8 transforms can be used, as is commonly applied to intra-frame prediction residuals. Therefore, when applying the MRL-ISP combination, the transform process needs to be defined. In this case, three different methods for determining the transform are proposed.

[0239] The first approach is based on implicit transformation selection. In this case, using Figure 5 The same independent major transform selection as described in . Therefore, in this embodiment, the same implicit transform selection is used for normal ISP mode and in coding units using MRL-ISP combination. According to a variant of this method, the CU level flag indicates the use of this implicit transform selection for the normal ISP case. In addition, this embodiment means that when ISP is used in combination with MRL, the explicit transform mechanism used with MRL in the prior art is no longer allowed.

[0240] The second method is based on implicitly selecting a DCT2 transform pair. This is a straightforward transform selection method and is an appropriate choice for the MRL-ISP residual since this is the same choice made when implicit transform selection is employed for all intra blocks. Fig.32 The derivation process of determining the implicitMTS variable according to the second method is illustrated. The figure shows an example application of the second method to the syntax and procedures of VVC Draft 7, and more specifically, the derivation process of implicitMTS. Using this method, when MRL is applied, the implicit ISP transform selection is disabled and DCT2 will be selected.

[0241] The third approach is based on explicitly selecting transform pairs from MTS. In fact, since MTS is allowed to be used with MRL, MRL-ISP residual can still use MTS. However, MTS with block sizes from 4×4 to 32×32 is allowed, while ISP may result in TUs with sizes greater than 32 or less than 4. Therefore, the corresponding TU size needs to be checked before encoding the MTS index. Fig.33 An example of encoding according to the third method is illustrated, and Fig.34The derivation process of determining the implicitMTS variable according to the third method is illustrated.

[0242] Regarding high-level syntax, MTS can be controlled by 3 flags present in the sequence parameter set (SPS): sps_mts_enabled_flag, sps_explicit_mts_intra_enabled_flag, and sps_explicit_mts_inter_enabled_flag. When set to zero, sps_mts_enabled_flag sets all transforms to DCT2, otherwise non-DCT2 transforms (DST7 and / or DCT8) are used. When sps_mts_enabled_flag is 1, the following configuration is defined:

[0243] 1 - when sps_explicit_mts_intra_enabled_flag is equal to 0: use implicit transform selection for all intra blocks (including ISP blocks) and sub-block transform (SBT) blocks, where SBT is a transform tile for inter prediction,

[0244] 2- When sps_explicit_mts_intra_enabled_flag is equal to 1: use explicit transform selection for all intra blocks, while SBT and ISP use implicit transform selection,

[0245] 3- When sps_explicit_mts_inter_enabled_flag is equal to 0: DCT2 is used for all inter blocks (excluding SBT),

[0246] 4- When sps_explicit_mts_inter_enabled_flag is equal to 1: use explicit transform selection for all inter blocks (excluding SBT).

[0247] Fig.35 A first example of a transformation process according to at least one embodiment is illustrated. sps_explicit_ mts_intra_enabled_flag=1 When , explicit MTS for ISP is enabled. Otherwise, when this flag is set to zero, the default implicit selection is used or DCT2 is used.

[0248] Fig.36 A second example of a transformation process according to at least one embodiment is illustrated. An additional flag selected to enable explicit transformation for ISP is used. That is, for example, in the encoding of SPS, sps_explicit_mts_ isp_flag , as shown in Table 17.

[0249]

[0250] Table 17

[0251] Fig.37A An example of an encoding process combining the above-described embodiments is illustrated. In this exemplary embodiment, in step 3710, the encoding process obtains a video block. In step 3720, the block is divided into sub-partitions. This can be performed using the above-described intra-frame sub-partition mode. Then, in step 3730, for each partition in the partition, one of the available encoding tools of the video encoding system is applied to the samples of the sub-partition as previously described in different embodiments. In step 3740, the block is then encoded as described above, and related signaling information is generated, which includes information representing the encoding tool and associated parameters for encoding.

[0252] Fig.37B An example of a decoding process combining the above-described embodiments is illustrated. In this exemplary embodiment, the decoding process decodes a coding unit that has been encoded using the above-described intra-frame sub-partitioning mode. In step 3750, a coding block and related signaling information are obtained, and the related signaling information includes information representing a coding tool and associated parameters for encoding. In step 3760, the block is divided into sub-partitions. This can be performed using the above-described intra-frame sub-partitioning mode. Then, in step 3770, for each partition in the partition, one of the available encoding or decoding tools of the video encoding system (depending on the tool used, it can be the inverse tool of the encoding tool used for encoding or the same tool) is applied to the samples of the sub-partitions as previously described in different embodiments. Then, in step 3780, the block is decoded as described above.

[0253] Various specific implementations involve decoding. As used in this application, "decoding" may encompass, for example, all or part of a process performed on a received coded sequence in order to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by a decoder of the various embodiments described in this application, such as in Fig.9A , Fig. 9B , Fig.25A , Fig.25B , Fig.28A , Fig.28B , Fig.37A and Fig.37B The implementation scheme presented in .

[0254] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" refers specifically to a subset of operations or broadly to a broader decoding process will be clear based on the context of the specific description, and is believed to be well understood by those skilled in the art.

[0255] Various specific implementations involve encoding. In a manner similar to the discussion above regarding "decoding", "encoding" as used in this application can encompass, for example, all or part of the processes performed on an input video sequence in order to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processes also or alternatively include processes performed by encoders of the various embodiments described in this application, such as in Fig.9A , Fig. 9B , Fig.25A , Fig.25B , Fig.28A , Fig.28B , Fig.37A or Fig.37B implementation plan.

[0256] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in yet another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" refers specifically to a subset of operations or broadly to a broader encoding process will be clear based on the context of the specific description, and is believed to be well understood by those skilled in the art.

[0257] Note that the grammatical elements used herein are descriptive terms. Therefore, they do not exclude the use of other grammatical element names.

[0258] This application describes various aspects, including tools, features, implementations, models, methods, etc. Many of these aspects are described in detail.

[0259] And at least individual characteristics are shown, often described in a way that may sound limited. However, this is for clarity of description and does not limit the application or scope of these aspects. In fact, all different aspects can be combined and interchanged to provide further aspects. In addition, these aspects can also be combined and interchanged with aspects described in previous filings. The aspects described and contemplated in this patent application can be implemented in many different forms. Figure 1 , Figure 2 and Figure 3Some embodiments are provided, but other embodiments are contemplated, and the discussion of these figures does not limit the breadth of the embodiments.

[0260] In this application, the terms "reconstruction" and "decoding" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, the terms "image", "picture" and "frame" are used interchangeably, and the terms "index" and "idx" are used interchangeably. Usually, but not necessarily, the term "reconstruction" is used at the encoding end, while "decoding" is used at the decoding end.

[0261] Various methods are described herein, and each method includes one or more steps or actions for implementing the method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0262] Various numerical values ​​are used in this application, such as regarding block size. The specific values ​​are for example purposes, and the aspects are not limited to these specific values.

[0263] Reference to "one embodiment" or "an embodiment" or "one implementation" or "an implementation" and other variations thereof means that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearance of the phrase "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" and any other variations thereof in various places throughout this specification are not necessarily all referring to the same embodiment.

[0264] Additionally, the application or its claims may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0265] Furthermore, this application or its claims may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, predicting information, or estimating information.

[0266] Additionally, the application or its claims may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing the information or retrieving the information (e.g., from a memory or optical media storage device). Furthermore, "receiving" generally involves in one way or another during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0267] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selecting only the first listed option (A), or selecting only the second listed option (B), or selecting both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such phrases are intended to encompass selecting only the first listed option (A), or selecting only the second listed option (B), or selecting only the third listed option (C), or selecting only the first listed option and the second listed option (A and B), or selecting only the first listed option and the third listed option (A and C), or selecting only the second listed option and the third listed option (B and C), or selecting all three options (A and B and C). This can be extended to as many items as listed, as would be apparent to one of ordinary skill in this and related arts.

[0268] It will be apparent to those skilled in the art that a specific implementation may generate various signals formatted to carry, for example, storable or transmittable information. The information may include, for example, instructions for executing a method or data generated by one of the specific implementations. For example, a signal may be formatted to carry a bit stream of the embodiment. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

Claims

1. A method, the method include: Partition the video chunks into subpartitions, selecting a single reference line among a plurality of reference lines that are common to all subpartitions of the block, and For each of the sub-partitions, predicting samples of the sub-partition based on reference samples of the selected single reference line; wherein the selected single reference line is used to predict all of the sub-partitions of the block, and wherein the selected single reference line can be any one of the plurality of reference lines common to all of the sub-partitions of the block. 2 . The method of claim 1 , wherein the reference line is selected from a plurality of reference lines adjacent to a top line of the block or a left column of the block. 3 . The method of claim 1 , wherein the reference line is selected from a plurality of reference lines adjacent to a top line of the sub-partition or a left column of the sub-partition.

4. The method according to any one of claims 1 to 3, wherein the prediction of the samples of the sub-partition is based on a 2D matrix and a 1D vector, which are applied to a vector based on average samples of a selected reference line.

5. A video encoding method, the video encoding method include: selecting a single reference line to be used for the prediction, the single reference line being selected from a plurality of reference lines common to all subpartitions of the block, Predicting samples of sub-partitions according to the method of claim 1, and The block and signaling information are encoded, wherein the block is encoded in an intra sub-partition mode, and the signaling information includes at least information indicating a selected single reference line to be used for the prediction.

6. A video decoding method, the video decoding method include: obtaining information representing a video block encoded using an intra subpartition mode and information representing a selected single reference line to be used for said prediction, said single reference line being selected from a plurality of reference lines common to all subpartitions of the block, Splitting the video block into sub-partitions, The method of claim 1, predicting samples of the video block based on a selected reference line, and The block is decoded.

7. A video encoding device, comprising an encoder, wherein the encoder is configured to: selecting a single reference line to be used for the prediction, the single reference line being selected from a plurality of reference lines common to all subpartitions of the block, Predicting samples of sub-partitions according to the method of claim 1, and The block and signaling information are encoded, wherein the block is encoded in an intra sub-partition mode and the signaling information includes at least information indicating a selected single reference line to be used for the prediction.

8. A video decoding device, the video decoding device comprising a decoder, the decoder being configured to: obtaining information representing a video block encoded using an intra subpartition mode and information representing a selected single reference line to be used for said prediction, said single reference line being selected from a plurality of reference lines common to all subpartitions of the block, Splitting the video block into sub-partitions, The method of claim 1, predicting samples of the video block based on a selected single reference line, and The block is decoded.

9. A non-transitory computer readable medium comprising program code instructions executable by a processor to implement the steps of the method according to claim 1.

Images