Method and apparatus for coding / decoding picture data

Abstract

The disclosure relates to a method for encoding image data, the method comprising: intra-predicting a first block of said image data by using an intra-prediction mode or predicting by combining inter-prediction and intra-prediction, wherein said intra-prediction mode uses a first single transform obtained by considering the size of said first block. The invention also relates to a method for encoding image data, variable coding length is used to signal a plurality of prediction modes by said encoding, the method comprising: - intra-predicting a first block of said image data by using an intra-prediction mode using a first transform, said first transform being obtained by considering the size of said first block, - encoding information signaling the use of said intra-prediction mode in a bitstream, said information being encoded as one of said plurality of prediction modes. The disclosure also relates to corresponding decoding methods, devices and media.

Description

1. TECHNICAL FIELD

[0001] Embodiments of the present invention relate generally to encoding / decoding of picture data. 2. BACKGROUND

[0002] This section is intended to introduce the reader to various aspects of art that can be related to various aspects of at least one embodiment of the present invention described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of at least one embodiment. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.

[0003] To achieve high compression efficiency, video coding schemes typically employ prediction and transform to exploit the spatial and temporal redundancy in the video content. Generally, intra- or inter-frame prediction is used to exploit the intra- or inter-frame correlation. Then, the differences between the original picture and the predicted picture, often denoted as prediction error or prediction residual, are transformed, quantized, and entropy coded. To reconstruct the picture, the compressed data is decoded by inverse processes corresponding to the prediction, transform, quantization, and entropy coding. 3. SUMMARY

[0004] Some embodiments of the present disclosure enable to solve at least one drawback by proposing a method for encoding image data, the method comprising: intra-prediction, or prediction by combining inter-prediction and intra-prediction. The present disclosure also relates to a corresponding decoding method, device and medium. 4. BRIEF DESCRIPTION OF DRAWINGS

[0005] Figure 1 A simplified block diagram of an exemplary encoder is shown in accordance with at least one embodiment;

[0006] Figure 2 A simplified block diagram of an exemplary decoder is shown in accordance with at least one embodiment;

[0007] Figure 3 An exemplary reference sample generation process is shown;

[0008] Figure 4 Exemplary prediction directions for square blocks are shown;

[0009] Figure 5 Exemplary top and left CU positions for deriving MPM lists for different block shapes are shown.

[0010] Figure 6 Exemplary MPM signaling is shown;

[0011] Figure 7 Exemplary MPM derivation is shown;

[0012] Figure 8 An exemplary MPM signaling when multiRefldx > 0 in VTM-5.0 is shown;

[0013] Figure 9 An exemplary affine linear weighted intra prediction process for 4x4 block is shown;

[0014] Figure 10 An exemplary affine linear weighted intra prediction process for 8x4 block is shown;

[0015] Figure 11 An exemplary affine linear weighted intra prediction process for 16x16 block is shown;

[0016] Figure 12 An exemplary MPM signaling when mipFlag is placed before the bits that signal the first MPM is shown;

[0017] Figure 13 An exemplary MPM signaling when mipFlag is placed before the bits that signal the first MPM is shown;

[0018] Figure 14 An exemplary MPM derivation when SMIP is put in the MPM list DC is shown;

[0019] Figure 15 An exemplary MPM derivation when SMIP is put in the MPM list DC is shown;

[0020] Figure 16 An exemplary MPM derivation when SMIP is put in the MPM list DC is shown;

[0021] Figure 17 An exemplary prediction map is shown;

[0022] Figure 18 An exemplary prediction of a current block is shown;

[0023] Figure 19 An exemplary values of U2, U18, U34 and U50 for 4x4 block is shown;

[0024] Figure 20 An exemplary illustration of and is shown;

[0025] Figure 21 An exemplary illustration of and is shown;

[0026] Figure 22 An exemplary illustration of and exemplary illustrations; and

[0027] Figure 23 shows a block diagram of an example of a system in which various aspects and embodiments can be implemented.

[0028] It should be noted that the drawings illustrate the exemplary embodiments and are therefore not to be considered limiting in their scope the embodiments of the application are not limited to the 5. DETAILED DESCRIPTION

[0029] This detailed description illustrates by way of example, not by way of limitation, the principles of the embodiments of the application. Thus, it is to be understood that other

[0030] All examples and conditional language recited herein are intended to be construed to teach only one example of how to make and use embodiments of the application and are not intended to

[0031] Also, all statements regarding the examples, or any experimental results, are intended to include all and only those examples that would be described and enabled by the language in the specification and claims, which will become apparent upon reading the specification and claims. Moreover, the indefinite articles "a" or "an," as used in the claims, are defined herein to mean one or more than one whenever they are used.

[0032] Accordingly, for example, those skilled in the art will appreciate that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the principles of the embodiments. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which can be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0033] The application will be described more fully hereinafter with reference to the accompanying drawings, in which examples of the application are shown. The application may, however, be embodied in many alternate forms and should not be construed as limited to the examples set forth herein. Accordingly, it should be understood that the application is not limited to the disclosed embodiments. Rather, the application is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the application.

[0034] When the drawings are presented as flowcharts, it will be understood that each block within the flowcharts can represent a module, segment, or portion of code that comprises one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the implementation, or these blocks can be replaced with a single block, or the like.

[0035] The functions of the various elements shown in the figures can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (DSP) hardware, read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and / or custom, can also be included. Similarly, any switches shown in the figures are conceptual only. Their function can be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

[0036] Similar or identical elements shown in the figures are designated by common reference numerals. Some of the figures represent syntax tables that are widely used in the specification of video compression standards to define the structure of bitstreams conforming to the described video compression standards. In these syntax tables, the term "..." indicates an unchanged part of the syntax with respect to the well-known definitions given in the specification of the video compression standard and is removed in the figures for ease of reading. Bold items in the syntax tables indicate that the value of this item is obtained by parsing the bitstream. The right column of the syntax table indicates the number of bits of data used to encode the syntax element. For example, u(4) indicates 4 bits are used to encode the data, u(8) indicates 8 bits, ae(v) indicates a syntax element that is context adaptively arithmetically entropy coded.

[0037] In its claims, any element expressed as a means for performing a specified function is intended to cover any way of performing that function including, for example, a) a combination of circuit elements that performs that function, or b) software in any form, including, therefore, firmware or microcode, etc., in combination with the appropriate circuitry for executing that software to perform that function. The embodiments defined by the claims are to be understood as follows: the functions provided by various described means are intended to be combined and arranged in the manner required by the claims. Thus, any means that can provide those functions are considered equivalent to those means shown.

[0038] It will be appreciated that the figures and descriptions have been simplified to illustrate elements that are relevant for a clear understanding of the embodiments, while

[0039] It should be understood that, although the terms first and second can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. The various methods described above can be implemented, individually and / or in combination, by one or more steps or actions of a method described above. The order of the steps and / or acts can be modified or combined, and / or the order of use can be changed, unless a particular order is required by the correct operation of the method.

[0040] In the following sections, the words “reconstruct” and “decode” are used interchangeably. Typically, but not necessarily, “reconstruct” is used on the encoder side, while “decode” is used on the decoder side. In addition, the words “code” and “encode” are used interchangeably. In addition, the words “image,” “picture,” and “frame” can be used interchangeably. In addition, the words “code,” “source code,” and “compress” can be used interchangeably.

[0041] It should be understood that a reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation” of the present disclosure, and other variants thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, the appearance of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation” or any other variants thereof, in various places throughout the specification is not necessarily referring to the same embodiment. Furthermore, the particular features, structures, characteristics, and so forth described in connection with the embodiments can be combined in any suitable manner in one or more embodiments.

[0042] Additionally, the present embodiments or claims can relate to “determining” various pieces of information. Determining, deriving, the information can include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory. Further, the present application or claims can relate to “providing” various pieces of information. Providing the information can include one or more of, for example, outputting the information, storing the information, transmitting the information, sending the information, displaying the information, showing the information, or moving the information. Further, the present application or claims can relate to “accessing” various pieces of information. Accessing the information can include one or more of, for example, receiving the information, retrieving the information (e.g., from memory), storing the information, processing the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information. Further, the present application or claims can relate to “receiving” various pieces of information. As with “accessing,” receiving is intended to be a broad term. Receiving the information can include one or more of, for example, accessing the information or retrieving the information (e.g., from memory). Further, “receiving” is typically involved, in one way or another, during operations such as storing the information, processing the information, sending the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0043] It should be understood that various features that are shown and described can be interchangeable. Features shown in one embodiment can be incorporated into another embodiment unless otherwise stated. Further, features described in the various embodiments can be combined or separated unless otherwise indicated as being inseparable or combinable.

[0044] As previously described, the functions of the various elements shown in the figures can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. Also, when provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared.

[0045] It should also be understood that, because some of the constituent system components and methods described in the detailed description can be implemented in software, the actual connections between system components or processing function blocks can differ depending upon the manner in which the present disclosure is programmed. Given the teachings herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the present disclosure.

[0046] Although illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the disclosure is not limited to those precise embodiments, and that various changes and modifications can be effected therein by one of ordinary skill in the related art without departing from the scope of the disclosure. Further, various implementations of the embodiments have been described herein, in terms of exemplary embodiments. The various implementations have been described with regard to particular implementations, and more specific language has been used in order to facilitate a thorough understanding of the application. However, the application is not limited to the specifically described implementations. Rather, the specific implementations are disclosed as exemplary implementations of the application, and one of ordinary skill in the art will understand that various changes and modifications can be made thereto without departing from the scope of the application. All such changes and modifications are meant to fall within the scope of the application as set forth in the appended claims.

[0047] It should be understood that any of the following “ / ”, “and / or”, and “at least one of” are intended to encompass only a selection of one or more of the items, “only a selection of” one or more of the items, or “only a selection of” one or more of the items. As a further example, in the case of “A, B, and / or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed item (A) only, or the selection of the second listed item (B) only, or the selection of each (A and B) only, or the selection of the first and third listed items (A and C) only, or the selection of the second and third listed items (B and C) only, or the selection of all three items (A and B and C). This can be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are chosen or are possible.

[0048] As will be appreciated by one of ordinary skill in the art, implementations can produce a variety of signals including but not limited to signals intended for consumption by a human user and signals intended for consumption by a machine, such as a machine-to-machine service. Such a signal can convey information, such as instructions for execution and / or data to be processed. Such a signal can be formatted, for example, according to a file format, a packetized format, a bit stream format, and / or another format. Further, it should be understood that software instructions can be embodied explicitly in code, in binary, in expressions, in logic, and / or the like, and can be transitory, such as electrical signals, magnetic signals, and / or the like.

[0049] It is to be understood that a picture (also denoted image or frame) can be a luma sample array in monochrome format, or a luma sample array and two corresponding chroma sample arrays in 4:2:0, 4:2:2 or 4:4:4 color format, or three arrays for three color components (e.g. RGB).

[0050] In video compression standards, pictures are partitioned into blocks that can have different sizes and / or different shapes. It is understood that a block is a two-dimensional array or matrix. The horizontal or x-direction (or axis) represents the width and the vertical or y-direction (or axis) represents the height. The indexing starts at 0. The x-direction represents the columns and the y-direction represents the rows. The maximum x-index is the width - 1 and the maximum y-index is the height - 1.

[0051] Encoding

[0052] Figure 1 A simplified block diagram of an exemplary encoder 100 is shown in accordance with at least one embodiment. The encoder 100 can be included in a transmitter or headend in a communication system.

[0053] For encoding a video sequence having one or more pictures, the pictures are partitioned into blocks that can have different sizes and / or different shapes (module 110). For example, in HEVC ("ITU-T H.265 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (10 / 2014), SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS, Infrastructure of audiovisual services - Coding of moving video, High efficiency video coding, Recommendation ITU-T H.265") pictures can be partitioned into square CTUs (coding tree units) having a configurable size. A contiguous set of CTUs can be grouped into a slice. A CTU is the root of a quadtree that is partitioned into coding units (CUs).

[0054] In the exemplary encoder 100, the pictures are encoded by a block-based encoding module as follows. Each block is encoded using either an intra prediction mode or an inter prediction mode. When a block is encoded in the intra prediction mode (module 160), the encoder 100 performs intra prediction (also denoted as spatial prediction) based on at least one sample of one block in the same picture (or based on a pre-defined value for the first block of the picture or slice). As an example, a prediction block is obtained by intra predicting a block from reconstructed neighboring samples.

[0055] When encoding a block in inter prediction mode, the encoder 100 performs inter prediction (also known as temporal prediction) based on at least one reference block of at least one reference picture or slice (which is stored in the reference picture buffer).

[0056] Inter prediction coding is performed by performing motion estimation (module 175) and motion compensation (module 170) on reference blocks stored in the reference picture buffer 180. In single inter prediction (a.k.a. uni-prediction) mode, the prediction block can be based on a reference picture that is earlier in time, generally (but not necessarily). In bi-inter prediction (a.k.a. bi-prediction) mode, the prediction block can be based on a reference picture that is earlier in time and a reference picture that is later in time, generally (but not necessarily).

[0057] The encoder 100 decides (module 105) which of the intra prediction mode or the inter prediction mode to use to encode the block, and indicates the intra / inter decision by a prediction mode syntax element.

[0058] A prediction residual block is computed by subtracting (module 120) the prediction block (also known as predictor) from the block. The prediction residual block is transformed (module 125) and quantized (module 130). The transform module 125 can transform the prediction residual block from pixel (spatial) domain to transform (frequency) domain. The transform can be, for example, a cosine transform, a sine transform, a wavelet transform, etc. The quantization (module 130) can be performed according to, for example, rate-distortion criteria.

[0059] The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (module 145) to output a bitstream. The entropy coding can be, for example, context adaptive binary arithmetic coding (CABAC), context adaptive variable length coding (CAVLC), Huffman, arithmetic, exp-Golomb, etc.

[0060] The encoder can also skip the transform, and apply quantization directly on the untransformed prediction residual block. The encoder can also bypass both the transform and quantization, i.e., directly code the prediction residual block without applying the transform or quantization processes.

[0061] In direct PCM coding, no prediction is applied, and the block samples are directly coded into the bitstream.

[0062] The encoder 100 includes a decoding loop and thus decodes the coded blocks to provide references for further prediction. The quantized transform coefficients are dequantized (module 140), also known as inverse quantization, and inverse transformed (module 150) to decode the prediction residual blocks. Then, the decoded prediction residual blocks are combined (module 155) with the prediction blocks, also known as predictors, to reconstruct the blocks. One or more in-loop filters (165) can be applied to the reconstructed picture, e.g., to perform deblocking / sample adaptive offset (SAO) filtering to reduce coding artifacts. The filtered picture is stored in the reference picture buffer 180.

[0063] The modules of the encoder 100 can be implemented in software and executed by a processor, or can be implemented using a circuit component well-known to those skilled in the art of compression. In particular, the video encoder 100 can be implemented as an integrated circuit (IC).

[0064] Decoding

[0065] Figure 2 A simplified block diagram of an exemplary decoder 200 according to at least one embodiment is shown. The decoder 200 can be included in a receiver of a communication system. The decoder 200 generally performs a decoding process that is reciprocal to the encoding process performed by the encoder 100 as described in Figure 1 but not all operations in the decoder are inverse operations of the encoding process (e.g., intra prediction and inter prediction). In particular, the input to the decoder 200 includes a video bitstream, which can be generated by the encoder 100.

[0066] The video bitstream is first entropy decoded (module 230) to obtain, for example, transform coefficients, motion vectors MVs, picture partitioning information, possible prediction mode flags, syntax elements, and other decoded data. For example, in HEVC, the picture partitioning information indicates the size of the CTU and the way the CTU is divided into CUs. Thus, the decoder can divide (235) the picture into CTUs and each CTU into CUs according to the picture partitioning information.

[0067] The transform coefficients are dequantized (module 240) and inverse transformed (module 250) to decode the prediction residual blocks. Then, the decoded prediction residual blocks are combined (module 255) with prediction blocks, also known as predictors, to obtain decoded / reconstructed blocks.

[0068] The prediction block can be obtained from either intra prediction (module 260) or motion-compensated prediction (i.e., inter prediction) (module 270) depending on the prediction mode flag (module 205). An in-loop filter (module 265) can be applied to the reconstructed picture. The in-loop filter can include a deblocking filter and / or a SAO filter. The filtered picture is stored in the reference picture buffer 280.

[0069] The modules of the decoder 200 can be implemented in software and executed by a processor, or can be implemented using circuit components well known in the art of compression. In particular, the decoder 200 can be implemented as an integrated circuit (IC), which can be separate or combined with the encoder 100 as a codec.

[0070] The present embodiment addresses intra prediction in Versatile Video Coding (VVC) VTM 5.0. At the JVET meeting held in Geneva in March 2019, it was agreed to use Matrix Intra Prediction (MIP), which is an affine transform set for intra prediction. With MIP, intra prediction in VTM-5.0 is now based on Intra Prediction with Multiple Reference Lines (MRL), Intra Prediction with Sub-partition (ISP), the traditional 67 intra prediction modes (Planar, DC, and 65 directional modes), and MIP. The present embodiment proposes to reduce this affine transform set to a single affine transform. This single affine transform for intra prediction is denoted here as SMIP. This reduces the memory footprint on both the encoder and the decoder side by a factor of 7.92.

[0071] The intra prediction process in H.266 / VVC includes the collection of reference samples, their processing, the derivation of the actual prediction of the samples of the current block, and finally the post-processing of the predicted samples. The reference sample generation process is explained in Figure 3 An exemplary reference sample for intra prediction in H.266 / VVC is shown in the case of a square current block with W = H = N, where the pixel value at coordinate (x, y) is denoted by P(x, y) in the figure. From the previously reconstructed top and top-right pixels of the current block, an array of 2W pixels above is formed, where W denotes the block width. Similarly, from the reconstructed left and bottom-left pixels, a column of 2H samples to the left is formed, where H denotes the block height. The corner pixel at the top-left position is also used to fill the gap between the top row and left column references. If some of the samples on the top or left are not available (due to the corresponding coding unit (CU) not being in the same slice or the current CU being at a frame boundary), a method called reference sample replacement is performed, where the missing samples are copied from the available samples in a clockwise direction. Next, the reference samples are filtered using a specified filter depending on the current CU size and prediction mode.

[0072] Figure 4 ​H.266 / VVC includes a range of prediction models derived from the prediction models in H.265 / HEVC. Planar and DC prediction modes are used to predict smooth and gradually changing regions, while angular prediction modes are used to capture different directional structures. There are 65 directional prediction modes, which are differently organized for each rectangular block shape. These prediction modes correspond to different prediction directions as shown in Figure 4 Intra prediction is further extended with tools such as Intra prediction with multiple reference lines (MRL) and Intra prediction with sub-partitions (ISP).

[0073] At the encoder side, the best Intra prediction mode according to the rate-distortion criterion is selected and its index is transmitted from the encoder to the decoder. To perform the signaling of the selected mode index via entropy coding, a list of most probable modes (MPM) is established.

[0074] In VTM-5.0, the MPM list contains 6 Intra prediction modes [1] for signaling the Intra prediction mode of the current block. The MPM list is created from the prediction modes of the Intra coded CUs on the top and left side of the current CU and some default modes. As shown in Figure 5 The top and left side CUs are on the right and bottom edges of the current block.

[0075] L≡prediction mode of the left side CU (value in [0-66] range)

[0076] A≡prediction mode of the above CU (value in [0-66] range)

[0077] offset = 61

[0078] mod = 64

[0079] Initialization of the MPM list:

[0080] MPM[0] = PLANAR IDX

[0081] MPM[1] = DC IDX

[0082] MPM[2] = VER IDX

[0083] MPM[3] = HOR IDX

[0084] MPM[4] = VER IDX - 4

[0085] MPM[5] = VER IDX + 4

[0086] Building of the MPM List:

[0087] if (L = A)

[0088] if (L > DC IDX)

[0089] MPM[0] = PLANAR IDX

[0090] MPM[1] = L

[0091] MPM[2] = ((L + offset) % mod) + 2

[0092] MPM[3] = ((L - 1) % mod) + 2

[0093] MPM[4] = DC IDX

[0094] MPM[5] = ((L + offset - 1) % mod) + 2

[0095] else

[0096] use initialized values

[0097] else

[0098] if ((L > DC IDX) && (A > DC IDX))

[0099] MPM[0] = PLANAR IDX

[0100] MPM[1] = L

[0101] MPM[2] = A

[0102] MPM[3] = DC IDX

[0103] MPM[4] = ((max(L, A) + offset) % mod) + 2 if L and A are not adjacent

[0104] = ((max(L, A) + offset - 1) % mod) + 2, otherwise

[0105] MPM[5] = ((max(L, A) - 1) % mod) + 2 if L and A are not adjacent

[0106] = ((max(L, A) - 0) % mod) + 2 otherwise

[0107] else if (L + A >= 2)

[0108] MPM[0] = PLANAR IDX

[0109] MPM[1] = max(L, A)

[0110] MPM[2] = DC IDX

[0111] MPM[3] = ((max(L, A) + offset) % mod) + 2

[0112] MPM[4] = ((max(L, A) - 1) % mod) + 2

[0113] MPM[5] = ((max(L, A) + offset - 1) % mod) + 2

[0114] else

[0115] use initialized values

[0116] Using circular adjacency over the range [2-66], it can be equivalently written

[0117] ((L + offset) % mod) + 2≡ L - 1

[0118] ((L + offset - 1) % mod) + 2≡ L - 2

[0119] ((L - 1) % mod) + 2≡ L + 1

[0120] ((L - 0) % mod) + 2≡ L + 2

[0121] Using the above relations, it can be shown that the MPM list derivation is Figure 6 the derivation in

[0122] In regular intra prediction, if the prediction mode of the current block corresponds to one of the six MPM modes, this prediction mode is signaled via mpmFlag having the value 1 and then by signaling the candidate mode from the MPM list using the variable length coding scheme shown in Figure 7 Figure 7 A and L denote the prediction modes of the above and left CUs, respectively. Otherwise, mpmFlag is equal to 0 and the candidate index in the set of the remaining 61 modes is signaled using 5 or 6 bit truncated-binary coding.

[0123] ​For intra prediction with MRL, the reference line used for prediction is signaled with the flag multiRefIdx. The valid values for multiRefIdx are 0, 1 and 3, which signal the first, second or fourth reference line. When multiRefIdx is non-zero (meaning that the second or fourth reference line is used), the prediction mode always belongs to the MPM list. Therefore, mpmFlag is not signaled. In addition, the planar mode is excluded from the list. This means that only five prediction modes are available as possible candidates when multiRefIdx is non-zero. When multiRefIdx is non-zero, the prediction mode is signaled as Figure 8 indicated in Table 1.

[0124] For intra prediction with ISP, the type of partitioning for a CU is signaled with a flag called ispMode. ispMode is only coded when multiRefIdx is equal to 0. The valid values for ispMode are 0, 1 and 2, which denote no partitioning, horizontal partitioning and vertical partitioning, respectively. If ispMode is non-zero, i.e. if the target CU is partitioned, the prediction mode always belongs to the MPM list. Therefore, mpmFlag is not signaled in this case. The prediction mode is coded using Figure 7 Table 2.

[0125] At the 12th JVET meeting in Macau, China, it was agreed to integrate the combined intra prediction (CIIP) in VVC. The tool was then simplified in [3]. In the simplified version of CIIP, the merge mode linearly combines the planar prediction and the prediction of one merge index. The intra- and inter-weights used to combine the intra- and inter-predicted samples depend on the number of neighboring intra-coded blocks. The notation in Figure 5 [2] is reused,

[0126] if (TopCU and LeftCU are intra coded)

[0127] w intra = 3

[0128] w inter = 1

[0129] else if (TopCU or LeftCU is intra coded)

[0130] w intra = 2

[0131] w inter = 2

[0132] else,

[0133] w intra = 1

[0134] w inter = 3

[0135] MIP (as employed in the current version of H.266 / VVC) is described in [2]. For MIP, the reference sample generation process consists in specifically fetching W top and H left reconstructed pixels. These reference samples are not filtered. Said reference samples are averaged, resulting in a vector of 4 values in case of W = H = 4, and a vector of 8 values in case of larger blocks. The vector of averaged reference samples is multiplied by a weight matrix and added to a biases vector. Finally, the prediction of the current block is obtained by linearly interpolating the result. The initial reference samples are used for interpolating the boundaries.

[0136] The three sets of matrices and vectors (S0, S1, S2) are available for the encoder to select. For blocks of size 4x4, the set S0 contains 18 matrices and biases vectors of size 16x4 and 16, respectively. For blocks of size 4x8, 8x4 and 8x8, S1 contains 10 matrices of size 16x8 and biases of size 16. Finally, for all other block shapes, the set S2 contains 7 matrices of size 64x8 and 6 biases vectors of size 64.

[0137] According to the mode and the block shape, the input vector bdry red is rearranged as follows:

[0138]

[0139] where and denote and describe the average boundary samples at the top and left, as illustrated in Figure 9 .

[0140] Then, the output reduced sample set pred red of the samples of the prediction block is generated as pred red = A - bdry red + b.

[0141] According to the mode and the block shape, the weight matrix A and the offset vector b are selected in the sets S0, S1, S2 as where

[0142]

[0143] and

[0144]

[0145] In the case of 8x8 and 8x4 blocks, the interpolation process performed on pred red is illustrated in Figure 10 and Figure 11 respectively.

[0146] For larger blocks, the reference samples are grouped and averaged so that the input vector is 8 samples long, as shown in Figure 12 for 16x16 blocks.

[0147] This method requires storing the sets S0, S1, S2, which correspond to

[0148] - 18 matrices of size 16x4 and 18 offsets of size 16

[0149] - 10 matrices of size 16x8 and 10 offsets of size 16

[0150] - 6 matrices of size 64x8 and 6 offsets of size 64

[0151] which represents 6336 parameters to code, which corresponds to 7.92 kilobytes of data when stored as 10-bit values.

[0152] The MIP mode is first signaled with a flag called mipFlag, with value 1 meaning that the MIP mode is used to predict the current block, and value 0 meaning that one of the 67 traditional intra prediction modes is used. When mipFlag is equal to 1, multiRefIdx must be equal to 0, which means that the first reference line is used, and ispMode is equal to 0, i.e. there is no target CU partition. Therefore, when mipFlag is equal to 1, multiRefIdx and ispMode are not written. If mipFlag is equal to 1, the MIP mode is then signaled via a binary decision tree based on a list with 3 MPMs.

[0153] To handle the case where the intra prediction mode used to predict the current block is one of the 67 traditional intra prediction modes, and the mode used to predict the CU to the right of the current block or the mode used to predict the CU at the bottom edge of the current block is a MIP mode, a mapping between each MIP mode and one of the traditional modes can make this MIP mode replaced by its mapped traditional mode. This mapping also enables to replace the way round, i.e. the traditional mode, by the MIP mode it is mapped to.

[0154] In this embodiment, the set S0 for block size 4x4 is not used, but a single matrix M0 and bias vector b0 of size 16x4 and 16, respectively, are used. The set S1 for block sizes 4x8, 8x4 and 8x8 is not used, but a single matrix M1 and bias vector b1 of size 16x8 and 16, respectively, are used. The set S2 for all other block shapes is not used, but a single matrix M2 and bias vector b2 of size 64x8 and 64, respectively, are used. This amounts to 800 parameters to code, which corresponds to 1.0 kilobyte when stored as 10-bit values.

[0155] According to some embodiments, the signaling of the SMIP using some of the above elements of H.266 / VVC can be done via a mipFlag, the value 1 now meaning a single MIP mode. Depending on the shape of the current block, either (M0, b0), (M1, b1) or (M2, b2) can be used to predict the current block.

[0156] Now, the above mapping reduces to SMIP to Planar.

[0157] A first variant of the above SMIP signaling is to place the mipFlag after the mpmFlag and before the bits signaling the first MPM. Now, if the mipFlag is equal to 1, the candidate modes are signaled from the MPM list using the variable length coding scheme shown in Table 1. Figure 13 A second variant is to place the mipFlag after the bits signaling the first MPM and before the bits signaling the second MPM. Now, if the mipFlag is equal to 1, the candidate modes are signaled from the MPM list using the variable length coding scheme shown in Table 2.

[0158] A second variant is to place the mipFlag after the bits signaling the first MPM and before the bits signaling the second MPM. Now, if the mipFlag is equal to 1, the candidate modes are signaled from the MPM list using the variable length coding scheme shown in Table 2. Figure 14 A second variant is to place the mipFlag after the bits signaling the first MPM and before the bits signaling the second MPM. Now, if the mipFlag is equal to 1, the candidate modes are signaled from the MPM list using the variable length coding scheme shown in Table 2.

[0159] In a third variant, instead of using a mipFlag, the SMIP can be put into the MPM list before the DC. The MPM derivation becomes the one described in Table 3. Figure 15

[0160] In a fourth variant, instead of using a mipFlag, the SMIP can also be put into the MPM list after the DC. The MPM derivation becomes the one described in Table 4. Figure 16

[0161] In a fifth variant, as mentioned above (third and fourth variant), no mipFlag is used. The SMIP replaces the Planar, which means that the SMIP has the signaling of the Planar.

[0162] ​​In a sixth variant, as mentioned above (third, fourth and fifth variants), the mipFlag is not used and the SMIP replaces the DC, which means that the SMIP owns the signaling about the DC.

[0163] The seventh variant can be combined with any of the variants of SMIP signaling described above. At the encoder side, when SMIP is selected as the intra prediction mode used to predict the current block, the multiple transform selection (MTS) and / or the non-separable secondary transform (NSST) transform can be deactivated, which means that for the deactivated transform, the signaling flag is not written in the bitstream. At the decoder side, after decoding the signaling flags for the intra prediction, if SMIP is selected as the intra prediction mode used to predict the current block, the flag of the deactivated transform is not decoded.

[0164] The planar mode is the only intra prediction mode used by the system in the simplified version of the CIIP described above, which can be replaced by the SMIP.

[0165] Since the SMIP has few parameters (800), these parameters can be learned online (e.g., when encoding the frames) and transmitted. More precisely, at the encoder side, for each group of n frames, the parameters of the SMIP can first be learned on pairs (decoded reference samples, block) extracted from the n luminance channels. Then, the learned parameters are encoded. Finally, the group of n frames is encoded using the learned parameters. At the decoder side, for each group of n frames, the parameters of the SMIP are first decoded. Then, the group is decoded using the learned parameters.

[0166] The training method of the SMIP described below can be applied to online training or offline training as described above. In combination with Figure 17 A training method of the SMIP is described, the figure illustrates an exemplary map for predicting a current block Y from its neighboring reference samples by the SMIP and each of the 67 traditional intra prediction modes in H.266 / VVC for 4x4 blocks (generalization to 8x8 blocks and 16x16 blocks is straightforward). According to some embodiments of the application, each pair of weight matrix and bias vector (M Figure 17 , b i ) is trained cooperatively with the traditional intra prediction modes in H.266 / VVC by defining a cost function on the map i and minimizing the weight matrix M i and the bias vector b i via backpropagation. The map component, called "SMIP prediction", is Figure 9 , Figure 10 or Figure 12The average, affine transformation and linear interpolation described in the middle, depending on the block size of the weight matrix and bias vector being learned. The graph component called "traditional prediction" is the prediction via each of the 64 traditional intra-predictions in H.266 / VVC, as Figure 18 shown. Figure 18 An exemplary prediction of the current block Y via each of the 67 intra-prediction modes in H.266 / VVC is shown for a 4x4 block (generalization to 8x8 blocks and 16x16 blocks is straightforward).

[0167] In Figure 18 the reference samples are vectorized. Then, the prediction of mode with index i is expressed as the dot product of the vector of reference samples and the mode matrix U i ,i = DC, planar, 2, 3,..., 66. The mode prediction as dot product is the graphical form that is suitable for training. As an example, U2, U 18 , U 34 and U 50 are depicted for a 4x4 block in Figure 19 .

[0168] During training, the SMIP prediction is clipped as Figure 17 shown. Since the pixel intensities of all pairs (reference samples, block) used for training are normalized from [-1.0, 1.0], the minimum value for clipping is -1.0 and the maximum value is 1.0. During backpropagation, the gradient of the output of the clipping function with respect to its input is replaced by the gradient of the identity function, i.e., the identity function For the simulation during the training of the intra-fast selection process in H.266 / VVC, the objective function to be minimized on the weight matrix M i and the bias vector combines the SAD between the current block Y and each mode prediction as well as the signaling cost of each mode estimate (see equation (1): minimization for learning (M i ,b i ).

[0169]

[0170] S = {DC, planar, 2, 3,..., 66, SMIP}

[0171] This factor 511.5 compensates for the normalization of the pixel intensities from [-1.0, 1.0] during training.

[0172] In the above minimization, the expectation is estimated by the average over all pairs (reference samples, block) used for training.

[0173] c j = a j (1 « 15) is the estimated signaling cost of the mode of index j.

[0174] a planar = 2

[0175] a DC = 3

[0176] a2= a 18 = a 50 = a 66 = 5

[0177]

[0178] a SMIP = 1

[0179] g = 0.001

[0180]

[0181] As training progresses, a is smoothly reduced from 0 to -10.0.

[0182] After running the above training algorithm, the learned bias is transformed by equation (2) (a transformation to compensate for the learned bias of the normalization of pixel intensities during training) to compensate for the above normalization of pixel intensities from [-1.0, 1.0] applied during the training phase but not during the testing phase inside H.266 / VVC.

[0183] b i,j = 511.5 - å k M i,jk + b i,j + 1.0) (2)

[0184] M i,kj denotes the coefficient at position (j, k) in the weight matrix M i

[0185] b i,j denotes the coefficient at position j in the bias vector b i

[0186] Each learned weight is then multiplied by b = 1 « p, where depends on the bit-shift involved in the SMIP affine transformation at the time of testing inside H.266 / VVC. Finally, both the resulting weights and the bias are rounded to the nearest integer, yielding the quantized weight matrix and the quantized bias vector ​​​

[0187] After the training method described above, in Figure 20 is shown in and in Figure 21 is shown in and in Figure 22 is shown in and

[0188] Figure 23 A block diagram illustrating an example of a system in which various aspects and embodiments can be implemented is shown. The system 2300 can be implemented as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 2300, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 2300 are distributed across multiple ICs and / or discrete components. The system 2300, in various embodiments, is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and / or output ports. In various embodiments, the system 2300 is configured to implement one or more of the aspects described herein.

[0189] The system 2300 includes at least one processor 2310 configured to execute instructions loaded therein for implementing, for example, the various aspects described herein. Processor 2310 can include embedded memory, input output interface, and various other circuitry known in the art. The system 2300 includes at least one memory 2320 (e.g., a volatile memory device and / or a non-volatile memory device). System 2300 includes a storage device 2340, which can include non-volatile memory and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash, magnetic disk drive, and / or optical disk drive. The storage device 2340 can include an internal storage device, an attached storage device (including a removable storage device and a non-removable storage device), and / or a network accessible storage device, as non-limiting examples.

[0190] System 2300 includes an encoder / decoder module 2330 that is configured, for example, to process data to provide encoded video or decoded video, and the encoder / decoder module 2330 can include its own processor and memory. The encoder / decoder module 2330 represents module(s) that can be included in a device to perform the encoding and / or decoding functions. As is known, a device can include one or both of the encoding and decoding modules. Additionally, the encoder / decoder module 2330 can be implemented as a separate element of system 2300 or can be incorporated in the processor 2310 as a combination of hardware and software as known to those skilled in the art. Program code to be loaded onto processor 2310 or encoder / decoder 2330 to perform the various aspects described in this document can be stored in the storage device 2340 and then loaded onto the memory 2320 for execution by the processor 2310. According to various embodiments, one or more of the processor 2310, the memory 2320, the storage device 2340, and the encoder / decoder module 2330 can store one or more of the various items during the performance of the processes described in this document. These stored items can include, but are not limited to, input video, decoded video or portions of that decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0191] In some embodiments, the processor 2310 and / or the memory within the encoder / decoder module 2330 are used to store instructions and provide working memory for processing needed during encoding or decoding. However, in other embodiments, memory external to the processing device, e.g., the processing device can be the processor 2310 or the encoder / decoder module 2330, is used for one or more of these functions. The external memory can be the memory 2320 and / or the storage device 2340, e.g., dynamic volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, fast external dynamic volatile memory, such as RAM, is used as working memory for video coding and decoding operations, such as working memory for MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard developed by the Joint Video Team experts, JVET).

[0192] As shown in block 2330, input to elements of system 2300 can be provided through various input devices. Such input devices include, but are not limited to: (i) an RF portion that receives a radio frequency (RF) signal transmitted, for example, over the air by a broadcaster, (ii) a component (COMP) input terminal (or set of component input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 23 Other examples, not shown in FIG. 24, include a synthesized video.

[0193] In various embodiments, the input devices of block 2430 have associated respective input processing elements as known in the art. For example, the RF portion can be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or limiting a signal band to a band of frequencies), (ii) downconverting the selected signal, (iii) limiting the band of the downconverted signal to a narrower band of frequencies to select a signal band (which can be referred to as a channel in certain embodiments, for example), (iv) demodulating the downconverted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF portion of various embodiments includes one or more elements to perform these functions, such as frequency selectors, signal selectors, band-limiting elements, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion can include a tuner that performs various of these functions, including downconverting the received signal to a lower frequency (such as an intermediate frequency or a near-baseband frequency), or baseband, for example. In one set-top box embodiment, the RF portion and its associated input processing elements receive an RF signal transmitted through a wired (e.g., cable) medium, and perform frequency selection to a desired frequency band by filtering, downconverting, and filtering again. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements performing similar or different functions. Adding elements can include inserting elements between existing elements, such as amplifiers and analog-to-digital converters. In various embodiments, the RF portion includes an antenna.

[0194] Additionally, the USB and / or HDMI terminals can include respective interface processors for connecting the system 2300 to other electronic devices through USB and / or HDMI connections. It will be appreciated that various aspects of input processing (e.g., Solomon error correction) can be implemented as desired within, for example, a separate input processing IC or processor 2310. Similarly, various aspects of USB or HDMI interface processing can be implemented within a separate interface IC or within the processor 2310 as desired. The demodulated, error corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 2310 and the encoder / decoder 2330, which operate in conjunction with memory and storage elements to process the data streams as desired for presentation on output devices.

[0195] The various elements of the system 2300 can be disposed within an integrated housing. Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangements, such as internal buses known in the art, including Inter-IC (I2C) buses, wiring, and printed circuit boards.

[0196] The system 2300 includes a communication interface 2350 that enables communication with other devices via a communication channel 2360. The communication interface 2350 can include, but is not limited to, a transceiver configured to transmit and to receive data via the communication channel 2360. The communication interface 2350 can include, but is not limited to, a modem or network card, and the communication channel 2360 can be implemented, for example, within wired and / or wireless media.

[0197] In various embodiments, data is streamed or otherwise provided to the system 2300 using a wireless network, such as a Wi-Fi network, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). Wi-Fi signals of these embodiments are received through the communication channel 2360 and the communication interface 2350, which are adapted for Wi-Fi communication. The communication channel 2360 of these embodiments is typically connected to an access point or router, which provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 2300 using a set-top box that delivers data through the HDMI connection of the input block 2330. Still other embodiments provide streamed data to the system 2300 using the RF connection of the input block 2330. As noted above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0198] The system 2300 can provide output signals to various output devices, including a display 2400, speakers 2410, and other peripheral devices 2420. The display 2400 of various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 2400 can be used in a television, a tablet computer, a laptop computer, a cellular phone (mobile phone), or other device. The display 2400 can also be integrated with other components (e.g., as in a smart phone), or separate (e.g., an external monitor for a laptop computer). In examples of various embodiments, the other peripheral devices 2420 include one or more of a digital video disc (or digital versatile disc) (DVR, for both), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 2420 that provide functionality based on the output of the system 2300. For example, a disc player performs the functionality of playing the output of the system 2300.

[0199] In various embodiments, control signals are communicated between the system 2300 and the display 2400, speakers 2410, or other peripheral devices 2420 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 2300 via dedicated connections through respective interfaces 2370, 2380, and 2390. Alternatively, the output devices can be connected to the system 2300 using the communication channel 2360 via the communication interface 2350. The display 2400 and speakers 2410 can be integrated in a single unit with other components of the system 2300 in an electronic device, for example, a television. In various embodiments, the display interface 2370 includes a display driver, for example, a timing controller (T Con) chip.

[0200] For example, if the RF portion of the input 2430 is part of a separate set-top box, the display 2400 and speakers 2410 can alternatively be separate from one or more of the other components. In various embodiments in which the display 2400 and speakers 2410 are external components, the output signals can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0201] The implementations described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation discussed can also be implemented in other forms (for example, an apparatus or program). An apparatus can be implemented in, for example, appropriate hardware, software, and firmware. The methods can be implemented in, for example, an apparatus such as, for example, a processor, which includes e.g., a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants (PDAs), and other devices that facilitate communication of information between end-users.

[0202] According to an aspect of embodiments of the present application, there is provided an apparatus 2100 for video encoding and / or decoding, the apparatus comprising a processor 2310 configured to perform any of the embodiments of the methods described above, and at least one memory 2320, 2340 coupled to the processor.

[0203] As will be appreciated by those skilled in the art, implementations can produce signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method, or data created by one of the described implementations. The signals can be formatted, for example, to carry information deemed sensitive or proprietary by a sender. Such information can be formatted in accordance with a communication protocol, such as a communication protocol used by a wireless network. Such information can be formatted in accordance with a file format, such as a file format used to carry a bitstream of a described embodiment. The information can be formatted in accordance with a standard or proprietary format for the exchange of information over a communications link. The information can be formatted in accordance with a standard or proprietary format for the storage of information on a storage medium. The information can be formatted by encoding the information in accordance with an encoding method.

[0204] Furthermore, any of the methods can be implemented as (independent of or jointly with) a computer program product that comprises computer-executable instructions, which, when executed by a processor, perform a method. The computer program product can be stored in a respective non-transitory computer-readable storage medium of the system 2300, the encoder 100, and / or the decoder 200.

[0205] It is important to note that in some embodiments, one or more elements of the processes can be combined, performed in a different order, or omitted, while still implementing aspects of the present disclosure. Other steps can be performed in parallel as well, where the processor does not wait for the complete completion of one step before starting another step.

[0206] Furthermore, aspects of the present embodiments can take the form of a computer readable storage medium. Any combination of one or more computer readable storage media can be utilized. The computer readable storage medium can take the form of a computer readable program product that is contained in one or more computer readable media and has computer readable program code embodied in it that can be executed by a computer. A computer readable storage medium, as used herein, is considered to be a non-transitory storage medium that is inherently tangibly embodied and is inherently capable of storing information that is static and unchanging. The computer readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[0207] It should be appreciated that the following list, while providing more specific examples of computer readable storage media to which the present disclosure can be applied, is merely illustrative and not an exhaustive list as would be readily appreciated by one of ordinary skill in the art. The example list includes a portable computer diskette, a hard disk, ROM, EPROM, flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0208] According to an aspect of an embodiment of the present application, there is provided a computer readable storage medium carrying a software program comprising program code instructions for executing any embodiment of any method of an embodiment of the present application including the above-mentioned method.

[0209] As detailed above, in the present application, some aspects relate to an apparatus comprising at least one processor adapted to encode image data, or a method for encoding image data, the at least one processor being adapted to or the method comprising: intra-predicting at least one first block of the image data by using an intra-prediction mode, the intra-prediction mode using a first single transform, the first single transform being obtained by considering the first block size. The image data can be, for example, an image frame. According to some embodiments, the intra-prediction mode uses the first single transform to predict a block of the image data having the first block size. According to some embodiments, the first single transform is obtained from a set comprising a single matrix and a single bias vector. According to some embodiments, the intra-prediction mode uses a second single transform to intra-predict a second block of the image data having a size different from the first block size.

[0210] According to some embodiments, when the block size is 4x4 pixels, the single matrix has a 16x4 pixels size and / or the single vector has a 16 pixels size. According to some embodiments, when the block size is 4x8 pixels, 8x4 pixels or 8x8 pixels, the single matrix has a 16x8 pixels size and / or the single vector has a 16 pixels size. According to some embodiments, when the block size is different from 4x4 pixels, 4x8 pixels, 8x4 pixels or 8x8 pixels, the single matrix has a 64x8 pixels size and / or the single vector has a 64 pixels size.

[0211] According to some embodiments, the at least one processor is adapted to or the method comprises encoding in a bitstream information signaling the use of the intra prediction mode. According to some embodiments, a plurality of prediction modes (MPM) is signaled in the bitstream using variable coding length, the variable coding length ordering the prediction modes, and the information is encoded as one of the plurality of prediction modes. According to some embodiments, the plurality of prediction modes contains at least one regular intra prediction mode. According to some embodiments wherein the plurality of prediction modes contains at least one DC element signaling a DC intra prediction mode, the information is coded before, after and / or alternatively to at least one of the DC elements. According to some embodiments wherein the plurality of prediction modes contains at least one plane element signaling a plane intra prediction mode, the information is coded before, after and / or alternatively to at least one of the plane elements.

[0212] According to some embodiments, the at least one processor is adapted to or the method comprises encoding in a bitstream (e.g. at frame level) at least one parameter of the first and / or second single transform. According to some embodiments, the parameter is obtained by training a deep neural network.

[0213] Some aspects of the disclosure relate to a method for encoding image data, wherein a plurality of prediction modes is signaled by the encoding using variable coding length, the method comprising:

[0214] - intra-predicting at least one first block of the image data by using an intra prediction mode, the intra prediction mode using at least one first transform, the at least one first transform being obtained by considering the first block size,

[0215] - encoding in a bitstream information representing the use of the intra prediction mode, the information being encoded as one of the plurality of prediction modes.

[0216] Some embodiments relate to a method for encoding image data, or a device comprising at least one processor adapted to encode image data, the method comprising or the processor being adapted to: predict at least one first block of said image data by using an intra prediction mode or by combining inter prediction and intra prediction, said intra prediction mode using a first single transform to predict a block of said first block size, said intra prediction mode using a second single transform to intra predict a second block of said image data of a size different from said first block size.

[0217] Some aspects of the disclosure relate to a device comprising at least one processor adapted to or a method comprising: reconstructing at least one first block of image data encoded in a bitstream by using an intra prediction mode signaled to use a first single transform, said first single transform considering said first block size.

[0218] According to some embodiments, according to said intra prediction mode, a block of said image data of said first block size is intra predicted by using said first single transform for prediction. According to some embodiments, according to said intra prediction mode, a block of said image data of a size different from said first block size is intra predicted by using a second single transform.

[0219] According to some embodiments, said first single transform is obtained from a set comprising a single matrix and a single bias vector.

[0220] According to some embodiments, when said block size is 4x4 pixels, said single matrix has a 16x4 pixels size and / or said single vector has a 16 pixels size. According to some embodiments, when said block size is 4x8 pixels, 8x4 pixels or 8x8 pixels, said single matrix has a 16x8 pixels size and / or said single vector has a 16 pixels size. According to some embodiments, wherein when said block size is different from 4x4 pixels, 4x8 pixels, 8x4 pixels or 8x8 pixels, said single matrix has a 64x8 pixels size and / or said single vector has a 64 pixels size.

[0221] According to some embodiments, a plurality of prediction modes (MPM) is signaled in the bitstream using a variable coding length, the variable coding length ordering the prediction modes, and the information is encoded as one of the plurality of prediction modes. According to some embodiments, the plurality of prediction modes includes at least one regular intra prediction mode. According to some embodiments, when the plurality of prediction modes includes at least one DC element signaling a DC intra prediction mode, the information is coded before, after and / or in place of at least one of the DC elements. According to some embodiments, when the plurality of prediction modes includes at least one plane element signaling a plane intra prediction mode, the information is coded before, after and / or in place of at least one of the plane elements.

[0222] According to some embodiments, the method comprises encoding at least one parameter of the first and / or second single transform in the bitstream (e.g. at frame level). According to some embodiments, the at least one parameter is obtained by training a deep neural network

[0223] Some aspects of the disclosure relate to a device comprising at least one processor adapted for, or a method comprising: decoding image data, a variable coding length being used for signaling the use of a plurality of prediction modes for encoding the image data, the processor being adapted for or the method comprising:

[0224] - reconstructing at least one first block of the image data by using an intra prediction mode, the intra prediction mode signaling the use of at least one first transform taking into account the first block size,

[0225] - decoding information signaling in the bitstream the use of the intra prediction mode, the information being encoded as one of the plurality of prediction modes.

[0226] Some aspects relate to a non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform any of the above methods of the present invention in any of its embodiments.

[0227] [1] A. M. Kotra, B. Wang, S. Esenlik, H. Gao, J. Chen, M. G. Sarwer, G. W. Hsu, Y. W. Huang, S. M. Lei, L. Li, J. Heo, J. Choi, S. Yoo, J. Lim, S. Kim, A. K. Ramasubramonian, G. Van der Auwera, M. Karczewicz. JVET-L0222 CE3 6.6.1 : a simple 6-MPM list construction with truncated binary coding for non-MPM signaling. Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 12th Meeting: Macau, CN, 3-12 October 2018.

[0228] [2] Jonathan Pfaff, Stallenberger, Michael Philipp Merkle, Philipp Helle, Tobias Hinz, Heiko Schwarz, Detlev Marpe, Thomas Wiegand. JVET-N0217 CE3: affine linear weighted intra prediction (CE3-4.1, CE3-4.2). Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 14th Meeting: Geneva, CH, 19-27 March 2019.

[0229] [3] Luong Pham Van, Geert Van der Auwera, Adarsh K. Ramasubramonian, Vadim Seregin, Marta Karczewicz. JVET-N0302 CE10: CIIP with position-independent weights. Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 14th Meeting: Geneva, CH, 19-27 March 2019.

Claims

1. An apparatus for encoding image data using a first intra-frame prediction mode, the apparatus comprising a processor configured to: An average reference sample set is obtained by averaging the reference samples obtained in the neighborhood of the current block of the image data. The average reference samples in the average reference sample set are weighted using a weight matrix, and a bias vector is applied to the weighted average reference samples to obtain an intermediate value block, wherein the weight matrix and the bias vector are selected exclusively based on the size of the current block; and A predictor for the current block is obtained by linearly interpolating the intermediate value block.

2. A method for encoding image data using a first intra-frame prediction mode, the method comprising: An average reference sample set is obtained by averaging the reference samples obtained in the neighborhood of the current block of the image data. The average reference samples in the average reference sample set are weighted using a weight matrix, and a bias vector is applied to the weighted average reference samples to obtain an intermediate value block, wherein the weight matrix and the bias vector are selected exclusively based on the size of the current block; and A predictor for the current block is obtained by linearly interpolating the intermediate value block.

3. An apparatus for decoding image data using a first intra-frame prediction mode, the apparatus comprising a processor configured to: An average reference sample set is obtained by averaging the reference samples obtained in the neighborhood of the current block of the image data. The average reference samples in the average reference sample set are weighted using a weight matrix, and a bias vector is applied to the weighted average reference samples to obtain an intermediate value block, wherein the weight matrix and the bias vector are selected exclusively based on the size of the current block; and A predictor for the current block is obtained by linearly interpolating the intermediate value block.

4. The device of claim 3, wherein in response to the current block size being 4×4, the weight matrix has a size of 16×4 and / or the deviation vector has a size of 16; in response to the current block size being 4×8, 8×4, or 8×8, the weight matrix has a size of 16×8 and / or the deviation vector has a size of 16; and in response to the current block size being different from 4×4, 4×8, 8×4, or 8×8, the weight matrix has a size of 64×8 and / or the deviation vector has a size of 64.

5. The apparatus of claim 3, wherein the processor is further configured to: construct an ordered list of the most probable intra-prediction modes for the current block, each intra-prediction mode in the ordered list being associated with a variable-length code according to its rank in the ordered list; add the first intra-prediction mode to the ordered list at a first highest position or a second highest position; and decode information from the bitstream based on the ordered list, the information signaling the use of the first intra-prediction mode.

6. The apparatus of claim 3, wherein the processor is further configured to: construct an ordered list of the most probable intra-prediction modes for the current block, each intra-prediction mode in the ordered list being associated with a variable-length code according to its rank in the ordered list; add the first intra-prediction mode to the ordered list at a position immediately before or immediately after the position of the DC prediction mode; and decode information from the bitstream based on the ordered list, the information signaling the use of the first intra-prediction mode.

7. The apparatus of claim 3, wherein the processor is further configured to: construct an ordered list of the most probable intra-prediction modes of the current block, each intra-prediction mode in the ordered list being associated with a variable-length code according to its rank in the ordered list; add the first intra-prediction mode to the ordered list in place of a second intra-prediction mode in the ordered list; associate the first intra-prediction mode with the variable-length code of the second intra-prediction mode, the second intra-prediction mode being a planar intra-prediction mode or a DC intra-prediction mode; and decode information from the bitstream based on the ordered list, the information signaling the use of the first intra-prediction mode.

8. The device of claim 3, wherein when the first intra-frame prediction mode is enabled, the coding mode based on multiple transform selection or based on indivisible quadratic transform is disabled, and syntax elements are not decoded for this coding mode.

9. The device according to claim 3, wherein the first intra-frame prediction mode is a combined intra-frame prediction mode.

10. The apparatus of claim 3, wherein the processor is further configured to decode from the bitstream at least one parameter representing the weight matrix and / or the bias vector.

11. The device of claim 3, wherein the weight matrix and / or the bias vector are learned online or offline using a training method.

12. A method for decoding image data using a first intra-frame prediction mode, the method comprising: An average reference sample set is obtained by averaging the reference samples obtained in the neighborhood of the current block of the image data. The average reference samples in the average reference sample set are weighted using a weight matrix, and a bias vector is applied to the weighted average reference samples to obtain an intermediate value block, wherein the weight matrix and the bias vector are selected exclusively based on the size of the current block; and A predictor for the current block is obtained by linearly interpolating the intermediate value block.

13. The method of claim 12, wherein in response to the current block size being 4×4, the weight matrix has a size of 16×4 and / or the deviation vector has a size of 16; in response to the current block size being 4×8, 8×4, or 8×8, the weight matrix has a size of 16×8 and / or the deviation vector has a size of 16; and in response to the current block size being different from 4×4, 4×8, 8×4, or 8×8, the weight matrix has a size of 64×8 and / or the deviation vector has a size of 64.

14. The method of claim 12, wherein the method further comprises: Construct an ordered list of the most probable intra-prediction modes for the current block, each intra-prediction mode in the ordered list being associated with a variable-length code according to its rank in the ordered list; add the first intra-prediction mode to the ordered list at a first highest position or a second highest position; and encode information in a bitstream based on the ordered list, the information signaling the use of the first intra-prediction mode.

15. The method of claim 12, wherein the method further comprises: Construct an ordered list of the most likely intra-prediction modes for the current block, each intra-prediction mode in the ordered list being associated with a variable-length code according to its rank in the ordered list; add the first intra-prediction mode to the ordered list at a position immediately before or immediately after the position of the DC prediction mode; and encode information in a bitstream based on the ordered list, the information signaling the use of the first intra-prediction mode.

16. The method of claim 12, wherein the method further comprises: Construct an ordered list of the most probable intra-prediction modes for the current block, each intra-prediction mode in the ordered list being associated with a variable-length code based on its rank in the ordered list; add a first intra-prediction mode to the ordered list in place of a second intra-prediction mode in the ordered list; associate the first intra-prediction mode with the variable-length code of the second intra-prediction mode, the second intra-prediction mode being either a planar intra-prediction mode or a DC intra-prediction mode; and encode information in a bitstream based on the ordered list, the information signaling the use of the first intra-prediction mode.

17. The method of claim 12, wherein when the first intra-frame prediction mode is enabled, the coding mode based on multiple transform selection or based on indivisible quadratic transform is disabled, and syntax elements are not decoded for this coding mode.

18. The method of claim 12, wherein the first intra-prediction mode is a combined intra-prediction mode.

19. The method according to claim 12, wherein, The weight matrix and / or the bias vector are learned online or offline using a training method.

20. A non-transitory information storage medium storing program code instructions for implementing the method according to claim 2 or claim 12.

Images