Coding using matrix based intra-prediction and secondary transforms

Abstract

A technique for effectively matrix-based prediction of blocks of a picture is disclosed. One embodiment relates to an apparatus for decoding a predetermined block (18) of a picture using intra prediction, configured to select (602) a predetermined intra prediction mode (604) from a plurality of intra prediction modes (600), including a first set (508) of intra prediction modes including a DC intra prediction mode (506) and an angle prediction mode (500), and a second set (520) of matrix-based intra prediction modes (510). Based on which samples of the predetermined block (18) are predicted according to each of these, a prediction vector (518) is obtained using a matrix vector product (512) between a vector (514) derived from a reference sample (17) in the neighborhood of the predetermined block (18) and a prediction matrix (516) associated with each matrix-based prediction matrix (510). The device is configured to derive a prediction signal (606) for the predetermined block using the predetermined intra prediction mode, and to select one or more subsets (610) of the set (612) of quadratic transformations in a manner that depends on the predetermined intra prediction mode, such that the subset (610) is not empty, when the predetermined intra prediction mode is included in the intra prediction mode of the first set (508) and the predetermined intra prediction mode is included in the matrix-based intra prediction mode (510) of the second set (520).Additionally, the device is configured (614) to derive from the data stream a transformed version (616) of the prediction residual for the predetermined block (18) associated with a spatial domain version (618) of the prediction residual for the predetermined block (18) through a transformation (T) defined by the connection of a predetermined second transformation (Ts) among a subset (610) of the second transformation applied to a subset (622) of the coefficients (620) of the first transformation, where the predetermined intra prediction mode is included in the intra prediction mode of the first set (508) and the predetermined intra prediction mode is included in the matrix-based intra prediction mode (510) of the second set (520). The device is configured (624) to restore the predetermined block (18) using the prediction signal (606) and the prediction residual for the predetermined block (18).

Description

Technology Field

[0001] This application relates to the field of matrix-based intra-prediction and quadratic transformation. Background Technology

[0002] For conventional intra-prediction modes such as planar mode, DC mode, and angular mode, the inseparable quadratic transformation (LFNST) is a tool used to transform the prediction residuals corresponding to these intra-prediction modes. Here, a set S of transformation sets is given such that each conventional intra-prediction mode is associated with one of these transformation sets. Then, the decoder can extract whether to apply a given block LFNST from the bitstream. In this case, one transformation set is provided from set S according to the intra-prediction mode used in the current block, and if this transformation set consists of two or more transformations, T can be extracted from this set from the bitstream used to transform. Then, the transformation T is applied as a quadratic transformation in the decoder, which means it is applied to a subset of the residual transformation coefficients of the separable quadratic transformation. However, the aforementioned quadratic transformation is defined a priori only for conventional intra-prediction modes.

[0003] Therefore, it is desired to provide a concept for rendering picture coding and / or video coding more efficiently to support matrix-based intra prediction (MIP), that is, a second transformation for block-based intra prediction.

[0004] This is achieved by the subject of the independent claim of the present application.

[0005] Further embodiments according to the present invention are defined by the subject matter of the dependent claims of this application.

[0006] According to a first aspect of the invention, the inventors of the present application have found that one problem that arises when attempting to associate a quadratic transformation with a matrix-based intra-prediction mode stems from the fact that providing a specific quadratic transformation for each MIP mode can be too costly in terms of memory requirements for storing additional transformations. According to a first aspect of the present application, this difficulty is overcome by selecting a subset of one or more quadratic transformations from a set of quadratic transformations that includes transformations associated with a matrix-based intra-prediction mode and a non-matrix-based intra-prediction mode. The quadratic transformations in the set of quadratic transformations may be defined for one or more prediction modes, which reduces the memory capacity required for the set of quadratic transformations. Transformations defined for a planar intra-prediction mode and / or transformations defined for a DC intra-prediction mode may also be selectable for a matrix-based intra-prediction mode. A specific selection of a subset of quadratic transformations for matrix-based intra-prediction modes can improve coding efficiency, but the bit stream and consequent signaling costs may increase due to additional syntax elements required in blocks associated with matrix-based intra-prediction modes to indicate the use of quadratic transformations.

[0007] Accordingly, according to the first aspect of the present application, an apparatus for decoding a predetermined block of a picture using intra prediction, i.e., a decoder, is configured to select a predetermined intra prediction mode among a plurality of intra prediction modes, including a first set of intra prediction modes and a second set of matrix-based intra prediction modes, based on a data stream. The first set of intra prediction modes includes a DC intra prediction mode and an angle prediction mode and optionally a planar intra prediction mode. When a matrix-based intra prediction mode among the second set is selected as the predetermined intra prediction mode, the decoder is configured to use a matrix-vector product between vectors derived from reference samples in the neighborhood of the predetermined block and to use a prediction matrix associated with each matrix-based intra prediction mode to obtain a prediction vector, based on the fact that the decoder is configured to predict samples of the predetermined block. If a predetermined intra-prediction mode is included in a first set of intra-prediction modes and if a predetermined intra-prediction mode is included in a second set of matrix-based intra-prediction modes, the decoder is configured to use the predetermined intra-prediction mode to derive a prediction signal for a predetermined block and to select one or more subsets of quadratic transformations from a set of quadratic transformations in a manner that relies on the predetermined intra-prediction mode such that the subset is not empty.

[0008] The first set and the second set define an intra-prediction mode capable of a second transformation. Accordingly, for a predetermined intra-prediction mode selected from the first set or the second set, the decoder is configured to select one or more subsets of second transformations from a set of second transformations specifically associated with the selected predetermined intra-prediction mode. Additionally, the decoder is configured to derive from the data stream a transformed version of the prediction residual for a predetermined block associated with a spatial domain version of the prediction residual for a predetermined block, through a transformation defined by the concatenation of a first transformation and a predetermined second transformation from a subset of second transformations applied to a subset of coefficients of the first transformation, when the predetermined intra-prediction mode is included in the first set of intra-prediction modes and when the predetermined intra-prediction mode is included in the second set of matrix-based intra-prediction modes. For example, the first transformation is set by default, and for example, the predetermined second transformation is selected by the decoder from a subset of second transformations. The decoder can be configured to select a predetermined second transformation from a subset of second transformations by deriving second transformation mark syntax elements from the data stream. The decoder is configured to restore a predetermined block using a prediction signal and a prediction residual for the predetermined block.

[0009] According to a first aspect of the present application, an encoder for encoding a predetermined block of a picture using intra prediction in parallel with a decoder device is configured to select a predetermined intra prediction mode among a plurality of intra prediction modes, comprising a first set of intra prediction modes including a DC intra prediction mode and an angle prediction mode and optionally a planar intra prediction mode, and a second set of matrix-based intra prediction modes, and obtain a prediction vector using a matrix vector product between a vector derived from reference samples in the neighborhood of the predetermined block and a prediction matrix associated with each matrix-based intra prediction mode, based on which sample of the predetermined block is predicted according to each of these. The encoder is configured to signal a predetermined intra prediction mode in a data stream and to derive a prediction signal for the predetermined block using the predetermined intra prediction mode. Additionally, if a predetermined intra-prediction mode is included in a first set of intra-prediction modes and if a predetermined intra-prediction mode is included in a second set of matrix-based intra-prediction modes, the encoder is configured to select one or more subsets of quadratic transformations from a set of quadratic transformations in a manner that depends on the predetermined intra-prediction mode such that the subsets are not empty.

[0010] The encoder is configured to encode into a data stream a transformed version of the prediction residual for a predetermined block associated with a spatial domain version of the prediction residual for a predetermined block, through a transformation defined by the concatenation of a first transformation and a predetermined second transformation from a subset of second transformations applied to a subset of coefficients of the first transformation, when a predetermined intra prediction mode is included in a first set of intra prediction modes and when a predetermined intra prediction mode is included in a second set of matrix-based intra prediction modes. The predetermined block can be reconstructed using the prediction signal and the prediction residual for the predetermined block.

[0011] According to one embodiment, the decoder / encoder is configured to select one or more subsets of secondary transformations from a secondary transformation set in such a manner that each secondary transformation of the secondary transformation set is included in a subset of one or more secondary transformations selected for at least one of the intra prediction modes in the first and second sets. For one or more intra prediction modes in the first set or the second set, each selected subset may include all secondary transformations of the secondary transformation set.

[0012] According to one embodiment, the decoder / encoder is configured to select one or more subsets of quadratic transformations from a set of quadratic transformations in a manner that depends on a predetermined intra-prediction mode, so that each quadratic transformation of each subset of quadratic transformations selected for any matrix-based intra-prediction mode is included by a subset of quadratic transformations selected for at least one intra-prediction mode in a first set that does not belong to an angle prediction mode. The subset selected for the matrix-based intra-prediction mode may include quadratic transformations associated with one or more non-angle intra-prediction modes in the first set, such as a DC intra-prediction mode and / or a plane intra-prediction mode.

[0013] Therefore, a quadratic transformation of a quadratic transformation set may be part of two or more subsets of one or more quadratic transformations. Additional quadratic transformations available only for blocks where the predetermined intra-prediction mode is a matrix-based intra-prediction mode do not need to be included in the quadratic transformation set. For a predetermined block where the predetermined intra-prediction mode is a matrix-based intra-prediction mode, the decoder / encoder is configured to select the same quadratic transformation from the quadratic transformation set, as in the case where the predetermined intra-prediction mode is a first set of non-angle prediction modes. The subset selected for the matrix-based intra-prediction mode may be the same as the subset selected for the first set of intra-prediction modes that do not belong to the angle prediction mode, or may include part of the quadratic transformation of the subset selected for the first set of intra-prediction modes that do not belong to the angle prediction mode, or may include part or all of the quadratic transformations of two or more subsets selected for the first set of intra-prediction modes that do not belong to the angle prediction mode.

[0014] The encoding or decoding method is based on the same considerations as the aforementioned encoding or decoding device. However, the method can be completed with all the features and functions described in relation to the encoding or decoding device. Brief explanation of the drawing

[0015] The drawings are not necessarily scaled to scale and generally focus on explaining the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings: FIG. 1 illustrates an example of encoding into a data stream; FIG. 2 illustrates an embodiment of an encoder; FIG. 3 illustrates an example of picture restoration; FIG. 4 illustrates an example of a decoder; FIG. 5 illustrates a schematic diagram of block prediction for encoding and / or decoding according to one embodiment; FIG. 6 illustrates a matrix operation for predicting a block for encoding and / or decoding according to one embodiment; FIG. 7.1 illustrates a prediction of a block having a reduced sample value vector according to one embodiment; FIG. 7.2 illustrates a prediction of a block using sample interpolation according to one embodiment; FIG. 7.3 illustrates a prediction of a block having a reduced sample value vector in which only some boundary samples are averaged, according to one embodiment; FIG. 7.4 illustrates a prediction of a block having a reduced sample value vector in which a group of four boundary samples is averaged, according to one embodiment; FIG. 8 illustrates a matrix operation performed by a device according to one embodiment; FIGS. 9a and 9b illustrate detailed matrix operations performed by an apparatus according to an embodiment; FIG. 10 illustrates a detailed matrix operation performed by a device using offset and scaling parameters according to one embodiment; FIG. 11 illustrates a detailed matrix operation performed by a device using offset and scaling parameters according to another embodiment; FIG. 12 illustrates a schematic diagram having details of an intra-prediction of a predetermined block using a prediction mode among a list of most likely modes according to one embodiment; FIG. 13 illustrates a schematic diagram of decoding a predetermined block using a second transformation according to one embodiment; FIG. 14 illustrates a schematic diagram of the application of a first transformation and a second transformation according to one embodiment; FIG. 15 illustrates a schematic diagram of a subset selection of a second transformation according to one embodiment; FIG. 16 illustrates a schematic diagram of a predetermined block having a non-zero transformation domain region according to one embodiment; FIG. 17 illustrates a block diagram of a method for decoding a predetermined block according to one embodiment; FIG. 18 illustrates a block diagram of a method for encoding a predetermined block according to one embodiment; and FIGS. 19a through 19d illustrate the syntax element portions of a data stream. Specific details for implementing the invention

[0016] Elements that are identical or equivalent, or elements that have the same or equivalent function, are indicated in the following description by the same or equivalent reference number, even if they occur in different drawings.

[0017] In the following description, a number of details are presented to provide a more complete description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other examples, well-known structures and devices are depicted in block diagram form without detail to avoid obscuring embodiments of the present invention. Additionally, features of different embodiments described below may be combined with one another unless specifically stated otherwise.

[0018] 1 Introduction

[0019] Hereinafter, different examples, embodiments, and aspects of the invention will be described. At least some of these examples, embodiments, and aspects relate particularly to methods and / or apparatus for performing video coding and / or intra-prediction and / or optimizing video delivery (e.g., broadcasting, streaming, file playback, etc.) using linear or affine transformations with adjacent sample reduction, for example, for video applications and / or virtual reality applications.

[0020] Additionally, examples, embodiments, and aspects relate to High Efficiency Video Coding (HEVC) or subsequent operations. Furthermore, additional embodiments, examples, and aspects will be defined by the appended claims.

[0021] It should be noted that any embodiments, examples, and aspects defined by the claims may be supplemented by any of the details (features and functions) described in the following chapter.

[0022] Additionally, the embodiments, examples, and aspects described in the following chapter may be used individually and may also be supplemented by any features included in the features of other chapters or claims.

[0023] Additionally, it should be noted that the individual examples, embodiments, and aspects described in this specification may be used individually or in combination. Accordingly, details may be added to each of the individual aspects without adding details to any other of the examples, embodiments, and aspects.

[0024] Additionally, it should be noted that the present disclosure explicitly or implicitly describes features of a decoding and / or encoding system and / or method.

[0025] Furthermore, the features and functions disclosed herein in relation to the method may also be used in the device. Furthermore, any features and functions disclosed herein in relation to the device may also be used in the corresponding method. In other words, the method disclosed herein may be complemented by any of the features and functions described in relation to the device.

[0026] Additionally, all features and functions described in this specification may be implemented using hardware or software, or a combination of hardware and software, as described in "Alternative Implementation Examples".

[0027] Furthermore, all features described in parentheses ((…)" or "[…]") may be considered optional in some examples, embodiments, or aspects.

[0028] 2 Encoder, Decoder

[0029] In the following, various examples that can help achieve more effective compression when using block-based prediction are described. Some examples achieve high compression efficiency by using a series of intra-prediction modes. The latter may be added to, for example, other empirically designed intra-prediction modes or provided exclusively. And other examples may also use all the singularities just discussed. However, as a variation of these embodiments, intra-prediction may be switched to inter-prediction by using a reference sample of a different picture instead.

[0030] To facilitate understanding of the following examples of the present application, the description begins with the presentation of an encoder and a decoder to which the examples subsequently outlined in the present application can be constructed. FIG. 1 illustrates a device for encoding a picture (10) into a data stream (12) in blocks. The device is indicated by reference numeral 14 and may be a still picture encoder or a video encoder. In other words, when the encoder (14) is configured to encode a video (16) containing the picture (10) into a data stream (12), or when the encoder (14) can exclusively encode the picture (10) into a data stream (12), the picture (10) may be the current picture of the video (16).

[0031] As mentioned, the encoder (14) performs encoding in a block-based or block-oriented manner. To do this, the encoder (14) subdivides the picture (10) into blocks, and the encoder (14) encodes the picture (10) into a data stream (12). Examples of possible subdivisions from the blocks (18) of the picture (10) are described in more detail below. Generally, subdivision can start with multi-tree subdivision into an array of tree blocks, for example, from the entire picture area of ​​the picture (10) or from a pre-partition of the picture (10), and end with blocks (18) of a fixed size or other block sizes, such as an array of blocks arranged in rows and columns by using hierarchical multi-tree subdivision, where these examples do not exclude other possible methods of subdividing the picture (10) into blocks (18).

[0032] Additionally, the encoder (14) is a predictive encoder configured to predictively encode the picture (10) into the data stream (12). For a specific block (18), this means that the encoder (14) determines the prediction signal for the block (18) and encodes the prediction residual, i.e., the prediction error where the prediction signal deviates from the actual picture content within the block (18) into the data stream (12).

[0033] The encoder (14) may support different prediction modes to induce a prediction signal for a specific block (18). In the following examples, an important prediction mode is an intra-prediction mode in which the interior of the block (18) is spatially predicted from adjacent samples of the picture (10) that has already been encoded. The picture (10) is encoded into a data stream (12), and the corresponding decoding procedure may be based on a specific coding order (20) defined between the blocks (18). For example, the coding order (20) may traverse the blocks (18) in a raster scan order, such as traversing each row from left to right and from top to bottom in a row direction. In the case of hierarchical multi-tree based segmentation, a raster scan order may be applied within each hierarchy level, whereby a depth-first traversal order may be applied, that is, leaf nodes within a block of a specific hierarchy level may precede blocks of the same hierarchy level that have the same parent block according to the coding order (20). According to the coding order (20), neighboring and already encoded samples of block (18) may generally be located on one or more sides of block (18). In the example presented here, for instance, neighboring and already encoded samples of block (18) are located on the top and left sides of block (18).

[0034] The intra prediction mode may not be the only mode supported by the encoder (14). For example, if the encoder (14) is a video encoder, the encoder (14) may also support an inter prediction mode in which the block (18) is temporarily predicted from a previously encoded picture of the video (16). This inter prediction mode may be a motion compensation prediction mode in which a motion vector is signaled for the block (18) representing a relative spatial offset of the portion to which the prediction signal of the block (18) is derived as a copy. Additionally or alternatively, if the encoder (14) is a multi-view encoder, other non-intra prediction modes may be used, as well as an inter prediction mode or a non-prediction mode in which the interior of the block (18) is coded as is, i.e., without any prediction.

[0035] Before focusing the description of the present application on the intra-prediction mode, a more specific example of a possible implementation of a possible block-based encoder, namely, as described in relation to FIG. 2, and then two corresponding examples of a decoder according to FIG. 1 and 2 are presented.

[0036] FIG. 2 is a possible implementation of the encoder (14) of FIG. 1, i.e., the encoder is configured to use transform coding to encode the predicted residual, but this is merely an example and the present application is not limited to this type of predicted residual coding. According to FIG. 2, the encoder (14) includes a subtractor (22) configured to subtract the corresponding predicted signal (24) from the current block (18) in units of inbound signals, i.e., pictures (10) or blocks, in order to obtain a predicted residual signal (26) to be encoded into a data stream (12) by the predicted residual encoder (28). The predicted residual encoder (28) consists of a lossy encoding step (28a) and a lossless encoding step (28b). The lossy step (28a) includes a quantizer (930) that receives the predicted residual signal (26) and quantizes a sample of the predicted residual signal (26). As described above, the present example utilizes a transform coding of the predicted residual signal (26), and accordingly, the loss encoding step (28a) includes a transform step (32) connected between the subtractor (22) and the quantizer (30) to transform the spectrally decomposed predicted residual (26) by the quantization of the quantizer (30) occurring in the transformed coefficients that provide the residual signal (26). The transform may be a DCT, DST, FFT, Hadamard transform, etc. The transformed and quantized predicted residual signal (34) is then subjected to lossless coding by a lossless encoding step (28b), which is an entropy coder that entropy-codes the quantized predicted residual signal (34) into a data stream (12).

[0037] The encoder (14) further includes a predicted residual signal recovery step (36) connected to the output of the quantizer (30) to recover the predicted residual signal from the transformed and quantized predicted residual signal (34) in a manner also available in the decoder. That is, it is the quantizer (30) that considers the coding loss. To this end, the predicted residual recovery step (36) includes an inverse quantizer (38) that performs the inverse of the quantization of the quantizer (30), and then an inverse converter (40) that performs the inverse of the transformation performed by the converter (32), such as the inverse of the spectral decomposition, such as the inverse of any of the specific transformation examples mentioned above. The encoder (14) includes an adder (42) that adds the recovered predicted residual signal and the predicted signal (24) output by the inverse converter (40) to output the recovered signal, i.e., the recovered sample. This output is fed to the predictor (44) of the encoder (14), and then the predicted signal (24) is determined based thereon. In relation to FIG. 1, the predictor (44) supports all prediction modes already discussed above. FIG. 2 also includes, where the encoder (14) is a video encoder, an in-loop filter (46) that filters a fully restored picture, which forms a reference picture for the predictor (44) for the inter-predicted block after filtering.

[0038] As already mentioned above, the encoder (14) operates on a block basis. For further explanation, the block basis of interest is to divide the picture (10) into blocks selected from a plurality of intra-prediction modes supported by the predictor (44) or the encoder (14), respectively, and the selected intra-prediction modes are performed individually. However, there may be other types of blocks in which the picture (10) is subdivided. For example, the aforementioned decision regarding whether the picture (10) is inter-coded or intra-coded may be performed at the level of blocks outside of the granularity or blocks (18). For example, the inter / intra-mode decision may be performed at the level of coding blocks in which the picture (10) is subdivided and each coding block is subdivided into prediction blocks. Prediction blocks having encoding blocks determined to use intra-prediction are each subdivided into intra-prediction mode decisions. To this end, for each of these prediction blocks, it is determined which supported intra-prediction mode should be used for each prediction block. These prediction blocks form the blocks of interest (18) in this specification. The prediction blocks within the coding block related to inter-prediction will be processed differently by the predictor (44). They will be inter-predicted from the reference picture by determining the motion vector and copying the prediction signal for this block from the position of the reference picture that the motion vector points to. Other block subdivisions relate to subdivision into a transformation block in the unit where the transformation by the converter (32) and the inverse converter (40) is performed. For example, the transformed block may be the result of further subdivision of the coding block. Naturally, the examples presented herein should not be treated as limiting, and other examples exist. For completeness only, subdivision into the coding block may use, for example, multi-tree subdivision, and prediction blocks and / or transformation blocks may be obtained by further subdividing the coding block using multi-tree subdivision.

[0039] A decoder (54) or device for block-unit decoding suitable for the encoder (14) of FIG. 1 is illustrated in FIG. 3. This decoder (54) performs the opposite of the encoder (14), namely, decoding a picture (10) from a data stream (12) in a block-unit manner and supports multiple intra-prediction modes for this purpose. The decoder (54) may include, for example, a residual provider (156). All other possibilities discussed above in relation to FIG. 1 are also valid for the decoder (54). For this purpose, the decoder (54) may be a still picture decoder or a video decoder, and all prediction modes and prediction possibilities are also supported by the decoder (54). The difference between the encoder (14) and the decoder (54) lies primarily in the fact that the encoder (14) selects coding decisions based on some optimization, such as to minimize some cost function that may depend, for example, on the coding rate and / or coding distortion. One of these coding options or coding parameters may include the selection of an intra prediction mode to be used for the current block (18) among available or supported intra prediction modes. The selected intra prediction mode may be signaled by the encoder (14) for the current block (18) within the data stream (12), and the decoder (54) re-executes the selection using this signaling in the data stream (12) for the block (18). Likewise, subdividing the picture (10) into blocks (18) may be optimized within the encoder (14), and the corresponding subdivision information may be transmitted within the data stream (12), and the decoder (54) restores the subdivision of the picture (10) into blocks (18) based on the subdivision information. To summarize the above, the decoder (54) may be a prediction decoder operating on a block-by-block basis, and in addition to the intra prediction mode, the decoder (54) may support other prediction modes, such as an inter prediction mode, for example, if the decoder (54) is a video decoder.In decoding, the decoder (54) may also use the coding order (20) discussed in relation to FIG. 1, and since this coding order (20) is observed in both the encoder (14) and the decoder (54), the same adjacent samples are available for the current block (18) in both the encoder (14) and the decoder (54). Thus, to avoid unnecessary repetition, the description of the operating mode of the encoder (14) also applies to the decoder (54), for example, as far as the picture (10) is subdivided into blocks, as far as prediction is concerned, and as far as the coding of prediction residuals is concerned. The difference is that the encoder (14) selects some coding options or coding parameters by optimization, and signals or inserts into the data stream (120) the coding parameters derived from the data stream (12) by the decoder (54) to re-execute prediction, subdivision, etc.

[0040] FIG. 4 illustrates a possible implementation of the decoder (54) of FIG. 3, that is, one suitable for the implementation of the encoder (14) of FIG. 1 as shown in FIG. 2. Since many elements of the encoder (54) of FIG. 4 are the same as those occurring in the corresponding encoder of FIG. 2, the same reference numerals provided with apostrophes are used in FIG. 4 to denote these elements. In particular, the adder (42'), optional in-loop filter (46'), and predictor (44') are connected to the prediction loop in the same way as in the encoder of FIG. 2. The recovery applied to the adder (42'), i.e., the inverse quantized and re-converted predicted residual signal, is derived by a sequence of entropy decoders (56) that inversely entropy encoding of the entropy encoder (28b), followed by a residual signal recovery step (36') consisting of an inverse quantizer (38') and an inverse converter (40'), as in the case of the encoding side. The output of the decoder is the recovery of the picture (10). The recovery of the picture (10) may be available directly from the output of the adder (42') or alternatively from the output of the in-loop filter (46'). Some post-filter may be placed at the output of the decoder to apply post-filtering to the recovery of the picture (10) to improve picture quality, but this option is not illustrated in FIG. 4.

[0041] Again, regarding FIG. 4, the description given above for FIG. 2 will also be valid for FIG. 4, except that the encoder performs optimization tasks and related decisions regarding coding options. However, all descriptions regarding block subdivision, prediction, inverse quantization and re-conversion are also valid for the decoder (54) of FIG. 4.

[0042] 3 ALWIP (Affine Linear Weighted Intra Prediction)

[0043] Although some non-limiting examples of ALWIP are discussed in this specification, ALWIP is not always required to implement the technology described in this specification.

[0044] This application relates particularly to an improved block-based prediction mode concept for block-unit picture coding, such as that available in video codecs like HEVC or any subsequent codec of HEVC. The prediction mode may be an intra-prediction mode, but theoretically, the concept described herein may also be transmitted as an inter-prediction mode, where the reference sample is part of another picture.

[0045] A block-based prediction concept is pursued that allows for efficient implementation, such as hardware-friendly implementation.

[0046] This objective is achieved by the subject of the independent claims of the present application.

[0047] Intra-prediction mode is widely used in picture and video coding. In video coding, intra-prediction mode competes with other prediction modes, such as motion-compensated prediction mode and inter-prediction mode. In intra-prediction mode, the current block is predicted based on adjacent samples—that is, samples that are already encoded as far as the encoder side is concerned and already decoded as far as the decoder side is concerned. Neighboring sample values ​​are extrapolated to the current block to form a prediction signal for the current block, along with the prediction residuals transmitted from the data stream for the current block. The better the prediction signal, the lower the prediction residuals; therefore, fewer bits are required to code the prediction residuals.

[0048] To be effective, several aspects must be considered to form an effective framework for intra-prediction in a block-based picture coding environment. For example, the greater the number of intra-prediction modes supported by the codec, the greater the consumption of additional information rate required to send selection signals to the decoder. On the other hand, the set of supported intra-prediction modes must be capable of providing a good prediction signal, that is, a prediction signal resulting in low prediction residuals.

[0049] In the following - as a comparative example or basic example - an apparatus (encoder or decoder) for decoding a picture from a data stream in blocks is disclosed, and the apparatus supports at least one intra prediction mode as an intra prediction signal for a block of a picture of a predetermined size is determined by applying a first template of a sample adjacent to the current block to an affine linear predictor called an affine linear weighted intra predictor (ALWIP).

[0050] The device may have one or more of the following attributes (this may apply, for example, to a method or other technique implemented in a non-transient storage unit that stores instructions that cause the processor to implement the method and / or act as a device when executed by the processor).

[0051] 3.1 Predictors can be complementary to other predictors

[0052] Intra-prediction modes that may form the subject of the implementation improvement described further below may be complementary to other intra-prediction modes of the codec. Thus, they may be complementary to the DC mode, planar mode, or angular prediction mode defined in the HEVC codec for the JEM reference software. The latter three types of intra-prediction modes are referred to hereafter as existing intra-prediction modes. Accordingly, for any block of intra-modes, a flag indicating whether to use one of the intra-prediction modes supported by the device must be parsed by the decoder.

[0053] 3.2 One or more proposed prediction modes

[0055] The device may include one or more ALWIP modes. Therefore, if the decoder knows that it will use one of the ALWIP modes supported by the device, the decoder must parse additional information indicating which of the ALWIP modes supported by the device is being used.

[0056] The signaling of supported modes may have the property that some ALWIP modes require fewer bins for coding than others. Among these modes, the modes requiring fewer bins and those requiring more bins may depend on the information that can be extracted from the already decoded bitstream or may be fixed in advance.

[0057] 4 Some aspects

[0058] FIG. 2 illustrates a decoder (54) for decoding a picture from a data stream (12). The decoder (54) may be configured to decode a predetermined block (18) of the picture. In particular, the predictor (44) may be configured to map a set of P adjacent samples adjacent to the predetermined block (18) using a linear or affine linear transformation [e.g., ALWIP] for a set of Q predicted values ​​for samples of the predetermined block.

[0059] As illustrated in FIG. 5, a predetermined block (18) contains a predicted Q value (which will be the "predicted value" at the end of the operation). If the block (18) has M rows and N columns, Q = M*N. The Q value of the block (18) may be in a spatial domain (e.g., pixels) or a transformation domain (e.g., DCT, Discrete Wavelet Transform, etc.). The Q value of the block (18) can generally be predicted based on a P value taken from neighboring blocks (17a-17c) adjacent to the block (18). The P value of the neighboring blocks (17a-17c) may be located at the position closest to the block (18) (e.g., adjacent). The P value of the neighboring blocks (17a-17c) has already been processed and predicted. The P value is indicated by the value of the part (17'a-17'c) to distinguish it from the block to which it is part (in some examples, 17'b is not used).

[0060] As illustrated in FIG. 6, to perform the prediction, one can work with a first vector (17P) with P entries (each entry is associated with a specific location in the adjacent part 17'a-17'c), a second vector (18Q) with Q entries (each entry is associated with a specific location in the block (18)), and a mapping matrix (17M) (each row is associated with a specific location in the block (18) and each column is associated with a specific location in the neighboring part (17'a-17'c)). Thus, the mapping matrix (17M) predicts the P value of the neighboring part (17'a-17'c) as the value of the block (18) according to a predetermined mode. Thus, the entries of the mapping matrix (17M) can be understood as weighting factors. In the following passage, the adjacent part of the boundary is referred to using the symbol 17a-17c instead of 17'a-17'c.

[0061] Several conventional modes are known in the industry, such as DC mode, planar mode, and 65-direction prediction mode. For example, 67 mode may be known.

[0062] However, it should be noted that different modes referred to herein as linear or affine linear transformations may be used. A linear or affine linear transformation comprises P*Q weighting factors, wherein at least ¼ of the P*Q weighting factors are non-zero weighting values, and for each Q prediction value, a series of P weighting factors associated with each prediction value. This series of P weighting factors forms an envelope that is non-linear in all directions when one is arranged below another according to the raster scan order among the samples of a predetermined block.

[0063] The P position of the neighbor value (17'a-17'c) (template) and the Q position of the adjacent sample (17'a-17'c) can be mapped to the values ​​of the P*Q weighting factors of the matrix (17M). The plane is an example of the envelope of the above series of P weighting factors for the DC transformation (it is a plane for the DC transformation). Since the envelope is clearly planar, it is excluded from the definition of a linear or affine linear transformation (ALWIP). Another example is a matrix resulting from the emulation of an angular mode: the envelope is excluded from the ALWIP definition and, frankly speaking, looks like a hill running obliquely from top to bottom along the direction of the P / Q plane. Although the planar mode and the 65-direction prediction mode have different envelopes, they will be linear in at least one direction, that is, all directions for the DC exemplified, for example, the direction of the hill for the angular mode.

[0064] Conversely, the envelope of a linear or affine transformation is not linear in all directions. It is understood that this type of transformation may be optimal for performing predictions for block (18) in some cases. It is noted that it is desirable for at least 1 / 4 of the weighting factors to be different from 0 (i.e., at least 25% of the P*Q weighting factors are not 0).

[0065] The weighting factors may not be related to each other according to normal mapping rules. Therefore, the matrix (17M) may not have an obvious recognizable relationship between the values ​​of its items. For example, the weighting factors cannot be described by analytical or differential functions.

[0066] In the examples, the ALWIP transformation ensures that the average of the maximum cross-correlation between the first series of weighting factors associated with each predicted value, whichever leads to a higher maximum value, and the second series of weighting factors associated with predicted values ​​other than each predicted value, or the inverted versions of the second series of weighting factors, whichever leads to a larger maximum value, is lower than a predetermined threshold (e.g., 0.2, 0.3, 0.35, or 0.1, or a threshold in the range of 0.05 to 0.035). For example, for each pair (i1, i2) of rows in the ALWIP matrix (17M), the cross-correlation can be calculated by multiplying the P-value of the i1th row by the P-value of the i2nd row. For each cross-correlation obtained, a maximum value can be obtained. Thus, an average can be calculated for the entire matrix (17M) (i.e., the maximum value of the cross-correlation across all combinations is averaged). After that, the threshold may be, for example, 0.2 or 0.3 or 0.35 or 0.1, or a threshold value in the range of, for example, 0.05 to 0.035.

[0067] P adjacent samples of blocks (17a-17c) may be located along a one-dimensional path extending along the boundaries of a predetermined block (18) (e.g., 18c, 18a). For each Q prediction value of a predetermined block (18), a series of P weighting factors for each prediction value may be aligned in a manner that traverses the one-dimensional path in a predetermined direction (e.g., from left to right, from top to bottom, etc.).

[0068] In examples, the ALWIP matrix (17M) may be non-diagonal or non-block diagonal.

[0069] An example of an ALWIP matrix (17M) for predicting a 4x4 block (18) from four already predicted adjacent samples may be as follows:

[0070] {

[0071] { 37, 59, 77, 28},

[0072] { 32, 92, 85, 25},

[0073] { 31, 69, 100, 24},

[0074] { 33, 36, 106, 29},

[0075] { 24, 49, 104, 48},

[0076] { 24, 21, 94, 59},

[0077] { 29, 0, 80, 72},

[0078] { 35, 2, 66, 84},

[0079] { 32, 13, 35, 99},

[0080] { 39, 11, 34, 103},

[0081] { 45, 21, 34, 106},

[0082] { 51, 24, 40, 105},

[0083] { 50, 28, 43, 101},

[0084] { 56, 32, 49, 101},

[0085] { 61, 31, 53, 102},

[0086] { 61, 32, 54, 100}

[0087] }.

[0088] (Here, {37, 59, 77, 28} is the first row, {32, 92, 85, 25} is the second row, and {61, 32, 54, 100} is the 16th row of matrix (17M).) Matrix (17M) has dimensions of 16x4 and contains 64 weight factors (resulting in 16*4 = 64). This is because matrix (17M) has dimensions of QxP, where Q = M*N is the number of samples in block (18) to be predicted (block (18) is a 4x4 block), and P is the number of samples already predicted. Here, M = 4, N = 4, Q = 16 (resulting in M*N = 4*4 = 16), P = 4. The matrix is ​​non-diagonal and non-block diagonal and is not explained by any specific rule.

[0089] As can be seen, less than 1 / 4 of the weight factors are 0 (in the case of the matrix shown above, one of the 64 weight factors is 0). The envelope formed by these values ​​forms a non-linear envelope in all directions when one is placed below another in the raster scan order.

[0090] Although the above description has been primarily explained with reference to a decoder (e.g., decoder (54)), this can also be done with an encoder (e.g., encoder (14)).

[0091] In some examples, for each block size (in a set of block sizes), the ALWIP transforms of the intra-prediction modes within a second set of intra-prediction modes for each block size are different from each other. Additionally or alternatively, the number of the second set of intra-prediction modes for the block sizes in the set of block sizes may match, but the associated linear or affine linear transforms of the intra-prediction modes within the second set of intra-prediction modes for different block sizes may not be transferable to each other by scaling.

[0092] In some examples, the ALWIP transformation can be defined to have "nothing to share" with the existing transformation (for example, even if the ALWIP transformation is mapped through one of the mappings above, it may have "nothing" to share with that existing transformation).

[0093] In some examples, ALWIP mode is used for both the luminar and chroma components, but in other examples, ALWIP mode is used for the luminar component but not for the chroma component.

[0094] 5 Encoder Speeded up Affin Linear weight Intra prediction mode (e.g., Test CE3-1.2.1)

[0095] 5.1 Description of the method or device

[0096] The affine linear weighted intra-prediction (ALWIP) mode tested in CE3-1.2.1 may be identical to that proposed in JVET-L0199 under test CE3-2.2.2, except for the following changes:

[0097] * Multiple Reference Line (MRL) intra-prediction, specifically harmony using encoder estimation and signaling, i.e., MRL is not combined with ALWIP, and the transmission of MRL indices is restricted to non-ALWIP blocks.

[0098] * Subsampling is mandatory for all blocks WxH ≥ 32x32 (previously optional for 32x32), and consequently, additional testing and transmission of subsampling flags in the encoder are removed.

[0099] * ALWIP for 64xN and Nx64 blocks (including N ≤ 32) is added by downsampling to 32xN and Nx32, respectively, and applying the corresponding ALWIP mode.

[0100] In addition, Test CE3-1.2.1 includes the following encoder optimizations for ALWIP:

[0101] * Combined mode estimation: The existing and ALWIP modes use a shared Hadamard candidate list for the total RD estimation, that is, ALWIP mode candidates are added to the same list as the existing (and MRL) mode candidates based on Hadamard costs.

[0102] With additional optimization to reduce the total number of RD tests, EMT Intra Fast and PB Intra Fast are supported for the combined mode list.

[0103] * Following the same approach as the existing mode, only the MPMs of the left and top blocks available for full RD estimation for ALWIP are added to the list.

[0104] 5.2 Complexity Assessment

[0105] In test CE3-1.2.1, excluding calculations that call the discrete cosine transform, up to 12 multiplications per sample are required to generate the prediction signal. Additionally, a total of 136,492 parameters are required, each 16 bits. This corresponds to 0.273 megabytes of memory.

[0106] 5.3 Experimental Results

[0107] Test evaluations were performed according to the common test conditions JVET-J1010 [2] for intra-only (AI) and random access (RA) configurations using VTM software version 3.0.1. The simulations were performed on an Intel Xeon cluster (E5-2697A v4, AVX2 on, Turbo Boost off) with a Linux OS and GCC 7.2.1 compiler.

[0108] Y U V enc time Dec time Class A1 -2,08% -1,68% -1,60% 155% 104% Class A2 -1,18% -0,90% -0,84% 153% 103% Class B -1,18% -0,84% -0,83% 155% 104% Class C -0,94% -0,63% -0,76% 148% 106% Class E -1,71% -1,28% -1,21% 154% 106% Overall -1,36% -1,02% -1,01% 153% 105% Class D -0,99% -0,61% -0,76% 145% 107% Class F (Optional) -1,38% -1,23% -1,04% 147% 104%

[0109] CE3-1.2.1 results for VTM AI configuration

[0110] Y U V enc time Dec time Class A1 -1,25% -1,80% -1,95% 113% 100% Class A2 -0,68% -0,54% -0,21% 111% 100% Class B -0,82% -0,72% -0,97% 113% 100% Class C -0,70% -0,79% -0,82% 113% 99% Class E 　 　 　 　 Overall -0,85% -0,92% -0,98% 113% 100% Class D -0,65% -1,06% -0,51% 113% 102% Class F (Optional) -1,07% -1,04% -0,96% 117% 99%

[0111] CE3-1.2.1 results for VTM RA configuration 5 .4 complexity Reduced Affin Linear weighted intra-prediction (e.g., test CE3-1.2.2)

[0112] The technology tested in CE2 is related to the "affine linear weighted intra-prediction" described in JVET-L0199 [1], but simplifies it in terms of memory requirements and computational complexity.

[0113] There may be three different sets of prediction matrices (e.g., S0, S1, S2, see below) and bias vectors containing all block shapes (e.g., for providing offset values). As a result, the number of parameters is reduced to 14,400 10-bit values, which is less memory than that stored in a 128x128 CTU.

[0114] The input and output sizes of the predictor variables are further reduced. Additionally, instead of transforming the boundaries through the DCT, averaging or downsampling can be performed on the boundary samples, and linear interpolation can be used instead of the inverse DCT when generating the prediction signal. Consequently, up to 4 multiplications per sample may be required to generate the prediction signal.

[0115] 6 examples

[0116] This specification describes a method for performing some predictions (e.g., see FIG. 6) using ALWIP prediction.

[0117] In principle, referring to FIG. 6, to obtain the Q = M*N value of the MxN block (18) to be predicted, the multiplication of the Q*P samples of the QxP ALWIP prediction matrix (17M) and the P samples of the Px1 adjacency vector (17P) must be performed. Thus, to obtain each of the Q = M*N values ​​of the MxN block (18) to be predicted in general, at least the multiplication of the P = M+N value is required.

[0118] Such multiplication has a very undesirable effect. The dimension P of the boundary vector (17P) generally depends on the number M+N of boundary samples (bins or pixels) (17a, 17c) adjacent (e.g., adjacent) to the MxN block (18) to be predicted. This means that if the size of the block (18) to be predicted is large, the number of boundary pixels (17a, 17c) M+N increases accordingly, and consequently, the dimension P = M+N of the Px1 boundary vector (17P), the length of each row of the QxP ALWIP prediction matrix (17M), and the number of multiplications required increase accordingly (generally Q = M*N = W*H, where W (width) is another symbol of N and H (height) is another symbol of M; if the boundary vector is formed by samples from only one row and / or one column, P = M+N = H+W).

[0119] This problem is generally exacerbated by the fact that multiplication is typically a power-consuming operation in microprocessor-based systems (or other digital processing systems). Performing a large number of multiplications on a very large number of samples across a large number of blocks can generally be said to result in an unwanted waste of computational power.

[0120] Therefore, it is desirable to reduce the number of multiplications Q*P required to predict MxN blocks (18).

[0121] It will be understood that by intelligently selecting an operation that is easier to process as an alternative to multiplication, the computational power required for each intra prediction of each block (18) to be predicted can be reduced in some way.

[0122] In particular, with reference to FIGS. 7.1 to 7.4, an encoder or decoder uses a plurality of adjacent samples (e.g., 17a, 17c),

[0123] A plurality of adjacent samples (e.g., 17a, 17c) are reduced (e.g., by averaging or downsampling) to obtain a reduced set of sample values ​​with fewer samples compared to a plurality of adjacent samples (e.g., in step 811),

[0124] By applying the set of reduced sample values ​​to a linear or affine linear transformation to obtain predicted values ​​for a predetermined sample of a predetermined block (e.g., in step 812),

[0125] It is understood that a predetermined block of the picture (e.g., 18) can be predicted.

[0126] In some cases, the decoder or encoder can also derive predicted values ​​for additional samples of a predetermined block through interpolation based on the predicted values ​​for a predetermined set of samples and a plurality of adjacent samples. Thus, an upsampling strategy can be achieved.

[0127] For example, by performing some averaging on the samples of the boundary (17) (e.g., in step 811), a reduced set (102) of samples with a reduced number of samples (Figs. 7.1 to 7.4) can be obtained (at least one of the samples of the reduced number of samples (102) may be the average of two samples of the original boundary samples or a selection of the original boundary samples). For example, if there are P = M+N samples at the original boundary, the reduced set of samples is M red <M 및 N red <N 중 하나인, P red = Mred +N red Since it can have, P red <P가 된다. 따라서, (예를 들어, 단계 812b에서) 실제로 예측에 사용될 경계 벡터(17P)는 Px1 항목이 아니라, P red It has item x1, and in this case P red <P이다. 유사하게, 예측을 위해 선택된 ALWIP 예측 행렬(17M)은 QxP 치수가 아니라 QxP red (or Q red xP red , see below) has, and in this case at least P red <P이기 때문에 행렬의 요소 수는 감소된다((M red <M 및 N red <N 중 적어도 하나에 의해).

[0128] In some examples (e.g., FIGs. 7.2, 7.3), M' red < M and / or N' red If < N, the block obtained by ALWIP (in step 812) is of size M' red × N' red If it is a reduced block having, it is possible to further reduce the number of multiplications (i.e., the number of samples directly predicted by ALWIP is less than the number of samples in the block (18) actually predicted). Therefore, Q red = M' red * N' red If you set it, Q*P red Instead of the product, Q red * P red ALWIP prediction can be obtained using the product (Q red* P red < Q*P red (If < Q*P). This multiplication is of size M' red × N' red Predicts reduced blocks. Nevertheless, reduced M' red × N' redIt is possible to perform upsampling from the prediction block to the final MxN prediction block (e.g., obtained by interpolation) (e.g., in the subsequent step 813).

[0129] This technique reduces the number of products (Q) of matrix multiplication. red * P red or Q*P red ...is included, but since both the initial reduction (e.g., averaging or downsampling) and the final transformation (e.g., interpolation) can be performed by reducing or avoiding multiplication, this may be desirable. For example, downsampling, averaging, and / or interpolation can be performed by employing binary operations that do not consume much computational power, such as addition and shifting (e.g., in steps 811 and / or 813).

[0130] In addition, addition is a very easy operation that can be performed easily without much computational effort.

[0131] The shifting operation can be used, for example, to average two boundary samples to obtain the final prediction block and / or to interpolate two samples (support values) of the reduced prediction block (or taken from the boundaries). (Two sample values ​​are required for interpolation. There are always two predetermined values ​​within a block. However, to interpolate samples along the left and top boundaries of a block, as shown in FIG. 7.2, there is only one predetermined value, so the boundary samples are used as support values ​​for interpolation.)

[0132] The following two-step procedure can be used:

[0133] After summing the values ​​of the two samples;

[0134] Halves the sum value (e.g., shifting to the right).

[0135] Or you can do the following:

[0137] * Half each sample first (e.g., shifting left);

[0138] Sum the values ​​of the two samples divided in half.

[0139] Since only one sample needs to be selected from a sample group (e.g., adjacent samples), downsampling (e.g., in step 811) can be performed more easily.

[0140] Therefore, we can now define techniques to reduce the number of multiplications to be performed. Some of these techniques may be based specifically on one or more of the following principles:

[0141] Even if the size of the predicted block (18) is actually MxN, the block can be reduced (in at least one of the two dimensions) to a reduced size Q red xP red ALWIP matrix (Q) having red = M' red* N' red , P red = N red + M red , at this time M' red < M and / or N' red <N 및 / 또는 M red <M 및 / 또는 N red <N)이 적용될 수 있다. 따라서 경계 벡터(17P)는 크기 P red With x1, this is P red <P 곱셈 만을 의미한다 (이 때, P red = M red + N red and P = M + N).

[0142] P red The x1 boundary vector (17P) is, for example:

[0143] By downsampling (e.g., selecting only some samples of the boundary); and / or

[0144] By averaging multiple samples of the boundary (which can be easily obtained through addition and shifting without multiplication), it can be easily obtained from the original boundary (17).

[0145] Additionally or alternatively, instead of predicting all Q = M*N values ​​of the block (18) to be predicted by multiplication, only the block reduced to a reduced dimension can be predicted (e.g., Q red = M' red* N' red At this time M' red < M and / or N' red < N). The remaining samples of the predicted block (18) are, for example, Q red The remaining QQ to predict the sample red It will be obtained by interpolation using the support value for the value.

[0146] According to one example illustrated in FIG. 7.1, a 4x4 block (18) (M = 4, N = 4, Q = M*N = 16) is to be predicted, and a sample (17) consisting of neighbor samples (17a) (vertical column with 4 already predicted samples) and neighbor samples (17c) (horizontal row with 4 already predicted samples) has already been predicted in a previous iteration (neighbor samples (17a and 17c) collectively denoted as 17). A priori, using the equation shown in FIG. 6, the prediction matrix (17M) must be a QxP = 16x8 matrix (due to Q = M*N = 4*4 and P = M + N = 4 + 4 = 8), and the boundary vector (17P) must have an 8x1 dimension (due to P = 8). However, this requires performing 8 multiplications for each of the 16 samples of the predicted 4x4 block (18), and thus a total of 16 * 8 = 128 multiplications must be performed. (The average number of multiplications per sample serves as an evaluation of computational complexity. In the case of conventional intra-prediction, 4 multiplications per sample are required, which increases the associated computational cost. Therefore, it is possible to use this as an upper limit for ALWIP, which ensures that the complexity is reasonable and does not exceed the complexity of conventional intra-prediction.)

[0147] Nevertheless, by using the present technique, the number of samples (17a and 17c) adjacent to block (18) to be predicted in step 811 is P to P red<P로 감소시킬 수 있음이 이해된다. 특히, 서로 인접한 경계 샘플(17a, 17c)을 평균 (예를 들어, 도 7.1에서 100)하여, 두 개의 수평 행과 두 개의 수직 열로 감소된 경계(102)를 얻을 수 있음이 이해된다. 따라서 블록(18)으로서 동작하는 것은 2x2 블록이었다 (감소된 경계는 평균값에 의해 형성됨). 또는 다운샘플링을 수행하므로, 행(17c)에 대해 2 개의 샘플을 선택하고 열(17a)에 대해 2 개의 샘플을 선택하는 것이 가능하다. 따라서, 4 개의 원래 샘플을 갖는 대신에, 수평 행(17c)은 2 개의 샘플 (예를 들어, 평균 샘플)을 갖는 것으로 처리되는 반면, 원래 4 개의 샘플을 갖는 수직 열(17a)은 2 개의 샘플(예를 들어, 평균 샘플)을 갖는 것으로 처리된다. 또한, 행(17c)와 열(17a)를 각각 2 개의 샘플 그룹(110)에서 세분화한 후, 하나의 단일 샘플이 유지된다 (예를 들어, 그룹(110)의 샘플들의 평균 또는 그룹(110)의 샘플들 중 간단한 선택). 따라서, 샘플 값의 소위 감소된 세트(102)는 단지 4 개의 샘플(M red = 2, N red = 2, P red = M red + N red = 4, P red <P)을 갖는 세트(102)에 의해 획득된다.

[0148] It has been understood that operations (e.g., averaging or downsampling (100)) can be performed without performing too many multiplications at the processor level: the averaging or downsampling (100) performed in step 811 can be simply achieved by simple and computationally non-power-consuming operations such as addition and shifting.

[0149] At this point, it is understood that it is possible to apply a set of reduced sample values ​​(102) to a linear or affine linear (ALWIP) transformation (19) (e.g., using a prediction matrix such as the matrix (17M) of FIG. 6). In this case, the ALWIP transformation (19) directly maps the four samples (102) to the sample values ​​(104) of block (18). In this case, interpolation is not required.

[0150] In this case, the ALWIP matrix (17M) has dimensions QxP red = 16x4: This follows the fact that all Q = 16 samples of the predicted block (18) are obtained directly by ALWIP multiplication (no interpolation needed).

[0151] Therefore, in step 812a, dimension QxP red A suitable ALWIP matrix (17M) having is selected. The selection may be based, for example, at least partially on signaling from the data stream (12). The selected ALWIP matrix (17M) is also A k It can be represented as, where k can be understood as an index, which can be signaled in the data stream (12) (in some cases, the matrix It is also indicated as (see below). The selection may be performed according to the following method: for each dimension (e.g., height / width pair of block (18) to be predicted), an ALWIP matrix (17M) is selected from, for example, one of three matrix sets S0, S1, S2. (Each of the three sets S0, S1, S2 may group multiple ALWIP matrices (17M) of the same dimension, and the ALWIP matrix to be selected for prediction will be one of them).

[0152] In step 812b, the selected QxP red ALWIP matrix(17M)(A k Also indicated as) and P red Multiplication between x1 boundary vectors (17P) is performed.

[0153] In step 812c, the offset value (e.g., b k ) can be added to all acquired values ​​(104) of the vector (18Q) acquired by ALWIP, for example. Offset value (b k or in some cases Indicated by (see below) is a specific selected ALWIP matrix (A k It can be associated with ) and based on an index (e.g., can be signaled in a data stream (12)).

[0154] Therefore, a comparison between using the present technology and not using the present technology is resumed below:

[0155] If this technology is not available:

[0156] Predicted block (18), this block has dimensions M = 4, N = 4;

[0157] Predicted M*N = 4*4 = 16 values;

[0158] P = M + N = 4 + 4 = 8 boundary samples

[0159] Multiplying P = 8 for the predicted Q = 16 value,

[0160] P*Q = 8*16 = 128 Total number of multiplications;

[0161] By using this technology, you have the following:

[0162] Predicted block (18), this block has dimensions M = 4, N = 4;

[0163] Predicted Q = M*N = 4*4 = 16 values ​​at termination;

[0164] Reduced dimension of boundary vector: P red = M red +N red = 2+2 = 4;

[0165] P for each Q = 16 value to be predicted by ALWIP red = 4 multiplication;,

[0166] P red*Q = 4 * 16 = a total of 64 multiplications (half of 128)

[0167] The ratio between the number of multiplications and the number of final values ​​to be obtained is P red * Q / Q = 4, i.e., P = 8 for each predicted sample, half of the multiplication.

[0169] As you can understand, it is possible to obtain the appropriate value in step 812 by relying on simple and computationally inefficient operations such as averaging (and in some cases, addition and / or shifting and / or downsampling).

[0170] Referring to FIG. 7.2, the block (18) to be predicted is an 8x8 block (M = 8, N = 8) with 64 samples in this specification. Here, a priori, the prediction matrix (17M) must have a size QxP = 64x16 (Q = 64 due to Q = M*N = 8*8 = 64, M = 8, N = 8 and P = M+N = 8 + 8 = 16). Thus, a priori, since P = 16 multiplications are required for each of the Q = 64 samples of the 8x8 block (18) to be predicted, 64*16 = 1024 multiplications are reached for the entire 8x8 block (18).

[0171] However, as can be seen in FIG. 7.2, a method (820) may be provided that uses only 8 values ​​instead of using all 16 samples of the boundary (e.g., 4 in the horizontal boundary row (17c) and 4 in the vertical boundary column (17a) between the original samples of the boundary). Instead of 8 samples in the boundary row (17c), 4 samples may be used (e.g., a selection of one of 2x2 average and / or 2 samples). Thus, the boundary vector is not a Px1 = 16x1 vector, but P red x1 is an 8x1 vector (P red = M red +N red = 4+4). Originally, P = 16 samples instead of P redIt is understood that it is possible to form a reduced set of sample values ​​(102) by selecting samples from the horizontal row (17c) and samples from the vertical column (17a) to have only boundary values ​​of = 8, or by averaging them (e.g., 2 x 2). This reduced set (102) is Q (instead of Q = M*N = 8*8 = 64). red = M red *N red = 4*4 = 16 samples, which enables obtaining a reduced version of block (18). M red xN red = An ALWIP matrix can be applied to predict a 4x4 block. A reduced version of the block (18) includes samples marked in gray in the method (106) of FIG. 7.2: the samples marked in gray squares (including samples 118' and 118") are Q obtained in processing step 812. red A 4x4 reduction block having a value of 16 is formed. The 4x4 reduction block is obtained by applying a linear transformation 19 in processing step 812. After obtaining the value of the 4x4 reduction block, the value of the remaining sample (the sample indicated as a white sample in scheme (106)) can be obtained, for example, by interpolation.

[0172] With respect to the method (810) of FIG. 7.1, this method (820) is, for example, the remaining QQ of the MxN = 8x8 block (18) to be predicted by interpolation. red = 64-16 = 48. Step 813, which derives the predicted value for the sample (white square), can be additionally included. The remaining QQ red = 64-16 = 48 samples are Q directly obtained through interpolation red= can be obtained from 16 samples (for example, interpolation may use the values ​​of boundary samples). As can be seen in FIG. 7.2, samples (118' and 118") are obtained in step 812 (indicated by gray squares), while sample (108') (the midpoint between samples (118' and 118") and indicated by white squares) is obtained in step 813 by interpolation between samples (118' and 118"). It will be understood that interpolation may also be obtained by operations similar to averaging, such as shifting and addition. Thus, in FIG. 7.2, the value (108') can generally be determined as the midpoint (which may be the average) between the value of sample (118') and the value of sample (118").

[0173] By performing interpolation, it is also possible to reach a final version of the MxN = 8x8 block (18) based on the multi-sample values ​​shown in 104 in step 813.

[0174] Therefore, the comparison between using and not using this technology is as follows:

[0175] If this technology is not available:

[0176] Predicted block (18), this block has dimensions M = 8, N = 8;

[0177] In the predicted block (18), Q = M*N = 8*8 = 64 samples;

[0178] At the boundary (17), P = M + N = 8 + 8 = 16 samples;

[0179] P = 16 multiplication for each predicted Q = 64 value,

[0180] P*Q = 16*64 = 1028 Total number of multiplications

[0181] The ratio between the number of multiplications and the number of final values ​​to be obtained is P*Q / Q = 16.

[0182] According to the present technology:

[0183] A predicted block (18) having dimensions M = 8, N = 8,

[0184] The predicted value of Q = M*N = 8*8 = 64 at the end;

[0185] However, the Q to be used red xP red ALWIP matrix, P red = M red + N red , Q red = M red * N red , M red = 4, N red = 4

[0186] P on the boundary red = M red +N red = 4+4 = 8 samples, P red <P

[0187] Predicted Q of 4x4 reduced blocks red = P for each of the 16 values red =8 multiplication (formed as a gray square in method (106),

[0188] P red* Q red = 8 * 16 = 128 Total number of multiplications (much less than 1024)

[0189] The ratio between the number of multiplicatives and the number of final values ​​is P red * Q red / Q = 128 / 64 = 2 (much less than the 16 obtained without this skill)

[0190] Therefore, the technology presented in this specification consumes eight times less power than the prior technology.

[0191] FIG. 7.3 illustrates another example (which may be based on method (820)), where the block (18) to be predicted is a rectangular 4x8 block (M = 8, N = 4) with Q = 4*8 = 32 samples to be predicted. The boundary (17) is formed by a horizontal row (17c) with N = 8 samples and a vertical column (17a) with M = 4 samples. Thus, a priori, the boundary vector (17P) has dimensions Px1 = 12x1, but since the predicted ALWIP matrix must be QxP = 32x12, Q*P = 32*12 = 384 multiplications are required.

[0192] However, it is possible to obtain a horizontal row reduced to only 4 samples (e.g., averaged samples) by, for example, averaging or downsampling at least 8 samples of the horizontal row (17c). In some examples, the vertical column (17a) will be kept as is (e.g., without averaging). The dimensions of the reduced boundary overall are P red = 8 and P red <P이다. 따라서, 경계 벡터(17P)는 치수 P red x1 will have 8x1. The ALWIP prediction matrix (17M) has dimensions of M*N red* P red It is a matrix of size Q, where 4*4*8 = 64. The 4x4 reduced block obtained directly in Subject Step 812 (formed by the gray columns of Schema 107) is of size Q red = M*N red = 4*4 = 16 samples will be obtained (instead of Q = 4*8 = 32 for the original 4x8 block (18) to be predicted). When the reduced 4x4 block is obtained by ALWIP, the offset value b k(Step 812c) can be added and interpolation can be performed in Step 813. As can be seen in Step 813 of FIG. 7.3, the reduced 4x4 block is expanded into a 4x8 block (18), where the value (108') not obtained in Step 812 is obtained in Step 813 by interpolating the value (118' and 118") obtained in Step 812 (gray square).

[0193] Therefore, the comparison between using and not using this technology is as follows.

[0195] If this technology is not available:

[0196] Predicted block (18), this block has dimensions M = 4, N = 8,

[0197] Predicted Q = M*N = 4*8 = 32 values;

[0198] At the boundary, P = M + N = 4 + 8 = 12 samples;

[0199] P = 12 multiplication for each predicted Q = 32 value,

[0200] P*Q = 12*32 = 384 Total number of multiplications

[0201] The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q = 12.

[0202] According to the present technology:

[0203] Predicted block (18), this block has dimensions M = 4, N = 8

[0204] Predicted Q at termination = M*N = 4*8 = 32 values

[0205] However, Q red xP red = A 16x8 ALWIP matrix is ​​used, M = 4, N red = 4, Q red = M*N red = 16, P red = M + N red = 4 + 4 = 8

[0206] P at the boundary red = M + N red = 4 + 4 = 8 samples, P red <P

[0207] Predicted reduced block Q red = For each of the 16 values, P red = 8 multiplication,

[0208] Q red* P red = 16 * 8 = 128 Total number of products (less than 384)

[0209] The ratio between the number of multiplications and the number of final values ​​to be obtained is P red * Q red / Q = 128 / 32 = 4 (much less than the 12 obtained without this skill).

[0210] Therefore, using this technology reduces computational effort by one-third.

[0211] FIG. 7.4 predicts the case of a block (18) with dimensions MxN = 16x16 and N = 16*16 = 256 values ​​predicted at termination, where P = M + N = 16 + 16 = 32 boundary samples. This leads to a prediction matrix with dimensions QxP = 256x32, which means 256*32 = 8192 multiplications.

[0212] However, by applying method (820), the number of boundary samples in step 811 can be reduced, for example, from 32 to 8 (e.g., by averaging or downsampling). For example, for every group (120) of four consecutive samples in row (17a), one single sample (e.g., selected from four samples or the average of the samples) remains. Also, for every group of four consecutive samples in column (17c), one single sample (e.g. selected from four samples or the average of the samples) remains.

[0213] Here, the ALWIP matrix (17M) is Q red xP red= is a 64x8 matrix: This is P red = 8 is derived from the fact that it is selected (using the average of 32 samples of the boundary or 8 selected samples) and the fact that the reduced block to be predicted in step 812 is an 8x8 block (in method (109), there are 64 gray squares).

[0214] Accordingly, when 64 samples of the reduced 8x8 block are obtained in step 812, the remaining QQ of the block (18) to be predicted in step 813 red It is possible to derive the value (104) = 256-64 = 192.

[0215] In this case, to perform interpolation, it is selected to use only all samples of the boundary column (17a) and replacement samples of the boundary row (17c), and other selections are also possible.

[0216] In this method, the ratio of the multiplicative number to the final obtained value is Q red * P red / Q = 8*64 / 256 = 2, which is much less than 32 multiplications for each value in the absence of this technique.

[0217] The comparison between using and not using this technology is as follows:

[0218] If this technology is not available:

[0219] Block to be predicted (18), this block has dimensions M = 16, N = 16

[0220] Predicted Q = M*N = 16*16 = 256 values;

[0221] At the boundary, P = M + N = 16*16 = 32 samples;

[0222] For each predicted Q = 256 value, multiply by P = 32,

[0223] P*Q = 32*256 = 8192 Total number of multiplications;

[0224] The ratio between the number of multiplications and the number of final values ​​to be obtained is P*Q / Q = 32.

[0225] If you use this technology:

[0226] Block to be predicted (18), this block has dimensions M = 16, N = 16

[0227] Predicted Q at termination = M*N = 16*16 = 256 values

[0228] However, use Q red xP red = 64x8 ALWIP matrix, M red = 4, N red = 4, Q predicted by ALWIP red = 8*8 = 64 samples, P red = M red + N red = 4 + 4 = 8

[0229] P at the boundary red = M red + N red = 4 + 4 = 8 samples, P red <P

[0230] Each Q of the predicted reduced block red P for the value of = 64 red = 8 multiplication,

[0231] Q red* P red = 64*4 = 256 Total number of multiplications (less than 8192)

[0232] The ratio between the number of multiplications and the number of final values ​​to be obtained is P red * Q red / Q = 8*64 / 256 = 2 (much less than the 32 obtained without this skill).

[0233] Therefore, the computational power required by this technology is 16 times less than that of existing technology.

[0234] Therefore, using multiple adjacent samples (17),

[0235] To obtain a reduced set (102) of sample values ​​with fewer samples compared to the number of adjacent samples (17), the number of adjacent samples is reduced (100, 813),

[0236] To obtain predicted values ​​for predetermined samples (104, 118', 188") of a predetermined block (18), a reduced set (102) of sample values ​​is applied to a linear or affine linear transformation (19, 17M) (812),

[0237] You can predict the predetermined block (18) of the picture.

[0238] In particular, the reduction (100, 813) can be performed by downsampling a plurality of adjacent samples to obtain a reduced set (102) of sample values ​​with fewer samples compared to a plurality of adjacent samples (17).

[0239] Alternatively, the reduction (100, 813) can be performed by averaging multiple adjacent samples to obtain a reduced set (102) of fewer sample values ​​in the number of samples compared with multiple adjacent samples (17).

[0240] Multiple adjacent samples (17a, 17c) may be extended one-dimensionally along two sides of a predetermined block (18) (e.g., right and bottom in FIG. 7.1-7.4). The predetermined samples (e.g., samples obtained by ALWIP in step 812) may be arranged in rows and columns, and along at least one of the rows and columns, the predetermined samples may be located at every n-th position from the sample (112) of the predetermined samples (112) adjacent to the two sides of the predetermined block (18).

[0241] Based on a plurality of adjacent samples (17), a support value (118) for one of a plurality of neighboring locations (118) for at least one row and column can be determined, which is aligned to each of at least one of the row and column. It is also possible to derive a prediction value (118) for additional samples (108, 108') of a predetermined block (18) by interpolation based on a prediction value for a predetermined sample (104, 118', 118") and a support value for an adjacent sample (118) aligned to at least one of the row and column.

[0242] A predetermined sample (104) may be positioned at every n-th position of a sample (112) adjacent to two sides of a predetermined block (18) along a row, and a predetermined sample may be positioned at every m-th position from a sample (112) of a predetermined sample adjacent to two sides of a predetermined block (18) along a column, where n, m > 1. In some cases, n = m (e.g., in FIG. 7.2 and 7.3, samples (104, 118', 118") obtained directly by ALWIP in 812 and indicated by gray squares appear alternately with samples (108, 108') obtained in step 813 along rows and columns).

[0243] For each support value, the determination of the support value can be performed by, for example, downsampling or averaging (122), on a group of adjacent samples (120) within a plurality of adjacent samples, including the adjacent sample (118) for which each support value is determined along at least one of row (17c) and column (17a). Thus, in FIG. 7.4, the value of sample (119) can be obtained using the value of the sample (118'") (previously obtained in step 812) and the value of the adjacent sample (118) determined in step 813 as the support value.

[0244] Multiple adjacent samples can be extended one-dimensionally along two sides of a predetermined block (18). The multiple adjacent samples (17) can be grouped into one or more consecutive groups of adjacent samples (110) to perform reduction (811), and downsampling or averaging can be performed for each of the one or more groups of adjacent samples (110) having two or more adjacent samples.

[0245] In the example, the linear or affine linear transformation is P red * Q red or P red * It may include Q weighting factors, and P red is the number of sample values ​​(102) in the reduced set of sample values ​​and Q red Or Q is the number of predetermined samples within the samples in the predetermined block (18). At least ¼P red* Q red or ¼P red * The Q weighting factor is a non-zero weight value. P red * Q red or P red * The Q weighting factor is Q or Q red For each predetermined sample, a series of P associated with each predetermined sample red It may include weighting factors, wherein the series of weighting factors forms an envelope that is non-linear in all directions when one of the predetermined samples of the predetermined block (18) is arranged below another according to the raster scan order. P red * Q or P red*Qred weighting factors may be unrelated to one another through normal mapping rules. The mean of the maximum cross-correlation between the first series of weighting factors associated with each predetermined sample and the second series of weighting factors or the inverse version of the latter concerning the predetermined sample that is not the predetermined sample, regardless of which leads to a higher maximum value, is lower than a predetermined threshold. The predetermined threshold may be 0.3 [or 0.2 or 0.1 in some cases]. P red Adjacent samples (17) may be located along a one-dimensional path extending along two sides of a predetermined block (18), and each Q or Q red For a predetermined sample, a series of P associated with each predetermined sample red The weighting factors are aligned in a manner that traverses a one-dimensional path in a predetermined direction.

[0246] 6.1 Description of Method and Device

[0247] To predict rectangular block samples of width W (also denoted as N) and height H (also denoted as M), Affine Linear Weighted Intra Prediction (ALWIP) can take as input one line of restored neighbor boundary samples H to the left of the block and one line of restored neighbor boundary samples W above the block. If restored samples are unavailable, they can be generated as performed in conventional intra prediction.

[0248] The generation of a prediction signal (e.g., a value for a complete block (18)) may be based on at least some of the following three steps:

[0250] *1. Among the boundary samples (17), samples (102) (e.g., 4 samples in the case of W = H = 4 and / or 8 samples in the other case) may be extracted by averaging or downsampling (e.g., step 811).

[0251] 2. Matrix vector multiplication and subsequent offset addition can be performed using the averaged samples (or samples remaining from downsampling) as input. The result may be a reduced prediction signal for a subsampled set of samples of the original block (e.g., step 812).

[0252] 3. For example, by linear interpolation, a prediction signal at the remaining positions can be generated by upsampling from a prediction signal for a subsampled set (e.g., step 813).

[0253] Thanks to step 1 (811) and / or step 3 (813), the total number of multiplications required for the calculation of matrix vector multiplication can always be less than or equal to 4*W*H. Additionally, the averaging operation for the boundaries and the linear interpolation of the reduced prediction signal are performed using only addition and bit shifting. That is, for the ALWIP mode in the example, a maximum of 4 multiplications per sample are required.

[0254] In some examples, the matrix (e.g., 17M) and offset vector (e.g., b) required to generate the prediction signal k ) can be taken from a set of matrices S0, S1, S2 that can be stored in the storage unit(s) of the decoder and encoder, for example.

[0255] In some examples, set S0 is a matrix n0 (e.g., n0=16 or n0=18 or other numbers). It can include (e.g., configure), 16 rows and 4 columns, and 18 offset vectors, each with a size of 16. It can have, which performs the technique according to FIG. 7.1. The matrix and offset vector of this set are used in a block (18) of size 4x4. The boundary vector is P red When reduced to a vector of = 4 (as in step 811 of FIG. 7.1), P of the reduced sample set (102) redIt is possible to directly map 4 samples to Q = 16 samples of a 4x4 block (18) to be predicted.

[0256] In some examples, set S1 is a matrix n1 (e.g., n1=8 or n1=18 or other numbers). It can include (e.g., configuration), each with 16 rows and 8 columns, and 18 offset vectors of size 16. It may have and perform the technique according to FIG. 7.2 or 7.3. The matrix and offset vector of this set S1 can be used for blocks of sizes 4x8, 4x16, 4x32, 4x64, 8x4, 8x8, 16x4, 32x4, and 64x4. Additionally, it can be used for blocks of size WxH where max(W, H) > 4 and min(W, H) = 4, i.e., blocks of sizes 4x16 or 16x4, 4x32 or 32x4, and 4x64 or 64x4. The 16x8 matrix represents a reduced version of the block (18), which is a 4x4 block, as achieved in FIG. 7.2 and 7.3.

[0257] Additionally or alternatively, set S2 is a matrix of n2 (e.g., n2=6 or n2=18 or other numbers). It can include (e.g., configured), 18 offset vectors each with 64 rows and 8 columns and a size of 64. It may have. A 64x8 matrix represents a reduced version of a block (18) that is an 8x8 block, as can be obtained in FIG. 7.4. This set of matrices and offset vectors can be used for blocks of sizes 8x16, 8x32, 8x64, 16x8, 16x16, 16x32, 16x64, 32x8, 32x16, 32×32, 32×64, 64×8, 64×16, 64×32, 64×64.

[0258] The matrices and offset vectors of the corresponding set, or parts of these matrices and offset vectors, can be used for all other block shapes.

[0259] 6.2 Averaging or Downsampling of Boundaries

[0260] The characteristics of step 811 are described below.

[0261] As explained above, boundary samples (17a, 17c) can be averaged and / or downsampled (e.g., P in sample P red <P 샘플로).

[0262] In step 1, the input boundary bdry top (e.g., 17c) and bdry left (e.g., 17a) is a smaller boundary and It is reduced to become a reduced set (102). Here, and It consists of 2 samples in the case of a 4x4 block and 4 samples in the other cases.

[0263] For a 4x4 block, the following can be defined.

[0264]

[0265] Similarly can be defined. Therefore, , is, for example, an average value obtained using bit shifting operations.

[0266] In all other cases (e.g., for blocks where the width or height is not 4), the block width W is W = 4*2 k Given as such, define the following for 0≤i < 4:

[0267]

[0268] Similarly Defines.

[0269] In another case, a reduced number of samples can be reached by downsampling the boundaries (e.g., by selecting a specific boundary sample from a group of boundary samples). For example, bdry top [0] and bdry top [1] Among can be selected and bdry top [2] and bdry top [3] of You can select. Similarly It can also be defined.

[0270] Two reduced boundaries and is the reduced boundary vector bdry, also represented as 17P. red It can be connected to (associated with the reduced set (102)). Therefore, the reduced boundary vector bdry red For a block with a shape of 4x4 (example in Fig. 7.1), size 4 (P red = 4) and for all other shaped blocks (example), size 8(P red = 8) It may be (Figs. 7.2 to 7.4).

[0271] Here, if mode < 18 (or the number of matrices in the matrix set), the following can be defined:

[0272]

[0273] For mode ≥ 18, corresponding to the prefix mode of mode-17, the following can be defined.

[0274]

[0275] Therefore, depending on a specific state (one state: mode < 18; another state: mode ≥ 18), the predicted values ​​of the output vector can be distributed according to different scan orders (e.g., one scan order: Another scan sequence :

[0276] Other strategies may be implemented. In other examples, the mode index 'mode' does not necessarily have to be in the range of 0 to 35 (other ranges may be defined). Also, the three sets S0, S1, and S2 do not each have to have 18 matrices (thus, instead of an expression like mode≥18, mode≥n0, n1, n2, which is the number of matrices for each set S0, S1, and S2, is possible). Furthermore, the sets can each have a different number of matrices (for example, S0 may have 16 matrices, S1 may have 8 matrices, and S2 may have 6 matrices).

[0277] The mode and transposed information do not necessarily need to be stored and / or transmitted as a single combined mode index 'mode': in some examples, it may be explicitly signaled by a transpose flag and matrix index (0-15 for S0, 0-7 for S1, 0-5 for S2).

[0278] In some cases, a combination of a transposed flag and a matrix index can be interpreted as a set index. For example, there may be a 1 bit acting as a transposed flag and some bits representing a matrix index that is set as a "set index".

[0279] 5.4 Generation of Predicted Signals Reduced by Matrix-Vector Multiplication

[0280] The characteristics of step 812 are described below.

[0281] Reduced input vector bdry red ( Among the boundary vectors (17P)) reduced prediction signal Pred red It can generate. The latter signal has a width W red and height H red It can be a signal on the downsampled block of. Here, W red and H red can be defined as follows:

[0282]

[0283] Reduced predictive signal pred red It can be calculated by performing matrix-vector multiplication and adding an offset:

[0284]

[0285] Here, in all other cases where W = H = 4 and there are 8 columns, A is W red * H red It is a matrix that can have rows and 4 columns (e.g., a prediction matrix (17M)) and b is of size W red* H red It is a vector that can be.

[0286] If W = H = 4, A can have 4 columns and 16 rows, so in this case, pred red To calculate, 4 multiplications per sample may be required. In all other cases, A can have 8 columns, in this case 8*W red * H red ≤4*W*, i.e., pred red It can be proven that at most 4 multiplications per sample are required to calculate.

[0287] Matrix A and vector b can be taken from one of sets S0, S1, or S2 as follows. The index idx = idx(W, H) is defined by setting idx(W, H) = 0 if W = H = 4, idx(W, H) = 1 if max(W, H) = 8, and idx(W, H) = 2 in all other cases. Additionally, if mode < 18 and m = mode-17, m = mode can be entered. Then, if idx ≤ 1 or idx = 2 and min(W, H) > 4, and You can input. If idx = 2 and min(W, H) = 4, A corresponds to the odd x-coordinate of the downsampled block when W = 4, or the odd y-coordinate of the downsampled block when H = 4, It becomes a matrix resulting from omitting all rows of. If mode ≥ 18, the reduced prediction signal is replaced with the transposed signal. In alternative examples, a different strategy may be performed. For example, instead of reducing the size of a larger matrix ("omitted"), W red = 4 and H red = 4 in S1 The smaller matrix of (idx = 1) is used. That is, this block is now assigned to S1 instead of S2.

[0288] Other strategies may be implemented. In other examples, the mode index 'mode' does not necessarily have to be in the range of 0 to 35 (other ranges may be defined). Also, the three sets S0, S1, and S2 do not each have to have 18 matrices (therefore, instead of an expression like mode < 18, mode, which is the number of matrices for each set S0, S1, S2, is used). <n0, n1, n2가 가능하다.) 또한, 세트는 각각 다른 수의 행렬을 가질 수 있다 (예를 들어, S0에는 16 개의 행렬이 있을 수 있고, S1에는 8 개의 행렬이 있고, S2에는 6 개의 행렬이 있다.)

[0289] 6.4 Linear Interpolation to Generate the Final Prediction Signal

[0290] The features regarding Step 812 are described below.

[0292] Interpolation of the subsampled prediction signal may require a second version of the boundary averaged over the large block. That is, if min(W, H) > 8 and W≥H, then W = 8*2 l Record , and if 0≤i < 8, define the following:

[0293]

[0294] If min(W, H) > 8 and H > W, similarly Defines.

[0295] Additionally or alternatively, "hard downsampling" can be used, at this time is as follows:

[0296]

[0297] also It can also be defined similarly.

[0298] pred red pred at the sample location omitted in the generation of red The final predicted signal can be generated by linear interpolation (e.g., step 813 in the examples of FIGS. 7.2 to 7.4). This linear interpolation may be unnecessary in some examples where W = H = 4 (e.g., the example of FIG. 7.1).

[0299] Linear interpolation can be given as follows (although other examples are possible). It is assumed that W≥H. In this case, H>H red If so, pred red Vertical upsampling of can be performed. In this case, pred red can be expanded upwards one line at a time as follows. If W = 8, as defined above, pred red The width of is W red = can be 4 and average boundary signal It can be extended upward by. If W > 8, pred as defined above red The width of is W red = 8 and average boundary signal It expands upward by. pred red pred for the first line of red [x][-1] can be recorded. Then width W red and height 2*H red Signal for the block can be given as follows:

[0300]

[0301] 0≤x <W red and 0≤y <H red . The latter process is 2 k* H red = It can be performed k times up to H. Therefore, if H = 8 or H = 16, it can be performed at most once. If H = 32, it can be performed twice. If H = 64, it can be performed three times. Next, a horizontal upsampling operation can be applied to the result of vertical upsampling. The latter upsampling operation can use the entire left boundary of the prediction signal. Finally, if H > W, it can proceed similarly by first upsampling in the horizontal direction (if necessary) and then upsampling in the vertical direction.

[0302] This is an example of interpolation that uses reduced boundary samples for the first interpolation (horizontal or vertical) and the original boundary samples for the second interpolation (vertical or horizontal). Depending on the block size, only the second interpolation may be required, or no interpolation may be required at all. If both horizontal and vertical interpolation are required, the order depends on the width and height of the block.

[0303] However, other techniques may be implemented: for example, the original boundary sample may be used for both the first and second interpolation, and the order may be fixed as horizontal first, then vertical (in other cases, vertical then horizontal).

[0304] Therefore, the interpolation order (horizontal / vertical) and the use of reduced / original boundary samples may be changed.

[0305] 6.5 Description of an Example of the Entire ALWIP Process

[0306] The entire process of averaging, matrix-vector multiplication, and linear interpolation is described for the different shapes in FIGS. 7.1 through 7.4. The remaining shapes are handled as in one of the illustrated examples.

[0307] 1. Given a 4x4 block, ALWIP can take the average of two values ​​along each axis of the boundary using the technique of Fig. 7.1. The final four input samples are put into matrix-vector-multiplication. The matrix is ​​taken from set S0. By adding an offset, 16 final prediction samples can be generated. Linear interpolation is not required for generating the prediction signal. Therefore, a total of (4*16) / (4*4) = 4 multiplications are performed per sample. For example, refer to Fig. 7.1.

[0308] 2. Given an 8x8 block, ALWIP can take four averages along each axis of the boundary. The final 8 input samples enter matrix-vector-multiplication using the technique of Fig. 7.2. The matrix is ​​taken from set S1. This generates 16 samples at the odd positions of the prediction block. Thus, a total of (8*16) / (8*8) = 2 multiplications are performed per sample. After adding an offset, these samples can be interpolated vertically, for example using the top boundary, or horizontally, for example using the left boundary. For example, refer to Fig. 7.2.

[0309] 3. Given an 8x4 block, ALWIP can use the technique of Fig. 7.3 to take the average of 4 values ​​along the horizontal axis of the boundary and the 4 original boundary values ​​at the left boundary. The final 8 input samples enter matrix vector multiplication. The matrix is ​​taken from set S1. This generates 16 samples for each odd horizontal position and each vertical position of the prediction block. Thus, a total of (8*16) / (8*4) = 4 multiplications are performed per sample. After adding an offset, these samples are horizontally interpolated, for example, using the left boundary. For example, refer to Fig. 7.3.

[0310] Cases that are transposed are also processed accordingly.

[0311] 4. Given a 16x16 block, ALWIP can take four averages along each axis of the boundary. The final 8 input samples enter matrix-vector-multiplication using the technique of Fig. 7.2. The matrix is ​​taken from set S2. This generates 64 samples at the odd positions of the prediction block. Thus, a total of (8*64) / (16*16) = 2 multiplications are performed per sample. After adding an offset, these samples are interpolated vertically using, for example, the top boundary and horizontally using the left boundary. For example, refer to Fig. 7.2. For example, refer to Fig. 7.4.

[0312] For larger shapes, the procedure can be essentially the same, and it is easy to check if the number of multiplications per sample is less than 2.

[0313] For W × 8 blocks, only horizontal interpolation is required because samples are provided at odd horizontal positions and at each vertical position. Therefore, in this case, a maximum of (8*64) / (16*8) = 4 multiplications are performed per sample.

[0314] Finally, for W × 4 blocks where W > 8, A k It becomes a matrix formed by excluding all rows corresponding to odd items along the horizontal axis of the downsampled block. Therefore, the output size can be 32, and only horizontal interpolation can be performed again. A maximum of (8*32) / (16*4) = 4 multiplications can be performed per sample.

[0315] The transposed case can be processed accordingly.

[0316] 6.6 Evaluation of the Number and Complexity of Required Parameters

[0317] The parameters required for all proposed intra-prediction modes can consist of matrices and offset vectors belonging to sets S0, S1, and S2. All matrix coefficients and offset vectors can be stored as 10-bit values. Therefore, based on the description above, the proposed method may require a total of 14,400 parameters, each with 10-bit precision. This corresponds to 0,018 megabytes of memory. In the current standard 4:2:0 chroma-subsampling, a CTU of size 128x128 consists of 24,576 values, each 10 bits. Therefore, the memory requirements of the proposed intra-prediction tool do not exceed the memory requirements of the current picture reference tool adopted at the last meeting. Additionally, it was noted that existing intra-prediction modes require four multiplications per sample due to PDPC tools or 4-tap interpolation filters for angle prediction modes with fractional angle positions. Therefore, in terms of operational complexity, the proposed method does not exceed existing intra-prediction modes.

[0318] 6.7 Signaling of the Proposed Intra Prediction Mode

[0319] For the Luma block, for example, 35 ALWIP modes are proposed (another number of modes may be used). For each CU (Coding Unit) of the intra mode, whether the ALWIP mode is applied to the corresponding prediction unit (PU) is transmitted as a bit stream. The signaling of the latter index can be harmonized with the MRL in the same way as the first CE test. If the ALWIP mode is applied, the ALWIP mode index predmode can be signaled using an MPM-list with 3 MPMSs.

[0320] Here, the derivation of the MPM can be performed as follows using the intra mode of the upper and left PUs. Tables, for example, three fixed tables map_angular_to_alwip idx, idx∈ {0,1,2} may exist, which corresponds to each existing intra prediction mode predmode Angular ALWIP mode can be assigned to.

[0321]

[0322] For each PU with width W and height H, define an index.

[0323]

[0324] This indicates which of the three sets will obtain the ALWIP parameters as in Section 4 above. The above forecast unit PU above If it is available, belongs to the same CTU as the current PU, and is in intra mode, idx(PU) = idx(PU above ) and ALWIP is ALWIP-mode PU having above When applied to, it becomes the following:

[0325]

[0326] The above PU is available, belongs to the same CTU as the current PU, is in intra mode, and the existing intra prediction mode If applied to the above PU, it becomes the following.

[0327]

[0328] In all other cases, it becomes the following.

[0329]

[0330] This means that this mode is unavailable. In the same way, it induces the next mode without the restriction that the left PU must belong to the same CTU as the current PU.

[0331]

[0332] Finally, 3 fixed basic lists idx, idx∈ {0,1,2} is provided, each containing three individual ALWIP modes. Basic list list idx (PU) and mode and Among them, configure three individual MPMs by replacing -1 with the default value and removing iterations.

[0333] The embodiments described herein are not limited by the signaling of the proposed intra-prediction mode described above. According to alternative embodiments, an MPM and / or mapping table is not used for MIP (ALWIP).

[0334] 6.8 Derivation of MPM list adjusted for existing Luma and Chroma Intra prediction modes

[0335] The proposed ALWIP mode can be harmonized with the MPM-based coding of existing intra prediction modes as follows. The process of deriving the luminal and chroma MPM lists for existing intra prediction modes uses the fixed table map_lwip_to_angularidx, idx∈{0,1,2} to derive the ALWIP mode predmode of a given PU. LWIP Maps to one of the existing intra prediction modes.

[0336]

[0337] For Luma MPM list induction, ALWIP mode predmode LWIP Whenever an adjacent luma block using occurs, this block is the existing intra prediction mode predmode Angular It can be processed as if using . In the case of chroma MPM list derivation, whenever the current luma block uses LWIP mode, the same mapping can be used to convert ALWIP mode to the existing intra-predicted mode.

[0338] It is evident that ALWIP mode can harmonize with existing intra prediction modes without using MPM and / or mapping tables. For example, in the case of a chroma block, it is possible for ALWIP mode to be mapped to a flat intra prediction mode whenever the current luma block uses ALWIP mode.

[0339] 7 Effective Implementation Example

[0340] Let us briefly summarize the above example, which can form the basis for further expanding the embodiments described below.

[0341] To predict a predetermined block (18) of a picture (10), multiple neighbor samples (17a,c) are used.

[0342] A reduction (100) by averaging multiple neighbor samples is performed to obtain a reduced set (102) of sample values ​​with fewer samples compared to the number of neighbor samples. This reduction is optional in the embodiments of this specification and yields the so-called sample value vector mentioned below. The reduced set of sample values ​​is subject to a linear or affine linear transformation (19) to obtain predicted values ​​for a predetermined sample (104) of a predetermined block. What is to be obtained by machine learning (ML) and efficiently performed is a transformation shown later using matrix A and offset vector b.

[0343] By interpolation, the predicted value for additional samples (108) of a predetermined block is derived based on the predicted values ​​for the predetermined sample and a plurality of neighboring samples. It is said that, theoretically, the result of the affine / linear transformation can be associated with sample locations rather than the entire pixel of the block (18) so that all samples of the block (18) can be obtained by interpolation according to an alternative embodiment. Interpolation may not be necessary at all.

[0344] Multiple neighbor samples may be extended one-dimensionally along two sides of a predetermined block, and the predetermined samples may be arranged along at least one of the rows and columns, wherein the predetermined samples may be located at every nth position from the sample (112) of the predetermined samples adjacent to the two sides of the predetermined block. Based on the multiple neighbor samples, for each of at least one of the rows and columns, a support value for one of the multiple neighbor positions (118) may be determined, which is aligned with each of at least one of the rows and columns, and by interpolation, a predicted value for an additional sample (108) of the predetermined block may be derived based on the predicted value for the predetermined sample and the support value for the neighbor sample aligned with at least one of the rows and columns. A predetermined sample may be located at every n-th position from the sample (112) of the predetermined sample adjacent to both sides of the predetermined block along the row, and a predetermined sample may be located at every m-th position from the sample (112) of the predetermined sample adjacent to two sides of the predetermined block along the column, where n, m > 1. n = m. For each support value, the determination of the support value along at least one of the row and column may be performed by averaging (122) a group of neighbor samples (120) within a plurality of neighbor samples including the neighbor sample (118) for which each support value is determined. The plurality of neighbor samples may be extended one-dimensionally along both sides of the predetermined block and reduced by grouping the plurality of neighbor samples into one or more consecutive groups of neighbor samples (110) and performing averaging on each of the one or more groups of neighbor samples having more than two neighbor samples.

[0345] For a predetermined block, the predicted residuals can be transmitted from the data stream. In the decoder, they can be derived therefrom, and the predetermined block can be reconstructed using the predicted residuals and predicted values ​​for the predetermined samples. In the encoder, the predicted residuals are encoded into the data stream.

[0346] The picture may be subdivided into multiple blocks of different block sizes, and the multiple blocks include a predetermined block. Then, a linear or affine linear transformation for the block (18) is selected according to the width W and height H of the predetermined block, so that if the width W and height H of the predetermined block are within a first set of width / height pairs, the linear or affine linear transformation selected for the predetermined block is selected in a first set of linear or affine linear transformations, and if the width W and height H of the predetermined block are within a second set of width / height pairs separated from the first set of width / height pairs, the linear or affine linear transformation selected for the predetermined block is selected in a second set of linear or affine linear transformations. In other words, it becomes clear later that the affine / linear transformation is expressed through other parameters, namely the weight of C and optionally offset and scale parameters.

[0347] The decoder and encoder are configured to subdivide the picture into multiple blocks of different block sizes containing predetermined blocks, and to select a linear or affine linear transformation according to the width W and height H of the predetermined blocks, so the linear or affine linear transformation selected for the predetermined blocks is

[0348] If the predetermined block width W and height H are within the first set of width / height pairs, the first set of linear or affine linear transformations,

[0349] If the width W and height H of a predetermined block are within a second set of width / height pairs separated from a first set of width / height pairs, a second set of linear or affine linear transformations, and

[0350] If the predetermined block width W and height H are within a third set of one or more width / height pairs separated from the first and second sets of width / height pairs, then a third set of linear or affine linear transformations

[0351] It is selected from among them.

[0352] One or more widths / heights of the third set (e.g., S0) include a single width / height pair, W', H', and each linear or affine linear transformation within the first set of linear or affine linear transformations is intended to transform N' sample values ​​into W'* H' predicted values ​​for the W'x H' array at the sample positions.

[0353] Each of the width / height pairs of the first and second sets is the first width / height pair W p , H p (W p is H p Not equal to) and the second width / height pair W q , H q (H q = W p and W q = H p It may include ).

[0354] Each of the 1st and 2nd width / height pairs is the 3rd width / height pair W p , H p Additionally include, at this time W p is H p Same as and H p > H q am.

[0355] For a predetermined block, a set index is transmitted to a data stream, which represents a linear or affine linear transformation selected for the block (18) from a predetermined set of linear or affine linear transformations.

[0356] The plurality of adjacent samples can be extended one-dimensionally along two sides of the predetermined block, and the reduction can be performed by grouping the first subset of the plurality of adjacent samples adjacent to the first side of the predetermined block into one or more consecutive adjacent samples (110) for the first subset of the plurality of adjacent samples adjacent to the second side of the predetermined block, and grouping the second subset into one or more consecutive adjacent samples (110) for the second subset of the plurality of adjacent samples adjacent to the second side of the predetermined block, and performing downsampling or averaging on each of the first and second groups of one or more adjacent samples having two or more adjacent samples.Next, the linear or affine linear transformation is selected according to the set index from among the predetermined set of linear or affine linear transformations, so that two different states of the set index select one of the linear or affine linear transformations of the predetermined set of linear or affine linear transformations, and may be configured to apply a reduced set of sample values ​​to the predetermined linear or pseudo-linear transformation, wherein the set index assumes a first state among the two different states in the form of a first vector to generate an output vector of predicted values ​​and distribute the predicted values ​​of the output vector to the predetermined samples of the predetermined block according to a first scan order, and the set index assumes a second state among the two different states in the form of a second vector, and wherein the first and second vectors are configured such that a component filled by one of the first sample values ​​of the first vector is filled by one of the second sample values ​​of the second vector, and a component filled by one of the second sample values ​​of the first vector is filled by one of the first sample values ​​of the second vector. Since they are different, an output vector of predicted values ​​is generated, and the predicted values ​​of the output vector are distributed to the predetermined samples of the predetermined block according to the second scan order transposed with respect to the first scan order.

[0357] Each linear or affine linear transformation in the first set of linear or affine linear transformations is configured to convert N1 sample values ​​into w1*h1 predicted values ​​for a w1xh1 array at a sample location, and each linear or affine linear transformation in the second set of linear or affine linear transformations is configured to convert N2 sample values ​​into w2*h2 predicted values ​​for a w2xh2 array at a sample location, wherein for a first predetermined width / height pair in the first set, w1 exceeds the width of the first predetermined width / height pair or h1 exceeds the height of the first predetermined width / height pair, and for a second predetermined width / height pair in the first set, w1 does not exceed the width of the second predetermined width / height pair or h1 does not exceed the height of the second predetermined width / height pair. Where the predetermined block is composed of the first predetermined width / height pair and the predetermined block is composed of the second predetermined width / height pair, the reduction (100) is performed by downsampling or averaging the plurality of adjacent samples to obtain the reduced set (102) of sample values ​​such that the reduced set (102) of sample values ​​has N1 sample values; thus, where the predetermined block is composed of the first predetermined width / height pair, only the first sub-part of the selected linear or affine linear transformation related to the subsampling of the w1xh1 array at the sample location is used along the width dimension when w1 exceeds the width of the one width / height pair, or along the height dimension when h1 exceeds the height of the one width / height pair, and where the predetermined block is composed of the second predetermined width / height pair, the selected linear or affine linear transformation is used in its entirety, thereby in the reduced set of sample values It can be configured to perform the application of a selected linear or affine linear transformation.

[0358] Each linear or affine linear transformation in the first set of linear or affine linear transformations is configured to transform N1 sample values ​​into w1*h1 predicted values ​​for a w1xh1 array at a sample location where w1=h1, and each linear or affine linear transformation in the second set of linear or affine linear transformations is configured to transform N2 sample values ​​into w2*h2 predicted values ​​for a w2xh2 array at a sample location where w2=h2.

[0359] All embodiments described above are merely illustrative in that they may form the basis of the embodiments described below. That is, the above concepts and details will aid in understanding the following embodiments and should serve as a repository for possible extensions and modifications of the embodiments described below. In particular, the details described above, such as the averaging of neighbor samples and the fact that neighbor samples are used as reference samples, are optional.

[0360] More generally, the embodiments described herein assume that the prediction signal for a rectangular block is generated from already restored samples, and the intra prediction signal for a rectangular block is generated from already restored samples adjacent to the left and top of the block. The generation of the prediction signal is based on the following steps.

[0361] 1. Among the reference samples currently referred to as boundary samples, samples can be extracted by averaging without excluding the possibility of moving the explanation to a reference sample located elsewhere. Here, the averaging is performed on both the left and top boundary samples of the block, or on only one of the boundary samples on one of the two sides. If averaging is not performed on one side, the samples on that side remain unchanged.

[0362] 2. Optionally, a matrix vector multiplication is performed followed by the addition of an offset, where the input vector for the matrix vector multiplication is the concatenation of the averaged boundary sample on the left side of the block and the original boundary sample above the block when the average is applied only to the left, or the concatenation of the original boundary sample on the left side of the block and the averaged boundary sample above the block when the average is applied only to the top, or the concatenation of the averaged boundary sample on the left side of the block and the averaged boundary sample above the block when the average is applied to both sides of the block. Again, as with cases where averaging is not used at all, alternatives will exist.

[0363] 3. The result of matrix vector multiplication and optional offset addition may optionally be a reduced prediction signal for a subsampled set of samples of the original block. The prediction signal for the remaining positions may be generated from the prediction signal for a subsampled set by linear interpolation.

[0364] In Step 2, the calculation of the matrix-vector product should preferably be performed using integer arithmetic. Therefore, x=(x1,…,x n If ) represents the input for a matrix-vector product, that is, if x represents the concatenation of (average) boundary samples to the left and above the block, the (reduced) prediction signal calculated in Step 2 at x must be computed using only bit shifts, addition of offset vectors, and multiplication with integers. Ideally, the prediction signal in Step 2 is given by Ax+b, where b is an offset vector that can be zero, and A is derived by some machine learning-based training algorithm. However, according to these training algorithms, the matrix A=A, which is typically provided in floating-point precision, float Only Equation A is the result. Therefore, Equation A float We face the problem of specifying integer operations in the aforementioned sense so that x is well approximated using these integer operations. Here, these integer operations are Equation A, which assumes a uniform distribution of vector x. floatIt is mentioned that it does not need to be chosen to approximate x, but generally, Equation A float The input vector x to which x is approximated is the component x of x i It is important to consider that these are (average) boundary samples from natural video signals from which some correlation can be expected.

[0365] FIG. 8 illustrates an improved ALWIP prediction. Samples of a given block can be predicted based on a first matrix-vector product between a matrix A (1100) and a sample value vector x (400) derived by some machine learning-based training algorithm. Optionally, an offset b (1110) may be added. To obtain an integer or fixed-point approximation of this first matrix-vector product, the sample value vector may undergo an invertible linear transformation (403) to determine an additional vector (402). A second matrix-vector product between the additional matrix B (1200) and the additional vector (402) may be the same as the result of the first matrix-vector product.

[0366] Due to the characteristics of the additional vector (402), the second matrix vector product may be an integer approximated by the matrix vector product (404) between the predetermined prediction matrix C (405) and the value of the sum of the additional vector (402) and the additional offset (408). The additional vector (402) and the additional offset (408) may consist of integer or fixed-point values. For example, all components of the additional offset are identical. The predetermined prediction matrix (405) may be a quantized matrix or a matrix to be quantized. The result of the matrix vector product (404) between the predetermined prediction matrix (405) and the additional vector (402) may be understood as the prediction vector (406).

[0367] Further details regarding this integer approximation are provided below.

[0368] Possible solutions according to Example I: Subtracting and adding average values

[0369] Equation A that can be used in the above scenario floatOne possible way to incorporate an integer approximation of x is x, i.e., the i0th component x of the sample value vector (400). i0 That is, replacing the predetermined component (1500) with the average value (x) of component x, i.e., the predetermined value (1400), and subtracting this average value from all other components. That is, as illustrated in FIG. 9a, the invertible linear transformation (403) is such that the predetermined component (1500) of the additional vector (402) becomes a, and each of the other components of the additional vector (402) excluding the predetermined component (1500) is equal to the corresponding component of the sample value vector (400) minus a, where a is the predetermined value (1400), which is an average such as, for example, the arithmetic mean or weighted mean of the components of the sample value vector (400). This operation on the input is provided by the invertible transformation T (403), which has an obvious integer implementation, particularly when the order n of x is a power of 2.

[0370] A float =(A float T - 1 Since )T, when performing this transformation on input x, we must find the integer approximation of the matrix-vector product By, where B=(A float T -1 ) and y=Tx. Matrix-vector product A float Since x represents a rectangular block, i.e., a prediction for a predetermined block, and x(400) consists of boundary samples (e.g., averaged) of that block, if all sample values ​​of x are the same, i.e., for all i, x i When =mean(x), the predicted signal A floatIt is expected that each sample value of x should be close to or exactly equal to mean(x). That is, the i0th column of B, i.e., the column corresponding to the predetermined component, should be expected to be very close to or equal to a column consisting only of 1s. Therefore, since M(i0), i.e., the integer matrix (1300), is a matrix where the i0th column consists of 1s and all other columns are 0, if By=Cy+M(i0)y is written as C=BM(i0), the i0th column of C (412), i.e., the predetermined prediction matrix (405), is expected to have somewhat fewer entries or be 0, as illustrated in FIG. 9b. Furthermore, since the components of x are correlated, for each i≠i_0, the i-th component y of y i =x i -mean(x) often has an absolute value much smaller than the i-th component of x. Since matrix M(i0)(1300) is an integer matrix, if an integer approximation of Cy is given, an integer approximation of By is achieved, and by the above argument, the quantization error resulting from quantizing each item of C(405) in an appropriate manner is A float It can be expected that it will have only a negligible effect on the error of the final quantization of x's By.

[0371] The predetermined value (1400) does not necessarily have to be the average value average (x). Formula A described in this specification float An integer approximation of x may also be achieved using the following alternative definition of a predetermined value (1400).

[0372] Formula A float In another possible integration of the integer approximation of x, the i0th component of x x i0 It remains unchanged and the same value x i0 is subtracted from all other components. That is, for each i≠i_0, y i0 = x i0 and y i =x i -x iThat is, the predetermined value (1400) may be a component of the sample value vector (400) corresponding to the predetermined component (1500).

[0373] Alternatively, a predetermined value (1400) is a default value or a value signaled in the data stream where the picture is coded.

[0374] The predetermined value (1400) is, for example, 2 bitdepth -1 It is the same as. In this case, the additional vector (402) is y0=2 for i>0 bitdepth-1 and can be defined by yi=xi-x0.

[0375] Alternatively, the predetermined component (1500) becomes a constant obtained by subtracting the predetermined value (1400). The constant is, for example, 2 bitdepth -1 It is the same as. According to one embodiment, the predetermined component y of the additional vector y (402) i0 (1500) is 2 bitdepth -1 Component x of the sample value vector (400) corresponding to the predetermined component (1500) in i0 It is equal to subtracting. All other components of the additional vector (402) are equal to subtracting the component of the sample value vector (400) corresponding to the predetermined component (1500) from the corresponding component of the sample value vector (400).

[0376] For example, it is advantageous for a predetermined value (1400) to have a small deviation from the predicted value of the samples of a predetermined block.

[0377] According to one embodiment, the device (1000) is configured to include a plurality of reversible linear transformations (403), each of which is associated with one component of an additional vector (402). Additionally, the device is configured to select a predetermined component (1500) from among the components of a sample value vector (400), for example, and to use a reversible linear transformation (403) from among a plurality of reversible linear transformations associated with the predetermined component (1500) as a predetermined reversible linear transformation. This is, for example, due to a different position in the i0th row, i.e., a row of the reversible linear transformation (403) corresponding to the predetermined component, depending on the position of the predetermined component in the additional vector. For example, if the first component of the additional vector (402), i.e., y1, is the predetermined component, the i0th row replaces the first row of the reversible linear transformation.

[0378] As illustrated in FIG. 9b, the column (412) of the predetermined prediction matrix (405) corresponding to the predetermined component (1500) of the additional vector (402), i.e., the matrix component (414) of the predetermined prediction matrix C (405) within the i0th column, is, for example, all zero. In this case, the device is configured to calculate the matrix vector product (404) by performing multiplication by calculating the matrix vector product (407) between the column (412) generated from the additional vector (402) by leaving the predetermined component (1500) as illustrated in FIG. 9b, and the reduced prediction matrix C' (405) resulting from the predetermined prediction matrix C (405) by leaving much more vector (410). Thus, the prediction vector (406) can be calculated with fewer multiplications.

[0379] As illustrated in FIGS. 8 and 9b, the device (1000) may be configured to calculate, in the step of predicting samples of a predetermined block based on a prediction vector (406), the sum of each component and a, i.e., a predetermined value (1400), for each component of the prediction vector (406). This sum may be represented as the sum of the prediction vector (406) and the vector (409), as illustrated in FIGS. 8 and 9b, where all components of the vector (409) are equal to the predetermined value (1400). Alternatively, the sum can be expressed as the sum of the prediction vector (406) and the matrix vector product (1310) of the integer matrix M (1300) and the additional vector (402), as illustrated in FIG. 9b, where the matrix component of the integer matrix (1300) is 1 in the column of the integer matrix (1300), i.e., the i0th column, corresponding to the predetermined component (1500) of the additional vector (402), and all other components are, for example, 0.

[0380] The result of the sum of the predetermined prediction matrix C (405) and the integer matrix (1300) is equal to or approximates the additional matrix B (1200) shown in Fig. 8, for example.

[0382] In other words, the matrix B (1200), which is the result of summing each matrix component of the predetermined prediction matrix C (405) within the i0th column (412) of the predetermined prediction matrix (405) corresponding to the predetermined component (1500) of the additional vector (402) multiplied by 1 (i.e., matrix B) by an invertible linear transformation (403), corresponds to, for example, a quantized version of the machine learning prediction matrix A (1100), as illustrated in FIG. 8, FIG. 9a and FIG. 9b. As illustrated in FIG. 9b, the sum of each matrix component of the predetermined prediction matrix C (405) within the i0th column (412) and 1 may correspond to the sum of the predetermined prediction matrix (405) and the integer matrix (1300). As illustrated in FIG. 8, the machine learning prediction matrix A (1100) may be equal to the result of multiplying an additional matrix (1200) by an invertible linear transformation (403). This is A*x=BT*yT - 1 This is due to. The predetermined prediction matrix (405) is, for example, a quantized matrix, an integer matrix and / or a fixed-point matrix, thereby enabling the realization of a quantized version of the machine learning prediction matrix A (1100).

[0383] Matrix multiplication using only integer operations

[0384] For low-complexity implementation (in terms of the complexity of adding and multiplying scalar values ​​and the storage space required for the entries of the participating matrix), it is desirable to perform matrix multiplication (404) using only integer operations. By using only integer operations, the approximation of z=Cy, i.e.

[0385]

[0386] To calculate, according to the embodiment, a real value C i,j is an integer value It must be mapped to. This is, for example, by uniform scalar quantization, or the value y iConsidering specific correlations between them, integer values ​​can be performed. For example, each represents a fixed-point number that can be stored as a fixed number of bits, n_bits (e.g., n_bits=8).

[0387] A matrix vector product (404) having a matrix, i.e., a predetermined prediction matrix (405) of size mxn, can be performed as shown in this pseudocode, where <<, >> are arithmetic binary left and right shift operations and +, -, and * are performed only on integer values.

[0388] (1)

[0389] final_offset = 1 << (right_shift_result - 1);

[0390] for i in 0..m-1

[0391] {

[0392] accumulator = 0

[0393] for j in 0..n-1

[0394] {

[0395] accumulator := accumulator + y[j]*C[i,j]

[0396] }

[0397] z[i] = (accumulator + final_offset) >> right_shift_result;

[0398] }

[0399] Here, matrix C, i.e., the predetermined prediction matrix (405), stores fixed-point numbers, for example, as integers. The final addition of final_offset and the right shift operation using right_shift_result reduce precision by rounding to obtain the fixed-point format required for the output.

[0400] To increase the range of actual values ​​that can be represented as integers in C, two additional matrix offsets, as shown in the embodiments of FIGS. 10 and 11 i,j and scale i,j Since matrix-vector multiplication can be used

[0401]

[0402] from y j Each coefficient b i,j Is

[0403]

[0404] It is given by.

[0405] value offset i,j and scale i,j is an integer value itself. For example, such integers are fixed-point numbers that can each be stored in a fixed number of bits (e.g., 8 bits), or, for example, values It can represent the same number of bits n_bits used to store.

[0406] In other words, the device (1000) predicts parameters, for example, integer values and value offset i,j and scale i,j It is configured to represent a predetermined prediction matrix (405) using and to calculate a matrix vector product (404) by performing multiplication and summation on the components of an additional vector (402), the prediction parameters, and the intermediate results generated therefrom, wherein the absolute value of the prediction parameters can be expressed in an n-bit fixed-point number representation, and n is less than or equal to 14, or alternatively 10, or alternatively 8. For example, the components of the additional vector (402) are multiplied with the prediction parameters to produce a product as an intermediate result, and then this is summed or forms the mantissa of the sum.

[0407] According to one embodiment, the prediction parameters include weights, each of which is associated with a corresponding matrix component of the prediction matrix. That is, the predetermined prediction matrix is ​​replaced or represented, for example, by the prediction parameters. The weights are, for example, integer and / or fixed-point values.

[0408] According to one embodiment, the prediction parameter is one or more scaling factors, e.g., value scale i,j It further includes weights, e.g., integer values, each associated with one or more corresponding matrix components of a predetermined prediction matrix (405). It is associated with one or more corresponding matrix components of a predetermined prediction matrix (405) for scaling. Additionally, or alternatively, the prediction parameter is associated with one or more offsets, e.g., value offset i,j and each of these is a weight, e.g., an integer value C, associated with one or more corresponding matrix components of a predetermined prediction matrix (405). i,j It is associated with one or more corresponding matrix components of a predetermined prediction matrix (405) for offsetting.

[0409] offset i,j and scale i,j To reduce the storage space required, the value can be chosen to be constant for a specific set of indices i,j. For example, the item can be constant for each column, constant for each row, or constant for all i,j as shown in Fig. 10.

[0410] For example, in one preferred embodiment, offset i,j and scale i,j is constant for all values ​​of the matrix of a single prediction mode, as shown in Fig. 11. Therefore, in the case where there are K prediction modes (k = 0..K-1), a single value o is used to calculate the prediction for mode k. k and a single value s kOnly [something] is needed.

[0411] According to one embodiment, offset i,j and / or scale i,j is constant, i.e., identical, for all matrix-based intra prediction modes. Additionally, or alternatively, the offset i,j and / or scale i,j It is constant for all block sizes.

[0412] o k The offset and s representing k As a scale representing , the calculation in (1) can be modified as follows:

[0413] (2)

[0414] final_offset = 0;

[0415] for i in 0..n-1

[0416] {

[0417] final_offset := final_offset - y[i];

[0418] }

[0419] final_offset *= final_offset * offset * scale;

[0420] final_offset += 1 << (right_shift_result - 1);

[0421] for i in 0..m-1

[0422] {

[0423] accumulator = 0

[0424] for j in 0..n-1

[0425] {

[0426] accumulator := accumulator + y[j]*C[i,j]

[0427] }

[0428] z[i] = (accumulator*scale + final_offset) >> right_shift_result;

[0429] }

[0430] Extended embodiments occurring in the solution

[0431] The above solution refers to the following embodiments.

[0432] 1. For the prediction method as in Section I, the following is performed for the integer approximation of the matrix-vector product involved in Step 2 of Section I: (mean) boundary samples x=(x1,…,x n Among ), for a fixed i0 such that 1 ≤ i0 ≤ n, the vector y=(y1,…,y n ) is calculated, where y for i≠i0 i =x i -mean(x) and yi0 = ,mean(x), where mean(x) represents the mean value of x. Then, since the vector y serves as the input for the (integer realization) of the matrix-vector product Cy, the (downsampled) prediction signal pred from Step 2 of Section I is given by pred = Cy + meanpred(x). In this equation, meanpred(x) represents the signal equivalent to mean(x) for each sample position in the domain of the (downsampled) prediction signal. (See, for example, Fig. 9b)

[0433] 2. The prediction method of Section I, where in Step 2 of Section I, for the integer approximation of the related matrix-vector product, the following is performed: (mean) boundary samples x=(x1,…,x n For a fixed i0 such that 1≤i0≤n among ), the vector y=(y1,…,y n - 1) is calculated, where i <i0인 경우 y i =x i y for -mean(x) and i≥i0 i =x i+1-mean(x), where mean(x) represents the mean value of x. Then, the vector y serves as the input for the matrix vector product Cy (an integer realization of y) such that the (downsampled) prediction signal pred from Step 2 of Section I is given by pred = Cy + meanpred(x). In this equation, meanpred(x) represents a signal identical to mean(x) for each sample position in the domain of the (downsampled) prediction signal. (See, for example, Fig. 9b)

[0434] 3. The same prediction method as in Section I, where the integer realization of the matrix-vector product Cy is the matrix-vector product coefficients in It is given using. (See, for example, Fig. 10)

[0435] 4. A prediction method as in Section I. Here, since Step 2 uses one of the K matrices, multiple prediction modes can be computed, each with a different matrix. Using (where k=1..K-1), here matrix-vector product C k The integer realization of y is the matrix-vector product coefficients in It is given using. (See, for example, Fig. 11)

[0436] That is, according to embodiments of the present application, the encoder and decoder operate as follows to predict a predetermined block (18) of a picture (10) (see FIG. 8). A plurality of reference samples are used for prediction. As described above, embodiments of the present application are not limited to intra-coding, and thus the reference samples are not limited to adjacent samples, i.e., samples of the picture (10) adjacent to the block (18). In particular, the reference samples are not limited to samples arranged along the outer edge of the block (18), such as samples adjacent to the outer edge of the block. However, such a situation certainly constitutes an embodiment of the present application.

[0437] To perform a prediction, a sample value vector (400) is formed from reference samples such as reference samples (17a, 17c). Possible formations have been described above. The formation may involve averaging, thereby reducing the number of components of the vector (400) compared to the number of samples (102) or reference samples (17) contributing to the formation. The formation may also depend on the dimensions or size of the block (18), such as width and height, as previously described.

[0438] It is this vector (400) that undergoes an affine or linear transformation to obtain a prediction of block (18). Different nomenclature was used above. Using the most recent one, the goal is to perform a prediction by applying the vector (400) to matrix A through matrix vector multiplication within performing a summation with the offset vector b. The offset vector b is optional. The affine or linear transformation determined by A or A and b can be determined by an encoder and a decoder, or more precisely, for a prediction based on the size and dimensions of block (18) as already described above.

[0439] However, to achieve the computational efficiency improvements described above or to make predictions more effective in terms of implementation, the affine or linear transformation is quantized, and the encoder and decoder, or their predictors, use the aforementioned C and T to represent and perform the linear or affine transformation, where C and T are applied in the manner described above and represent a quantized version of the affine transformation. In particular, instead of applying the vector (400) directly to matrix A, the predictors of the encoder and decoder apply the resulting vector (402) from the sample value vector (400) in a manner that applies it to the mapping through a predetermined invertible linear transformation T. As long as the vector (400) has the same size, the transformation T used herein may be the same, that is, it does not depend on the dimensions of the block, i.e., width and height, or at least is the same for different affine / linear transformations. In the above, the vector (402) is denoted as y. The exact matrix for performing the affine / linear transformation determined by machine learning is B. However, instead of performing B exactly, predictions in the encoder and decoder are performed through approximations or quantized versions thereof. In particular, the representation is performed by appropriately expressing C in the manner described above, along with C + M, which represents the quantized version of B.

[0440] Accordingly, prediction in the encoder and decoder is further performed by calculating a matrix vector product (404) between a vector (402) appropriately represented and stored in the encoder and decoder in the manner described above and a predetermined prediction matrix C. The vector (406) resulting from this matrix vector product is used to predict the sample (104) of the block (18). As described above, for prediction, each component of the vector (406) is summed with the parameter a, as indicated in 408, to compensate for the corresponding definition of C. An optional sum of the offset vector b and the vector (406) may also be included in the derivation of the prediction of the block (18) based on the vector (406). As described above, each component of the vector (406) and, therefore, each component of the sum of the vector (406), all vectors of a and the optional vector b, as indicated in 408, can directly correspond to the sample (104) of the block (18) and thus represent the predicted value of the sample. Also, only a subset of the samples (104) of the block is predicted in this way, and the remaining samples of the block (18), such as 108, can be derived by interpolation.

[0441] As previously mentioned, there are various embodiments for setting a. For example, this may be the arithmetic mean of the components of vector (400). For this case, refer to FIG. 9a. The invertible linear transformation T (403) may be as shown in FIG. 9a. i0 is a predetermined component of the sample value vector and vector (402), respectively, which is replaced by a. However, as described above, other possibilities exist. However, it was also mentioned above that, as far as the representation of C is concerned, it may be implemented differently. For example, the matrix-vector product (404) may end up as a smaller matrix-vector product with a lower dimensionality in the actual calculation. In particular, as noted above, since the entire i0th column (412) may be zero due to the definition of C, the actual calculation of the product (404) may be performed by a reduced version of vector (402), which is the component y i0 By omitting, that is, by omitting the i0th column (412), the result from C is obtained by multiplying the reduced matrix C' resulting from C and this reduced vector (410) to obtain the result from vector (402).

[0442] The weights of C or the weights of C', i.e., the components of this matrix, can be expressed and stored as fixed-point number representations. However, these weights (414) can also be stored in a manner associated with different scales and / or offsets as described above. The scale and offset can be defined for the entire matrix C, that is, they must be the same for all weights (414) of matrix C or matrix C', or they can be defined to be constant or equal for all weights (414) of the same row or all weights (414) of matrix C and matrix C', respectively. FIG. 10 illustrates, in this regard, that the calculation of the matrix vector product, i.e., the result of the product, can be performed slightly differently, for example, by shifting the multiplication with the scale(s) toward the vector (402 or 410), thereby reducing the number of additional multiplications that need to be performed. FIG. 11 illustrates the case where one scale and one offset are used for all weights (414) of C or C', as is done in the calculation (2).

[0443] According to one embodiment, the device described herein for predicting a predetermined block of a picture may be configured to use matrix-based intra-sample prediction comprising the following features:

[0444] The device is configured to form a sample value vector pTemp[x] (400) from a plurality of reference samples (17). Assuming pTemp[x] is 2*boundarySize, pTemp[x] can be filled—for example, by direct copying, subsampling, or pooling—with neighbor samples located at the top of a predetermined block, redT[x] (where x = 0..boundarySize-1), followed by neighbor samples located to the left of a predetermined block, redL[x] (where x = 0..boundarySize-1), or in the case of transposition (for example, when isTransposed=0), or vice versa.

[0445] The input value p[x] (where x = 0.. inSize-1) is derived, that is, the device derives an additional vector p[x] from the sample value vector pTemp[x], to which the sample value vector pTemp[x] is mapped by a predetermined invertible linear transformation, or is configured to be a more specific predetermined invertible affine linear transformation as follows:

[0446] - If mipSizeId is 2, the following applies.

[0447] p[ x ] = pTemp[ x + 1 ] - pTemp

[0000]

[0448] - Otherwise (if mipSizeId is less than 2), the following applies:

[0449] p

[0000] = ( 1 << ( BitDepth - 1 ) ) - pTemp

[0000]

[0450] p[ x ] = pTemp[ x ] - pTemp

[0000] , x = 1..inSize-1

[0451] Here, the variable mipSizeId represents the size of a predetermined block. That is, according to the present embodiment, the invertible transformation in which an additional vector is derived from a sample value vector depends on the size of the predetermined block. The dependency can be given as follows.

[0452] mipSizeId boundarySize predSize 0 2 4 1 4 4 2 4 8

[0453] Here, predSize represents the number of predicted samples within a predetermined block, 2*bondarySize represents the size of the sample value vector, and inSize is related to the size of the additional vector, i.e., inSize, according to inSize = (2 * boundarySize) - (mipSizeId = 2) ? 1:0. More precisely, inSize represents the number of components of the additional vector that actually participate in the calculation. For small block sizes, inSize is as large as the size of the sample value vector, and for large block sizes, it is one component smaller. In the former case, one component, that is, the predetermined component of the additional vector, can be ignored, and since the contribution of that vector component generates 0 anyway, as in the matrix-vector multiplication to be calculated later, it does not actually need to be calculated. In the case of an alternative embodiment, the dependency on the block size may be omitted, in which case only one of the two alternatives is inevitably used, that is, regardless of the block size (the option corresponding to mipSizeId is less than 2, or the option corresponding to mipSizeId is equal to 2). That is, the predetermined invertible linear transformation is defined, for example, such that the predetermined component of the additional vector p is a, whereas everything else corresponds to the value obtained by subtracting a from the component of the sample value vector, for example, where a = pTemp[0]. If the first option corresponding to mipSizeId is 2, this is easily seen, and only the differentially formed component of the additional vector is additionally considered. That is, in the case of the first option, the additional vector is actually {p[0…inSize];pTemp[0]}, where pTemp[0] is 0, and the actual calculated part of the matrix-vector multiplication to yield the result of the multiplication is limited only to the inSize component of the additional vector and the corresponding column of the matrix. This is because the matrix has zero columns that do not require calculation.In other cases, corresponding to mipSizeId less than 2, a=pTemp[0] is selected as all components of the additional vector excluding p[0]; that is, excluding the predetermined component p[0], each of the other components p[x] of the additional vector p (for x = 1..inSize - 1) is equal to the corresponding component of the sample value vector pTemp[x] minus a, but p[0] is selected as the value of a minus a constant. Then, the matrix-vector product is calculated. The constant is an representable value, i.e., 2. x-1 It is the average of (i.e., 1 << (BitDepth - 1)), where x represents the bit depth of the computational representation used. Note that if p[0] is selected as pTemp[0], the product can deviate by the constant vector considered when predicting the block interior based on the product from what is calculated using p[0] as indicated above, i.e., the prediction vector (p[0] = (1 << (BitDepth-1)) - pTemp[0]). Therefore, the value of a is a predetermined value, e.g., pTemp[0]. Thus, this value is the predetermined value pTemp[0]. The predetermined component p[0] is, in this case, the component of the sample value vector pTemp corresponding to the predetermined component p[0]. It may be a sample adjacent to the top of the predetermined block or the top-left of the predetermined block, and is closest to the top-left corner of the predetermined block.

[0454] For example, in the case of an intra-sample prediction process according to predModeIntra which specifies an intra-prediction mode, the device is configured to apply, for example, the following steps, for example, to perform at least the first step.

[0455] 1. The matrix-based intra-prediction sample predMip[ x ][ y ](x = 0..predSize - 1, y = 0..predSize - 1) is derived as follows:

[0457] * - The variable modeId is set to be the same as predModeIntra.

[0458] - The weight matrix mWeight[ x ][ y ] with x = 0..inSize - 1 and y = 0..predSize * predSize - 1 is derived by calling the MIP weight matrix derivation process with mipSizeId and modeId as inputs.

[0459] - The matrix-based intra-prediction sample predMip[ x ][ y ](x = 0..predSize - 1, y = 0..predSize - 1) is derived as follows:

[0460]

[0461]

[0462] In other words, the device is configured to calculate a matrix vector product between an additional vector p[i] or {p[i];pTemp[0]} when mipSizeId is 2 and a predetermined prediction matrix mWeight, or when mipSizeId is less than 2, a prediction matrix mWeight having additional zero weight lines corresponding to the omitted components of p, where this is already assigned to an array of block positions {x,y} distributed within a predetermined block, becoming the array predMip[ x ][ y ]. The prediction vector corresponds to a connection of the row or column of predMip[ x ][ y ], respectively.

[0463] According to one embodiment, or according to another interpretation, the component is understood as a prediction vector, and when the device predicts a sample of a predetermined block based on the prediction vector, it is configured to calculate the sum of each component and a, for example, pTemp[0], for each component of the prediction vector.

[0464] The device uses a prediction vector, for example, predMip or Based on this, it can be optionally configured to perform the following additional steps when predicting samples of a predetermined block.

[0465] 2. Matrix-based intra-prediction sampling predMip[ x ][ y ] (x = 0..predSize - 1, y = 0..predSize - 1) is clipped, for example, as follows:

[0466] predMip[x][y] = Clip1( predMip[x][y])

[0467] 3. When isTransposed is TRUE, the array predMip[ x ][ y ] (x = 0..predSize - 1, y = 0..predSize - 1) is transposed as, for example, as follows:

[0468] predTemp[y][x] = predMip[x][y]

[0469] predMip = predTemp

[0470] 4. The predicted samples predSamples[ x ][ y ](x = 0..nTbW - 1, y = 0..nTbH - 1) are derived as follows, for example.

[0471] - If nTbW, which specifies the transform block width, is greater than predSize, or nTbH, which specifies the transform block height, is greater than predSize, the MIP prediction upsampling process is called with input block size predSize, matrix-based intra-predicted sample predMip[x][y] (x = 0..predSize - 1, y = 0..predSize - 1), transform block width nTbW, transform block height nTbH, upper reference sample refT[x] (x = 0..nTbW - 1) and left reference sample refL[y] (y = 0..nTbH - 1) as input, and the output is an array of predicted samples predSamples.

[0472] - Otherwise, predSamples[x][y] (x = 0..nTbW - 1, y = 0..nTbH - 1) is set to be the same as predMip[ x ][ y ].

[0473] In other words, the device is configured to predict predSamples of a predetermined block of samples based on the prediction vector predMip.

[0474] 8 other Intra prediction Mode and Block / Matrix based together Intra prediction mode use

[0475] The following description again indicates the possibility of combining block / matrix-based prediction with other intra-prediction modes. This represents an additional expression of the possibility in which the embodiments described in the subsequent section can be implemented.

[0476] Note that in the following, the term block-based intra-prediction is used to denote an intra-prediction mode that is identical to or can be emulated from the one indicated by the ALWIP above.

[0477] In conjunction with this, the embodiment described below in relation to FIG. 12 relates to a decoder and an encoder that support intra prediction for decoding / encoding a predetermined block (18) in which different intra prediction modes are supported. There is an angle intra prediction mode (500) in which reference samples (17) adjacent to the predetermined block (18) are used to fill the predetermined block (18) in order to obtain an intra prediction signal for the predetermined block (18). In particular, reference samples (17) arranged along the boundaries of the predetermined block (18), such as along the upper and left edges of the predetermined block (18), represent picture content that is extrapolated or copied into the interior of the predetermined block (18) along the predetermined direction (502). Before extrapolation or copying, the picture content represented by the neighbor samples (17) may be subject to interpolation filtering, or in other words, may be derived from the neighbor samples (17) using interpolation filtering. Angle intra prediction modes (500) differ from each other in intra prediction directions (502). Each angle intra prediction mode (500) may have an associated index, wherein the association between the angle intra prediction mode (500) and the index may cause the direction (500) to rotate monotonically clockwise or counterclockwise when the angle intra prediction modes (500) are ordered according to the associated mode index.

[0478] Additionally, there may be non-angle intra prediction modes. For example, 504 illustrates a planar intra prediction mode optionally included in the set (508) as is derived based on neighboring samples (17) according to a two-dimensional linear function defined by the horizontal slope, vertical slope, and offset in FIG. 12, which defines the predicted sample values ​​of a predetermined block (18). The horizontal slope, vertical slope, and offset are derived based on the adjacent samples (17). According to one embodiment, the intra prediction mode of the first set (508) includes a planar intra prediction mode (504).

[0479] A specific non-angle intra prediction mode, DC mode, included in set (508) is exemplified in 506. Here, one value, i.e., the DC value, is derived based on neighbor samples (17), and this one DC value is attributed to all samples of a predetermined block (18) to obtain an intra prediction signal. Two examples of non-intra prediction modes are shown, but only one or more may exist.

[0480] Intra-prediction modes (500, 504, and 506) form a set (508) of intra-prediction modes supported by encoders and decoders, competing with the block-based intra-prediction mode generally indicated by reference numeral 510 in terms of rate / distortion optimization, for which the example discussed above is given using the abbreviation ALWIP. As previously mentioned, according to these block-based intra-prediction modes (510), a matrix vector multiplication (520) is performed between a vector (514) derived from neighbor samples (17) and a predetermined prediction matrix (516). The result of the multiplication (520) is a prediction vector (518) used to predict samples of a predetermined block (18). The block-based intra-prediction modes (510) differ from one another in the prediction matrix (516) associated with each mode.

[0481] Accordingly, to summarize briefly, the encoder and decoder according to the embodiment described in this specification include a set of intra prediction modes (508), i.e., a first set of intra prediction modes and a set of block-based intra prediction modes (520), i.e., a second set of matrix-based intra prediction modes, and they compete with each other.

[0482] According to embodiments of the present application, a predetermined block (18) is coded / decoded using intra prediction in the following manner. In particular, first, a set selected syntax element (522) indicates whether the predetermined block (18) is predicted using any of the sets (508) of intra prediction modes or any of the sets (520) of block-based intra prediction modes. If the set selected syntax element indicates that the predetermined block (18) is to be predicted using any of the sets (508), i.e., any of the first sets of intra prediction modes, a list (528) of the most likely candidates among the sets (508) is interpreted / formed in an encoder and a decoder based on the intra prediction mode using neighboring blocks, neighboring blocks (18), exemplarily indicated by 524 and 526. Neighbor blocks (524, 526) may be determined for the position of a predetermined block (18) in a predetermined manner, such as determining a neighbor block that overlays a specific neighbor sample of block (18). For example, block 526 includes a sample at the top of the top left sample of block (18) and a sample to the left of the corner sample just mentioned. Of course, this is merely an example. The same applies to the number of neighbor blocks used for mode prediction, and is not limited to two for all embodiments. Two or more, or just one, may be used. If either of these blocks (524 and 526) is missing, the default intra prediction mode may be used by default as a substitute for the intra prediction mode of the missing neighbor block. The same applies even if any of the blocks (524 and 526) are coded / decoded using an inter prediction mode, such as motion-compensated prediction.

[0483] The composition of the list of modes, i.e., the list of most likely intra-predicted modes (528), among the set (508) is as follows. The length of the list (528), i.e., the number of most likely modes within it, can be fixed by default. The length can be 4 as shown in FIG. 12, or it can be different, such as 5 or 6. The latter case applies to the specific example described below. The index of the data stream described below may indicate the mode to be used in a predetermined block (18) among the list (528). Indexing is performed according to the list order or rank (530), for example, with a list index of variable length coded so that the length of the index increases monotonically according to the order (530). Thus, it is worth first filling the list (528) only with the most likely modes among the set (508), and placing the more likely modes upstream compared to the modes less likely to be suitable for the block (18) according to the order (530). The mode of the list (528) is derived based on the mode used in the neighboring blocks adjacent to the predetermined block (18), namely blocks (524 and 526). If any of the blocks (524 and 526) are intra-predicted using the block-based mode (510) in the set (520), the aforementioned mapping from this “ALWIP” or block-based mode (510) to the mode within the set (508), namely the non-ALWIP mode, is used. The latter mapping may, for example, map the majority (i.e., more than half) of the block-based mode (510) to the DC mode (506) (or any of the DC (506) or planar mode (504)).

[0484] According to one embodiment, the list of most likely intra prediction modes (528) is filled with planar intra prediction modes (504) in a manner independent of the intra prediction mode used when predicting neighbor blocks. Thus, for example, only DC intra prediction modes (506) and angle intra prediction modes (500) are filled in the list (528) according to the intra prediction modes used for predicting neighbor blocks (524, 526). Planar intra prediction modes (504) are located in the first position of the list of most likely intra prediction modes (528) independent of the intra prediction mode used when predicting neighbor blocks (524, 526), ​​for example.

[0485] In a manner described more specifically and exemplarily below, when neighbor blocks (524 and 526) are exclusively predicted by any angle intra prediction mode (500), the list of most likely intra prediction modes (528) is constructed in such a way that there is no DC intra prediction mode (506) in the list (528). When one neighbor block (524 or 526) is predicted by any angle intra prediction mode (500) and / or when both neighbor blocks (524 and 526) are predicted by any angle intra prediction mode (500), the DC intra prediction mode (506) is not in the list of most likely intra prediction modes (528). For example, according to the embodiment described below, the list (528) is populated with DC mode (506) only if the following condition is true for all neighboring blocks (524 and 526): coded using either of the non-angle intra-prediction modes (504 and 506) or intra-predicted using the block-based intra-prediction mode (510), which is mapped to either of the non-angle intra-prediction modes (504 and 506) through the aforementioned mapping from the block-based intra-prediction mode (510) to the mode within the set (508). Only in that case is the DC intra-prediction mode (506) located in the list (528). In that case, as can be seen in the subsequent example, it may be located before any angle intra-prediction mode (500) in order (530).

[0486] In other words, the list of most likely intra prediction modes (528) is filled with the DC intra prediction mode (506) only when each neighbor block (524, 526) is predicted using any of at least one non-angle intra prediction mode (504 and 506) together with a first set (508) including, for example, the DC intra prediction mode (506), or when the prediction is made using any of the block-based intra prediction modes (510) mapped to any of the at least one non-angle intra prediction mode (500) through mapping to the intra prediction mode within the first set (508) used to form the list of most likely intra prediction modes (528) among the block-based intra prediction modes (510) of the second set (520). For example, the DC intra prediction mode (506) is positioned before any angle intra prediction mode (500) in the list of most likely intra prediction modes (528).

[0487] Accordingly, resuming the description of how a predetermined block (18) is coded into a data stream (12), the data stream (12) includes the following: where a set selection syntax element (522) indicates that the predetermined block (18) will be coded by any mode among the first set (508), there is optionally an MPM syntax element (532) in the list (528) indicating whether an intra-prediction mode will be used for the predetermined block (18), and if yes, the data stream (12) includes an MPM list index (534) pointing to the list (528) indicating the mode to be used for the predetermined block (18) among the list (528) by the same indexing in order (530), namely the predetermined intra-prediction mode. However, as indicated by the MPM syntax element (532), if a mode in the set (508) is not in the list (528), the data stream (12) includes an additional syntax element (536) indicating which mode, namely a predetermined intra-predicted mode, will be used for the block (18) from the set (508). The additional syntax element (536) may indicate the mode in a manner that simply distinguishes the modes in the set (508) that are not included in the list (528).

[0488] In other words, the device for decoding a predetermined block (18) is configured to derive from a data stream an MPM syntax element (532) indicating whether the predetermined intra prediction mode of the first set (508) of intra prediction modes is within the list (528) of most likely intra prediction modes, for example, when a set selection syntax element (522) indicates that the predetermined block (18) will be predicted using one of the first set (508) of intra prediction modes. When an MPM syntax element (532) indicates that a predetermined intra prediction mode of a first set (508) of intra prediction modes is within a list (528) of most likely intra prediction modes, the device is configured to perform the formation of a list (528) of most likely intra prediction modes based on the intra prediction mode used when neighbor blocks (524, 526) adjacent to a predetermined block (100) are predicted, for example, and to derive an MPM list index (534) pointing to a predetermined intra prediction mode of the list (528) of most likely intra prediction modes from a data stream (12). If an MPM syntax element (532) from a data stream (12) indicates that a predetermined intra prediction mode of a first set (508) of intra prediction modes is not within the list (528) of most likely intra prediction modes, the device is configured to derive an additional list index (536) from the data stream representing a predetermined intra prediction mode from the first set of intra prediction modes. Accordingly, based on the MPM syntax element (532), the data stream (12) includes an MPM list index (534) or an additional list index (536) for the prediction of a predetermined block (18).

[0489] By eliminating the situation in which the list (528) includes a DC intra prediction mode (506), the following advantages are achieved. In particular, the inventors of the present application have found that "consuming" a valuable list position of the list (528) to a DC intra prediction mode (506) among the set (508) for coding / decoding a predetermined block (18) that must use any of the intra prediction modes of the set (508) as indicated by the syntax element (522), i.e., the set selection syntax element, has a negative effect on coding efficiency because the DC intra prediction mode (506) among the set (508) competes with the block-based intra prediction mode (510). Therefore, "consuming" the list position of the list (528) with the DC intra prediction mode (506) among the set (508) results in increasing the likelihood of a situation where the intra prediction mode to be finally used for the predetermined block (18), i.e., the predetermined intra prediction mode, is not within the list (528), so the syntax element (536), i.e., the additional list index, needs to be transmitted from the data stream (12).

[0490] In particular, since the syntax element (522) already indicates for block (18) whether the same should be predicted using any of the modes within set (508) or the block-based mode (510) of set (520), if the syntax element (522) indicates that the mode within set (508) is preferred for block (18) and accordingly the block-based mode (510) is not used for block (18), it appears that the probability that the DC prediction mode (506) in set (508) is suitable for block (18) is too low, and therefore its occurrence in list (528) should be limited to a very limited set of constellations of modes used in neighboring blocks (524 and 526), ​​that is, the constellations set above.

[0491] In another case, that is, when the set selection syntax element (522) indicates that a predetermined block (18) should be predicted using any of the block-based intra prediction modes (510), coding the block (18) into a data stream (12) and decoding from it can be performed in the manner described above. To this end, indexing can be used to index the selected one or to indicate which of the block-based intra prediction modes (510) of the set (520), i.e., the second set of block-based intra prediction modes, is used. An additional MPM syntax element (538) may indicate whether indexing is performed by an index (540), i.e., by an additional MPM list index representing the block-based intra prediction mode (510) to be used for block (18) among the list (542) of the most likely block-based intra prediction modes (510), i.e., by indexing along the list order (544), or whether the block-based intra prediction mode (510) to be used for block (18) is represented by an additional syntax element (546), i.e., by an additional list index representing the block-based intra prediction mode (510) among the set (520), where the latter syntax element (546) may distinguish, for example, only between modes (510) within the set (520) that are not already included in the list (542). The list composition of the list (542) may be performed based on the predicted modes of blocks (524 and 526). If any of the blocks (524 and 526) are outside the picture or are not available since inter-predicted, a default intra-predicted mode such as one of the sets (508) may be used instead.For each block (524, 526) intra-predicted using a mode in set (508) rather than set (520), the aforementioned mapping from the mode in set (508) to the mode in set (520) is used to obtain an intra-predicted mode (510), i.e., a predetermined block-based intra-predicted mode, for each block, i.e., a predetermined block (18), and a list (542) is interpreted based on the final block-based intra-predicted mode for blocks (524 and 526).

[0492] According to one embodiment, an apparatus for decoding a predetermined block (18) is configured to derive an additional MPM syntax element (538) from a data stream (12) indicating whether a predetermined block-based intra-prediction mode of a block-based intra-prediction mode (510) of a second set (520) is within a list (542) of most likely block-based intra-prediction modes, when a set selection syntax element (522) indicates that the predetermined block (18) is not predicted using one of the first set (508) of intra-prediction modes. If an additional MPM syntax element (538) indicates that a predetermined block-based intra prediction mode of a second set (520) of block-based intra prediction modes (510) is within a list (542) of most likely block-based intra prediction modes, the device is configured to form a list (542) of most likely block-based intra prediction modes based on the intra prediction mode used when neighbor blocks (524, 526) adjacent to a predetermined block (18) are predicted, for example, and to derive an additional MPM list index (540) from a data stream (12) that points to the list (542) of most likely block-based intra prediction modes as the predetermined block-based intra prediction mode. If an additional MPM syntax element (538) indicates that a predetermined block-based intra prediction mode of a second set (520) of block-based intra prediction modes is not within the list (542) of most likely block-based intra prediction modes, the device is configured to derive an additional list index (546) from the data stream (12) representing a predetermined block-based intra prediction mode from the block-based intra prediction modes of the second set (520). Accordingly, based on the additional MPM syntax element (538), the data stream (12) includes an additional MPM list index (540) or an additional list index (546) for the prediction of a predetermined block (18).

[0493] Although the additional MPM syntax element (538), additional MPM list index (540), and additional list index (546) are shown as being in parallel with the MPM syntax element (532), MPM list index (534), and additional list index (536) in the data stream (12) of FIG. 12, it is clear that the additional MPM syntax element (538) and the index associated with the additional MPM syntax element (538), e.g., additional MPM list index (540) or additional list index (546), are composed of the data stream (12) or the MPM syntax element (532) and the index associated with the MPM syntax element (532), e.g., MPM list index (534) or additional list index (536). Which of these syntax elements and indexes is included in the data stream (12) depends, for example, on the set selection syntax element (522).

[0494] Examples of the syntactic element parts of a data stream (12) written in pseudocode are as shown in FIGS. 19a through 19d, where reference numerals indicate which syntactic element corresponds to a previously discussed syntactic element.

[0495] The composition of the list of list (528) can be defined as follows, where, if the corresponding block (524 or 526) is intra-predicted using any of the block-based intra-predicted modes (510), candIntraPredModeA / B indicates which of the blocks (524 and 526) is predicted, such as A for block (524) and B for block (526), ​​or uses a mode in which the intra-predicted mode maps outside the set (508). INTRA_DC is used to indicate the mode (506), and the angle mode (500) is indicated by INTRA_ANGULAR# with a number (#) that orders the angle mode in a monotonically decreasing or increasing manner as the angle direction (502) increases, as exemplified above. The order among the modes within the set (508) may be as defined in the subsequent table where INTRA_PLANAR indicates the mode (504).

[0496] In the above example, note that the index (534) is actually distributed to the syntax elements (534' and 534''). The former (534') specifies the first location (530) of the list (528) where the INTRA_PLANAR mode (504) is inevitably located according to this example. The latter (534'') points to one of the subsequent locations of the list (528) where the DC mode (506) is included only in the special circumstances described as described.

[0497] Additionally, in the above example, if the syntax element (522) indicates that any mode among the modes within the set (508) is used, an additional syntax element is included in the data stream, and the additional syntax element somehow parameterizes the prediction mode within the set (500). For example, the syntax element (600) parameterizes or varies the area where the reference sample (17) is located based on the mode of the set (508) intra-predicting the interior of the block (18) in terms of the distance toward the outer perimeter of the block (18). Additionally, or alternatively, the syntax element (602) parameterizes or varies whether the reference sample (17) is used by the mode of the set (508) to intra-predict the interior of the block (18) entirely or in the en-block, or whether the intra-predicting is performed on the subdivided parts of the block (18), and the prediction residual coded into the data stream for one part is intra-predicted sequentially so that it can serve to recruit a new reference sample for intra-predicting the next part. Only when the syntax element (600) has a predetermined state corresponding, for example, to a region, is the latter coding option controlled by the syntax element available (and the corresponding syntax element may exist only in the data stream), whereby the reference sample (17) is adjacent to the block (18). The block may be subdivided and defined along a predetermined direction, such as horizontally, thereby leading to a part as high as the block (18), or vertically, thereby extending the part as wide as the block (18). When the partition controlling the use of the partition direction is signaled as active, the syntax element (604) may exist in the data stream.As can be seen, for example, the location of the list (528) reserved for the INTRA_PLANAR mode may be available only in the case of a specific parameterization of the mode by the just-mentioned parameterization syntax element, such as when the syntax element (600) has a predetermined state corresponding to the region where the reference sample (17) adjacent to the block (18) is, and / or when the partial intra prediction mode is not in an active state as signaled by the syntax element (602).

[0498] All syntax elements shown in the table and not specifically mentioned above are optional and will not be explained further.

[0499] - If candIntraPredModeB is equal to candIntraPredModeA and candIntraPredModeA is greater than INTRA_DC, candModeList[ x ] for x = 0..4 is derived as follows:

[0500] candModeList[ 0 ] = candIntraPredModeA

[0501] candModeList[ 1 ] = 2 + ( ( candIntraPredModeA + 61 ) % 64 )

[0502] candModeList[ 2 ] = 2 + ( ( candIntraPredModeA - 1 ) % 64 )

[0503] candModeList[ 3 ] =2 + ( ( candIntraPredModeA + 60 ) % 64 )

[0504] candModeList[ 4 ] = 2 + ( candIntraPredModeA % 64 )

[0505] - Otherwise, if candIntraPredModeB is not equal to candIntraPredModeA and candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC, the following applies:

[0506] - The variables minAB and maxAB are derived as follows:

[0507] minAB = Min(candIntraPredModeA, candIntraPredModeB)

[0508] maxAB = Max( candIntraPredModeA, candIntraPredModeB )

[0509] - If both candIntraPredModeA and candIntraPredModeB are greater than INTRA_DC, candModeList[x] for x = 0..4 is derived as follows:

[0510] candModeList[ 0 ] = candIntraPredModeA

[0511] candModeList[ 1 ] = candIntraPredModeB

[0512] If - maxAB - minAB is 1, the following applies:

[0513] candModeList[ 2 ] = 2 + ( ( minAB + 61 ) % 64 )

[0514] candModeList[ 3 ] = 2 + ( ( maxAB - 1 ) % 64 )

[0515] candModeList[ 4 ] = 2 + ( ( minAB + 60 ) % 64 )

[0516] - Otherwise, if maxAB - minAB is greater than or equal to 62, the following applies.

[0517] candModeList[ 2 ] = 2 + ( ( minAB - 1 ) % 64 )

[0518] candModeList[ 3 ] = 2 + ( ( maxAB + 61 ) % 64 )

[0519] candModeList[ 4 ] = 2 + ( minAB % 64 )

[0520] - Otherwise, if maxAB - minAB is 2, the following applies. candModeList[ 2 ] = 2 + ( ( minAB - 1 ) % 64 )

[0521] candModeList[ 3 ] = 2 + ( ( minAB + 61 ) % 64 )

[0522] candModeList[ 4 ] = 2 + ( ( maxAB - 1 ) % 64 )

[0523] - Otherwise, the following applies.

[0524] candModeList[ 2 ] = 2 + ( ( minAB + 61 ) % 64 )

[0525] candModeList[ 3 ] = 2 + ( ( minAB - 1 ) % 64 ) (8-36)

[0526] candModeList[ 4 ] = 2 + ( ( ( maxAB + 61 ) ) % 64 )

[0527] - Otherwise (candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC), candModeList[ x ] for x = 0..4 is derived as follows:

[0528] candModeList[ 0 ] = maxAB

[0529] candModeList[ 1 ] = 2 + ( ( maxAB + 61 ) % 64 )

[0530] candModeList[ 2 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-41)

[0531] candModeList[ 3 ] = 2 + ( ( maxAB + 60 ) % 64 )

[0532] candModeList[ 4 ] = 2 + ( maxAB % 64 )

[0533] - Otherwise, the following applies.

[0534] candModeList[ 0 ] = INTRA_DC

[0535] candModeList[ 1 ] = INTRA_ANGULAR50 (

[0536] candModeList[ 2 ] = INTRA_ANGULAR18

[0537] candModeList[ 3 ] = INTRA_ANGULAR46

[0538] candModeList[ 4 ] = INTRA_ANGULAR54

[0539] Intra prediction mode Related names 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66

[0540] 9 other Intra prediction mode and block / matrix based with the use of quadratic transformations Intra prediction mode Implementation using yesThe following description presents an example for combining block / matrix-based prediction with other intra prediction modes, along with using a quadratic transformation to code prediction residuals. The above presentation regarding the possibility of matrix / based intra prediction (ALWIP) and its combination with other intra prediction modes will serve as an example for implementing the example described below. For example, all details regarding the configuration of the MPM list related to the limited inclusion of the DC mode in FIG. 12 are optional.

[0541] As described above, matrix-based intra prediction (MIP), also referred to herein as block-based intra prediction and ALWIP, generates an intra prediction signal for a rectangular block by performing matrix-vector multiplication, wherein the output of the matrix-vector multiplication can be considered as a prediction signal for a downsampled block and the input of the matrix-vector multiplication can consist of downsampled boundary samples. If the output is considered as a prediction signal for a downsampled block, this prediction signal must undergo an upsampling (or linear interpolation) step before obtaining the final prediction signal.

[0542] Meanwhile, for existing intra-prediction modes such as planar mode (504), DC mode (506), and angle mode (500) (also indicated as the mode of set (508) in the description above), the inseparable second transformation (LFNST) is a tool used to transform the prediction residuals corresponding to these intra-prediction modes. Here, a set S of transformation sets is given such that each existing intra-prediction mode is associated with one of these transformation sets. Then, the decoder can extract whether to apply the given block LFNST from the bitstream. In this case, depending on the intra-prediction mode used in the current block, one transformation set is provided in set S, and if this transformation set consists of two or more transformations, which transformation T is used in this set can be extracted from the bitstream. Then, in the decoder, the transformation T is applied as a second transformation Ts, which means that it is applied to a subset (622) of the residual transformation coefficients (620) of the separable first transformation Tp, as shown in FIG. 14, for example.

[0543] The problem is that the aforementioned second transformation (T) is defined a priori only for existing intra prediction modes. Providing specific second transformations (Ts) for each MIP mode (510) can be too costly in terms of memory requirements for storing additional transformations.

[0544] FIG. 13 illustrates a decoder that solves this problem. The decoder decodes a specific block (18) of a picture using intra-prediction. An encoder according to an embodiment includes parallel features and / or functions as a decoder.

[0545] The decoder / encoder is configured to select (602) a predetermined intra prediction mode (604) from a plurality of intra prediction modes (600), including an intra prediction mode of a first set (508) and a matrix-based intra prediction mode (510) of a second set (520). This intra mode selection (602) is performed by the decoder based on the data stream (12), wherein the encoder is configured to signal the predetermined intra prediction mode (604) in the data stream (12). The intra mode selection (602) may be performed as described in relation to FIG. 12.

[0546] The intra prediction mode of the first set (508) includes a DC intra prediction mode (506) and an angle prediction mode (500), and optionally a plane intra prediction mode (504). If the predetermined intra prediction mode (604) is a matrix-based intra prediction mode (510) of the second set (520), the decoder / encoder is configured to obtain a prediction vector (518) using a matrix vector product (512) between a vector (514) derived from a reference sample (17) in the neighborhood of the predetermined block (18) and a prediction matrix (516) associated with each matrix-based intra prediction mode (510), based on which samples of the predetermined block (18) are predicted. Prediction of a predetermined block (18) using a matrix-based intra prediction mode (510) as a predetermined intra prediction mode (604) can be performed by a decoder / encoder according to the embodiment of FIGS. 6 and FIGS. 11. The decoder / encoder is configured to derive a prediction signal (606) for a predetermined block (18) using the predetermined intra prediction mode (604).

[0547] The decoder / encoder relies on one or more quadratic transformations Ts in such a way that when the predetermined intra prediction mode (604) is included in the intra prediction mode of the first set (508) and when the predetermined intra prediction mode (604) is included in the matrix-based intra prediction mode (510) of the second set (520), the subset (610) is not empty. (i1) -Ts (in) It is configured to select (608) a subset (610) of, at this time, from the set (612) of the second transformation Ts, for example, Ts (1) - Ts (N) Among them, i1 is in the range of 1 to N and i n It is in the range of i1 to N.

[0548] According to one embodiment, the decoder / encoder is configured to select (608) a subset (610) such that each of the second transformation Ts of the set (612) of second transformation Ts is included in one or more subsets (610) of second transformation Ts selected for at least one of the intra prediction modes within the first set (508) and the second set (520). Thus, the subset (610) may be identical to the set (612) of second transformations. Such a subset may be selected for one or more matrix-based intra prediction modes (510). For one or more matrix-based intra-prediction modes (510), it may be possible to select all quadratic transformations Ts of a set of quadratic transformations (612), thereby including a subset (610) of the one or more matrix-based intra-prediction modes (510) that includes quadratic transformations T selectable for an intra-prediction mode within a first set (508) of intra-prediction modes. Thus, for at least one of the matrix-based intra-prediction modes (510), a specific additional quadratic transformation is not required in the set of quadratic transformations (612).

[0549] According to one embodiment, the decoder / encoder has a matrix (610) of quadratic transformations selected for any matrix-based intra-prediction mode (510), and each quadratic transformation Ts is a subset of quadratic transformations selected for at least one intra-prediction mode within a first set (508) that does not belong to an angle prediction mode (500), e.g., 610 DC and / or 610 planar The decoder / encoder is configured to select (608) a subset (610) in a manner that includes it. FIG. 15 illustrates different possible subsets of the quadratic transformation Ts selected for any matrix-based intra prediction mode (510). For each matrix-based intra prediction mode (510), the decoder / encoder selects a first combination (611) of the subset (610) of the quadratic transformation for the matrix-based intra prediction mode (510). matrix It can be configured to select a subset (610) of ).

[0550] As illustrated in FIG. 15, a selectable subset (610) for one or more matrix-based intra prediction modes (510) is a selectable subset for a DC intra prediction mode (506), e.g., 610 DC1 or 610 DC2 , or a selectable subset for the planar intra prediction mode (504), e.g., 610 planar1 or 610 planar2 It may be the same as. A selectable subset (610) for one or more matrix-based intra prediction modes (510) is a subset (610 matrix4 As indicated by ), a subset of secondary transformation Ts, e.g., 610, selected for one intra prediction mode within a first set (508) that does not belong to an angle prediction mode (500). DC One or more quadratic transformations of, e.g., Ts (ax) Ts (ay) It can include only.

[0551] The selectable subset (610) for the matrix-based intra-prediction mode (510) is the subset (610) matrix1 As indicated by ), two or more subsets selectable for the DC intra prediction mode (506), e.g., 610 DC1 and 610 DC2 It may include one or more secondary transformations among, or a subset (610 matrix3 As indicated by ), two or more subsets selectable for the planar intra prediction mode (504), e.g., 610 planar1 and 610 planar2 It may include one or more second transformations among them.

[0552] An additional possible subset (610) selectable for the matrix-based intra prediction mode (510) is one or more subsets selectable for the DC intra prediction mode (506), e.g., 610 DC2 One or more of the second transformations and subsets (610 matrix2 One or more subsets selectable for the planar intra prediction mode (504) as indicated by ), e.g., 610 planar1 It may include one or more second transformations among them.

[0553] According to one embodiment, the decoder / encoder has a first combination (611) of a subset (610) of a second transformation selected for a matrix-based intra prediction mode (510). matix ) and a subset of the selected quadratic transformations for all angle intra prediction modes (610 angular The second combination of ) (611 angular It is configured to select (608) a subset (610) in such a way that the intersection between ) is empty. A third combination (611 DC ) includes all subsets (610DC) selected for the DC intra prediction mode (506) and the fourth combination (611 planar) is all subsets (610) selected for the planar intra prediction mode (504) planar ...includes ). For example, due to downsampling optionally applied to the reduced prediction signal (606), the MIP mode (510) is omnidirectional like the planar mode (504) and DC mode (506), so their prediction residuals (618) have more statistical similarity to those of the DC mode (506) and planar mode (504) than to those of the angle mode (500).

[0554] Additionally, as illustrated in FIG. 13, the decoder is configured to derive (616) a transformed version of the prediction residual for a predetermined block (18) associated with a spatial domain version (618) of the prediction residual for a predetermined block (18) through a transformation T defined by the connection of a first transformation Tp and a predetermined second transformation Ts from a subset (610) of second transformations from a data stream encoded by the encoder. As illustrated in FIG. 14, the encoder may be configured to apply the transformation T to a subset (622) of coefficients (620) of the first transformation Tp when the predetermined intra prediction mode is included in the first set (508) of intra prediction modes and when the predetermined intra prediction mode is included in the second set (520) of matrix-based intra prediction modes (510). The decoder obtains the inverse T of the transformation T to obtain the spatial domain version (618) of the predicted residual of the predetermined block (18). -1 It can be configured to use. For example, the first transformation is a separable 2D transformation, and the second transformation is, for example, an inseparable 2D transformation.

[0555] The decoder is configured to restore (624) a predetermined block (18) using a prediction signal (606) and a prediction residual (618) for a predetermined block (18).

[0556] If a subset (610) of one or more secondary transformations Ts includes one or more secondary transformations Ts, the decoder may be configured to select a predetermined secondary transformation Ts from the subset (610) of one or more secondary transformations according to a secondary transformation indicator syntax element transmitted in the data stream (12) for a predetermined block. The secondary transformation indicator syntax element may be an index pointing to the selected subset (610) of one or more secondary transformations. In this case, the encoder may be configured to transmit a secondary transformation indicator syntax element in the data stream (12).

[0557] According to one embodiment, if the dimensions of a predetermined block (18) satisfy a predetermined criterion, the decoder is configured to infer that the transformation T associated with the spatial domain version (618) of the prediction residuals for the predetermined block (18) is a first-order transformation Tp. Otherwise, the transformation T may be a concatenation of the first-order transformation Tp and the predetermined second-order transformation Ts. The decoder makes available a second set (520) of matrix-based intra-prediction modes (510) for the selection (602) of a predetermined intra-prediction mode (604), regardless of the dimensions of the predetermined block (18) satisfying the predetermined criterion. The predetermined criterion for the dimensions of the predetermined block (18) may be relevant only to the subset selection (608) and not to the intra-mode selection (602). For example, there are block dimensions where MIP / ALWIP (510) is available but LFNST, i.e., subset selection (608), is not. For blocks (18) of these dimensions, secondary transformation mark syntax elements are transmitted from the data stream and do not need to be read from the data stream. For example, if the dimensions fall below a predetermined threshold, the predetermined criteria are satisfied. For small predetermined blocks (18), this may not be beneficial, which means that in the latter case, the additional signaling cost to indicate whether LFNST is applied to the block in MIP mode is, on average, higher than the gain obtained by allowing LFNST transformation for MIP mode.

[0558] According to one embodiment, the decoder is configured to read non-zero region indications transmitted from the data stream (12) for each predetermined block (18) representing non-zero transformed domain regions (623) within the transformed version (616) of the prediction residual for the predetermined block (18) as illustrated in FIG. 16. All non-zero coefficients are located only in the non-zero transformed domain regions (623). The non-zero region indications are transmitted from the data stream (12), for example, by the encoder. The decoder / encoder is configured to decode / encode coefficients within the non-zero transformed domain regions (623) from / to the data stream (12). An LP syntax element may serve as a non-zero region indication. Along the scan path leading from the DC coefficient location to the opposite or highest frequency coefficient location, the last non-zero coefficient location is indicated by an LP syntax element. The LP syntax element can serve as a measure of the expected count of non-zero coefficients within the non-zero transformation domain region (623).

[0559] According to one embodiment, the decoder is configured to infer that the transformation T is a first transformation Tp such that the transformed version (616) of the prediction residual for a predetermined block (18) is associated with the spatial domain version (618) of the prediction residual of the predetermined block (18) according to the extension and / or location of a non-zero transformation domain region (623) satisfying a first predetermined criterion and / or a plurality of non-zero coefficients within a non-zero transformation domain region (623) satisfying a second predetermined criterion.

[0560] For example, the first predetermined criterion is satisfied if the non-zero transformation domain region (623) does not exclusively contain the subset (622) of the coefficients of the first transformation Tp to which the second transformation Tp is applied by connection. This is based on the idea that the second transformation Ts must cover all non-zero coefficients of the transformation coefficients of the first transformation Tp. If the non-zero coefficients are outside the subset (622) of the coefficients of the first transformation Tp, it may not be advantageous to apply the predetermined second transformation, and for this reason, the decoder infers that the transformation T is the first transformation Tp. If a non-zero transformation domain region (623) is entirely located within a subset (622) of coefficients of the first transformation Tp to which the second transformation Tp is applied by connection, the decoder may perform a subset selection (608) from a subset (610) of the second transformation applied to the subset (622) of coefficients of the first transformation Tp to obtain a spatial domain version (618) of the predicted residual of a predetermined block (18), and may apply a transformation T defined by the connection of the first transformation Tp and the predetermined second transformation Ts. For example, refer to FIG. 16.

[0561] A second predetermined criterion is equally satisfied, for example, when the number of non-zero coefficients within a non-zero transformation domain region (623) falls below a predetermined threshold. This is based on the idea that if the number falls below a predetermined threshold, the number of non-zero coefficients within the non-zero transformation domain region (623) does not need to be further reduced by the second transformation Ts. If the number of non-zero coefficients of the first transformation Tp is small, it may not be advantageous to apply the predetermined second transformation further, and thus the decoder infers that the transformation T is the first transformation Tp. If the number of non-zero coefficients falls below a predetermined threshold, the additional signaling cost for the predetermined second transformation will be higher than the increased coding efficiency achieved by the predetermined second transformation. If the number of non-zero coefficients within the non-zero transformation domain region (623) is equal to or exceeds a predetermined threshold, the decoder performs a subset selection (608) and can obtain a spatial domain version (618) of the predicted residual of a predetermined block (18) by applying a transformation T defined by the connection of the predetermined secondary transformation Ts and the primary transformation Tp from among the subset (610) of the secondary transformations applied to the subset (622) of the coefficients of the primary transformation Tp.

[0562] According to one embodiment, the decoder / encoder is configured to derive / encode a set selection syntax element (522) from / to the data stream (12) indicating whether a predetermined block (18) is predicted using one of a first set (508) of intra prediction modes, for example, as described in relation to FIG. 12. If the set selection syntax element (522) indicates that the predetermined block (18) should be predicted using one of the intra prediction modes of the first set (508), the decoder / encoder is configured to form a list (528) of the most likely intra prediction modes based on the intra prediction modes in which neighboring blocks (524 and 526) adjacent to the predetermined block (18) are predicted, and to derive / signal an MPM list index (534) from / to the data stream (12) pointing to the list (528) of the most likely intra prediction modes for the predetermined intra prediction mode (604). If the set selection syntax element (522) indicates that the predetermined block (18) is not predicted using one of the intra prediction modes of the first set (508), the decoder / encoder is configured to derive / encode an additional index (540 and / or 546) from / to the data stream representing the predetermined intra prediction mode (604) among the matrix-based intra prediction modes (510) of the second set (520).

[0563] The decoder and encoder described in relation to FIG. 13 may include additional features and / or functions described in relation to FIG. 12.

[0564] If the predetermined intra-prediction mode (604) for the predetermined block (18) is a matrix-based intra-prediction mode (510) of the second set (520), the device for decoding the predetermined block (18), i.e., the decoder according to FIG. 13, and / or the device for encoding the predetermined block (18), i.e., the encoder according to FIG. 13, may include one or more of the following functions.

[0565] According to one embodiment, the device is configured to form a sample value vector, e.g., a sample value vector (400), among a plurality of reference samples (17) as described in connection with one of the embodiments of FIGS. 6 to 9, and to derive a vector (514) from the sample value vector such that the sample value vector is mapped to a vector (514) by a predetermined inverse linear transformation. In this case, the vector (514) may be understood as an additional vector. The vector (514) is determined and / or defined as described for the additional vector (402) in connection with one of the embodiments of FIGS. 8 to 11, e.g.

[0566] According to one embodiment, the device is configured to form a sample value vector among a plurality of reference samples (17) by adopting one of a plurality of reference samples as an individual component of the sample value vector for each component of the sample value vector, and / or by averaging two or more components of the sample value vector to obtain each component of the sample value vector.

[0567] Multiple reference samples (17) are arranged within the picture along the outer edge of a predetermined block (18), for example.

[0568] The invertible linear transformation is defined such that, for example, a predetermined component of the vector (514), for example, the additional vector, becomes a, and each of the other components of the vector (514), excluding the predetermined component, is equal to the corresponding component of the sample value vector minus a. The value a is, for example, a predetermined value (1400).

[0569] According to one embodiment, a predetermined value (1400) is one of an average such as the arithmetic mean or weighted average of the components of a sample value vector, a base value, a value signaled in a data stream in which a picture is coded, and a component of the sample value vector corresponding to the predetermined component.

[0570] An invertible linear transformation is defined such that, for example, a predetermined component of a vector (514), for example, an additional vector, is a, and each of the other components of the vector (514), excluding the predetermined component, is defined such that a is the result of subtracting a from the corresponding component of the sample value vector, where a is the arithmetic mean of the components of the sample value vector.

[0571] A reversible linear transformation is defined such that, for example, a predetermined component of a vector (514), for example, an additional vector, is a, and each of the other components of the vector (514), excluding a predetermined component, is defined such that a is the corresponding component of the sample value vector minus a, where a is the component of the sample value vector corresponding to the predetermined component. The device is configured to include, for example, a plurality of reversible linear transformations—each of which is associated with one component of the vector (514)—select a predetermined component from the components of the sample value vector, and use a reversible linear transformation from among the plurality of reversible linear transformations associated with the predetermined component as the predetermined reversible linear transformation.

[0572] According to one embodiment, all matrix components of the prediction matrix (516) within the columns of the prediction matrix (516) corresponding to the predetermined components of the vector (514), for example, the additional vector, are zero. The device is configured to calculate the matrix-vector product (512) between the reduced prediction matrix resulting from the prediction matrix (516) by removing columns and the additional vector (410) resulting from the vector (514) by removing predetermined components, as illustrated in FIG. 9b.

[0573] According to one embodiment, the device is configured to calculate the sum of each component and a for each component of the prediction vector (518) when predicting samples of a predetermined block (18) based on the prediction vector (518).

[0574] A matrix resulting from the sum of each matrix component of the prediction matrix (516) within the columns of the prediction matrix (516) corresponding to a predetermined component of the additional vector, for example, a matrix obtained by multiplying the vector (514) by an inverse linear transformation (i.e., the sum of matrix C (405) and matrix M (1300) exemplified in FIG. 9a, resulting in matrix B of FIG. 8), corresponds, for example, to a quantized version of the machine learning prediction matrix (i.e., prediction matrix A (1100) shown in FIG. 8).

[0575] According to one embodiment, the device is configured to calculate the matrix vector product (512) using fixed-point arithmetic operations.

[0576] According to one embodiment, the device is configured to calculate the matrix vector product (512) without floating-point arithmetic operations.

[0577] According to one embodiment, the device is configured to store a fixed-point number representation of the prediction matrix (516).

[0578] According to one embodiment, the device is configured to represent a prediction matrix (516) using prediction parameters and to calculate a matrix-vector product (512) by performing multiplication and summation on a vector (514), e.g., components of an additional vector, and the prediction parameters and intermediate results generated therefrom, wherein the absolute value of the prediction parameters can be expressed in an n-bit fixed-point number representation, where n is equal to or less than 14, or alternatively 10, or alternatively 8. This can be done similarly or as described in FIG. 10 or FIG. 11.

[0579] The prediction parameters include, for example, weights in which each weight is associated with a corresponding matrix component of the prediction matrix (516).

[0580] The prediction parameters further include, for example, one or more scaling factors associated with one or more corresponding matrix components of the prediction matrix (516) to scale weights associated with one or more corresponding matrix components of the prediction matrix (516), and / or one or more offsets associated with one or more corresponding matrix components of the prediction matrix (516) to offset weights associated with one or more corresponding matrix components of the prediction matrix (516).

[0581] According to one embodiment, the device is configured to use interpolation to calculate at least one sample location of a predetermined block (18) based on a prediction vector (518) in predicting samples of a predetermined block (18) based on a prediction vector (518) in which each component is associated with a corresponding location within the predetermined block (18).

[0582] The decoder and encoder described in relation to FIG. 13 may include additional characteristics and / or functions as described in relation to one or more embodiments of FIG. 6 to 11.

[0583] Accordingly, the solution provided by the present invention is to associate one specific set of transformations (160) from a given set of transformations S (612) with each MIP mode (510), the latter being originally defined for the existing intra-prediction mode, i.e., the intra-prediction mode within the first set (508). One specific way to do this is for all MIP modes (510) to use the LFNST transformation, i.e., the second transformation, which was originally designed for the plane mode (504) and the DC mode (506). For example, because the MIP modes (510) are omnidirectional like the plane mode (504) and the DC mode (506) due to downsampling optionally applied to the reduced prediction signal, their prediction residuals have more statistical similarity to those of the DC plane (506) and plane mode (504) than to those of the angle mode (500).

[0584] It has been found that allowing LFNST for MIP (510) in the manner described above, that is, allowing the use of a second transformation, may be advantageous in terms of coding efficiency for some block shapes, but may not be beneficial for other block shapes, which means that for other shapes, the additional signaling cost of signaling whether LFNST is applied to block (18) in MIP mode (510) is higher on average than the gain obtained by allowing LFNST transformation for MIP mode (510). Therefore, according to an embodiment of the present invention, the LFNST and MIP (510) combination described above is, in principle, allowed only for a subset of all block shapes where such combination is possible.

[0585] FIG. 17 illustrates a method (6000) for decoding a predetermined block (18) of a picture using intra prediction, the method comprising the step (602) of selecting a predetermined intra prediction mode (604) based on a data stream from a plurality of intra prediction modes (600), including a first set (508) of intra prediction modes including a DC intra prediction mode (506) and an angle prediction mode (500) and optionally a planar intra prediction mode (504), and a second set (520) of matrix-based intra prediction modes (510), and obtaining a prediction vector (518) using a matrix vector product (512) between a vector (514) derived from a reference sample (17) in the neighborhood of the predetermined block and a prediction matrix (516) associated with each matrix-based intra prediction mode, based on which sample of the predetermined block is predicted according to each of these. If a predetermined intra-prediction mode is included in the intra-prediction mode of the first set (508) and the predetermined intra-prediction mode is included in the matrix-based intra-prediction mode (510) of the second set (520), a prediction signal (606) for a predetermined block is derived using the predetermined intra-prediction mode (6100), and one or more subsets (610) of secondary transformations from a secondary transformation set (612) are selected in a manner that depends on the predetermined intra-prediction mode so that the subset (610) is not empty (608).The method (6000) includes the step (614) of deriving from a data stream a transformed version (616) of the prediction residual for a predetermined block (18) associated with a spatial domain version (618) of the prediction residual for a predetermined block through a transformation (T) defined as a connection of a first transformation (Tp) and a predetermined second transformation (Ts), among a subset (610) of second transformations applied to a subset (622) of coefficients (620) of a first transformation. Additionally, the method (6000) includes the step (624) of restoring a predetermined block using a prediction signal and a prediction residual for the predetermined block (18).

[0586] FIG. 18 illustrates a method (7000) for encoding a predetermined block (18) of a picture using intra prediction, the method comprising the step (602) of selecting a predetermined intra prediction mode (604) from a plurality of intra prediction modes (600), including a first set (508) of intra prediction modes including a DC intra prediction mode (506) and an angle prediction mode (500), and a second set (520) of matrix-based intra prediction modes (510), and obtaining a prediction vector (518) using a matrix vector product (512) between a vector (514) derived from a reference sample (17) in the neighborhood of the predetermined block and a prediction matrix (516) associated with each matrix-based intra prediction, based on which sample of the predetermined block is predicted according to each of these. Additionally, the method (7000) includes the steps of signaling (7100) a predetermined intra-prediction mode (604) in a data stream, deriving a prediction signal (606) for a predetermined block using the predetermined intra-prediction mode (7200), and selecting one or more subsets (610) of a set (612) of quadratic transformations in a manner dependent on the predetermined intra-prediction mode such that the subset (610) is not empty when the predetermined intra-prediction mode is included in the intra-prediction mode of a first set (508) and the predetermined intra-prediction mode is included in the matrix-based intra-prediction mode (510) of a second set (520).The above method includes the step (614) of encoding (616) a transformed version of the prediction residual for a predetermined block (18) into a data stream, which is associated with a spatial domain version (618) of the prediction residual for a predetermined block, when the predetermined intra prediction mode is included in the intra prediction mode of the first set (508) and when the predetermined intra prediction mode is included in the matrix-based intra prediction mode (510) of the second set (520), and the transformation (T) defined by the connection of the predetermined second transformation (Ts) among the subset (610) of the second transformation applied to the first transformation (Tp) and the coefficients (620) of the first transformation, wherein the predetermined block can be restored using the prediction signal and prediction residual for the predetermined block (18) (624).

[0587] References

[0588] [1] P. Helle et al., "Nonlinear Weighted Intra Prediction", JVET-L0199, Macau, China, October 2018.

[0589] [2] F. Bossen, J. Boyce, K. Suehring, X. Li, V. Seregin, "JVET Common Test Conditions and Software Reference Configurations for SDR Video", JVET-K1010, Ljubljana, SI, July 2018.

[0590] Additional implementation examples and examples

[0591] Generally, an example can be implemented as a computer program product having program instructions, and these computer instructions operate to perform one of the methods when the computer program product is executed on a computer. The program instructions can be stored, for example, on a machine-readable medium.

[0592] Another example includes a computer program for performing one of the methods described herein stored in a machine-readable carrier. That is, an example of a method is a computer program having program instructions for performing one of the methods described herein when the computer program is executed on a computer.

[0593] Accordingly, an additional example of the method is a data carrier medium (or digital storage medium or computer-readable medium) comprising a computer program for performing one of the methods described herein. The data carrier medium, digital storage medium or recording medium is tangible and / or non-transient rather than intangible and transient signals.

[0594] Accordingly, an additional example of the method is a data stream or signal sequence representing a computer program for performing one of the methods described herein. The data stream or signal sequence may be transmitted, for example, via a data communication connection, for example, the Internet.

[0595] Additional examples include processing means, e.g., a computer, or a programmable logic device that performs one of the methods described herein.

[0596] Additional examples include a computer with a computer program installed for performing one of the methods described in this specification.

[0597] Additional examples include a device or system that transmits (e.g., electronically or optically) a computer program to perform one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The device or system may include, for example, a file server for transmitting the computer program to the receiver.

[0598] In some examples, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the method described herein.

[0599] In some examples, an electric field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, the method may be performed by any suitable hardware device.

[0600] The examples described above are merely illustrative of the principles described above. It is understood that modifications and changes to the arrangements and details described herein will be apparent. Accordingly, the scope of the pending claims is limited to the specific details presented by the description of the examples in this specification, rather than to the specific details provided.

[0601] Identical or equivalent elements or components having the same or equivalent function are indicated in the following description by the same or equivalent reference number, even if they occur in different drawings.

Claims

Claim 1 A method for decoding a picture from a data stream, the method comprising: selecting a matrix-based intra-prediction mode for a block of the picture based at least partially on a mark included in the data stream; deriving a prediction signal for the block using the selected matrix-based intra-prediction mode; selecting a subset of the second-order transformations from a set of second-order transformations for transforming prediction residuals, wherein the subset comprises a low-frequency non-separable second-order transformation (LFNST) associated with both a planar intra-prediction mode and a DC intra-prediction mode; deriving the prediction residuals for the block from the data stream; transforming the prediction residuals using the LFNST from the selected subset; and restoring the block using the prediction signal and the transformed prediction residuals for the block. Claim 2 A method according to claim 1, wherein when the matrix-based intra-prediction mode is selected, the step of deriving the prediction signal comprises: deriving a prediction vector generated from a matrix vector product between a vector derived from a reference sample in the neighborhood of the block and a prediction matrix associated with the selected matrix-based intra-prediction mode; and upsampling the prediction vector to obtain a prediction signal of the block. Claim 3 In claim 1, the step of selecting a subset of the second transformation The above-mentioned selected matrix-based intra-prediction mode, and A method based on the size of the above block. Claim 4 The method of claim 1 further comprises the steps of: decoding a mark representing an LFNST from a subset of the selected second transformations; and selecting an LFNST from the subset for transforming the predicted residual based on the mark designating one of two LFNSTs. Claim 5 The method of claim 1, wherein the step of transforming the predicted residual comprises: applying LFNST(Ts) to a subset of coefficients of a first-order transformation (Tp) to obtain the transformation; and transforming the predicted residual using the transformation, wherein the first-order transformation (Tp) is a separable 2D transformation. Claim 6 The method of claim 1, wherein the step of selecting the intra prediction mode for a block of the picture comprises: decoding a set of syntax elements from the data stream indicating whether the block is predicted using one of a set of intra prediction modes including the planar intra prediction mode, at least one angle prediction mode, and the DC intra prediction mode; and selecting the matrix-based intra prediction mode from a set of matrix-based intra prediction modes in response to a determination that the set of syntax elements indicates that the block is not predicted using any mode of the set of intra prediction modes. Claim 7 In claim 1, the step of transforming the predicted residual comprises: using a transformation defined by the connection of a first transformation (Tp) and a second transformation (Ts) selected from a subset of the second transformations, wherein the second transformation (Ts) corresponds to the LFNST. Claim 8 A non-transient computer-readable storage medium, wherein the non-transient computer-readable storage medium stores instructions for performing the method of any one of claims 1 to 7 when executed by at least one processor of an electronic device. Claim 9 An apparatus for decoding a picture from a data stream, the apparatus comprising at least one processor, wherein the at least one processor comprises: for a block of the picture, selecting a matrix-based intra-prediction mode based at least partially on a mark included in the data stream; deriving a prediction signal for the block using the selected matrix-based intra-prediction mode; selecting a subset of the second-order transformations from a set of second-order transformations for transforming prediction residuals, wherein the subset comprises a low-frequency non-separable second-order transformation (LFNST) associated with both a planar intra-prediction mode and a DC intra-prediction mode; deriving a prediction residual for the block from the data stream; transforming the prediction residual using the LFNST from the selected subset; and restoring the block using the prediction signal and the transformed prediction residual for the block. Claim 10 In claim 9, for deriving the prediction signal when the matrix-based intra-prediction mode is selected, the at least one processor is configured to: derive a prediction vector generated from a matrix vector product between a vector derived from a reference sample in the neighborhood of the block and a prediction matrix associated with the selected matrix-based intra-prediction mode, and to upsample the prediction vector to obtain the prediction signal of the block. Claim 11 In claim 9, the above at least one processor is: The above-mentioned selected matrix-based intra-prediction mode, and A device further configured to select a subset of the second transformation based on the size of the block. Claim 12 In claim 9, the device is further configured such that the at least one processor: decodes a mark indicating an LFNST from a subset of the selected second transformations; and also selects an LFNST from the subset for transforming the predicted residual based on the mark specifying one of two LFNSTs. Claim 13 In claim 9, for transforming the predicted residuals, the at least one processor is configured to: apply LFNST(Ts) to a subset of coefficients of a first-order transformation (Tp) to obtain a transformation; and also to transform the predicted residuals using said transformation, wherein the first-order transformation (Tp) is a separable 2D transformation. Claim 14 In claim 9, for selecting the intra prediction mode for a block of the picture, the at least one processor: decodes a set of syntax elements from the data stream indicating whether the block is predicted using one of a set of intra prediction modes including a planar intra prediction mode, at least one angle prediction mode, and a DC intra prediction mode; and is also configured to select the matrix-based intra prediction mode from a set of matrix-based intra prediction modes in response to a determination that the set of syntax elements indicates that the block is not predicted using any mode of the set of intra prediction modes. Claim 15 In claim 9, for transforming the predicted residual, the at least one processor is configured to use a transformation defined by the connection of a first transformation (Tp) and a second transformation (Ts) selected from a subset of the second transformations, wherein the second transformation (Ts) corresponds to the LFNST. Claim 16 A method for encoding a picture into a data stream, the method comprising: generating a representation of a selected matrix-based intra-prediction mode for a block of the picture; deriving a prediction signal for the block using the selected matrix-based intra-prediction mode; selecting a subset of the second-order transformations from a set of second-order transformations, wherein the subset includes a low-frequency non-separable second-order transformation (LFNST) associated with both a planar intra-prediction mode and a DC intra-prediction mode; deriving a prediction residual for the block; transforming the prediction residual using the LFNST from the selected subset; and encoding the transformed prediction residual for the block into the data stream. Claim 17 A method according to claim 16, further comprising: deriving a prediction vector generated from a matrix vector product between a vector derived from a reference sample in the neighborhood of the block and a prediction matrix associated with the matrix-based intra prediction mode, in response to the fact that the matrix-based intra prediction mode is the selected intra prediction mode; and generating the prediction signal based on the prediction vector. Claim 18 In claim 16, the step of selecting a subset of the second transformation is: a method based on the selected matrix-based intra-prediction mode and the size of the block. Claim 19 A non-transient computer-readable storage medium, wherein the non-transient computer-readable storage medium stores instructions for performing the method of any one of claims 16 to 19 when executed by at least one processor of an electronic device. Claim 20 An apparatus for encoding a picture into a data stream, the apparatus comprising at least one processor, wherein the at least one processor comprises: generating a representation of a selected matrix-based intra-prediction mode for a block of the picture; deriving a prediction signal for the block using the selected matrix-based intra-prediction mode; selecting a subset of the second-order transformations from a set of second-order transformations, wherein the subset comprises a low-frequency non-separable second-order transformation (LFNST) associated with both a planar intra-prediction mode and a DC intra-prediction mode; deriving a prediction residual for the block; transforming the prediction residual using the LFNST from the selected subset; and also configured to encode the transformed prediction residual for the block into the data stream. Claim 21 In claim 20, in response to the fact that the matrix-based intra-prediction mode is the selected intra-prediction mode, the at least one processor is configured to: derive a prediction vector generated from the matrix vector product between a vector derived from a reference sample in the neighborhood of the block and a prediction matrix associated with the matrix-based intra-prediction mode; and also to generate the prediction signal based on the prediction vector. Claim 22 In claim 20, the at least one processor is configured to select a subset of the quadratic transformations based on the selected matrix-based intra-prediction mode and the size of the block.

Images