Block based prediction

Abstract

To provide a device and a method of executing an improved block based prediction mode for picture encoding by block unit.SOLUTION: A method of predicting a block 18 of a picture comprises: acquiring a set 102 of one or more sample values from a plurality of adjacent samples 17a and 17c arranged along a boundary of the block 18; determining an input value from the one or more sample values on the basis of an instruction of a size of the block; determining an intra prediction sample of a matrix base by applying a predetermined prediction matrix to the determined input value; and predicting samples 104, 118', and 118" of the block 18 on the basis of the determined intra prediction sample of the matrix base.SELECTED DRAWING: Figure 7.2

Description

[Technical field]

[0001] The present application relates to the field of block-based prediction. The present invention relates to an advantageous method for [Background technology]

[0002] Today, there are different block-based intra- and inter-prediction modes. The samples adjacent to the block to be predicted or samples obtained from other pictures are , a sample vector on which matrix multiplication can be performed to determine a prediction signal for the block to be predicted. A curve can be formed.

[0003] Matrix multiplication should preferably be performed with integer arithmetic and some kind of machine learning based The matrix derived by the training algorithm should be used for the matrix multiplication. Summary of the Invention [Problem to be solved by the invention]

[0004] However, such training algorithms usually use linear regression with floating-point precision. columns. Therefore, matrix multiplication is well approximated using these integer arithmetic operations. specifying integer arithmetic to be used in a multi-threaded manner and / or achieving improved computational efficiency; Challenges are being faced in making predictions more effective in terms of implementation and / or [Means for solving the problem]

[0005] This is achieved by the subject matter of the independent claims of the present application. Further embodiments of the invention are defined by the subject matter of the dependent claims of this application.

[0006] According to a first aspect of the present invention, the inventors of the present application have One problem encountered when trying to determine the predicted vector for a given block is In the present application, the inventors have recognized that it is possible to calculate the torque without using integer arithmetic. According to this aspect, the difficulty is solved by the matrix vector for calculating the prediction vector. From the sample vector, the sample vector is By deriving a further vector that is mapped by a reversible linear transformation of Alternatively, to calculate the predicted vector, a further vector and a predefined prediction matrix A matrix vector product is calculated between and . Further vectors can be calculated using, for example, integer arithmetic and / Or that a given block of samples is predicted by a device using fixed-point arithmetic. This is because the components of the sample vectors are correlated, For example, integer matrices and / or matrices with fixed point values ​​and / or predictors A matrix with a small quantization error can be used as the given prediction matrix. To obtain a further vector having the trie, a certain advantageous reversible linear transformation is used. It is based on the idea that it is possible.

[0007] Therefore, according to a first aspect of the present application, a picture is reconstructed using multiple reference samples. The apparatus for predicting a given block derives a sample value vector from a plurality of reference samples. The reference sample is, for example, a predetermined block in intra prediction. The sample is either a neighboring sample in the same block, or a sample in another picture in inter prediction. According to an embodiment, for example, a sample value vector having a reduced number of values ​​is obtained. By averaging, the reference samples can be reduced. The apparatus converts the sample-value vector into a vector of a matrix by a predetermined reversible linear transformation. To obtain the predicted vector, a further vector is calculated. A matrix-vector product is calculated between the vector and a predetermined prediction matrix, and a predetermined vector is calculated based on the prediction vector. based on a further vector, The prediction of a block of samples is done by multiplying the samples to get the predicted samples of a given block. It is possible to represent an integer approximation of the direct matrix-vector product between the vector of sample values ​​and the matrix.

[0008] A direct matrix-vector product between a sample vector and a matrix is ​​a further vector and a second The second matrix vector product between the matrix and / or The matrix is, for example, a machine learning prediction matrix. According to an embodiment, the second matrix is ​​a predetermined The second matrix may be based on a prediction matrix and an integer matrix. The second matrix may be, for example, a predetermined prediction matrix and an integer matrix. In other words, the second vector between the further vector and the second matrix The matrix vector product is the matrix vector product between the further vector and the predetermined predictor matrix, and It can be represented by a further matrix-vector product between an integer matrix and a further vector. An integer matrix is, for example, a matrix in which a certain column i0 is made up of 1s and column i≠i0 is made up of 0s. Therefore, a good integer approximation of the first and / or second matrix-vector product and / or A good fixed-point value approximation can be achieved by the device. This is due to the fact that The torque mainly contains small values ​​and the possibility of approximating the first and / or second matrix-vector products is A given prediction matrix can be quantized to yield a small effect of quantization error. The idea is that the matrix is ​​a quantized matrix, or has already been quantized.

[0009] According to an embodiment, the reversible linear transformation of multiplying a predetermined prediction vector by the sum of an integer matrix is The integer matrix may correspond to a quantized version of the machine learning prediction matrix. A matrix in which the constant column i0 is 1 and column i ≠ i0 is 0.

[0010] According to an embodiment, the reversible linear transformation is such that a given component of the further vector becomes a and a given Each of the other components of the further vector, except for the component of is defined to be equal to the component minus a, where a is a given value. Thus, Further vectors of small values ​​can be implemented, allowing quantization of a given prediction matrix. This results in a small effect of quantization error in the predicted samples of a given block. This further vector allows the desired It is possible to predict the samples of a block of .

[0011] According to an embodiment, the predetermined value is an arithmetic mean or a weighted mean of the components of the sample value vector. Which average, default value, picture is signaled in the data stream being encoded? The value to be sampled and one of the components of the sample value vector that corresponds to the given component. The sample vector may be, for example, a vector of values ​​determined by or from a number of reference samples. The group of reference samples is composed of the average of the , for example, includes at least two reference samples, preferably adjacent reference samples.

[0012] The predetermined value may be, for example, a part of the components of the sample value vector (e.g., at least two components element) or the arithmetic mean or weighted mean of all the components of a sample vector. The components of the sample vector are correlated, i.e., the values ​​of the components may be similar. and / or at least some of the components may have equal values, whereby , the component of the further vector is not equal to the given component of the further vector, i.e., i Component i with ≠ i0 (where i0 represents a given component) is probably the corresponding The idea is that the sigma has a smaller absolute value than the component. Vectors can be realized.

[0013] The predetermined value may be a default value, for example, Selected from a list or the same for all block sizes, prediction modes, etc. The components of the default list are different block sizes, prediction modes, and sample values. It can be related to the vector size, the average of the values ​​of the sample vector, etc. Thus, for example, depending on a given block, i.e., associated with a given block, Depending on the decoding or encoding settings, the optimized default value is set by the device. is selected from a list.

[0014] Alternatively, the predetermined value is signaled in the data stream in which the picture is encoded. In this case, for example, the encoding device determines the predetermined value. The decision can be based on the same considerations as discussed above in the context of default values.

[0015] The components of the further vector are not equal to the predetermined components of the further vector, i.e. Component i, where i ≠ i0, represents a given component, may be, for example, a default value or a data set. The corresponding sample vector is calculated using the value signaled in the stream as the given value. has a smaller absolute value than the corresponding component.

[0016] According to an embodiment, the predetermined value is a component of the sample value vector corresponding to the predetermined component. In other words, the value of the component of the sample vector corresponding to a given component can be reversibly It does not change when a linear transformation is applied. The value of an element of the pull value vector is, for example, equal to the value of a predetermined element of the further vector.

[0017] The predetermined components may be, for example, default values ​​as described above with respect to the predetermined values. It is clear that a given component may be selected by an alternative procedure. The predetermined component may be selected, for example, as a predetermined value. According to an embodiment, the predetermined component may be min is the value of the corresponding component of the sample vector that is equal to the average of the values ​​of the sample vector. or selected to have only marginal deviations from the mean of the values ​​of the sample value vector. .

[0018] According to an embodiment, in a column of the predetermined prediction matrix corresponding to a predetermined component of the further vector The matrix elements of a given prediction matrix are all zeros. The system leaves behind a column, i.e., a column of zeros. The reduced prediction matrix resulting from the given prediction matrix and the given components are left Therefore, we calculate the matrix-vector product between the further vector and the further vector that results from the further vector. By performing the multiplication by computing The method is configured to calculate a matrix-vector product between the vector and a predetermined prediction matrix. , a predetermined component of the further vector is set to a predetermined value, and the values ​​of the sample value vector are correlated. If the predetermined value is equal to or close to the sample value in the predicted signal for the given block, then the predetermined value is said to be exact or close to the sample value in the predicted signal for the given block. Therefore, the prediction of the samples of a given block is optionally further A given prediction matrix multiplied by a vector, or rather by a further vector, and an integer matrix, whose column i0 corresponds to a given component, The other columns i ≠ i0 are all 0, multiplied by a further vector. In other words, the machine learning predictions transformed, for example, by the inverse transformation of a given reversible linear transformation. The matrix is ​​a predefined predictor matrix, or rather a reduced predictor matrix, based on a further vector and an integer Therefore, only the prediction matrix is ​​quantized, and the machine learning prediction We should obtain integer approximations of the measured matrix and / or the transformed machine learning prediction matrix. This means that if no other vectors contain the specified component, all other components are the machine learning prediction matrix and / or the quantization in the resulting quantization of the transformed machine learning prediction matrix is much smaller than the corresponding component of the sample vector, allowing for a small effect of errors It is advantageous to have absolute values. Furthermore, the reduced prediction matrix and the further vectors By using this, fewer multiplications need to be performed to obtain the prediction vector, reducing the complexity. Optionally, when predicting samples of a given block, Then, a vector whose components are all the predetermined value a can be added to the predicted vector. A vector is multiplied by a matrix-vector product between an integer matrix and a further matrix, as described above. can be obtained by

[0019] According to an embodiment, in a column of the predetermined prediction matrix corresponding to a predetermined component of the further vector The matrix resulting from summing each matrix element of a given predictor matrix by 1 times a given reversible linear transformation corresponds to a quantized version of the machine learning prediction matrix. Summing each matrix element of a given predictor matrix in the corresponding column of the given predictor matrix with 1, e.g. For example, the transformed machine learning prediction matrix is ​​represented as follows: Represents the machine learning prediction matrix transformed by the inverse of a given reversible linear transformation. The sum is an integer matrix and a given predictor matrix whose columns correspond to given components. Column i0 of consists of ones, and all other columns i≠i0 are zeros.

[0020] According to an embodiment, the device uses the prediction parameters to represent a predetermined prediction matrix, The components of the vectors to be calculated, as well as the prediction parameters and intermediate results resulting therefrom, are multiplied and The prediction parameter is configured to calculate a matrix vector product by performing The absolute value of the meter can be expressed by an n-bit fixed-point number, where n is 14 or less. , or 10 or less, or 8 or less. In other words, the prediction parameters are Multiply the elements of a matrix-vector product, such as a vector, a given predictor matrix and / or a predictor vector. The multiplication and addition operations are performed to obtain a predetermined prediction matrix, a prediction The fixed-point format of the vector and / or predicted samples for a given block is taken. can be obtained.

[0021] An embodiment of the present invention may further comprise the steps of any of the embodiments described herein to obtain the prediction signal. To predict a given block of a picture using any of a number of reference samples, The present invention also relates to an apparatus for encoding a picture, the apparatus comprising: An entry configured to encode a prediction residual of a given block to correct the measured signal. For the prediction of a given block to obtain a prediction signal, a tropy coder is provided. For example, the positioning may include forming a sample value vector from a plurality of reference samples, From the sample vector, a further vector is generated to which the sample vector is mapped by a given reversible linear transformation. To derive the vector and obtain the predicted vector, a further vector and a given prediction matrix are , and predicts the samples of a given block based on the prediction vector. The system is configured to measure

[0022] An embodiment of the present invention may further comprise the steps of any of the embodiments described herein to obtain the prediction signal. To predict a given block of a picture using any of a number of reference samples, The present invention also relates to an apparatus for decoding a picture, the apparatus comprising: an entropy decoder configured to decode a prediction residual of the block of the and a prediction corrector configured to correct the prediction signal using the prediction signal. For prediction of a given block for a given time, the device may, for example, select samples from a number of reference samples. and forming a vector of sample values ​​from the vector of sample values, the vector of sample values ​​being determined by a predetermined reversible linear To derive a further vector that is mapped by the transformation to obtain a predicted vector. , a matrix-vector product is calculated between the further vector and the predetermined prediction matrix, and the predicted vector The method is configured to predict samples of a given block based on

[0023] An embodiment of the present invention uses multiple reference samples to estimate a given block of a picture. forming a sample value vector from a plurality of reference samples, From the sample-value vector, the sample-value vector is transformed by a predetermined reversible linear transformation. To obtain a predicted vector, a further vector is derived, which is then mapped to the vector. calculating a matrix vector product between a vector consisting of the above-mentioned elements and a predetermined prediction matrix; and predicting samples of the given block based on the prediction vector.

[0024] An embodiment of the present invention is a method for coding a picture, comprising the steps of: obtaining a prediction signal; To do this, a given block of a picture is estimated using a number of reference samples according to the method described above. The prediction signal is then predicted by inputting the prediction residual of a given block to correct the prediction signal. and (b) encoding the encoded data in a 3D form.

[0025] An embodiment of the present invention is a method for decoding a picture, comprising: To this end, a given pixel of a picture is obtained using a number of reference samples according to one of the methods described above. predicting a block and entropy decoding a prediction residual for the given block; and correcting the prediction signal using the prediction residual. An embodiment of the present invention uses the method described herein for encoding a picture. The present invention relates to a data stream having pictures coded by a multi-bit coding scheme.

[0026] When implemented on a computer, the embodiments of the present invention may be implemented as described herein. A computer program having a program code for carrying out any of the methods of the present invention. Regarding.

[0027] The drawings are not necessarily to scale, emphasis instead being placed generally upon illustrating the principles of the invention. In the following description, various embodiments of the present invention will be described with reference to the following drawings, in which: Explained. [Brief description of the drawings]

[0028] [Figure 1] 1 illustrates an embodiment of encoding into a data stream. [Diagram 2] 2 shows an embodiment of an encoder. [Diagram 3] 1 illustrates an embodiment of picture reconstruction. [Figure 4] 2 shows an embodiment of a decoder. [Diagram 5] 1 shows a schematic diagram of prediction of a block for encoding and / or decoding according to an embodiment; [Figure 6] 1 illustrates a matrix operation for prediction of a block for encoding and / or decoding according to an embodiment. [Figure 7.1] 1 illustrates a prediction of a block with a reduced sample value vector according to an embodiment. [Figure 7.2] 1 illustrates block prediction using sample interpolation according to an embodiment. [Figure 7.3] 13 illustrates prediction of a block with a reduced sample value vector in which only some boundary samples are averaged, according to an embodiment. [Figure 7.4] 13 illustrates prediction of a block with a reduced sample value vector in which groups of four boundary samples are averaged, according to an embodiment. [Figure 8]1 shows a schematic diagram of an apparatus for predicting a block according to an embodiment; [Figure 9] 4 illustrates a matrix operation performed by an apparatus according to an embodiment. [Figure 10a] 4 illustrates detailed matrix operations performed by an apparatus according to an embodiment. [Figure 10b-10c] 4 illustrates detailed matrix operations performed by an apparatus according to an embodiment. [Figure 11] 13 illustrates detailed matrix operations performed by an apparatus using offset and scaling parameters according to an embodiment. [Figure 12] 4 illustrates detailed matrix operations performed by an apparatus using offset and scaling parameters according to different embodiments; [Figure 13] 1 shows a block diagram of a method for predicting a given block according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Equal or equivalent elements or elements having equal or equivalent functions may be shown in different figures. Even if they occur, they are indicated in the following description by the same or equivalent reference numerals. .

[0030] In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present invention. However, embodiments of the invention may be practiced without these specific details. In other examples, it will be apparent to one skilled in the art that embodiments of the present invention may be obfuscated. In order to avoid obscuring the illustrations, well-known structures and devices are shown in block diagram form rather than in detail. In addition, the features of the different embodiments described below are the same unless otherwise stated. , can be combined with each other. 1 Introduction

[0031] In the following, different examples, embodiments and aspects of the invention are described. These examples, embodiments and at least some of the aspects relate, inter alia, to video coding, and and / or block transformations using, for example, linear or affine transformations with adjacent sample reduction. and / or for performing query-based predictions, e.g., in video applications and Video distribution (e.g., broadcasting, streaming) for and / or virtual reality applications The present invention refers to a method and / or apparatus for optimizing video playback (video, file playback, etc.).

[0032] Further, examples, embodiments, and aspects may be implemented using High Efficiency Video Coding (HEVC) or Further embodiments, examples and aspects are set forth in the appended claims. It is defined by a range.

[0033] Any embodiments, examples, and aspects defined by the claims are described in the following sections: It should be noted that the present invention may be supplemented by any of the details (features and functions) described in the specification. I want to.

[0034] Also, the embodiments, examples, and aspects described in the following sections may be used individually. may be supplemented by any of the features of another chapter or by any feature included in the claims. It can also be done.

[0035] Additionally, each example, embodiment, and aspect described herein may be used individually or in any combination. It should be noted that the above examples, embodiments and and each of said individual aspects without adding details to another one of the aspects. It can be done. The present disclosure explicitly or implicitly describes features of the decoding and / or encoding systems and / or methods. Note also that the explanation is implicit.

[0036] Additionally, the features and functions disclosed herein in relation to the method may be used in the apparatus. Furthermore, any of the features and functions disclosed herein with respect to the device may be used. The functions can also be used in the corresponding methods. The method described may be supplemented by any of the features and functions described with respect to the apparatus. This can be done.

[0037] Additionally, any of the features and functionality described herein may be implemented in any manner that is not specifically described in the "Implementation Alternatives" section. As described in the section, The present invention can be implemented using a combination of software.

[0038] Additionally, any of the characteristics listed in brackets ("(...)" or "[...]") may be considered optional in some examples, embodiments, or aspects. can.

[0039] 2 Encoder and Decoder In the following, we will discuss how to achieve more effective compression when using block-based prediction. We will now discuss various examples that can be useful. Some examples are: The latter achieves high compression efficiency by using a heuristic These modes may be added to or offered exclusively by other intra prediction modes designed for the And even other examples can take advantage of both of the disciplines described here. While these embodiments are not identical, intra prediction uses a reference frame instead of a frame in another picture. The prediction can be converted to inter prediction by using the reference samples.

[0040] In order to facilitate understanding of the following examples of this application, the description will be based on the examples outlined thereafter in this application. We start by presenting possible encoders and decoders that can be constructed to accommodate FIG. 1 shows an apparatus for block-by-block encoding of a picture 10 into a data stream 12. The apparatus is indicated using the reference numeral 14 and is a still picture encoder or In other words, picture 10 includes picture 10 If the encoder 14 is configured to encode the video 16 into the data stream 12, or the encoder 14 encodes the picture 10 exclusively into the data stream 12. If so, it may be the current picture from the video 16.

[0041] As previously mentioned, the encoder 14 performs the encoding in a block-by-block manner or on a block basis. To this end, the encoder 14 subdivides the picture 10 into blocks and encodes them. A unit of the decoder 14 encodes the picture 10 into a data stream 12. Examples of possible subdivisions of the blocks 18 are given in more detail below. The division is into blocks of fixed size, such as an array of blocks arranged in rows and columns. or from the entire picture area of ​​Picture 10 or from a pre-partition of Picture 10 of hierarchical multitree subdivision that initiates a multitree subdivision into an array of treeblocks. Due to different block sizes used, you may end up with block 18. is treated as excluding other possible ways of subdividing the picture 10 into blocks 18. This should not be allowed.

[0042] Additionally, the encoder 14 predictively encodes the picture 10 into the data stream 12. For a particular block 18, this is the encoder 14 determines the prediction signal of block 18 and the prediction residual, i.e., the prediction signal of block 18 The prediction error, which deviates from the actual picture content in the image, is encoded into the data stream 12. This means that

[0043] The encoder 14 may use different prediction modes to derive a prediction signal for a particular block 18. The prediction modes that are important in the following example are: is an int that is spatially predicted from samples of adjacent already coded pictures 10. The coding of the picture 10 into the data stream 12 and thus the corresponding The decoding procedure can be based on a specific coding order 20 defined between blocks 18. For example, coding order 20 can be written from top to bottom, traversing each row from left to right. The blocks 18 can be traversed in a raster scan order, such as by unit. In the case of tree-based subdivision, the order of raster traversal may be applied within each hierarchical level. A depth-first traversal order can be applied, i.e., Leaf nodes in a block of Bell have the same parent block according to coding order 20. Depending on the coding order 20, a block may precede a block of the same hierarchical level. The adjacent already coded samples of block 18 are usually located on one or more sides of block 18. In the example presented here, for example, adjacent blocks of block 18 The already coded samples are located at the top and to the left of block 18 .

[0044] Encoder 14 may not only support intra-prediction modes. For example, if encoder 14 is a video encoder, encoder 14 also includes block 18. Inter prediction mode in which the image is temporally predicted from a previously encoded video 16 picture. Such an inter prediction mode can support the prediction signal of block 18. Such a block indicates the relative spatial offset of the part from which the code is derived as a copy. 18, the motion compensation prediction mode in which the motion vector is signaled. Additionally or alternatively, inter-prediction when the encoder 14 is a multi-view encoder prediction mode, or non-prediction mode in which the inside of block 18 is coded as is, i.e. without prediction. Other non-intra prediction modes, such as prediction modes, may also be available.

[0045] Before beginning the description of this application with a focus on intra-prediction modes, A more specific example of a block-based encoder, i.e., The possible implementations of the encoder 14 are shown in Figs. 1 and 2, which present two corresponding examples of decoders conforming to the present invention. This section describes possible implementations.

[0046] FIG. 2 shows a possible implementation of the encoder 14 of FIG. 1, i.e., the encoder encodes the prediction residual. The figure shows one that is configured to use transform coding to obtain the Therefore, the present application is not limited to such predictive residual coding. The encoder 14 receives an inbound signal, i.e. a picture 10, or a current The corresponding prediction signal 24 is subtracted from the block 18 for later use by a predictive residual coder 28. A subtraction circuit configured to obtain a prediction residual signal 26 that is encoded into the data stream 12. The predictive residual encoder 28 comprises a lossy encoding stage 28a and a lossless encoder 28b. The lossy stage 28a receives the prediction residual signal 26 and , a quantizer 30 for quantizing the samples of the prediction residual signal 26. ,This example uses transform coding of the prediction residual signal 26 and therefore a lossy coding stage 28a is the result of the quantization performed by the quantizer 30 on the transformed coefficients representing the residual signal 26. The subtractor 22 and the quantizer 23 are arranged to convert such a spectrally decomposed prediction residual 26 into a The transform stage 32 is connected between the transformer 30 and the DCT stage 32. The transforms include DCT, DST, FF T, the Hadamard transform, etc. Then, the transformed and quantized prediction residual signal No. 34 is an entropy coding signal for entropy coding the quantized prediction residual signal 34 into the data stream 12. The lossless encoding is performed by a lossless encoding stage 28b, which is an entropy coder. 14 is a prediction signal that is obtained from the transformed and quantized prediction residual signal 34 in a manner that is also usable by the decoder. A prediction residual signal reconstruction switch is connected to the output of the quantizer 30 so as to reconstruct the prediction residual signal. It further comprises a quantizer 30 which takes into account the coding loss. To this end, the prediction residual reconstruction stage 36 performs the inverse of the quantization of the quantizer 30. followed by a spectrum quantizer 38 which performs the inverse of any of the specific transformation examples mentioned above. An inverse transformer that performs an inverse transform to the transform performed by the transformer 32, such as the inverse of the decomposition The encoder 14 outputs the reconstructed signal, i.e. the reconstructed samples, The reconstructed prediction residual signal and the prediction signal output by the inverse transformer 40 are 24. The output of this adder is fed to a predictor 44 in the encoder 14. The predictor 44 determines the predicted signal 24 based thereon. It is the predictor 44 that supports all prediction modes. In the case of a video encoder, the encoder 14 also outputs the inter-prediction blocks after filtering. The fully reconstructed picture that forms the reference picture for the predictor 44 for the block is filtered. 4 shows that an in-loop filter 46 for detecting the phase difference can be provided.

[0047] As already mentioned above, the encoder 14 operates on a block basis. The block base of the object is a subdivision of the picture 10 into blocks, For each block, the set or An intra prediction mode is selected from a plurality of intra prediction modes, and the selected intra prediction mode is However, the other types of subdivision of the picture 10 may be implemented separately. For example, if picture 10 is inter-coded, The above determination of whether a block is intra- or intra-coded may be made at a granularity or block level. This can be done in blocks that deviate from 18. For example, intermode / intramode The lamode decision is made by subdividing the picture 10 and subdividing each coding block into prediction blocks. Intra prediction can be used at the level of the coding block being processed. The prediction block having the coding block determined to be the intra prediction mode Therefore, for each of these prediction blocks, the supported It is determined which of the available intra prediction modes should be used for each prediction block. These predicted blocks form the block 18 of interest here. Predicted blocks within a related coding block may be treated differently by predictor 44. They determine the motion vector and the reference picture pointed to by the motion vector. The prediction signal of this block is obtained from the reference picture by copying it from its location in the Another block subdivision is the transformation by the transformer 32 and the inverse transformer 40. It concerns the subdivision into transformation blocks in the units where the transformation is performed. The transformed blocks are , for example, may be the result of further subdivision of the coding block. Therefore, the examples provided should not be treated as limiting and other examples may exist. Just for the sake of completeness, the subdivision into coding blocks may be implemented as, for example, a multi-tree subdivision. Similarly, the prediction block and / or the transform block may be multi-partitioned. The subdivision is obtained by further subdividing the coding block using a subtree subdivision. Note that it is possible to

[0048] A decoder 54 or device for block-wise decoding compatible with the encoder 14 of FIG. 1 is shown in FIG. This decoder 54 does the opposite of the encoder 14. That is, The picture 10 is decoded block-wise from the data stream 12, and for this purpose a number of The decoder 54 may, for example, include a residual provider 156. All other possibilities discussed above with respect to FIG. 1 are also valid for decoder 54. Thus, the decoder 54 can be a still picture decoder or a video decoder. , all prediction modes and predictability are supported by the decoder 54. The differences between the encoder 14 and the decoder 54 are primarily due to the fact that the encoder 14 has different coding speeds and / or or to minimize some cost function that may depend on the coding distortion, etc. The problem lies in the fact that the coding decisions are selected or chosen according to some optimization. One of the encoding options or encoding parameters is not available or supported. The intra prediction mode to be used for the current block 18 from among the available intra prediction modes. The selected intra prediction mode may then be used to select a data stream. The decoder 54 receives a signal from the encoder 14 of the current block 18 in the memory 12. , this signaling in data stream 12 in block 18 is used to redo the selection. Similarly, the subdivision of picture 10 into blocks 18 is subject to optimization within encoder 14. and corresponding subdivision information can be conveyed in the data stream 12. , the decoder 54 restores the subdivision of the picture 10 into blocks 18 based on the subdivision information. To summarise the above, the decoder 54 may be a predictive decoder operating on a block basis. In addition to the intra prediction mode, the decoder 54 may also include a video decoder, e.g. If so, other prediction modes, such as inter-prediction modes, can be supported. In this case, the decoder 54 may also use the coding order 20 described with respect to FIG. Since the coding order 20 is followed by both the encoder 14 and the decoder 54, Neighboring samples are available for the current block 18 at both the encoder 14 and the decoder 54. Therefore, in order to avoid unnecessary repetition, the operating mode of the encoder 14 is The description is given in terms of the picture, for example as far as prediction is concerned and as far as the coding of the prediction residual is concerned. As far as the subdivision of the channel 10 into blocks is concerned, this must also be applied to the decoder 54. The difference is that the encoder 14 can use several encoding options or codes depending on the optimization. The initialization parameters and signals are selected within the data stream 12 or The fact that the data is inserted into the 12th row is a problem that requires some effort, such as repartitioning, to redo the prediction. The signal is derived from the data stream 12 by a decoder 54 .

[0049] FIG. 4 shows a possible implementation of the decoder 54 of FIG. 3, i.e., the decoder 54 of FIG. 1 as shown in FIG. 4. Many of the elements of encoder 54 in FIG. To denote these elements, we use the a The same reference numbers with posttrophies are used in FIG. 4. In particular, adder 42', optional The in-loop filter 46' and predictor 44' are the same as they are in the encoder of FIG. The reconstructed sum is applied to the adder 42'. That is, the inverse quantized and retransformed prediction residual signal is fed to the entropy encoder 28b. The sequence of the entropy decoder 56 that reverses the entropy coding, followed by the coding side The residual signal reconstruction stage is constructed by an inverse quantizer 38' and an inverse transformer 40' in the same manner as in The output of the decoder is a reconstruction of picture 10. The reconstruction of the filter 10 can be performed directly at the output of the adder 42' or by using an in-loop filter 4 6' output. To improve the picture quality, To subject the reconstruction of Cha10 to some post filtering, The filter can be placed at the output of the decoder, but this option is not shown in Figure 4. Not yet.

[0050] Again, with respect to FIG. 4, the explanation given above with respect to FIG. 4, except that it only performs the relevant decisions regarding the However, all the details regarding block subdivision, prediction, inverse quantization, and retransformation are The above explanations are also valid for the decoder 54 of FIG.

[0051] 3. ALWIP (Affine Linear Weighted Intra Predictor) Some non-limiting examples of ALWIP include those embodying the techniques described herein. Even though ALWIP is not necessarily required to ensure that the

[0052] This application is applicable, inter alia, to video codecs such as HEVC or a successor to HEVC. Improved block-based prediction for block-based picture coding, such that The prediction mode may be an intra prediction mode, but theoretically In particular, the concept described herein applies to interleaved images in which the reference samples are part of another picture. It can also be converted into a prediction mode. Block-based prediction schemes enable efficient implementations, including hardware-friendly implementations. We need to be more cautious. This object is achieved by the subject matter of the independent claims of the present application.

[0053] Intra prediction modes are widely used in picture and video coding. In coding, intra prediction modes are used in combination with other inter prediction modes such as motion compensation prediction modes. In intra prediction modes, the current block is predicted based on the neighboring samples. that is, as far as the encoder is concerned, it has already been encoded and as far as the decoder is concerned, it has already been decoded. The neighboring sample values ​​are extrapolated to the current block. The prediction signal of the current block is formed, and the prediction residual is stored in the data stream of the current block. The better the prediction signal, the smaller the prediction residual will be, and therefore Fewer bits are needed to code the prediction residual.

[0054] To be effective, a block-based picture coding environment requires a To create an effective framework, several aspects need to be considered. For example, the more intra-prediction modes a codec supports, the more options the decoder has to choose from. On the other hand, the consumption of the side information rate for signaling the supported The set of prediction modes provides a good prediction signal, i.e., a prediction signal with a low prediction residual. It is necessary to be able to provide

[0055] In the following, as a comparative embodiment or a basic example, a picture is extracted from a data stream as a block. A device (encoder or decoder) for decoding a picture in a block-by-block manner, the device comprising: The intra prediction signal for the block of the current block is the first of the samples adjacent to the current block. The template is then processed using an affine linear weighted predictor (ALWIP). at least one intra-prediction mode, determined by applying a linear predictor to the An apparatus for supporting the above is disclosed.

[0056] The apparatus has the following characteristics, which, when executed by, for example, a processor: A non-transitory memory device that stores instructions that cause the processor to perform a method and / or operate as a device. may be implemented in a storage unit, method, or another technique) You can also have one.

[0057] 3.1 A predictor can be complementary to other predictors. The intra prediction modes, which may form the subject of implementation improvements as further described below, are , can be complementary to other intra-prediction modes of the codec. They are based on the DC prediction model defined in the HEVC codec and the JEM reference software. It can complement the planar prediction mode, the angular prediction mode, or the 3D prediction mode. The types of intra prediction modes are referred to as conventional intra prediction modes. , for a given block of intra modes, intra prediction supported by the device A flag must be parsed by the decoder to indicate whether one of the modes should be used. There is a need.

[0058] 3.2 Two or more proposed prediction modes A device may contain more than one ALWIP mode. If the decoder knows that one of the supported ALWIP modes should be used, If so, the decoder determines which of the ALWIP modes supported by the device is being used. Additional information needs to be analyzed to indicate whether the

[0059] Signaling of supported modes is done by coding some ALWIP modes differently than other ALWIP modes. This has the advantage that it may require fewer bins than the WIP mode. Which of the modes require fewer bins and which require more bins? The bitstream to be decoded can be extracted from the already decoded bitstream or can be It may rely on information that may have been previously fixed.

[0060] 4. Some Aspects FIG. 2 shows a decoder 54 for decoding pictures from the data stream 12. The decoder 54 may be configured to decode a given block 18 of a picture. In particular, the predictor 44 uses a linear or affine linear transformation [e.g., ALWIP]. A set of P adjacent samples adjacent to a given block 18 is then calculated as the sample of the given block. The method may be configured to map the Q predictions to a set of Q predictions of the Q filter.

[0061] As shown in FIG. 5, a given block 18 is a set of predicted Q values ​​(which are If block 18 has M rows and N columns, then Q = M N. The Q value of block 18 can be calculated in the spatial domain (e.g., pixel) or in the transform domain (e.g., DC T, discrete wavelet transform, etc.). The Q value of block 18 is generally Prediction based on P values ​​obtained from adjacent blocks 17a to 17c adjacent to block 18 The P values ​​of the adjacent blocks 17a to 17c are closest to the block 18. The P values ​​of the adjacent blocks 17a to 17c may be in different positions (for example, adjacent). The P values ​​are calculated by dividing the 17'b In order to distinguish it from blocks that are part of the is shown as:

[0062] As shown in FIG. 6, to perform prediction, a first vector 1 7P (each entry is associated with a particular position in the adjacent portion 17'a-17'c) , a second vector 18Q having Q entries (each entry is a specific 17M (each row is associated with a position in block 18) Each row is associated with a specific location within the adjacent portion 17'a-17'c. Therefore, the mapping matrix 17M can be calculated by the following formula: According to the mode, the P values ​​of the adjacent parts 17'a to 17'c are predicted to the value of the block 18. Thus, the entries in the mapping matrix 17M are understood as weighting coefficients. In the following description, the reference numbers 17a to 17c are used instead of 17'a to 17'c. to reference the adjacent part of the boundary.

[0063] In the art, there are several modes such as DC mode, planar mode, and 65-way prediction mode. There are several known conventional modes. For example, 67 modes are known.

[0064] However, we use a different mode called linear or affine linear transformation. Note that it is also possible to use a linear or affine linear transformation. The weighting coefficients are at least 1 / 4P Q weight coefficients that are non-zero. , for each of the Q predictors, a set of P weighting factors for each predictor is The sequences are arranged one above the other in raster scan order among the samples of a given block. When combined, they form an envelope that is omnidirectionally nonlinear.

[0065] P positions of adjacent values ​​17'a to 17'c (template), adjacent sample 17 Map the Q positions of 'a to 17'c and the P*Q weight coefficient values ​​of matrix 17M The plane is an example of the envelope of the series for DC conversion ( (The envelope is obviously planar, and therefore linear or affine.) Another example is the following angular mode emitter: The envelope is excluded from the ALWIP definition and is simply For example, it looks like a hill going diagonally from top to bottom along a direction in the P / Q plane. Planar mode The 65 and 66 directional prediction modes have different envelopes, but this is because the 65 directional prediction modes are different in at least one direction, i.e. That is, in all directions of the DC, for example, and in the hill direction of the angle mode, for example, It is a shape.

[0066] Conversely, the envelope of a linear or affine transformation is not linear in all directions. Such a kind This transformation may be optimal for performing predictions for block 18 in some circumstances. It is understood that at least one-quarter of the weighting coefficients are different from 0 (i.e., P* Note that it is preferred that at least 25% of the Q weighting coefficients be different from 0.

[0067] The weighting factors may be independent of each other according to any regular mapping rule. Thus, matrix 17M is a set of entries whose values ​​have no obvious discernible relationship. For example, the weighting coefficients may be calculated by any analytical or differential function. It cannot be described even if it is written in a non-standard way.

[0068] In the example, the ALWIP transformation uses the weighting coefficients of the first series associated with each predicted value and The maximum cross-correlation between the weighting coefficients of the second series associated with each of the predicted values ​​other than the predicted value Although the average of the two series, or a reversed version of the latter series, would lead to a higher maximum value. Regardless, a given threshold (e.g., 0.2 or 0.3 or 0.35 or 0.1, e.g. For example, the threshold may be lower than For example, for each combination (i1,i2) of rows in the ALWIP matrix 17M, the P value in row i1 is multiplied by i2. The cross-correlation can be calculated by multiplying the P values ​​of the first and second rows. For correlation, the maximum value can be obtained. Thus, for the entire matrix 17M, The average (mean) can be obtained (i.e., the cross-correlation across all combinations). (The maximum of the correlations is averaged). The threshold is then set to, for example, 0.2 or 0.3 or 0. 35 or 0.1, for example the threshold value may be in the range between 0.05 and 0.035.

[0069] The P adjacent samples of the blocks 17a to 17c are arranged on the boundary of a given block 18 (e.g. , 18c, 18a). For each of the Q predictors in class 18, a set of P weights associated with each predictor is The weighting coefficients are calculated by traversing a one-dimensional path in a given direction (e.g., left to right, top to bottom, etc.). may be ordered as follows: In an example, the ALWIP matrix 17M may be off-diagonal or non-block diagonal. ALWIP for predicting 4x4 blocks 18 from 4 already predicted adjacent samples An example of a matrix 17M may be: { { 37, 59, 77, 28}, { 32, 92, 85, 25}, { 31,69,100,24}, { 33,36,106,29}, { 24,49,104,48}, { 24,21,94,59}, { 29,0,80,72}, { 35,2,66,84}, { 32,13,35,99}, { 39,11,34,103}, { 45,21,34,106}, { 51,24,40,105}, { 50,28,43,101}, { 56,32,49,101}, { 61,31,53,102}, { 61,32,54,100} }.

[0070] (where {37,59,77,28} is the first row, and {32,92,85,25 } is the second row. {61,32,54,100} is the 16th row of matrix 17M. ) Matrix 17M has dimensions 16×4, resulting in 64 weight coefficients (as a result of 16*4=64). This is because the matrix 17M has dimensions QxP, where Q=M*N. , the number of samples in the prediction target block 18 (block 18 is a 4 × 4 block), and P is the number of samples in the prediction target block 18 (block 18 is a 4 × 4 block), and where M=4, N=4, Q=16(M*N = 4*4 = 16, so P = 4. The matrix is ​​off-diagonal and non-block It is diagonal and is not described by any particular rule.

[0071] As can be seen, less than a quarter of the weight coefficients are zero (1 out of 64 in the above matrix). (One of the weighting factors is 0.) The envelope formed by these values ​​is Thus, when placed one level down, it creates an envelope that is nonlinear in every direction.

[0072] Where the above description has been primarily described with reference to a decoder (e.g., decoder 54), Alternatively, the same may be performed in an encoder (eg, encoder 14).

[0073] In some examples, for each block size (in the set of block sizes), Intra prediction modes within a second set of intra prediction modes for each block size The ALWIP transformations in the prediction modes are different from each other. Cardinality of a second set of intra prediction modes for block sizes in the set The dimensionality can be matched, but the intra prediction models for different block sizes are The associated linear transformation or affine transformation of the intra prediction modes in the second set of modes. Shape transformations can be made inconvertible to each other by scaling.

[0074] In some cases, ALWIP transformations have "nothing in common" with traditional transformations. (For example, an ALWIP transformation may be defined as Even if a transformation is mapped through a (It can also be stated as "not having any".)

[0075] In the example, the ALWIP mode is used for both the luma and chroma components, but not for the other In the example, the ALWIP mode is used for the luma component but not for the chroma components.

[0076] 5 Affine linear weighted intra prediction mode with faster coder (e.g., test CE3 -1.2.1) 5.1 Description of the method or apparatus Affine Linear Weighted Intra Prediction (ALWIP) model tested in CE3-1.2.1 The code was tested in JVET-L0199 under test CE3-2.2.2, with the following changes: This could be the same as proposed in:

[0077] Multiple Reference Line (MRL) intra prediction, especially aligned with the encoder estimation and signaling , i.e., MRL is not combined with ALWIP, and the transmission of MRL index is non Restricted to ALWIP blocks.

[0078] Subsampling is mandatory for all blocks W×H≧32×32 (previously was optional for 32x32). Therefore, additional tests in the coder And the transmission of the subsampling flag has been removed.

[0079] ALWIP for 64×N and N×64 blocks (N≦32) is 32×N and 32×64, respectively. and N×32 downsampling and applying the corresponding ALWIP mode. It has been added. In addition, test CE3-1.2.1 includes the following encoder optimizations for ALWIP: :

[0080] Combined mode estimation: Conventional and ALWIP modes are combined for a complete RD estimation. Use a shared Hadamard candidate list, i.e., the ALWIP mode candidates are Hadamard They will be added to the same list as conventional (and MRL) mode candidates based on cost. EMT Intra Fast and PB Intra Fast are supported for the combined mode list. and has additional optimizations to reduce the number of full RD checks. Only the MPMs of the left and top blocks available are used in the same way as in the conventional mode. Therefore, it is added to the list for ALWIP full RD estimation.

[0081] 5.2 Complexity Assessment Except for calculations that call for discrete cosine transforms, exam CE3-1.2.1 does not require the predicted signal to be A maximum of 12 multiplications were required per sample to generate them. A total of 136492 parameters of 0.273 MB were required. This corresponds to the memory of the

[0082] 5.3 Experimental results The test was evaluated using the VTM software version 3.0.1. ) and Random Access (RA) configurations under the common test conditions JVET-J1010 The corresponding simulations were performed using the Linux Intel Xeon cluster (E5 -2697A v4, AVX2 on, Turbo Boost off).

[0083] [Table 1]

[0084] [Table 2]

[0085] 5.4 Affine Linear Weighted Intra Prediction with Reduced Complexity (e.g., test CE3-1. 2.2) The technology tested in CE2 is based on the "Affine Line" method described in JVET-L0199[1]. It is related to "intra-prediction" but has the following disadvantages in terms of memory requirements and computational complexity: Simplifying this:

[0086] Prediction matrices covering all block shapes (e.g. S0, S1, S2, see below) The three different vectors are the vector (to be used for the purpose of As a result, the number of parameters is 14400. is reduced to a 10-bit value, which is

number

[0087] The input and output sizes of the predictor are further reduced. Furthermore, the boundaries are decomposed via DCT. Instead of converting, averaging or downsampling is performed on the boundary samples. and the generation of the prediction signal can use linear interpolation instead of the inverse DCT. As a result, up to four multiplications per sample may be required to generate the predicted signal. .

[0088] 6. Example Here, we perform some predictions (e.g., as shown in Figure 6) using ALWIP prediction. This section explains how to do this. In principle, referring to FIG. 6, to obtain Q=M*N values ​​for the M×N block 18 to be predicted, To do this, we use a Q×P ALWIP prediction matrix of 17M Q*P samples and a P×1 neighbor vector of 1 7P should be multiplied with P samples. Therefore, in general, To obtain each of the Q=M*N values ​​in the ×N block 18, you need at least P=M +N number of values ​​must be multiplied.

[0089] These multiplications have a highly undesirable effect. The dimension P of the boundary vector 17P is In general, the boundary samples (e.g., adjacent) to the M×N block 18 to be predicted are It depends on the number M+N of bins or pixels 17a, 17c of the block to be predicted. If the size of 8 is large, the number of boundary pixels (17a, 17c) M+N will be large accordingly. Therefore, the P×1 boundary vector 17P has dimensions P=M+N, and the Q×P ALWIP prediction row This means that the length of each row of the 17M column, and therefore the number of multiplications required, will be large (typically In other words, Q=M*N=W*H, where W (width) is another symbol for N and H (height). is another symbol for M, and P is the boundary vector that is in only one row and / or one column of samples. Therefore, when it is formed, P=M+N=H+W).

[0090] This problem is typically solved by using microprocessor-based systems (or other digital processing This is exacerbated by the fact that in multiplication (high-speed systems), multiplication is generally a power-intensive operation. A large number of multiplications performed on a large number of samples in a large number of blocks is generally undesirable. It can be assumed that this would cause a waste of computational power. Therefore, it is possible to reduce the number of multiplications Q*P required to predict the M×N block 18. preferable.

[0091] By intelligently choosing an alternative, easier to process operation to multiply, each block to be predicted is It is possible to somehow reduce the computational power required for each intra prediction of block 18. It is understood that. In particular, referring to Figures 7.1 to 7.4, the encoder or decoder:

[0092] A reduced sample value with fewer samples compared to multiple adjacent samples. To obtain a set, several adjacent samples (e.g., 17a, 17c) are scaled down. (e.g., in step 811), (e.g., averaging or downsampling) by Gu) and

[0093] The reduced set of sample values ​​is subjected to a linear or affine-linear transformation (e.g., obtaining a prediction value for a given sample of a given block;

[0094] By using multiple adjacent samples (e.g., 17a, 17c), It is understood that certain blocks (eg, 18) can be predicted.

[0095] In some cases, the decoder or encoder may also adjust the size of a given sample, for example by interpolation. and further samples of the given block based on the predictions of a number of adjacent samples. Therefore, an upsampling strategy can be obtained. This can be done.

[0096] In an example, several averages are performed on the samples of boundary 17 (e.g., in step 811) so as to reach a reduced set 102 of samples with a reduced number of samples (Figs. 7.1 - 7.4). (At least one of the samples of the reduced number of samples 102 may be the average of two samples of the original boundary samples, or a selection of the original boundary samples). For example, if the original boundary has P = M + N samples, the reduced set of samples can have P < P, where at least one of M < M and N < N. Thus, the boundary vector 17P actually used for prediction (e.g., in step 812b) does not have P × 1 entries, but has P < P, P × 1 entries. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have Q × P dimensions, but has at least P < P (where at least one of M < M and N < N), so that the number of elements of the matrix is reduced to Q × P (or Q × P, see below). It is possible to perform several averages on the samples of boundary 17 so as to reach a reduced set 102 of samples with a reduced number of samples (e.g., in step 811). (At least one of the samples of the reduced number of samples 102 may be the average of two samples of the original boundary samples, or a selection of the original boundary samples). (e.g., in step 811). (At least one of the samples of the reduced number of samples 102 may be the average of two samples of the original boundary samples, or a selection of the original boundary samples). At least one of the samples of the reduced number of samples 102 may be the average of two samples of the original boundary samples, or a selection of the original boundary samples). For example, if the original boundary has P = M + N samples, the reduced set of samples can have P < P, where at least one of M red < M and N red < M and N red < N. Thus, the reduced set of samples can have P red = M red + N red and can have P < P. Thus, the boundary vector 17P actually used for prediction (e.g., in step 812 b) does not have P × 1 entries, but has P red < P, P red × 1 entries. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have Q × P dimensions, but has (M red < M and N red < N, where at least one of M < M and N red < N) at least P red (or Q red × P red ; see below).

[0097] In some examples (e.g., Figs. 7.2, 7.3), the block obtained by AL WIP (in step 812) is

Number

[0098] These techniques involve matrix multiplication with a reduced number of multiplications (Q red *P red or Q*P re d ) while including initial reduction (for example, averaging or downsampling) and final transformation Both conversions (e.g., interpolation) are performed by reducing (and even avoiding) multiplications. This can be advantageous because it can reduce non-computational power, such as additions and shifts. By employing binary operations that require and / or interpolation may be performed (e.g., steps 811 and / or 813 in. Also, addition is a very simple operation that can be easily performed without much computational effort. be.

[0099] This shift operation is, for example, done by shifting two boundary samples to obtain the final predicted block. To average the pull, and / or take from the (or boundary of) the reduced prediction block. The two samples (support values) obtained by the interpolation can be used to interpolate between them. For this, two sample values ​​are needed. Within a block, there are always two predefined values. But to interpolate samples along the left and top boundaries of the block, we use the , has only one predetermined value and therefore uses the boundary sample as the support value for the interpolation do.)

[0100] A two-step procedure can be used as follows: First add the values ​​of the two samples, The sum value is then halved (eg, by a right shift). Alternatively, you can: Each of the samples is first halved (e.g., by left shifting),

[0101] The values ​​of the two halved samples are then added. Select one sample amount and a group of samples (e.g. samples adjacent to each other) Since it is only necessary to do so, easier operations can be performed during downsampling (e.g., at step 811). Therefore, it is possible to define techniques for reducing the number of multiplications to be performed here. Some of these techniques may be based, in particular, on at least one of the following principles: Even if the size of block 18 actually predicted is M×N, the block is reduced (in at least one of the two dimensions) to a reduced size of Q ×P

[0102] where red ×P red (

Number

Number

Number

[0103] Additionally or alternatively, all Q=M*N values ​​of the block 18 to be predicted are multiplied by Instead of predicting, we use a reduced block with reduced dimensions (e.g.

number

number

number

[0104] According to the example shown in Figure 7.1, 18 4x4 blocks (M=4, N=4, Q=M*N=16 ) is predicted, and the neighborhood 17 of sample 17a (the four already predicted samples 17c (vertical matrix with 4 already predicted samples) and 17d (horizontal row with 4 already predicted samples) Neighborhoods 17a and 17c are collectively denoted as 17, as already predicted in the previous iteration. A priori, by using the formula shown in FIG. 5, the prediction matrix 17M can be calculated as Q× P = 16x8 matrix (because Q = M*N = 4*4 and P = M+N = 4+4 = 8) and the boundary vector 17P should have dimensions 8x1 (because P=8). However, this leads to the necessity of performing 8 multiplications for each of the 16 samples of the 4×4 block 18 to be predicted, and thus to the necessity of performing a total of 16 * 8 = 128 multiplications. (Note that the average number of multiplications per sample is a favorable evaluation of the computational complexity. In conventional intra prediction, 4 multiplications per sample are required, which increases the computational effort involved. Therefore, this can be used as the upper limit of ALWIP, ensuring that the complexity is reasonable and does not exceed that of conventional intra prediction.) This leads to the necessity of performing 8 multiplications for each of the 16 samples of the 4×4 block 18 to be predicted, and thus to the necessity of performing a total of 16 * 8 = 128 multiplications. (Note that the average number of multiplications per sample is a favorable evaluation of the computational complexity. In conventional intra prediction, 4 multiplications per sample are required, which increases the computational effort involved. Therefore, this can be used as the upper limit of ALWIP, ensuring that the complexity is reasonable and does not exceed that of conventional intra prediction.) However, by using this technique, it is understood that in step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2×2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, averaged samples), and the vertical column 17a, which originally had four samples, is processed as having two samples. However, by using this technique, it is understood that in step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c

[0105] However, by using this technique, it is understood that in step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c red <P is reduced However, by using this technique, it is understood that in step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2×2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, averaged samples), and the vertical column 17a, which originally had four samples, is processed as having two samples. <P. In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2×2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, averaged samples), and the vertical column 17a, which originally had four samples, is processed as having two samples. However, by using this technique, it is understood that in step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2×2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, averaged samples), and the vertical column 17a, which originally had four samples, is processed as having two samples. However, by using this technique, it is understood that in step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P (For example, the averaged sample). After dividing row 17c and column 17a into two groups of samples 110 respectively, a single sample is maintained (for example, the average of the samples in group 11 0 or a simple selection between the samples in group 110). It can also be understood that . Therefore, a so-called reduced set 102 of sample values is obtained by a set 102 having only four samples (M red = 2, N red = 2, P red = M red + N red = 4, P red < P). A so-called reduced set 102 of sample values is obtained.

[0106] It is understood that operations (such as averaging or downsampling in the ring 100) can be performed without performing an excessive number of multiplications at the processor level: The averaging or downsampling 100 performed in step 811 can be easily obtained by simple and computationally non-power-consuming operations such as addition and shift.

[0107] At this point, it is understood that the reduced set 102 of sample values can be subjected to a linear or affine linear (ALW IP) transformation 19 (for example, using a prediction matrix such as matrix 17M in FIG. 5). In this case, the ALWIP transformation 19 directly maps the four samples 1 02 to the sample values 104 of block 18. In this case, interpolation is not required.

[0108] In this case, the ALWIP matrix 17M has dimensions Q × P red = 16 × 4: This means that all Q = 16 samples of the block 18 to be predicted are directly taken by ALWIP multiplication. This is subject to the fact that the eigenvalues ​​are obtained (no interpolation is required).

[0109] Therefore, in step 812a, the dimensions Q×P red Having proper ALWIP The matrix 17M is selected. The selection may be based, for example, on the signaling from the data stream 12. The selected ALWIP matrix 17M may also be based at least in part on A k Shown in where k may be signaled in data stream 12. (In some cases, matrices can also be understood as indices that can be

number

[0110] In step 812b, the selected Q×P red ALWIP matrix 17M(A k year (also shown) and P red x1 Multiplication with boundary vector 17P is performed.

[0111] In step 812c, an offset value (e.g., b k ) is, for example, ALWIP Thus, it can be added to all the obtained values ​​104 of the obtained vector 18Q. . Offset(bk , or in some cases

number

[0112] As can be seen, averaging (possibly with addition and / or shifting) and / or downsampling) By having the above information, it is possible to obtain the appropriate value in step 812.

[0113] Referring to Figure 7.2, the block to be predicted, 18, is an 8x8 block of 64 samples. lock (M=8, N=8), where a priori, the prediction matrix 17M is of size Q×P = 64 x 16 (Q = M * N = 8 * 8 = 64, because M = 8 and N = 8, and P = M + N = 8 + 8 = 16, so Q = 64). Therefore, a priori, the prediction P = 16 multiplications for each of the Q = 64 samples in the 18 8x8 blocks of interest is needed, amounting to 64*16=1024 multiplications for the entire 18 8x8 blocks. You will!

[0114] However, as can be seen in Figure 7.2, we use all 16 samples of the boundary. Instead, eight values ​​(e.g., the four in row 17c and the four in row 17d) are used to determine the horizontal and vertical boundaries between the original samples of the boundary. A method 820 can be provided in which only four of the direct boundary columns 17a are used. From 17c, four samples may be used instead of eight (e.g., they are (It may be the average of two and / or the selection of one sample out of two). So the boundary vector is not a P×1=16×1 vector, but a P red x1=8x1 vector Tor only (P red =M red +N red =4+4). Original P=16 samples Instead of P red = 8 boundary values. and samples in vertical column 17a are selected or averaged (e.g., 2×2) to reduce the sample values. It is understood that it is possible to form a reduced set 102. 102 allows to obtain a reduced version of block 18, which is , (instead of Q=M*N=8*8=64) red =M red *N red =4*4=1 There are 6 samples. Size M red ×N red = AL predicting 4x4 blocks It is possible to apply the WIP matrix. A reduced version of block 18 is shown in Figure 7.2. Including the samples shown in grey in Scheme 106: Samples shown in grey squares (including samples 118' and 118'') are the samples obtained in step 812 of interest. Q red = 16 values. The 4x4 reduced block is obtained by applying the linear transformation 19 in step 812 After obtaining the values ​​of the 4x4 reduced block, the remaining samples (s It is possible to obtain the value of the sample (shown as a white sample in scheme 106). It is.

[0115] With respect to the method 810 of FIG. 7.1, the method 820 may be implemented, for example, by using M×N=8× 8 Block 18 Remaining QQ red =64-16=48 sample (white square) prediction The method may further include deriving 813 the value by interpolation. re d =64-16=48 samples are interpolated (interpolation also uses the values ​​of the boundary samples, e.g. Q obtained directly by red = 16 samples taken As can be seen in Figure 7.2, samples 118' and 118'' are The sample 108' (shown as a square) was taken in step 812. The pulls 118' and 118'' are shown in white in FIG. 3 is obtained by interpolation between samples 118' and 118''. The interpolation is also , can be obtained by operations similar to those for averaging, such as shifting and adding. Therefore, in Figure 7.2, the value 108' is generally The value can be determined as the intermediate value between the value of sample 118' and the value of sample 118'' ( can be taken as an average).

[0116] By performing the interpolation, in step 813, the plurality of samples shown in 104 are obtained. It is also possible to arrive at the final version of M×N=8×8 blocks 18 based on the pull value. be. Therefore, the comparison between using this technology and not using it is as follows: : Without this technology: a block 18 to be predicted, which is a block having dimensions M=8, N=8; and Q=M*N=8*8=64 samples of block 18 to be predicted, P=M+N=8+8=16 samples for boundary 17, P=16 multiplications for each of the Q=64 values ​​to be predicted, P*Q=16*64=1028 total multiplications The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q=16. When using this technology: Block 18 to be predicted, with dimensions M=8, N=8 The last Q = M * N = 8 * 8 = 64 values to be predicted,

[0117] However, P red = M red + N red , Q red = M red * N red , M r ed = 4, N red = 4, Q having red × P red The ALWIP matrix is used P red <P, the boundary P red = M red + N red = 4 + 4 = 8 samples For each of the 16 values of the 4×4 reduced block Q to be predicted red = P red = 8 times multiplication (formed by the gray squares in scheme 106), P red * Q red = 8 * 16 = 128 total multiplications (far less than 1024!)

[0118] The ratio of the number of multiplications to the number of final values to be obtained is P red * Q red / Q = 128 / 6 4 = 2 (much less than 16 obtained without using this technology!). Therefore, the technology presented in this specification requires 8 times less power than the previous technology.

[0119] Figure 7.3 shows another example (which can be based on method 820), the block to be predicted 18 is a rectangular 4×8 block (M = 8, N = 4) having Q = 4 * 8 = 32 samples to be predicted. The boundary 17 is formed by the horizontal row 17c of N = 8 samples and the vertical column 17a of M = 4 samples. Therefore, a priori, the boundary vector 17P is in inches It has the equation P×1 = 12×1, and the predicted ALWIP matrix is a Q×P = 32×12 matrix. It should be, and thus, Q*P = 32*12 = 384 multiplications are required.

[0120] However, for example, at least 8 samples of horizontal row 17c are averaged or downsampled to obtain a reduced horizontal row of only 4 samples (e.g., the averaged samples). In some examples, vertical column 17a remains as it is (e.g., without averaging). In total, the reduced boundary has dimension P = 8, and P < P. Thus, the boundary vector 17P has dimension P ×1 = 8×1 red = 8×1 red < P. Thus, the boundary vector 17P has dimension P red ×1 = 8×1 and has. The ALWIP prediction matrix 17M has dimension M*N red *P red = 4*4*8 = 64 and becomes a matrix with. The 4×4 reduced block directly obtained in the target step 812 (formed by the gray columns of schema 107) has size Q = M*N red = M*N red = 4*4 = 16 samples (instead of the original Q = 4*8 = 32 of the 4×8 block 18 to be predicted). When the reduced 4×4 block is obtained by ALWIP, an offset value b is added (step 812c), and interpolation can be performed in step 813. As can be seen in step 813 of FIG. 7.3, the reduced 4×4 block is expanded to a 4×8 block 18, and the value 108' not obtained in step 812 is obtained in step 813 by interpolating the values 118' and 118'' (gray squares) obtained in step 812. is added (step 812c), and interpolation can be performed in step 813. As can be seen in step 813 of FIG. 7.3, the reduced 4×4 block k is expanded to a 4×8 block 18, and the value 108' not obtained in step 812 is obtained in step 813 by interpolating the values 118' and 118'' (gray squares) obtained in step 812. is added (step 812c), and interpolation can be performed in step 813. As can be seen in step 813 of FIG. 7.3, the reduced 4×4 block is expanded to a 4×8 block 18, and the value 108' not obtained in step 812 is obtained in step 813 by interpolating the values 118' and 118'' (gray squares) obtained in step 812. 08’ is obtained in step 813 by interpolating the values 118’ and 118’’ (gray squares) obtained in step 812. 08’ is obtained in step 813 by interpolating the values 118’ and 118’’ (gray squares) obtained in step 812. Therefore, comparing using and not using this technology, it is as follows : When not using this technology: Block 18 to be predicted, which is a block with dimensions M = 4 and N = 8 Q = M * N = 4 * 8 = 32 values to be predicted, P = M + N = 4 + 8 = 12 boundary samples, P = 12 multiplications for each of the Q = 32 values to be predicted, Total number of multiplications: P * Q = 12 * 32 = 384 Ratio of the number of multiplications to the number of final values obtained: P * Q / Q = 12 When using this technology: Block 18 to be predicted, which is a block with dimensions M = 4 and N = 8 Finally, Q = M * N = 4 * 8 = 32 values to be predicted,

[0121] However, M = 4, N red = 4, Q red = M * N red = 16, P red = M + N red = 4 + 4 = 8, Q red × P red = 16 × 8, the ALWIP matrix can be used P red < P, the boundary P red = M + N red = 4 + 4 = 8 samples Q of the reduced block to be predicted red = 16 values each, P red = 8 multiplications Q red * P red = 16 * 8 = 128 total multiplications (less than 384!)

[0122] Ratio of the number of multiplications to the number of final values to be obtained: P red * Q red / Q = 128 / 3 ​​2=4 (much less than the 12 obtained without using this technique!). Therefore, the computational effort is reduced by a factor of 3 with this technique.

[0123] Figure 7.4 shows the example of a block 18 to be predicted having dimensions M×N=16×16, The final prediction target has Q=M*N=16*16=256 values, and P=M+N=16+1 6=32 boundary samples. This means that the prediction row has dimensions Q×P=256×32. This results in a sequence, which means 256*32=8192 multiplications!

[0124] However, by applying method 820, the boundary sample is Reduce the number of samples, for example from 32 to 8 (for example by averaging or downsampling) For example, for each group 120 of four consecutive samples in row 17a In addition, a single sample (e.g., selected from among four samples or Also, for each group of four consecutive samples in column 17c, a single of samples (e.g., selected from among four samples or the average of the samples) Remains.

[0125] Here, the ALWIP matrix 17M is Q red ×P red = 64 × 8 matrix: , it uses 8 averaged samples or samples selected from 32 boundaries. By using red = 8 and the fact that in step 812 The reduced block to be predicted is now an 8x8 block (in scheme 109, the grey squares are 64 This is due to the fact that

[0126] Therefore, the 64 samples of the 8×8 block reduced in step 812 are Once obtained, in step 813, the remaining QQ of the block 18 to be predicted red It is possible to derive 256-64=192 values ​​104.

[0127] In this case, to perform the interpolation, all samples of the boundary column 17a and the boundary row 17c are It has been chosen to use only alternate samples of . Other choices may be made.

[0128] In this method, the ratio between the number of multiplications and the number of values ​​finally obtained is Q red *P red / Q=8*64 / 256=2, which is less than 32 multiplications of each value without this technique. Much less! Therefore, the comparison between using this technology and not using it is as follows: : Without this technology: Block 18 to be predicted, which is a block with dimensions M=16, N=16 Q = M*N = 16*16 = 256 values ​​to predict, P=M+N=16*16=32 samples on the boundary, P=32 multiplications for each of the Q=256 values ​​to be predicted, P*Q=32*256=8192 total multiplications The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q=32. When using this technology: Block 18 to be predicted, which is a block with dimensions M=16, N=16 The final Q=M*N=16*16=256 values ​​to predict,

[0129] However, M red =4, N red = 4, Q of the target to be predicted by ALWIP red= 8 * 8 = 64 samples, P red = M red + N red = 4 + 4 = 8, Q re d × P red = 64 × 8 ALWIP matrix is used P red <P, the boundary P red = M red + N red = 4 + 4 = 8 samples Q of the reduced block to be predicted red = P for each of the 64 values red = 8 times of multiplication calculation Q red * P red = 64 * 4 = 256 total multiplications (less than 8192!)

[0130] The ratio of the number of multiplications to the number of final values to be obtained is P red * Q red / Q = 8 * 64 / 256 = 2 (much less than 32 obtained without using this technology!). Therefore, the computational power required by this technology is 16 times smaller than that of the conventional technology! Therefore

[0131] Reduce a plurality of adjacent samples (100, 813), and compare with a plurality of adjacent samples (1 7) to obtain a set of reduced sample values (102) with fewer samples obtained

[0132] Subject the set of reduced sample values (102) to linear or affine linear transformation (19, 17 M) to obtain the predicted values of the predetermined samples (104, 118’, 188’ ’) of the predetermined block (18) (812) By doing so, a plurality of adjacent samples (17) are used to determine a predetermined block of the picture ( 18) can be predicted.

[0133] In particular, we downsample multiple adjacent samples to obtain a multi-adjacent sample (17). By obtaining a reduced set of sample values ​​(102) with a relatively small number of samples, Therefore, it is possible to perform the reduction (100, 813).

[0134] Alternatively, multiple adjacent samples can be averaged and compared to multiple adjacent samples (17). By obtaining a reduced set of sample values ​​(102) with a smaller number of samples, It is possible to perform a reduction (100, 813).

[0135] Furthermore, given sample (104, 118', 118'') and a number of adjacent samples Based on the predicted value of (17), further samples (108, 1 08') can be derived (813) by interpolation.

[0136] A number of adjacent samples (17a, 17c) are arranged on either side (e.g. For example, it may extend in one dimension along the line (for example, toward the right and bottom of Figures 7.1 to 7.4). A given sample (e.g., obtained by ALWIP in step 812) Also, the pixels may be arranged in rows and columns, and a predetermined size may be provided along at least one of the rows and columns. The samples are the samples (11) of a given sample 112 adjacent to either side of a given block 18. 2) may be placed at every nth position.

[0137] For each of at least one of the rows and columns based on a plurality of adjacent samples (17), The support value (118) of one of the adjacent positions (118) aligned in a row is and for each of the columns. The predictions 118 of further samples (108, 108') of the block (18) are The predicted value of (104, 118', 118'') is aligned with at least one of the rows and columns. It is also possible to derive the value by interpolation based on the support values ​​of the adjacent samples (118) that are located in the same position. It is.

[0138] A given sample (104) is adjacent to the samples on either side of a given block 18 along a row. The samples may be arranged at every nth position along the row (112), and a given sample may be The samples (112) of a given sample adjacent (112) on both sides of a given block (18) ), where n, m>1. In some cases, n=m. (For example, in Figures 7.2 and 7.3, the data obtained directly by ALWIP in 812 Samples 104, 118', and 118'', shown in grey boxes, are In step 813, the samples 108, 108' are subsequently acquired. (Reciprocated).

[0139] Along at least one of the rows (17c) and columns (17a), for example, for each support value: In the plurality of adjacent samples, including the adjacent sample (118) for which the respective support values ​​are determined, Downsample or average (122) groups of adjacent samples (120) of It may be possible to perform the determination of the support value by In 7.4, in step 813, the location (previously obtained in step 812) is The values ​​of a given sample 118''' and adjacent samples 118 are used as support values. It is possible to obtain the value of sample 119 by:

[0140] A plurality of adjacent samples extend in one dimension along both sides of a given block (18). The plurality of adjacent samples (17) may be divided into one or more groups of consecutive adjacent samples (1 10) and groups of one or more adjacent samples with two or more adjacent samples. By performing downsampling or averaging on each of the groups (110), Thus, it may be possible to perform the reduction (811).

[0141] In the example, a linear or affine linear transformation is red *Q red or P red *Q weighting It can include coefficients, P red is the sample value (10 2) is the number of Q red or Q is the number of given samples in a given block (18). At least 1 / 4 of P red *Q red or 1 / 4 P red *Q weighting factor is , are non-zero weight values. P red *Q red or P red *Q weighting coefficients are Q or Q red For each of the given samples, A series of P red The weighting coefficients may be included in the set of weighting coefficients. If the samples are arranged one above the other in the raster scan order, then the Form a linear envelope. P red *Q or P red *Q red The weighting coefficients are For each given sample, The weighting coefficients of the associated first sequence and the weighting coefficients of the associated second sequence for each given sample other than the respective given sample are The average of the maximum cross-correlation between the weighting coefficients of the second series and the corresponding The one that is lower than the predetermined threshold value, even though it results in a higher maximum value. The threshold for can be 0.3 [or possibly 0.2 or 0.1]. red The adjacent samples (17) are arranged along two sides of a given block (18). The original path may be arranged along the Q or Q red For each of the given samples For each given sample, red The weighting coefficients are given by The images are ordered to be traversed in a specific direction.

[0142] 6.1 Method and Apparatus Description width

number

number

number

[0143] 1. Among the boundary samples 17, sample 102 (for example, when W=H=4, there are four samples, and / or 8 samples in other cases) are averaged or downsampled The rings can be extracted (eg, step 811).

[0144] 2. Matrix vector multiplication followed by adding an offset results in the averaged samples (also can be performed with the input (samples remaining from downsampling). The result is a reduced prediction on a set of subsampled samples in the original block. This may be a signal (eg, step 812).

[0145] 3. The predicted signal for the remaining positions is subsampled, e.g. by upsampling. The prediction signal can be generated from the set of prediction signals, for example by linear interpolation (e.g. step 813).

[0146] Thanks to steps 1.(811) and / or 3.(813), The total number of multiplications required for a calculation is always

number

[0147] In some examples, the matrices (e.g., 17M) and offsets required to generate the predicted signal are offset vector (e.g., b k ) is stored in the memory units of the decoder and the encoder, for example. A set of matrices that can be stored (e.g., three sets), e.g.

number

number

[0148] In some cases, the set

number

number

number

number

number

number

number

[0149] In some cases, the set

number

number

number

number

number

number

number

number

number

number

number

[0150] Additionally or alternatively, a set

number

number

number

number

number

Number

[0151] 6.2 Averaging or Downsampling of Borders Here, features are provided regarding step 811. As described above, boundary samples (17a, 17c) can be averaged and / or down sampled (e.g., from P samples to P red <P samples). It can be done.

[0152] In the first step, the input boundary

Number

Number

Number

Number

Number

Number

number

number

[0153] It is possible to define

number

number

number

number

number

[0154] In all other cases (e.g., for blocks with a wither width or height different from 4) Regarding the block width W,

number

number

number

number

[0155] In still other cases, (e.g., one particular boundary sample from a group of boundary samples) Downsample the boundaries (by selecting For example,

number

number

number

number

number

number

number

[0156] Two reduced boundaries

number

number

number

number

number

number

number

number

number

number

[0157] Therefore, in a particular state (

number

number

[0158] Other strategies may be implemented. In another example, the mode index "mode" is always nor in the range 0 to 35 (other ranges may be defined). There is no need for each of the subgroups S0, S1, and S2 to have 18 matrices (so

number

number

number

number

[0159] The mode and transposition information are not necessarily combined into one combined mode index, "mode" In some cases, the transposition flag and and matrix index (0-15 for S0, 0-7 for S1, 0-5 for S2) )

[0160] In some cases, the combination of the transpose flag and the matrix index is a set index. For example, one bit may act as a transposition flag and the other may be interpreted as a "set in" bit. There are several bits that indicate a matrix index, collectively referred to as the "matrix index." It is possible to exist.

[0161] 6.3 Generating reduced prediction signals by matrix-vector multiplication Characteristics are provided here with respect to step 812. The reduced input vector

number

number

number

number

number

number

number

number

number

[0162] Reduced predicted signal

number

number

[0163] Where:

number

number

number

number

number

number

number

number

[0164] Matrix A and vector

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0165] Other strategies may be implemented. In another example, the mode index "mode" is always nor in the range 0 to 35 (other ranges may be defined). There is no need for each of the subgroups S0, S1, and S2 to have 18 matrices (so

number

number

number

number

[0166] 6.4 Linear Interpolation to Generate the Final Prediction Characteristics are provided here with respect to step 812. In the case of the interpolation of the subsampled prediction signal in large blocks, the averaged boundary A second version of the following may be required:

number

number

number

number

number

number

number

number

[0167] Additionally or alternatively, it is possible to have "hard downsampling", So,

number

number

[0168] Also,

number

number

number

number

[0169] The linear interpolation may be given as follows (other examples are possible):

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0170] This uses the downscaled border samples for the first interpolation (horizontal or vertical) and This is an example of interpolation that uses original boundary samples between blocks (vertically or horizontally). Depending on the need, only the second interpolation or no interpolation is needed. Both horizontal and vertical interpolation are needed. If so, the order depends on the width and height of the block. However, different techniques may be implemented, for example if the original boundary samples are the first and The second interpolation may be used for both, and the order may be fixed, e.g., horizontal first, It may then be vertical (or in other cases first vertical, then horizontal). Therefore, the interpolation order (horizontal / vertical) and shrinking / using original border samples is changed. It is possible.

[0171] 6.5 Explaining the Overall ALWIP Process Example The whole process of averaging, matrix vector multiplication and linear interpolation is shown in different figures in Figure 7.1 to Figure 7.4. The remaining shapes are treated as one of the illustrated examples. Please note that.

[0172] 1.

number

number

number

[0173] 2.

number

number

number

[0174] 3.

number

number

number

[0175] 4.

number

number

number

number

[0176] Finally, for W×4 blocks where W>8,

number

number

[0177] 6.6 Evaluating the Number and Complexity of Required Parameters The necessary parameters for all possible proposed intra prediction modes are given in the set

number

number

number

[0178] 6.7 Proposed Signaling of Intra Prediction Modes For the luma block, for example, 35 ALWIP modes have been proposed (although there are other numbers For each coding unit (CU) in intra mode, Indicates whether the ALWIP mode should be applied to the corresponding prediction unit (PU). A flag indicating the index is transmitted in the bitstream. The latter index signaling is It can be harmonized with MRL in the same way as CE testing. ALWIP mode is applied. If so, the index in ALWIP mode

number

[0179] Here, the derivation of MPM is as follows, using the intra modes of the top and left PUs: Each of the conventional intra-prediction modes

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0180] The embodiments described herein are based on the above-mentioned signaling of the proposed intra prediction modes. According to an alternative embodiment, the MPM and / or the mapping table are , not used for MIP (ALWIP).

[0181] 6.8 Adaptation for Conventional Luma Intra Prediction Mode and Chroma Intra Prediction Mode MPM list derivation The proposed ALWIP mode is an MPM-based alternative to conventional intra prediction modes as follows: The luma and chroma M of conventional intra prediction modes can be coordinated with the coding of the signal. The PM list derivation process is a fixed table

number

number

number

number

[0182] For Luma MPM list derivation, ALWIP mode

number

number

[0183] 7. Efficient implementation The above is intended to provide a basis for further extension of the embodiments described below. The above examples are briefly summarized below. To predict a given block 18 of a picture 10, a number of adjacent samples 17a, c are To be used is to be used.

[0184] Reduction by averaging multiple adjacent samples 100 reduces the sample size compared to multiple adjacent samples. A smaller number of samples are taken to obtain a reduced set of sample values ​​102. The reduction of is optional in the embodiments of the present specification and is referred to below as the so-called sample value vector. The reduced set of sample values ​​is To obtain the predicted value of 104, the vector undergoes a linear or affine linear transformation 19. This transformation is ,Later, the matrix obtained by machine learning (ML) and the ,implementation should be performed efficiently. A and an offset vector b.

[0185] Interpolation allows predictions of other samples 108 in a given block to be calculated based on the given sample and It is derived based on the predictions of several neighboring samples. In theory, it is possible to The result is that all samples in block 18 are obtained by interpolation according to an alternative embodiment. It can be associated with the non-full-pel sample positions of block 18 so that Interpolation may not be necessary at all.

[0186] The plurality of adjacent samples may extend in one dimension along both sides of a given block, the samples are arranged in rows and columns, arranged along at least one of the rows and columns; A given sample is determined by dividing the given sample by the samples (112 ) may be placed at every nth position from the row and For each of at least one of the rows, a A support value can be determined, which corresponds to each of the rows and / or columns. , and by interpolation, the predicted values ​​for the further samples 108 of the given block are is the predicted value for a given sample and is aligned to the row and / or column The support values ​​for the neighboring samples may be derived based on the support values ​​for the neighboring samples. Samples 112 to n of a given sample adjacent to either side of a given block along a row A given sample may be placed at every position on either side of a given block along a column. may be located at every m-th position from the sample 112 of the adjacent given sample, , m>1. n=m may be used. Along at least one of the rows and columns, the support values The determination of includes, for each support value, the neighboring samples 118 for which the respective support value is determined. By averaging (122) groups of adjacent samples 120 within a plurality of adjacent samples The multiple adjacent samples extend in one dimension along both sides of a given block. The shrinking may be performed by dividing a plurality of adjacent samples into a group of one or more consecutive adjacent samples. 110, and one or more groups of adjacent samples having three or more adjacent samples. This may be done by performing an averaging over each of the loops.

[0187] For a given block, the prediction residual can be transmitted in the data stream. It can be derived therefrom in the decoder, and a given block is a given sample. The prediction residual and the predicted value of the input signal can be used to reconstruct the signal. The residual is encoded into a data stream in an encoder. A picture can be subdivided into multiple blocks of different block sizes. The plurality of blocks includes a predetermined block. Next, the linear or affine line of block 18 The shape transformation is selected according to the width W and height H of a given block. For a given block, as long as the width W and height H of the block are within the first set of width / height pairs, The linear or affine linear transformation selected as the first set of linear or affine linear transformations. a first set of width / height pairs, and a width W and height H of a given block are selected from the set of width / height pairs. The second set of linear or affine-linear transformations can be performed on the second set of width / height pairs, as long as they are within a second set of different width / height pairs. Similarly, later, the affine / linear transformation is performed on the other parameters, i.e., represented by weights, and optionally offset and scale parameters becomes clear.

[0188] The decoder and the encoder divide the picture into blocks of different block sizes, including the predetermined block. Subdivision into multiple blocks and linear or axial alignment depending on the width W and height H of a given block. The method may be configured to select a linear transformation such that for a given block The linear or affine linear transformation selected by As long as the width W and height H of a given block are within the first set of width / height pairs, linear or or a first set of affine linear transformations, A given block has a width W and height H that is a different set of width / height pairs from the first set of width / height pairs. a second set of linear or affine linear transformations, and The width W and height H of a given block are different from the first and second sets of width / height pairs. A third set of one or more width / height pairs can be transformed using a linear or affine-linear transformation. , . . . , selected from the set

[0189] The third set of one or more width / height pairs simply includes one width / height pair W', H', and is Each linear or affine linear transformation in the first set of linear or affine linear transformations has N' to convert the sample values ​​of It is something. Each of the first and second sets of width / height pairs is W p H p First width not equal to / Height vs W p , H p And, Hq =W p and W q =H p The second width / height pair W q , H q and may include.

[0190] Each of the first and second sets of width / height pairs is a third width / height pair W p , H p Wow It can also be included in W p H p is equal to H p >H q It is. For a given block, which of a given set of linear or affine linear transformations or a set index indicating which affine linear transformation should be selected for block 18. The text can be transmitted within the data stream.

[0191] The plurality of adjacent samples may extend in one dimension along either side of a given block; The downsizing is performed on a first subset of a plurality of adjacent samples adjacent to a first side of the given block. The first subset is then divided into a first group 110 of one or more consecutive adjacent samples. a second subgroup of a plurality of adjacent samples adjacent to a second side of the given block; For the set, a second subset is defined as a second group of one or more contiguous adjacent samples. 110, and a first sample value from the first group and a second sample value from the second group are 1 or more adjacent samples having three or more adjacent samples to obtain the second sample value of By performing averaging on each of the first and second groups of samples Then, a linear or affine linear transformation can be done by , and the set index from a predetermined set of Two different states of bit index are given a set of linear or affine linear transformations. The output vector of predicted values ​​is also A set index that assumes the first of two different states in the form of a first vector to In the case of a filter, the reduced set of sample values ​​is subjected to a given linear or affine linear transformation. and a predicted value of the output vector can be calculated based on the location of the predetermined block along the first scan order. The second vector can be used to represent the two different states of the second. For a given index, the first The input element is the first sample value of the vector, and the input element is the second sample value of the second vector. The second vector is input to one of the input channels and the second sample value of the first vector is input to one of the input channels. The first and second vectors are different so that one of the first samples of the vector A second scan to a given sample of a given block is then performed which is transposed with respect to the first scan order. The predicted values ​​of the output vectors along the order can be distributed.

[0192] Each linear or affine linear transformation in the first set of linear or affine linear transformations is One sample value is converted to w1*h1 predicted values ​​for a w1×h1 array of sample positions. In a first set of linear or affine linear transformations, Each linear or affine-linear transformation maps the N2 sample values ​​to a w2 × h2 array of sample positions. The first width / height pair is then converted to w2*h2 predicted values ​​for the first For a given one of 1, w1 can be greater than the width of the first given width / height pair, and h1 can be greater than the height of a first given width / height pair and greater than the height of a second given width / height pair. For a given one, w1 cannot exceed the width of a second given width / height pair, and h1 The height of the second predetermined width / height pair cannot be exceeded. By taking multiple adjacent samples to obtain a reduced set of sample values ​​(102), The step of (100) includes: if the given block is of a first given width / height pair; and and if a given block is of a second given width / height pair, a reduced version of the sample values ​​is obtained. The set 102 may be made to have N1 sample values, and the reduced sample Subjecting the set of values ​​to a selected linear or affine linear transformation determines whether a given block is For the first given width / height pair, if w1 exceeds the width of one width / height pair along the width dimension, or along the height dimension if h1 exceeds the height of one width / height pair The selected linear or This may be done by using only a first sub-part of the affine linear transformation, given If the block is of a second given width / height pair, the selected linear or affine A linear transformation may be performed entirely.

[0193] Each linear or affine linear transformation in the first set of linear or affine linear transformations is For a w1×h1 array with sample positions 1=h1, we define N1 sample values ​​as w1*h It can be used to convert one predictor into a linear or affine linear transformation. Each linear or affine linear transformation in the set of 2 is a function of w2 of sample positions where w2 = h2. for converting N2 sample values ​​for a ×h2 array into w2*h2 predicted values It is.

[0194] All of the above-mentioned embodiments may form the basis of the embodiments described herein below. That is, the above concepts and details are merely exemplary in that they may be implemented in the following embodiments: This will be helpful in understanding the present invention and will provide a basis for possible extensions and amendments to the embodiments described later in this specification. In particular, many of the details mentioned above involve averaging adjacent samples, It is optional, such as the fact that the sample is used as a reference sample.

[0195] More generally, the embodiments described herein are directed to an intra prediction signal on a rectangular block: A rectangular block is generated from the adjacent already reconstructed samples to the left and above the block. We assume that the predicted signal on the clock is generated from already reconstructed samples. The generation is based on the following steps:

[0196] 1. However, it excludes the possibility of transferring the description to a reference sample placed elsewhere. Without any additional calculations, samples are extracted by averaging from reference samples, called boundary samples. Here, the averaging is done on both the left and top boundary samples of the block. Averaging is performed only on the boundary samples on either side. If not, the sample on that side remains unchanged.

[0197] 2. A matrix vector multiplication is performed, optionally followed by the addition of an offset. The input vector of the matrix vector multiplication is the block vector if averaging is applied only to the left side. The averaged boundary sample of the block is concatenated to the left of the block with the original boundary sample above the block, or If equalization is applied to the top side only, the left of the original border samples of the block and the top of the block are evenly spaced. If averaging or concatenation with smoothed boundary samples is applied to both sides of the block, The averaged boundary sample to the left of the block and the averaged boundary sample above the block. Again, there are alternatives, such as no averaging at all. There is.

[0198] 3. The result of the matrix vector multiplication and optional offset addition is optionally The reduced prediction signal on the set of subsampled samples in the block is The prediction signal at the remaining positions can be subsampled by linear interpolation. The signal can be generated from a set of predicted signals.

[0199] The matrix-vector product calculation in step 2 should preferably be performed in integer arithmetic. There is. Therefore,

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0200] FIG. 8 illustrates a method for predicting a given block 18 of a picture 10 using multiple reference samples 17. 1 shows an embodiment of an apparatus 1000 for detecting a plurality of reference samples 17 in a predetermined block. Dependent on the prediction mode used by the device 1000 to predict the block 18. When the prediction mode is, for example, intra prediction, a reference block adjacent to a given block can be In other words, a plurality of reference samples 17 can be used, e.g. For example, the prediction model may be arranged in the picture 10 along the outer edge of a given block 18. If the prediction mode is inter prediction, the reference samples 172 of the other picture 10' are used. It is possible.

[0201] The apparatus 1000 forms a sample value vector 400 from a plurality of reference samples 17. The sample vector can be obtained by different techniques. The sample value vector can, for example, include all the reference samples 17. Optionally, the reference samples can be weighted. According to another example, the sample values Vector 400 is one of the samples in Figure 7.1 to Figure 7.4 for sample value vector 102. In other words, the sample value vector 40 The zero can be formed by averaging or downsampling. Thus, For example, a group of reference samples may be averaged to produce a reduced set of sample values. In other words, the device can obtain a sample value vector 400, for example. For each component of the 400 samples, one of several reference samples17 was used. as components of a sample vector and / or Averaging two or more components of the sample vector 400, i.e. To obtain each component, two or more reference samples from multiple reference samples17 were used. By averaging the pulls, a sample value vector 102 is obtained from multiple reference samples 17. The device is configured to form 100.

[0202] The apparatus 1000 extracts from the sample value vector 400 a sample value vector 400 corresponding to a given Derive 401 a further vector 402 that is mapped by a reversible linear transformation 403 The further vector 402 may be, for example, an integer value and / or a fixed point value. The reversible linear transform 403 includes, for example, a prediction of the samples of a given block 18. is selected to be performed using integer or fixed-point arithmetic.

[0203] Furthermore, the device 1000 may also perform a computation between the further vector 402 and a predefined prediction matrix 405. A matrix vector product 404 is calculated to obtain a predicted vector 406, and the predicted vector 406 is It is configured to predict the samples of a given block 18 based on the Based on the vector 402, a given prediction matrix is ​​quantized to produce a prediction subsequence for a given block 18. Integer and / or fixed-point arithmetic with only minor effects of quantization error on samples It can be made possible.

[0204] According to an embodiment, the apparatus 1000 performs the matrix vector product 404 using fixed-point arithmetic. Alternatively, integer arithmetic can be used. According to an embodiment, the apparatus 1000 calculates the matrix vector product 404 without floating point operations. The device is configured to:

[0205] According to an embodiment, the device 1000 stores a fixed-point representation of the predetermined prediction matrix 405. Additionally or alternatively, an integer representation of the predefined prediction matrix 405 is stored. It can be done. According to an embodiment, the device 1000 determines the predetermined block 18 based on the prediction vector 406. In predicting the next sample, at least one of the samples in a given block 18 is predicted based on the prediction vector 406. Each component is constructed to use interpolation to calculate a single sample position. , are associated with corresponding positions in a given block 18. Interpolation is shown in Figs. 7.1 to 7.4. It can be carried out as described with respect to any of the embodiments shown in .

[0206] FIG. 9 illustrates the inventive concept described herein. A sample of a given block is The matrices A1100 and san derived by several machine learning based training algorithms. The prediction can be made based on a first matrix-vector product between the input value vector 400 and the Optionally, an offset b 1110 can be added. To achieve an integer or fixed-point approximation of the vector product, the sampled vector is further The vector 402 can then be subjected to a reversible linear transformation 403 to determine the vector 402. The second matrix-vector product between the matrix B 1200 and the further vector 402 is It can be made equal to the result of a column vector product.

[0207] For the further vector 402 features, the second matrix vector product is A matrix vector product between 405, a further vector 402 and a further offset 408 404. Further vectors 402 and further objects can be approximated by The offset 408 can consist of integer or fixed point values. All elements of the matrix are, for example, the same. The given prediction matrix 405 is a quantized matrix. The matrix 405 may be a predefined prediction matrix or may be a matrix to be quantized. The result of the matrix vector product 404 between the input vector 402 and the input vector 404 is known as the predicted vector 406. It is possible.

[0208] Further details regarding this integer approximation are provided below. Possible Solutions for Embodiment I: Subtracting and Adding Average Values Usable expressions for the above scenarios

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0209] The specified value 1400 is not necessarily the average value.

number

number

number

number

number

number

number

number

number

number

[0210] Alternatively, the predetermined value 1400 may be a default value or a data stream in which the picture is encoded. This is the value signaled in the stream. The predetermined value 1400 is, for example, 2 bitdepth-1 In this case, the further vector Tor 402, when i>0, y0=2 bitdepth-1 and y i =x i -x0 Thus, it can be defined.

[0211] Alternatively, the predetermined component 1500 is a constant obtained by subtracting the predetermined value 1400. The constant is, for example, For example, 2 bitdepth-1 According to an embodiment, a further vector y 40 2. Prescribed ingredients

number

number

[0212] According to an embodiment, the device 1000 is configured to include a number of reversible linear transforms 403. , each of which is associated with a component of a further vector 402. For example, a predetermined component 1500 is selected from the components of the sample value vector 400, and A reversible linear transformation 4 among a plurality of reversible linear transformations associated with a given component 1500 of 03 as its predetermined reversible linear transform. Depending on the position of the given component in the vector 402 consisting of due to different positions of the rows of the reversible linear transform 403 corresponding to If the first component of 402, i.e., y1, is a given component, then the i o The rows of are reversible linear variables. Replace the first line of the transformation.

[0213] As shown in FIG. 10b, a predetermined vector 402 corresponding to a predetermined component 1500 of the further vector 402 Column 412 of the prediction matrix 405, i.e., the matrix i0 of the given prediction matrix C 405 in the i0th column Component 414 is, for example, all zeros. In this case, the device may, for example, Then, the resulting reduced matrix C 405 by leaving column 412 The predicted matrix C' 405 and the further vector C' are obtained by leaving the predetermined components 1500. A matrix vector product 407 between the yet further vector 410 resulting from 402 To calculate the matrix vector product 404, we perform the multiplication by calculating Therefore, the prediction vector 406 can be calculated with fewer multiplications. can.

[0214] As shown in FIG. 9, FIG. 10b and FIG. 10c, the device 1000 may For each component of the prediction vector 406, The sum of each component and a, i.e., a predetermined value 1400, is calculated. This addition can be done by adding the predicted vector 406 and the vector as shown in FIG. 9 and FIG. 10c. The vector 409 can be expressed as a sum of the vector 409, and all components of the vector 409 are expressed as a predetermined value. 1400. Alternatively, the addition can be done by adding the predicted vector 406 and The sum of the matrix vector product 1310 between the integer matrix M 1300 and the further vector 402 and the matrix elements of the integer matrix 1300 can be represented by a further vector 402 A column of the integer matrix 1300 corresponding to a given element 1500 of the matrix i0, i.e., a 1 in the i0th column, and all other components are, for example, 0.

[0215] The result of the sum of the predetermined prediction matrix 405 and the integer matrix 1300 is, for example, the further The matrix 1200 is equal to or approximates the matrix 1200 , In other words, a given prediction matrix 402 corresponding to a given element 1500 of the further vector 402 Each matrix element of a given prediction matrix C 405 in the i0th column 412 of the i0th column is calculated as The matrix obtained by multiplying the inverse linear transform 403 by 1 (i.e., matrix B) is The matrix B 1200 is, for example, a machine learning matrix as shown in FIG. 9, FIG. 10a, and FIG. 10b. It corresponds to a quantized version of prediction matrix A 1100. Summing each matrix element of matrix C 405 with 1 results in a given predicted It can correspond to the sum of matrix 405 and integer matrix 1300. As shown in FIG. The machine-learned prediction matrix A 1100 is the result of a reversible linear transform 403 of a further matrix 1200. This can be equated to

number

[0216] Matrix multiplication using integer arithmetic only Low complexity implementation (in terms of the complexity of adding and multiplying scalar values, as well as the parting In terms of storage required for column entries, matrix multiplication can be done in 40 steps using integer arithmetic only. It is advisable to do 4.

number

number

number

number

number

[0217] A matrix of size m×n, i.e. a matrix vector product 404 with a given prediction matrix 405 is , can be implemented as shown in this pseudocode, where <<,>> is an arithmetic binomial. left and right shift operations, and +, -, and * operate only on integer values. .

[0218] (1) final_offset=1<<(right_shift_result-1); For i in 0...m-1 { Accumulator = 0 For j in 0...n-1 { Accumulator:=Accumulator+y[j]*C[i,j] } z[i] = (accumulator + final_offset) >> right_shift_result; }

[0219] Here, the array C, i.e., the predefined prediction matrix 405, is expressed as a fixed-point number, e.g., as an integer. The final addition of final_offset and right shift operation by right_shift_result. reduces precision by rounding to obtain the required fixed-point format at output. To allow for an increase in the range of real values ​​representable by integers in C, Figures 11 and 1 As shown in the second embodiment, two additional matrices

number

number

number

number

number

number

number

number

number

[0220] In other words, the apparatus 1000 may estimate a prediction parameter, e.g., an integer value

number

number

number

[0221] According to an embodiment, the prediction parameters are each associated with a corresponding matrix element of the prediction matrix. In other words, a given prediction matrix includes weights assigned to the prediction parameters, e.g. The weights can be expressed as integers and / or prediction parameters. or a fixed point value.

[0222] According to an embodiment, the prediction parameters are determined by one or more scaling factors, e.g. the value sca le i,j each of which corresponds to one or more of the given prediction matrices 405 weights associated with matrix elements, e.g. integer values

number

number

number

number

[0223] For example, in one preferred embodiment,

number

number

number

number

[0224] According to an embodiment,

number

number

number

number

[0225] Offset Expression

number

number

[0226] A wide range of embodiments resulting from the solution The above solution implies the following embodiments: 1. A method of prediction as described in Section I, comprising the steps of step 2 of Section I. For integer approximations of the matrix-vector products involved, the following is done:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0227] 2. A method of prediction as described in Section I, comprising the steps of Section I. In 2, the following is done for integer approximations of the matrix-vector products involved:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0228] 3. A method of prediction as described in Section I, comprising the steps of:

number

number

number

[0229] 4. A method of prediction as described in Section I, wherein step 2 comprises: One of K matrices is used so that the measurement modes can be calculated, each of which is Different matrices with k=0...K-1

number

number

number

number

[0230] That is, according to an embodiment of the present application, the encoder and decoder are To predict a given block 18 of the model 10, we operate as follows: As outlined above, the embodiments of the present application are Therefore, the reference samples are adjacent samples, i.e., The reference samples are not limited to the samples of the 10 adjacent blocks 18. In particular, the reference samples are The samples may be arranged along the outer edge of the block 18, such as, but not limited to, samples abutting the outer edge. However, this situation is certainly one embodiment of the present application.

[0231] To make predictions, samples from reference samples such as reference samples 17a and 17c A value vector 400 is formed. Possible formations have been described above. Formation may include averaging. and thereby the number of sample 102 compared to the reference sample 17 that contribute to the formation This can reduce the number of components of the vector 400. 18. The block 18 may be in any way dependent on the dimensions or size of the block 18, such as the width and height of the block 18. It is possible.

[0232] This vector should undergo an affine or linear transformation to obtain the prediction of block 18. Tor 400. Different nomenclature is used above. Use the latest one and Vectors are added to matrix A by matrix-vector multiplication within the scope of performing addition with set vector b. The goal is to perform prediction by applying the rule 400. b is arbitrary. An affine or linear transformation determined by A or A and b has the sign or, more precisely, by the encoder and decoder of block 18 as already mentioned above. A determination can be made for the prediction based on size and dimensions.

[0233] However, to achieve the improvements in computational efficiency outlined above, or to perform the predictions To make it more effective, the affine or linear transformation is quantized and the encoder and and the decoder, or its predictor, represents a quantized version of the affine transformation. To represent and perform linear or affine transformations with C and T applied in In particular, the predictors in the encoder and decoder use vector 400 Instead of applying directly to matrix A, we apply vector 400 obtained from sample vector 400. 2 by subjecting it to a mapping through a given invertible linear transformation T, where The transformation T used is the same as long as the vectors 400 have the same size, i.e. Independent of block dimensions, i.e. width and height, or dependent on different affine / linear transformations In the above, vector 402 is denoted by y. The exact matrix for performing the affine / linear transformation determined by the study was B. However, instead of implementing B exactly, the predictions in the encoder and decoder are This can be done by an approximation or quantized version. In particular, the representation can be expressed as C in the manner outlined above. where C+M represents a quantized version of B.

[0234] Therefore, the prediction in the encoder and decoder is made by the vector 402 and the encoder and decoder A matrix vector product 404 between a predetermined prediction matrix C appropriately represented and stored in the decoder is performed. This is further accomplished by calculating in the manner described above. Then, from this matrix-vector product The resulting vector 406 is used to predict the samples 104 of the block 18 . As mentioned above, for prediction purposes, each element of the vector 406 is a vector that compensates for the corresponding definition of C. To do this, it can be summed with a parameter a, as shown at 408. The optional addition of vector 406 with an offset vector b is also based on vector 406 As mentioned above, the vector 40 can be included in the derivation of the prediction of block 18 based on the 6, and therefore each component of the sum of vector 406, all of which are shown in 408 The vector a, and the optional vector b, are then directly input to sample 104 of block 18. The sample 104 of the block corresponds to the sample 104 of the block 102 and can therefore indicate the predicted value of the sample. Only a subset of,is predicted in this way, and the remaining samples in block 18,such as 108, It may also be derived by interpolation.

[0235] As mentioned above, there are different embodiments for setting a. For example, the vector 400 In this case, see FIG. 10. The shape transformation T can be as shown in FIG. vector and a given component of vector 402, which is replaced by a. As shown above, there are other possibilities. However, as far as the representation of C is concerned, It has also been shown above that this can be implemented differently. For example, The matrix product 404 is, in its actual calculation, a smaller matrix vector having a lower dimension. In particular, as stated above, by the definition of C, the i0th Since the entire column 412 of is zero, the actual calculation of the product 404 is

number

[0236] The weights of C or C', i.e., the elements of this matrix, are expressed in fixed-point notation. However, these weights 414 can also be stored as , may be stored in a manner that is associated with different scales and / or offsets. The rules and offsets may be defined for the entire matrix C, i.e., for matrix C or It may be equal for all weights 414 of matrix C', or it may be equal for matrices C and C'. A constant or equal value is set for all weights 414 in the same row or for all weights 414 in the same column, respectively. In this regard, FIG. 11 shows the calculation of the matrix vector product, That is, the result of the product may be implemented slightly differently in practice, i.e., For example, by shifting the multiplication with the scale towards vector 402 or 404, This reduces the number of multiplications that must be performed. Figure 12 shows that the eigenvalues ​​C and C' are calculated as in calculation (2) above. 414. The following shows the case where one scale and one offset are used for all weights 414 in the is doing.

[0237] According to an embodiment, a device as described herein for predicting a given block of a picture can be configured to use matrix-based in-sample prediction that includes the following features: Cut: The apparatus forms a sample value vector pTemp[x] 400 from a plurality of reference samples 17. Assume that pTemp[x] is 2*boundarySize. Then pTemp[x] can be calculated, for example, by direct copying or by subsampling. Or by pooling, the adjacent samples at the beginning of a given block, x=0..bo redT[x] with undarySize-1, followed by the The neighboring samples, redL[x], with x=0..boundarySize-1 (e.g. by the transposition process (e.g., when transposition = 0) or by the transposition process (e.g., when transposition = 1). can be populated by its inverse.

[0238] The input values ​​p[x] for x=0..inSize-1 are derived, i.e., the device Thus, from the sample value vector pTemp[x], derive a further vector p[x] such that p[x] is mapped by a given invertible linear transformation or more specifically, a predetermined invertible affine linear transformation. If -mipSizeId is equal to 2, the following applies: p[x]=pTemp[x+1]-pTemp[0] - Otherwise (mipSizeId is less than 2), the following applies: p[0]=(1<<(BitDepth-1))-pTemp[0] p[x]=pTemp[x]-pTemp[0] for x=1..inSize-1

[0239] Here, the variable mipSizeId indicates the size of a given block. According to an embodiment, a further vector is derived from the sample value vector using The reversible transform depends on the size of the given block. The dependency is given by It is possible.

[0240] [Table 3]

[0241] predSize indicates the number of predicted samples in a given block, and 2*bondar ySize indicates the size of the sample vector, and inSize, i.e., inSi ze=(2*boundarySize)-(mipSizeId==2)?1:0 Therefore, inSize is related to the size of the further vector: 1:0. More precisely, inSize is , indicates the number of components of the further vector actually involved in the calculation. The small block size is the same as the size of the sample vector, and the large block size is In the former case, one component is smaller, i.e., the component calculated later. Like a matrix-vector multiplication, the components of a further vector that correspond to a given component are ignored. The contribution of the corresponding vector component will be zero anyway, and therefore it is practically The dependence on block size requires that only one of the two alternatives is In the case of an alternative embodiment where the can be left alone (if the option corresponding to mipSizeId is less than 2 or or the option corresponding to mipSizeId is equal to 2).

[0242] In other words, a given invertible linear transformation is, for example, a given component of a further vector p is transformed into a and all other components are defined to correspond to the components of the sample vector minus a. For example, a = pTemp[0]. The first one corresponds to mipSizeId equal to 2. In the case of the option, this is easily seen and only the separately formed components of the further vectors is further considered. That is, in the first option, the further vector is actually is {p[0...inSize];pTemp[0]}, where pTemp[0] is a The actual calculated part of the matrix-vector multiplication to produce the matrix-vector product, i.e. That is, the result of the multiplication is a further vector and and the inSize elements of the corresponding columns of the matrix. For all components of the further vectors except p[0], , a = pTemp[0] is selected, i.e., the further vector excluding the given component p[0] is Each of the other components p[x] (for x=1..inSize-1) of the matrix p is a sample. The corresponding element of the vector pTemp[x] is equal to a minus a, whereas p[0] is equal to a constant The matrix vector product is then calculated. The constants are chosen to be minus a. The average of the possible values, i.e. 2 x-1 (i.e. 1<<(BitDepth-1)) where x indicates the bit depth of the computation representation used. p[0] is replaced by pTemp[0 ], the computed product is calculated using p[0] as above. From the product (p[0]=(1<<(BitDepth-1))-pTemp[0]), This can be taken into account when predicting the inside of a block based on the product, i.e., the prediction vector. Note that the values ​​of a deviate by a constant vector. Thus, the value a is a predefined value, e.g. pTemp[0]. In this case, the predetermined value pTemp[0] is, for example, The component of the sample value vector pTemp that corresponds to [0]. the sample above or to the left of a given block that is closest to the top-left corner of It can be said that:

[0243] For example, specifying the intra prediction mode, etc. In the case of an in-pull prediction process, the device is configured to apply, for example, the following steps: , for example, run at least the first step: 1. Matrix-based iterative iteration with x=0..predSize-1, y=0..predSize-1. The intra prediction sample predMip[x][y] of the input signal is derived as follows: - The variable modeId is set equal to predModeIntra. -x=0..inSize-1, y=0..predSize*predSize-1 The weight matrix mWeight[x][y] is calculated using mipSizeId and modeId as inputs. The weight matrix is ​​derived by invoking the MIP weight matrix derivation process. - Matrix base where x=0..predSize-1, y=0..predSize-1 The intra prediction samples predMip[x][y] are derived as follows: oW=32-32*(

number

number

[0244] In other words, the device now performs the following operations to yield the array predMip[x][y]: Already assigned to an array of block positions {x,y} distributed inside a given block To obtain the predicted vector p[i], we perform a matrix-vector multiplication between the further vector p[i], or if mipSizeId is equal to 2, {p[i];pTemp[0]} and the location If the specified prediction matrix mWeight or mipSizeId is less than 2, the number of p is omitted. To calculate the prediction matrix mWeight with additional zero weight lines corresponding to the omitted components, The prediction vector is constructed as follows: These correspond to the concatenation of the columns of p[x][y], respectively.

[0245] According to an embodiment, or according to a different interpretation, the component (((

number

[0246] The apparatus optionally predicts samples of a given block based on the prediction vector. When doing this, use the following steps, e.g. predMip or ((

number

[0247] 2. Matrix vector with x=0..predSize-1, y=0..predSize-1 For example, the intra prediction sample predMip[x][y] of the input image is clipped as follows: To be uploaded: predMip[x][y]=Clip1(predMip[x][y])

[0248] 3. If the transpose is equal to true, then x=0..predSize-1, y=0..predS The predSize×predSize array predMip[x][y] of ize-1 is , for example, is transposed as follows: predTemp[y][x]=predMip[x][y] predMip=predTemp

[0249] 4. Prediction sample predS, where x=0..nTbW-1 and y=0..nTbH-1 For example, amples[x][y] can be derived as follows: - nTbW, which specifies the transform block width, is greater than predSize, or the transform block If the height nTbH is greater than predSize, MIP prediction upsampling is performed. The ring process operates on input block size predSize, x=0..predSize -1, y=0..predSize-1 matrix-based intra prediction samples pr edMip[x][y], transform block width nTbW, transform block height nTbH, x=0 ..nTbW-1, and y=0..nTbH It is called with the left reference sample refL[y] with -1 as input, and the output is the predicted sample The lu array is predSamples. - otherwise, predS with x=0..nTbW-1, y=0..nTbH-1 amples[x][y] is set equal to predMip[x][y]. In other words, the device estimates the samples of a given block based on the prediction vector predMip. It is configured to predict predSamples.

[0250] FIG. 13 illustrates a method for predicting a given block of a picture using multiple reference samples. A method 2000, comprising forming a sample value vector from a plurality of reference samples. 00 and the sample-value vector is transformed by a given reversible linear transformation. deriving 2200 a further vector to be mapped and To do this, a matrix-vector product is calculated between the further vector and the predetermined prediction matrix 230. 0, and predicting 2400 samples of a given block based on a prediction vector; 2 shows a method 2000 including:

[0251] References [1] P. Helle et al., “Non-linear weighted intra p ,JVET-L0199,, Macau, China, October 2018. [2] F.Bossen, J.Boyce, K.Suehring, X.Li,VS eregin, “JVET common test conditions and so ftware reference configurations for SDR vi deo”, JVET-K1010, Ljubljana, Slovenia, July 2018.

[0252] Further embodiments and examples Generally, the examples are implemented as a computer program product comprising program instructions. The program instructions may be used to cause the computer program product to execute on a computer. The program instructions, when executed, function to perform one of the methods. It may be stored in the body. Another example is a computer readable medium for carrying out one of the methods described herein, stored on a machine readable carrier. It is equipped with a computer program for this purpose.

[0253] Thus, an example method is one in which the computer program, when executed on a computer, A computer processor having program instructions for carrying out one of the methods described herein. grams.

[0254] Thus, a further example of a method is a computer program for carrying out one of the methods described herein. a data carrier medium (or digital storage medium) having a computer program recorded thereon; or computer readable medium). The recording medium is tangible and / or non-transitory, rather than an intangible, transitory signal. Thus, a further example of a method is a computer program for carrying out one of the methods described herein. A data stream or sequence of signals that represents a computer program. The stream or sequence of signals may be transmitted over a data communication connection, for example the Internet. Can be forwarded.

[0255] A further example is a processing means for carrying out one of the methods described herein, e.g. a computer or a programmable logic device. Further examples include computer programs for carrying out one of the methods described herein. The computer includes a RAM installed thereon.

[0256] A further example is a computer program for carrying out one of the methods described herein. A receiver includes a device or system that transmits (e.g., electronically or optically) the received signal to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The device or system may, for example, include a file for transferring a computer program to a receiver. A rule server may be provided.

[0257] In some instances, programmable logic devices (e.g., field programmers) A multi-gate array (MGA) may be used to provide some or all of the functionality of the methods described herein. In some examples, the field programmable gate array may include: A microprocessor capable of cooperating with the device to carry out one of the methods described herein. In general, the methods may be performed by any suitable hardware apparatus.

[0258] The above examples are merely illustrative of the principles described above. It is understood that various modifications and variations of the invention will be apparent to those skilled in the art. The specific details set forth in the description and explanation are not to be construed as limiting the scope of the appended claims. It is intended to be limited by Equal or equivalent elements or elements with equal or equivalent functions occur in different figures. Even if so, they are indicated in the following description by equal or similar reference numerals.

Claims

1. 1. A method for predicting a block of a picture, comprising: obtaining one or more sample values ​​from a plurality of adjacent samples located along a boundary of the block; determining an input value from the one or more sample values ​​based on an indication of a size of the block; determining a matrix-based intra prediction sample by applying a predefined prediction matrix to the determined input values; predicting samples of the block based on the determined matrix-based intra prediction samples; A method for providing the above.

2. The method of claim 1 , wherein the plurality of adjacent samples are located along a top and left boundary of the block.

3. Determining the input value (p[x]) based on the indication of the size of the block (mipSizeId) comprises: determining p[x] from one or more sample values ​​(pTemp[x]) according to p[x]=pTemp[x+1]-pTemp[0] if mipSizeId is equal to 2; if mipSizeId is less than 2, determining p[x] according to p[x]=pTemp[x]-pTemp[0], where x=1, 2, ..., inSize-1, where inSize corresponds to the number of components of p[x]; The method of claim 1 , comprising:

4. determining a matrix-based intra-prediction sample (predMip) for the determined input value (p[x]) by applying a predetermined prediction matrix (mWeight), [Number 362] 2. The method of claim 1, wherein predSize indicates a number of predicted samples in the block, pTemp[0] is a sample value of one or more sample values ​​(pTemp[x]), and inSize corresponds to a number of components of p[x].

5. obtaining a prediction signal based on the predicted samples of the block; encoding a prediction residual of said block to correct said prediction signal; The method of claim 1 , further comprising encoding the block by:

6. obtaining a prediction signal based on the predicted samples of the block; decoding a prediction residual for said block; correcting the prediction signal using the prediction residual; The method of claim 1 , further comprising: decoding the block by:

7. An apparatus for predicting a block of a picture, comprising: a non-transitory computer readable medium; At least one processor configured to execute instructions from a non-transitory computer-readable medium to perform the method of any one of claims 1 to 6; An apparatus comprising:

8. A computer program configured, when executed by at least one processor, to cause said at least one processor to perform the method according to any one of claims 1 to 6.

9. A computer readable storage medium storing a computer program having a program code for performing the method according to any one of claims 1 to 6, when the computer program is executed on a computer.

Images