Block-based prediction

Abstract

To provide a device and a method of executing an improved block based prediction mode for picture encoding by block unit.SOLUTION: A method of predicting a block 18 of a picture comprises: acquiring a set 102 of one or more sample values from a plurality of adjacent samples 17a and 17c arranged along a boundary of the block 18; determining an input value from the one or more sample values on the basis of an instruction of a size of the block; determining an intra prediction sample of a matrix base by applying a predetermined prediction matrix to the determined input value; and predicting samples 104, 118', and 118" of the block 18 on the basis of the determined intra prediction sample of the matrix base.SELECTED DRAWING: Figure 7.2

Description

[Technical Field]

[0001] The present application relates to the field of block-based prediction. The present invention relates to an advantageous method for [Background technology]

[0002] Today, there are different block-based intra and inter prediction modes. Samples adjacent to the block to be predicted or samples obtained from other pictures are used. , a sample base on which matrix multiplication can be performed to determine the prediction signal for the block to be predicted. A curve can be formed.

[0003] Matrix multiplication should preferably be performed in integer arithmetic and should be implemented using some kind of machine learning-based algorithm. The matrix derived by the training algorithm should be used in the matrix multiplication. Summary of the Invention [Problem to be solved by the invention]

[0004] However, such training algorithms typically use rows given in floating-point precision. columns only. Therefore, matrix multiplication can be well approximated using these integer operations. Specifying integer arithmetic to be more efficient and / or achieving improved computational efficiency; Challenges are faced in making forecasts more effective in terms of implementation and / or implementation. [Means for solving the problem]

[0005] This is achieved by the subject matter of the independent claims of the present application. Further embodiments of the invention are defined by the subject matter of the dependent claims of the present application.

[0006] According to a first aspect of the present invention, the inventors of the present application have proposed a method for predicting a time domain signal by an encoder or a decoder. One problem encountered when trying to determine the predicted vector for a given block is We have recognized the possibility of not using integer arithmetic to calculate the torque. According to this aspect, this difficulty is solved by using a matrix vector to calculate a prediction vector. The sample vector is then multiplied by a given by deriving a further vector that is mapped by an invertible linear transformation of Alternatively, to calculate the predicted vector, a further vector and a given prediction matrix are used. Further vectors can be calculated using integer arithmetic and / or Or that the samples of a given block are predicted by a device using fixed-point arithmetic. This is because the components of the sample vector are correlated, by, for example, integer matrices and / or matrices with fixed point values ​​and / or predictors A matrix with a small estimation error can be used as the given prediction matrix. To obtain a further vector with the tri, a predetermined reversible linear transformation is used. It is based on the idea that it is possible.

[0007] Therefore, according to a first aspect of the present application, a plurality of reference samples is used to estimate the quality of a picture. The apparatus for predicting a given block derives a sample value vector from a plurality of reference samples. The reference sample is, for example, a predetermined block in intra prediction. It is a sample of a neighboring picture in the same frame, or a sample of another picture in inter prediction. According to an embodiment, for example, a sample vector having a reduced number of values ​​is obtained. By averaging, the reference samples can be reduced. The apparatus converts the sampled vector into a vector of matrices by a predetermined reversible linear transformation. To obtain the predicted vector, a further vector is derived and further vectors are used. and calculating a matrix-vector product between the vector and a predetermined prediction matrix, and based on a further vector, The prediction of a block of samples is done by performing a sample It is possible to represent an integer approximation of the direct matrix-vector product between a vector of values ​​and a matrix.

[0008] A direct matrix-vector product between a sample vector and a matrix is ​​a further vector and a second The second matrix and / or The matrix is, for example, a machine learning prediction matrix. According to an embodiment, the second matrix is ​​a predetermined The second matrix may be based on a predictor matrix and an integer matrix. The second matrix may be based on, for example, a predetermined predictor matrix. and an integer matrix. In other words, the second vector between a further vector and a second matrix The matrix vector product is the matrix vector product between the further vector and the predetermined predictor matrix, and It can be represented by a further matrix-vector product between an integer matrix and a further vector. An integer matrix is, for example, a matrix in which a predetermined column i0 is made up of 1s and columns i≠i0 are made up of 0s. Therefore, a good integer approximation of the first and / or second matrix-vector product and / or A good fixed-point value approximation can be achieved by the device. The torque mainly contains small values ​​and the possibility of approximating the first and / or second matrix-vector product is A given prediction matrix can be quantized to yield a small effect of possible quantization error. It is based on the idea that the matrix is ​​a quantized matrix or has already been quantized.

[0009] According to an embodiment, the reversible linear transformation of multiplying a predetermined prediction vector by the sum of an integer matrix is: The integer matrix can correspond to a quantized version of the machine learning prediction matrix. It is a matrix where every column i0 is 1 and every column i≠i0 is 0.

[0010] According to an embodiment, the reversible linear transformation is such that a predetermined component of the further vector becomes a and a predetermined Each of the other components of the further vector, except for the component of is defined to be equal to the component minus a, where a is a given value. Further vectors of small values ​​can be implemented, allowing for quantization of a given prediction matrix. This results in a small effect of quantization error in the predicted samples of a given block. This further vector allows integer and / or fixed-point arithmetic to be used to calculate the desired It is possible to predict the samples of a block of .

[0011] According to an embodiment, the predetermined value is an arithmetic mean or a weighted mean of the components of the sample value vector. which average, default value, picture is signaled in the encoded data stream. The value to be sampled is one of the components of the sample vector that corresponds to the given component. The sample vector may be, for example, a vector of values ​​obtained by or from a number of reference samples. The group of reference samples is composed of the average of the group of reference samples. , for example, includes at least two reference samples, preferably adjacent reference samples.

[0012] The predetermined value may be, for example, a part of the components of the sample value vector (for example, at least two components element) or the arithmetic or weighted average of all components of a sampled vector. is that the components of the sample vector are correlated, i.e., the values ​​of the components may be similar. and / or at least some of the components may have equal values, whereby , the component of the further vector is not equal to the predetermined component of the further vector, i.e., i Component i with ≠ i0 (where i0 represents a given component) is probably the corresponding Therefore, the further Vectors can be implemented.

[0013] The predetermined value can be a default value, and the default value can be, for example, Selected from a list or the same for all block sizes, prediction modes etc. The components of the default value list are different block sizes, prediction modes, and sample values. It can be related to the vector size, the average value of the sample vector, etc. Thus, for example, depending on a given block, i.e., associated with a given block, Depending on the decoding or encoding settings, the optimized default value is set by the device. is selected from the list.

[0014] Alternatively, the predetermined value may be signaled in the data stream in which the picture is encoded. In this case, for example, the encoding device determines the predetermined value. The decision can be based on the same considerations as discussed above in the context of default values.

[0015] The components of the further vector are not equal to the predetermined components of the further vector, i.e. Component i, where i≠i0 (i0 represents a predetermined component), is, for example, a default value or a data set. The value signaled in the stream is used as the predetermined value to find the corresponding sample value vector. has a smaller absolute value than the corresponding component.

[0016] According to an embodiment, the predetermined value is a component of the sample value vector corresponding to the predetermined component. In other words, the value of the component of the sample value vector corresponding to a given component can be reversibly It does not change when a linear transformation is applied. The value of an element of the pull value vector is, for example, equal to the value of a predetermined element of the further vector.

[0017] The predetermined component may be, for example, a default value, as described above with respect to the predetermined value. It is clear that a given component can be selected by an alternative procedure. The predetermined component may be selected, for example, as a predetermined value. According to an embodiment, the predetermined component may be The value of the corresponding component of the sample vector is equal to the average of the values ​​of the sample vector. or selected to have only marginal deviations from the mean of the values ​​of the sampled vector. .

[0018] According to an embodiment, in a column of a predetermined prediction matrix corresponding to a predetermined component of the further vector The matrix elements of a given prediction matrix are all 0. The system leaves a column, i.e., a column of zeros. The reduced prediction matrix resulting from the given prediction matrix and the remaining components are Therefore, we calculate the matrix-vector product between a further vector and a further vector resulting from the further vector. By performing the multiplication by calculating the matrix vector product, i.e., It is configured to calculate a matrix-vector product between the vector and a predetermined prediction matrix. , a predetermined component of the further vector is set to a predetermined value, and the values ​​of the sample value vector are correlated. If the predetermined value is found to be accurate or close to the sample value in the predicted signal for the given block, Therefore, the prediction of the samples of a given block is optionally further A given prediction matrix multiplied by a vector of is based on the reduced prediction matrix and an integer matrix, whose column i0 corresponds to a given component, The other columns i≠i0 are all 0, multiplied by a further vector. In other words, machine learning predictions transformed by, for example, the inverse of a predetermined reversible linear transformation. The matrix is ​​a predetermined predictor matrix, or rather a reduced predictor matrix, based on a further vector and an integer Therefore, only the prediction matrix is ​​quantized and the machine learning prediction We should obtain integer approximations of the measurement matrix and / or the transformed machine learning prediction matrix, This means that if no other vectors contain the specified component, and all other components are in the machine learning prediction matrix and / or the resulting quantization of the transformed machine learning prediction matrix is much smaller than the corresponding component of the sample vector, allowing for a small effect of errors It is advantageous to have absolute values. Furthermore, the reduced prediction matrix and the further vectors By using this, fewer multiplications need to be performed to obtain the prediction vector, reducing complexity. Optionally, when predicting samples of a given block, A vector whose components are all the predetermined value a can be added to the predicted vector. A vector is multiplied by a matrix-vector product between an integer matrix and a further matrix, as described above. can be obtained by

[0019] According to an embodiment, in a column of a predetermined prediction matrix corresponding to a predetermined component of the further vector The matrix resulting from summing each matrix element of a given predictor matrix by 1 times a given reversible linear transformation corresponds to a quantized version of the machine learning prediction matrix. Summing each matrix element of a given predictor matrix in the corresponding column of the given predictor matrix with 1 is, e.g. For example, it represents a transformed machine learning prediction matrix. A transformed machine learning prediction matrix is, for example, Represents the machine learning prediction matrix transformed by the inverse of a given reversible linear transformation. The sum is an integer matrix and a given predictor matrix whose columns correspond to given components. The column i0 of consists of ones, and all other columns i≠i0 are zeros.

[0020] According to an embodiment, the device uses the prediction parameters to represent a predetermined prediction matrix, and further The components of the vectors and the prediction parameters and the intermediate results resulting from them are multiplied and and performing an addition to calculate a matrix-vector product. The absolute value of the meter can be expressed by an n-bit fixed-point number, where n is 14 or less. , or 10 or less, or 8 or less. In other words, the prediction parameters are Multiply the elements of a matrix-vector product, such as a vector, a given predictor matrix and / or predictor vector, etc. The multiplication and addition operations are used to obtain, for example, a predetermined prediction matrix, a prediction The fixed-point format of the vector and / or predicted samples for a given block is taken. can be obtained.

[0021] An embodiment of the present invention may be implemented by combining the embodiments described herein to obtain a predicted signal. To predict a given block of a picture using any of a number of reference samples, Furthermore, the apparatus relates to an apparatus for encoding a picture, the apparatus comprising: an entry configured to encode a prediction residual of a given block to correct the measured signal; For the prediction of a given block to obtain a prediction signal, a tropy coder is provided. For example, the positioning may be such that a sample value vector is formed from a plurality of reference samples, and the sample value vector is From this, a further vector is generated to which the sample value vector is mapped by a predetermined reversible linear transformation. To derive the vector and obtain the predicted vector, we use a further vector and a given prediction matrix. and calculates the matrix vector product between the samples of a given block based on the prediction vector. It is configured to measure

[0022] An embodiment of the present invention may be implemented by combining the embodiments described herein to obtain a predicted signal. To predict a given block of a picture using any of a number of reference samples, Furthermore, the apparatus relates to an apparatus for decoding a picture, the apparatus comprising: an entropy decoder configured to decode a prediction residual of the block; and a prediction corrector configured to correct the prediction signal using the prediction signal. For prediction of a given block for a given time, the device may, for example, select a sample from a plurality of reference samples. and form a vector of sample values ​​from the vector of sample values, such that the vector of sample values ​​is a predetermined reversible linear vector. To derive a further vector that is mapped by the transformation to obtain a predicted vector. , a matrix-vector product is calculated between the further vector and the predetermined prediction matrix, and the predicted vector The method is configured to predict the samples of a given block based on

[0023] An embodiment of the present invention uses multiple reference samples to estimate a given block of a picture. 1. A method for predicting a value of a sample, comprising forming a sample value vector from a plurality of reference samples. and from the sampled vector, the sampled vector is transformed into a matrix by a predetermined reversible linear transformation. To obtain the predicted vector, a further vector is derived. calculating a matrix-vector product between a vector consisting of the above and a predetermined prediction matrix; and predicting samples of a given block based on the prediction.

[0024] An embodiment of the present invention is a method for coding a picture, comprising the steps of: obtaining a prediction signal; To do this, a given block of a picture is estimated using a number of reference samples according to the method described above. and inputting the prediction residual of a given block to correct the prediction signal. and encoding the image data in a tropy-encoded form.

[0025] An embodiment of the present invention is a method for decoding a picture, comprising: To do this, a given pixel of the picture is calculated using a number of reference samples according to one of the methods described above. predicting a block and entropy decoding the prediction residual for the given block; and correcting the prediction signal using the prediction residual. Embodiments of the present invention use the methods described herein for encoding pictures. The present invention relates to a data stream having pictures coded by the above method.

[0026] When implemented on a computer, embodiments of the present invention may be implemented in a manner similar to that described herein. A computer program having a program code for performing any of the methods of the present invention. Regarding.

[0027] The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the present invention will be described with reference to the following drawings, in which: It will be explained. [Brief explanation of the drawings]

[0028] [Figure 1] 1 illustrates an embodiment of encoding into a data stream. [Figure 2] 1 shows an embodiment of an encoder. [Figure 3] 1 illustrates an embodiment of picture reconstruction. [Figure 4] 1 shows an embodiment of a decoder. [Figure 5] 1 shows a schematic diagram of prediction of a block for encoding and / or decoding according to an embodiment; [Figure 6] 1 illustrates a matrix operation for prediction of a block for encoding and / or decoding according to an embodiment. [Figure 7.1] 10 illustrates prediction of a block with a reduced sample value vector, according to an embodiment. [Figure 7.2] 10 illustrates block prediction using sample interpolation, according to an embodiment. [Figure 7.3] 10 illustrates prediction of a block with a reduced sample value vector, in which only some boundary samples are averaged, according to an embodiment. [Figure 7.4] 10 illustrates prediction of a block with a reduced sample value vector in which groups of four boundary samples are averaged, according to an embodiment. [Figure 8]1 shows a schematic diagram of an apparatus for predicting a block according to an embodiment; [Figure 9] 1 illustrates a matrix operation performed by an apparatus according to an embodiment. [Figure 10a] 1 illustrates detailed matrix operations performed by an apparatus according to an embodiment. [Figures 10b-10c] 1 illustrates detailed matrix operations performed by an apparatus according to an embodiment. [Figure 11] 10 illustrates detailed matrix operations performed by the device using offset and scaling parameters, according to an embodiment. [Figure 12] 10 illustrates detailed matrix operations performed by an apparatus using offset and scaling parameters according to different embodiments. [Figure 13] 1 shows a block diagram of a method for predicting a given block according to an embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0029] Equal or equivalent elements or elements with equal or equivalent functions may be shown in different figures. Even if they occur, they are indicated in the following description by the same or equivalent reference numerals. .

[0030] In the following description, numerous details are set forth to provide a more thorough explanation of embodiments of the present invention. However, embodiments of the invention may be practiced without these specific details. It will be apparent to one skilled in the art that other embodiments of the present invention may be used. In order to avoid obscuring the details, well-known structures and devices are shown in block diagram form, rather than in detail. Furthermore, the features of the different embodiments described below are the same unless otherwise specified. , can be combined with each other. 1. Introduction

[0031] Below, different examples, embodiments and aspects of the present invention are described. , and at least some of the aspects are, inter alia, for video coding; and / or block transformations using, for example, linear or affine transformations with adjacent sample reduction. for performing network-based predictions and / or for example video applications and Video distribution (e.g., broadcasting, streaming) for / or virtual reality applications , file playback, etc.).

[0032] Additionally, examples, embodiments, and aspects may be implemented using High Efficiency Video Coding (HEVC) or Further embodiments, examples and aspects are set forth in the appended claims. Defined by a range.

[0033] Any embodiments, examples, and aspects defined by the claims are described in the following sections. It should be noted that the invention may be supplemented by any of the details (features and functions) described in the specification. I want to be.

[0034] Also, the embodiments, examples, and aspects described in the following sections may be used individually. may be supplemented by any feature of another chapter or by any feature included in the claims. It can also be done.

[0035] Additionally, the individual examples, embodiments, and aspects described herein may be used individually or in combination. It should be noted that the above examples, embodiments and and each of said individual aspects without adding details to another one of the aspects. It can be done. This disclosure explicitly or implicitly describes features of decoding and / or encoding systems and / or methods. Note also the implicit explanation.

[0036] Additionally, the features and functions disclosed herein in connection with the method may be used in the apparatus. Furthermore, any of the features and functions disclosed herein with respect to the device may be used. The functions can also be used in the corresponding methods. The method described may be supplemented by any of the features and functions described with respect to the apparatus. This can be done.

[0037] Also, any of the features and functionality described herein may be implemented in the "Implementation Alternatives" section. as described in the section, in hardware or software, or in hardware and software. The method may be implemented using a combination of software.

[0038] Additionally, any of the features listed in brackets ("(...)" or "[...]") may be considered optional in some examples, embodiments, or aspects. can.

[0039] 2 Encoder and decoder Below we will discuss how to achieve more effective compression when using block-based prediction. We will now discuss various examples that can be useful. Some examples are intra prediction modes High compression efficiency is achieved by using a set of heuristics. It may be in addition to or offered exclusively as other intra prediction modes designed for the And even other examples may utilize both of the disciplines described here. While these embodiments are not identical, intra prediction is performed using a reference frame instead of a frame in another picture. The prediction can be converted to inter prediction by using the reference sample.

[0040] To facilitate understanding of the following examples of this application, the description will be based on the examples outlined thereafter in this application. We start by presenting possible coders and decoders that can be adapted to it. FIG. 1 shows an apparatus for block-by-block encoding of a picture 10 into a data stream 12. The apparatus is indicated using the reference numeral 14 and is a still picture encoder or In other words, picture 10 includes picture 10 If the encoder 14 is configured to encode the video 16 into the data stream 12, or encoder 14 encodes picture 10 exclusively into data stream 12. If so, it can be the current picture from the video 16.

[0041] As previously mentioned, the encoder 14 performs the encoding in a block-by-block manner or on a block basis. To do this, the encoder 14 subdivides the picture 10 into blocks and encodes them. The unit of the decoder 14 encodes the picture 10 into a data stream 12. Examples of possible subdivisions of the blocks 18 are given in more detail below. The division may be into blocks of a fixed size, such as an array of blocks arranged in rows and columns. or from the entire picture area of ​​Picture 10 or from a pre-partition of Picture 10 of hierarchical multitree subdivision that initiates multitree subdivision into an array of treeblocks. Due to different block sizes used, you may end up with block 18. is treated as excluding other possible ways of subdividing the picture 10 into blocks 18. This should not be allowed.

[0042] Additionally, the encoder 14 predictively encodes the pictures 10 into a data stream 12. For a particular block 18, this is the encoder 14 determines the prediction signal of block 18 and calculates the prediction residual, i.e., the predicted signal The prediction error, which deviates from the actual picture content in the image, is coded into the data stream 12. This means that

[0043] The encoder 14 may use different prediction modes to derive a prediction signal for a particular block 18. The prediction modes that are important in the following example are the internal is an input image spatially predicted from samples of adjacent, already coded pictures 10. The coding of the picture 10 into the data stream 12 and therefore the corresponding The decoding procedure can be based on a specific coding order 20 defined between the blocks 18. For example, encoding order 20 traverses each row from left to right, going from top to bottom. The blocks 18 can be traversed in a raster scan order, such as by unit. In the case of tree-based subdivision, the order of raster traversal may be applied within each hierarchical level. depth-first traversal order can be applied, i.e., Leaf nodes in a block of a tree have the same parent block according to coding order 20. Depending on the coding order 20, a block may precede a block at the same hierarchical level. The adjacent, already coded samples of block 18 are typically located on one or more sides of block 18. In the example presented here, for example, adjacent blocks of block 18 The already coded samples are located at the top and left of the block 18 .

[0044] Encoder 14 may not only support intra-prediction modes. For example, if encoder 14 is a video encoder, encoder 14 also includes block 18 Inter prediction mode in which ∇ ... Such inter prediction modes can support the prediction signal of block 18. Such blocks indicate the relative spatial offset of the parts from which the code is derived as a copy. 18, the motion compensation prediction mode in which the motion vectors are signaled. Additionally or alternatively, inter-prediction when the encoder 14 is a multi-view encoder prediction mode, or non-prediction mode in which the interior of the block 18 is coded as is, i.e., without prediction. Other non-intra prediction modes may also be available, such as the .intg.prediction mode.

[0045] Before starting the description of this application by focusing on intra-prediction modes, it is important to consider the possible block A more specific example of a block-based encoder, i.e., The possibilities of the encoder 14 are shown in FIGS. 1 and 2, which present two corresponding examples of decoders that conform to the encoder 14. This section describes possible implementations.

[0046] FIG. 2 shows a possible implementation of the encoder 14 of FIG. 1, i.e., an encoder that encodes the prediction residual. Although the example shown is one that is configured to use transcoding to Therefore, the present application is not limited to such predictive residual coding. The encoder 14 receives the inbound signal, i.e., the picture 10, or the current The corresponding prediction signal 24 is subtracted from the block 18 and subsequently coded by a predictive residual coder 28. A subtractor configured to obtain a prediction residual signal 26 that is encoded into the data stream 12. The predictive residual encoder 28 comprises a lossy encoding stage 28a and a lossless encoding stage 28b. The lossy stage 28a receives the prediction residual signal 26 and , a quantizer 30 for quantizing the samples of the prediction residual signal 26. , this example uses transform coding of the prediction residual signal 26 and therefore a lossy coding stage 28a is the result of the quantization performed by the quantizer 30 on the transformed coefficients representing the residual signal 26. The subtractor 22 and the quantizer 24 are used to transform such a spectrally decomposed prediction residual 26 into a The transform stage 32 is connected between the transformer 30 and the DCT stage 32. The transforms include DCT, DST, FF, T, Hadamard transform, etc. Then, the transformed and quantized prediction residual signal The signal 34 is an encoder for entropy coding the quantized prediction residual signal 34 into the data stream 12. The lossless encoding is performed by a lossless encoding stage 28b, which is an entropy coder. 14 is a prediction signal obtained from the transformed and quantized prediction residual signal 34 in a manner that can be used by the decoder. A prediction residual signal reconstruction switch is connected to the output of the quantizer 30 so as to reconstruct the prediction residual signal. It further comprises a stage 36. That is, it is the quantizer 30 that takes into account the coding loss. To this end, the prediction residual reconstruction stage 36 performs the inverse of the quantization of the quantizer 30. followed by a spectral quantizer 38, such as the inverse of any of the specific transform examples mentioned above. an inverse transformer that performs an inverse transform to the transform performed by the transformer 32, such as the inverse of the decomposition 40. The encoder 14 outputs the reconstructed signal, i.e., the reconstructed samples, as The reconstructed prediction residual signal and the prediction signal output by the inverse transformer 40 are 24. The output of this adder 42 is fed to a predictor 44 in the encoder 14. The predictor 44 determines the predicted signal 24 based thereon. It is the predictor 44 that supports all prediction modes. If the encoder 14 is a video encoder, the encoder 14 also generates the inter-prediction blocks after filtering. The fully reconstructed picture that forms the reference picture for the predictor 44 is then filtered. 10 shows that an in-loop filter 46 for filtering may be provided.

[0047] As already mentioned above, the encoder 14 operates on a block basis. The target block base is a subdivision of the picture 10 into blocks, For each block, the set or blocks supported by the predictor 44 or the encoder 14, respectively, are An intra prediction mode is selected from a plurality of intra prediction modes, and the selected intra prediction mode is However, other types of subdivision of the picture 10 are possible. For example, if picture 10 is inter-coded, The above determination of whether a block is intra- or intra-coded can be made at a granularity or block level. It can be done in units of blocks that deviate from 18. For example, The mode decision is made by subdividing the picture 10 and subdividing each coding block into prediction blocks. Intra prediction can be used to perform the coding at the level of the coding block being processed. The prediction blocks having the coding blocks determined as the intra prediction mode are respectively Therefore, for each of these prediction blocks, the supported It is determined which of the available intra prediction modes should be used for each prediction block. These predicted blocks form the block of interest 18. Predicted blocks within related coding blocks may be treated differently by predictor 44. They determine the motion vector and the reference picture pointed to by the motion vector. The prediction signal for this block is then obtained from the reference picture by copying it from its location in the image. Another block subdivision is the transformation by the transformer 32 and the inverse transformer 40. It concerns the subdivision into transformation blocks in the units where the transformation is performed. The transformed blocks are , for example, can be the result of further subdivision of the coding block. Therefore, the examples provided here should not be treated as limiting and other examples may exist. Just for the sake of completeness, the subdivision into coding blocks may be implemented as a multi-tree subdivision. Similarly, the prediction block and / or the transform block may be multi-partitioned. The coding block is obtained by further subdividing it using a tree subdivision. Note that it is possible to

[0048] A decoder 54 or device for block-wise decoding compatible with the encoder 14 of FIG. 1 is shown in FIG. This decoder 54 does the opposite of the encoder 14, i.e. The picture 10 is decoded block by block from the data stream 12, and for this purpose, a plurality of The decoder 54 may include, for example, a residual provider 156. All other possibilities discussed above with respect to FIG. 1 are also valid for decoder 54. Thus, the decoder 54 can be a still picture decoder or a video decoder. , all prediction modes and predictability are supported by the decoder 54. The differences between the encoder 14 and the decoder 54 are primarily due to the fact that the encoder 14 has different encoding speeds and / or or to minimize some cost function that may depend on the coding distortion, etc. The main difference lies in the fact that the coding decisions are selected or chosen according to some optimization. One of the encoding options or encoding parameters is not available or supported. The intra prediction mode to be used for the current block 18 from among the available intra prediction modes. The selected intra prediction mode can then be used to select the data stream. The decoder 54 receives the signal from the encoder 14 of the current block 18 in the program 12. , this signaling in the data stream 12 of block 18 is used to redo the selection. Similarly, the subdivision of picture 10 into blocks 18 is subject to optimization within encoder 14. and corresponding subdivision information can be conveyed in the data stream 12. , the decoder 54 restores the subdivision of the picture 10 into blocks 18 based on the subdivision information. To summarize the above, the decoder 54 may be a predictive decoder operating on a block basis. In addition to the intra prediction mode, the decoder 54 may also be configured to, for example, If so, other prediction modes such as inter-prediction modes can be supported. In this case, the decoder 54 may also use the encoding order 20 described with respect to FIG. This coding order 20 is followed by both the encoder 14 and the decoder 54, so that the same Neighboring samples are available to the current block 18 at both the encoder 14 and the decoder 54. Therefore, to avoid unnecessary repetition, the operating mode of the encoder 14 is The description is given in terms of the picture, e.g., as far as prediction is concerned and as far as the coding of the prediction residual is concerned. As far as the subdivision of the channel 10 into blocks is concerned, this must also be applied to the decoder 54. The difference is that the encoder 14 can optimize several encoding options or codes. The parameters and signals are selected within the data stream 12 or The main reason is that the data is inserted into the memory 12, and these are used for re-prediction, such as re-division. The signal is derived from the data stream 12 by decoder 54 .

[0049] FIG. 4 shows a possible implementation of the decoder 54 of FIG. 3, i.e., the decoder 54 of FIG. 1, as shown in FIG. 4. Many elements of encoder 54 in FIG. To denote these elements, we use the A The same reference numerals with posttrophies are used in FIG. 4. In particular, adder 42', optional The in-loop filter 46' and predictor 44' are similar to those in the encoder of FIG. The reconstructed sum is applied to adder 42'. That is, the dequantized and retransformed prediction residual signal is fed to the entropy encoder 28b. The sequence of the entropy decoder 56 that reverses the entropy coding, followed by the coding side The residual signal reconstruction stage is composed of an inverse quantizer 38' and an inverse transformer 40' in the same manner as in the The output of the decoder is a reconstruction of picture 10. The reconstruction of the filter 10 can be performed either directly at the output of the adder 42' or by using an in-loop filter 44. 6' output. To improve the picture quality, To subject the reconstruction of Cha10 to some post-filtering, A filter can be placed at the output of the decoder, but this option is not shown in Figure 4. Not yet.

[0050] Again, with respect to FIG. 4, the explanation given above with respect to FIG. 2 is that the encoder 4, except that it only performs the relevant decisions regarding the encoding options. However, all the details regarding block subdivision, prediction, inverse quantization, and retransformation are All the explanations are also valid for the decoder 54 of FIG.

[0051] 3. ALWIP (Affine Linear Weighted Intra Predictor) Some non-limiting examples of ALWIP include: Even though ALWIP is not necessarily required to achieve this, it is described herein.

[0052] This application is applicable, inter alia, to video codecs such as HEVC or a successor to HEVC. Improved block-based prediction for block-based picture coding, such that The prediction mode may be an intra prediction mode, but the In particular, the concepts described herein are applicable to interleaved images where the reference samples are part of another picture. It can also be converted into a prediction mode. Block-based prediction schemes enable efficient implementations, including hardware-friendly implementations. Mindfulness is required. This object is achieved by the subject matter of the independent claims of the present application.

[0053] Intra prediction modes are widely used in picture and video coding. In coding, intra prediction modes are used in combination with other inter prediction modes such as motion compensated prediction modes. In intra prediction mode, the current block is predicted by the neighboring samples. that is, it has already been coded as far as the encoder is concerned and it has already been decoded as far as the decoder is concerned. The neighboring sample values ​​are extrapolated to the current block. The prediction signal of the current block is formed, and the prediction residual is The better the predicted signal, the smaller the prediction residual will be, and therefore Fewer bits are needed to code the prediction residual.

[0054] To be effective, a block-based picture coding environment for intra prediction requires a To create an effective framework, several aspects need to be considered. For example, the more intra-prediction modes a codec supports, the more options the decoder has to choose from. On the other hand, the side information rate for signaling is high. The set of prediction modes provides a good prediction signal, i.e., a prediction signal with a low prediction residual. It is necessary to be able to provide

[0055] In the following, as a comparative embodiment or a basic example, a picture is extracted from a data stream as a block. A device (encoder or decoder) for decoding a picture in a block-by-block manner, The intra prediction signal for the current block is the first of the samples adjacent to the current block. The template is then used as an affine linear weighted predictor (ALWIP). at least one intra-prediction mode determined by applying a linear predictor to the An apparatus for supporting the method is disclosed.

[0056] The apparatus has the following characteristics (which, when executed by, for example, a processor, a non-transitory memory device that stores instructions that cause the processor to perform a method and / or operate as a device; at least one of the following: You can also have one.

[0057] 3.1 Predictors can be complementary to other predictors. Intra prediction modes that may form the subject of implementation improvements that are further described below are , can be complementary to other intra-prediction modes of the codec. , they are based on the DC prediction model defined in the HEVC codec and the JEM reference software. It can complement the planar prediction mode, the angular prediction mode, or the holographic prediction mode. These types of intra prediction modes are referred to as conventional intra prediction modes. , for a given block of intra modes, intra prediction supported by the device A flag must be parsed by the decoder to indicate whether one of the modes should be used. There is a need.

[0058] 3.2 Two or more proposed prediction modes A device may contain more than one ALWIP mode, thus The decoder knows that one of the supported ALWIP modes should be used. If so, the decoder determines which of the ALWIP modes supported by the device is being used. The additional information that indicates whether the

[0059] Signaling of supported modes is done by encoding some ALWIP modes in a way that other ALWIP modes are not. This can have the property that it may require fewer bins than the WIP mode. Which of the modes requires fewer bins and which requires more bins? The bitstream to be decoded can be extracted from an already decoded bitstream or It may rely on information that may have been previously fixed.

[0060] 4. Some Aspects FIG. 2 shows a decoder 54 for decoding pictures from the data stream 12. The decoder 54 may be configured to decode a predetermined block 18 of a picture. In particular, the predictor 44 uses a linear or affine linear transformation [e.g., ALWIP]. A set of P adjacent samples adjacent to a given block 18 is then calculated as the sample of the given block. The method can be configured to map the Q predictions to a set of Q predictions.

[0061] As shown in FIG. 5, a given block 18 is composed of Q predicted values ​​(which are the sum of the Q predicted values ​​at the end of the calculation). If block 18 has M rows and N columns, then Q = M N. The Q value of block 18 can be calculated in the spatial domain (e.g., pixel) or the transform domain (e.g., DC T, discrete wavelet transform, etc.). The Q value of block 18 is generally Prediction based on P values ​​obtained from neighboring blocks 17a to 17c adjacent to block 18 The P values ​​of the adjacent blocks 17a to 17c can be determined by the P value of the block 18. The P values ​​of the adjacent blocks 17a to 17c may be in different positions (for example, adjacent). The P value indicates that they are (in some cases, 17'b In order to distinguish it from blocks that are part of the is shown as:

[0062] As shown in Figure 6, to perform prediction, a first vector 1 with P entries is used. 7P (each entry is associated with a specific position in the adjacent parts 17'a to 17'c) , a second vector 18Q having Q entries (each entry is a specific 17M (each row is associated with a location in block 18) Each row is associated with a specific location within the adjacent sections 17'a-17'c. Therefore, the mapping matrix 17M can be calculated by the following formula: According to the mode, the P values ​​of the adjacent portions 17'a to 17'c are predicted to the value of the block 18. Therefore, the entries in the mapping matrix 17M can be understood as weighting factors. In the following description, the symbols 17a to 17c are used instead of 17'a to 17'c. to refer to the adjacent part of the boundary.

[0063] In the art, there are several modes such as DC mode, planar mode, and 65-way prediction mode. There are many known conventional modes, for example 67 known modes.

[0064] However, we will use a different mode called linear or affine-linear transformation. Note that it is also possible to use a linear or affine-linear transformation with P·Q overlapping The weighting coefficients are at least 1 / 4P·Q, of which at least 1 / 4P·Q are non-zero. , for each of the Q predicted values, a set of P weighting factors for each predicted value is The sequences are arranged one above the other in raster scan order between the samples of a given block. When coupled together, they form an envelope that is omnidirectionally nonlinear.

[0065] P positions of adjacent values ​​17'a to 17'c (template), adjacent sample 17 Map the Q positions of 'a~17'c and the P*Q weight coefficient values ​​of matrix 17M The plane is an example of the envelope of the series for DC conversion ( The envelope is obviously planar and therefore linear or affine. Another example is the following angular mode emitter: The envelope is excluded from the ALWIP definition and, in layman's terms, For example, it looks like a hill going diagonally from top to bottom along the direction in the P / Q plane. The 65 and 66 directional prediction modes have different envelopes, but this is because the 65 directional prediction modes are different in at least one direction, i.e. That is, in all directions of the DC, for example, and in the hill direction of the angular mode, for example, It is a shape.

[0066] Conversely, the envelope of a linear or affine transformation is not linear in all directions. This transformation may be optimal for performing block 18 predictions in some situations. It is understood that at least one-quarter of the weighting coefficients are different from 0 (i.e., P* Note that it is preferable that at least 25% of the Q weighting factors are different from 0.

[0067] The weighting factors may be independent of each other according to any regular mapping rule. Thus, the matrix 17M is such that the values ​​of its entries have no obvious discernible relationship. For example, the weighting factors may be calculated by any analytical or differential function. It cannot be described in any way.

[0068] In the example, the ALWIP transformation involves the first series of weighting factors associated with each predicted value and the maximum value of the cross-correlation between the weighting coefficients of the second series associated with the respective predicted values ​​other than the predicted value Although the average of the two series, or an inverted version of the latter series, leads to higher maximum values, Regardless, a predetermined threshold (e.g., 0.2 or 0.3 or 0.35 or 0.1, e.g. For example, the threshold may be lower than For example, for each row combination (i1,i2) of the ALWIP matrix 17M, the P value of the i1th row is The cross-correlation can be calculated by multiplying the P values ​​of the first and second rows. For correlation, the maximum value can be obtained. Thus, for the entire matrix 17M, The average (mean) can be obtained (i.e., the cross-correlation of all combinations). (The maximum values ​​of the correlations are averaged). The threshold is then set to, for example, 0.2 or 0.3 or 0. 35 or 0.1, for example, the threshold may range between 0.05 and 0.035.

[0069] P adjacent samples of blocks 17a to 17c are arranged on the boundary of a given block 18 (e.g. , 18c, 18a). For each of the Q predictors in the metric, a set of P weights associated with each predictor is The weighting coefficients are calculated as the weights are used to traverse a one-dimensional path in a given direction (e.g., left to right, top to bottom, etc.). may be ordered as follows: In examples, the ALWIP matrix 17M may be non-diagonal or non-block diagonal. ALWIP for predicting a 4x4 block 18 from four already predicted neighboring samples An example of a matrix 17M may be: { { 37,59,77,28}, { 32,92,85,25}, { 31,69,100,24}, { 33,36,106,29}, { 24,49,104,48}, { 24,21,94,59}, { 29,0,80,72}, { 35,2,66,84}, { 32,13,35,99}, { 39,11,34,103}, { 45,21,34,106}, { 51,24,40,105}, { 50,28,43,101}, { 56,32,49,101}, { 61,31,53,102}, { 61,32,54,100} }.

[0070] (where {37,59,77,28} is the first row. {32,92,85,25 } is the second row. {61,32,54,100} is the 16th row of matrix 17M. ) Matrix 17M has dimensions 16x4 and contains 64 weight coefficients (as a result of 16*4=64). This is because the matrix 17M has dimensions QxP, where Q=M*N , the number of samples in the prediction target block 18 (block 18 is a 4x4 block), and P is the number of samples in the prediction target block 18 (block 18 is a 4x4 block), and where M=4, N=4, Q=16(M*N = 4 * 4 = 16), and P = 4. The matrix is ​​non-diagonal and non-block It is diagonal and is not described by any particular rule.

[0071] As can be seen, less than a quarter of the weighting coefficients are 0 (1 out of 64 in the case of the matrix above). (One weighting factor is 0.) The envelope formed by these values ​​is Thus, when placed one level below it, it forms an envelope that is nonlinear in all directions.

[0072] Where the above description has been primarily described with reference to a decoder (e.g., decoder 54), However, the same may be performed in an encoder (eg, encoder 14).

[0073] In some examples, for each block size (in the set of block sizes), Intra prediction within the second set of intra prediction modes for each block size The ALWIP transformations in the measurement mode are different from each other. Additionally or alternatively, the block size Cardinality of a second set of intra prediction modes for block sizes in the set The densities can be matched, but the intra prediction models for different block sizes are associated linear transformations or affine transformations of the intra prediction modes in the second set of modes; Shape transformations can be made inconvertible to each other by scaling.

[0074] In some instances, ALWIP transformations have nothing in common with traditional transformations. (For example, an ALWIP transformation may be defined as Even if they are mapped via a transformation, they share nothing in common with their traditional counterparts. (It can also be stated as "there is no

[0075] In the example, the ALWIP mode is used for both the luma and chroma components, but not for the other In the example, the ALWIP mode is used for the luma component but not for the chroma component.

[0076] 5 Affine linear weighted intra prediction mode with faster encoder (e.g., test CE3 -1.2.1) 5.1 Description of the method or apparatus Affine Linear Weighted Intra Prediction (ALWIP) model tested in CE3-1.2.1 The code is identical to that used in JVET-L0199 under test CE3-2.2.2, with the following modifications: This could be the same as the one proposed in:

[0077] Multiple Reference Line (MRL) intra prediction, especially matching with coder estimation and signaling , i.e., MRL is not combined with ALWIP, and the transmission of MRL index is Restricted to ALWIP blocks.

[0078] Subsampling is mandatory for all blocks W×H≧32×32 (previously was optional for 32x32). Therefore, additional tests in the coder and the transmission of the sub-sampling flag has been removed.

[0079] ALWIP for 64×N and N×64 blocks (N≦32) is 32×N and 32×64, respectively. and Nx32 downsampling and applying the corresponding ALWIP mode. It has been added. Additionally, test CE3-1.2.1 includes the following encoder optimizations for ALWIP: :

[0080] Combined mode estimation: Conventional and ALWIP modes are combined for complete RD estimation Use a shared Hadamard candidate list, i.e., the ALWIP mode candidates are Hadamard They will be added to the same list as conventional (and MRL) mode candidates based on cost. EMT Intra Fast and PB Intra Fast are supported for the combined mode list and has additional optimizations to reduce the number of full RD checks. Only the MPMs of the left and top blocks available are used in the same way as in the conventional mode. Therefore, it is added to the list for ALWIP full RD estimation.

[0081] 5.2 Complexity Assessment Except for calculations that call for the discrete cosine transform, exam CE3-1.2.1 requires that the predicted signal Up to 12 multiplications per sample were required to generate them. A total of 136492 parameters of 0.273 Mbytes were required. This corresponds to the memory of the

[0082] 5.3 Experimental results The test was evaluated intravenously using VTM software version 3.0.1 (AI). ) and Random Access (RA) configurations, common test conditions JVET-J1010 The corresponding simulations were performed according to [2]. Intel Xeon cluster (E5 -2697A v4, AVX2 on, Turbo Boost off).

[0083] [Table 1]

[0084] [Table 2]

[0085] 5.4 Affine Linear Weighted Intra Prediction with Complexity Reduction (e.g., Test CE3-1. 2.2) The technology tested in CE2 is based on the "Affine Line" method described in JVET-L0199[1]. It is related to "intra-prediction" but has the following advantages in terms of memory requirements and computational complexity: To simplify this:

[0086] Prediction matrix covering all block shapes (e.g., S0, S1, S2, see below) The three different vectors are the sine wave vector (to be used as a reference) and the bias vector (to provide an offset value, for example). As a result, the number of parameters is 14400. is reduced to a 10-bit value, which is

number

[0087] The input and output sizes of the predictor are further reduced. Furthermore, the boundaries are decomposed via DCT. Instead of converting, averaging or downsampling is performed on the boundary samples. The prediction signal can be generated using linear interpolation instead of the inverse DCT. As a result, up to four multiplications per sample may be required to generate the predicted signal. .

[0088] 6. Example Here, we perform several predictions (e.g., as shown in Figure 6) using ALWIP prediction. This section explains how to do this. In principle, referring to FIG. 6, to obtain Q=M*N values ​​for the M×N block 18 to be predicted, To do this, we use a Q×P ALWIP prediction matrix of 17M Q*P samples and a P×1 neighbor vector of 1 7P should be multiplied by P samples. To obtain each of the Q=M*N values ​​in the ×N block 18, at least P=M +N values ​​must be multiplied.

[0089] These multiplications have a highly undesirable effect. The dimension P of the boundary vector 17P is Generally, the boundary samples (e.g., adjacent) to the M×N block 18 to be predicted are It depends on the number M+N of bins or pixels 17a, 17c of the block to be predicted. If the size of 8 is large, the number M+N of boundary pixels (17a, 17c) will be large accordingly. Therefore, the P×1 boundary vector 17P has dimensions P=M+N, and the Q×P ALWIP prediction row This means that the length of each row of the 17M column, and therefore the number of multiplications required, will be large (generally In other words, Q = M * N = W * H, where W (width) is another symbol for N and H (height). is another notation for M, and P is the boundary vector that is in only one row and / or one column of samples. Thus, when formed, P = M + N = H + W).

[0090] This problem is commonly encountered in microprocessor-based systems (or other digital processing This is exacerbated by the fact that in a multiplication system, multiplication is generally a power-consuming operation. A large number of multiplications performed on a large number of samples in a large number of blocks is generally undesirable. It can be assumed that this will not cause any waste of computational power. Therefore, it is possible to reduce the number of multiplications Q*P required to predict the M×N block 18. preferable.

[0091] By intelligently choosing easier-to-process operations instead of multiplication, each block to be predicted is It is possible to somehow reduce the computational power required for each intra prediction of block 18. It is understood that In particular, referring to Figures 7.1 to 7.4, the encoder or decoder:

[0092] A reduced sample value with fewer samples compared to multiple adjacent samples. To obtain a set, several adjacent samples (e.g., 17a, 17c) are scaled down. (e.g., in step 811), (e.g., averaging or downsampling) by g) and

[0093] The reduced set of sample values ​​is subjected to a linear or affine-linear transformation (e.g., obtaining a prediction of a given sample of a given block;

[0094] By using multiple adjacent samples (e.g., 17a, 17c), It is understood that certain blocks (eg, 18) can be predicted.

[0095] In some cases, the decoder or encoder may also interpolate a given sample and further samples of the given block based on the predictions of multiple adjacent samples. Therefore, an upsampling strategy can be obtained. This can be done.

[0096] In an example, several averages are performed on the samples of boundary 17 (e.g., in step 811) so as to reach a reduced set 102 of samples with a reduced number of samples (Figures 7.1 to 7.4). At least one of the samples of the reduced number of samples 102 may be an average of two samples of the original boundary samples or a selection of the original boundary samples. For example, if the original boundary has P = M + N samples, the reduced set of samples can have P < P, where at least one of M < M and N < N. Thus, the boundary vector 17P actually used for prediction (e.g., in step 812b) does not have P × 1 entries but has P < P, P × 1 entries. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have Q × P dimensions, and since at least one of M < M and N < N is at least P < P, the number of elements of the matrix is reduced to Q × P (or Q × P, see below). It is possible to perform several averages on the samples of boundary 17 so as to reach a reduced set 102 of samples with a reduced number of samples (e.g., in step 811). At least one of the samples of the reduced number of samples 102 may be an average of two samples of the original boundary samples or a selection of the original boundary samples. For example, if the original boundary has P = M + N samples, the reduced set of samples can have P < P, where at least one of M < M and N < N. Thus, the boundary vector 17P actually used for prediction (e.g., in step 812b) does not have P × 1 entries but has P < P, P × 1 entries. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have Q × P dimensions, and since at least one of M < M and N < N is at least P < P, the number of elements of the matrix is reduced to Q × P (or Q × P, see below). It is possible to perform several averages on the samples of boundary 17 so as to reach a reduced set 102 of samples with a reduced number of samples (e.g., in step 811). At least one of the samples of the reduced number of samples 102 may be an average of two samples of the original boundary samples or a selection of the original boundary samples. For example, if the original boundary has P = M + N samples, the reduced set of samples can have P < P, where at least one of M < M and N < N. Thus, the boundary vector 17P actually used for prediction (e.g., in step 812b) does not have P × 1 entries but has P < P, P × 1 entries. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have Q × P dimensions, and since at least one of M < M and N < N is at least P < P, the number of elements of the matrix is reduced to Q × P (or Q × P, see below). It is possible to perform several averages on the samples of boundary 17 so as to reach a reduced set 102 of samples with a reduced number of samples (e.g., in step 811). At least one of the samples of the reduced number of samples 1​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​ [Number] (i.e., the size that is [Number] a reduced block with, it is even possible to further reduce the number of multiplications (i.e., the samples directly predicted by ALWIP are less than the number of samples of the actually predicted block 18). Therefore, [Number] when setting this, this takes the ALWIP prediction by using red Q red *P red (Q red *P red < Q*P red < Q*P, i.e., Q*P). This multiplication predicts a reduced block with dimensions [Number] [Number] However, it is possible to perform upsampling (e.g., obtained by interpolation) from the reduced [Number] [[ID=5​​​​​​​​​​​​​​​​​​Both conversions (e.g., interpolations) are performed by reducing (and even avoiding) multiplications. This can be advantageous because it allows for non-computational power savings, such as adding and shifting. By employing binary operations that require and / or interpolation can be performed (e.g., steps 811 and / or 813 in). Also, addition is a very simple operation that can be easily performed without much computational effort. be.

[0099] This shifting operation is done, for example, by shifting two boundary samples to obtain the final predicted block. and / or taken from the (or boundary of) the downscaled prediction block to average the The two samples (support values) can be used to interpolate between them. For this, two sample values ​​are required. Within a block, there are always two predetermined values. but to interpolate samples along the left and top boundaries of the block, as in Figure 7.2 , has only one predetermined value and therefore uses the boundary sample as the support value for interpolation do.)

[0100] A two-step procedure can be used as follows: First add the values ​​of the two samples, The sum value is then halved (eg, by right shifting). Alternatively, you can: Each of the samples is first halved (e.g., by left-shifting),

[0101] The values ​​of the two halved samples are then added together. Select one sample volume and a group of samples (e.g., samples adjacent to each other) Since it is only necessary to do so, easier operations can be performed during downsampling (for example, in step 811). Therefore, it is possible to define techniques for reducing the number of multiplications to be performed here. Some of these techniques may be based, among other things, on at least one of the following principles: Even if the size of the actually predicted block 18 is M×N, the block is reduced (in at least one of the two dimensions) to a reduced size of Q ×P

[0102] where P =N red ×P red (

Number

Number

Number

[0103] Additionally or alternatively, all Q=M*N values ​​of the block 18 to be predicted are multiplied by Instead of predicting, we use a reduced block with reduced dimensions (e.g.

number

number

number

[0104] According to the example shown in Figure 7.1, 18 4x4 blocks (M=4, N=4, Q=M*N=16) ) is predicted, and the neighborhood of sample 17a (four already predicted samples ) and 17c (horizontal row with four already predicted samples) Neighbors 17a and 17c are collectively denoted by 17, as they were already predicted in the previous iteration. A priori, by using the formula shown in Figure 5, the prediction matrix 17M is Q × P = 16x8 matrix (since Q = M*N = 4*4 and P = M+N = 4+4 = 8) and the boundary vector 17P should have dimensions 8x1 (because P=8). However, this leads to the necessity of performing 8 multiplications for each of the 16 samples in the 4×4 block 18 to be predicted, and thus a total of 16 * 8 = 128 multiplications. (It should be noted that the average number of multiplications per sample is a favorable evaluation of the computational complexity. In conventional intra prediction, 4 multiplications per sample are required, which increases the computational effort involved. Therefore, it is possible to use this as the upper limit of ALWIP, ensuring that the complexity is reasonable and does not exceed that of conventional intra prediction.) This leads to the necessity of performing 8 multiplications for each of the 16 samples in the 4×4 block 18 to be predicted, and thus a total of 16 * 8 = 128 multiplications. (It should be noted that the average number of multiplications per sample is a favorable evaluation of the computational complexity. In conventional intra prediction, 4 multiplications per sample are required, which increases the computational effort involved. Therefore, it is possible to use this as the upper limit of ALWIP, ensuring that the complexity is reasonable and does not exceed that of conventional intra prediction.) This leads to the necessity of performing 8 multiplications for each of the 16 samples in the 4×4 block 18 to be predicted, and thus a total of 16 * 8 = 128 multiplications. (It should be noted that the average number of multiplications per sample is a favorable evaluation of the computational complexity. In conventional intra prediction, 4 multiplications per sample are required, which increases the computational effort involved. Therefore, it is possible to use this as the upper limit of ALWIP, ensuring that the complexity is reasonable and does not exceed that of conventional intra prediction.) However, by using this technology, in step 811, it is understood that the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain the reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average the adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2××2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, the averaged samples), and the vertical column 17a originally having four samples is processed as having two samples.) However, by using this technology, in step 811, it is understood that the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P

[0105] <P. In particular, in order to obtain the reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average the adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2××2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, the averaged samples), and the vertical column 17a originally having four samples is processed as having two samples.) red <P. In particular, in order to obtain the reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average the adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2××2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, the averaged samples), and the vertical column 17a originally having four samples is processed as having two samples.) However, by using this technology, in step 811, it is understood that the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain the reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average the adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2××2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, the averaged samples), and the vertical column 17a originally having four samples is processed as having two samples.) However, by using this technology, in step 811, it is understood that the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain the reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average the adjacent boundary samples (17a, 17c ) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as the block 18 is a 2××2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for the row 17c and two samples for the column 17a. Therefore, the horizontal row 17c is processed as having two samples instead of having four original samples (for example, the averaged samples), and the vertical column 17a originally having four samples is processed as having two samples.) However, by using this technology, in step 811, it is understood that the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced from P to P <P. In particular, in order to obtain the reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average the adjacent boundary samples (17a, 17c (For example, an averaged sample). After dividing row 17c and column 17a into two groups of samples 110 respectively, a single sample is maintained (for example, the average of the samples in group 11 0 or a simple selection between the samples in group 110). It can also be understood that . Therefore, a so-called reduced set 102 of sample values is obtained by a set 102 having only four samples (M red = 2, N red = 2, P red = M red + N red = 4, P red < P). A so-called reduced set 102 of sample values is obtained.

[0106] It is understood that operations (such as averaging or downsampling ring 100) can be performed without performing too many multiplications at the processor level: the averaging or downsampling 100 performed in step 811 can be easily obtained by simple and computationally non-power-consuming operations such as addition and shift.

[0107] At this point, it is understood that the reduced set 102 of sample values can be subjected to a linear or affine linear (ALW IP) transform 19 (for example, using a prediction matrix such as matrix 17M in FIG. 5). In this case, the ALWIP transform 19 directly maps the four samples 1 0 to the sample values 104 of block 18. In this case, interpolation is unnecessary. ""

[0108] 7]In this case, the ALWIP matrix 17M has dimensions Q × P red "" = 16 × 4: This means that all Q = 16 samples of the block to be predicted 18 are directly taken by ALWIP multiplication. This is due to the fact that the

[0109] Therefore, in step 812a, the dimensions Q×P red Having the right ALWIP The matrix 17M is selected. The selection may be based on, for example, the signaling from the data stream 12. The selected ALWIP matrix 17M may also be based at least in part on A k Shown in where k may be signaled in data stream 12. can be understood as an index that can be used to

number

[0110] In step 812b, the selected Q×P red ALWIP matrix 17M(A k year (also shown) and P red x1 Multiplication with boundary vector 17P is performed.

[0111] In step 812c, an offset value (e.g., b k ) is, for example, ALWIP Thus, the obtained vector 18Q can be added to all the obtained values ​​104. offset(bk , or in some cases

number

[0112] As can be seen, averaging (possibly adding and / or shifting) and / or downsampling) By having the appropriate value, it is possible to obtain the appropriate value in step 812.

[0113] Referring to Figure 7.2, the block to be predicted, 18, is an 8x8 block of 64 samples. lock (M=8, N=8), where a priori, the prediction matrix 17M is of size Q × P = 64 x 16 (Q = M * N = 8 * 8 = 64, since M = 8 and N = 8, and P = M + N = 8 + 8 = 16, so Q = 64. Therefore, a priori, the prediction P = 16 multiplications for each of the Q = 64 samples in the 18 8x8 blocks of interest are needed, amounting to 64*16=1024 multiplications for the entire 8x8 block of 18. You will!

[0114] However, as can be seen in Figure 7.2, all 16 boundary samples are used. Instead, eight values ​​(e.g., four in horizontal boundary row 17c and four in vertical boundary row 17d) are used between the original samples of the boundary. A method 820 can be provided in which only four of the direct boundary columns 17a are used. From 17c, four samples may be used instead of eight (e.g., they may be (This may be the average of two and / or the selection of one sample out of two). So the boundary vector is not a P×1=16×1 vector, but a P red ×1=8×1 vector Tor only (P red =M red +N red =4+4). Original P=16 samples Instead of P red = 8 boundary values. and samples in vertical column 17a are selected or averaged (e.g., 2×2) to reduce the sample values. It is understood that it is possible to form a reduced set 102. 102 allows to obtain a reduced version of block 18, and the reduced version is , (instead of Q=M*N=8*8=64) Q red =M red *N red =4*4=1 There are 6 samples. Size M red ×N red = 4 × 4 block prediction AL A reduced version of block 18 can be applied to the WIP matrix shown in Figure 7.2. Including the samples shown in grey in Scheme 106: Samples shown in grey squares (including samples 118′ and 118″) are obtained in step 812 of interest. Q red = 16 values. The 4x4 reduced block is obtained by applying the linear transformation 19 in step 812 of interest After obtaining the values ​​of the 4x4 reduced block, the remaining samples (s It is possible to obtain the values ​​of the samples (shown as white samples in scheme 106) is.

[0115] Regarding the method 810 of FIG. 7.1, the method 820 may be implemented, for example, by using the M×N=8× 8 Block 18 Remaining QQ red =64-16=48 sample (white square) prediction The method may further include step 813 of deriving the value by interpolation. re d = 64 - 16 = 48 samples are interpolated (interpolation also uses the values ​​of boundary samples, e.g. Q obtained directly by red = 16 samples As can be seen from Figure 7.2, samples 118' and 118'' are The sample 108' (shown as a square) was taken in step 812. The pulls 118' and 118'' are shown as white squares in step 81. 3 is obtained by interpolation between samples 118' and 118''. , can be obtained by operations similar to those for averaging, such as shifting and adding. Therefore, in Figure 7.2, the value 108' is generally The value can be determined as the midpoint between the value of the sample 118' and the value of the sample 118'' ( can be taken as an average).

[0116] By performing interpolation, in step 813, a plurality of samples shown at 104 are obtained. It is also possible to arrive at the final version of M × N = 8 × 8 blocks 18 based on the pull value. be. Therefore, the comparison between using this technology and not using it is as follows: : Without this technology: a block 18 to be predicted, which is a block having dimensions M=8, N=8, and Q=M*N=8*8=64 samples of block 18 to be predicted, P=M+N=8+8=16 samples for boundary 17, P=16 multiplications for each of the Q=64 values ​​to be predicted, P*Q=16*64=1028 total multiplications The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q=16 When using this technology: Block 18 to be predicted, with dimensions M=8, N=8 The last Q = M * N = 8 * 8 = 64 values to be predicted,

[0117] However, P red = M red + N red , Q red = M red * N red , M r ed = 4, N red = 4, Q has red × P red ALWIP matrix is used P red <P, the boundary P red = M red + N red = 4 + 4 = 8 samples For each of the 16 values of the 4×4 reduced block Q to be predicted red P red = 8 times of multiplication (formed by the gray squares in scheme 106), P red * Q red = 8 * 16 = 128 total multiplications (much less than 1024!)

[0118] The ratio of the number of multiplications to the number of final values to be obtained is P red * Q red / Q = 128 / 6 4 = 2 (much less than 16 obtained without using this technology!) Therefore, the technology presented in this specification requires 8 times less power than the previous technology.

[0119] Figure 7.3 shows another example (which can be based on method 820), the block to be predicted 18 is a rectangular 4×8 block (M = 8, N = 4) with Q = 4 * 8 = 32 samples to be predicted. The boundary 17 is formed by the horizontal row 17c of N = 8 samples and the vertical column 17a of M = 4 samples. Therefore, a priori, the boundary vector 17P is in inches It has the equation P×1 = 12×1, and the predicted ALWIP matrix is a Q×P = 32×12 matrix. It should be, and thus, Q*P = 32*12 = 384 multiplications are required.

[0120] However, for example, at least 8 samples of horizontal row 17c are averaged or downsampled to obtain a reduced horizontal row of only 4 samples (e.g., the averaged samples). In some examples, vertical column 17a remains as it is (e.g., without averaging). In total, the reduced boundary has dimension P = 8, and P < P. Thus, the boundary vector 17P has dimension P ×1 = 8×1 red = 8×1 red < P. Thus, the boundary vector 17P has dimension P red ×1 = 8×1 and the ALWIP prediction matrix 17M has dimension M*N red *P [[ID=2C]] red = 4*4*8 = 64 and becomes a matrix with this dimension. The 4×4 reduced block directly obtained in the target step 812 (formed by the gray columns of schema 107 ) has size Q red = M*N red = 4*4 = 16 samples (instead of Q = 4*8 = 32 for the original 4×8 block 18 to be predicted). When the reduced 4×4 block is obtained by ALWIP, an offset value b is added (step 812c), and interpolation can be performed in step 813 . As can be seen in step 813 of FIG. 7.3, the reduced 4×4 block is expanded to a 4×8 block 18, and the value 108’ not obtained in step 812 is obtained in step 813 by interpolating the values 118’ and 118’’ (gray squares) obtained in step 812. k is added (step 812c), and interpolation can be performed in step 813 . As can be seen in step 813 of FIG. 7.3, the reduced 4×4 block is expanded to a 4×8 block 18, and the value 108’ not obtained in step 812 is obtained in step 813 by interpolating the values 118’ and 118’’ (gray squares) obtained in step 812. is expanded to a 4×8 block 18, and the value 108’ not obtained in step 812 is obtained in step 813 by interpolating the values 118’ and 118’’ (gray squares) obtained in step 812. 08’ is obtained in step of FIG. 7.3 by interpolating the values 118’ and 118’’ (gray squares) obtained in step 812. 08’ is obtained in step 813 by interpolating the values 118’ and 118’’ (gray squares) obtained in step 812. Therefore, comparing using and not using this technology, it is as follows : When not using this technology: Block 18 to be predicted, which is a block with dimensions M = 4 and N = 8 Q = M * N = 4 * 8 = 32 values to be predicted, P = M + N = 4 + 8 = 12 boundary samples, P = 12 multiplications for each of the Q = 32 values to be predicted, Total number of multiplications: P * Q = 12 * 32 = 384 Ratio of the number of multiplications to the number of final values obtained: P * Q / Q = 12 When using this technology: Block 18 to be predicted, which is a block with dimensions M = 4 and N = 8 Finally, Q = M * N = 4 * 8 = 32 values to be predicted,

[0121] However, M = 4, N red = 4, Q red = M * N red = 16, P red = M + N red = 4 + 4 = 8, Q red × P red = 16 × 8 ALWIP matrix can be used P red < P, boundary P red = M + N red = 4 + 4 = 8 samples Q of the reduced block to be predicted red = 16 values for each P red = 8 multiplications Q red * P red = 16 * 8 = 128 total multiplications (less than 384!)

[0122] Ratio of the number of multiplications to the number of final values to be obtained: P red * Q red / Q = 128 / 3 ​​2 = 4 (much less than the 12 obtained without using this technique!). Therefore, the present technique reduces the computational effort by a factor of three.

[0123] Figure 7.4 shows the case of a block 18 to be predicted with dimensions M × N = 16 × 16, The final prediction target has Q=M*N=16*16=256 values, and P=M+N=16+1 6 = 32 boundary samples. This means that the predicted row has dimensions Q × P = 256 × 32. This results in a sequence, which means 256*32=8192 multiplications!

[0124] However, by applying method 820, the boundary samples are Reduce the number of samples, for example from 32 to 8 (for example by averaging or downsampling) For example, for each group 120 of four consecutive samples in row 17a In this case, a single sample (e.g., selected from among four samples or Also, for each group of four consecutive samples in column 17c, a single A sample (e.g., selected from four samples or the average of the samples) is Remains.

[0125] Here, the ALWIP matrix 17M is Q red ×P red = 64 × 8 matrix: , it uses 8 averaged samples or 32 boundary-selected samples. By using red = 8 and the fact that in step 812 The reduced block to be predicted is an 8x8 block (in scheme 109, the gray squares are 64 This is due to the fact that

[0126] Therefore, the 64 samples of the 8x8 block reduced in step 812 are Once obtained, in step 813, the remaining QQ of the block 18 to be predicted is red =256-64=192 values ​​104 can be derived.

[0127] In this case, to perform the interpolation, all samples of the boundary column 17a and the boundary row 17c are It has been chosen to use only alternate samples of . Other choices may be made.

[0128] In this method, the ratio between the number of multiplications and the number of values ​​finally obtained is Q red *P red / Q=8*64 / 256=2, which is less than 32 multiplications of each value without this technique. Much less! Therefore, the comparison between using this technology and not using it is as follows: : Without this technology: Block 18 to be predicted, which is a block with dimensions M=16, N=16 Q=M*N=16*16=256 values ​​to be predicted, Boundary P=M+N=16*16=32 samples, P=32 multiplications for each of the Q=256 values ​​to be predicted, P*Q=32*256=8192 total multiplications The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q=32 When using this technology: Block 18 to be predicted, which is a block with dimensions M=16, N=16 The final Q=M*N=16*16=256 values ​​to predict,

[0129] However, M red =4, N red = 4, Q of the target to be predicted by ALWIP red= 8 * 8 = 64 samples, P red = M red + N red = 4 + 4 = 8, Q re d × P red = 64 × 8 ALWIP matrix is used P red <P, the boundary P red = M red + N red = 4 + 4 = 8 samples Q of the reduced block to be predicted red = P for each of the 64 values red = 8 times of multiplication calculation Q red * P red = 64 * 4 = 256 total multiplications (less than 8192!)

[0130] The ratio of the number of multiplications to the number of final values to be obtained is P red * Q red [[ID=,46]] / Q = 8 * 64 / 256 = 2 (much less than 32 obtained without using this technology!) Therefore, the computational power required by this technology is 16 times smaller than that of the conventional technology! Therefore

[0131] Reduce a plurality of adjacent samples (100, 813), and compare with a plurality of adjacent samples (1 7) to obtain a set of reduced sample values (102) with fewer samples obtained

[0132] Subject the set of reduced sample values (102) to a linear or affine linear transformation (19, 17 M) to obtain the predicted values of the predetermined samples (104, 118’, 188’ ’) of the predetermined block (18) (812) By doing so, a plurality of adjacent samples (17) are used to predict a predetermined block of the picture ( 18) can be predicted.

[0133] In particular, we downsample multiple adjacent samples to obtain a By obtaining a reduced set of sample values ​​(102) with a relatively small number of samples, Therefore, it is possible to perform the reduction (100, 813).

[0134] Alternatively, multiple adjacent samples can be averaged and compared with multiple adjacent samples (17). By obtaining a reduced set of sample values ​​(102) with fewer samples, It is possible to perform a reduction (100, 813).

[0135] Furthermore, a given sample (104, 118', 118'') and a number of adjacent samples Based on the predicted value of (17), further samples (108, 1 08') can be derived (813) by interpolation.

[0136] A plurality of adjacent samples (17a, 17c) are arranged on either side (e.g., For example, it may extend in one dimension along the line (to the right and bottom of Figures 7.1 to 7.4). A given sample (e.g., obtained by ALWIP in step 812) The pixels may also be arranged in rows and columns, and a predetermined size may be provided along at least one of the rows and columns. The samples are the samples (11) of a given sample 112 adjacent to either side of a given block 18. 2) may be placed at every nth position.

[0137] For each of at least one of the rows and columns, based on a plurality of adjacent samples (17), The support value (118) of one of the plurality of adjacent positions (118) aligned in a row is and for each of the columns. The predicted values ​​118 of the further samples (108, 108') of the block (18) are calculated based on the given sample The predicted value of (104, 118', 118'') and the It can also be derived by interpolation based on the support values ​​of the adjacent samples (118) is.

[0138] A given sample (104) is adjacent to the samples on either side of a given block 18 along a row. The sample may be placed at every nth position along the column (112), and a given sample may be placed at every nth position along the column (112). The samples (112) of a given sample adjacent (112) on both sides of a given block (18) ), where n, m>1. In some cases, n=m. (For example, in Figures 7.2 and 7.3, the data obtained directly by ALWIP in 812 Samples 104, 118', and 118'', shown in grey squares, are In step 813, the samples 108, 108' are subsequently acquired. (Reciprocated).

[0139] Along at least one of the rows (17c) and columns (17a), for example, for each support value: and a plurality of adjacent samples, including adjacent samples (118) for which respective support values ​​are determined. Downsample or average (122) groups of adjacent samples (120) of It may be possible to perform the determination of the support value by In 7.4, in step 813, the location (previously obtained in step 812) The values ​​of a given sample 118''' and adjacent samples 118 are used as support values. It is possible to obtain the value of sample 119 by

[0140] A plurality of adjacent samples extend in one dimension along both sides of a given block (18). The plurality of adjacent samples (17) may be divided into one or more groups of consecutive adjacent samples (1 10) and groups of one or more adjacent samples with two or more adjacent samples. By performing downsampling or averaging on each of the groups (110), Therefore, it may be possible to perform the reduction (811).

[0141] In the example, a linear or affine linear transformation is red *Q red or P red *Q weighting It may include coefficients, P red is the sample value (10 2) is the number of Q red or Q is the number of given samples in a given block (18) At least 1 / 4 of P red *Q red or 1 / 4 P red *Q weighting factor is , are non-zero weight values. P red *Q red or P red *Q weighting coefficients are Q red For each of the predetermined samples, A series of P red The weighting coefficients may be included in the predetermined block (18). If the samples are arranged one above the other in the raster scan order, the It forms a linear envelope. P red *Q or P red *Q red The weighting coefficients are may be unrelated to each other through a general mapping rule. The weighting coefficients of the associated first sequence and the weighting coefficients of the associated second sequence for the given samples other than the respective given samples are The average of the maximum cross-correlation between the weighting coefficients of the second series and the corresponding sequence, or the inverse of the latter series. The ones that are lower than the predetermined threshold, despite resulting in a higher maximum value. The threshold for P can be 0.3 [or possibly 0.2 or 0.1]. red The adjacent samples (17) are arranged in a linear pattern extending along both sides of a given block (18). The original path may be arranged along Q or Q red For each of the given samples For each given sample, red The weighting coefficients are given by the one-dimensional path. The images are ordered to be traversed in a specific direction.

[0142] 6.1 Method and Apparatus Description width

number

number

number

[0143] 1. Among the boundary samples 17, sample 102 (for example, when W=H=4, there are four samples, and / or 8 samples in other cases) are averaged or downsampled The rings can be extracted (eg, step 811).

[0144] 2. Matrix vector multiplication followed by adding an offset results in the averaged samples (also can be performed with the remaining samples from the downsampling as input. The result is a reduced prediction on a set of subsampled samples in the original block. This may be a signal (eg, step 812).

[0145] 3. The predicted signal for the remaining positions is subsampled, e.g., by upsampling. The predicted signal can be generated from the set of predicted signals, for example by linear interpolation (e.g. Step 813).

[0146] By virtue of steps 1.(811) and / or 3.(813), the matrix-vector product The total number of multiplications required for a calculation is always

number

[0147] In some examples, the matrices (e.g., 17M) and offset vector (e.g., b k ) are stored in the memory units of the decoder and the encoder, for example. A set of matrices (e.g., three sets) that can be stored, e.g.,

number

number

[0148] In some cases, the set

number

number

number

number

number

number

number

[0149] In some cases, the set

number

number

number

number

number

number

number

number

number

number

number

[0150] Additionally or alternatively, a set

number

number

number

number

number

[0151] 6.2 Averaging or downsampling of boundaries Here, features are provided regarding step 811. As described above, boundary samples (17a, 17c) can be averaged and / or downsampled (e.g., from P samples to P < P samples). red can be done.

[0152] In the first step, the input boundary [Number] (e.g., 17c) and [Number] (e.g., 17a) are reduced to a smaller boundary [Number] and [Number] and can reach the reduced set 102. Here, [Number] <*[0001873]*> and [[ID=...]] [Number] If both are 4x4 blocks, they consist of 2 samples, otherwise they consist of 4 samples. It consists of For 4x4 blocks,

number

number

[0153] It is possible to define

number

number

number

number

number

[0154] In all other cases (e.g., blocks with a wither width or height different from 4) Regarding the block width W,

number

number

number

number

[0155] In still other cases, (e.g., one particular boundary sample from a group of boundary samples) downsample the boundaries (by selecting For example,

number

number

number

number

number

number

number

[0156] Two reduced boundaries

number

number

number

number

number

number

number

number

number

number

[0157] Therefore, in a particular state (

number

number

[0158] Other strategies may be implemented. In another example, the mode index "mode" is always and not within the range 0 to 35 (other ranges may be defined). It is not necessary for each of the subgroups S0, S1, and S2 to have 18 matrices (thus,

number

number

number

number

[0159] The mode and transposition information are not necessarily combined into one combined mode index, "mode" In some cases, the transposition flag and and matrix index (0-15 for S0, 0-7 for S1, 0-5 for S2) )

[0160] In some cases, the combination of the transpose flag and the matrix index is a set index. For example, one bit may act as a transposition flag and the other as a "set in" bit. There are several bits that indicate the matrix index, collectively referred to as the "matrix index." It can exist.

[0161] 6.3 Generating reduced prediction signals by matrix-vector multiplication Here, features are provided with respect to step 812. The reduced input vector

number

number

number

number

number

number

number

number

number

[0162] Reduced prediction signal

number

number

[0163] where:

number

number

number

number

number

number

number

number

[0164] matrix A and vector

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0165] Other strategies may be implemented. In another example, the mode index "mode" is always and not within the range 0 to 35 (other ranges may be defined). It is not necessary for each of the subgroups S0, S1, and S2 to have 18 matrices (thus,

number

number

number

number

[0166] 6.4 Linear Interpolation to Generate the Final Prediction Signal Here, features are provided with respect to step 812. For the interpolation of the subsampled prediction signal in large blocks, the averaged boundary A second version of the following may be required:

number

number

number

number

number

number

number

number

[0167] Additionally or alternatively, it is possible to have "hard downsampling", which So,

number

number

[0168] Also,

number

number

number

number

[0169] The linear interpolation may be given as follows (other examples are possible):

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0170] This uses the scaled boundary samples for the first interpolation (horizontal or vertical) and the This is an example of interpolation using original boundary samples between blocks (vertically or horizontally). Depending on the need, only the second interpolation or no interpolation is required. Both horizontal and vertical interpolation are required. If so, the order depends on the width and height of the block. However, different techniques may be implemented, for example, if the original boundary samples are the first and The second interpolation may be used for both, and the order may be fixed, e.g., horizontal first, Then vertical (or in other cases vertical first, then horizontal). Therefore, the interpolation order (horizontal / vertical) and the use of reduced / original boundary samples are changed. It is possible.

[0171] 6.5 Explaining the complete ALWIP process example The whole process of averaging, matrix vector multiplication, and linear interpolation is shown in Figure 7.1 to Figure 7.4. The remaining shapes are treated as one of the illustrated cases. Please note that.

[0172] 1.

number

number

number

[0173] 2.

number

number

number

[0174] 3.

number

number

number

[0175] 4.

number

number

number

number

[0176] Finally, for W×4 blocks where W>8,

number

number

[0177] 6.6 Evaluating the number and complexity of parameters required The parameters required for all possible proposed intra prediction modes are given in the set

number

number

number

[0178] 6.7 Proposed Intra Prediction Mode Signaling For the luma block, for example, 35 ALWIP modes have been proposed (other numbers For each coding unit (CU) in intra mode, Indicates whether the ALWIP mode should be applied to the corresponding prediction unit (PU). The latter index signaling is transmitted in the bitstream. It can be harmonized with MRL in the same way as CE testing. ALWIP mode is applied. If so, the index in ALWIP mode

number

[0179] Here, the derivation of MPM is as follows, using the intra-modes of the top and left PUs: Each conventional intra-prediction mode may be performed.

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0180] The embodiments described herein utilize the above-described signaling of proposed intra-prediction modes. According to an alternative embodiment, the MPM and / or mapping table may be ,Not used for MIP (ALWIP).

[0181] 6.8 Adaptation for Conventional Luma Intra Prediction Mode and Chroma Intra Prediction Mode MPM list derivation The proposed ALWIP mode is an MPM-based alternative to conventional intra prediction modes as follows: The luma and chroma M of conventional intra prediction modes can be integrated with the coding of the signal. The PM list derivation process is a fixed table

number

number

number

number

[0182] For Luma MPM list derivation, ALWIP mode

number

number

[0183] 7. Implementation Efficiency The above is intended to be illustrative and not restrictive, as it may form the basis for further extensions of the embodiments described below. The above example is briefly summarized below. To predict a given block 18 of a picture 10, a plurality of adjacent samples 17a, c are used. To be used is to be used.

[0184] Reduction by averaging multiple adjacent samples 100 reduces the sample size compared to multiple adjacent samples. A smaller number of samples are taken to obtain a reduced set of sample values ​​102. The reduction of is optional in the embodiments herein and is referred to below as the so-called sample value base The reduced set of sample values ​​is To obtain the predicted value of 104, it undergoes a linear or affine-linear transformation 19. This transformation is ,Later, the matrix obtained by machine learning (ML) and the ,implementation should be performed efficiently. A and an offset vector b.

[0185] Interpolation allows a prediction of another sample 108 in a given block to be calculated based on the given sample and It is derived based on the prediction of multiple neighboring samples. In theory, it is possible to The result is that all samples in block 18 are obtained by interpolation according to an alternative embodiment. can be associated with non-full-pel sample positions in block 18 so that You may not even need interpolation at all.

[0186] The plurality of adjacent samples may extend in one dimension along both sides of a given block, the samples are arranged in rows and columns, and are arranged along at least one of the rows and columns; A given sample is divided into two blocks, one for each of the adjacent samples (112 ) may be placed at every n-th position from the row and For each of at least one of the rows, a A support value can be determined, which is located at each of the rows and / or columns. and by interpolation, predictive values ​​for the further samples 108 of the given block. is the predicted value for a given sample and is aligned to the row and / or column. The given sample can be derived based on the support values ​​for the neighboring samples. Samples 112 to n of a given sample adjacent to either side of a given block along a row A given sample may be placed at every position on either side of a given block along a column. may be located at every m-th position from the sample 112 of the adjacent given sample, and n , m>1. n=m may be used. Along at least one of the rows and columns, the support values The determination of includes, for each support value, the neighboring samples 118 for which the respective support value is determined. By averaging (122) groups of adjacent samples 120 within a plurality of adjacent samples The adjacent samples are arranged in one dimension along both sides of a given block. The shrinking may be performed by dividing multiple adjacent samples into groups of one or more consecutive adjacent samples. Grouping into groups 110, groups of one or more adjacent samples having three or more adjacent samples This may be done by performing an averaging over each of the loops.

[0187] For a given block, the prediction residual can be transmitted in the data stream. It can be derived therefrom in the decoder, and a given block is The prediction residual and the predicted value can be used to reconstruct the The residual is coded into a data stream in an encoder. A picture can be subdivided into multiple blocks of different block sizes, The plurality of blocks includes a predetermined block. Next, a linear or affine line of the block 18 The shape transformation is selected depending on the width W and height H of a given block, so that For a given block, as long as the width W and height H of the block are within the first set of width / height pairs, The linear or affine-linear transformation selected as the first set of linear or affine-linear transformations the width W and height H of a given block are selected from the set of width / height pairs. The second set of linear or affine-linear transformations can be used as long as they are within a second set of different width / height pairs. Similarly, later, the affine / linear transformation is performed on the other parameters, i.e., represented by weights, and optionally offset and scale parameters becomes clear.

[0188] The decoder and encoder divide the picture into blocks of different block sizes, including the predetermined block. Subdivide into multiple blocks and linear or axial depending on the width W and height H of a given block. The method may be configured to select a linear transformation such that for a given block The linear or affine linear transformation chosen by As long as the width W and height H of a given block are within the first set of width / height pairs, linear or or a first set of affine linear transformations, The width W and height H of a given block are different from the first set of width / height pairs. a second set of linear or affine-linear transformations, and The width W and height H of a given block are different from the first and second sets of width / height pairs. a third of a linear or affine-linear transformation, as long as it is within a third set of one or more width / height pairs , which is selected from the set

[0189] The third set of one or more width / height pairs includes only one width / height pair W', H', and the line Each linear or affine linear transformation in the first set of linear or affine linear transformations has N' to convert the sample values ​​of It is something. Each of the first and second sets of width / height pairs is W p H p First width not equal to / Height vs. W p , H p And, Hq =W p and W q =H p The second width / height pair W is q , H q and may include:

[0190] Each of the first and second sets of width / height pairs is connected to a third width / height pair W p , H p Osa It can also be included in W p is H p is equal to H p >H q is. For a given block, which of a given set of linear or affine-linear transformations or a set index indicating which affine linear transformation should be selected for block 18. The text can be transmitted within the data stream.

[0191] The plurality of adjacent samples may extend in one dimension along either side of the given block; The downscaling is performed on a first subset of a plurality of adjacent samples adjacent to a first side of the given block. The first subset is then divided into a first group 110 of one or more consecutive adjacent samples. and grouping a second subset of the plurality of adjacent samples adjacent to the second side of the given block into a For a set, a second subset is defined as a second group of one or more consecutive adjacent samples. The first sample value from the first group and the second sample value from the second group are grouped into groups 110. to obtain the second sample value of one or more adjacent samples having three or more adjacent samples. By performing averaging on each of the first and second groups of samples Then, a linear or affine linear transformation can be can be selected according to a set index from a predetermined set of Two different states of bit index are given by a given set of linear or affine-linear transformations. The output vector of predicted values ​​is also To achieve this, we first assume two different states in the form of a vector. In the case of a filter, the reduced set of sample values ​​is subjected to a given linear or affine-linear transformation. and a predicted value of the output vector can be calculated for a given block along the first scanning order. The second vector can be distributed to a certain sample and can represent two different states of the second Assuming the index is set, the first The first sample value of the vector is input and the second sample value of the second vector is input. The second vector is input to one of the input channels, and the second vector is input to one of the input channels. The first and second vectors are different so that one of the first samples of the vector is input. The second scan to a given sample of a given block is transposed relative to the first scan order. The predicted values ​​of the output vectors along the order can be distributed.

[0192] Each linear or affine-linear transformation in the first set of linear or affine-linear transformations is Convert one sample value into w1*h1 predicted values ​​for a w1×h1 array of sample locations In the first set of linear or affine linear transformations, Each linear or affine-linear transformation maps the N2 sample values ​​to a w2 × h2 array of sample positions. The first width / height pair is used to convert the first width / height pair into w2*h2 predicted values ​​for the first For a given one of the widths, w1 can exceed the width of the first given width / height pair, and h1 can exceed the height of the first given width / height pair, and For a given one, w1 cannot exceed the width of a second given width / height pair, and h1 The height of the second predetermined width / height pair cannot be exceeded. Taking multiple adjacent samples to obtain a reduced set of sample values ​​(102) The step (100) is performed if the predetermined block is of a first predetermined width / height pair, and and if a given block is of a second given width / height pair, the reduced size of the sample values The set 102 may be made to have N1 sample values, and the reduced sample Subjecting a set of values ​​to a selected linear or affine-linear transformation determines whether a given block is If it is of the first given width / height pair, then w1 is greater than the width of one width / height pair. along the width dimension, or along the height dimension if h1 exceeds the height of one width / height pair The selected linear or This may be done by using only the first sub-part of the affine linear transformation, If the block is of a second given width / height pair, the selected linear or affine A linear transformation may be performed entirely.

[0193] Each linear or affine-linear transformation in the first set of linear or affine-linear transformations is For a w1×h1 array of sample positions where 1=h1, we calculate N1 sample values ​​as w1*h It can be used to convert one predictor into a linear or affine linear transformation. Each linear or affine linear transformation in the set of 2 is a function of w2 of sample locations where w2 = h2. for converting N sample values ​​into w * h predicted values ​​for a x h array is.

[0194] All of the above-described embodiments may form the basis for the embodiments described herein below. That is, the concepts and details above are merely exemplary in that they may be implemented in the following embodiments: This will be helpful in understanding the present invention and will provide a basis for possible extensions and modifications of the embodiments described later in this specification. In particular, many of the details mentioned above involve averaging adjacent samples, The fact that the sample is used as a reference sample is optional.

[0195] More generally, the embodiments described herein are directed to the case where the intra prediction signal on a rectangular block is A rectangular block is generated from adjacent already reconstructed samples to the left and above the block. Assume that the predicted signal on the clock is generated from already reconstructed samples. The generation is based on the following steps:

[0196] 1. However, it precludes the possibility of transferring the description to a reference sample located elsewhere. Without any further analysis, the samples are extracted by averaging from a reference sample called the boundary sample. where the averaging is done on both the left and top boundary samples of the block. Averaging is performed only on the boundary samples on either side. If not, the sample on that side remains unchanged.

[0197] 2. A matrix vector multiplication is performed, optionally followed by the addition of an offset. The input vector of the matrix vector multiplication is the block vector if averaging is applied only to the left side. The averaged boundary sample of the block is concatenated to the left of the original boundary sample above the block, or If equalization is applied to the top side only, the original boundary samples to the left of the block and the top of the block are evenly spaced. If averaging or concatenation with smoothed boundary samples is applied to both sides of the block, The averaged boundary samples to the left of the block and the averaged boundary samples above the block. Again, there are alternatives, such as no averaging at all. There is.

[0198] 3. The result of the matrix vector multiplication and optional offset addition is optionally The reduced prediction signal is based on a set of subsampled samples within a block. The predicted signal at the remaining positions can be subsampled by linear interpolation. It can be generated from a set of predicted signals.

[0199] The matrix-vector product calculation in step 2 should preferably be performed in integer arithmetic. There is. Therefore,

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0200] FIG. 8 illustrates a method for predicting a given block 18 of a picture 10 using multiple reference samples 17. 1 shows an embodiment of an apparatus 1000 for measuring the number of reference samples 17 in a predetermined block. Dependent on the prediction mode used by the device 1000 to predict the block 18 When the prediction mode is, for example, intra prediction, a reference block adjacent to a given block can be In other words, multiple reference samples 17 can be used, e.g. For example, it is arranged in the picture 10 along the outer edge of a predetermined block 18. If the prediction mode is inter prediction, the reference samples 172 of the other picture 10' are used. It is possible.

[0201] The apparatus 1000 forms a sample value vector 400 from a plurality of reference samples 17. The sample vector can be obtained by different techniques. The sample value vector can, for example, include all reference samples 17. Optionally, the reference samples can be weighted. According to another example, the sample values Vector 400 is one of Figures 7.1 to 7.4 for sample value vector 102. In other words, the sample value vector 40 The zeros can be formed by averaging or downsampling. For example, a group of reference samples may be averaged to provide a reduced set of sample values. In other words, the device can obtain a sample value vector 400, for example. For each of the 400 components, one reference sample from multiple reference samples17 was used. as each component of a sample vector and / or Averaging two or more components of the sample vector 400, i.e., To obtain each component, two or more reference samples from multiple reference samples17 were used. By averaging the samples, a sample value vector 102 is obtained from multiple reference samples 17. It is configured to form 100.

[0202] The apparatus 1000 extracts from the sample value vector 400 a sample value vector 400 corresponding to a predetermined Derive 401 a further vector 402 that is mapped by a reversible linear transformation 403 The further vector 402 may be, for example, an integer value and / or a fixed-point value. The reversible linear transform 403 includes only point values. For example, the prediction of the samples of a given block 18 is selected to be performed using integer or fixed-point arithmetic.

[0203] Furthermore, the device 1000 may also perform a calculation between the further vector 402 and a predetermined prediction matrix 405. A matrix vector product 404 is calculated to obtain a predicted vector 406, and the predicted vector 406 is It is configured to predict the samples of a given block 18 based on the Based on the vector 402, a given prediction matrix is ​​quantized to produce a prediction matrix for a given block 18. Integer and / or fixed-point arithmetic with only a small effect of quantization error on the sample It can be made possible.

[0204] According to an embodiment, the apparatus 1000 performs the matrix vector product 404 using fixed-point arithmetic. Alternatively, integer arithmetic can be used. According to an embodiment, the device 1000 calculates the matrix vector product 404 without floating-point operations. It is configured to:

[0205] According to an embodiment, the device 1000 stores a fixed-point representation of the predetermined prediction matrix 405. Additionally or alternatively, an integer representation of the predetermined prediction matrix 405 is stored. It can be done. According to an embodiment, the device 1000 determines a predetermined block 18 based on the prediction vector 406. In predicting the samples of a given block 18 based on the prediction vector 406, It is also configured to use interpolation to calculate a single sample position, and each component of , are associated with corresponding positions within a given block 18. Interpolation is shown in Figures 7.1 to 7.4 2. The method of claim 1, wherein the method is implemented as described with respect to any of the embodiments shown in .

[0206] Figure 9 illustrates the inventive concept described herein. The samples of a given block are The matrix A1100 and the sample can be predicted based on a first matrix-vector product between the vector value vector 400 Optionally, an offset b 1110 can be added. To achieve an integer or fixed-point approximation of the vector product, the sampled vectors are further 402. The second matrix-vector product between matrix B 1200 and the further vector 402 is It can be made equal to the result of a column vector product.

[0207] For the additional vector 402 features, a second matrix-vector product is performed to obtain a predetermined prediction matrix C A matrix vector product between 405, a further vector 402 and a further offset 408 404. Further vectors 402 and further objects can be approximated by integers. The offset 408 can consist of integer or fixed point values. All elements of the matrix are, for example, the same. The given prediction matrix 405 is a quantized matrix. The matrix may be a predetermined prediction matrix 405 and a further vector. The result of the matrix vector product 404 between the vector 402 is known as the predicted vector 406. It is possible.

[0208] Further details regarding this integer approximation are provided below. Possible Solutions According to Embodiment I: Subtracting and Adding Average Values Expressions that can be used in the above scenarios

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0209] The predetermined value 1400 is not necessarily the average value.

number

number

number

number

number

number

number

number

number

number

[0210] Alternatively, the predetermined value 1400 may be a default value or the data stream in which the picture is encoded. This is the value signaled in the stream. The predetermined value 1400 is, for example, 2 bitdepth-1 In this case, a further vector Tor 402, when i>0, y0=2 bitdepth-1 and y i =x i -x0 Thus, it can be defined as:

[0211] Alternatively, the predetermined component 1500 is a constant minus the predetermined value 1400. The constant is, for example, For example, 2 bitdepth-1 According to an embodiment, a further vector y 40 2 prescribed ingredients

number

number

[0212] According to an embodiment, the device 1000 is configured to include a plurality of reversible linear transforms 403. , each of which is associated with one component of a further vector 402. For example, select a predetermined component 1500 from the components of the sample value vector 400, and A reversible linear transformation 4 among a plurality of reversible linear transformations associated with a predetermined component 1500 of 03 as its predetermined reversible linear transformation. Depending on the position of the given component in the vector 402, the i0th row, i.e., the given component due to different positions of the rows of the reversible linear transform 403 corresponding to If the first component of 402, i.e., y1, is a given component, then the i o The rows of are reversible linear variables Replace the first line of the transformation.

[0213] As shown in FIG. 10b, a predetermined vector 402 corresponding to a predetermined component 1500 of the further vector 402 Column 412 of the prediction matrix 405, i.e., the matrix of the given prediction matrix C 405 in the i0th column Component 414 is, for example, all 0. In this case, the device may, for example, , the resulting reduced matrix C 405 by leaving column 412 The predicted matrix C' 405 and the further vector C' are obtained by leaving the predetermined components 1500. A matrix vector product 407 between the still further vector 410 resulting from 402 To calculate the matrix vector product 404, we perform the multiplication by calculating Therefore, the prediction vector 406 can be calculated with fewer multiplications. can.

[0214] As shown in FIGS. 9, 10b and 10c, the device 1000 may For each component of the prediction vector 406, The sum of each component and a, i.e., a predetermined value 1400, is calculated. This addition can be done by adding the predicted vector 406 and the vector as shown in Figures 9 and 10c. The vector 409 can be expressed as a sum of the vector 409 and the vector 409, and all components of the vector 409 have a predetermined value. 1400. Alternatively, the addition can be performed using the predicted vector 406 and The sum of the matrix-vector product 1310 between the integer matrix M 1300 and the further vector 402 and the matrix elements of the integer matrix 1300 can be represented by a further vector 402 The column of the integer matrix 1300 corresponding to the predetermined element 1500 of the matrix i, i.e., the i0th column, is 1. and all other components are, for example, 0.

[0215] The result of the sum of the predetermined prediction matrix 405 and the integer matrix 1300 is, for example, the further is equal to or approximates a matrix 1200 which is In other words, a given prediction matrix 4 corresponding to a given element 1500 of the further vector 402 Each matrix element of the given prediction matrix C 405 in the i0th column 412, i.e., the i0th column, is calculated as follows: The matrix obtained by multiplying the inverse linear transform 403 by 1 (i.e., matrix B) is The matrix B 1200 can be obtained by machine learning, for example, as shown in FIGS. 9, 10a, and 10b. corresponds to a quantized version of prediction matrix A 1100. Summing each matrix element of matrix C 405 with 1 yields a given predicted It can correspond to the sum of matrix 405 and integer matrix 1300. As shown in FIG. The machine learning prediction matrix A 1100 is the result of a reversible linear transformation 403 of a further matrix 1200. can be equated to, which means

number

[0216] Matrix multiplication using only integer arithmetic Low complexity implementation (in terms of the complexity of adding and multiplying scalar values, as well as the partaking In terms of storage required for the column entries, matrix multiplication is 40x using only integer arithmetic. It is recommended to perform step 4.

number

number

number

number

number

[0217] The matrix vector product 404 with a matrix of size m×n, i.e., a given prediction matrix 405, is , can be implemented as shown in this pseudocode, where <<,>> is an arithmetic binary left and right shift operations, and +, - and * operate only on integer values .

[0218] (1) final_offset=1<<(right_shift_result-1); For i in 0...m-1 { Accumulator = 0 for j in 0...n-1 { Accumulator:=Accumulator+y[j]*C[i,j] } z[i]=(accumulator+final_offset)>>right_shift_result; }

[0219] where array C, i.e., the predetermined prediction matrix 405, is a fixed-point number, e.g., as an integer. The final addition of final_offset and right shift operation by right_shift_result are stored. reduces precision by rounding to obtain the required fixed-point format at output. To allow for an increase in the range of real values ​​representable by integers in C, Figures 11 and 1 As shown in the second embodiment, two additional matrices

number

number

number

number

number

number

number

number

number

[0220] In other words, the apparatus 1000 determines the predicted parameters, e.g., integer values.

number

number

number

[0221] According to an embodiment, the prediction parameters are each associated with a corresponding matrix element of the prediction matrix. In other words, a given prediction matrix includes weights assigned to the prediction parameters, e.g. The weights can be expressed as integers and / or prediction parameters. or a fixed point value.

[0222] According to an embodiment, the prediction parameters may be one or more scaling factors, e.g., values ​​sca le i,j each of which corresponds to one or more of the given prediction matrices 405 weights associated with matrix elements, e.g. integer values

number

number

number

number

[0223] For example, in one preferred embodiment:

number

number

number

number

[0224] According to an embodiment,

number

number

number

number

[0225] Offset Expression

number

number

[0226] A wide range of embodiments resulting from the solution The above solution implies the following embodiments: 1. A forecasting method as described in Section I, comprising: In this case, the following is done for integer approximations of the matrix-vector products involved:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0227] 2. A method of prediction as described in Section I, comprising the steps of Section I. In 2, the following is done for integer approximations of the matrix-vector products involved:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0228] 3. A prediction method as described in Section I, comprising the steps of:

number

number

number

[0229] 4. A forecasting method as described in Section I, wherein step 2 comprises: The observation modes can be calculated using one of K matrices, each of which is Different matrices with k=0...K-1

number

number

number

number

[0230] That is, according to an embodiment of the present application, the encoder and decoder are To predict a given block 18 of the image 10, we operate as follows: As outlined above, embodiments of the present application utilize the Therefore, the reference samples are neighboring samples, i.e., The reference samples are not limited to samples of the 10 adjacent blocks 18. In particular, the reference samples are The samples may be positioned along the outer edge of the block 18, such as, but not limited to, samples abutting the outer edge. However, this situation is certainly one embodiment of the present application.

[0231] To make predictions, samples were taken from reference samples such as reference samples 17a and 17c. A value vector 400 is formed. Possible formations are described above. Formation may include averaging. and thereby contribute to the formation of a number of samples in comparison with the reference sample 17. can reduce the number of components of the vector 400. The formation can also be done by using blocks, as described above. Depends in any way on the dimensions or size of the block 18, such as the width and height of the block 18. It is possible.

[0232] This vector should undergo an affine or linear transformation to obtain the prediction of block 18. The Tor 400. Different nomenclature is used above. Use the latest one and Set vector b to matrix A by matrix-vector multiplication, within which it performs addition. The goal is to perform prediction by applying rule 400. b is arbitrary. Affine or linear transformations determined by A or A and b are expressed as by the encoder and decoder, or more precisely by the encoder and decoder of block 18, as already mentioned above. A determination can be made for the prediction based on size and dimensions.

[0233] However, to achieve the computational efficiency improvements outlined above, or to perform the predictions To make it more effective, the affine or linear transform is quantized and the encoder and and the decoder, or its predictor, represents a quantized version of the affine transformation. To represent and perform a linear or affine transformation, with C and T applied in In particular, the predictors in the encoder and decoder are based on vectors 400 Instead of applying directly to matrix A, we apply the vector 40 obtained from the sample vector 400. 2 is applied by subjecting it to a mapping through a predetermined invertible linear transformation T, where The transformation T used is the same as long as the vectors 400 have the same size, i.e. Independent of block dimensions, i.e. width and height, or different affine / linear transformations In the above, vector 402 is denoted by y. The exact matrix to perform the affine / linear transformation determined by the experiment was B. However, instead of implementing B exactly, the prediction in the encoder and decoder is This is done by an approximation or quantized version. In particular, the representation is where C+M represents a quantized version of B.

[0234] Therefore, prediction in the encoder and decoder is performed by the vector 402 and the encoder and decoder A matrix-vector product 404 is performed between a predetermined prediction matrix C appropriately represented and stored in the decoder. This is further accomplished by calculating in the manner described above. Then, from this matrix-vector product, The resulting vector 406 is used to predict the samples 104 of the block 18 . As mentioned above, for prediction, each element of the vector 406 is a vector that compensates for the corresponding definition of C. To do this, it can be added with a parameter a as shown at 408. The optional addition of vector 406 with an offset vector b is also based on vector 406. As mentioned above, vector 40 can be included in the derivation of the prediction of block 18 based on the 6, and therefore each component of the sum of vector 406, all shown in 408 The vector a, and the optional vector b, are directly input to sample 104 of block 18. The corresponding sample 104 can therefore indicate the predicted value of the sample. Only a subset of ,is predicted in this way, and the remaining samples in block 18, e.g., 108, It can also be derived by interpolation.

[0235] As mentioned above, there are different embodiments for setting a. For example, vector 400 In this case, see Figure 10. The shape transformation T can be as shown in Figure 10. i0 is the sample value vector and a given component of vector 402, which is replaced by a. As shown above, there are other possibilities. However, as far as the representation of C is concerned, It has also been shown above that this can be implemented differently. For example, The matrix product 404 is, in its actual calculation, a smaller matrix vector with lower dimensions. In particular, as stated above, by the definition of C, the i0th Since the entire column 412 of

number

[0236] The weights of C or C', i.e., the elements of this matrix, are expressed in fixed-point number notation. However, these weights 414 can also be stored as , may be stored in a manner that relates to different scales and / or offsets. The rule and offset may be defined for the entire matrix C, i.e., for matrix C or It may be equal for all weights 414 of matrix C', or it may be equal for matrix C and matrix C'. A constant or equal weight is assigned to all weights 414 in the same row or all weights 414 in the same column, respectively. In this regard, Figure 11 shows the calculation of the matrix vector product, That is, the result of the product may be implemented slightly differently in practice, i.e., For example, by shifting the multiplication with the scale towards vector 402 or 404, can be performed in a multiplication step, thereby reducing the number of further multiplications that must be performed. Figure 12 shows that the C or C' 4 shows the case where one scale and one offset are used for all weights 414 in the is doing.

[0237] According to an embodiment, the device described herein for predicting a given block of a picture can be configured to use matrix-based in-sample prediction, which includes the following features: Cut: The device forms a sample value vector pTemp[x] 400 from a plurality of reference samples 17. Assume that pTemp[x] is 2*boundarySize. Then pTemp[x] can be obtained by, for example, direct copying or by subsampling. Alternatively, pooling may be used to select adjacent samples at the beginning of a given block, x=0..bo redT[x] with undarySize-1, followed by the the adjacent samples, redL[x] (e.g., redL[x]) with x=0..boundarySize-1. by a transposition operation (e.g., when transposition = 0) or by a transposition operation (e.g., when transposition = 1). can be populated by its inverse.

[0238] The input values ​​p[x] for x=0..inSize-1 are derived, i.e., the device In this way, from the sample value vector pTemp[x], ] is mapped by a given invertible linear transformation. or more specifically, a predetermined invertible affine linear transformation. If -mipSizeId is equal to 2, the following applies: p[x]=pTemp[x+1]-pTemp[0] - Otherwise (mipSizeId is less than 2), the following applies: p[0]=(1<<(BitDepth-1))-pTemp[0] p[x]=pTemp[x]-pTemp[0] for x=1..inSize-1

[0239] Here, the variable mipSizeId indicates the size of a predetermined block. According to an embodiment, a further vector is derived from the sample value vector using The reversible transformation depends on the size of the given block. The dependency is given by It is possible.

[0240] [Table 3]

[0241] predSize indicates the number of predicted samples in a given block, and 2*bondar ySize indicates the size of the sample vector, and inSize, i.e., inSi ze=(2*boundarySize)-(mipSizeId==2)?1:0 Therefore, it is related to the size of the further vector: 1:0. More precisely, inSize is , indicates the number of components of the further vector actually involved in the calculation. The small block size is the same as the size of the sample vector, and the large block size is In the former case, one component is smaller, i.e., the component calculated later. Like a matrix-vector product, the components corresponding to a given component of a further vector are ignored. The contribution of the corresponding vector component will be zero anyway, and therefore it is not actually calculated. The dependency on block size requires that only one of the two alternatives is In the case of an alternative embodiment where the can be left (if the option corresponding to mipSizeId is less than 2 or or the option corresponding to mipSizeId is equal to 2).

[0242] In other words, a given invertible linear transformation is, for example, a given component of a further vector p is transformed into a and all other components are defined to correspond to the components of the sampled vector minus a. For example, a = pTemp[0]. The first one corresponds to mipSizeId equal to 2. In the case of the option, this is easily seen, and only the separately formed components of the further vectors is further considered, i.e., in the first option, the further vector is actually is {p[0...inSize];pTemp[0]}, where pTemp[0] is a and the actual calculated part of the matrix-vector multiplication to yield the matrix-vector product, i.e. That is, the result of the multiplication is a matrix with zero columns that do not require any further calculations, and therefore no further vectors and and the inSize elements of the corresponding columns of the matrix. As all components of the further vectors except p[0], corresponding to the new mipSizeId , a=pTemp[0] is selected, i.e., a further vector excluding the given component p[0] is Each of the other components p[x] (for x=1..inSize-1) of the matrix p is a sample. The value of the vector pTemp[x] is equal to the corresponding component of the vector pTemp[x] minus a, but p[0] is a constant The matrix vector product is then calculated. The constant is The average of the possible values, i.e., 2 x-1 (i.e., 1<<(BitDepth-1)) where x indicates the bit depth of the computation representation used. p[0] is replaced by pTemp[0 ], the computed product is calculated using p[0] as above. From the product (p[0]=(1<<(BitDepth-1))-pTemp[0]), This can be taken into account when predicting the interior of a block based on the product, i.e., the prediction vector. Note that the value a deviates by a constant vector. Therefore, the value a is a predetermined value, e.g. pTemp[0]. In this case, the predetermined value pTemp[0] is, for example, a predetermined component p The component of the sample value vector pTemp corresponding to [0] is the component of the sample value vector pTemp corresponding to the given block the sample above or to the left of a given block that is closest to the top-left corner of It can be said that:

[0243] For example, specifying the intra prediction mode, etc. In the case of an in-pull prediction process, the device is configured to apply, for example, the following steps: , e.g., run at least the first step: 1. Matrix base where x=0..predSize-1, y=0..predSize-1 The intra prediction samples predMip[x][y] are derived as follows: The variable modeId is set equal to predModeIntra. -x=0..inSize-1, y=0..predSize*predSize-1 The weight matrix mWeight[x][y] takes mipSizeId and modeId as inputs. The weight matrix is ​​derived by invoking the MIP weight matrix derivation process. - Matrix base where x=0..predSize-1, y=0..predSize-1 The intra prediction samples predMip[x][y] are derived as follows: oW=32-32*(

number

number

[0244] In other words, the device now performs a predMip[x][y] operation to yield the array predMip[x][y]. Already assigned to an array of block positions {x,y} distributed inside a given block To obtain the predicted vector, a matrix vector product is performed between the further vector p[i], Or if mipSizeId is equal to 2, {p[i];pTemp[0]} and If the specified prediction matrix mWeight or mipSizeId is less than 2, the number of p is omitted. To calculate the prediction matrix mWeight with additional zero weight lines corresponding to the omitted components: The prediction vector is constructed as follows: correspond to the concatenation of the columns of p[x][y], respectively.

[0245] According to an embodiment, or according to a different interpretation, the component (((

number

[0246] The apparatus optionally predicts samples of a given block based on a prediction vector. When doing this, use the following steps, e.g., predMip or ((

number

[0247] 2. Matrix vector with x=0..predSize-1, y=0..predSize-1 The intra prediction sample predMip[x][y] of the source is calculated as follows, for example: Uploaded: predMip[x][y]=Clip1(predMip[x][y])

[0248] 3. If transpose is equal to true, then x=0..predSize-1, y=0..predS The predSize×predSize array predMip[x][y] of size-1 is , for example, transposed as follows: predTemp[y][x]=predMip[x][y] predMip=predTemp

[0249] 4. Prediction sample predS where x=0..nTbW-1 and y=0..nTbH-1 For example, amples[x][y] can be derived as follows: - The transform block width nTbW is greater than predSize, or the transform block If the height nTbH is greater than predSize, MIP prediction upsampling is performed. The ring process uses input block size predSize, x=0..predSize -1, y=0..predSize-1 edMip[x][y], transform block width nTbW, transform block height nTbH, x=0 ..nTbW-1 upper reference sample refT[x], and y=0..nTbH It is called with the left reference sample refL[y] with -1 as input, and the output is the predicted sample Luarray predSamples. - otherwise, predS with x=0..nTbW-1, y=0..nTbH-1 amples[x][y] is set equal to predMip[x][y]. In other words, the device calculates the sample of a given block based on the prediction vector predMip. It is configured to predict predSamples.

[0250] FIG. 13 illustrates a method for predicting a given block of a picture using multiple reference samples. A method 2000, comprising forming a sample value vector from a plurality of reference samples. 00 and the sampled vector is transformed into a vector by a given reversible linear transformation. deriving 2200 a further vector to be mapped and To do this, calculate a matrix-vector product between the further vector and the predetermined predictor matrix 230. 0, and predicting 2400 samples of a given block based on the prediction vector. 2 shows a method 2000 including:

[0251] References [1] P. Helle et al., “Non-linear weighted intra p rediction,” JVET-L0199,, Macau, China, October 2018. [2] F.Bossen, J.Boyce, K.Suehring, X.Li,VS eregin, “JVET common test conditions and so ftware reference configurations for SDR vi deo”, JVET-K1010, Ljubljana, Slovenia, July 2018.

[0252] Further Embodiments and Examples Generally, the examples are implemented as a computer program product comprising program instructions. The program instructions may be used to execute the computer program product on a computer. The program instructions, when executed, function to perform one of the methods. It may be stored in the body. Another example is a computer readable medium for performing one of the methods described herein, stored on a machine readable carrier. It is equipped with a computer program for this purpose.

[0253] Thus, an example method is one in which the computer program, when executed on a computer, A computer processor having program instructions for carrying out one of the methods described herein. It is grams.

[0254] Thus, a further example of a method is a computer program for carrying out one of the methods described herein. a data carrier medium (or digital storage medium, or computer-readable medium). The recording medium is tangible and / or non-transitory, rather than an intangible, transitory signal. Thus, a further example of a method is a computer program for carrying out one of the methods described herein. A data stream or sequence of signals representing a computer program. The stream or sequence of signals may be transmitted over a data communication connection, e.g., the Internet. can be transferred.

[0255] A further example is a processing means for performing one of the methods described herein, e.g. a computer It comprises a programmable logic device. A further example is a computer program for carrying out one of the methods described herein. The computer includes a RAM installed thereon.

[0256] A further example is a computer program for carrying out one of the methods described herein. A receiver includes a device or system for transmitting (e.g., electronically or optically) the The receiver may be, for example, a computer, a mobile device, a memory device, etc. The device or system may, for example, include a file system for transferring a computer program to a receiver. A server can be provided.

[0257] In some instances, programmable logic devices (e.g., field programmers) A multi-gate array (GGA) may be used to perform some or all of the functionality of the methods described herein. In some examples, the field programmable gate array may A microprocessor capable of cooperating with the microprocessor to carry out one of the methods described herein. In general, the methods may be performed by any suitable hardware apparatus.

[0258] The above examples are merely illustrative of the principles described above. It is understood that various modifications and variations of the present invention will be apparent to those skilled in the art. The scope of the present claims is not limited to the specific details set forth in the description and explanation. It is intended to be limited by Equal or equivalent elements or elements with equal or equivalent functions occur in different figures. Even if they do, they are indicated in the following description by the same or equivalent reference numerals.

Claims

1. 1. A method for predicting a block of a picture, comprising: obtaining one or more sample values ​​from a plurality of adjacent samples located along a boundary of the block; determining an input value from the one or more sample values ​​based on an indication of a size of the block; determining matrix-based intra-prediction samples by applying a predetermined prediction matrix to the determined input values; predicting samples of the block based on the determined matrix-based intra prediction samples; A method for providing

2. The method of claim 1 , wherein the plurality of adjacent samples are located along a top and left boundary of the block.

3. Determining the input value (p[x]) based on the indication of the size of the block (mipSizeId) comprises: determining p[x] from one or more sample values ​​(pTemp[x]) according to p[x]=pTemp[x+1]-pTemp[0] if mipSizeId is equal to 2; If mipSizeId is less than 2, determining p[x] according to p[x]=pTemp[x]-pTemp[0], where x=1, 2, ..., inSize-1, and inSize corresponds to the number of components of p[x]; The method of claim 1 , comprising:

4. determining a matrix-based intra-predicted sample (predMip) for the determined input value (p[x]) by applying a predetermined prediction matrix (mWeight), [Number 362] 2. The method of claim 1, wherein predSize indicates the number of predicted samples in the block, pTemp[0] is a sample value of one or more sample values ​​(pTemp[x]), and inSize corresponds to the number of components of p[x].

5. obtaining a prediction signal based on the predicted samples of the block; encoding a prediction residual of the block to correct the prediction signal; The method of claim 1 , further comprising encoding the block by:

6. obtaining a prediction signal based on the predicted samples of the block; decoding a prediction residual for said block; correcting the prediction signal using the prediction residual; The method of claim 1 , further comprising: decoding the block by:

7. 1. An apparatus for predicting a block of a picture, comprising: a non-transitory computer-readable medium; At least one processor configured to execute instructions from a non-transitory computer-readable medium to perform the method of any one of claims 1 to 6; An apparatus comprising:

8. A computer program configured, when executed by at least one processor, to cause said at least one processor to perform the method of any one of claims 1 to 6.

9. A computer readable storage medium storing a computer program having a program code for performing the method according to any one of claims 1 to 6 when the computer program is executed on a computer.

Images