Encoding / decoding method and apparatus, and communication system

Abstract

To provide a method, an apparatus and a system for improving the computation time and resource consumption of matrix-based intra prediction.SOLUTION: A method includes: determining prediction parameters of a current block in a bitstream S601; determining a matrix-based intra prediction (MIP) input sample of the current block based on neighboring samples of the current block and the prediction parameters S602; setting a shifting number parameter (sW) as a first fixed value S603; setting a shifting offset parameter (fO) as a second fixed value S604; and determining an MIP prediction sample of the current block based on an MIP weighting matrix, the MIP input sample, the shifting offset parameter (fO), and the shifting number parameter (sW) S606.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] Cross - reference to related applications This application claims the benefit and priority of U.S. Provisional Patent Application No. 62 / 911,166, filed on October 4, 2019, the entire content of which is incorporated herein by reference.

[0002] This disclosure relates to the field of telecommunications technology, and more particularly to methods for encoding and decoding images such as still images and videos.

Background Art

[0003] Versatile Video Coding (VVC) is a next - generation video compression standard used to replace current standards such as the High Efficiency Video Coding standard (H.265 / HEVC). The VVC encoding standard provides higher encoding quality than current standards. To achieve this goal, various intra - prediction modes and inter - prediction modes are considered. When using these prediction modes, the video can be compressed so that the data transmitted in the bitstream (binary format) can be reduced. Matrix - based intra - prediction (MIP) is one such mode. MIP is an intra - prediction mode. When executed in the MIP mode, an encoder or decoder can derive an intra - prediction block based on the current coding block (e.g., a collection of bits or digits that are transmitted as a unit and can be encoded and / or decoded together). However, deriving such a prediction block can require a significant amount of computational resources and additional storage space. Therefore, an improved method to address this problem is beneficial and desirable.

Summary of the Invention

[0004] When executing the MIP process, various prediction parameters are determined and used. Traditionally, some of these prediction parameters can be determined by multiple lookup tables. These lookup tables need to be stored in the encoder and / or decoder components (e.g., memory, cache, etc.), and therefore require memory space. Furthermore, accessing these lookup tables consumes computation time and resources. Therefore, it is beneficial to have improved methods, devices, and systems to address the aforementioned problems.

[0005] This disclosure provides a method for predicting, encoding, and / or decoding an image based on a MIP process. The MIP process can generate a prediction block for the current block, the size of which the prediction block is smaller than the size of the current block. For example, an 8x8 current block may have a 4x4 prediction block. A MIP prediction block smaller than the current block is derived by performing a matrix calculation, which consumes fewer computational resources than performing a matrix calculation for a larger block. After the matrix calculation, an upsampling process is applied to the MIP prediction block to derive an intra-prediction block the same size as the current block. For example, an 8x8 intra-prediction block can be derived from a 4x4 MIP prediction block by calling an interpolation and / or extrapolation upsampling process.

[0006] More specifically, this method involves, for example, (i) determining the prediction parameters of the current block in the bitstream, (ii) determining the MIP input sample of the current block (e.g., "p[x]" in equations (P-1), (P-2), and (P-3) discussed in detail below) based on the adjacent samples and prediction parameters of the current block, (iii) setting the numerical shift parameter (e.g., "sW" in equation (B) discussed in detail below) to a first fixed value, and (iv) setting the offset shift parameter (e.g., "fO" in equation (B) discussed in detail below) to a second fixed value. (v) determining the MIP weighting matrix of the current block based on prediction parameters; (vi) determining the MIP prediction samples of the current block (e.g., "predMip[x][y]" in equation (C), which will be discussed in detail below) based on the MIP weighting matrix, MIP input samples, offset shift parameter "fO", and numerical shift parameter (sW); and (vii) performing an upsampling process on the MIP prediction samples to generate intra prediction samples (e.g., "predSamples[x][y]" in equation (G), which will be discussed in detail below).

[0007] While we do not wish to be constrained by theory, setting either or both of the numerical shift parameter and the offset shift parameter to fixed values ​​improves the overall encoding / decoding efficiency without significantly impacting the accuracy of the encoding / decoding process. This arrangement provides a solution that significantly reduces computation time and the required memory space when executing the MIP process.

[0008] Another aspect of this disclosure includes a system for encoding / decoding images and videos. This system may include an encoding subsystem (or encoder) and a decoding subsystem (or decoder). The encoding subsystem includes a splitter, a first prediction unit, and an entropy encoding unit. The splitter is configured to receive an input video and split the input video into one or more encoding units (CUs). The first intra-prediction unit is configured to generate prediction blocks corresponding to each CU based on prediction parameters derived from the encoding of the input video. The entropy encoding unit is configured to convert the parameters for deriving the prediction blocks into a bitstream. The decoding subsystem includes a parser and a second intra-prediction unit. The parser is configured to parse the bitstream to obtain numerical values ​​(e.g., values ​​associated with one or more CUs). The second intra-prediction unit is configured to convert the numerical values ​​into an output video based on the prediction parameters. [Brief explanation of the drawing]

[0009] To make the technical solutions described in the embodiments of this disclosure clearer, the drawings used to describe the embodiments are briefly described below. Obviously, the drawings described below are for illustrative purposes only and not limiting. Those skilled in the art should understand that they can obtain other drawings based on these without original work.

[0010] [Figure 1A] This is a schematic diagram of a system according to one embodiment of the present disclosure. [Figure 1B] This is a schematic diagram showing an MIP process according to an embodiment of the present disclosure. [Figure 2] This is a schematic diagram of an encoding system according to one embodiment of the present disclosure. [Figure 3] This is a schematic diagram showing the derivation of an intra-prediction block using the MIP mode according to the embodiment of this disclosure. [Figure 4] This is a schematic diagram of a decoding system according to one embodiment of the present disclosure. [Figure 5]This is a flowchart showing the method of the embodiment of the disclosure. [Figure 6] This flowchart shows the method according to the embodiments of the disclosed embodiment. [Figure 7] This is a schematic diagram of a device (e.g., an encoder) according to one embodiment of the present disclosure. [Figure 8] This is a schematic diagram of a device (e.g., an encoder) according to one embodiment of the present disclosure. [Figure 9] This is a schematic diagram of a device (e.g., a decoder) according to one embodiment of the present disclosure. [Figure 10] This is a schematic diagram of a device (e.g., a decoder) according to one embodiment of the present disclosure. [Figure 11] This is a schematic diagram of a device (e.g., an encoder) according to one embodiment of the present disclosure. [Figure 12] This is a schematic diagram of a device (e.g., a decoder) according to one embodiment of the present disclosure. [Figure 13] A schematic diagram of a communication system according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0011] To facilitate understanding of this disclosure, it is described in more detail below with reference to the attached drawings.

[0012] Figure 1A is a schematic diagram of a system 100 according to one embodiment of the present disclosure. The system 100 can encode, transmit, and decode images. The system 100 can also be applied to encode, transmit, and decode a video consisting of a series of images. More specifically, the system 100 can receive an input image, process the input image, and generate an output image. The system 100 includes an encoding device 100a and a decoding device 100b. The encoding device 100a includes a splitting unit 101, a first intra-prediction unit 103, and an entropy encoding unit 105. The decoding device 100b includes a syntactic analysis unit 107 and a second intra-prediction unit 109.

[0013] The splitting unit 101 is configured to receive the input video 10 and split the input video 10 into one or more coded tree units (CTUs) or coded units (CUs) 12. One or more CUs 12 are sent to the first intra-prediction unit 103. The first intra-prediction unit 103 is configured to derive a prediction block for each of the one or more CUs 12 by performing an MIP process. In some embodiments, the MIP process has different techniques based on the size of the CUs 12 to handle CUs 12 of different sizes. For example, for each type of one or more CUs 12, the MIP process has a designated MIP size identifier (e.g., 0, 1, 2, etc.).

[0014] The first intra-prediction unit 103 first determines the prediction parameters of CU12 (e.g., width, height, size, etc.). Accordingly, the first intra-prediction unit 103 can determine the MIP size identifier of CU12. The first intra-prediction unit 103 further derives a set of reference samples for CU12 (for example, using adjacent samples of CU12, such as upper adjacent samples and left adjacent samples, which will be discussed in detail with reference to Figure 3). Then, the first intra-prediction unit 103 derives the MIP prediction of CU12 based on the set of reference samples and the corresponding MIP weighting matrix. The first intra-prediction unit 103 can use the MIP prediction as the intra-prediction 14 of CU12. The intra-prediction 14 and the prediction parameters for deriving the intra-prediction 14 are then sent to the entropy coding unit 105 for further processing.

[0015] The entropy coding unit 105 is configured to convert prediction parameters for deriving the intra-prediction 14 into binary format. Accordingly, the entropy coding unit 105 generates a bitstream 16 based on the intra-prediction 14. In some embodiments, the bitstream 16 can be transmitted over a communication network or stored on disk or a server.

[0016] The decoding device 100b receives the bit stream 16 as the input bit stream 17. The syntax analysis unit 107 analyzes the input bit stream 17 (binary format) and converts it into a numerical value 18. The numerical value 18 indicates the features (e.g., color, brightness, density, etc.) of the input video 10. The numerical value 18 is transmitted to the second intra prediction unit 109. Then, the second intra prediction unit 109 can convert these numerical values 18 into the output video 19 (based on, for example, the same process as that performed by the first intra prediction unit 103, and related embodiments are discussed in detail with reference to FIG. 4). Thereafter, the output video 19 can be stored, transmitted, and / or drawn by an external device (e.g., a storage device, a transmitter, etc.). The stored video can be further displayed by a display device.

[0017] FIG. 1B is a schematic diagram showing the MIP process S100 according to an embodiment of the present disclosure. The MIP process includes four main steps, S101, S102, S103, and S104. In step S101, prediction parameters for the MIP process S100 are determined. The prediction parameters include the type or size of the current block (e.g., indicated by a MIP size identifier such as "mipSizeId"), the number of reference samples within each boundary (e.g., "boundySize"), the number of MIP input samples (e.g., "inSize"), and the dimensions of the MIP prediction block generated by matrix multiplication (e.g., "predSize×predSize"). Based on the size of the current block, the MIP size identifier can be "0", "1", or "2", and the MIP size identifier is recorded in the parameter "mipSizeId". Different blocks may have different prediction parameters.

[0018] In step S102, the MIP process acquires reference pixels. The reference pixels can be taken from the upper adjacent block and / or the left adjacent block. The pixels from the upper adjacent block are stored as the parameter "refT", and the pixels from the left adjacent block are stored as the parameter "refL".

[0019] In step S103, an input sample for the MIP process is determined. The input sample can be determined by three sub-steps, S1031, S1032, and S1033. In sub-step S1031, a downsampling process is performed based on reference pixels (e.g., from step S102) to generate a reference sample. In sub-step S1031, a reference sampling area (or buffer sampling area) is determined. In some embodiments, a matrix transposition process may be included in sub-step S1032. In sub-step S1033, matrix multiplication calculations are performed based on the input sample.

[0020] In step S104, an intra MIP prediction sample is generated. Step S104 includes four sub-steps, S1041, S1042, S1043, and S1044. In sub-step S1041, a MIP prediction block is generated based on a MIP weighting matrix, an offset shift parameter (e.g., "fO"), and a numerical shift parameter (e.g., "sW"). In the present disclosure, the offset shift parameter and the numerical shift parameter are set to fixed values. In sub-step S1042, an interpolation process is performed on the MIP prediction block. In sub-step S1043, the MIP prediction block is transposed. In sub-step S1044, an intra MIP prediction sample is generated. In some embodiments, the intra MIP prediction sample can be generated by considering prediction samples from upper adjacent pixels. In other embodiments, the intra MIP prediction sample can be generated without considering prediction samples from upper adjacent pixels (e.g., by only considering prediction samples from left adjacent pixels). Details of the MIP process are discussed below with reference to FIG. 3 and the corresponding equations.

[0021] Figure 2 is a schematic diagram of an encoding system 200 according to one embodiment of the present disclosure. The encoding system 200 is configured to encode, compress, and / or process an input image 20 to generate an output bitstream 21 in binary format. The encoding system 200 includes a splitter 201 configured to split the input image 20 into one or more encoding tree units (CTUs) 22. In some embodiments, the splitter 201 can split the image into slices, tiles, and / or bricks. Each brick may contain one or more essential CTUs 22 and / or partial CTUs 22. In some embodiments, the splitter 201 may also form one or more subpictures, each of which may contain one or more slices, tiles, or bricks. The splitter 201 sends the CTUs 22 to a prediction unit 202 for further processing.

[0022] The prediction unit 202 is configured to generate prediction blocks 23 for each of the CTUs 22. The prediction blocks 23 can be generated by using various interpolation and / or extrapolation schemes based on one or more interprediction or intraprediction methods. As shown in Figure 2, the prediction unit 202 may further include a block division unit 203, a motion estimation (ME) unit 204, a motion compensation (MC) unit 205, and an intraprediction unit 206. The block division unit 203 is configured to divide the CTUs 22 into smaller coded units (CUs) or coded blocks (CBs). In some embodiments, CUs can be generated from the CTUs 22 by various methods such as quadtree division, binary tree division, and ternary tree division. The ME unit 204 is configured to estimate the motion of an object shown in the input image 20, or the changes caused by the motion of the image acquisition device that generates the input image 20. The MC unit 205 is configured to adjust and compensate for the changes caused by the aforementioned motion. Both the ME unit 204 and the MC unit 205 are configured to derive interprediction blocks of the CU (or a portion of the CU) at different time points. In some embodiments, the ME unit 204 and the MC unit 205 can derive interprediction blocks using a rate-distortion optimized motion estimation method.

[0023] The intra-prediction unit 206 is configured to derive an intra-prediction block (at the same time point) of the CU (or a portion of the CU) using various intra-prediction modes, including the MIP mode. Details of the derivation of the intra-prediction block using the MIP mode (hereinafter referred to as the "MIP process") are discussed with reference to Figures 1B and 3. During the MIP process, the intra-prediction unit 206 first derives one or more reference samples from the adjacent samples of the CU, for example, by directly using adjacent samples as reference samples, downsampling adjacent samples, or directly extracting from adjacent samples (e.g., step 301 in Figure 3).

[0024] Secondly, the intra-prediction unit 206 derives predicted samples at a number of sample locations within the CU using a reference sample, the MIP matrix, and shift parameters. The sample locations can be pre-set sample locations within the CU. For example, the sample locations can be locations within the CU with odd horizontal and vertical coordinate values ​​(e.g., x=1, 3, 5, etc., and y=1, 3, 5, etc.). The shift parameters include offset shift parameters and numerical shift parameters, which can be used in the shift calculation when generating predicted samples. This arrangement allows the intra-prediction unit 206 to generate predicted samples in the CU (i.e., "MIP prediction" or "MIP prediction block" refers to a group of such predicted samples) (e.g., step 302 in Figure 3). In some embodiments, the sample locations can be locations within the CU with even horizontal and vertical coordinate values.

[0025] Thirdly, the intra-prediction unit 206 can derive predicted samples at the remaining positions of the CUs (e.g., they are not sample positions) (e.g., step 303 in Figure 3). In some embodiments, the intra-prediction unit 206 can use interpolation filters to derive predicted samples at the remaining positions. Through the processes described above, the intra-prediction unit 206 can generate predicted blocks 23 for the CUs in the CTU 22.

[0026] Referring to Figure 2, the prediction unit 202 outputs the prediction block 23 to the adder 207. The adder 207 calculates the difference (e.g., residual R) between the output of the division unit 201 (e.g., CU in CTU 22) and the output of the prediction block 202 (i.e., the prediction block 23 of CU). The transformation unit 208 reads the residual R, performs one or more transformation operations on the prediction block 23, and obtains coefficients 24 to be used later. The quantization unit 209 can quantize the coefficients 24 and output the quantized coefficients 25 (e.g., levels) to the inverse quantization unit 210. The inverse quantization unit 210 performs a scaling operation on the quantized coefficients 25 and outputs the reconstructed coefficients 26 to the inverse transformation unit 211. The inverse transformation unit 211 performs one or more inverse transformations corresponding to the transformations in the transformation unit 208 and outputs the reconstructed residuals 27.

[0027] The adder 212 then calculates the reconstructed CU by adding the prediction block 23 of the CU from the prediction unit 202 to the reconstructed residual 27. The adder 212 also sends its output 28, which is used as an intra-prediction reference, to the prediction unit 202. After all CUs in the CTU 22 have been reconstructed, the filter processing unit 213 can perform in-loop filtering on the reconstructed picture 29. The filter processing unit 213 includes one or more filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luminance mapping (LMCS) filter with chroma scaling, a neural network-based filter, and other suitable filters to suppress encoding distortion or improve the encoding quality of the picture.

[0028] Subsequently, the filter processing unit 213 can send the decoded picture 30 (or subpicture) to the decoded picture buffer (DPB) 214. The DPB 214 outputs the decoded picture 31 based on the control information. The picture 31 stored in the DPB 214 may also be used as a reference picture for inter-prediction or intra-prediction by the prediction unit 202.

[0029] The entropy coding unit 215 is configured to convert the picture 31, parameters from each part of the coding system 200, and supplementary information (for example, information for controlling or communicating with the system 200) into binary format. The entropy coding unit 215 can generate the output bitstream 21 accordingly.

[0030] In some embodiments, the encoding system 200 may be a computer device comprising a processor and a storage medium having one or more encoding programs. When the processor reads and executes an encoding program, the encoding system 200 can receive an input image 20 and generate an output bitstream 21 accordingly. In some embodiments, the encoding system 200 may be a computer device comprising one or more chips. Each part or element of the encoding system 200 may be implemented as an integrated circuit on a chip.

[0031] Figure 3 is a schematic diagram showing the MIP process according to an embodiment of the present disclosure. The MIP process can be carried out by an intra-prediction unit (e.g., intra-prediction unit 206). As shown in Figure 3, the intra-prediction unit may include a prediction module 301 and a filtering module 302. Also as shown in Figure 3, the MIP process includes three steps 301, 302, and 303. The MIP process can generate a prediction block based on the current block or encoded block 300 (such as a CU or multiple segments of a CU).

[0032] Step 301 In step 301, the intra-prediction unit can generate reference samples 32 and 34 using adjacent samples 31 and 33 of the coded block 300. In the shown embodiment, adjacent sample 31 is the upper adjacent sample, and adjacent sample 33 is the left adjacent sample. The intra-prediction unit 206 can calculate the average of the values ​​of every two samples from adjacent samples 31 and 33, and set these average values ​​as the values ​​for reference samples 32 and 34, respectively. In some embodiments, the intra-prediction unit 206 can select one value from each of the two samples from adjacent samples 31 and 33 as the value for reference sample 32 or 34. In the shown embodiment, the intra-prediction unit 206 derives four reference samples 32 from the eight upper adjacent samples 31 of the coded block 300, and another four reference samples 34 from the eight left adjacent samples 33 of the coded block 300.

[0033] In step 301, the intra-prediction unit determines the width and height of the encoded block 300, which are denoted by the variables "cbWidth" and "cbHeight," respectively. In some embodiments, the intra-prediction unit 206 can employ a rate-distortion optimized mode determination process to determine the intra-prediction mode (e.g., whether the MIP mode is used). In such embodiments, the encoded block 300 can be divided into one or more transform blocks, the width and height of which are denoted by the variables "nTbW" and "nTbH," respectively. If the MIP mode is used as the intra-prediction mode, the intra-prediction unit determines the MIP size identifier (denoted by the variable "mipSizeId") based on the following conditions A to C. [Condition A] If both "nTbW" and "nTbH" are 4, set "mipSizeId" to 0. [Condition B] If not, set "mipSizeId" to 1 if "cbWidth" or "cbHeight" is 4. [Condition C] If none of the above apply, set "mipSizeId" to 2.

[0034] For example, if the size of encoding block 300 is "8x8" (i.e., both "cbWidth" and "cbHeight" are 8), then "mipSizeId" is set to 2. Another example is if the size of the block to be transformed by encoding block 300 is "4x4" (i.e., both "nTbW" and "nTbH" are 4), then "mipSizeId" is set to 0. Yet another example is if the size of encoding block 300 is "4x8", then "mipSizeId" is set to 1.

[0035] In the embodiments shown, there are three types of “mipSizeId”, namely “0”, “1”, and “2”. Each type of MIP size identifier (i.e., the variable “mipSizeId”) corresponds to a specific way of performing the MIP process (e.g., using different MIP matrices). In other embodiments, there may be four or more types of MIP size identifiers.

[0036] Based on the MIP size identifier, the intra-prediction unit can determine the variables ("Size" and "predSize") based on the following conditions D to F. [Condition D] If "mipSizeId" is 0, set "boundarySize" to 2 and "predSize" to 4. [Condition E] If "mipSizeId" is 1, set "boundarySize" to 4 and "predSize" to 4. [Condition F] If "mipSizeId" is 2, set "boundarySize" to 4 and "predSize" to 8.

[0037] In the embodiment shown, "boundarySize" represents the number of reference samples 32 and 34 derived from the upper adjacent sample 31 and the left adjacent sample 33 of the coded block 300, respectively. The variable "predSize" is used in the following calculation (i.e., equation (C) below).

[0038] Furthermore, in some embodiments, the intra-prediction unit can derive a variable "isTransposed" to indicate the order of reference samples 32 and 34 stored in the temporal array. For example, "isTransposed" being "0" indicates that the intra-prediction unit presents reference sample 32 derived from the upper adjacent sample 31 of the encoded block 300 before reference sample 34 derived from the left adjacent sample 33. Alternatively, "isTransposed" being "1" indicates that the intra-prediction unit presents reference sample 34 derived from the left adjacent sample 33 of the encoded block 300 before reference sample 32 derived from the upper adjacent sample 31. In the implementation of the encoding system 200, the value of "isTransposed" is sent to the entropy encoding unit (e.g., entropy encoding unit 215) as one of the parameters of the MIP process that is encoded and written to a bitstream (e.g., output bitstream 21). Accordingly, in the implementation of the decoding system 400 shown in Figure 4 of this disclosure, the value of "isTransposed" can be received from the parser (e.g., parser 401) by parsing the input bitstream (which may be the output bitstream 21).

[0039] The intra-prediction unit can further determine the variable "inSize" to indicate the number of reference samples 32 and 34 used when deriving the MIP prediction. The value of "inSize" is determined by the following equation (A). In this disclosure, the meaning and operation of all operators in the equation are the same as the corresponding operators as defined in the ITU-T H.265 standard.

[0040] inSize = (2 * boundarySize) - (mipSizeId = = 2) ? 1:0; (A) For example, "==" is a comparison operator meaning "equal to". For example, if "mipSizeId" is 2, then "inSize" is 7 (calculated by (2 × 4) - 1). For example, if "mipSizeId" is 1, then "inSize" is 8 (calculated by (2 × 4) - 0). In some embodiments, the parameter "inSize" can be found in the following Size-Id table.

[0041] [Table 1]

[0042] The intra-prediction unit can derive reference sample sets 32 and 34 to be stored in the array p[x] (where "x" ranges from "0" to "inSize-1") by calling the following process. The intra-prediction unit can derive "nTbW" samples from the upper adjacent sample 31 of the coded block 300 (and store them in the array "refT"), and can also derive "nTbH" samples from the left adjacent sample 33 of the coded block 300 (and store them in the array "refL").

[0043] The intra prediction unit can assign an initial letter to the downsampling process on "refT", obtain "boundarySize" samples, and store the "boundarySize samples" in "refT". The intra prediction unit 206 can start a downsampling process on "refL", obtain "boundarySize" samples, and store the "boundarySize" samples in "refL".

[0044] In some embodiments, the intra-prediction unit can combine the arrays "refT" and "refL" into a single array "pTemp" based on the order indicated by the variable "isTransposed". The intra-prediction unit can derive "isTransposed" to indicate the order of reference samples stored in the temporal array "pTemp". For example, if "isTransposed" is "0" (or FALSE), it indicates that the intra-prediction unit presents the reference sample 32 derived from the upper adjacent sample 31 of the coded block 300 before the reference sample 34 derived from the left adjacent sample 33. In other cases, if "isTransposed" is "1" (or TRUE), it indicates that the intra-prediction unit presents the reference sample 34 derived from the left adjacent sample 33 of the coded block 300 before the reference sample 32 derived from the upper adjacent sample 31. In some embodiments, in the implementation of the coding system 200, the intra-prediction unit can determine the value of "isTransposed" by using a rate-distortion optimization method. In some embodiments, in the implementation of the encoding system 200, the intra prediction unit can determine the value of "isTransposed" based on a comparison and / or correlation between adjacent samples 32, 34 and the encoding block 300. In the implementation of the encoding system 200, the value of "isTransposed" can be sent to the entropy encoding unit (e.g., entropy encoding unit 215) as one of the parameters of the MIP process that is written to the bitstream (e.g., output bitstream 21). Accordingly, in the implementation of the decoding system 400 shown in Figure 4 described herein, the value of "isTransposed" can be received from the parser (e.g., parser 401) by parsing the input bitstream (which may be the output bitstream 21).

[0045] In some embodiments, the parameter "mipSizeId" can be used to determine whether the selected size parameter of the current block is within a given range. More specifically, "mipSizeId" equal to "0" or "1" means that the selected size parameter is within the given range. If "mipSizeId" is equal to "2", it means that the selected size parameter is not within the given range. That is, the MIP input samples can be determined based on the buffer area (e.g., "pTemp"), the MIP size identifier (e.g., "mipSizeId"), and the bit depth of adjacent samples (e.g., "BitDepth"), as will be discussed in more detail below.

[0046] The intra prediction unit can determine the array "p[x]" (where x ranges from "0" to "inSize-1") based on the following conditions G and H. [Condition G] If "mipSizeId" is 2, then p[x] = pTemp[x+1] - pTemp[0]. [Condition H] Otherwise (for example, if "mipSizeId" is less than 2), p[0] = pTemp[0] - (1 << (BitDepth - 1)) and p[x] = pTemp[x] - pTemp[0] (for x from 1 to "inSize - 1").

[0047] In the above condition H, "BitDepth" is the bit depth of the color component (e.g., Y component) of the sample within coding block 300. The symbol "<<" is the bit shift operator used in the ITU-T H.265 standard.

[0048] Alternatively, the intra prediction unit can derive the sequence p[x] (for x from "0" to "inSize-1") based on the following conditions I and J. [Condition I] If "mipSizeId" is 2, then p[x] = pTemp[x+1] - pTemp[0]. [Condition J] Otherwise (for example, if "mipSizeId" is less than 2), p[0]=(1<<(BitDepth-1))-pTemp[0] and p[x]=pTemp[x]-pTemp[0] (for x from 1 to "inSize-1").

[0049] In some embodiments, the intra-prediction unit can determine the values ​​of the array p[x] by using a unified calculation method without determining the value of "mipSizeId". For example, the intra-prediction unit can add "(1<<(BitDepth-1))" as an additional element to "pTemp", and calculate p[x] as "pTemp[x]-pTemp[0]".

[0050] In some embodiments, the parameter "(1<<(BitDepth-1))" can be a constant. In some embodiments, the parameter "(1<<(BitDepth-1))" can be expressed as "2^(BitDepth-1)". Accordingly, the parameters "p[0]" and "p[x]" can be calculated based on the following equations P-1, P-2, and P-3.

[0051]

number

number

number

[0052] Equations P-1 and P-2 apply when "mipSizeId" is equal to "0" or "1", meaning that the selected size parameter is within a predetermined range. After determining that the selected size parameter of the current block is within a predetermined range, "1<<(BitDepth-1)" can be determined. Then, the difference between "1<<(BitDepth-1)" and "pTemp[0]" can be determined. The difference is set to p[0]. Then, p[x] can be calculated based on the above equations P-1 and P-2.

[0053] Equation P-3 applies when "mipSizeId" is equal to "2", meaning the selected size parameter is not within the given range. Assuming the current block is a 4x4 block, this means there are four values ​​in the buffer area "pTemp". These four values ​​are pTemp[0], pTemp[1], pTemp[2], and pTemp[3]. Based on the above equation P-3 and these four values, the array "p[x]" can be determined. The values ​​in the array "p[x]" are sometimes called MIP input samples.

[0054] Step 302 In step 302, the intra-prediction unit (or prediction module 301) derives the MIP prediction for the coding block 300 using the reference sample sets 32 and 34 and the MIP matrix. The MIP matrix is ​​selected from a predefined set of MIP matrices based on its corresponding MIP mode identifier (i.e., the variable "mipModeId") and MIP size identifier (i.e., the variable "mipSizeId").

[0055] The MIP prediction derived by the intra-prediction unit includes partial prediction samples 35 for all or some of the sample locations within the coded block 300. The MIP prediction is represented as "predMip[x][y]".

[0056] In the embodiment shown in Figure 3, the partial prediction sample 35 is the sample displayed as a gray square within the current block 300. The reference samples 32 and 34 in the array "p[x]", derived in step 301, are used as input to the prediction module 301. The prediction module 301 calculates the partial prediction sample 35 using the MIP matrix and shift parameters. The shift parameters include an offset shift parameter ("fO") and a numerical shift parameter ("sW"). In some embodiments, the prediction module 301 derives a partial prediction sample 35 having coordinates (x, y) based on equations (B) and (C).

[0057]

number

[0058]

number

[0059] In equation (B) above, the parameter "fO" is an offset shift parameter used to determine the parameter "oW" or the offset value. The parameter "sW" is a numerical shift parameter. "p[i]" is a reference sample. The symbol ">>" is the binary right shift operator as defined in the H.265 standard.

[0060] In equation (C) above, "mWeight[i][j]" is the MIP weighting matrix, and the matrix elements are fixed constants in both encoding and decoding. Alternatively, in some embodiments, an adaptive MIP matrix is ​​used in the implementation of the encoding system 200. For example, the MIP weighting matrix can be updated by various training methods using one or more encoded pictures as input, or using images supplied to the encoding system 200 by external means. The intra-prediction unit can send "mWeight[i][j]" to the entropy encoding unit (e.g., entropy encoding unit 215) when the MIP mode is determined. The entropy encoding unit can then write "mWeight[i][j]" into the bitstream, for example, into one or more special data units in the bitstream containing the MIP data. Accordingly, in some embodiments, the implementation of the decoding system 400 using an adaptive MIP matrix can update the MIP matrix using a training method that inputs, for example, one or more encoded pictures or blocks, or pictures from other bitstreams supplied by external meaning, or obtained by the parsing unit 401 parsing special data units in the input bitstream containing the MIP matrix data.

[0061] The prediction unit 301 can determine the values ​​of "sW" and "fO" based on the size of the current block 300 and the MIP mode used for the current block 300. In some embodiments, the prediction unit 301 can obtain the values ​​of "sW" and "fO" by using a lookup table. For example, "sW" can be determined using Table 1 below.

[0062] [Table 2]

[0063] In some embodiments, the numerical shift parameter "sW" can be set to a first fixed value such as 5 or 6. In such embodiments, it is not necessary to use Table 1 above to determine the value of the numerical shift parameter "sW". For example, if "mipSizeId" is equal to "0" or "2", the numerical shift parameter "sW" can be set to "5". As another example, if "mipSizeId" is equal to "2", the numerical shift parameter "sW" can be set to "5". Table 2 below shows different settings for the numerical shift parameter "sW". In some embodiments, the numerical shift parameter "sW" can be set by the prediction module 301.

[0064] [Table 3]

[0065] In some embodiments, the prediction module can set "sW" to a constant. For example, the prediction module can set "sW" to "5" for blocks of various sizes having different MIP modes. As another example, the prediction module 301 can set "sW" to "6" for blocks of various sizes having different MIP modes. As yet another example, the prediction module can set "sW" to "7" for blocks of various sizes having different MIP modes.

[0066] In some embodiments, the prediction unit 301 can determine the offset shift parameter "fO" using Table 3 below.

[0067] [Table 4]

[0068] In some embodiments, the offset shift parameter "fO" can be set to a second fixed value such as 23, 32, 46, 56, or 66. The second fixed value has a preferred range of 1 to 100. In such embodiments, it is not necessary to use Table 3 above to determine the value of the offset shift parameter "fO". In some embodiments, the offset shift parameter "fO" can be set based on the parameter "mipSizeId". For example, if "mipSizeId" is equal to "0", the offset shift parameter "fO" can be set to "34". As another example, if "mipSizeId" is equal to "1", the offset shift parameter "fO" can be set to "23". As yet another example, if "mipSizeId" is equal to "1", the offset shift parameter "fO" can be set to "46". Table 4 below shows different settings for the offset shift parameter "fO". In some embodiments, the offset shift parameter "fO" can be set by the prediction module 301.

[0069] [Table 5]

[0070] Referring to equation (B) above, if the numerical shift parameter "sW" and the offset shift parameter "fO" are set to fixed values, the parameter "oW" can also be set to a constant, thereby improving the overall calculation speed of the MIP process.

[0071] In some embodiments, the intra prediction unit can perform a clipping operation on the values ​​of the MIP prediction samples stored in the array "predMip". If "isTransposed" is 1 (or TRUE), the array "predMip[x][y]" of "predSize × preSize" (for x from 0 to "predSize-1", and for y from 0 to "predSize-1") is transposed to "predTemp[y][x] = predMip[x][y]", resulting in "predMip = predTemp".

[0072] More specifically, if the size of the coded block 303 is "8x8" (i.e., both "cbWidth" and "cbHeight" are 8), the intra prediction unit can derive an "8x8" "predMip" sequence.

[0073] Step 303 In step 303 of Figure 3, the intra-prediction unit derives predicted samples 37 of the remaining samples in the coded block 300 other than the partial sample 35. As shown in Figure 3, the intra-prediction unit can derive predicted samples 37 of the remaining samples in the coded block 300 other than the partial sample 35 using the filtering module 302. The input to the filtering module 302 can be the partial sample 35 in step 302. The filtering module 302 can derive predicted samples 37 of the remaining samples in the coded block 300 other than the partial sample 35 using one or more interpolation filters. The intra-prediction unit (or filtering module 302) can generate predictions (including a number of predicted samples 37) for the coded block 300 and store the predictions 37 in the array "predSamples[x][y]" (for x from 0 to "nTbW-1" and for y from 0 to "nTbH-1") based on the following conditions K and L.

[0074] [Condition K] If the intra prediction unit determines that "nTbW" is greater than "predSize" or "nTbH" is greater than "predSize", the intra prediction unit starts the upsampling process and derives "predSamples" based on "predMip".

[0075] [Condition L] Otherwise, the intra prediction unit sets the prediction of coding block 300 to the MIP prediction of coding block.

[0076] In other words, the intra prediction unit can set "predSamples[x][y]" (for x from 0 to "nTbW-1" and for y from 0 to "nTbH-1") to be equal to "predMip[x][y]". For example, the intra prediction unit can set "predSamples" for an encoded block whose size is equal to "8x8" (i.e., both "cbWidth" and "cbHeight" are 8) to its "predMip[x][y]".

[0077] Through steps 301 to 303, the intra-prediction unit can generate a prediction for the current block 300. The generated prediction can be used in further processes (for example, the prediction block 23 described above, see Figure 2).

[0078] Figure 4 is a schematic diagram of a decoding system 400 according to one embodiment of the present disclosure. The decoding system 400 is configured to receive and process an input bitstream 40 and convert it into an output video 41. The input bitstream 40 may be a bitstream representing a compressed / encoded picture / video. In some embodiments, the input bitstream 40 may be from an output bitstream (e.g., output bitstream 21) generated by an encoding system (e.g., encoding system 200).

[0079] The decoding system 400 includes a parser 401 configured to analyze the input bitstream 40 to obtain the values ​​of its syntax elements. The parser 401 also converts the binary representation of the syntax elements into numerical values ​​(i.e., decoding blocks 42) and sends these numerical values ​​to a prediction unit 402 (for example, for decoding). In some embodiments, the parser 401 may also send one or more variables and / or parameters to the prediction unit 402 for decoding the numerical values.

[0080] The prediction unit 402 is configured to determine the decoded block 42 (for example, a prediction block 43 of a portion of a CU, such as a CU or a transform block). If it is indicated that an inter-encoded mode is used to decode the decoded block 42, the motion compensation (MC) unit 403 of the prediction unit 402 receives the relevant parameters from the parsing unit 401 and can decode in the inter-encoded mode accordingly. If it is indicated that an intra-prediction mode (for example, MIP mode) is used to decode the decoded block 42, the intra-prediction unit 404 of the prediction unit 402 receives the relevant parameters from the parsing unit 401 and can decode in the intra-encoded mode accordingly. In some embodiments, the intra-prediction mode (for example, MIP mode) can be identified by a specific flag (for example, an MIP flag) embedded in the input bitstream 40.

[0081] For example, if the MIP mode is identified, the intra-prediction unit 404 can determine the prediction block 43 (containing a number of prediction samples) based on the following method (similar to steps 301-303 described in Figure 3).

[0082] First, the intra-prediction unit 404 derives one or more reference samples from adjacent samples in the decoded block 42 (similar to step 301 in Figure 3). For example, the intra-prediction unit 404 can generate reference samples by downsampling adjacent samples or by directly extracting a portion from adjacent samples.

[0083] The intra-prediction unit 404 can then derive partial prediction samples within the decoding block 42 using the reference sample, MIP matrix, and shift parameters (similar to step 302 in Figure 3). In some embodiments, the positions of the partial prediction samples can be pre-set within the decoding clock 42. For example, the positions of the partial prediction samples can be positions with odd horizontal and vertical coordinate values ​​within the coding block. The shift parameters may include offset shift parameters and numerical shift parameters, which can be used in the shift operations when generating the partial prediction samples.

[0084] Finally, once the partially predicted sample of the decoded block 42 has been derived, the intra-prediction unit 404 derives the predicted sample of the remaining sample in the decoded block 42 other than the partially predicted sample (similar to step 303 in Figure 3). For example, the intra-prediction unit 404 can derive the predicted sample by using an interpolation filter, with the partially predicted sample and adjacent samples as inputs to the interpolation filter.

[0085] The decoding system 400 includes a scaling unit 405 that has a function similar to that of the inverse quantization unit 210 of the encoding system 200. The scaling unit 405 performs a scaling operation on the quantized coefficients 44 (e.g., levels) from the parsing unit 401 to generate reconstructed coefficients 45.

[0086] The transformation unit 406 has a function similar to that of the inverse transformation unit 211 of the encoding system 200. The transformation unit 406 performs one or more transformation operations (for example, the inverse operation of one or more transformation operations performed by the inverse transformation unit 211) to obtain the reconstructed residual 46.

[0087] The adder 407 adds the predicted block 43 from the prediction unit 402 and the reconstructed residual 46 from the transform unit 406 to obtain the reconstructed block 47 of the decoded block 42. The reconstructed block 47 is then sent to the prediction unit 402 to be used as a reference (for example, for other blocks encoded in intra-prediction mode).

[0088] After all decoded blocks 42 within a picture or subpicture have been reconstructed (i.e., a reconstructed block 48 has been formed), the filter processing unit 408 can perform in-loop filtering on the reconstructed block 49. The filter processing unit 408 includes one or more filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luminance mapping (LMCS) filter with chromatic difference scaling, and a neural network-based filter. In some embodiments, the filter processing unit 408 can perform in-loop filtering on only one or more target pixels within the reconstructed block 48.

[0089] The filter processing unit 408 then sends the decoded picture 49 (or picture) or subpicture to the DPB (decoded picture buffer) 409. The DPB 409 outputs the decoded picture as output video 41 based on timing and control information. The decoded picture 49 stored in the DPB 409 can also be used as a reference picture by the prediction unit 402 when performing interpretation or intrapretation.

[0090] In some embodiments, the decoding system 400 may be a computer device comprising a processor and a storage medium recording one or more decoding programs. Once the processor reads and executes a decoding program, the decoding system 400 can receive an input video bitstream and generate a corresponding decoded video.

[0091] In some embodiments, the decoding system 400 may be a computer device comprising one or more chips. Each part or element of the decoding system 400 may be implemented as an integrated circuit on a chip.

[0092] Figure 5 is a flowchart of Method 500 according to one embodiment of the present disclosure. Method 500 begins in block 501 by determining the prediction parameters for the current block. In some embodiments, the prediction parameters include a parameter for prediction (e.g., “predModeIntra” as defined in the H.265 standard) and a size identifier (e.g., “mipSizeId”). In block 502, if the prediction parameters indicate that an MIP mode is applicable, Method 500 continues by determining the MIP input samples (e.g., values ​​in the array “p[x]”) based on adjacent samples. In some embodiments, adjacent samples may include upper adjacent samples and / or left adjacent samples.

[0093] In block 503, method 500 continues determining the product of the offset shift parameter (fO) and the sum of the MIP input samples. As shown in equation (B) above, the sum of the MIP input samples is

[0094]

number

[0095]

number

[0096] In block 503, method 500 continues to determine the first constant based on the numerical shift parameter (sW). As shown in equation (B) above, the first constant can be "1 << (sW-1)". In block 504, method 500 continues to determine the offset value (oW) by multiplying the first constant by the product. As shown in equation (B) above, the first offset value (oW) is

number

[0097] In block 505, method 500 determines the MIP weighting matrix based on the prediction parameters. In equation (B) above, the MIP weighting matrix is ​​the matrix "mWeight". In some embodiments, the MIP weighting matrix can be generated based on various methods, such as training processes including machine learning (ML) or artificial intelligence (AI).

[0098] In block 507, method 500 continues determining the MIP prediction based on the MIP weighting matrix, the numerical shift parameter (sW), and the offset value (oW). The MIP prediction can be the matrix “predMip[x][y]” shown in equation (C) above. In some embodiments, the MIP prediction can be determined as described in equation (C) above. In some embodiments, the MIP prediction can be determined based on the following equation (D).

[0099]

number

[0100] In equation (D) above, "[x][y]" are the coordinates of the pixel location. "x" is for the horizontal direction of the MIP prediction matrix, and "y" is for the vertical direction of the MIP prediction matrix. The parameters "incH", "predC", and "incW" are parameters for obtaining matrix values ​​from the MIP weighting matrix. The other parameters in equation (D) are described above with reference to equation (C).

[0101] In some embodiments, the offset shift parameter (fO) can be set to "32" and the numerical shift parameter (sW) can be set to 6. In such embodiments, the MIP prediction matrix can be determined based on the following equations (E) and (F).

[0102]

number

[0103]

number

[0104] In block 508, method 500 generates intra-predictions for the current block by a filtering process based on the MIP prediction matrix. If the size of the MIP prediction matrix is ​​the same as the size of the current block, method 500 can set the values ​​in the MIP prediction matrix as MIP intra-prediction samples (e.g., "preSamples[x][y]") for the current block, as shown in equation (G) below. Otherwise, method 500 can perform a filtering process to adjust the size of the MIP prediction matrix. In some embodiments, the filtering process can be an upsampling process or a low-pass filtering process. An example of an upsampling process is discussed in detail above with reference to Figure 3 (e.g., step 303). predSamples[x][y]=predMip[x][y] (G)

[0105] Figure 6 is a flowchart of Method 600 according to one embodiment of the present disclosure. Method 600 begins in block 601 by determining the prediction parameters for the current block in the bitstream. In some embodiments, the prediction parameters may include “predModeIntra” and “mipSizeId” as defined in the H.265 standard.

[0106] In block 602, method 600 continues by determining the matrix-based intra-prediction (MIP) input sample for the current block based on the adjacent samples and prediction parameters of the current block. In some embodiments, the MIP input sample may be a value in the array "p[x]". In some embodiments, the adjacent sample includes the left adjacent sample and / or the upper adjacent sample.

[0107] Embodiments of determining the MIP input samples are described above with reference to Figure 3 (e.g., step 301). For example, method 600 may include performing a downsampling process on adjacent samples to generate a temporary reference array (pTemp[x]) based on a size identifier.

[0108] In block 603, method 600 continues by setting the numerical shift parameter (sW) to a first fixed value. In some embodiments, the first fixed value can be 5 or 6. In block 604, method 600 continues by setting the offset shift parameter (fO) to a second fixed value. In some embodiments, the second fixed value can be 23, 34, or 46.

[0109] Once the first and second fixed values ​​are set, the first constant "1 << (sW-1)" can be determined. The offset value (oW) can also be calculated (for example, equation (B)).

[0110] In block 605, method 600 continues by determining the MIP weighting matrix for the current block based on the prediction parameters. An example of the MIP weighting matrix is ​​described above with reference to Figure 3 (e.g., step 302).

[0111] In block 606, method 600 continues by determining the MIP predicted sample for the current block (e.g., the values ​​in the array "predMip[x][y]") based on the MIP weighting matrix, the MIP input sample, the offset shift parameter (fO), and the numerical shift parameter (sW). Examples of the MIP predicted sample are discussed in detail above with reference to equations (C), (D), (E), and (F).

[0112] In block 607, method 600 continues by performing an upsampling process on the MIP predicted samples of the current block to generate intra-predicted samples of the current block (e.g., "predSamples[x][y]"). In some embodiments, the MIP predicted samples of the current block may include predicted samples for at least a portion of the sampling points of the current block. Examples of MIP predicted samples and MIP weighting matrices are discussed in detail above with reference to equation (G) and Figure 3 (e.g., step 303).

[0113] Figure 7 is a schematic diagram of an encoder 700 according to one embodiment of the present disclosure. As shown, the encoder 700 includes a first determination unit 701, a first calculation unit 702, and a first prediction unit 703. The first determination unit 701 sets prediction parameters for the current block and determines the MIP input samples (e.g., values ​​in the array "p[x]"). In some embodiments, the MIP input samples can be determined based on adjacent samples. The prediction parameters include parameters indicating which prediction model (e.g., MIP model) was used and the corresponding parameters (e.g., size identifiers). The first determination unit 701 may also determine the MIP metric matrix. The first calculation unit 702 is configured to calculate an offset value (e.g., the aforementioned "oW") based on a numerical shift parameter (e.g., "sW") and an offset shift parameter (e.g., "fO"). The first prediction unit 703 generates an MIP prediction for the current block based on the MIP metric matrix, the offset value, the numerical shift parameter, and the offset shift parameter.

[0114] In some embodiments, the encoder 700 includes a first lookup unit 704 configured to determine the MIP model of the current block. In such embodiments, the first determination unit 701 determines the MIP model index of the current block. The first lookup unit 704 can then obtain corresponding parameters (e.g., sW, fO, MIP size identifier, etc.) based on the MIP model index.

[0115] In this disclosure, the term “unit” may refer to a processor, circuit, software, module, or a combination thereof. In some embodiments, a “unit” may refer to an integrated component such as a System-on-a-Chip (SoC). In some embodiments, a “unit” may refer to a set of instructions stored on a storage medium such as a disk, hard drive, or memory.

[0116] Figure 8 is a schematic diagram of an encoder 800 according to one embodiment of the present disclosure. The encoder 800 may include a first communication interface 801 coupled by a first system bus 804, a first storage device 802, and a first processor 803. The first system bus 804 may include power lines, control lines, and / or signal lines. The first communication interface 801 is configured to communicate with other external devices by sending and receiving signals. The first storage device 802 is configured to store data, information, and / or instructions that can be executed by the first processor 803 (such as the steps discussed in Figures 5 and 6).

[0117] The first processor 803 may be a chip, an integrated circuit, or another device capable of processing signals. The first processor 803 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic circuit (PLD), or another suitable logic component.

[0118] Figure 9 is a schematic diagram of a decoder 900 according to one embodiment of the present disclosure. As shown, the decoder 900 includes an analysis unit 901, a second calculation unit 902, a second prediction unit 903, a second prediction unit 904, and a second investigation unit 905. The analysis unit 901 is configured to analyze the bitstream to obtain prediction parameters for the current block. The second determination unit 902 sets appropriate prediction parameters for the current block and determines MIP input samples (e.g., values ​​in the array "p[x]"). In some embodiments, the MIP input samples can be determined based on adjacent samples. The prediction parameters include parameters indicating which prediction model (e.g., MIP model) was used and the corresponding parameters (e.g., size identifiers). The second determination unit 903 may also determine the MIP metric matrix. The second calculation unit 904 is configured to calculate an offset value (e.g., "oW") based on a numerical shift parameter (e.g., "sW") and an offset shift parameter (e.g., "fO"). The second prediction unit 904 generates an MIP prediction for the current block based on the MIP metric matrix, offset value, numerical shift parameter, and offset shift parameter.

[0119] In some embodiments, the second investigation unit 905 is configured to determine the MIP model of the current block. In such embodiments, the second determination unit 902 determines the MIP model index of the current block. The second investigation unit 904 can then obtain the corresponding parameters (e.g., sW, fO, MIP size identifier, etc.) based on the MIP model index.

[0120] Figure 10 is a schematic diagram of a decoder 1000 according to one embodiment of the present disclosure. The decoder 1000 may include a second communication interface 1001 coupled by a second system bus 1004, a second storage device 1002, and a second processor 1003. The second system bus 1004 may include power lines, control lines, and / or signal lines. The second communication interface 1001 is configured to communicate with other external devices by sending and receiving signals. The second storage device 1002 is configured to store data, information, and / or instructions that can be executed by the second processor 1003 (such as the steps discussed in Figures 5 and 6).

[0121] The second processor 1003 may be a chip, an integrated circuit, or another device capable of processing signals. The second processor 1003 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic circuit (PLD), or another suitable logic component.

[0122] Figure 11 is a schematic diagram of a device 1100 according to one embodiment of the present disclosure. The device 1100 may be a “transmitting” device. More specifically, the device 1100 is configured to acquire, encode, and store / transmit one or more images. The device 1100 includes an acquisition unit 1001, an encoder 1102, and a storage / transmitting unit 1103.

[0123] The acquisition unit 1101 is configured to acquire or receive an image and send the image to the encoder 1102. The acquisition unit 1101 may also be configured to acquire or receive a video consisting of a series of images and send the video to the encoder 1102. In some embodiments, the acquisition unit 1101 may be a device including one or more cameras (e.g., image cameras, depth cameras, etc.). In some embodiments, the acquisition unit 1101 may be a device capable of partially or completely decoding a video bitstream to generate an image or video. The acquisition unit 1101 may also include one or more elements for capturing audio signals.

[0124] The encoder 1102 is configured to encode an image from the acquisition unit 1101 to generate a video bitstream. Alternatively, the encoder 1102 can be configured to encode video from the acquisition unit 1101 to generate a bitstream. In some embodiments, the encoder 1102 can be implemented as the encoding system 200 described in Figure 2. In some embodiments, the encoder 1102 may include one or more audio encoders for encoding audio signals to generate an audio bitstream.

[0125] The storage / transmission unit 1103 is configured to receive one or both of the video bitstream and / or audio bitstream from the encoder 1102. The storage / transmission unit 1103 can encapsulate the video bitstream together with the audio bitstream to form a media file (e.g., an ISO-based media file) or transport stream. In some embodiments, the storage / transmission unit 1103 can write or store this media file or transport stream in a storage device such as a hard drive, disk, DVD, cloud storage, or portable memory device. In some embodiments, the storage / transmission unit 1103 can transmit the video / audio bitstream to an external device via a transfer network such as the Internet, a wired network, a cellular network, or a wireless local area network.

[0126] Figure 12 is a schematic diagram of a device 1200 according to one embodiment of the present disclosure. The device 1200 may be a “transmitter” device. More specifically, the device 1200 is configured to receive, decode, and draw images or videos. The device 1200 includes a receiving unit 1201, a decoder 1202, and a drawing unit 1203.

[0127] The receiver 1201 is configured to receive a media file or transport stream from, for example, a network or storage device. The media file or transport stream includes a video bitstream and / or an audio bitstream. The receiver 1201 can separate the video bitstream and the audio bitstream. In some embodiments, the receiver 1201 can generate a new video / audio bitstream by extracting the video / audio bitstream.

[0128] Decoder 1202 includes one or more video decoders, such as the decoding system 400 described above. Decoder 1202 may also include one or more audio decoders. Decoder 1202 decodes the video bitstream and / or audio bitstream from receiver 1201 to obtain a decoded video file and / or one or more (corresponding to one or more channels) decoded audio files.

[0129] The drawing unit 1203 receives the decoded video / audio file and processes it to obtain a video / audio signal suitable for display / playback. These adjustment / reconstruction operations may include one or more of the following: noise reduction, synthesis, color space conversion, upsampling, downsampling, etc. The drawing unit 1203 can improve the quality of the decoded video / audio file.

[0130] Figure 13 is a schematic diagram of a communication system 1300 according to one embodiment of the present disclosure. The communication system 1300 includes a source device 1301, a storage medium or transfer network 1302, and a destination device 1303. In some embodiments, the source device 1301 may be the device 1100 described above with reference to Figure 11. The source device 1301 transmits media files to the storage medium or transfer network 1302 for storage or transport. The destination device 1303 may be the device 1200 described above with reference to Figure 12. The communication system 1300 is configured to encode media files, transport or store encoded media files, and decode encoded media files. In some embodiments, the source device 1301 may be a first smartphone, the storage medium 1302 may be cloud storage, and the destination device may be a second smartphone.

[0131] The embodiments described above are merely examples of some embodiments of the present disclosure, and their descriptions are specific and detailed. The above embodiments should not be construed as limiting the present disclosure. It should be noted that numerous variations and modifications can be made by those skilled in the art without departing from the spirit and scope of the present disclosure. Therefore, the scope of the present disclosure should be limited to the appended claims.

Claims

1. An image prediction method, Determining the prediction parameters for the current block, Based on the adjacent samples of the current block and the prediction parameters, the matrix-based intra-prediction (MIP) input samples of the current block are determined. Based on the aforementioned prediction parameters, the MIP weighting matrix of the current block is determined, Determining a MIP prediction sample (predMip[x][y]) having coordinates (x, y) of the current block using equations (A) and (B), [Math 1] sW is a parameter for numerical shifting, fO is a parameter for offset shift, p[i] is the i-th value in the MIP input sample, inSize is the number of values ​​in the MIP input sample, mWeight is an MIP weighting matrix corresponding to the MIP mode identifier and size identifier of the current block, predSize represents the size of the MIP prediction block. pTemp is a temporary reference array, and This includes performing an upsampling process on the MIP prediction samples of the current block to generate intra-prediction samples (predSamples[x][y]) of the current block, A method in which, for a block having an arbitrary MIP mode identifier and an arbitrary size identifier, the values ​​of sW are both first fixed values, and for a block having an arbitrary MIP mode identifier and an arbitrary size identifier, the values ​​of fO are both second fixed values.

2. The first fixed value is 6. The method according to claim 1.

3. The second fixed value mentioned above is 32. The method according to claim 2.

4. The MIP prediction sample of the current block includes a portion of the predicted values ​​of the sample of the current block. The method according to claim 1.

5. The prediction parameter includes the size identifier (mipSizeId) of the current block, The method according to claim 1.

6. The aforementioned method, The further step includes performing a downsampling process on the adjacent samples based on the size identifier to generate the temporary reference array pTemp. The method according to claim 5.

7. The aforementioned method, In the case where both the width and height of the current block are equal to 4, the size identifier of the current block is determined to be equal to 0. In the case where both the width and height of the current block are equal to 8, or in the case where either the width or height of the current block is equal to 4, the size identifier of the current block is determined to be equal to 1. If the width and height of the current block are not in the above case, the size identifier of the current block is determined to be equal to 2, further comprising: The method according to claim 5.

8. The aforementioned method, The process further includes generating an MIP input sample (p[x]) based on a second constant value and the temporary reference array pTemp, wherein the second constant value is "1 << (BitDepth - 1)" and BitDepth represents the bit depth of the current block. The method according to claim 1.

9. A decoding device comprising an analysis unit, an intra prediction unit, and, The analysis unit is configured to determine the prediction parameters of the current block. The aforementioned intra prediction unit, Based on the adjacent samples of the current block and the prediction parameters, the matrix-based intra-prediction (MIP) input samples of the current block are determined. Based on the aforementioned prediction parameters, the MIP weighting matrix of the current block is determined, Determining a MIP prediction sample (predMip[x][y]) having coordinates (x, y) of the current block using equations (A) and (B), [Math 2] sW is a parameter for numerical shifting, fO is a parameter for offset shift, p[i] is the i-th value in the MIP input sample, inSize is the number of values ​​in the MIP input sample, mWeight is an MIP weighting matrix corresponding to the MIP mode identifier and size identifier of the current block, predSize represents the size of the MIP prediction block. pTemp is a temporary reference array, and The system is configured to perform an upsampling process on the MIP prediction samples of the current block to generate intra-prediction samples (predSamples[x][y]) of the current block, A decoding device in which, for any block having an arbitrary MIP mode identifier and an arbitrary size identifier, the value of sW is a first fixed value, and for any block having an arbitrary MIP mode identifier and an arbitrary size identifier, the value of fO is a second fixed value.

10. A decoding device, A decoding device comprising a first processor configured to perform the method described in any one of claims 1 to 8.

11. A method for encoding an image, The process involves receiving an input image and dividing the input image into one or more current blocks. Based on the adjacent samples of the current block and the prediction parameters of the current block, the matrix-based intra-prediction (MIP) input samples of the current block are determined, Based on the aforementioned prediction parameters, the MIP weighting matrix of the current block is determined, Determining a MIP prediction sample (predMip[x][y]) having coordinates (x, y) of the current block using equations (A) and (B), [Math 3] sW is a parameter for numerical shifting, fO is a parameter for offset shift, p[i] is the i-th value in the MIP input sample, inSize is the number of values ​​in the MIP input sample, mWeight is an MIP weighting matrix corresponding to the MIP mode identifier and size identifier of the current block, predSize represents the size of the MIP prediction block. pTemp is a temporary reference array, and This includes performing an upsampling process on the MIP prediction sample to generate an intra-prediction sample (predSamples[x][y]) of the current block, A method in which, for a block having an arbitrary MIP mode identifier and an arbitrary size identifier, the values ​​of sW are both first fixed values, and for a block having an arbitrary MIP mode identifier and an arbitrary size identifier, the values ​​of fO are both second fixed values.

12. The first fixed value is 6. The method according to claim 11.

13. The second fixed value mentioned above is 32. The method according to claim 12.

14. The MIP prediction sample of the current block includes a portion of the predicted values ​​of the sample of the current block. The method according to claim 11.

15. The prediction parameter includes the size identifier (mipSizeId) of the current block, The method according to claim 11.

16. The aforementioned method, The further step includes performing a downsampling process on the adjacent samples based on the size identifier to generate the temporary reference array pTemp. The method according to claim 15.

17. The aforementioned method, In the case where both the width and height of the current block are equal to 4, the size identifier of the current block is determined to be equal to 0. In the case where both the width and height of the current block are equal to 8, or in the case where either the width or height of the current block is equal to 4, the size identifier of the current block is determined to be equal to 1. If the width and height of the current block are not in the above case, the size identifier of the current block is determined to be equal to 2, further comprising: The method according to claim 15.

18. The aforementioned method, The process further includes generating an MIP input sample (p[x]) based on a second constant value and the temporary reference array pTemp, wherein the second constant value is "1 << (BitDepth - 1)" and BitDepth represents the bit depth of the current block. The method according to claim 11.

19. An encoding device comprising a division unit and a prediction unit, The division unit is configured to receive an input image and divide the input image into one or more current blocks. The prediction unit, Based on the adjacent samples of the current block and the prediction parameters of the current block, the matrix-based intra-prediction (MIP) input samples of the current block are determined, Based on the aforementioned prediction parameters, the MIP weighting matrix of the current block is determined, Using equations (A) and (B), determine the MIP prediction sample (predMip[x][y]) having the coordinates (x, y) of the current block, [Math 4] sW is a parameter for numerical shifting, fO is a parameter for offset shift, p[i] is the i-th value in the MIP input sample, inSize is the number of values ​​in the MIP input sample, mWeight is an MIP weighting matrix corresponding to the MIP mode identifier and size identifier of the current block, predSize represents the size of the MIP prediction block. pTemp is a temporary reference array, and The system is configured to perform an upsampling process on the MIP prediction samples to generate intra-prediction samples (predSamples[x][y]) of the current block, An encoding device in which, for any block having an arbitrary MIP mode identifier and an arbitrary size identifier, the value of sW is a first fixed value, and for any block having an arbitrary MIP mode identifier and an arbitrary size identifier, the value of fO is a second fixed value.

20. An encoding device, An encoding apparatus comprising a second processor configured to perform the method described in any one of claims 11 to 18.

21. A method for transmitting a bitstream, comprising generating a bitstream by performing the method according to any one of claims 11 to 18, and transmitting the bitstream.

Images