Encoding and decoding method, apparatus and communication system

Abstract

To provide a method, an apparatus and a system for improving the computation time and resource consumption of matrix-based intra prediction.SOLUTION: A method includes: determining prediction parameters of a current block in a bitstream S601; determining a matrix-based intra prediction (MIP) input sample of the current block based on neighboring samples of the current block and the prediction parameters S602; setting a shifting number parameter (sW) as a first fixed value S603; setting a shifting offset parameter (fO) as a second fixed value S604; and determining an MIP prediction sample of the current block based on an MIP weighting matrix, the MIP input sample, the shifting offset parameter (fO), and the shifting number parameter (sW) S606.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit and priority of U.S. Provisional Patent Application No. 62 / 911,166, filed on October 4, 2019, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to the field of telecommunications technology, and more particularly to methods for encoding and decoding images such as still images and videos.

Background Art

[0003] Versatile Video Coding (VVC) is the next - generation video compression standard used to replace current standards such as the High Efficiency Video Coding standard (H.265 / HEVC). The VVC encoding standard provides higher encoding quality than current standards. To achieve this goal, various intra - prediction modes and inter - prediction modes are considered. When using these prediction modes, the video can be compressed so that the data transmitted in the bit - stream (binary format) can be reduced. Matrix - based intra - prediction (MIP) is one of such modes. MIP is an intra - prediction mode. When executed in the MIP mode, an encoder or decoder can derive an intra - prediction block based on the current coding block (e.g., a collection of bits or digits that are transmitted as a unit and can be encoded and / or decoded together). However, deriving such prediction blocks can require a significant amount of computational resources and additional storage space. Therefore, an improved method to address this problem is beneficial and desirable.

Summary of the Invention

[0004] When performing the MIP process, various prediction parameters are determined and utilized. Conventionally, some of these prediction parameters can be determined by multiple look-up tables. These look-up tables need to be stored in components of the encoder and / or decoder (such as memory, cache, etc.), and thus require storage space. Furthermore, accessing these look-up tables consumes computation time and computational resources. Therefore, it is beneficial to have improved methods, apparatuses, and systems to address the aforementioned problems.

[0005] The present disclosure provides a method for predicting, encoding, and / or decoding an image based on the MIP process. The MIP process can generate a prediction block of a current block, and the size of the prediction block is smaller than the size of the current block. For example, an "8×8" current block can have a "4×4" prediction block. The MIP prediction block with a size smaller than the current block is derived by performing matrix calculations, and the computational resources consumed are less than performing matrix calculations on a larger block. After the matrix calculations, an upsampling process is applied to the MIP prediction block to derive an intra prediction block of the same size as the current block. For example, an "8×8" intra prediction block can be derived from a "4×4" MIP prediction block by invoking an upsampling process of interpolation and / or extrapolation.

[0006] More specifically, the method includes, for example: (i) determining prediction parameters of a current block in a bitstream; (ii) determining MIP input samples of the current block (e.g., "p[x]" in equations (P-1), (P-2), and (P-3) discussed in detail below) based on adjacent samples and prediction parameters of the current block; (iii) setting a parameter for numerical shift (e.g., "sW" in equation (B) discussed in detail below) to a first fixed value; (iv) setting a parameter for offset shift (e.g., "fO" in equation (B) discussed in detail below) to a second fixed value; (v) determining an MIP weighting matrix of the current block based on the prediction parameters; (vi) determining MIP prediction samples of the current block (e.g., "predMip[x][y]" in equation (C) discussed in detail below) based on the MIP weighting matrix, the MIP input samples, the offset shift parameter "fO", and the numerical shift parameter (sW); and (vii) performing an upsampling process on the MIP prediction samples to generate intra prediction samples (e.g., "predSamples[x][y]" in equation (G) discussed in detail below).

[0007] Although not desired to be restricted by theory, setting either or both of the parameter for numerical shift and the parameter for offset shift to fixed values improves the overall encoding / decoding efficiency without significantly affecting the accuracy of the encoding / decoding process. This decision provides a solution that significantly reduces the computation time and the required storage space when the MIP process is executed.

[0008] Another aspect of the present disclosure includes a system for encoding / decoding images and videos. This system may include an encoding subsystem (or encoder) and a decoding subsystem (or decoder). The encoding subsystem includes a splitting unit, a first prediction unit, and an entropy encoding unit. The splitting unit is configured to receive an input video and split the input video into one or more coding units (CUs). The first intra prediction unit is configured to generate a prediction block corresponding to each CU based on prediction parameters derived from the encoding of the input video. The entropy encoding unit is configured to convert the parameters for deriving the prediction block into a bitstream. The decoding subsystem includes a syntax analysis unit and a second intra prediction unit. The syntax analysis unit is configured to analyze the bitstream to obtain numerical values (e.g., values related to one or more CUs). The second intra prediction unit is configured to convert the numerical values into an output video based on the prediction parameters.

Brief Description of the Drawings

[0009] To make the technical solutions described in the embodiments of the present disclosure clearer, the drawings used in the description of the embodiments are briefly described. Obviously, the drawings described below are for illustration only and not for limitation. It should be understood that those skilled in the art can obtain other drawings based on these drawings without creative work.

[0010]

Figure 1A

Figure 1B

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

DETAILED DESCRIPTION OF THE INVENTION

[0011] To facilitate understanding of the present disclosure, the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0012] FIG. 1A is a schematic diagram of a system 100 according to an embodiment of the present disclosure. The system 100 can encode, transmit, and decode images. Also, the system 100 can be applied to encode, transmit, and decode a video composed of a series of images. More specifically, the system 100 can receive an input image and process the input image to generate an output image. The system 100 includes an encoding device 100a and a decoding device 100b. The encoding device 100a includes a splitting unit 101, a first intra prediction unit 103, and an entropy encoding unit 105. The decoding device 100b includes a syntax analysis unit 107 and a second intra prediction unit 109.

[0013] The splitting unit 101 is configured to receive the input video 10 and split the input video 10 into one or more coding tree units (CTUs) or coding units (CUs) 12. One or more CUs 12 are sent to the first intra prediction unit 103. The first intra prediction unit 103 is configured to derive a prediction block for each of one or more CUs 12 by performing the MIP process. In some embodiments, the MIP process has different methods based on the size of the CU 12 to handle CUs 12 of different sizes. For example, for each type of one or more CUs 12, the MIP process has a specified MIP size identifier (e.g., 0, 1, 2, etc.).

[0014] The first intra prediction unit 103 first determines the prediction parameters (e.g., width, height, size, etc.) of the CU 12. Accordingly, the first intra prediction unit 103 can determine the MIP size identifier of the CU 12. The first intra prediction unit 103 further derives a reference sample group for the CU 12 (e.g., using adjacent samples of the CU 12 such as upper adjacent samples and left adjacent samples, which will be discussed in detail with reference to FIG. 3). Then, the first intra prediction unit 103 derives the MIP prediction of the CU 12 based on the reference sample group and the corresponding MIP weighting matrix. The first intra prediction unit 103 can use the MIP prediction as the intra prediction 14 of the CU 12. Then, the intra prediction 14 and the prediction parameters for deriving the intra prediction 14 are sent to the entropy coding unit 105 for further processing.

[0015] The entropy coding unit 105 is configured to convert the prediction parameters for deriving the intra prediction 14 into a binary format. Accordingly, the entropy coding unit 105 generates a bitstream 16 based on the intra prediction 14. In some embodiments, the bitstream 16 can be transmitted via a communication network or stored on a disk or server.

[0016] The decoding device 100b receives the bitstream 16 as the input bitstream 17. The syntax analysis unit 107 analyzes the input bitstream 17 (binary format) and converts it into a numerical value 18. The numerical value 18 indicates the characteristics of the input video 10 (e.g., color, brightness, density, etc.). The numerical value 18 is transmitted to the second intra prediction unit 109. Then, the second intra prediction unit 109 can convert these numerical values 18 into the output video 19 (based on a process similar to that performed by the first intra prediction unit 103, and related embodiments are discussed in detail with reference to FIG. 4). Thereafter, the output video 19 can be stored, transmitted, and / or drawn by an external device (e.g., a storage device, a transmitter, etc.). The stored video can be further displayed by a display device.

[0017] FIG. 1B is a schematic diagram showing the MIP process S100 according to an embodiment of the present disclosure. The MIP process includes four main steps, S101, S102, S103, and S104. In step S101, prediction parameters for the MIP process S100 are determined. The prediction parameters include the type or size of the current block (e.g., indicated by an MIP size identifier such as "mipSizeId"), the number of reference samples within each boundary (e.g., "boundySize"), the number of MIP input samples (e.g., "inSize"), and the dimensions of the MIP prediction block generated by matrix multiplication (e.g., "predSize×predSize"). Based on the size of the current block, the MIP size identifier can be "0", "1", or "2", and the MIP size identifier is recorded in the parameter "mipSizeId". Different blocks may have different prediction parameters.

[0018] In step S102, the MIP process acquires reference pixels. The reference pixels can be from the upper adjacent block and / or the left adjacent block. The pixels from the upper adjacent block are stored as the parameter "refT", and the pixels from the left adjacent block are stored as the parameter "refL".

[0019] In step S103, an input sample for the MIP process is determined. The input sample can be determined by three sub-steps, S1031, S1032, and S1033. In sub-step S1031, a downsampling process is performed based on a reference pixel (e.g., from step S102) to generate a reference sample. In sub-step S1031, a reference sampling region (or buffer sampling region) is determined. In some embodiments, a matrix transposition process may be included in sub-step S1032. In sub-step S1033, a matrix multiplication calculation is performed based on the input sample.

[0020] In step S104, an intra MIP prediction sample is generated. Step S104 includes four sub-steps, S1041, S1042, S1043, and S1044. In sub-step S1041, a MIP prediction block is generated based on a MIP weighting matrix, a parameter for offset shift (e.g., "fO"), and a parameter for numerical shift (e.g., "sW"). In the present disclosure, the parameter for offset shift and the parameter for numerical shift are set to fixed values. In sub-step S1042, an interpolation process is performed on the MIP prediction block. In sub-step S1043, the MIP prediction block is transposed. In sub-step S1044, an intra MIP prediction sample is generated. In some embodiments, the intra MIP prediction sample can be generated by considering the prediction sample from the upper adjacent pixel. In other embodiments, the intra MIP prediction sample can be generated without considering the prediction sample from the upper adjacent pixel (e.g., by only considering the prediction sample from the left adjacent pixel). Details of the MIP process are discussed below with reference to FIG. 3 and the corresponding equations.

[0021] Figure 2 is a schematic diagram of an encoding system 200 according to an embodiment of the present disclosure. The encoding system 200 is configured to encode, compress, and / or process the input image 20 to generate an output bitstream 21 in binary format. The encoding system 200 includes a splitting unit 201 configured to split the input image 20 into one or more coding tree units (CTUs) 22. In some embodiments, the splitting unit 201 can split the image into slices, tiles, and / or blocks. Each of the blocks may include one or more essential CTUs 22 and / or partial CTUs 22. Also, in some embodiments, the splitting unit 201 can form one or more sub-pictures, each of which may include one or more slices, tiles, or blocks. The splitting unit 201 sends the CTUs 22 to the prediction unit 202 for further processing.

[0022] The prediction unit 202 is configured to generate a prediction block 23 for each of the CTUs 22. The prediction block 23 can be generated by using various interpolation methods and / or extrapolation methods based on one or more inter-prediction methods or intra-prediction methods. As shown in FIG. 2, the prediction unit 202 may further include a block partitioning unit 203, a ME (motion estimation) unit 204, a MC (motion compensation) unit 205, and an intra-prediction unit 206. The block partitioning unit 203 is configured to partition the CTU 22 into smaller coding units (CUs) or coding blocks (CBs). In some embodiments, the CUs can be generated from the CTU 22 by various methods such as quadtree partitioning, binary tree partitioning, ternary tree partitioning, etc. The ME unit 204 is configured to estimate the motion of the object shown in the input image 20 or the change caused by the motion of the image capturing device that generates the input image 20. The MC unit 205 is configured to adjust and compensate for the change caused by the motion. Both the ME unit 204 and the MC unit 205 are configured to derive an inter-prediction block of the CU (or a part of the CU) at different time points. In some embodiments, the ME unit 204 and the MC unit 205 can derive the inter-prediction block using a rate-distortion optimized motion estimation method.

[0023] The intra-prediction unit 206 is configured to derive an intra-prediction block of the CU (or a part of the CU) at the same time point using various intra-prediction modes including the MIP mode. The details of the derivation of the intra-prediction block using the MIP mode (hereinafter referred to as the "MIP process") are discussed with reference to FIGS. 1B and 3. During the MIP process, the intra-prediction unit 206 first derives one or more reference samples from the adjacent samples of the CU, for example, by directly using the adjacent samples as reference samples, or downsampling the adjacent samples, or directly extracting from the adjacent samples (e.g., step 301 in FIG. 3).

[0024] Second, the intra prediction unit 206 derives predicted samples at a number of sample positions within the CU using a reference sample, an MIP matrix, and shift parameters. The sample positions can be preset sample positions within the CU. For example, the sample positions can be positions having odd horizontal and vertical coordinate values within the CU (e.g., x = 1, 3, 5, etc. and y = 1, 3, 5, etc.). The shift parameters include an offset shift parameter and a numerical shift parameter, which can be used in a shift operation when generating predicted samples. With this arrangement, the intra prediction unit 206 can generate predicted samples in the CU (i.e., the "MIP prediction" or "MIP prediction block" refers to a group of such predicted samples) (e.g., step 302 in FIG. 3). In some embodiments, the sample positions can be positions having even horizontal and vertical coordinate values within the CU.

[0025] Third, the intra prediction unit 206 can derive predicted samples at the remaining positions of the CU (e.g., they are not sample positions) (e.g., step 303 in FIG. 3). In some embodiments, the intra prediction unit 206 can use an interpolation filter to derive predicted samples at the remaining positions. Through the foregoing process, the intra prediction unit 206 can generate a prediction block 23 for the CU within the CTU 22.

[0026] Referring to FIG. 2, the prediction unit 202 outputs the prediction block 23 to the adder 207. The adder 207 calculates the difference (e.g., residual R) between the output of the division unit 201 (e.g., CU within the CTU 22) and the output of the prediction block 202 (i.e., the prediction block 23 of the CU). The conversion unit 208 reads the residual R, performs one or more conversion operations on the prediction block 23, and obtains the coefficient 24 for later use. The quantization unit 209 can quantize the coefficient 24 and output the quantized coefficient 25 (e.g., level) to the inverse quantization unit 210. The inverse quantization unit 210 performs a scaling operation on the quantized coefficient 25 and outputs the reconstructed coefficient 26 to the inverse conversion unit 211. The inverse conversion unit 211 performs one or more inverse conversions corresponding to the conversion in the conversion unit 208 and outputs the reconstructed residual 27.

[0027] Then, the adder 212 calculates the reconstructed CU by adding the reconstructed residual 27 to the prediction block 23 of the CU from the prediction unit 202. Also, the adder 212 sends its output 28 used as an intra prediction reference to the prediction unit 202. After all the CUs within the CTU 22 are reconstructed, the filter processing unit 213 can perform processing by an in-loop filter on the reconstructed picture 29. The filter processing unit 213 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luminance (luma) mapping (LMCS) filter with chroma scaling, a neural network-based filter, and other suitable filters for suppressing coding distortion or improving the coding quality of the picture.

[0028] After that, the filter processing unit 213 can send the decoded picture 30 (or sub-picture) to the decoded picture buffer (DPB) 214. The DPB 214 outputs the decoded picture 31 based on the control information. Also, the picture 31 stored in the DPB 214 may be used as a reference picture for performing inter prediction or intra prediction by the prediction unit 202.

[0029] The entropy encoding unit 215 is configured to convert the picture 31, the parameters from each part within the encoding system 200, and the supplementary information (for example, the information for controlling the system 200 or communicating with the system 200) into a binary format. Accordingly, the entropy encoding unit 215 can generate an output bitstream 21.

[0030] In some embodiments, the encoding system 200 can be a computer device including a processor and a storage medium having one or more encoding programs. When the processor reads and executes the encoding program, the encoding system 200 can receive the input image 20 and accordingly generate an output bitstream 21. In some embodiments, the encoding system 200 can be a computer device including one or more chips. Each part or element of the encoding system 200 can be implemented as an integrated circuit on the chip.

[0031] FIG. 3 is a schematic diagram showing the MIP process according to an embodiment of the present disclosure. The MIP process can be implemented by an intra prediction unit (for example, the intra prediction unit 206). As shown in FIG. 3, the intra prediction unit can include a prediction module 301 and a filtering processing module 302. Also as shown in FIG. 3, the MIP process includes three steps 301, 302, and 303. The MIP process can generate a prediction block based on the current block or the encoding block 300 (such as a CU or a plurality of sections of a CU).

[0032] Step 301 In step 301, the intra prediction unit can generate reference samples 32 and 34 using adjacent samples 31 and 33 of the encoding block 300. In the illustrated embodiment, the adjacent sample 31 is an upper adjacent sample, and the adjacent sample 33 is a left adjacent sample. The intra prediction unit 206 can calculate the average of the values for every two of the adjacent samples 31 and 33, and set the average of these values as the values of the reference samples 32 and 34 respectively. In some embodiments, the intra prediction unit 206 can select one of the values of every two of the adjacent samples 31 and 33 as the value of the reference sample 32 or 34. In the illustrated embodiment, the intra prediction unit 206 derives four reference samples 32 from eight upper adjacent samples 31 of the encoding block 300, and another four reference samples 34 from eight left adjacent samples 33 of the encoding block 300.

[0033] In step 301, the intra prediction unit determines the width and height of the encoding block 300, and represents them with variables "cbWidth" and "cbHeight" respectively. In some embodiments, the intra prediction unit 206 can adopt a rate-distortion optimized mode decision process to determine the intra prediction mode (for example, whether the MIP mode is used). In such embodiments, the encoding block 300 can be divided into one or more transform blocks, and its width and height are represented by variables "nTbW" and "nTbH" respectively. When the MIP mode is used as the intra prediction mode, the intra prediction unit determines the MIP size identifier (represented by the variable "mipSizeId") based on the following conditions A to C. [Condition A] If both "nTbW" and "nTbH" are 4, set "mipSizeId" to 0. [Condition B] Otherwise, if "cbWidth" or "cbHeight" is 4, set "mipSizeId" to 1. [Condition C] If neither of the above, set "mipSizeId" to 2.

[0034] As an example, when the size of the encoding block 300 is "8×8" (i.e., when both "cbWidth" and "cbHeight" are 8), "mipSizeId" is set to 2. As another example, when the size of the block to be transformed in the encoding block 300 is "4×4" (i.e., when both "nTbW" and "nTbH" are 4), "mipSizeId" is set to 0. As yet another example, when the size of the encoding block 300 is "4×8", "mipSizeId" is set to 1.

[0035] In the illustrated embodiment, there are three types of "mipSizeId", namely "0", "1", and "2". Each type of MIP size identifier (i.e., the variable "mipSizeId") corresponds to a specific way of performing the MIP process (e.g., using different MIP matrices). In other embodiments, there may be four or more types of MIP size identifiers.

[0036] Based on the MIP size identifier, the intra prediction unit can determine variables ("Size" and "predSize") based on the following conditions D to F. [Condition D] When "mipSizeId" is 0, set "boundarySize" to 2 and "predSize" to 4. [Condition E] When "mipSizeId" is 1, set "boundarySize" to 4 and "predSize" to 4. [Condition F] When "mipSizeId" is 2, set "boundarySize" to 4 and "predSize" to 8.

[0037] In the illustrated embodiment, "boundarySize" represents the number of reference samples 32 and 34 derived from the upper adjacent sample 31 and the left adjacent sample 33 of the encoding block 300, respectively. The variable "predSize" is used in subsequent calculations (i.e., the following formula (C)).

[0038] In some embodiments, the intra prediction unit may also derive a variable "isTransposed" to indicate the order of reference samples 32 and 34 stored in the temporal array. For example, the fact that "isTransposed" is "0" indicates that the intra prediction unit presents the reference sample 32 derived from the upper adjacent sample 31 of the coding block 300 before the reference sample 34 derived from the left adjacent sample 33. Alternatively, the fact that "isTransposed" is "1" indicates that the intra prediction unit presents the reference sample 34 derived from the left adjacent sample 33 of the coding block 300 before the reference sample 32 derived from the upper adjacent sample 31. In the implementation of the coding system 200, the value of "isTransposed" is sent to an entropy coding unit (e.g., entropy coding unit 215) as one of the parameters of the MIP process written to the coded bitstream (e.g., output bitstream 21). Accordingly, in the implementation of the decoding system 400 of FIG. 4 described in the present disclosure, the value of "isTransposed" can be received from a syntax analysis unit (e.g., syntax analysis unit 401) by analyzing the input bitstream (which may be the output bitstream 21).

[0039] The intra prediction unit may further determine a variable "inSize" to indicate the number of reference samples 32 and 34 used when deriving the MIP prediction. The value of "inSize" is determined by the following equation (A). In the present disclosure, the meaning and operation of all operators in the equation are the same as the corresponding operators defined in the ITU-T H.265 standard.

[0040] inSize = (2 * boundarySize) - (mipSizeId == 2)? 1:0; (A) For example, "==" is a comparison operator and means "equal to". For example, when "mipSizeId" is 2, "inSize" is 7 (calculated by (2 × 4) - 1). For example, when "mipSizeId" is 1, "inSize" is 8 (calculated by (2 × 4) - 0). In some embodiments, the parameter "inSize" can be found in the following Size-Id table.

[0041]

Table 1

[0042] The intra prediction unit can call the following process to derive the reference sample groups 32 and 34 stored in the array p[x] ("x" ranges from "0" to "inSize - 1"). The intra prediction unit can derive "nTbW" samples from the upper adjacent sample 31 of the coding block 300 (and store them in the array "refT"), and can also derive "nTbH" samples from the left adjacent sample 33 of the coding block 300 (and store them in the array "refL").

[0043] The intra prediction unit can mark the first letter for the downsampling process on "refT" to obtain "boundarySize" samples and store the "boundarySize samples" in "refT". The intra prediction unit 206 can start the downsampling process for "refL" to obtain "boundarySize" samples and store the "boundarySize" samples in "refL".

[0044] In some embodiments, the intra prediction unit can incorporate the arrays "refT" and "refL" into a single array "pTemp" based on the order indicated by the variable "isTransposed". The intra prediction unit can derive "isTransposed" to indicate the order of the reference samples stored in the temporal array "pTemp". For example, "isTransposed" being "0" (or FALSE) indicates that the intra prediction unit presents the reference sample 32 derived from the upper adjacent sample 31 of the coding block 300 before the reference sample 34 derived from the left adjacent sample 33. In another case, "isTransposed" being "1" (or TRUE) indicates that the intra prediction unit presents the reference sample 34 derived from the left adjacent sample 33 of the coding block 300 before the reference sample 32 derived from the upper adjacent sample 31. In some embodiments, in the implementation of the coding system 200, the intra prediction unit can determine the value of "isTransposed" by using a rate-distortion optimization method. In some embodiments, in the implementation of the coding system 200, the intra prediction unit can determine the value of "isTransposed" based on the comparison and / or correlation between the adjacent samples 32, 34 and the coding block 300. In the implementation of the coding system 200, the value of "isTransposed" can be sent to the entropy coding unit (e.g., entropy coding unit 215) as one of the parameters of the MIP process written to the bitstream (e.g., output bitstream 21). Accordingly, in the implementation of the decoding system 400 of FIG. 4 described in the present disclosure, the value of "isTransposed" can be received from the syntax analysis unit (e.g., syntax analysis unit 401) by analyzing the input bitstream (which can be the output bitstream 21).

[0045] In some embodiments, the parameter "mipSizeId" can be used to determine whether the selected size parameter of the current block is within a predetermined range. More specifically, "mipSizeId" equal to "0" or "1" means that the selected size parameter is within a predetermined range. If "mipSizeId" is equal to "2", it means that the selected size parameter is not within the predetermined range. That is, the MIP input sample can be determined based on a buffer area (e.g., "pTemp"), a MIP size identifier (e.g., "mipSizeId"), and the bit depth of adjacent samples (e.g., "BitDepth"), as discussed in more detail below.

[0046] The intra prediction unit can determine the array "p[x]" (where x ranges from "0" to "inSize - 1") based on the following conditions G and H. [Condition G] When "mipSizeId" is 2, p[x] = pTemp[x + 1] - pTemp[0]. [Condition H] Otherwise (e.g., when "mipSizeId" is less than 2), p[0] = pTemp[0] - (1 << (BitDepth - 1)) and p[x] = pTemp[x] - pTemp[0] (for x from 1 to "inSize - 1").

[0047] In the above Condition H, "BitDepth" is the bit depth of the color component (e.g., the Y component) of the samples within the coding block 300. The symbol "<<" is the bit shift operation symbol used in the ITU - T H.265 standard.

[0048] Alternatively, the intra prediction unit can derive the array p[x] (for x ranging from "0" to "inSize - 1") based on the following conditions I and J. [Condition I] When "mipSizeId" is 2, p[x] = pTemp[x + 1] - pTemp[0]. [Condition J] Otherwise (for example, if "mipSizeId" is less than 2), p[0] = (1 << (BitDepth - 1)) - pTemp[0] and p[x] = pTemp[x] - pTemp[0] (for x from 1 to "inSize - 1").

[0049] In some embodiments, the intra prediction unit can determine the value of the array p[x] by using a unified calculation method without determining the value of "mipSizeId". For example, the intra prediction unit can add "(1 << (BitDepth - 1))" as an additional element into "pTemp", and calculate p[x] as "pTemp[x] - pTemp[0]".

[0050] In some embodiments, the parameter "(1 << (BitDepth - 1))" can be a constant. In some embodiments, the parameter "(1 << (BitDepth - 1))" can be denoted as "2 ^ (BitDepth - 1)". Accordingly, the parameters "p[0]" and "p[x]" can be calculated based on the following equations P-1, P-2, and P-3.

[0051]

Equation

Equation

Equation

[0052] Equations P-1 and P-2 apply when "mipSizeId" is equal to "0" or "1", meaning that the selected size parameter is within a predetermined range. After determining that the selected size parameter of the current block is within the predetermined range, "1<<(BitDepth-1)" can be determined. Then, the difference between "1<<(BitDepth-1)" and "pTemp[0]" can be determined. The difference is set to p[0]. And p[x] can be calculated based on the above equations P-1 and P-2.

[0053] Equation P-3 applies when "mipSizeId" is equal to "2", meaning that the selected size parameter is not within the predetermined range. Assuming that the current block is a 4×4 block, this means that there are four values in the buffer area "pTemp". These four values are pTemp[0], pTemp[1], pTemp[2], pTemp[3]. Based on the above equation P-3 and these four values, the array "p[x]" can be determined. The values within the array "p[x]" may be called MIP input samples.

[0054] Step 302 In step 302, the intra prediction unit (or prediction module 301) derives the MIP prediction of the coding block 300 by using the reference sample groups 32, 34 and the MIP matrix. The MIP matrix is selected from a group of predefined MIP matrices based on its corresponding MIP mode identifier (i.e., the variable "mipModeId") and MIP size identifier (i.e., the variable "mipSizeId").

[0055] The MIP prediction derived by the intra prediction unit includes partial prediction samples 35 at all or some of the sample positions within the coding block 300. The MIP prediction is represented as "predMip[x][y]".

[0056] In the embodiment shown in FIG. 3, the partial prediction sample 35 is a sample displayed as a gray square within the current block 300. The reference samples 32, 34 in the array "p[x]" derived in step 301 are used as inputs to the prediction module 301. The prediction module 301 calculates the partial prediction sample 35 by using the MIP matrix and the shift parameters. The shift parameters include an offset shift parameter ("fO") and a numerical shift parameter ("sW"). In some embodiments, the prediction module 301 derives the partial prediction sample 35 having coordinates (x, y) based on the following equations (B) and (C).

[0057]

Number

[0058]

Number

[0059] In the above equation (B), the parameter "fO" is an offset shift parameter used to determine the parameter "oW" or an offset value. The parameter "sW" is a numerical shift parameter. "p[i]" is a reference sample. The symbol ">>" is a binary right shift operator defined in the H.265 standard.

[0060] In the above formula (C), "mWeight[i][j]" is the MIP weighting matrix, and the matrix elements are fixed constants for both encoding and decoding. Alternatively, in some embodiments, an adaptive MIP matrix is used in the implementation of the encoding system 200. For example, the MIP weighting matrix can be updated by various training methods using one or more encoded pictures as input or pictures supplied to the encoding system 200 by external means. When the MIP mode is determined, the intra prediction unit can send "mWeight[i][j]" to the entropy encoding unit (e.g., entropy encoding unit 215). Thereafter, the entropy encoding unit can write "mWeight[i][j]" into one or more special data units within the bitstream, e.g., within the bitstream containing MIP data. Correspondingly, in some embodiments, the implementation of the decoding system 400 using an adaptive MIP matrix can update the MIP matrix using, for example, one or more encoded pictures or blocks, or pictures from other bitstreams supplied by external meaning or obtained by the syntax analysis unit 401 analyzing special data units within the input bitstream containing MIP matrix data, through a training method.

[0061] The prediction unit 301 can determine the values of "sW" and "fO" based on the size of the current block 300 and the MIP mode used for the current block 300. In some embodiments, the prediction unit 301 can obtain the values of "sW" and "fO" by using a look-up table. For example, the value of "sW" can be determined using Table 1 below.

[0062] [Table 2]

[0063] In some embodiments, the numerical shift parameter "sW" can be set to a first fixed value such as 5 or 6. In such embodiments, it is not necessary to use Table 1 above to check the value of the numerical shift parameter "sW". For example, when "mipSizeId" is equal to "0" or "2", the numerical shift parameter "sW" can be set to "5". As another example, when "mipSizeId" is equal to "2", the numerical shift parameter "sW" can be set to "5". Table 2 below shows different settings of the numerical shift parameter "sW". In some embodiments, the numerical shift parameter "sW" can be set by the prediction module 301.

[0064]

Table 3

[0065] In some embodiments, the prediction module can set "sW" to a constant. For example, the prediction module can set "sW" to "5" for blocks of various sizes having different MIP modes. As another example, the prediction module 301 can set "sW" to "6" for blocks of various sizes having different MIP modes. As yet another example, the prediction module can set "sW" to "7" for blocks of various sizes having different MIP modes.

[0066] In some embodiments, the prediction unit 301 can determine the offset shift parameter "fO" using Table 3 below.

[0067]

Table 4

[0068] In some embodiments, the offset shift parameter "fO" can be set to a second fixed value such as 23, 32, 46, 56, 66. The second fixed value has a preferred range of 1 to 100. In such embodiments, it is not necessary to use Table 3 above to check the value of the offset shift parameter "fO". In some embodiments, the offset shift parameter "fO" can be set based on the parameter "mipSizeId". For example, when "mipSizeId" is equal to "0", the offset shift parameter "fO" can be set to "34". As another example, when "mipSizeId" is equal to "1", the offset shift parameter "fO" can be set to "23". As yet another example, when "mipSizeId" is equal to "1", the offset shift parameter "fO" can be set to "46". Table 4 below shows different settings of the offset shift parameter "fO". In some embodiments, the offset shift parameter "fO" can be set by the prediction module 301.

[0069]

Table 5

[0070] Referring to the above formula (B), when the numerical shift parameter "sW" and the offset shift parameter "fO" are set to fixed values, the parameter "oW" can also be a constant, thereby improving the calculation speed of the entire MIP process.

[0071] In some embodiments, the intra prediction unit can perform a clipping operation on the values of the MIP prediction samples stored in the array "predMip". When "isTransposed" is 1 (or TRUE), the array "predMip[x][y]" of "predSize×preSize" (for x from 0 to "predSize - 1" and y from 0 to "predSize - 1") is transposed as "predTemp[y][x]=predMip[x][y]" and becomes "predMip = predTemp".

[0072] More specifically, when the size of the coding block 303 is "8×8" (i.e., both "cbWidth" and "cbHeight" are 8), the intra prediction unit can derive an "8×8" "predMip" array.

[0073] Step 303 In step 303 of FIG. 3, the intra prediction unit derives prediction samples 37 of the remaining samples other than the partial samples 35 within the coding block 300. As shown in FIG. 3, the intra prediction unit can use the filtering module 302 to derive prediction samples 37 of the remaining samples other than the partial samples 35 within the coding block 300. The input to the filtering module 302 can be the partial samples 35 in step 302. The filtering module 302 can use one or more interpolation filters to derive prediction samples 37 of the remaining samples other than the partial samples 35 within the coding block 300. The intra prediction unit (or the filtering module 302) can generate a prediction (including a number of prediction samples 37) of the coding block 300 and store the prediction 37 in the array "predSamples[x][y]" (for x from 0 to "nTbW - 1" and y from 0 to "nTbH - 1") based on the following conditions K and L.

[0074] If the Intra prediction unit determines that "nTbW" is greater than "predSize" or "nTbH" is greater than "predSize" under [Condition K], the Intra prediction unit starts an upsampling process and derives "predSamples" based on "predMip".

[0075] Otherwise, under [Condition L], the Intra prediction unit sets the prediction of the coding block 300 to the MIP prediction of the coding block.

[0076] That is, the Intra prediction unit can set "predSamples[x][y]" (for x from 0 to "nTbW - 1" and y from 0 to "nTbH - 1") to be equal to "predMip[x][y]". For example, for a coding block whose size is equal to "8×8" (i.e., both "cbWidth" and "cbHeight" are 8), the Intra prediction unit can set "predSamples" to its "predMip[x][y]".

[0077] Through steps 301 to 303, the Intra prediction unit can generate the prediction of the current block 300. The generated prediction can be used in a further process (e.g., prediction block 23 described above with reference to FIG. 2).

[0078] FIG. 4 is a schematic diagram of a decoding system 400 according to an embodiment of the present disclosure. The decoding system 400 is configured to receive and process an input bitstream 40 and convert it into an output video 41. The input bitstream 40 can be a bitstream representing a compressed / encoded picture / video. In some embodiments, the input bitstream 40 can be from an output bitstream (e.g., output bitstream 21) generated by an encoding system (such as encoding system 200).

[0079] The decoding system 400 includes a syntax analysis unit 401 configured to analyze the input bit stream 40 to obtain the values of its syntax elements. Further, the syntax analysis unit 401 converts the binary representation of the syntax element into a numerical value (i.e., the decoding block 42) and sends this numerical value to the prediction unit 402 (for example, for decoding). In some embodiments, the syntax analysis unit 401 can also send one or more variables and / or parameters for decoding to the prediction unit 402.

[0080] The prediction unit 402 is configured to determine a decoding block 42 (for example, a prediction block 43 that is a part of a CU such as a CU or a conversion block). When it is shown that an inter-coding mode is used to decode the decoding block 42, the MC (motion compensation) unit 403 of the prediction unit 402 can receive the relevant parameters from the syntax analysis unit 401 and perform decoding in the inter-coding mode accordingly. When it is shown that an intra prediction mode (for example, the MIP mode) is used to decode the decoding block 42, the intra prediction unit 404 of the prediction unit 402 can receive the relevant parameters from the syntax analysis unit 401 and perform decoding in the intra-coding mode accordingly. In some embodiments, the intra prediction mode (for example, the MIP mode) can be identified by a specific flag (for example, the MIP flag) embedded in the input bit stream 40.

[0081] For example, when the MIP mode is specified, the intra prediction unit 404 can determine the prediction block 43 (including a number of prediction samples) based on the following method (similar to steps 301 - 303 described in FIG. 3).

[0082] First, the intra prediction unit 404 derives one or more reference samples from the adjacent samples of the decoding block 42 (similar to step 301 in FIG. 3). For example, the intra prediction unit 404 can generate the reference samples by downsampling the adjacent samples or directly extracting a part from the adjacent samples.

[0083] Then, the intra prediction unit 404 can derive partial prediction samples within the decoded block 42 using the reference sample, the MIP matrix, and the shift parameters (similar to step 302 in FIG. 3). In some embodiments, the positions of the partial prediction samples can be pre-set within the decoded block 42. For example, the positions of the partial prediction samples can be set to the positions having odd horizontal coordinate values and vertical coordinate values within the encoded block. The shift parameters can include an offset shift parameter and a numerical shift parameter, which can be used in the shift operation when generating the partial prediction samples.

[0084] Finally, once the partial prediction samples of the decoded block 42 are derived, the intra prediction unit 404 derives prediction samples for the remaining samples other than the partial prediction samples within the decoded block 42 (similar to step 303 in FIG. 3). For example, the intra prediction unit 404 can use an interpolation filter to derive prediction samples by using the partial prediction samples and adjacent samples as the input to the interpolation filter.

[0085] The decoding system 400 includes a scaling unit 405 having a function similar to that of the inverse quantization unit 210 of the encoding system 200. The scaling unit 405 performs a scaling operation on the quantized coefficients 44 (e.g., levels) from the syntax analysis unit 401 to generate the reconstructed coefficients 45.

[0086] The conversion unit 406 has a function similar to that of the inverse conversion unit 211 of the encoding system 200. The conversion unit 406 performs one or more conversion operations (e.g., the inverse operations of one or more conversion operations by the inverse conversion unit 211) to obtain the reconstructed residual 46.

[0087] The adder 407 adds the prediction block 43 from the prediction unit 402 and the reconstructed residual 46 from the conversion unit 406 to obtain the reconstructed block 47 of the decoding block 42. Also, the reconstructed block 47 is transmitted to the prediction unit 402 and used as a reference (e.g., for other blocks encoded in the intra prediction mode).

[0088] After all the decoding blocks 42 within a picture or sub - picture are reconstructed (i.e., the reconstructed block 48 is formed), the filter processing unit 408 can perform in - loop filter processing on the reconstructed block 49. The filter processing unit 408 includes one or more filters such as a de - blocking filter, a sample - adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luminance mapping with chroma scaling (LMCS) filter, a neural network - based filter, etc. In some embodiments, the filter processing unit 408 can perform in - loop filter processing on only one or more target pixels within the reconstructed block 48.

[0089] Then, the filter processing unit 408 transmits the decoded picture 49 (or picture) or sub - picture to the DPB (decoded picture buffer) 409. The DPB 409 outputs the decoded picture as the output video 41 based on timing and control information. Also, the decoded picture 49 stored in the DPB 409 can be used as a reference picture by the prediction unit 402 when performing inter - prediction or intra - prediction.

[0090] In some embodiments, the decoding system 400 can be a computer device comprising a processor and a storage medium storing one or more decoding programs. When the processor reads and executes the decoding program, the decoding system 400 can receive an input video bitstream and generate a corresponding decoded video.

[0091] In some embodiments, the decoding system 400 can be a computer device comprising one or more chips. Each part or element of the decoding system 400 can be implemented as an integrated circuit on a chip.

[0092] FIG. 5 is a flowchart showing a method 500 according to an embodiment of the present disclosure. The method 500 begins at block 501 by determining the prediction parameters of the current block. In some embodiments, the prediction parameters can include prediction parameters (e.g., "predModeIntra" defined in the H.265 standard) and size identifiers (e.g., "mipSizeId"). At block 502, if the prediction parameters indicate that the MIP mode is applicable, the method 500 continues to determine the MIP input samples (e.g., values within the array "p[x]") based on adjacent samples. In some embodiments, the adjacent samples can include upper adjacent samples and / or left adjacent samples.

[0093] At block 503, the method 500 continues to determine the product of the offset shift parameter (fO) and the sum of the MIP input samplings. As shown in Equation (B) above, the sum of the MIP input samplings is

[0094]

Number

[0095]

Number

[0096] At block 503, the method 500 continues to determine a first constant based on the numerical shift parameter (sW). As shown in Equation (B) above, the first constant can be "1<<(sW - 1)". At block 504, the method 500 continues to determine the offset value (oW) by multiplying the first constant and the product. As shown in Equation (B) above, the first offset value (oW) is

Number

[0097] In block 505, method 500 determines a MIP weighting matrix based on prediction parameters. In the above formula (B), the MIP weighting matrix is the matrix "mWeight". In some embodiments, the MIP weighting matrix can be generated based on various methods such as a training process including machine learning (ML) or artificial intelligence (AI).

[0098] In block 507, method 500 continues to determine the MIP prediction based on the MIP weighting matrix, the numerical shift parameter (sW), and the offset value (oW). The MIP prediction can be the matrix "predMip[x][y]" shown in the above formula (C). In some embodiments, the MIP prediction can be determined as described in the above formula (C). In some embodiments, the MIP prediction can be determined based on the following formula (D).

[0099]

Number

[0100] In the above formula (D), "[x][y]" are the coordinates of the pixel location. "x" is for the horizontal direction of the MIP prediction matrix, and "y" is for the vertical direction of the MIP prediction matrix. The parameters "incH", "predC", "incW" are parameters for obtaining matrix values from the MIP weighting matrix. The other parameters in formula (D) were described above with reference to formula (C).

[0101] In some embodiments, the offset shift parameter (fO) can be set to "32", and the numerical shift parameter (sW) can be set to 6. In such embodiments, the MIP prediction matrix can be determined based on the following formulas (E) and (F).

[0102]

Number

[0103]

Number

[0104] At block 508, method 500 generates an intra prediction of the current block by a filtering process based on the MIP prediction matrix. If the size of the MIP prediction matrix is the same as the size of the current block, method 500 can set the values in the MIP prediction matrix as MIP intra prediction samples (e.g., "preSamples[x][y]") in the current block as shown in the following equation (G). Otherwise, method 500 can perform a filtering process to adjust the size of the MIP prediction matrix. In some embodiments, the filtering process can be an upsampling process or a low-pass filtering process. An example of the upsampling process is discussed in detail above with reference to FIG. 3 (e.g., step 303). predSamples[x][y]=predMip[x][y] (G)

[0105] FIG. 6 is a flowchart showing a method 600 according to an embodiment of the present disclosure. Method 600 begins at block 601 by determining the prediction parameters of the current block in the bitstream. In some embodiments, the prediction parameters can include "predModeIntra" and "mipSizeId" defined in the H.265 standard.

[0106] At block 602, method 600 continues by determining matrix-based intra prediction (MIP) input samples for the current block based on adjacent samples of the current block and prediction parameters. In some embodiments, the MIP input samples can be values within the array "p[x]". In some embodiments, the adjacent samples include a left adjacent sample and / or an upper adjacent sample.

[0107] Embodiments of the determination of the MIP input samples are described above with reference to FIG. 3 (e.g., step 301). For example, method 600 may include performing a downsampling process on the adjacent samples to generate a temporary reference array (pTemp[x]) based on a size identifier.

[0108] At block 603, method 600 continues by setting the parameter for numerical shift (sW) to a first fixed value. In some embodiments, the first fixed value can be 5 or 6. At block 604, method 600 continues by setting the parameter for offset shift (fO) to a second fixed value. In some embodiments, the second fixed value can be 23, 34, or 46.

[0109] Once the first fixed value and the second fixed value are set, the first constant "1<<(sW - 1)" can be determined. An offset value (oW) can also be calculated (e.g., Equation (B)).

[0110] At block 605, method 600 continues by determining the MIP weighting matrix for the current block based on the prediction parameters. Examples of the MIP weighting matrix are described above with reference to FIG. 3 (e.g., step 302).

[0111] At block 606, method 600 continues by determining the MIP prediction sample of the current block (e.g., the value in the array "predMip[x][y]") based on the MIP weighting matrix, the MIP input sample, the parameter for offset shift (fO), and the parameter for numerical shift (sW). Examples of MIP prediction samples are discussed in detail above with reference to equations (C), (D), (E), and (F).

[0112] At block 607, method 600 continues by performing an upsampling process on the MIP prediction sample of the current block to generate the intra prediction sample of the current block (e.g., "predSamples[x][y]"). In some embodiments, the MIP prediction sample of the current block may include prediction samples for at least some of the sampling points of the current block. Examples of the MIP prediction sample and the MIP weighting matrix are discussed in detail above with reference to equation (G) and FIG. 3 (e.g., step 303).

[0113] FIG. 7 is a schematic diagram of an encoder 700 according to an embodiment of the present disclosure. As shown, the encoder 700 includes a first determination unit 701, a first calculation unit 702, and a first prediction unit 703. The first determination unit 701 sets the prediction parameters of the current block and determines the MIP input sample (e.g., the value in the array "p[x]"). In some embodiments, the MIP input sample can be determined based on adjacent samples. The prediction parameters include parameters indicating which prediction model (e.g., the MIP model) was used and the corresponding parameters (e.g., size identifiers). Also, the first determination unit 701 can determine the MIP metric matrix. The first calculation unit 702 is configured to calculate an offset value (e.g., "oW" described above) based on the parameter for numerical shift (e.g., "sW") and the parameter for offset shift (e.g., "fO"). The first prediction unit 703 generates the MIP prediction of the current block based on the MIP metric matrix, the offset value, the parameter for numerical shift, and the parameter for offset shift.

[0114] In some embodiments, encoder 700 includes a first investigation unit 704 configured to determine the MIP model of the current block. In such embodiments, the first determination unit 701 determines the MIP model index of the current block. Then, the first investigation unit 704 can obtain corresponding parameters (e.g., sW, fO, MIP size identifier, etc.) based on the MIP model index.

[0115] In the present disclosure, the term "unit" can be a processor, a circuit, software, a module, or a combination thereof. In some embodiments, the "unit" can be an integrated component such as a SoC (system on chip). In some embodiments, the "unit" can include a set of instructions stored in a storage medium such as a disk, a hard drive, or a memory.

[0116] FIG. 8 is a schematic diagram of an encoder 800 according to an embodiment of the present disclosure. Encoder 800 may include a first communication interface 801, a first storage device 802, and a first processor 803 coupled by a first system bus 804. The first system bus 804 may include power lines, control lines, and / or signal lines. The first communication interface 801 is configured to communicate with other external devices by transmitting and receiving signals. The first storage device 802 is configured to store data, information, and / or instructions (such as the steps discussed in FIGS. 5 and 6) that can be executed by the first processor 803.

[0117] The first processor 803 can be a chip, an integrated circuit, or other device capable of processing signals. The first processor 803 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic circuit (PLD), or other suitable logic component.

[0118] FIG. 9 is a schematic diagram of a decoder 900 according to an embodiment of the present disclosure. As shown, the decoder 900 includes an analysis unit 901, a second calculation unit 902, a second prediction unit 903, a second prediction unit 904, and a second investigation unit 905. The analysis unit 901 is configured to analyze a bitstream to obtain prediction parameters of a current block. The second determination unit 902 sets appropriate prediction parameters for the current block and determines MIP input samples (for example, values in the array "p[x]"). In some embodiments, the MIP input samples can be determined based on adjacent samples. The prediction parameters include parameters indicating which prediction model (for example, the MIP model) was used and the corresponding parameters (for example, size identifiers). Also, the second determination unit 903 can determine the MIP metric matrix. The second calculation unit 904 is configured to calculate an offset value (for example, "oW") based on a numerical shift parameter (for example, "sW") and an offset shift parameter (for example, "fO"). The second prediction unit 904 generates an MIP prediction of the current block based on the MIP metric matrix, the offset value, the numerical shift parameter, and the offset shift parameter.

[0119] In some embodiments, the second investigation unit 905 is configured to determine the MIP model of the current block. In such embodiments, the second determination unit 902 determines the MIP model index of the current block. Then, the second investigation unit 904 can obtain the corresponding parameters (for example, sW, fO, MIP size identifier, etc.) based on the MIP model index.

[0120] FIG. 10 is a schematic diagram of a decoder 1000 according to an embodiment of the present disclosure. The decoder 1000 may include a second communication interface 1001, a second storage device 1002, and a second processor 1003 coupled by a second system bus 1004. The second system bus 1004 may include power lines, control lines, and / or signal lines. The second communication interface 1001 is configured to communicate with other external devices by transmitting and receiving signals. The second storage device 1002 is configured to store data, information, and / or instructions (such as the steps discussed in FIGS. 5 and 6) that can be executed by the second processor 1003.

[0121] The second processor 1003 can be a chip, an integrated circuit, or other device capable of processing signals. The second processor 1003 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic circuit (PLD), or other suitable logic component.

[0122] FIG. 11 is a schematic diagram of a device 1100 according to an embodiment of the present disclosure. The device 1100 can be a "transmission" device. More specifically, the device 1100 is configured to acquire, encode, and store / transmit one or more images. The device 1100 includes an acquisition unit 1001, an encoder 1102, and a storage / transmission unit 1103.

[0123] The acquisition unit 1101 is configured to acquire or receive an image and send the image to the encoder 1102. Also, the acquisition unit 1101 can be configured to acquire or receive a video composed of a series of images and send the video to the encoder 1102. In some embodiments, the acquisition unit 1101 can be a device including one or more cameras (such as an image camera, a depth camera, etc.). In some embodiments, the acquisition unit 1101 can be a device capable of partially or completely decoding a video bitstream to generate an image or a video. Also, the acquisition unit 1101 can include one or more elements for capturing an audio signal.

[0124] The encoder 1102 is configured to encode the image from the acquisition unit 1101 and generate a video bitstream. Also, the encoder 1102 can be configured to encode the video from the acquisition unit 1101 and generate a bitstream. In some embodiments, the encoder 1102 can be implemented as the encoding system 200 described in FIG. 2. In some embodiments, the encoder 1102 may include one or more audio encoders for encoding an audio signal and generating an audio bitstream.

[0125] The memory / transmission unit 1103 is configured to receive one or both of the video bitstream and the audio bitstream from the encoder 1102. The memory / transmission unit 1103 can encapsulate the video bitstream together with the audio bitstream to form a media file (e.g., an ISO-based media file) or a transport stream. In some embodiments, the memory / transmission unit 1103 can write or store this media file or transport stream in a storage unit such as a hard drive, disk, DVD, cloud storage, portable memory device, etc. In some embodiments, the memory / transmission unit 1103 can transmit the video / audio bitstream to an external device via a transfer network such as the Internet, a wired network, a cellular network, a wireless local area network, etc.

[0126] FIG. 12 is a schematic diagram of an apparatus 1200 according to an embodiment of the present disclosure. The apparatus 1200 can be a "destination" apparatus. More specifically, the apparatus 1200 is configured to receive, decode, and render an image or video. The apparatus 1200 includes a reception unit 1201, a decoder 1202, and a rendering unit 1203.

[0127] The receiving unit 1201 is configured to receive a media file or a transport stream from, for example, a network or a storage device. The media file or the transport stream includes a video bitstream and / or an audio bitstream. The receiving unit 1201 can separate the video bitstream and the audio bitstream. In some embodiments, the receiving unit 1201 can generate a new video / audio bitstream by extracting the video / audio bitstream.

[0128] The decoder 1202 includes one or more video decoders such as the above-described decoding system 400. Further, the decoder 1202 may also include one or more audio decoders. The decoder 1202 decodes the video bitstream and / or the audio bitstream from the receiving unit 1201 to obtain a decoded video file and / or one or more (corresponding to one or more channels) decoded audio files.

[0129] The rendering unit 1203 receives the decoded video / audio file, processes the video / audio file, and obtains a video / audio signal suitable for display / playback. Such adjustment / reconfiguration operations may include one or more of noise removal, synthesis, color space conversion, upsampling, downsampling, etc. The rendering unit 1203 can improve the quality of the decoded video / audio file.

[0130] FIG. 13 is a schematic diagram of a communication system 1300 according to an embodiment of the present disclosure. The communication system 1300 includes a source device 1301, a storage medium or transfer network 1302, and a destination device 1303. In some embodiments, the source device 1301 can be the device 1100 described above with reference to FIG. 11. The source device 1301 transmits to the storage medium or transfer network 1302 for storing or transporting a media file. The destination device 1303 can be the device 1200 described above with reference to FIG. 12. The communication system 1300 is configured to encode a media file, transport or store the encoded media file, and decode the encoded media file. In some embodiments, the source device 1301 can be a first smartphone, the storage medium 1302 can be cloud storage, and the destination device can be a second smartphone.

[0131] The above-described embodiments merely illustrate some embodiments of the present disclosure, and the description thereof is specific and detailed. The above embodiments should not be construed as limiting the present disclosure. It should be noted that numerous modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present disclosure. Therefore, the scope of the present disclosure should be in accordance with the appended claims.

Claims

1. 1. A method for predicting an image, comprising: determining prediction parameters for the current block; determining matrix-based intra-prediction (MIP) input samples for the current block based on neighboring samples of the current block and the prediction parameters; determining a MIP weighting matrix for the current block based on the prediction parameters; determining a MIP predicted sample (predMip[x][y]) having coordinates (x, y) of the current block using equations (A) and (B), [Equation 1] sW is a parameter for the numerical shift, fO is a parameter for the offset shift, p[i] is the i-th value in the MIP input sample; inSize is the number of values ​​in the MIP input sample; mWeight is a MIP weighting matrix corresponding to the MIP mode identifier and size identifier of the current block; predSize represents the size of the MIP prediction block; pTemp is a temporary reference sequence; and performing an upsampling process on the MIP predicted samples of the current block to generate intra predicted samples (predSamples[x][y]) of the current block; A method wherein, for blocks having any MIP mode identifier and any size identifier, all values ​​of sW are a first fixed value, and, for blocks having any MIP mode identifier and any size identifier, all values ​​of fO are a second fixed value.

2. the first fixed value is 6; The method of claim 1.

3. the second fixed value is 32; The method of claim 2.

4. The MIP predicted samples of the current block include predicted values ​​of a portion of the samples of the current block. The method of claim 1.

5. The prediction parameters include a size identifier (mipSizeId) of the current block. The method of claim 1.

6. The method comprises: performing a downsampling process on the adjacent samples based on the size identifier to generate the temporary reference sequence pTemp. The method of claim 5.

7. The method comprises: determining that the size identifier of the current block is equal to 0 if the width and height of the current block are both equal to 4; determining that the size identifier of the current block is equal to 1 if both the width and height of the current block are equal to 8, or if one of the width and height of the current block is equal to 4; If the width and height of the current block are not the above cases, determining that the size identifier of the current block is equal to 2. The method of claim 5.

8. The method comprises: generating a MIP input sample (p[x]) based on a second constant value and the temporary reference array pTemp, where the second constant value is 1<<(BitDepth-1), and BitDepth indicates a bit depth of the current block; The method of claim 1.

9. A decoding device, comprising: an analysis unit; and an intra prediction unit, the analyzer is configured to determine prediction parameters for a current block; The intra prediction unit determining matrix-based intra-prediction (MIP) input samples for the current block based on neighboring samples of the current block and the prediction parameters; determining a MIP weighting matrix for the current block based on the prediction parameters; determining a MIP predicted sample (predMip[x][y]) having coordinates (x, y) of the current block using equations (A) and (B), [Equation 2] sW is a parameter for the numerical shift, fO is a parameter for the offset shift, p[i] is the i-th value in the MIP input sample; inSize is the number of values ​​in the MIP input sample; mWeight is a MIP weighting matrix corresponding to the MIP mode identifier and size identifier of the current block; predSize represents the size of the MIP prediction block; pTemp is a temporary reference sequence; and performing an upsampling process on the MIP predicted samples of the current block to generate intra predicted samples (predSamples[x][y]) of the current block; A decoding device, wherein for blocks having any MIP mode identifier and any size identifier, the value of sW is a first fixed value, and for blocks having any MIP mode identifier and any size identifier, the value of fO is a second fixed value.

10. A decoding device, comprising: A decoding device comprising a first processor configured to perform the method of any one of claims 1 to 8.

11. 1. A method for encoding an image, comprising: receiving an input image and dividing the input image into one or more current blocks; determining matrix-based intra-prediction (MIP) input samples for the current block based on neighboring samples of the current block and prediction parameters of the current block; determining a MIP weighting matrix for the current block based on the prediction parameters; determining a MIP predicted sample (predMip[x][y]) having coordinates (x, y) of the current block using equations (A) and (B), [Equation 3] sW is a parameter for the numerical shift, fO is a parameter for the offset shift, p[i] is the i-th value in the MIP input sample; inSize is the number of values ​​in the MIP input sample; mWeight is a MIP weighting matrix corresponding to the MIP mode identifier and size identifier of the current block; predSize represents the size of the MIP prediction block; pTemp is a temporary reference sequence; and performing an upsampling process on the MIP predicted samples to generate intra predicted samples (predSamples[x][y]) of the current block; A method wherein, for blocks having any MIP mode identifier and any size identifier, all values ​​of sW are a first fixed value, and, for blocks having any MIP mode identifier and any size identifier, all values ​​of fO are a second fixed value.

12. the first fixed value is 6; The method of claim 11.

13. the second fixed value is 32; The method of claim 12.

14. The MIP predicted samples of the current block include predicted values ​​of a portion of the samples of the current block. The method of claim 11.

15. The prediction parameters include a size identifier (mipSizeId) of the current block. The method of claim 11.

16. The method comprises: performing a downsampling process on the adjacent samples based on the size identifier to generate the temporary reference sequence pTemp.

16. The method of claim 15.

17. The method comprises: determining that the size identifier of the current block is equal to 0 if the width and height of the current block are both equal to 4; determining that the size identifier of the current block is equal to 1 if both the width and height of the current block are equal to 8, or if one of the width and height of the current block is equal to 4; If the width and height of the current block are not the above cases, determining that the size identifier of the current block is equal to 2.

16. The method of claim 15.

18. The method comprises: generating a MIP input sample (p[x]) based on a second constant value and the temporary reference array pTemp, where the second constant value is 1<<(BitDepth-1), and BitDepth indicates a bit depth of the current block; The method of claim 11.

19. An encoding device, comprising: a division unit; and a prediction unit; the divider is configured to receive an input image and divide the input image into one or more current blocks; The prediction unit determining matrix-based intra-prediction (MIP) input samples for the current block based on neighboring samples of the current block and prediction parameters of the current block; determining a MIP weighting matrix for the current block based on the prediction parameters; determining a MIP predicted sample (predMip[x][y]) having coordinates (x, y) of the current block using equations (A) and (B), [Equation 4] sW is a parameter for the numerical shift, fO is a parameter for the offset shift, p[i] is the i-th value in the MIP input sample; inSize is the number of values ​​in the MIP input sample; mWeight is a MIP weighting matrix corresponding to the MIP mode identifier and size identifier of the current block; predSize represents the size of the MIP prediction block; pTemp is a temporary reference sequence; and performing an upsampling process on the MIP predicted samples to generate intra predicted samples (predSamples[x][y]) of the current block; an encoding device, wherein for blocks having any MIP mode identifier and any size identifier, the value of sW is a first fixed value, and for blocks having any MIP mode identifier and any size identifier, the value of fO is a second fixed value.

20. 1. An encoding device, comprising: Encoding device comprising a second processor configured to perform the method of any one of claims 11 to 18.

21. A computer-readable medium having a computer program and a bitstream stored thereon, comprising: A computer readable medium which, when executed by a processor, produces the bitstream by implementing the steps of the method of any one of claims 11 to 18.

Images