Image encoding method, image decoding method, and related device and system

Abstract

To provide an image decoding method and an image encoding method, as well as a decoder and an encoder.SOLUTION: In a system 100 for encoding, transmitting, and decoding an image, a first intra-prediction unit of an image encoding device 100a determines a width and a height of a coding block in an image (input video 10), and determines a matrix-based intra-prediction (MIP) size identifier if the width and the height are equal to N (N is a positive integer power of 2). The MIP size identifier indicates that a MIP prediction size is equal to N. The first intra prediction unit derives a group of reference samples for the coding block, and derives a MIP prediction value 14 for the coding block based on the group of the reference samples and a MIP matrix corresponding to the MIP size identifier.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the field of telecommunications technology, and in particular to an image (picture, video, etc.) encoding method and an image decoding method. [Background technology]

[0002] Versatile Video Coding (VVC) is a next-generation video compression standard that replaces current standards such as the high efficiency video coding (H.265) standard. Compared with current standards, the VVC coding standard provides higher coding quality. To achieve this goal, various intra-prediction and inter-prediction modes are being considered. Using these prediction modes, the video can be compressed, which reduces the data transmitted in the bitstream (binary format). Matrix-based Intra Prediction (MIP) is one of the modes. MIP is an intra-prediction mode. When implemented under the MIP mode, an encoder (or coder) or decoder can derive an intra-prediction block based on a current block (e.g., a group of bits or numbers that are transmitted as a unit and can be coded and / or decoded together). However, deriving such a prediction block requires a large amount of computational resources and additional storage space. Therefore, an improved method for solving this problem is desirable and necessary. Summary of the Invention

[0003] Under the current MIP mode, to generate a predicted block of a current block, the size of the predicted block is smaller than that of the current block. For example, a current block of "8x8" can have a predicted block of "4x4". Under the current MIP mode, a MIP predicted block with a size smaller than that of the current block is derived by performing a matrix calculation. The matrix calculation consumes less computational resources compared with performing the matrix calculation using a larger block. After the matrix calculation, an intra predicted block having the same size as the current block is derived by performing an upsampling process on the MIP predicted block. For example, an intra predicted block of "8x8" can be derived from a MIP predicted block of "4x4" by invoking an upsampling process of interpolation and / or extrapolation. The present invention provides a method for realizing the MIP mode without performing an upsampling process, thereby greatly reducing the computational complexity and improving the overall efficiency. More specifically, when implementing MIP mode, the method determines a suitable size identifier (or MIP size identifier) ​​so that the size of the MIP prediction block (e.g., "8x8") is the same as the size of the current block ("8x8"), which eliminates the need for an upsampling process.

[0004] In an embodiment of the present invention, an image encoding method is provided. The method can further be applied to encoding a video including a sequence of images. The method includes, for example, (i) determining a width and height of a coding block (e.g., a coding block) in an image; (ii) determining a matrix-based intra-prediction (MIP) size identifier if the width and height are equal to N (N is a positive integer power of 2), where the MIP size identifier indicates that the MIP prediction size is equal to N; (iii) deriving a group of reference samples of the coding block (e.g., by using neighboring samples of the coding block); (iv) deriving an MIP prediction value of the coding block using the group of reference samples and a MIP weighting matrix according to the MIP size identifier; and (v) setting the prediction value of the coding block equal to the MIP prediction value of the coding block. In some embodiments, the method further includes generating a bitstream based on the prediction value of the coding block.

[0005] In another aspect of the present invention, an image decoding method is provided, which may include the following: (a) determine a width, height and prediction mode (e.g., whether the bitstream indicates that MIP mode is used) of a decoded block (e.g., a decoded block) by analyzing a bitstream; (b) determine a MIP size identifier if the width and height are equal to N and the MIP mode is used, and the identifier indicates that the MIP prediction size is equal to N (N is a positive integer power of 2); (c) derive a group of reference samples of the decoded block (e.g., by using adjacent samples of the decoded block); (d) derive a MIP prediction value of the decoded block according to the group of reference samples and a MIP matrix according to the MIP size identifier; and (e) set a prediction value of the decoded block equal to the MIP prediction value of the decoded block.

[0006] In some embodiments, the MIP prediction value may include values ​​of "NxN" prediction samples (e.g., "8x8"). In some embodiments, the MIP matrix may be selected from a group of predefined MIP matrices.

[0007] In another aspect of the present invention, a system for encoding / decoding images and videos is included. The system includes an encoding subsystem (or encoder) and a decoding subsystem (or decoder). The encoding subsystem includes a partitioning unit, a first intra prediction unit, and an entropy coding unit. The partitioning unit is configured to receive an input video and partition the input video into one or more coding units (CUs). The first intra prediction unit is configured to generate a prediction block corresponding to each CU and a MIP size identifier derived from the encoding of the input video. The entropy coding unit is configured to convert parameters to write the prediction values ​​into a bitstream. The decoding subsystem includes a parsing unit and a second intra prediction unit. The parsing unit is configured to obtain values ​​(e.g., values ​​associated with one or more CUs) by parsing the bitstream. The second intra prediction unit is configured to convert the values ​​into an output video based at least in part on the MIP size identifier.

[0008] The width and height of the CU may be equal to N, where N is a positive integer power of 2. The MIP size identifier indicates that the MIP prediction size used by the first intra prediction unit to generate the MIP prediction block is N. A MIP size identifier equal to "2" indicates that the MIP prediction size is "8x8". [Brief description of the drawings]

[0009] In order to make the technical solutions of the embodiments of the present invention clearer, the following briefly introduces the drawings used in the description of the embodiments. Obviously, the drawings described below are only used to describe the present invention, and do not limit the present invention. Those skilled in the art can obtain other drawings by these drawings without creative efforts. [Figure 1] FIG. 1 is a schematic diagram showing a system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram illustrating an encoding system according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a schematic diagram illustrating the derivation of an intra-prediction block using MIP mode according to an embodiment of the present invention. [Figure 4] FIG. 4 is a schematic diagram illustrating a decoding system according to an embodiment of the present invention. [Diagram 5] FIG. 5 is a schematic diagram illustrating an apparatus (eg, an encoder) according to an embodiment of the present invention. [Figure 6] FIG. 6 is a schematic diagram illustrating an apparatus (eg, a decoder) according to an embodiment of the present invention. [Figure 7] FIG. 7 is a schematic diagram showing a communication system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In order to facilitate an understanding of the invention, the invention will now be described in more detail with reference to the accompanying drawings, in which: FIG.

[0011] FIG. 1 is a schematic diagram illustrating a system 100 according to an embodiment of the present invention. The system 100 can encode, transmit, and decode an image. The system 100 can also be applied to encode, transmit, and decode a video including a sequence of images. More specifically, the system 100 can receive an input image, process the input image, and generate an output image. The system 100 includes an encoding device 100a and a decoding device 100b. The encoding device 100a includes a division unit 101, a first intra prediction unit 103, and an entropy coding unit 105. The decoding device 100b includes an analysis unit 107 and a second intra prediction unit 109.

[0012] The division unit 101 is configured to receive an input video 10 and then divide the input video 10 into one or more coding tree units (CTUs) or coding units (CUs) 12. The CUs 12 are transmitted to a first intra prediction unit 103. The first intra prediction unit 103 is configured to derive a prediction block for each CU 12 by performing a MIP process. Based on the size of the CU 12, the MIP process has different approaches to process CUs 12 of different sizes. Each type of CU 12 has a designated MIP size identifier (e.g., 0, 1, 2, etc.). The MIP size identifier is used to derive the size of the MIP prediction block (i.e., the variable "predSize"), the number of reference samples from the upper or left boundary of the CU (i.e., the variable "boundarySize"), and select a MIP matrix from multiple predefined MIP matrices. For example, if the MIP size identifier is "0", the size of the MIP prediction block is "4x4" (e.g., "predSize" is set equal to 4) and "boundarySize" is set equal to 2. If the MIP size identifier is "1", "predSize" is set equal to 4 and "boundarySize" is set equal to 2. If the MIP size identifier is "2", "predSize" is set equal to 8 and "boundarySize" is set equal to 4.

[0013] The first intra prediction unit 103 first determines the width and height of CU12. For example, the first intra prediction unit 103 may determine that the height of CU12 is "8" and the width is "8". In this example, the width and height are "8". Correspondingly, the first intra prediction unit 103 determines that the MIP size identifier of CU12 is "2", which indicates that the size of MIP prediction is "8×8". The first intra prediction unit 103 further derives a group of reference samples for CU12 (e.g., by using neighboring samples of CU12 (e.g., above-neighboring samples or left-neighboring samples), as shown in the detailed description of FIG. 3. Then, the first intra prediction unit 103 derives a MIP prediction value for CU12 based on the group of reference samples and a corresponding MIP matrix. The first intra prediction unit 103 may use the MIP prediction value as an intra prediction value 14 for CU12. The intra prediction value 14 and parameters for deriving the intra prediction value 14 are transmitted to the entropy encoding unit 105 for further processing.

[0014] The entropy encoding unit 105 is configured to convert the parameters for deriving the intra prediction value 14 into a binary format. Correspondingly, the entropy encoding unit 105 generates a bitstream 16 based on the intra prediction value 14. In some embodiments, the bitstream 16 can be transmitted over a communication network or stored on a disk or a server.

[0015] The decoding device 100b receives the bitstream 16 as an input bitstream 17. The parsing unit 107 parses the input bitstream 17 (in binary format) and converts it into numerical values ​​18, which indicate the characteristics of the input video 10 (such as color, brightness, depth, etc.). The numerical values ​​18 are sent to the second intra prediction unit 109, which can then convert these numerical values ​​18 into an output video 19 (e.g., based on a process similar to that performed by the first intra prediction unit 103; related embodiments are described in detail with reference to FIG. 4). The output video 19 can then be stored, transmitted, and / or rendered by an external device (e.g., storage, transmitter, etc.). The stored video can further be displayed by a display.

[0016] FIG. 2 is a schematic diagram illustrating an encoding system 200 according to an embodiment of the present invention. The encoding system 200 is configured to encode, compress, and / or process an input image 20 and generate an output bitstream 21 in a binary format. The encoding system 200 includes a division unit 201 configured to divide the input image 20 into one or more coding tree units (CTUs) 22. In some embodiments, the division unit 201 may divide the image into slices, tiles, and / or bricks. Each brick may include one or more whole and / or partial CTUs 22. In some embodiments, the division unit 201 may also form one or more sub-images, each of which may include one or more slices, tiles, or bricks. The division unit 201 transmits the CTUs 22 to a prediction unit 202 for further processing.

[0017] The prediction unit 202 is configured to generate a prediction block 23 for each CTU 22. The prediction block 23 may be generated based on one or more inter- or intra-prediction methods by utilizing various interpolation and / or extrapolation schemes. As shown in FIG. 2, the prediction unit 202 may further include a block division unit 203, a motion estimation (ME) unit 204, a motion compensation (MC) unit 205, and an intra-prediction unit 206. The block division unit 203 is configured to divide the CTU 22 into smaller coding units (CUs) or coding blocks (CBs). In some embodiments, the CUs may be generated from the CTU 22 through various methods, such as quad-tree division, binary tree division, and ternary tree division. The ME unit 204 is configured to estimate changes caused by the motion of objects shown in the input image 20 or the motion of an image capture device generating the input image 20. The MC unit 205 is configured to adjust and compensate for the changes caused by the motion. Both the ME unit 204 and the MC unit 205 are configured to derive inter (e.g., at different times) predictive blocks for a CU. In some embodiments, the ME unit 204 and the MC unit 205 can utilize a rate-distortion optimized motion estimation method to derive the inter predictive blocks.

[0018] The intra prediction unit 206 is configured to derive an intra (e.g., same-time) predicted block of a CU (or a part of a CU) using various intra prediction modes including a MIP mode. Details of the derivation of an intra prediction block using a MIP mode (hereinafter referred to as a "MIP process") will be described with reference to Figure 3. During the MIP process, the intra prediction unit 206 first derives one or more reference samples from neighboring samples of the CU (e.g., step 301 of Figure 3), for example, by directly using the neighboring samples as reference samples, downsampling the neighboring samples, or directly extracting from the neighboring samples.

[0019] Next, the intra prediction unit 206 derives prediction samples at multiple sample positions in the CU using the reference samples, the MIP matrix, and a shifting parameter. The sample positions may be preset sample positions in the CU. For example, the sample positions may be positions in the CU where the horizontal and vertical coordinate values ​​are odd numbers (e.g., x=1, 3, 5, etc., y=1, 3, 5, etc.). The shifting parameters may include a shifting offset parameter and a shifting number parameter, and may be used for the shifting operation when generating the prediction samples. With such an arrangement, the intra prediction unit 206 may generate prediction samples in the CU (i.e., the term "MIP prediction" or "MIP prediction block" refers to such a set of prediction samples) (e.g., step 302 of FIG. 3). In some embodiments, the sample positions may be positions in the CU where the horizontal and vertical coordinate values ​​are even numbers.

[0020] Third, the intra prediction unit 206 may derive prediction samples at remaining positions (e.g., which are not sample positions) of the CU (e.g., step 303 in FIG. 3). In some embodiments, the intra prediction unit 206 may derive prediction samples at remaining positions by utilizing an interpolation filter. Through the above process, the intra prediction unit 206 may generate a prediction block 23 of the CU in the CTU 22.

[0021] Referring to FIG. 2, the prediction unit 202 outputs a prediction block 23 to an adder 207. The adder 207 calculates a difference (e.g., a residual R) between the output of the split unit 201 (e.g., a CU in the CTU 22) and the output of the prediction unit 202 (i.e., the prediction block 23 of the CU). The transform unit 208 reads the residual R and performs one or more transform operations on the prediction block 23 to obtain coefficients 24 for further use. The quantization unit 209 can quantize the coefficients 24 and output the quantized coefficients 25 (e.g., levels) to the inverse quantization unit 210. The inverse quantization unit 210 performs an inverse quantization operation on the quantized coefficients 25 to output the reconstructed coefficients 26 to the inverse transform unit 211. The inverse transform unit 211 performs one or more inverse transforms corresponding to the transform in the transform unit 208, and outputs the reconstructed residuals 27.

[0022] Then, the adder 212 calculates a reconstructed CU by adding the reconstructed residual 27 and the prediction block 23 of the CU from the prediction unit 202. The adder 212 also forwards its output 28 to the prediction unit 202 so that the output 28 is used as an intra-prediction reference. After all CUs in the CTU 22 are reconstructed, the filtering unit 213 can perform in-loop filtering on the reconstructed image 29. The filtering unit 213 includes one or more filters for suppressing coding distortion or enhancing the coding quality of the image, examples of which include a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping with chroma scaling (LMCS) filter, a neural-network-based filter, and other suitable filters.

[0023] Then, the filtering unit 213 can send the decoded image 30 (or sub-image) to a decoded picture buffer (DPB) 214. The DPB 214 outputs the decoded image 31 based on the control information. The image 31 stored in the DPB 214 can also be used as a reference image for the prediction unit 202 to perform inter prediction or intra prediction.

[0024] The entropy encoding unit 215 is configured to convert the image 31, parameters from units in the encoding system 200, and supplemental information (e.g., information for controlling or communicating with the system 200) into a binary format. The entropy encoding unit 215 can generate the output bitstream 21 accordingly.

[0025] In some embodiments, the encoding system 200 can be a computing device with a processor and a storage medium having one or more encoding programs. When the processor reads and executes the encoding programs, the encoding system 200 can receive an input image 20 and generate an output bitstream 21 accordingly. In some embodiments, the encoding system 200 can be a computing device with one or more chips. The units or elements of the encoding system 200 can be implemented as integrated circuits on a chip.

[0026] FIG. 3 is a schematic diagram illustrating a MIP process according to an embodiment of the present invention. The MIP process may be realized by an intra prediction unit (e.g., the intra prediction unit 206). As shown in FIG. 3, the intra prediction unit may include a prediction module 301 and a filtering module 302. As shown in FIG. 3, the MIP process includes three steps: 301, 302, and 303. The MIP process may generate a prediction block based on a current block or coding block 300 (such as a CU or a portion of a CU).

[0027] Step 301

[0028] In step 301, the intra prediction unit can generate reference samples 32, 34 using neighboring samples 31, 33 of the coding block 300. In the illustrated embodiment, the neighboring sample 31 is an upper neighboring sample, and the neighboring sample 33 is a left neighboring sample. The intra prediction unit 206 can calculate the average value of the values ​​of every two neighboring samples 31 and the average value of the values ​​of every two neighboring samples 33, and set the average values ​​as the values ​​of the reference samples 32, 34, respectively. In some embodiments, the intra prediction unit 206 can select one value from the two neighboring samples 31 or one value from the two neighboring samples 33 as the value of the reference sample 32 or 34. In the illustrated embodiment, the intra prediction unit 206 derives four reference samples 32 from the eight upper neighboring samples 31 of the coding block 300, and derives another four reference samples 34 from the eight left neighboring samples 33 of the coding block 300.

[0029] In step 301, the intra prediction unit determines the width and height of the coding block 300, which are denoted as variables "cbWidth" and "cbHeight", respectively. In some embodiments, the intra prediction unit 206 may utilize a rate-distortion optimized mode decision process to determine the intra prediction mode (e.g., whether MIP mode is used). In such embodiments, the coding block 300 may be divided into one or more transform blocks, whose widths and heights are denoted as variables "nTbW" and "nTbH", respectively. If MIP mode is used as the intra prediction mode, the intra prediction unit determines a MIP size identifier (denoted as variable "mipSizeId") based on the following conditions A to C.

[0030] [Condition A] If both “nTbW” and “nTbH” are 4, “mipSizeId” is set to 0.

[0031] [Condition B] Otherwise, if either "cbWidth" or "cbHeight" is 4, "mipSizeId" is set to 1.

[0032] [Condition C] Otherwise, "mipSizeId" is set to 2.

[0033] As one example, if the size of the coding block 300 is "8x8" (i.e., "cbWidth" and "cbHeight" are both 8), "mipSizeId" is set to 2. As another example, if the size of the transform block of the coding block 300 is "4x4" (i.e., "nTbW" and "nTbH" are both 4), "mipSizeId" is set to 0. As yet another example, if the size of the coding block 300 is "4x8", "mipSizeId" is set to 1.

[0034] In the illustrated embodiment, there are three types of "mipSizeId": "0", "1", and "2". Each type of MIP size identifier (i.e., the variable "mipSizeId") corresponds to a particular way of performing the MIP process (e.g., using different MIP matrices). In other embodiments, there may be more than three types of MIP size identifiers.

[0035] Based on the MIP size identifier, the intra prediction unit can determine the variables "boundarySize" and "predSize" based on the following conditions D to F.

[0036] [Condition D] If "mipSizeId" is 0, "boundarySize" is set to 2 and "predSize" is set to 4.

[0037] [Condition E] If "mipSizeId" is 1, then "boundarySize" is set to 4 and "predSize" is set to 4.

[0038] [Condition F] If "mipSizeId" is 2, then "boundarySize" is set to 4 and "predSize" is set to 8.

[0039] In the illustrated embodiment, "boundarySize" represents the number of reference samples 32, 34 derived from each of the upper neighboring samples 31 and the left neighboring samples 33 of the coding block 300. The variable "predSize" is used in the subsequent calculations (i.e., the following formula: (C) ) is used for

[0040] In some embodiments, the intra prediction unit may derive a variable "isTransposed" to indicate the order of the reference samples 32, 34 stored in the temporal array. For example, "isTransposed" being 0 indicates that the intra prediction unit presents the reference sample 32 derived from the upper neighboring sample 31 of the coding block 300 before the reference sample 34 derived from the left neighboring sample 33. Alternatively, "isTransposed" being 1 indicates that the intra prediction unit presents the reference sample 34 derived from the left neighboring sample 33 of the coding block 300 before the reference sample 32 derived from the upper neighboring sample 31. In the implementation of the coding system 200, the value of "isTransposed" is transmitted to an entropy coding unit (e.g., the entropy coding unit 215) as one of the parameters of the MIP process to be coded and written into a bitstream (e.g., the output bitstream 21). Correspondingly, in the implementation of the decoding system 400 shown in FIG. 4 of the present invention, the value of “isTransposed” can be received from a parsing unit (e.g., parsing unit 401) by parsing the input bitstream (which can be the output bitstream 21).

[0041] The intra prediction unit may further determine a variable "inSize" to indicate the number of reference samples 32, 34 used to derive the MIP predicted value. The value of "inSize" is determined by the following formula (A). In the present invention, the meaning and operation of all operators in the formula are the same as the corresponding operators defined in the ITU-T H.265 standard.

[0042] inSize = ( 2 * boundarySize) - (mipSizeId = = 2) ? 1:0; (A)

[0043] For example, "= =" is the relational operator "equal to." For example, if "mipSizeId" is 2, then "inSize" is 7 (calculated as (2*4)-1). If "mipSizeId" is 1, then "inSize" is 8 (calculated as (2*4)-0).

[0044] The intra prediction unit may invoke the following process to derive a group of reference samples 32, 34 stored in an array p[x] (where "x" ranges from "0" to "inSize-1"). The intra prediction unit may derive "nTbW" samples from the upper neighboring samples 31 of the coding block 300 (and store them in an array "refT") and derive "nTbH" samples from the left neighboring samples 33 of the coding block 300 (and store them in an array "refL").

[0045] The intra prediction unit may start a downsampling operation on 'refT' to obtain 'boundarySize' samples and store the 'boundarySize' samples in 'refT'. Similarly, the intra prediction unit 206 may start a downsampling operation on 'refL' to obtain 'boundarySize' samples and store the 'boundarySize' samples in 'refL'.

[0046] In some embodiments, the intra prediction unit can combine the arrays "refT" and "refL" into a single array "pTemp" based on the order indicated by the variable "isTransposed". The intra prediction unit can derive "isTransposed" to indicate the order of the reference samples stored in the temporal array "pTemp". For example, "isTransposed" being 0 (or FALSE) indicates that the intra prediction unit presents the reference sample 32 derived from the upper neighboring sample 31 of the coding block 300 before the reference sample 34 derived from the left neighboring sample 33 of the coding block 300. In other cases, "isTransposed" being 1 (or TRUE) indicates that the intra prediction unit presents the reference sample 34 derived from the left neighboring sample 33 of the coding block 300 before the reference sample 32 derived from the upper neighboring sample 31. In some embodiments, in the implementation of the coding system 200, the intra prediction unit can determine the value of "isTransposed" in a rate-distortion optimization manner. In some embodiments, in the implementation of the encoding system 200, the intra prediction unit can determine the value of "isTransposed" based on a comparison and / or correlation between the neighboring samples 32, 34 and the coding block 300. In the implementation of the encoding system 200, the value of "isTransposed" can be transferred to an entropy coding unit (e.g., the entropy coding unit 215) as one of the parameters of the MIP process to be written to a bitstream (e.g., the output bitstream 21). Correspondingly, in the implementation of the decoding system 400 shown in FIG. 4 of the present invention, the value of "isTransposed" can be received from a parsing unit (e.g., the parsing unit 401) by parsing an input bitstream (which can be the output bitstream 21).

[0047] The intra prediction unit can determine the array p[x] (x ranges from "0" to "inSize-1") based on the following conditions G and H.

[0048] [Condition G] If "mipSizeId" is 2, p[x] = pTemp[x+1] - pTemp[0].

[0049] [Condition H] Otherwise (for example, when "mipSizeId" is smaller than 2), p[0] = pTemp[0] - (1 << (BitDepth-1)) and p[x] = pTemp[x] - pTemp[0] (x is 1 to "inSize-1").

[0050] In the above condition H, "BitDepth" is the bit depth of a color component (e.g., Y component) of a sample in the coding block 300. Also, the symbol "<<" is a bit shifting symbol used in the ITU-T H.265 standard.

[0051] Optionally, the intra prediction unit may derive array p[x] (x ranges from 0 to “inSize−1”) based on the following conditions I and J:

[0052] [Condition I] If "mipSizeId" is 2, p[x] = pTemp[x+1] - pTemp[0].

[0053] [Condition J] Otherwise (for example, when "mipSizeId" is smaller than 2), p[0] = (1<<(BitDepth-1)) - pTemp[0] and p[x] = pTemp[x] - pTemp[0] (x is from 1 to "inSize-1").

[0054] In some embodiments, the intra prediction unit may not determine the value of "mipSizeId" and may determine the value of the array p[x] in a unified calculation method. For example, the intra prediction unit may add "(1<<(BitDepth-1))" as an additional element to "pTemp" and calculate p[x] as "pTemp[x]-pTemp[0]".

[0055] Step 302

[0056] In step 302, an intra prediction unit (or prediction module 301) derives a MIP prediction for the coding block 300 by utilizing a group of reference samples 32, 34 and a MIP matrix. The MIP matrix is ​​selected from a group of predefined MIP matrices based on a corresponding MIP mode identifier (i.e., the variable "mipModeId") and a MIP size identifier (i.e., the variable "mipSizeId").

[0057] The MIP prediction derived by the intra prediction unit includes partial predicted samples 35 for all or some of the sample positions in the coding block 300. The MIP prediction is denoted as "predMip[x][y]".

[0058] In the embodiment shown in FIG. 3, the partial prediction samples 35 are the samples marked as grey rectangles in the current block 300. The reference samples 32, 34 of the array p[x] derived in step 301 are used as input to the prediction module 301. The prediction module 301 calculates the partial prediction samples 35 by utilizing a MIP matrix and shift parameters. The shift parameters include a shift offset parameter and a shift number parameter. In some embodiments, the prediction module 301 derives the partial prediction samples 35 with coordinates (x, y) based on the following equations (B) and (C):

[0059] JPEG2025024219000002.jpg9147

[0060] JPEG2025024219000003.jpg15147

[0061] In the above formula (B), the parameter "fO" is a shift offset parameter used to determine the parameter "oW". The parameter "sW" is a shift number parameter. "p[i]" is a reference sample. The symbol ">>" is a binary right shifting operator defined in the H.265 standard.

[0062] The above formula (C) In the above, "mWeight[i][j]" is a MIP weighting matrix whose matrix elements are fixed constants for both encoding and decoding. Optionally, in some embodiments, the implementation of the encoding system 200 utilizes an adaptive MIP matrix. For example, the MIP weighting matrix can be updated through various training methods, such as using one or more encoded images as input, or using images provided to the encoding system 200 by external means. The intra prediction unit can forward "mWeight[i][j]" to an entropy coding unit (e.g., the entropy coding unit 215) when the MIP mode is determined. The entropy coding unit can then write "mWeight[i][j]" to the bitstream, for example, to one or more special data units in the bitstream that includes the MIP data. Correspondingly, in some embodiments, an implementation of the decoding system 400 utilizing an adaptive MIP matrix may, for example, update the MIP matrix with a training method, the output of which is one or more encoded images or blocks, or images from another bitstream provided to the decoder 200 by external means, or images obtained from the parsing unit 401 by parsing special data units in the input bitstream that contain MIP matrix data.

[0063] The prediction unit 301 may determine the values ​​of "sW" and "fO" based on the size of the current block 300 and the MIP mode used for the current block 300. In some embodiments, the prediction unit 301 may obtain the values ​​of "sW" and "fO" by using a look-up table. For example, the following Table 1 may be used to determine "sW".

[0064] [Table 1]

[0065] Alternatively, the following Table 2 can also be used to determine "sW".

[0066] [Table 2]

[0067] In some embodiments, the prediction module can set "sW" to a constant. For example, for blocks of various sizes that utilize different MIP modes, the prediction module can set "sW" to "5." As another example, for blocks of various sizes that utilize different MIP modes, the prediction module 301 can set "sW" to "6." As yet another example, for blocks of various sizes that utilize different MIP modes, the prediction module can set "sW" to "7."

[0068] In some embodiments, the prediction unit 301 may determine "fO" in Table 3 or Table 4 below.

[0069] [Table 3]

[0070] [Table 4]

[0071] In some embodiments, the prediction module 301 can set "fO" directly to a constant (e.g., a value between 0 and 100). For example, for blocks of various sizes utilizing different MIP modes, the prediction module 301 can set "fO" to "46". As another example, the prediction module 301 can set "fO" to "56". As yet another example, the prediction module 301 can set "fO" to "66".

[0072] In some embodiments, the intra prediction unit may perform a "clipping" operation on the values ​​of the MIP predicted samples stored in the array "predMip". If "isTransposed" is 1 (or TRUE), the "predSize x preSize" array "predMip[x][y]", with x running from 0 to "predSize-1" and y running from 0 to "predSize-1", is transposed to "predTemp[y][x] = predMip[x][y]", and then "predMip = predTemp".

[0073] More specifically, if the size of the coding block 303 is 8×8 (i.e., both cbWidth and cbHeight are 8), the intra prediction unit can derive an 8×8 predMip array.

[0074] Step 303

[0075] In step 303 of FIG. 3, the intra prediction unit derives prediction samples 37 of remaining samples other than the partial prediction samples 35 in the coding block 300. As shown in FIG. 3, the intra prediction unit can use a filtering module 302 to derive prediction samples 37 of remaining samples other than the partial prediction samples 35 in the coding block 300. The input to the filtering module 302 can be the partial prediction samples 35 in step 302. The filtering module 302 can use one or more interpolation filters to derive prediction samples 37 of remaining samples other than the partial prediction samples 35 in the coding block 300. The intra prediction unit (or the filtering module 302) can generate a prediction block (including multiple prediction samples 37) of the coding block 300 based on the following conditions K and L, and store the prediction samples 37 in an array "predSamples[x][y]" (x runs from 0 to "nTbW-1" and y runs from 0 to "nTbH-1").

[0076] [Condition K] If the intra prediction unit determines that "nTbW" is greater than "predSize" or "nTbH" is greater than "predSize", the intra prediction unit starts an upsampling process to derive "predSamples" based on "predMip".

[0077] [Condition L] Otherwise, the intra prediction unit sets the predicted value of the coding block 300 to the MIP predicted value of the coding block.

[0078] In other words, an intra prediction unit can set "predSamples[x][y]" (x ranging from 0 to "nTbW-1" and y ranging from 0 to "nTbH-1") equal to "predMip[x][y]". For example, an intra prediction unit can set "predSamples" of a coding block whose size is equal to "8x8" (i.e., both "cbWidth" and "cbHeight" are 8) to its "predMip[x][y]".

[0079] Through steps 301-303, the intra prediction unit can generate a prediction value of the current block 300. The generated prediction value can be used for further processing (e.g., prediction block 23 above with reference to FIG. 2).

[0080] 4 is a schematic diagram illustrating a decoding system 400 according to an embodiment of the present invention. The decoding system 400 is configured to receive, process, and convert an input bitstream 40 into an output video 41. The input bitstream 40 can be a bitstream representing compressed / encoded images / video. In some embodiments, the input bitstream 40 can come from an output bitstream (e.g., output bitstream 21) generated by an encoding system (such as encoding system 200).

[0081] The decoding system 400 includes a parsing unit 401 configured to parse the input bitstream 40 to obtain values ​​of syntax elements from the input bitstream 40. The parsing unit 401 further converts binary representations of the syntax elements into numerical values ​​(i.e., the decoding block 42) and forwards the numerical values ​​to the prediction unit 402 (e.g., for decoding). In some embodiments, the parsing unit 401 may also forward one or more variables and / or parameters for decoding the numerical values ​​to the prediction unit 402.

[0082] The prediction unit 402 is configured to determine a prediction block 43 (e.g., a CU or a part of a CU, such as a transform block, etc.) of the decoding block 42. If an inter-decoding mode is indicated to be used to decode the decoding block 42, a motion compensation (MC) unit 403 of the prediction unit 402 can receive related parameters from the analysis unit 401 and decode under the inter-decoding mode accordingly. If an intra-prediction mode (e.g., MIP mode) is indicated to be used to decode the decoding block 42, an intra-prediction unit 404 of the prediction unit 402 can receive related parameters from the analysis unit 401 and decode under the indicated intra-decoding mode accordingly. In some embodiments, the intra-prediction mode (e.g., MIP mode) can be identified by a specific flag (e.g., MIP flag) embedded in the input bitstream 40.

[0083] For example, if the MIP mode is identified, the intra prediction unit 404 performs the following method (described in FIG. 3): Steps 301 to 303 A prediction block 43 (comprising a number of prediction samples) can be determined based on the

[0084] First, the intra prediction unit 404 derives one or more reference samples from the neighboring samples of the decoding block 42 (see FIG. 3 ). Step 301 For example, the intra prediction unit 404 may generate the reference sample by downsampling the neighboring samples or by directly extracting a portion from the neighboring samples.

[0085] The intra prediction unit 404 can then use the reference samples, the MIP matrix and the shift parameters to derive partial prediction samples in the decoding block 42 (see FIG. 3). Step 302In some embodiments, the position of the partial predicted sample may be preset in the decoding clock 42. For example, the position of the partial predicted sample may be a position in the decoding block whose horizontal and vertical coordinate values ​​are odd numbers. The shift parameters may include a shift offset parameter and a shift number parameter, which may be used for the shifting operation in generating the partial predicted sample.

[0086] Finally, when the partial prediction samples of the decoding block 42 are derived, the intra prediction unit 404 derives prediction samples of the remaining samples other than the partial prediction samples in the decoding block 42 (see FIG. 3 ). Step 303 For example, the intra prediction unit 404 may derive the prediction samples using an interpolation filter, using the partial prediction samples and the neighboring samples as inputs to the interpolation filter.

[0087] The decoding system 400 includes a scaling unit 405 that has a similar function to the inverse quantization unit 210 of the encoding system 200. The scaling unit 405 performs a scaling operation on the quantized coefficients 44 (e.g., levels) from the analysis unit 401 to generate reconstructed coefficients 45.

[0088] The transform unit 406 has a similar function to the inverse transform unit 211 in the encoding system 200. The transform unit 406 performs one or more transform operations (e.g., the inverse of one or more transform operations performed by the inverse transform unit 211) to obtain a reconstructed residual 46.

[0089] An adder 407 adds the prediction block 43 from the prediction unit 402 and the reconstructed residual 46 from the transform unit 406 to obtain a reconstructed block 47 of the decoded block 42. The reconstructed block 47 is also transmitted to the prediction unit 402 to be used as a reference (e.g., for other blocks coded under intra prediction modes).

[0090] After all decoded blocks 42 in an image or sub-image are reconstructed (i.e., the reconstructed block 48 is formed), the filtering unit 408 may perform in-loop filtering on the reconstructed block 48. The filtering unit 408 may include one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping with chroma scaling (LMCS) filter, a neural network-based filter, etc. In some embodiments, the filtering unit 408 may perform in-loop filtering only on one or more target pixels in the reconstructed block 48.

[0091] The filtering unit 408 then sends the decoded image 49 (or image) or sub-image to a decoded picture buffer (DPB) 409. The DPB 409 outputs the decoded image as the output video 41 based on timing and control information. The decoded image 49 stored in the DPB 409 can also be adopted as a reference image when the prediction unit 402 performs inter prediction or intra prediction.

[0092] In some embodiments, the decoding system 400 can be a computing device including a processor and a storage medium having one or more decoding programs recorded thereon, which, when read and executed by the processor, can cause the decoding system 400 to receive an input video bitstream and generate a corresponding decoded video.

[0093] In some embodiments, the decoding system 400 may be a computing device with one or more chips. The units or elements of the decoding system 400 may be implemented as integrated circuits on a chip.

[0094] 5 is a schematic diagram illustrating an apparatus 500 according to an embodiment of the present invention. The apparatus 500 can be a "transmitting" apparatus. More specifically, the apparatus 500 is configured to capture, encode, and store / transmit one or more images. The apparatus 500 includes an capturing unit 501, an encoder 502, and a storage / transmitting unit 503.

[0095] The acquisition unit 501 is configured to acquire or receive an image and forward the image to the encoder 502. The acquisition unit 501 may also be configured to acquire or receive a video including a sequence of images and forward the video to the encoder 502. In some embodiments, the acquisition unit 501 may be a device including one or more cameras (e.g., image cameras, depth cameras, etc.). In some embodiments, the acquisition unit 501 may be a device capable of partially or fully decoding a video bitstream to generate an image or video. The acquisition unit 501 may also include one or more elements for capturing an audio signal.

[0096] The encoder 502 is configured to encode the image from the acquisition unit 501 and generate a video bitstream. The encoder 502 is also configured to encode the video from the acquisition unit 501 and generate a bitstream. In some embodiments, the encoder 502 may be implemented as the encoding system 200 illustrated in Figure 2. In some embodiments, the encoder 502 may include one or more audio encoders to encode an audio signal and generate an audio bitstream.

[0097] The storage / transmission unit 503 is configured to receive the video and / or audio bitstreams from the encoder 502. The storage / transmission unit 503 can encapsulate the video bitstream with the audio bitstream to form a media file (e.g., an ISO-based media file) or a transport stream. In some embodiments, the storage / transmission unit 503 can write or store the media file or transport stream in a storage unit, examples of which include a hard drive, a disk, a DVD, cloud storage, a portable memory device, etc. In some embodiments, the storage / transmission unit 503 can transmit the video / audio bitstreams to an external device via a transmission network (e.g., the Internet, a wired network, a cellular network, a wireless local area network, etc.).

[0098] 6 is a schematic diagram illustrating an apparatus 600 according to an embodiment of the present invention. The apparatus 600 may be a "target" apparatus. More specifically, the apparatus 600 is configured to receive, decode and render images or videos. The apparatus 600 includes a receiving unit 601, a decoder 602 and a rendering unit 603.

[0099] The receiving unit 601 is configured to receive a media file or a transmission stream, for example, from a network or a storage device. The media file or the transmission stream includes a video bitstream and / or an audio bitstream. The receiving unit 601 can separate the video bitstream and the audio bitstream. In some embodiments, the receiving unit 601 can generate a new video / audio bitstream by extracting the video / audio bitstream.

[0100] The decoder 602 may include one or more video decoders, such as the decoding system 400 described above. The decoder 602 may also include one or more audio decoders. The decoder 602 decodes the audio and / or video bitstreams from the receiving unit 601 to obtain one or more decoded audio and / or video files (corresponding to one or more channels).

[0101] The rendering unit 603 receives the decoded video / audio files and processes the video / audio files to obtain suitable video / audio signals for display / playback. These adjustment / reconstruction operations may include one or more of noise removal, synthesis, color space conversion, upsampling, downsampling, etc. The rendering unit 603 may improve the quality of the decoded video / audio files.

[0102] FIG. 7 is a schematic diagram illustrating a communication system 700 according to an embodiment of the present invention. The communication system 700 includes a source device 701, a storage medium or transmission network 702, and a target device 703. In some embodiments, the source device 701 can be the above-mentioned device 500 with reference to FIG. 5. The source device 701 transmits a media file to the storage medium or transmission network 702 for storage or transmission. The target device 703 can be the above-mentioned device 600 with reference to FIG. 6. The communication system 700 is configured to encode a media file, transmit or store the encoded media file, and then decode the encoded media file. In some embodiments, the source device 701 can be a first smartphone, the storage medium 702 can be a cloud storage, and the target device can be a second smartphone.

[0103] The above embodiments are merely illustrative of some embodiments of the present invention, and the description is specific and detailed. The above embodiments cannot be construed as limiting the present invention. It should be noted that many variations and modifications can be made by those skilled in the art without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be determined according to the appended claims.

Claims

1. An image decoding method, comprising: determining a width, height and prediction mode of a transform block by analyzing the bitstream; determining a matrix-based intra-prediction (MIP) size identifier when the prediction mode indicates that a matrix-based intra-prediction (MIP) mode is used to decode the transform block; deriving a group of reference samples for the transform block; deriving a MIP prediction for the transform block based on the group of reference samples and a MIP matrix corresponding to the MIP size identifier; the MIP size identifier indicates a MIP prediction size equal to N, where N is a positive integer power of 2, and determining the MIP size identifier includes determining the MIP size identifier to be 0 if the width and the height are equal to 4; Deriving a MIP prediction for the transform block based on the group of reference samples and a MIP matrix corresponding to the MIP size identifier, comprising: deriving the MIP prediction for the transform block based on the following formula: x ranges from 0 to "predSize-1", y ranges from 0 to "predSize-1", "sW" represents a shift number parameter, "fO" represents a shift offset parameter, "inSize" represents a variable indicating the number of reference samples used to derive the MIP prediction value, "p[i]" represents a reference sample, and "predMip[x][y]" represents the MIP prediction value. , "mWeight[i][j]" represents the MIP weighting matrix, and "predSize" represents the MIP prediction block. pTemp[0] represents the size of the reference sample buffer, pTemp[0] represents the 0th value in the reference sample buffer, the symbol << represents the binary left shift operator, and the symbol >> represents the binary right shift operator.

2. An image decoding method comprising:

2. The image decoding method further comprises: obtaining the reference sample buffer by downsampling the group of reference samples of the transform block; determining an input sample value based on a reference sample in the reference sample buffer, the MIP size identifier, and a bit depth of a luma component; the reference sample buffer includes a group of the reference samples of the downsampled transform block; 2. The image decoding method according to claim 1,

3. determining input sample values ​​based on reference samples in the reference sample buffer, the MIP size identifier, and a bit depth of a luma component, deriving input sample values ​​based on the following formula: If the MIP size identifier is equal to 2, then p[x]=pTemp[x+1]−pTemp[0]; If the MIP size identifier is less than 2, "p[x]" represents the reference sample, "pTemp[x]" represents the x-th value in the reference sample buffer, and "BitDepth" represents the bit depth of the luma component.

3. The image decoding method according to claim 2,

4. Deriving a MIP prediction for the transform block based on the group of reference samples and a MIP matrix corresponding to the MIP size identifier, comprising: determining said shift number parameter; deriving a MIP prediction value of the transform block based on the group of reference samples, the shift number parameter, and a MIP matrix according to the MIP size identifier; Different MIP size identifiers correspond to the same shift number parameter, said shift number parameter being a constant equal to 6; 2. The image decoding method according to claim 1,

5. Deriving a MIP prediction for the transform block based on the group of reference samples and a MIP matrix corresponding to the MIP size identifier, comprising: determining the shift offset parameter; deriving a MIP prediction value of the transform block based on the group of reference samples, the shift offset parameters, and a MIP matrix according to the MIP size identifier; Different MIP size identifiers correspond to the same shift offset parameter, the shift offset parameter being a constant.

2. The image decoding method according to claim 1,

6. The image decoding method further includes deriving the group of reference samples of the transform block based on neighboring samples; the adjacent samples include an upper adjacent sample and / or a left adjacent sample; 2. The image decoding method according to claim 1,

7. setting a predicted value of the transform block equal to the MIP predicted value of the transform block.

2. The image decoding method according to claim 1,

8. 1. An image encoding method, comprising: determining a width and height of a transformation block in the image; determining a matrix based intra prediction (MIP) size identifier; deriving a group of reference samples for the transform block; deriving a MIP prediction of the transform block based on the group of reference samples and a MIP matrix according to the MIP size identifier; the MIP size identifier indicates a MIP prediction size equal to N, where N is a positive integer power of 2, and determining the MIP size identifier includes determining the MIP size identifier to be 0 if the width and the height are equal to 4; deriving a MIP prediction for the transform block based on the group of reference samples and a MIP matrix according to the MIP size identifier, deriving the MIP prediction for the transform block based on the following formula: x ranges from 0 to "predSize-1", y ranges from 0 to "predSize-1", "sW" represents a shift number parameter, "fO" represents a shift offset parameter, "inSize" represents a variable indicating the number of reference samples used to derive the MIP prediction value, "p[i]" represents a reference sample, and "predMip[x][y]" represents the MIP prediction value. , "mWeight[i][j]" represents the MIP weighting matrix, and "predSize" represents the MIP prediction block. pTemp[0] represents the size of the reference sample buffer, pTemp[0] represents the 0th value in the reference sample buffer, the symbol << represents the binary left shift operator, and the symbol >> represents the binary right shift operator.

13. An image coding method comprising:

9. The image encoding method further comprises: obtaining the reference sample buffer by downsampling the group of reference samples of the transform block; determining an input sample value based on a reference sample in the reference sample buffer, the MIP size identifier, and a bit depth of a luma component; the reference sample buffer includes a group of the reference samples of the downsampled transform block; 9. The image coding method according to claim 8.

10. determining input sample values ​​based on reference samples in the reference sample buffer, the MIP size identifier, and a bit depth of a luma component, deriving input sample values ​​based on the following formula: If the MIP size identifier is equal to 2, then p[x]=pTemp[x+1]−pTemp[0]; If the MIP size identifier is less than 2, "p[x]" represents the reference sample, "pTemp[x]" represents the x-th value in the reference sample buffer, and "BitDepth" represents the bit depth of the luma component.

10. The image coding method according to claim 9.

11. deriving a MIP prediction for the transform block based on the group of reference samples and a MIP matrix according to the MIP size identifier, determining said shift number parameter; deriving a MIP prediction value of the transform block based on the group of reference samples, the shift number parameter, and a MIP matrix according to the MIP size identifier; Different MIP size identifiers correspond to the same shift number parameter, said shift number parameter being a constant equal to 6; Or, deriving a MIP prediction value of the transform block based on the group of reference samples and a MIP matrix according to the MIP size identifier may include: determining the shift offset parameter; deriving a MIP prediction value of the transform block based on the group of reference samples, the shift offset parameters, and a MIP matrix according to the MIP size identifier; Different MIP size identifiers correspond to the same shift offset parameter, the shift offset parameter being a constant.

9. The image coding method according to claim 8.

12. 1. A decoder for decoding an image, comprising: a processor and a memory; The memory stores one or more computer programs which, when executed by the processor, cause the processor to perform a method according to any one of claims 1 to 7.

23. A decoder for decoding an image, comprising:

13. 1. An encoder for encoding an image, comprising: a processor and a memory; The memory stores one or more computer programs which, when executed by the processor, cause the processor to perform a method according to any one of claims 8 to 11.

16. An encoder for encoding an image, comprising:

Images