Method for encoding and decoding images, and related apparatus and system

Abstract

The present application relates to a method for encoding an image. In some embodiments, the method comprises: (i) determining a width and a height of a coded block in an image; (ii) if the width and height are equal to N, determining a matrix-based intra prediction (MIP) size identifier indicating that the MIP prediction size is equal to N, where N is a positive integer power of 2; (iii) determining a set of reference samples of the coded block; (iv) determining a MIP predictor for the encoded block based on the set of reference samples and a MIP matrix corresponding to the MIP size identifier.

Description

Technical Field

[0001] The present application relates to the field of telecommunication technology, and in particular to a method for encoding and decoding an image (eg, an image or a video). Background Art

[0002] Versatile video coding (VVC) is a next-generation video compression standard that replaces current standards such as H.265 / high efficiency video coding (HEVC). The VVC coding standard provides higher coding quality than current standards. To achieve this goal, various intra-frame and inter-frame prediction modes are considered. When these prediction modes are used, the video can be compressed, so that the data transmitted in the code stream (binary form) can be reduced. Matrix-based intra prediction (MIP) is one of these modes. MIP is an intra-frame prediction mode. When implemented in MIP mode, an encoder (or codec) or decoder can predict an intra-frame prediction block based on a current block (e.g., a group of bits or numbers that are transmitted as a unit and can be encoded and / or decoded together). However, predicting such a prediction block may require a lot of computing resources and additional storage space. Therefore, an improved method for solving this problem is advantageous and necessary. Summary of the invention

[0003] In the current MIP mode, in order to generate a prediction block for the current block, the size of the prediction block is smaller than the size of the current block. For example, a current block of "8x8" may have a prediction block of "4x4". In the current MIP mode, a MIP prediction block smaller than the size of the current block is obtained by matrix calculation, and the matrix calculation consumes less computing resources than matrix calculation with a larger block. After the matrix calculation, the MIP prediction block is upsampled to obtain an intra-frame prediction block of the same size as the current block. For example, an intra-frame prediction block of "8x8" can be determined from a MIP prediction block of "4x4" by calling an upsampling process of interpolation and / or extrapolation. The present application provides an implementation method of a MIP mode without an upsampling process, thereby significantly reducing the computational complexity and improving the overall efficiency. More specifically, when implementing the MIP mode, the method determines a suitable size identifier (or MIP size identifier) ​​so that the size of the MIP prediction block (e.g., "8x8") is the same as the size of the current block ("8x8"), so there is no need to perform an upsampling process.

[0004] An embodiment of the present application provides a method for encoding an image. The method can also be applied to encoding a video containing an image sequence. The method includes, for example, (i) determining the width and height of a coding block (e.g., a coding block) in an image; (ii) if the width and height are "N" ("N" is a positive integer power of 2), determining a matrix-based intra prediction (MIP) size identifier, which indicates that the MIP prediction size is equal to "N"; (iii) determining a set of reference samples of the coding block (e.g., determined using adjacent samples of the coding block); (iv) determining the MIP prediction value of the coding block using the set of reference samples and a MIP weight matrix according to the MIP size identifier; (v) setting the prediction value of the coding block to be equal to the MIP prediction value of the coding block. In some embodiments, the method also includes generating a bitstream based on the prediction value of the coding block.

[0005] According to another aspect of the present application, a method for decoding an image may include, for example, (a) parsing a bitstream to determine the width, height, and prediction mode of a decoding block (e.g., a decoding block) (e.g., whether the bitstream indicates that a MIP mode is used); (b) if the width and height are "N" and the MIP mode is used, determining a MIP size identifier indicating that the MIP prediction size is equal to "N" ("N" is a positive integer power of 2); (c) determining a set of reference samples of the decoding block (e.g., determined using adjacent samples of the decoding block); (d) determining a MIP prediction value of the decoding block using the set of reference samples and a MIP weight matrix based on the MIP size identifier; and (e) setting the prediction value of the decoding block to be equal to the MIP prediction value of the decoding block.

[0006] In some embodiments, the MIP prediction values ​​may include values ​​of "NxN" prediction samples (eg, "8x8"). In some embodiments, the MIP matrix may be selected from a set of predefined MIP matrices.

[0007] Another aspect of the present application includes a system for encoding / decoding images and videos. The system may include an encoding subsystem (or encoder) and a decoding subsystem (or decoder). The encoding subsystem includes a partitioning unit, a first prediction unit, and an entropy coding unit. The partitioning unit is used to receive an input video and divide the input video into one or more coding units (coding unit, CU). The first intra-frame prediction unit is used to generate a prediction block corresponding to each CU and a MIP size identifier determined from encoding the input video. The entropy coding unit is used to transform parameters to write the prediction block into a bitstream. The decoding subsystem includes a parsing unit and a second intra-frame prediction unit. The parsing unit is used to parse the bitstream to obtain a numerical value (e.g., a value associated with one or more CUs). The second intra-frame prediction unit is used to convert the above-mentioned numerical value into an output video based at least in part on the MIP size identifier.

[0008] The width and height of the CU may be equal to "N", where "N" is a positive integer power of 2. The MIP size identifier indicates that the MIP prediction size used by the first intra prediction unit to generate the MIP prediction block is "N". For example, the MIP size identifier equal to "2" indicates that the MIP prediction size is "8x8". BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to make the technical solutions described in the embodiments of the present application clearer, the drawings used to describe the embodiments are briefly described. Obviously, the drawings described below are only for illustration and are not intended to be limiting. It should be understood that those skilled in the art can obtain other drawings based on these drawings without creative work.

[0010] Figure 1 is a schematic diagram of a system according to an embodiment of the present application.

[0011] Figure 2 is a schematic diagram of a coding system according to an embodiment of the present application.

[0012] Figure 3 It is a schematic diagram of determining an intra-frame prediction block using the MIP mode according to an embodiment of the present application.

[0013] Figure 4 is a schematic diagram of a decoding system according to an embodiment of the present application.

[0014] Figure 5 is a schematic diagram of a device (eg, an encoder) according to an embodiment of the present application.

[0015] Figure 6 is a schematic diagram of a device (eg, a decoder) according to an embodiment of the present application.

[0016] Figure 7 is a schematic diagram of a communication system according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] To facilitate understanding of the present application, the present application will be described in more detail below with reference to the accompanying drawings.

[0018] Figure 1 1 is a schematic diagram of a system 100 according to an embodiment of the present application. The system 100 can encode, send and decode images. The system 100 can also be applied to encode, send and decode videos containing image sequences. More specifically, the system 100 can receive input images, process input images, and generate output images. The system 100 includes an encoding device 100a and a decoding device 100b. The encoding device 100a includes a partitioning unit 101, a first intra-frame prediction unit 103 and an entropy encoding unit 105. The decoding device 100b includes a parsing unit 107 and a second intra-frame prediction unit 109.

[0019] The partitioning unit 101 is used to receive an input video 10, and then divide the input video 10 into one or more coding tree units (CTU) or coding units (CU) 12. The CU 12 is sent to a first intra prediction unit 103. The first intra prediction unit 103 is used to determine a prediction block for each CU 12 by performing a MIP process. Based on the size of the CU 12, the MIP process has different processing methods for CUs 12 of different sizes. For each type of CU 12, there is a specified MIP size identifier (e.g., 0, 1, 2, etc.). The MIP size identifier is used to determine the size of the MIP prediction block (i.e., a variable "predSize"), the number of reference samples from the upper boundary or left boundary of the CU (i.e., a variable "boundarySize"), and to select a MIP matrix from a plurality of predefined MIP matrices. For example, when the MIP size identifier is "0", the size of the MIP prediction block is "4x4" (e.g., "predSize" is set equal to 4) and "boundarySize" is set equal to 2; when the MIP size identifier is "1", "predSize" is set equal to 4 and "boundarySize" is set equal to 2; when the MIP size is 2, "predSize" is set equal to 8 and "boundarySize" is set equal to 4.

[0020] The first intra prediction unit 103 first determines the width and height of CU 12. For example, the first intra prediction unit 103 may determine that the height of CU 12 is "8" and the width is "8". In this example, the width and height are "8". Accordingly, the first intra prediction unit 103 determines that the MIP size identifier of CU 12 is "2", which indicates that the size of the MIP prediction is "8x8". The first intra prediction unit 103 also determines a set of reference samples of CU 12 (for example, using neighboring samples of CU 12, such as upper neighboring samples or left neighboring samples, reference samples). Figure 3 Detailed discussion). The first intra prediction unit 103 then determines a MIP prediction value for CU 12 based on the set of reference samples and the corresponding MIP matrix. The first intra prediction unit 103 may use the MIP prediction value as an intra prediction value 14 for CU 12. The intra prediction value 14 and parameters used to determine the intra prediction value 14 are then sent to the entropy coding unit 105 for further processing.

[0021] The entropy coding unit 105 is used to convert the parameters used to determine the intra-frame prediction value 14 into a binary form. Accordingly, the entropy coding unit 105 generates a bit stream 16 based on the intra-frame prediction value 14. In some embodiments, the bit stream 16 can be sent through a communication network or stored in a disk or server.

[0022] The decoding device 100b receives the code stream 16 as an input code stream 17. The parsing unit 107 parses the input code stream 17 (in binary form) and converts it into numerical values ​​18. The numerical values ​​18 indicate the characteristics of the input video 10 (e.g., color, brightness, depth, etc.). The numerical values ​​18 are sent to the second intra-frame prediction unit 109. The second intra-frame prediction unit 109 can then convert these numerical values ​​18 into an output video 19 (e.g., based on processes similar to those performed by the first intra-frame prediction unit 103; see related embodiments for reference). Figure 4 Detailed discussion). The output video 19 may then be stored, transmitted, and / or presented by an external device (e.g., memory, transmitter, etc.). The stored video may also be displayed by a display.

[0023] Figure 220 is a schematic diagram of a coding system 200 according to an embodiment of the present application. The coding system 200 is used to encode, compress and / or process an input image 20, and generate an output code stream 21 in binary form. The coding system 200 includes a partitioning unit 201, which is used to divide the input image 20 into one or more coding tree units (CTUs) 22. In some embodiments, the partitioning unit 201 may divide the image into slices, tiles and / or bricks. Each brick may contain one or more complete and / or partial CTUs 22. In some embodiments, the partitioning unit 201 may also form one or more sub-pictures, each of which may contain one or more slices, tiles or bricks. The partitioning unit 201 sends the CTU 22 to the prediction unit 202 for further processing.

[0024] The prediction unit 202 is used to generate a prediction block 23 for each CTU 22. The prediction block 23 may be generated based on one or more inter-frame or intra-frame prediction methods by using various interpolation and / or extrapolation schemes. Figure 2 As shown, the prediction unit 202 may also include a block partitioning unit 203, a motion estimation (ME) unit 204, a motion compensation (MC) unit 205, and an intra-frame prediction unit 206. The block partitioning unit 203 is used to divide the CTU 22 into smaller coding units (CUs) or coding blocks (CBs). In some embodiments, a CU may be generated from the CTU 22 by various methods, such as quadtree partitioning, binary partitioning, and trifurcated partitioning. The ME unit 204 is used to estimate changes caused by the motion of an object shown in the input image 20 or the motion of an image acquisition device that generates the input image 20. The MC unit 205 is used to adjust and compensate for changes caused by the above motion. Both the ME unit 204 and the MC unit 205 are used to determine an inter-frame (e.g., at different time points) prediction block of the CU. In some embodiments, the ME unit 204 and the MC unit 205 may use a rate-distortion optimized motion estimation method to determine the inter-frame prediction block.

[0025] The intra prediction unit 206 is used to determine the inter-frame (eg, at the same time point) prediction block of the CU (or a portion of the CU) using various intra prediction modes (including the MIP mode). Figure 3 The details of determining the intra prediction block using the MIP mode (hereinafter referred to as the "MIP process") are discussed. In the MIP process, the intra prediction unit 206 first determines one or more reference samples from the neighboring samples of the CU, for example, directly using the neighboring samples as reference samples, downsampling the neighboring samples, or directly extracting from the neighboring samples (for example, Figure 3 of Step 301).

[0026] Secondly, the intra-frame prediction unit 206 uses the reference samples, the MIP matrix and the shift parameters to determine the prediction samples of multiple sample positions in the CU. The sample positions may be preset sample positions in the CU. For example, the sample positions may be positions with odd horizontal and vertical coordinate values ​​in the CU (e.g., x=1, 3, 5, etc., y=1, 3, 5, etc.). The shift parameters include a shift offset parameter and a shift quantity parameter, which may be used for shift operations when generating prediction samples. With this arrangement, the intra-frame prediction unit 206 may generate prediction samples in the CU (i.e., "MIP prediction" or "MIP prediction block" refers to a collection of such prediction samples) (e.g. Figure 3 of Step 302 ). In some embodiments, the sample position may be a position within the CU having even horizontal and vertical coordinate values.

[0027] Third, the intra prediction unit 206 may determine the predicted sample values ​​(eg, Figure 3 of Step 303 ). In some embodiments, the intra prediction unit 206 may use an interpolation filter to determine the predicted sample values ​​at the remaining positions. Through the above processing, the intra prediction unit 206 may generate a prediction block 23 of a CU in a CTU 22.

[0028] refer to Figure 2 , the prediction unit 202 outputs the prediction block 23 to the adder 207. The adder 207 calculates the difference between the output of the partition unit 201 (e.g., the CU in the CTU 22) and the output of the prediction unit 202 (i.e., the prediction block 23 of the CU). The transform unit 208 reads the residual R and performs one or more transform operations on the prediction block 23 to obtain the coefficient 24 for further use. The quantization unit 209 can quantize the coefficient 24 and output the quantized coefficient 25 (e.g., level) to the inverse quantization unit 210. The inverse quantization unit 210 performs an inverse quantization operation on the quantized coefficient 25 to output the reconstructed coefficient 26 to the inverse transform unit 211. The inverse transform unit 211 performs one or more inverse transforms corresponding to the transform in the transform unit 208 and outputs the reconstructed residual 27.

[0029] Then, the adder 212 calculates the reconstructed CU by adding the reconstructed residual 27 to the prediction block 23 of the CU from the prediction unit 202. The adder 212 also forwards its output 28 to the prediction unit 202 for use as an intra-frame prediction reference. After all CUs of the CTU 22 have been reconstructed, the filtering unit 213 can perform in-loop filtering on the reconstructed picture 29. The filtering unit 213 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping with chroma scaling (LMCS) filter, a neural network-based filter, and other suitable filters for suppressing coding distortion or enhancing the coding quality of the image.

[0030] The filtering unit 213 may then send the decoded image 30 (or sub-image) to a decoded picture buffer (DPB) 214. The DPB 214 outputs a decoded image 31 based on the control information. The image 31 stored in the DPB 214 may also be used as a reference image for the prediction unit 202 to perform inter-frame or intra-frame prediction.

[0031] The entropy coding unit 215 is used to convert the image 31, parameters from the units in the coding system 200, and supplementary information (eg, information used to control or communicate with the system 200) into binary form. The entropy coding unit 215 can generate an output code stream 21 accordingly.

[0032] In some embodiments, the encoding system 200 may be a computing device having a processor and a storage medium having one or more encoding programs. When the processor reads and executes the encoding program, the encoding system 200 may receive an input image 20 and generate an output code stream 21 accordingly. In some embodiments, the encoding system 200 may be a computing device having one or more chips. The units or elements of the encoding system 200 may be implemented as an integrated circuit on a chip.

[0033] Figure 3 is a schematic diagram of a MIP process according to an embodiment of the present application. The MIP process may be implemented by an intra-frame prediction unit (eg, intra-frame prediction unit 206). Figure 3 As shown, the intra-frame prediction unit may include a prediction module 301 and a filtering module 302. Figure 3 As shown, the MIP process includes three steps 301, 302 and 303. The MIP process may generate a prediction block based on a current block or a coding block 300 (eg, a CU or a partition of a CU).

[0034] Step 301

[0035] In step 301, the intra prediction unit 206 may generate reference samples 32, 34 using neighboring samples 31, 33 of the coding block 300. In the illustrated embodiment, the neighboring sample 31 is an upper neighboring sample and the neighboring sample 33 is a left neighboring sample. The intra prediction unit 206 may calculate the average of the values ​​of every two neighboring samples 31, 33 and set these average values ​​as the values ​​of the reference samples 32, 34, respectively. In some embodiments, the intra prediction unit 206 may select the value of one of every two neighboring samples 31 or 33 as the value of the reference sample 32 or 34. In the illustrated embodiment, the intra prediction unit 206 determines 4 reference samples 32 from the 8 upper neighboring samples 31 of the coding block 300 and determines another 4 reference samples 34 from the 8 left neighboring samples 33 of the coding block 300.

[0036] In step 301, the intra prediction unit determines the width and height of the coding block 300, and records them as variables "cbWidth" and "cbHeight", respectively. In some embodiments, the intra prediction unit 206 can use a rate-distortion optimization mode decision process to determine the intra prediction mode (for example, whether to use the MIP mode). In such an embodiment, the coding block 300 can be divided into one or more transform blocks, and the width and height of these transform blocks are recorded as variables "nTbW" and "nTbH", respectively. When the MIP mode is used as the intra prediction mode, the intra prediction unit determines the MIP size identifier (recorded as the variable "mipSizeId") based on the following conditions AC.

[0037] [Condition A] If both "nTbW" and "nTbH" are 4, set "mipSizeId" to 0.

[0038] [Condition B] Otherwise, if "cbWidth" or "cbHeight" is 4, set "mipSizeId" to 1.

[0039] [Condition C] Otherwise, set mipSizeId to 2.

[0040] For example, if the size of the coding block 300 is "8x8" (i.e., both "cbWidth" and "cbHeight" are 8), "mipSizeId" is set to 2. For another example, if the size of the transform block of the coding block 300 is "4x4" (i.e., both "nTbW" and "nTbH" are 4), "mipSizeId" is set to 0. For another example, if the size of the coding block 300 is "4x8", "mipSizeId" is set to 1.

[0041] In the illustrated embodiment, there are three "mipSizeId", namely "0", "1" and "2". Each MIP size identifier (i.e., the variable "mipSizeId") corresponds to a specific way of performing the MIP process (e.g., using a different MIP matrix). In other embodiments, there may be more than three MIP size identifiers.

[0042] Based on the MIP size identifier, the intra prediction unit may determine the variables "boundarySize" and "predSize" based on the following condition DF.

[0043] [Condition D] If mipSizeId is 0, set boundarySize to 2 and predSize to 4.

[0044] [Condition E] If mipSizeId is 1, set boundarySize to 4 and predSize to 4.

[0045] [Condition F] If mipSizeId is 2, set boundarySize to 4 and predSize to 8.

[0046] In the illustrated embodiment, "boundarySize" represents the number of reference samples 32, 34 determined from each of the upper reference sample 31 and the left reference sample 33 of the coding block 300. The variable "predSize" is used in subsequent calculations (i.e., the following equation (C) ).

[0047] In some embodiments, the intra-frame prediction unit may also determine a variable "isTransposed" to indicate the order in which the reference samples 32 and 34 are stored in the temporary array. For example, "isTransposed" being 0 indicates that the intra-frame prediction unit presents the reference sample 32 determined from the upper adjacent sample 31 of the coding block 300 before the reference sample 34 determined from the left adjacent sample 33. Alternatively, "isTransposed" being 1 indicates that the intra-frame prediction unit presents the reference sample 34 determined from the left adjacent sample 33 of the coding block 300 before the reference sample 32 determined from the upper adjacent sample 31. In an embodiment of the encoding system 200, the value of "isTransposed" is sent to the entropy encoding unit (e.g., the entropy encoding unit 215) as one of the parameters of the MIP process that is encoded and written to the codestream (e.g., the output codestream 21). Accordingly, in the present application Figure 4 In the implementation of the decoding system 400, the value of "isTransposed" may be received from a parsing unit (eg, the parsing unit 401) by parsing an input code stream (which may be the output code stream 21).

[0048] The intra prediction unit may also determine a variable "inSize" to indicate the number of reference samples 32, 34 used to determine the MIP prediction value. The value of "inSize" is determined by the following equation (A). In the present application, the meaning and operation of all operators in the equation are the same as the corresponding operators defined in the ITU-T H.265 standard.

[0049] inSize=(2*boundarySize)-(mipSizeId==2)? 1:0; (A)

[0050] For example, "==" is the relational operator "equal to". For example, if "mipSizeId" is 2, "inSize" is 7 (calculated by (2*4)-1). If "mipSizeId" is 1, "inSize" is 8 (calculated by (2*4)-0).

[0051] The intra prediction unit may call the following process to determine a set of reference samples 32, 34, which are stored in an array p[x] ("x" from "0" to "inSize-1"). The intra prediction unit may determine "nTbW" samples from the upper neighboring samples 31 of the coding block 300 (and store them in the array "refT"), and determine "nTbH" samples from the left neighboring samples 33 of the coding block 300 (and store them in the array "refL").

[0052] The intra prediction unit 206 may perform a downsampling process on “refT” to obtain “boundarySize” samples, and store the “boundarySize” samples in “refT”. Similarly, the intra prediction unit 206 may perform a downsampling process on “refL” to obtain “boundarySize” samples, and store the “boundarySize” samples in “refL”.

[0053] In some embodiments, the intra-frame prediction unit may merge the arrays "refT" and "refL" into a single array "pTemp" based on the order indicated by the variable "isTransposed". The intra-frame prediction unit may determine "isTransposed" to indicate the order in which the reference samples are stored in the temporary array "pTemp". For example, "isTransposed" being 0 (or FALSE) indicates that the intra-frame prediction unit presents the reference sample 32 determined from the upper neighboring sample 31 of the coding block 300 before the reference sample 34 determined from the left neighboring sample 33. In other cases, "isTransposed" being 1 (or TRUE) indicates that the intra-frame prediction unit presents the reference sample 34 determined from the left neighboring sample 33 of the coding block 300 before the reference sample 32 determined from the upper neighboring sample 31. In some embodiments, in an embodiment of the encoding system 200, the intra-frame prediction unit may determine the value of "isTransposed" by a rate-distortion optimization method. In some embodiments, in an implementation of the encoding system 200, the intra prediction unit may determine the value of "isTransposed" based on the comparison and / or relationship between the adjacent samples 32, 34 and the encoding block 300. In an implementation of the encoding system 200, the value of "isTransposed" may be forwarded to the entropy encoding unit (e.g., the entropy encoding unit 215) as one of the parameters of the MIP process to be written into the bitstream (e.g., the output bitstream 21). Accordingly, in the present application Figure 4 In the implementation of the decoding system 400, the value of "isTransposed" may be received from a parsing unit (eg, the parsing unit 401) by parsing an input code stream (which may be the output code stream 21).

[0054] The intra prediction unit may determine an array p[x] ("x" from "0" to "inSize-1") based on the following conditions G and H.

[0055] [Condition G] If "mipSizeId" is 2, then p[x] = pTemp[x+1] - pTemp[0].

[0056] [Condition H] Otherwise (e.g., "mipSizeId" is less than 2), then p[0] = pTemp[0] - (1<<(BitDepth-1)) and p[x] = pTemp[x] - pTemp[0] (x is from 1 to "inSize-1").

[0057] In the above condition H, "BitDepth" is the bit depth of the color component (eg, Y component) of the samples in the coding block 300. The symbol "<<" is a shift symbol used in the ITU-T H.265 standard.

[0058] Alternatively, the intra prediction unit may determine an array p[x] (x from 0 to "inSize-1") based on the following conditions I and J.

[0059] [Condition I] If "mipSizeId" is 2, then p[x] = pTemp[x+1] - pTemp[0].

[0060] [Condition J] Otherwise (e.g., "mipSizeId" is less than 2), then p[0] = (1<<(BitDepth-1))-pTemp[0] and p[x] = pTemp[x]-pTemp[0] (x is from 1 to "inSize-1").

[0061] In some embodiments, the intra prediction unit may determine the value of the array p[x] by using a unified calculation method without determining the value of "mipSizeId". For example, the intra prediction unit may append "(1<<(BitDepth-1))" as an additional element to "pTemp" and calculate p[x] as "pTemp[x]-pTemp[0]".

[0062] Step 302

[0063] In step 302, the intra prediction unit (or prediction module 301) determines the MIP prediction value of the coding block 300 by using the above-mentioned set of reference samples 32, 34 and the MIP matrix. The MIP matrix is ​​selected from a set of predefined MIP matrices based on the corresponding MIP mode identifier (i.e., variable "mipModeId") and MIP size identifier (i.e., variable "mipSizeId").

[0064] The MIP prediction value determined by the intra prediction unit includes the values ​​of the partial prediction samples 35 of all or part of the sample positions in the coding block 300. The MIP prediction value is denoted as "predMip[x][y]".

[0065] exist Figure 3 In the illustrated embodiment, the partial prediction samples 35 are the samples marked as gray squares in the current block 300. The reference samples 32, 34 in the array p[x] determined in step 301 are used as inputs to the prediction module 301. The prediction module 301 calculates the partial prediction samples 35 by using the MIP matrix and the shift parameters. The shift parameters include a shift offset parameter and a shift amount parameter. In some embodiments, the prediction module 301 determines the partial prediction samples 35 with coordinates (x, y) based on the following equations (B) and (C):

[0066]

[0067]

[0068] In the above equation (B), parameter "fO" is a shift offset parameter used to determine parameter "oW". Parameter "sW" is a shift amount parameter. "p[i]" is a reference sample. The symbol ">>" is a binary right shift operator as defined in the H.265 standard.

[0069] In the above equation (C) In the example, "mWeight[i][j]" is a MIP weight matrix, where the matrix elements are fixed constants for both encoding and decoding. Alternatively, in some embodiments, an implementation of the encoding system 200 uses an adaptive MIP matrix. For example, the MIP weight matrix can be updated by various training methods using one or more encoded images as input, or using images provided to the encoding system 200 by an external device. The intra-frame prediction unit can forward "mWeight[i][j]" to an entropy coding unit (e.g., entropy coding unit 215) when the MIP mode is determined. The entropy coding unit can then write "mWeight[i][j]" to the bitstream, for example, to one or more special data units in the bitstream containing MIP data. Accordingly, in some embodiments, an implementation of the decoding system 400 using an adaptive MIP matrix can update the MIP matrix using, for example, a training method, wherein the input of the training method is one or more encoded images or blocks, or images from other bitstreams provided to the encoding system 200 by an external device, or images obtained by parsing special data units in the input bitstream containing MIP matrix data by the parsing unit 401.

[0070] The prediction module 301 may determine the values ​​of "sW" and "fO" based on the size of the current block 300 and the MIP mode for the current block 300. In some embodiments, the prediction module 301 may obtain the values ​​of "sW" and "fO" by using a lookup table. For example, the following Table 1 may be used to determine "sW".

[0071] Table 1

[0072] Alternatively, the following Table 2 may also be used to determine "sW".

[0073] Table 2 MipSizeId w 0 5 1 6 2 5

[0074] In some embodiments, the prediction module may set "sW" to a constant. For example, for blocks of different sizes using different MIP modes, the prediction module may set "sW" to "5". For another example, for blocks of different sizes using different MIP modes, the prediction module 301 may set "sW" to "6". For another example, for blocks of different sizes using different MIP modes, the prediction module may set "sW" to "7".

[0075] In some embodiments, the prediction module 301 may determine “fO” using Table 3 or Table 4 below.

[0076] Table 3

[0077] Table 4 MipSizeId fO 0 34 1 23 2 46

[0078] In some embodiments, the prediction module 301 may directly set "f0" to a constant (e.g., a value from 0 to 100). For example, for blocks of different sizes using different MIP modes, the prediction module 301 may set "f0" to "46". For another example, the prediction module 301 may set "f0" to "56". For another example, the prediction module 301 may set "f0" to "66".

[0079] In some embodiments, the intra prediction unit may perform a "clipping" operation on the values ​​of the MIP prediction samples stored in the array "predMip". When "isTransposed" is 1 (or TRUE), the array "predMip[x][y]" of "predSize x preSize" (x from 0 to "predSize-1", y from 0 to "predSize-1") is transposed to "predTemp[y][x] = predMip[x][y]", then "predMip = predTemp".

[0080] More specifically, when the size of the coding block 303 is "8x8" (ie, both "cbWidth" and "cbHeight" are 8), the intra prediction unit may determine an "8x8" "predMip" array.

[0081] Step 303

[0082] exist Figure 3 In step 303, the intra prediction unit determines the prediction sample 37 of the remaining samples in the coding block 300 except for the partial samples 35. Figure 3As shown, the intra prediction unit can use the filtering module 302 to determine the prediction samples 37 of the remaining samples in the coding block 300 except the partial samples 35. The input of the filtering module 302 can be the partial samples 35 in step 302. The filtering module 302 can use one or more interpolation filters to determine the prediction samples 37 of the remaining samples in the coding block 300 except the partial samples 35. The intra prediction unit (or the filtering module 302) can generate a prediction block (which includes multiple prediction samples 37) of the coding block 300 according to the following conditions K and L and store the values ​​of the prediction samples 37 in the array "predSamples[x][y]" (x ranges from 0 to "nTbW-1", y ranges from 0 to "nTbH-1").

[0083] [Condition K] If the intra prediction unit determines that "nTbW" is greater than "predSize" or "nTbH" is greater than "predSize", the intra prediction unit initiates an upsampling process to determine "predSamples" based on "predMip".

[0084] [Condition L] Otherwise, the intra prediction unit sets the prediction value of the coding block 300 to the MIP prediction value of the coding block.

[0085] In other words, the intra prediction unit can set "predSamples[x][y]" (x from 0 to "nTbW-1", y from 0 to "nTbH-1") equal to "predMip[x][y]". For example, the intra prediction unit can set the "predSamples" of the coding block whose size is equal to "8x8" (that is, "cbWidth" and "cbHeight" are both 8) to its "predMip[x][y]".

[0086] Through steps 301-303, the intra prediction unit can generate a prediction value for the current block 300. The generated prediction value can be used for further processing (for example, the above reference Figure 2 Prediction block discussed in 23).

[0087] Figure 4 4 is a schematic diagram of a decoding system 400 according to an embodiment of the present application. The decoding system 400 is used to receive and process an input code stream 40 and transform the input code stream 40 into an output video 41. The input code stream 40 may be a code stream representing a compressed / encoded image / video. In some embodiments, the input code stream 40 may come from an output code stream (e.g., output code stream 21) generated by an encoding system (e.g., encoding system 200).

[0088] The decoding system 400 includes a parsing unit 401, which is used to parse the input code stream 40 to obtain the value of the syntax element therefrom. The parsing unit 401 also converts the binary representation of the syntax element into a numerical value (i.e., a decoding block 42), and forwards these numerical values ​​to the prediction unit 402 (e.g., for decoding). In some embodiments, the parsing unit 401 may also forward one or more variables and / or parameters used to decode the above numerical values ​​to the prediction unit 402.

[0089] The prediction unit 402 is used to determine a prediction block 43 of a decoded block 42 (e.g., a CU or a portion of a CU, such as a transform block). When it is indicated that the decoded block 42 is decoded using an inter-frame decoding mode, a motion compensation (MC) unit 403 of the prediction unit 402 may receive relevant parameters from the parsing unit 401 and decode in the inter-frame decoding mode accordingly. When it is indicated that the decoded block 42 is decoded using an intra-frame prediction mode (e.g., a MIP mode), the intra-frame prediction unit 404 of the prediction unit 402 receives relevant parameters from the parsing unit 401 and decodes in the indicated intra-frame decoding mode accordingly. In some embodiments, the intra-frame prediction mode (e.g., a MIP mode) may be identified by a specific identifier (e.g., a MIP identifier) ​​embedded in the input code stream 40.

[0090] For example, when the MIP mode is identified, the intra-frame prediction unit 404 may be based on the following method (similar to Figure 3 The Steps 301-303 ) determines a prediction block 43 (which comprises a plurality of prediction samples).

[0091] First, the intra-frame prediction unit 404 determines one or more reference samples from the neighboring samples of the decoded block 42 (similar to Figure 3 In Step 301 ). For example, the intra prediction unit 404 may generate a reference sample by downsampling adjacent samples or directly extracting a portion of adjacent samples.

[0092] Then, the intra prediction unit 404 may use the reference samples, the MIP matrix, and the shift parameters to determine the partial prediction samples in the decoding block 42 (similar to Figure 3 In Step 302 ). In some embodiments, the position of the partial prediction sample may be pre-set in the decoding block 42. For example, the position of the partial prediction sample may be a position with odd horizontal and vertical coordinate values ​​within the decoding block. The shift parameters may include a shift offset parameter and a shift amount parameter, which may be used for the shift operation when generating the partial prediction sample.

[0093] Finally, if the partial prediction samples of the decoding block 42 are determined, the intra prediction unit 404 determines the prediction samples of the remaining samples in the decoding block 42 except for the partial prediction samples (similar to Figure 3 of Step 303 ). For example, the intra prediction unit 404 may determine the prediction sample using an interpolation filter, wherein a portion of the prediction sample and neighboring samples are used as inputs of the interpolation filter.

[0094] The decoding system 400 includes a scaling unit 405 whose function is similar to that of the inverse quantization unit 210 of the encoding system 200. The scaling unit 405 performs a scaling operation on the quantization coefficients 44 (eg, levels) from the parsing unit 401, thereby generating reconstruction coefficients 45.

[0095] The function of the transform unit 406 is similar to the function of the inverse transform unit 211 in the encoding system 200. The transform unit 406 performs one or more transform operations (eg, the inverse operations of the one or more transform operations performed by the inverse transform unit 211) to obtain a reconstructed residual 46.

[0096] The adder adds the prediction block 43 from the prediction unit 402 and the reconstructed residual 46 from the transform unit 406 to obtain a reconstructed block 47 of the decoded block 42. The reconstructed block 47 is also sent to the prediction unit 402 for use as a reference (eg, a reference for other blocks encoded in intra prediction mode).

[0097] After all decoded blocks 42 in the image or sub-image are reconstructed (i.e., forming a reconstructed block 48), a filtering unit 408 may perform in-loop filtering on the reconstructed block 48. The filtering unit includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping and chroma scaling (LMCS) filter, a neural network-based filter, etc. In some embodiments, the filtering unit 408 may perform in-loop filtering only on one or more target pixels in the reconstructed block 48.

[0098] The filtering unit 408 then sends the decoded picture 49 (or picture) or sub-picture to a decoded picture buffer (DPB) 409. The DPB 409 outputs the decoded picture as an output video 41 based on the timing and control information. The decoded picture 49 stored in the DPB 409 can be used as a reference picture when the prediction unit 402 performs inter-frame prediction or intra-frame prediction.

[0099] In some embodiments, the decoding system 400 may be a computing device having a processor and a storage medium recording one or more decoding programs. When the processor reads and executes the decoding program, the decoding system 400 may receive an input video bitstream and generate a corresponding decoded video.

[0100] In some embodiments, decoding system 400 may be a computing device having one or more chips. The units or elements of decoding system 400 may be implemented as an integrated circuit on a chip.

[0101] Figure 5 is a schematic diagram of an apparatus 500 according to an embodiment of the present application. The apparatus 500 may be a "sending" apparatus. More specifically, the apparatus 500 is used to acquire, encode, and store / send one or more images. The apparatus 500 includes an acquisition unit 501, an encoder 502, and a storage / sending unit 503.

[0102] The acquisition unit 501 is used to acquire or receive an image and forward the image to the encoder 502. The acquisition unit 501 can also be used to acquire or receive a video consisting of an image sequence and forward the video to the encoder 502. In some embodiments, the acquisition unit 501 can be a device including one or more cameras (e.g., an image camera, a depth camera, etc.). In some embodiments, the acquisition unit 501 can be a device capable of partially or completely decoding a video code stream to generate an image or video. The acquisition unit 501 can also include one or more components to collect audio signals.

[0103] The encoder 502 is used to encode the image from the acquisition unit 501 and generate a video code stream. The encoder 502 can also be used to encode the video from the acquisition unit 501 to generate a code stream. In some embodiments, the encoder 502 can be implemented as Figure 2 The encoding system 200. In some embodiments, the encoder 502 may include one or more audio encoders to encode the audio signal to generate an audio bitstream.

[0104] The storage / transmission unit 503 is used to receive the video and / or audio code stream from the encoder 502. The storage / transmission unit 503 can encapsulate the video code stream and the audio code stream together to form a media file (e.g., an ISO-based media file) or a transport stream. In some embodiments, the storage / transmission unit 503 can write or store the media file or the transport stream into a storage unit, such as a hard disk drive, a disk, a DVD, a cloud storage, a portable storage device, etc. In some embodiments, the storage / transmission unit 503 can send the video / audio code stream to an external device via a transmission network (e.g., the Internet, a wired network, a cellular network, a wireless local area network, etc.).

[0105] Figure 6 6 is a schematic diagram of an apparatus 600 according to an embodiment of the present application. The apparatus 600 may be a "target" apparatus. More specifically, the apparatus 600 is used to receive, decode and present an image or video. The apparatus 600 includes a receiving unit 601, a decoder 602 and a presenting unit 603.

[0106] The receiving unit 601 is used to receive a media file or a transport stream from, for example, a network or a storage device. The media file or the transport stream includes a video code stream and / or an audio code stream. The receiving unit 601 can separate the video code stream and the audio code stream. In some embodiments, the receiving unit 601 can generate a new video / audio code stream by extracting the video / audio code stream.

[0107] The decoder 602 includes one or more video decoders, such as the decoding system 400 discussed above. The decoder 602 may also include one or more audio decoders. The decoder 602 decodes the video bitstream and / or audio bitstream from the receiving unit 601 to obtain a decoded video file and / or one or more decoded audio files (corresponding to one or more channels).

[0108] The rendering unit 603 receives the decoded video / audio file and processes the video / audio file to obtain a suitable video / audio signal for display / playback. These adjustment / reconstruction operations may include one or more of the following: denoising, synthesis, color space conversion, upsampling, downsampling, etc. The rendering unit 603 can improve the quality of the decoded video / audio file.

[0109] Figure 7 700 according to an embodiment of the present application. The communication system 700 includes a source device 701, a storage medium or a transmission network 702, and a target device 703. In some embodiments, the source device 701 may be the above-mentioned Figure 5 The source device 701 sends the media file to the storage medium or the transmission network 702 for storage or transmission. The target device 703 can be the above-mentioned Figure 6 The communication system 700 is used to encode a media file, transmit or store the encoded media file, and then decode the encoded media file. In some embodiments, the source device 701 can be a first smart phone, the storage medium 702 can be a cloud storage, and the target device can be a second smart phone.

[0110] The above embodiments are only illustrations of several embodiments of the present invention, and the description is specific and detailed. The above embodiments cannot be interpreted as limiting the present application. It should be noted that, without departing from the spirit and scope of the present invention, those skilled in the art may make various changes and modifications. Therefore, the scope of the present application shall be subject to the appended claims.

Claims

1. A method for encoding an image, comprising: Determine the width and height of the encoding block; If the width and the height are equal to N, determining a matrix-based intra prediction (MIP) size identifier, the MIP size identifier indicating that the MIP prediction size is equal to N, where N is a positive integer power of 2; determining a set of reference samples for the coding block; and Determining a MIP prediction value for the coding block based on the set of reference samples and a MIP matrix according to the MIP size identifier includes: Determining the MIP prediction value of the coding block based on a set of reference samples, a shift offset parameter, and the MIP matrix according to the MIP size identifier; Different MIP size identifiers correspond to the same shift offset parameter, wherein the shift offset parameter is a constant value.

2. The method according to claim 1, wherein: The determining the MIP prediction value of the coding block based on the set of reference samples and the MIP matrix according to the MIP size identifier further comprises: The MIP prediction value of the coding block is determined based on the following equation: x is from 0 to "predSize-1", y is from 0 to "predSize-1", Among them, "sW" is the shift quantity parameter, "fO" is the shift offset parameter, "inSize" indicates the number of reference samples used to determine the MIP prediction value, "p[i]" represents the reference sample, "predMip[x][y]" represents the MIP prediction value, "mWeight[i][j]" represents the MIP weighting matrix, "predSize" represents the size of the MIP prediction block, "pTemp[0]" represents the 0th value in the reference sample buffer, the symbol "<<" represents a binary left shift operator, and the symbol ">>" represents a binary right shift operator.

3. The method according to claim 2, further comprising: Downsampling the set of reference samples of the coding block to obtain a reference sampling buffer, where the reference sampling buffer contains the set of reference samples of the coding block after downsampling; An input sample value is determined based on the reference samples in the reference sample buffer, the MIP size identifier, and the luma bit depth.

4. The method according to claim 3, wherein: Determining the input sample value according to the reference sample in the reference sample buffer, the MIP size identifier and the luma bit depth comprises: The input sampling value is derived by the following formula: If the MIP size identifier is equal to 2, then p[x] = pTemp[x+1] - pTemp[0]; If the MIP size identifier is less than 2, then Wherein, "p[x]" represents a reference sample, "pTemp[x]" represents the xth value in the reference sample buffer, BitDepth is the brightness bit depth.

5. The method according to claim 4, wherein: In a case where the width and the height of the coding block are equal to 4, the MIP size identifier is equal to 0; In a case where the width multiplied by the height of the coding block is Nx4 or 4xN, the MIP size identifier is equal to 1; In a case where the width and the height of the coding block are not equal to 4 and the width multiplied by the height of the coding block is not Nx4 or 4xN, the MIP size identifier is equal to 2.

6. The method according to claim 2, wherein: The determining the MIP prediction value of the coding block based on the set of reference samples and the MIP matrix according to the MIP size identifier further comprises: determining the shift amount parameter; and The MIP prediction value of the coding block is determined based on the set of reference samples, the shift amount parameter, and the MIP matrix according to the MIP size identifier.

7. The method according to claim 6, wherein: Different MIP size identifiers correspond to the same shift amount parameter.

8. The method according to claim 6, wherein: The shift amount parameter is a constant value.

9. The method according to any one of claims 6 to 8, wherein: The shift amount parameter is equal to 6.

10. The method according to claim 1, wherein: The coding block includes a coding block of size "8x8".

11. The method according to claim 1, further comprising: The set of reference samples of the coding block is determined based on neighboring samples.

12. The method according to claim 11, wherein: The adjacent samples include upper adjacent samples.

13. The method according to claim 11, wherein: The neighboring samples include left neighboring samples.

14. The method according to claim 1, further comprising, The prediction value of the coding block is set equal to the MIP prediction value of the coding block.

15. The method according to claim 1, wherein: The MIP prediction value includes the value of NxN prediction samples.

16. A method for decoding an image, comprising: Determine the width, height and prediction mode of the decoding block; When the prediction mode indicates that the decoded block is decoded using a matrix-based intra prediction (MIP) mode, if the width and the height are equal to N, determining a MIP size identifier indicating that a MIP prediction size is equal to N, where N is a positive integer power of 2; determining a set of reference samples for the decoding block; and Determining a MIP prediction value of the decoding block based on the set of reference samples and a MIP matrix corresponding to the MIP size identifier comprises: determining the MIP prediction value of the decoding block based on a set of reference samples, a shift offset parameter, and the MIP matrix corresponding to the MIP size identifier; Different MIP size identifiers correspond to the same shift offset parameter, wherein the shift offset parameter is a constant value.

17. The method according to claim 16, wherein: The determining the MIP prediction value of the decoding block based on the set of reference samples and the MIP matrix corresponding to the MIP size identifier further comprises: The MIP prediction value of the decoded block is determined based on the following equation: x is from 0 to "predSize-1", y is from 0 to "predSize-1", Among them, "sW" is the shift quantity parameter, "fO" is the shift offset parameter, "inSize" indicates the number of reference samples used to determine the MIP prediction value, "p[i]" represents the reference sample, "predMip[x][y]" represents the MIP prediction value, "mWeight[i][j]" represents the MIP weighting matrix, "predSize" represents the size of the MIP prediction block, "pTemp[0]" represents the 0th value in the reference sample buffer, the symbol "<<" represents a binary left shift operator, and the symbol ">>" represents a binary right shift operator.

18. The method according to claim 17, further comprising: Downsampling the set of reference samples of the decoding block to obtain a reference sample buffer, wherein the reference sample buffer contains the set of reference samples of the decoding block after downsampling; An input sample value is determined based on the reference samples in the reference sample buffer, the MIP size identifier, and the luma bit depth.

19. The method according to claim 18, wherein: Determining the input sample value according to the reference sample in the reference sample buffer, the MIP size identifier and the luma bit depth comprises: The input sampling value is derived by the following formula: If the MIP size identifier is equal to 2, then p[x] = pTemp[x+1] - pTemp[0]; If the MIP size identifier is less than 2, then Wherein, "p[x]" represents a reference sample, "pTemp[x]" represents the xth value in the reference sample buffer, BitDepth is the brightness bit depth.

20. The method according to claim 19, wherein: In case the width and the height of the decoding block are equal to 4, the MIP size identifier is equal to 0; In the case where the width multiplied by the height of the decoding block is Nx4 or 4xN, the MIP size identifier is equal to 1; In case the width and the height of the decoding block are not equal to 4 and the width multiplied by the height of the decoding block is not Nx4 or 4xN, the MIP size identifier is equal to 2.

21. The method according to claim 17, wherein: The determining the MIP prediction value of the decoding block based on the set of reference samples and the MIP matrix corresponding to the MIP size identifier further comprises: determining the shift amount parameter; and The MIP prediction value of the decoded block is determined based on the set of reference samples, the shift amount parameter, and the MIP matrix corresponding to the MIP size identifier.

22. The method according to claim 21, wherein: Different MIP size identifiers correspond to the same shift amount parameter.

23. The method according to claim 21, wherein: The shift amount parameter is a constant value.

24. The method according to any one of claims 21 to 23, wherein: The shift amount parameter is equal to 6.

25. The method of claim 16, wherein: The decoding block includes a decoding block of size "8x8".

26. The method according to claim 16, further comprising The set of reference samples for the decoded block is determined based on neighboring samples.

27. The method according to claim 26, wherein: The adjacent samples include upper adjacent samples.

28. The method according to claim 26, wherein: The neighboring samples include left neighboring samples.

29. The method according to claim 16, further comprising The prediction value of the decoded block is set equal to the MIP prediction value of the decoded block.

30. The method of claim 16, wherein: The MIP prediction value includes the value of NxN prediction samples.

31. A coding device for encoding an image, comprising: a partitioning unit, configured to receive an input image and partition the input image into one or more coding blocks; a prediction unit configured to determine a width and a height of the coding block, wherein if the width and the height are equal to N, the prediction unit is configured to determine a matrix-based intra prediction (MIP) size identifier, the MIP size identifier indicating that a MIP prediction size is equal to N, N being a positive integer power of 2, and wherein the prediction unit is configured to determine a set of reference samples for the coding block, and the prediction unit is configured to determine a MIP prediction value for the coding block based on the set of reference samples and a MIP matrix according to the MIP size identifier; and An entropy coding unit, configured to transform parameters so as to write the MIP prediction value into a bitstream; The prediction unit is specifically configured to determine the MIP prediction value of the coding block based on a set of reference samples, a shift offset parameter, and the MIP matrix according to the MIP size identifier; Different MIP size identifiers correspond to the same shift offset parameter, wherein the shift offset parameter is a constant value.

32. A decoding device for decoding an image, comprising: A parsing unit for determining the width, height and prediction mode of a decoded block; an intra prediction unit for, when the prediction mode indicates that the decoded block is decoded using a matrix-based intra prediction (MIP) mode, determining a MIP size identifier if the width and the height are equal to N, the MIP size identifier indicating that a MIP prediction size is equal to N, wherein N is a positive integer power of 2, and wherein the intra prediction unit is for determining a set of reference samples of the decoded block, and wherein the intra prediction unit is for determining a MIP prediction value of the decoded block based on the set of reference samples and a MIP matrix corresponding to the MIP size identifier; The intra prediction unit is specifically configured to determine the MIP prediction value of the decoding block based on a set of reference samples, a shift offset parameter, and the MIP matrix corresponding to the MIP size identifier; Different MIP size identifiers correspond to the same shift offset parameter, wherein the shift offset parameter is a constant value.

33. A system for processing an image, the system comprising: The encoding device as claimed in claim 31 and the decoding device as claimed in claim 32.

Images