Intraframe prediction method, apparatus, and decoder and encoder

Abstract

Provided is an intraframe prediction method, an apparatus, and an encoder and a decoder. In the embodiment of the present application, two or more different intraframe prediction modes are respectively adopted to perform intraframe prediction on a pending block, two or more prediction blocks are obtained, and the obtained two or more prediction blocks are combined based on a weight matrix, thereby obtaining a prediction block for the pending block. The embodiment of the present application determines multiple prediction blocks according to multiple intraframe prediction modes, realizes prediction processing of complex textures, improves the quality of intraframe prediction, and thereby improves compression performance. Furthermore, by applying the intraframe prediction method provided in the embodiment of the present application, a diversified weight matrix is ​​used to provide a guarantee for prediction processing of more complex textures, improves the quality of intraframe prediction, and thereby improves compression performance.

Description

[Technical field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) The present application relates to video processing technology, and in particular to an intraframe prediction method, apparatus, and decoder and encoder. [Background technology]

[0002] There exists strong correlation between adjacent pixels in one frame of a video. In video coding and decoding technology, intra-frame prediction methods are used to remove the spatial redundancy between adjacent pixels, thereby improving the coding efficiency.

[0003] In a typical intra prediction mode, prediction can be performed for simple textures. Complex textures need to be divided into smaller blocks or more residuals need to be coded, which increases the complexity of intra prediction. That is, the quality of intra prediction is reduced at the expense of higher distortion or higher complexity in the related intra prediction solutions. Summary of the Invention

[0004] The following is a summary of the invention described herein. This summary is not intended to limit the scope of protection of the claims.

[0005] The present application provides an intraframe prediction method, an apparatus, and a decoder and an encoder, which can improve the quality of intraframe prediction.

[0006] An embodiment of the present application provides an intra-frame prediction method applied to a decoder, the intra-frame prediction method comprising: performing intra-frame prediction on a pending block by adopting two or more different intra-frame prediction modes, respectively, to obtain two or more types of predicted blocks corresponding to the different intra-frame prediction modes; and obtaining a target predicted block for the pending block based on the weight matrix and the obtained two or more predicted blocks.

[0007] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, the computer-executable instructions being for implementing the intra-frame prediction method according to any one of the above claims.

[0008] An embodiment of the present application provides a decoder including a memory and a processor, the memory storing instructions executable by the processor, the instructions being for implementing steps of the intra-frame prediction method according to any one of the preceding claims.

[0009] An embodiment of the present application provides a decoder including a decoding module, a prediction module, and a combination module.

[0010] The decoding module is configured to perform decoding on the received codestream to obtain two or more different intra-frame prediction modes, pending blocks, and weight matrices; The prediction module is configured to perform intra-frame prediction on the pending block by adopting two or more different intra-frame prediction modes, respectively, to obtain two or more types of prediction blocks corresponding to the different intra-frame prediction modes; The combination module is configured to derive a target prediction block for the pending block based on the weight matrix and the obtained two or more prediction blocks.

[0011] An embodiment of the present application provides an intra-frame prediction method applied to an encoder, the intra-frame prediction method comprising: performing intra-frame prediction on a pending block by adopting two or more different intra-frame prediction modes, respectively, to obtain two or more types of predicted blocks corresponding to the different intra-frame prediction modes; and obtaining a target predicted block for the pending block based on the weight matrix and the obtained two or more predicted blocks.

[0012] An embodiment of the present application provides a computer-readable storage medium having stored thereon computer-executable instructions for implementing the intra-frame prediction method according to any one of the above claims, applied to an encoder.

[0013] An embodiment of the present application provides an encoder including a memory and a processor, the memory storing instructions executable by the processor for implementing steps of the intra-frame prediction method according to any one of the above claims applied to the encoder.

[0014] An embodiment of the present application provides an encoder including a prediction module, a combination module, and a processing module.

[0015] The prediction module is configured to perform intra-frame prediction on the pending block by adopting two or more different intra-frame prediction modes, respectively, to obtain two or more types of prediction blocks corresponding to the different intra-frame prediction modes; The combination module is configured to obtain a target prediction block of the pending block based on the weight matrix and the obtained two or more prediction blocks; The processing module is configured to perform trials for all or some of the possible combinations of prediction modes and weight matrix derivation modes, calculate a loss cost, select a combination with a smaller loss cost, determine the two or more different intra-frame prediction modes and weight matrices in the combination as two or more different intra-frame prediction modes and weight matrices used for intra-frame prediction, and write information of the determined two or more different intra-frame prediction modes and weight matrix derivation modes into the codestream according to the grammar.

[0016] An embodiment of the present application provides an intra-frame prediction method, the intra-frame prediction method comprising: performing intra-frame prediction on a pending block by employing each of two or more different intra-frame prediction modes; After completing prediction of the preset number of pixel points by prediction of each type of intraframe prediction mode, obtaining a preset number of predicted pixel points of the block to be processed according to the weight matrix and the pixel points corresponding to each intraframe prediction mode that has been predicted; and obtaining a target predicted block for the pending block based on the obtained plurality of preset quantities of predicted pixel points.

[0017] An embodiment of the present application provides an intra-frame prediction device, which includes a prediction module and a combination module.

[0018] The prediction module is configured to perform intra-frame prediction on a waiting block by adopting each of two or more different intra-frame prediction modes obtained by decoding, and obtain two or more types of prediction blocks corresponding to the different intra-frame prediction modes; The combination module is configured to derive a target prediction block for the pending block based on the weight matrix and the obtained two or more prediction blocks.

[0019] The intra prediction method, device, and encoder and decoder provided in the embodiments of the present application are applied to perform intra prediction on a pending block by adopting two or more different intra prediction modes, respectively, to obtain two or more predicted blocks. Then, the obtained two or more predicted blocks are combined based on a weight matrix, thereby obtaining a predicted block for the pending block. The embodiments of the present application determine multiple predicted blocks according to multiple intra prediction modes, realize complex texture prediction processing, improve the quality of intra prediction, and thereby improve compression performance.

[0020] Furthermore, by applying the intra prediction method provided in the embodiment of the present application, a diversified weight matrix is ​​used to provide a guarantee for a more complex texture prediction process, thereby improving the quality of the intra prediction, and thereby improving the compression performance. Furthermore, the intra prediction method provided in the embodiment of the present application can be applied to more scenes.

[0021] Other features and advantages of the embodiments of the present application will be set forth in the following specification, will be apparent from the specification, or may be learned by the present application. The objectives and other advantages of the present application may be realized and obtained by the structure particularly set forth in the written description, claims, and drawings.

[0022] Other aspects may be appreciated after reading and understanding the drawings and detailed description. [Brief description of the drawings]

[0023] [Figure 1(a)] FIG. 1 is a schematic diagram of a block-based hybrid coding framework in an embodiment of the present application; [Figure 1(b)] FIG. 1 is a structural block diagram of an example of a video encoding system according to an embodiment of the present application; [Figure 1(c)] FIG. 1 is a schematic block diagram of an example of a video decoding system according to an embodiment of the present application; [Diagram 2] 1 is a schematic diagram of an embodiment of an intra-frame prediction method according to an embodiment of the present application; [Diagram 3] FIG. 2 is a schematic diagram of an embodiment of implementing intra-frame prediction using four reference rows / columns in an embodiment of the present application; [Figure 4] FIG. 2 is a schematic diagram of nine modes of intra-frame prediction for 4×4 blocks in H.264 according to an embodiment of the present application. [Diagram 5] FIG. 11 is a weight diagram of 64 modes of GPM in a square block in an embodiment of the present application. [Figure 6] FIG. 11 is a weight diagram of 56 types of modes of AWP in a square block in an embodiment of the present application. [Figure 7] 1 is a flowchart of an example of an intra-frame prediction method according to an embodiment of the present application. [Figure 8] FIG. 2 is a schematic diagram illustrating intra-frame prediction using two different intra-frame prediction modes in an embodiment of the present application; [Figure 9(a)] FIG. 1 is a schematic diagram in which the change position of the weight is represented as a straight line in an embodiment of the present application. [Figure 9(b)] FIG. 1 is a schematic diagram showing the position of change in weighting as a curve in an embodiment of the present application. [Figure 10] FIG. 2 is a schematic diagram of the process of a first embodiment of the mutually exclusive cases in the present application. [Figure 11] FIG. 13 is a schematic diagram of the process of the second embodiment of the mutually exclusive case in the present application. [Figure 12] FIG. 2 is a schematic diagram of storing intra-frame prediction modes in an embodiment of the present application; [Figure 13] 1 is a schematic structural diagram of an intra-frame prediction device according to an embodiment of the present application; [Figure 14] 11 is a flowchart of another example of an intra-frame prediction method according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] The drawings described above are for providing further understanding of the present disclosure and constitute a part of this application. The exemplary embodiments and the description thereof are for interpreting the present application and are not intended to be limiting of the present application.

[0025] The present application will be described in detail below with reference to the drawings and embodiments. It should be noted that, unless conflicting, the embodiments and features in the embodiments in the present application can be combined with each other.

[0026] The intra-frame prediction method provided in the embodiments of the present application is applied to the basic process of a video codec in a block-based hybrid coding framework as shown in FIG. 6(a), but is not limited to this framework and process.

[0027] The basic working principle of the video codec in the block-based hybrid coding framework shown in Figure 1(a) is as follows: on the coding side, a frame image is divided into blocks, intra-frame prediction is used for the current block to generate a predicted block of the current block, and the predicted block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain a quantized coefficient matrix, and the quantized coefficient matrix is ​​entropy coded and output into a code stream.

[0028] Here, in the above image frame block division, each frame is divided into square largest coding units (LCU) of the same size (for example, 128×128, 64×64, etc.). Each largest coding unit can be divided into rectangular coding units (CU) according to a rule. The coding unit can be further divided into prediction units (PU), transform units (TU), etc.

[0029] On the decoding side, on the one hand, the current block is subjected to intraframe prediction or interframe prediction to generate a prediction block of the current block, and on the other hand, the code stream is analyzed to obtain a quantization coefficient matrix, and the quantization coefficient matrix is ​​subjected to inverse quantization and inverse transformation to obtain a residual block, and the prediction block and the residual block are summed to obtain a reconstructed block. The reconstructed block constitutes a reconstructed image, and the reconstructed image is subjected to in-loop filtering based on the image or block to obtain a decoded image. The encoding side also needs to obtain a decoded image by a similar operation to the decoding side. The decoded image can be a reference frame for predicting the subsequent frame. The decoded image obtained on the encoding side is usually also called a reconstructed image. During prediction, the current block can be divided into prediction units, and during transformation, the current block can be divided into transformation units, and the division of the prediction units and the transformation units can be different. The block division information, prediction, transformation, quantization, entropy coding, in-loop filtering, and other mode information or parameter information determined on the encoding side need to be output to the code stream as necessary. The decoding side determines the same block division information, mode information or parameter information such as prediction, transformation, quantization, entropy coding, in-loop filtering, etc. as the encoding side through analysis and analysis based on existing information, thereby ensuring that the decoded image obtained at the encoding side is the same as the decoded image obtained at the decoding side.

[0030] The intra prediction method provided in the embodiment of the present application functions by an intra prediction module in a frame shown in Fig. 1(a) and can be applied to the encoding side and also to the decoding side. At the encoding side, information such as an adopted intra prediction mode and a weight matrix is ​​determined, and intra prediction in the present application is performed based on the determined intra prediction mode and weight matrix. At the decoding side, information such as an adopted intra prediction mode and a weight matrix is ​​obtained by decoding the code stream, and intra prediction in the present application is performed based on the obtained intra prediction mode and weight matrix.

[0031] FIG. 1(b) is a structural block schematic diagram of an example of a video encoding system in an embodiment of the present application. As shown in FIG. 1(b), the video encoding system 11 can include a transform unit 111, a quantization unit 112, a mode selection and coding control logic unit 113, an intraframe prediction unit 114, an interframe prediction unit 115 (including motion compensation and motion estimation), an inverse quantization unit 116, an inverse transform unit 117, an in-loop filtering unit 118, an encoding unit 119, and a decoded image cache unit 110. For an input original video signal, a video reconstruction block can be obtained by dividing a coding tree block (CTU), and a coding mode is determined by the mode selection and coding control logic unit 113. Then, for residual pixel information obtained by intraframe prediction or interframe prediction, the transform unit 111 and the quantization unit 112 perform a transform on the video reconstruction block, which transforms the residual information from the pixel domain to a transform domain, and quantizes the obtained transform coefficients, thereby further reducing the bit rate. The intra prediction unit 114 is for performing intra prediction on the video reconstruction block. Wherein, the intra prediction unit 114 is for determining an optimal intra prediction mode (i.e., a target prediction mode) of the video reconstruction block, and the inter prediction unit 115 is for performing inter prediction coding on one or more blocks in one or more reference frames of the received video reconstruction block to provide temporal prediction information. Wherein, the motion estimation is a process of generating a motion vector, which can estimate the motion of the video reconstruction block, and then the motion compensation performs motion compensation based on the motion vector determined by the motion estimation. After determining the inter prediction mode, the inter prediction unit 115 is further for providing the selected inter prediction data to the encoding unit 119, and also for sending the calculated and determined motion vector data to the encoding unit 119.Besides, the inverse quantization unit 116 and the inverse transform unit 117 are for reconstructing the video reconstruction block, reconstructing a residual block in pixel domain, filtering the reconstructed residual block by the in-loop filtering unit 118 to remove blocking artifacts, and then adding the reconstructed residual block to one predictive block in a frame in the decoded image cache unit 110, thereby generating a reconstructed video reconstruction block. The encoding unit 119 is for encoding various coding parameters and the quantized transform coefficients. The decoded image cache unit 110 is for storing the reconstructed video reconstruction block as a prediction reference. As the video image encoding progresses, new reconstructed video reconstruction blocks are continuously generated, and all of these reconstructed video reconstruction blocks are stored in the decoded image cache unit 110.

[0032] Fig. 1(c) is a structural block schematic diagram of an example of a video decoding system in an embodiment of the present application. As shown in Fig. 1(c), the video decoding system 12 may include a decoding unit 121, an inverse transform unit 127, an inverse quantization unit 122, an intraframe prediction unit 123, a motion compensation unit 124, an in-loop filtering unit 125, and a decoded image cache unit 126. After the input video signal is encoded by the video encoding system 11, a code stream of the video signal is output. The code stream is input to the video decoding system 12, which first obtains decoded transform coefficients by the decoding unit 121, and processes the transform coefficients by the inverse transform unit 127 and the inverse quantization unit 122, thereby generating a residual block in pixel domain. The intraframe prediction unit 123 may be for generating prediction data of a current video decoding block according to the determined intraframe prediction direction and data of a previously decoded block of a current frame or image. The motion compensation unit 124 determines prediction information for the video decoded block by analyzing the motion vector and other related grammar elements, and uses the prediction information to generate a predictive block for the video decoded block being decoded. The decoded video block is generated by performing summation on the residual block from the inverse transform unit 127 and the inverse quantization unit 122 and the corresponding predictive block generated by the intraframe prediction unit 123 or the motion compensation unit 124. The decoded video signal can be filtered by the in-loop filtering unit 125 to remove blocking artifacts and improve the video quality. The decoded video block is then stored in the decoded image cache unit 126, which stores the reference image for subsequent intraframe prediction or motion compensation, and at the same time outputs the video signal to obtain the restored original video signal.

[0033] The intra prediction method provided in the embodiment of the present application is implemented by the intra prediction unit 114 of the video encoding system 11 and the intra prediction unit 123 of the video decoding system 12, to perform prediction on a current block (a block to be encoded or a block to be decoded) and thereby obtain a corresponding predicted block. That is, the intra prediction method provided in the embodiment of the present application may be based on the intra prediction in the video encoding method, or may be based on the intra prediction in the video decoding method.

[0034] In the intraframe prediction method, prediction is performed on the current block using reconstructed pixels that have been coded and decoded around the current block as reference pixels. For example, as shown in FIG. 2, a white 4×4 block is the current block, and the gray pixels in the left row and upper column of the current block are reference pixels for the current block, and intraframe prediction uses these reference pixels to perform prediction on the current block. These reference pixels may all be obtainable, i.e., all have been coded and decoded, or some of them may be unobtainable. For example, if the current block is the leftmost block in the frame, the reference pixels on the left side of the current block are unobtainable. Or, if the lower left part of the current block has not been coded and decoded when coding and decoding the current block, the reference pixels on the lower left side are also unobtainable. If the reference pixels are unobtainable, filling may be performed using obtainable reference pixels or several values ​​or several methods, or filling may not be performed. In the multiple reference line (MRL) intraframe prediction method, the efficiency of coding can be improved by using more reference pixels. For example, as shown in FIG. 3, FIG. 3 is a schematic diagram of an embodiment of implementing intra-frame prediction using four reference rows / columns in the related art.

[0035] Intra prediction has a number of modes. As shown in FIG. 4, FIG. 4 shows nine modes of intra prediction for a 4×4 block in H.264. Here, in mode 0, the pixel above the current block is copied to the current block as a predicted value along the vertical direction. In mode 1, the reference pixel on the left side is copied to the current block as a predicted value along the horizontal direction. In mode 2 (DC), the average value of the eight points A to D and I to L is used as the predicted value of all points. In each of modes 3 to 8, the reference pixel is copied to the corresponding position of the current block according to a certain angle. Some positions of the current block cannot completely correspond to the reference pixel. Therefore, the weighted average value of the reference pixel, that is, the subpixel of the reference pixel for interpolation, may be used. In addition, in related technologies, there are also modes such as Plane and Planar. As technology develops and blocks are enlarged, the number of angle prediction modes increases. For example, the intraframe prediction modes used in High Efficiency Video Coding (HEVC) are Planar, DC, and 33 types of angle modes, totaling 35 types of prediction modes.Further, for example, the intraframe modes used in VVC are Planar, DC, and 65 types of angle modes, totaling 67 types of prediction modes.Further, for example, AVS3 uses DC, Plane, Bilinear, and 63 types of angle modes, totaling 66 types of prediction modes.

[0036] Furthermore, some techniques improve intraframe prediction, such as improved sub-pixel interpolation of reference pixels, filtering of predicted pixels, etc. For example, Multiple Intra Prediction Filter (MIPF) in AVS3 generates predicted values ​​using different filters for different block sizes. For pixels at different positions in the same block, one type of filter is used to generate predicted values ​​for pixels close to the reference pixel, and another type of filter is used to generate predicted values ​​for pixels far from the reference pixel. The technique of filtering predicted pixels can include, for example, Intra Prediction Filter (IPF) in AVS3, which can filter predicted values ​​using reference pixels.

[0037] Intra prediction includes intra prediction modes such as directional angle (DC) mode, planar mode, smooth mode, bilinear mode, etc. However, these modes can only handle simple texture prediction. There are many angle modes, however, the prediction of these modes can only be performed along a line of one angle.

[0038] The Versatile Video Coding (VVC, also known as H.266) standard will introduce the Geometric Partitioning Mode (GPM) inter-frame prediction mode. The Audio Video coding Standard (AVS3), developed independently by China, will introduce the Angular Weighted Prediction (AWP) inter-frame prediction mode.

[0039] GPM or AWP uses two reference blocks with the same size as the current block. In this case, in some pixel positions, the pixel value of the corresponding position of the first reference block is used 100%, in other pixel positions, the pixel value of the corresponding position of the second reference block is used 100%, and in the boundary area, the pixel values ​​of the corresponding positions of the two reference blocks are used according to a certain proportion. These weights are specifically determined by the mode of GPM or AWP. GPM or AWP may use two reference blocks with different sizes from the current block, that is, use a part of each as a reference block as necessary, that is, use the part with a weight other than 0 as a reference block, and remove the part with a weight of 0.

[0040] 5 is a weight diagram of 64 modes of square blocks of GPM in an embodiment of the present application. As shown in FIG. 5, black represents a weight value of the corresponding position of the first reference block being 0%, white represents a weight value of the corresponding position of the first reference block being 100%, and gray area represents a weight value of the corresponding position of the first reference block being 0% to 100% according to the difference in color depth. The weight value of the corresponding position of the second reference block is a value obtained by subtracting the weight value of the corresponding position of the first reference block from 100%.

[0041] 6 is a weight diagram of 56 types of modes of square blocks of AWP in an embodiment of the present application. As shown in FIG. 6, black represents a weight value of the corresponding position of the first reference block being 0%, white represents a weight value of the corresponding position of the first reference block being 100%, and gray area represents a weight value of the corresponding position of the first reference block being 0% to 100% according to the difference in color depth. The weight value of the corresponding position of the second reference block is a value obtained by subtracting the weight value of the corresponding position of the first reference block from 100%.

[0042] GPM and AWP have different weight derivation methods. For GPM, we determine the angle and offset amount based on each type of mode, and then calculate the weight matrix for each mode. For AWP, we first create a one-dimensional weight line, and then tile the entire matrix with the one-dimensional weight line using a method similar to intra-frame angle prediction.

[0043] In previous coding and decoding techniques, only rectangular division methods exist for the division of CU, PU, ​​or TU. In contrast, GPM and AWP achieve the effect of non-rectangular division of prediction without division. GPM and AWP use a mask of the weights of two reference blocks, i.e., the weight diagram above. This mask determines the weights of the two reference blocks when generating a prediction block, or, as can be easily understood, some of the positions of the prediction block are from the first reference block and the other is from the second reference block, and the transition area is obtained by weighting the corresponding positions of the two reference blocks, so that the transition becomes smoother. In GPM and AWP, the current block is not divided into two CUs or PUs based on the division line. And the transform, quantization, inverse transform, inverse quantization, etc. of the residual after prediction are processed as a whole current block.

[0044] The intra prediction method provided in the embodiment of the present application can include the steps of: adopting two or more different intra prediction modes to perform intra prediction on a pending block, and obtaining two or more prediction blocks corresponding to the different intra prediction modes; and obtaining a prediction block for the pending block according to a weight matrix and the obtained two or more prediction blocks. The embodiment of the present application determines multiple prediction blocks according to multiple intra prediction modes, realizes complex texture prediction processing, improves the quality of intra prediction, and thereby improves compression performance.

[0045] Furthermore, by applying the intra prediction method provided in the embodiment of the present application, a diversified weight matrix is ​​used to provide a guarantee for a more complex texture prediction process, thereby improving the quality of the intra prediction, and thereby improving the compression performance. Furthermore, the intra prediction method provided in the embodiment of the present application can be applied to more scenes.

[0046] 7 is a flow chart of an intra-frame prediction method in an embodiment of the present application. As shown in FIG. 7, the intra-frame prediction method includes the following steps:

[0047] In step 700, two or more different intra prediction modes are employed to perform intra prediction on a pending block, and two or more predicted blocks corresponding to the different intra prediction modes are obtained.

[0048] In the embodiment of the present application, the processing-awaiting block may be a block to be encoded to be processed by an encoder, or a block to be decoded to be processed by a decoder.

[0049] In one illustrative example, the intra prediction modes may include, but are not limited to, intra prediction modes such as DC mode, Planar mode, Plane mode, Bilinear mode, and Angular Prediction (AP) mode, and intra prediction improvement techniques, such as improved sub-pixel interpolation of reference pixels, filtering of predicted pixels, Multiple Combination Intra Prediction Filter (MIPF), Intra Prediction Filter (IPF), etc.

[0050] An intra-frame prediction mode that independently generates a prediction block without depending on other intra-frame prediction modes is called a first intra-frame prediction mode (also referred to herein as a basic intra-frame prediction mode), and may include intra-frame prediction modes such as DC mode, Planar mode, Plane mode, Bilinear mode, and angle prediction mode. That is, the basic intra-frame prediction mode may determine a prediction block when a reference pixel and a basic intra-frame prediction mode are determined. An intra-frame prediction mode that may determine a prediction block depending on a basic intra-frame prediction mode is called a second intra-frame prediction mode (also referred to herein as an improved intra-frame prediction mode), and may include an improved technique for intra-frame prediction such as MIPF and IPF. That is, the improved intra-frame prediction mode may not independently generate a prediction block. For example, a basic intra-frame prediction mode, such as a certain type of angle prediction mode, may determine and generate a prediction block based on a reference pixel. An improved intra-frame prediction mode, such as MIPF, may use different filters for pixels at different positions based on the above angle prediction mode, thereby generating or determining a prediction block.

[0051] In one illustrative example, the two or more different intra prediction modes include at least one base intra prediction mode.

[0052] In one illustrative example, two different intra prediction modes are adopted to perform intra prediction on the pending block, and the two different intra prediction modes are both basic intra prediction modes.

[0053] In one illustrative example, the base intra prediction mode may be combined with the improved intra prediction mode, i.e. the base intra prediction mode employed may be further combined with the improved intra prediction mode to perform prediction for the pending block.

[0054] In one illustrative example, two different intra prediction modes are adopted to perform intra prediction on pending blocks, and the two different intra prediction modes include one basic intra prediction mode and one improved intra prediction mode. For example, the first intra prediction mode and the second intra prediction mode both use the same type of angle prediction mode, but the first intra prediction mode does not use a certain type of improved intra prediction mode, for example, the IPF of the certain type of improved intra prediction mode, and the second intra prediction mode uses this type of improved intra prediction mode, for example, the IPF of the improved intra prediction mode. Furthermore, for example, the first intra prediction mode and the second intra prediction mode use the same type of angle prediction mode, but the first intra prediction mode uses one type of selection of a certain type of improved intra prediction mode, and the second intra prediction mode uses another type of selection of the certain type of improved intra prediction mode.

[0055] In the embodiment of the present application, the prediction of the pending block adopts at least two different intra-frame prediction modes, in this way, the prediction of the pending block can be performed from multiple viewpoints, which is suitable for processing the prediction of complex texture, and can improve the quality of the intra-frame prediction.

[0056] The above intraframe prediction method steps are applied to the encoder and also to the decoder.

[0057] In one illustrative example, on the decoder side, the pending block is a pending block to be decoded, and before step 700, the intraframe prediction method further comprises: The method includes analyzing the codestream to obtain two or more different intra-frame prediction modes, pending blocks, and weighting matrices.

[0058] In one illustrative example, on the encoder side, before step 700, the intra-frame prediction method further comprises: The method includes a step of performing trials for all or some of the possible combinations of prediction modes and weight matrix derivation modes, calculating a loss cost, and selecting a combination with a small loss cost, and a step of determining two or more different intra-frame prediction modes and weight matrices in the combination as two or more different intra-frame prediction modes and weight matrices used for intra-frame prediction.

[0059] The intraframe prediction method further includes a step of writing information on the determined two or more different intraframe prediction modes and a weighting matrix derivation mode into a codestream according to a grammar.

[0060] Here, the combination of all possible prediction modes and weight matrix derivation modes includes all possible modes of the first intra-frame prediction mode, all possible modes of the second intra-frame prediction mode, and all possible combinations of the weight matrix derivation modes. For example, assuming that there are 66 types of all available intra-frame prediction modes, there are 66 possible types of the first intra-frame prediction mode, and there are 65 possible types of the second intra-frame prediction mode, which is necessarily different from the first intra-frame prediction mode. If there are 56 types of weight matrix derivation modes (taking AWP as an example), the combination of any two different available intra-frame prediction modes and any one available weight matrix derivation mode has 66×65×56 types.

[0061] In one illustrative example, the manner of calculating the loss cost may include one or any combination of algorithms such as Sum of Absolute Differences (SAD), Sum of Absolute Transformed Difference (SATD), and Rate Distortion Optimization (RDO).

[0062] In one illustrative example, SATD and / or SAD are employed to perform a first selection, e.g., a rough selection, to determine candidate combinations from all or a portion of the possible combinations of prediction modes and weight matrix derivation modes, and then RDO is employed to perform a second selection, e.g., a fine selection, to determine the combination with the smallest loss cost from the candidate combinations.

[0063] In one embodiment, the coarse selection may include using some fast algorithms to reduce the number of trials, for example, if the cost of an intra-frame angular prediction mode is high, a pre-set number of intra-frame prediction modes adjacent to the intra-frame angular prediction mode are not tried.

[0064] In one illustrative example, before trying the above combinations of prediction modes and weight matrix derivation modes, the intra-frame prediction method further comprises: This may include performing an analysis on the texture of the current pending block, for example by gradient.

[0065] Further attempts at combinations of the above prediction modes and weight matrix derivation modes include: The method may include determining a trial intra-frame prediction mode based on the texture analysis.

[0066] For example, for a direction of rough texture (e.g., larger than a preset high threshold) of a current block waiting to be processed, a trial is performed by selecting as many intraframe prediction modes as possible that approximate the rough texture direction (i.e., within a range of a predetermined angle value from the direction) in a rough trial selection of combinations of prediction modes and weight matrix derivation modes.Furthermore, for a direction of smooth texture (e.g., smaller than a preset low threshold) of a current block waiting to be processed, a trial is performed by selecting as few intraframe prediction modes as possible that approximate the smooth texture direction.

[0067] It should be noted that the loss cost includes not only the cost of the codewords occupied in the code stream according to the first intraframe prediction mode, the second intraframe prediction mode, and the weight matrix derivation mode, but also, for example, the cost of transforming and quantizing the prediction residual, the cost of various flags such as entropy coding that need to be transmitted in the code stream, the cost of the quantization coefficients, and the cost of the distortion of the reconstruction block. In general, the cost is not the space occupied, but the cost of the distortion, that is, the difference between the prediction block and the original block, or the difference value of the distortion between the original image and the image obtained by encoding and decoding. Here, the smallest cost of the selection refers to the smallest distortion, that is, the smallest loss in the compression process and the highest coding quality.

[0068] In one illustrative example, after determining the combination with the smallest loss cost, before step 700, the intra prediction method further comprises: The smallest selected loss cost is less than or equal to the cost of other prediction modes, and the other prediction modes may include other intraframe prediction modes or interframe prediction modes, etc., in which case the encoder side selects the intraframe prediction mode in the combination with the smallest loss cost selected in the present application as the prediction mode for the block to be processed, and if the smallest selected loss cost is greater than the cost of the other prediction modes, the encoder includes a step of selecting another type of prediction mode as the prediction mode for the block to be processed.

[0069] In one illustrative example, on the encoder side, the intra-frame prediction method further comprises: The coding method may include a step of writing information on the two or more different types of finally determined intra-frame prediction modes and the weighting matrix derivation mode into a code stream according to the grammar.

[0070] The intraframe prediction method may further include a step of performing intraframe prediction and subsequent encoding processing on the pending block according to the intraframe prediction method of the present application using the determined two or more different intraframe prediction modes and weighting matrices.

[0071] In step 701, a target prediction block for a pending block is obtained based on a weight matrix and the obtained two or more prediction blocks.

[0072] In one illustrative example, at the encoding side, the weight matrix can be determined by a manner of calculating the loss cost, as in step 700. At the decoding side, the codestream is parsed according to the grammar, and the weight matrix is ​​obtained based on the obtained weight matrix derivation mode.

[0073] The weight matrix determination method may be implemented by referring to the weight derivation method of GPM or AWP in inter-frame prediction. In the same encoding / decoding standard or codec, if the prediction mode of GPM or AWP is used, the weight matrix may be determined using the weight derivation method of GPM or AWP in the embodiment of the present application. In this way, some of the same logic can be reused. For example, if the AVS3 inter-frame prediction uses AWP, in AVS3, the embodiment of the present application may determine the weight matrix using the weight derivation method of AWP. The weight matrix determination method in the embodiment of the present application may be different from the method of GPM or AWP used in the same encoding / decoding standard or codec, for example, by using a different mode number, using a different transition region algorithm, or using different parameters.

[0074] In one illustrative example, as shown in FIG. 8, two different intra prediction modes are respectively adopted to perform intra prediction on the pending block to obtain a first predicted block and a second predicted block, as an example, step 701 includes: calculating a first product of a matrix corresponding to the first prediction block and a first weighting matrix, and calculating a second product of a matrix corresponding to the second prediction block and a second weighting matrix; calculating a sum of the first product, the second product, and a preset value; and performing a normalization process on the calculated sum to obtain the target predicted block.

[0075] In one illustrative example, the second weighting matrix is ​​the difference between the maximum weighting value (eg, 8) and the first weighting matrix.

[0076] The normalization process involves right-shifting the calculated sum by a preset number of bits (eg, 3 bits) to obtain a target prediction block for the pending block.

[0077] For example, the value of element predMatrixSawp[x][y] in predMatrixSawp=((predMatrix0[x][y]*AwpWeightArrayY[x][y]+predMatrix1[x][y]*(8-AwpWeightArrayY[x][y])+4)>>3), where predMatrixSawp represents a target prediction block, predMatrixSawp[x][y] represents a target prediction block matrix, predMatrix0[x][y] represents a matrix corresponding to the first prediction block, predMatrix1[x][y] represents a matrix corresponding to the second prediction block, and AwpWeightArrayY[x][y] represents a first weight matrix.

[0078] In one illustrative example, after step 701, the intra-frame prediction method further comprises: The method may include a step of adopting an improved intraframe prediction mode, performing intraframe prediction on a target prediction block of the obtained block awaiting processing, and setting the predicted result as the target prediction block of the block awaiting processing.

[0079] In one illustrative example, in all possible weight matrices determined, not all points in each weight matrix have the same weight, i.e., in all possible weight matrices, there is at least one possible weight matrix that includes at least two different weight values.

[0080] In one illustrative example, all possible weight matrices include at least two different weight values.

[0081] In one illustrative example, there is at least one weight matrix in which all possible weight matrices contain at least two different weight values, and there is at least one weight matrix in which only identical weight values ​​are included. For example, if the minimum weight value is 0, then the maximum weight value is 8, e.g., some points in a weight matrix have a weight value of 0 and some points have a weight value of 8. In a weight matrix where all points are 4 and only one weight value is included, this weight value can be any value greater than the minimum weight value and less than the maximum weight value.

[0082] In one illustrative example, for one or several weight matrices including at least two different weight values, each point in the pending block is obtained by weighting the predicted values ​​derived by the two intra-frame prediction modes according to the minimum weight value and the maximum weight value. For example, the weight has eight values ​​from 0 to 8. Taking the example of performing intra-frame prediction on the pending block by adopting two different intra-frame prediction modes, 0 represents that this point is completely obtained from the predicted value derived by one intra-frame prediction mode, and 8 represents that this point is completely obtained from the predicted value derived by another intra-frame prediction mode. If the minimum weight value is 1 and the maximum weight value is 7, all points in this type of weight matrix need to be obtained by weighting the predicted values ​​derived by the two intra-frame prediction modes. However, the weights of all points are not the same.

[0083] In one illustrative example, taking two different intra prediction modes for performing intra prediction on a pending block as an example, there can be only one weight matrix that contains only two weight values ​​among all possible weight matrices, one weight value representing that the predicted value of the corresponding point is entirely from the value of the corresponding point of the first predicted block, and the other weight value representing that the predicted value of the corresponding point is entirely from the value of the corresponding point of the second predicted block, for example, the two weight values ​​are 0 and 1, respectively.

[0084] In one illustrative example, taking two different intra prediction modes for performing intra prediction on a block to be decoded as an example, in all possible weight matrices, one weight matrix can include multiple types of weight values, and each of the maximum and minimum values ​​(e.g., 0) of the weight values ​​represents that the predicted value of the corresponding point is entirely from the value of the corresponding point of the first prediction block and the value of the corresponding point of the second prediction block, and the weight values ​​that are not the maximum and minimum represent that the predicted value of the corresponding point is entirely from the value of the corresponding point of the first prediction block and the value of the corresponding point of the second prediction block. Here, the area formed by the weight values ​​other than the maximum and minimum values ​​can be called a transition area (blending area).

[0085] In one illustrative example, as shown in Fig. 9(a), in all possible weight matrices, if the weight matrix contains only two kinds of weight values, the positions of the weight value changes are represented as one straight line, and if the weight matrix contains multiple kinds of weight values, the positions of the same weight values ​​in the transition region are represented as one straight line. In one embodiment, all of the above straight lines are horizontal and vertical, or all of the above straight lines are not horizontal and vertical.

[0086] In one illustrative example, as shown in FIG. 9(b), in all possible weight matrices, if the weight matrix contains only two types of weight values, the positions of weight value change are represented as one curve, and if the weight matrix contains multiple types of weight values, the positions where the weight values ​​in the transition region are the same are represented as one curve.

[0087] The diversified weight matrix provided in the embodiments of the present application provides the guarantee that more diverse prediction blocks can be predicted, and further, the intra-frame prediction method provided in the embodiments of the present application can be applied to more scenes.

[0088] In one embodiment, AWP mode is used for inter-frame prediction in AVS3 as an example, and the weight matrix of AWP includes 56 kinds. In the embodiment of the present application, 64 kinds of weight matrices are used in intra-frame prediction, of which 56 kinds are the same as the weight matrix of AWP. For example, the first 56 kinds are the same as the weight matrix of AWP, and each weight matrix of the remaining 8 kinds of weight matrices includes only one weight value, and the weight values ​​are 1, 2, ..., 7, 8. The total weight value of these 8 kinds of weight matrices is 16, that is, a weight value of 1 represents a weight of 1:15, and a weight value of 2 represents a weight of 2:14. In this way, when binarizing the mode numbers of 64 kinds of weight matrices, a 6-bit codeword can be used for each weight matrix. Optionally, the total weight value is 8, in this case, 8 is the maximum weight value, that is, a weight value of 1 represents a weight of 1:7, and a weight value of 2 represents a weight of 2:6.

[0089] Inter-frame prediction utilizes the correlation in the time domain, so it uses the reconstructed image in the reference frame as the reference block. Meanwhile, intra-frame prediction utilizes the correlation in the space domain, so it uses the reconstructed pixels around the block to be processed as the reference pixels. The closer the distance in the space domain, the stronger the correlation, and the farther the distance, the worse the correlation. Therefore, if a weight matrix makes all pixel positions of the prediction block farther from the reference pixels, it is not necessary to use such a weight matrix in the embodiment of the present application in order to guarantee the effect of intra-frame prediction.

[0090] In one illustrative example, in the intra-frame prediction method provided in the embodiments of the present application, the size of a block (e.g., a pending block) may include, but is not limited to, the following cases:

[0091] The width of the block is greater than or equal to a first threshold TH1, and the height of the block is greater than or equal to a second threshold TH2, and the values ​​of the first and second thresholds TH1 and TH2 may be 8, 16, 32, 64, 128, etc., and the first threshold TH1 may be equal to the second threshold TH2, e.g., first threshold TH1=second threshold TH2=8. Or, the number of pixels of the block is greater than or equal to a third threshold TH3, and the value of the third threshold TH3 may be 8, 16, 32, 64, 128, etc.

[0092] Alternatively, the width of the block is less than or equal to a fourth threshold TH4 and the height of the block is less than or equal to a fifth threshold TH5, and the values ​​of the fourth and fifth thresholds TH4 and TH5 may be 8, 16, 32, 64, 128, etc., and the fourth threshold TH4 may be equal to the fifth threshold TH5. Alternatively, the number of pixels of the block is less than or equal to a sixth threshold TH6, and the value of the sixth threshold TH6 may be 8, 16, 32, 64, 128, etc.

[0093] By restricting the size of the block, the impact of the complexity of prediction using multiple prediction modes on the overall system is reduced, and by restricting the use of some sizes of blocks, the compression performance and the complexity are well balanced, thereby better ensuring the applicability of the present application.

[0094] In one illustrative example, with the development of technology, the division of blocks becomes more and more flexible. Besides square blocks, the division method can support blocks with aspect ratios of 1:2, 1:4, 1:8, 2:1, 4:1, 8:1, etc. In the intraframe prediction method provided in the embodiment of the present application, according to the research of the inventors of the present application, the compression performance of some aspect ratio blocks, i.e., some size aspect ratio blocks, such as 1:4 or 4:1 blocks and 1:8 or 8:1 blocks, or 8×32, 8×64, 32×8, 64×8, etc., has not been good or significant so far. In this case, in one embodiment of the present application, the size of the block can be set by setting the aspect ratio of the block. For example, the proportion of width to height is less than or equal to a preset proportion threshold THR, and the proportion of width to height is less than or equal to the proportion threshold THR.

[0095] In one illustrative example, the size of the block and the aspect ratio of the block may be set simultaneously. For example, for the size of the block, if the height of the block is 8 or more, the width of the block is 8 or more, the ratio of the width of the block to the height of the block is 4 or less, and the ratio of the height of the block to the width of the block is 4 or less, the intra-frame prediction method provided in the embodiment of the present application can be used; otherwise, the intra-frame prediction method provided in the embodiment of the present application is not used by default.

[0096] In one illustrative example, on the encoder side, before step 700 of the present application, the intra-frame prediction method further comprises: The method may include a step of setting a frame-level flag to indicate whether the currently pending frame uses the intra-frame prediction method of the embodiment of the present application, i.e., whether to execute step 700, and writing the flag into the codestream according to a grammar, thereby executing the intra-frame prediction method at the decoder side based on the flag.

[0097] In one embodiment, when the intra-frame prediction method of the embodiment of the present application is used for intra-frames (e.g., I-frames) and the intra-frame prediction method of the embodiment of the present application is not used for inter-frames (e.g., B-frames, P-frames), if the flag indicates that the currently pending frame is an intra-frame, it indicates that the decoding side continues to perform step 700. If the flag indicates that the currently pending frame is an inter-frame, it indicates that the decoding side can exit from the process of the present application and adopt related technology to perform intra-frame prediction.

[0098] Furthermore, for example, when the intraframe prediction method of the embodiment of the present application is not used for intraframes (e.g., I frames) but is used for interframes (e.g., B frames, P frames), if the flag indicates that the currently pending frame is an intraframe, it indicates that the decoding side can exit from the process of the present application and adopt related technology to perform intraframe prediction. If the flag indicates that the currently pending frame is an interframe, it indicates that the decoding side continues to execute step 700.

[0099] Furthermore, for example, when the present disclosure is used for some inter frames and not used for other inter frames, if the flag indicates that the currently pending frames are some inter frames, it indicates that the decoding side continues to execute step 600. If the flag indicates that the currently pending frames are some other inter frames, it indicates that the decoding side can exit the process of the present application and adopt related techniques to perform intra-frame prediction.

[0100] Correspondingly, on the decoder side, before step 700 of the intra-frame prediction method shown in FIG. 7 above, the intra-frame prediction method may further include a step of parsing the codestream according to the grammar and obtaining a flag.

[0101] For example, when the intraframe prediction method of the embodiment of the present application is used for intraframes (e.g., I frames) and not for interframes (e.g., B frames, P frames), if the flag obtained by decoding indicates that the currently waiting frame is an intraframe, continue to execute step 700. If the flag obtained by decoding indicates that the currently waiting frame is an interframe, exit from the process of the present application and adopt related technology to perform intraframe prediction.

[0102] Furthermore, for example, when the intraframe prediction method of the embodiment of the present application is not used for intraframes (e.g., I frames) but is used for interframes (e.g., B frames, P frames), if the flag obtained by decoding indicates that the currently waiting frame is an intraframe, the process of the present application can be terminated and intraframe prediction can be performed by adopting related techniques. If the flag obtained by decoding indicates that the currently waiting frame is an interframe, step 700 is continued.

[0103] Furthermore, for example, when the present disclosure is used for some inter frames and not used for other inter frames, if the decoded flag indicates that the currently pending frames are some inter frames, continue to execute step 700. If the decoded flag indicates that the currently pending frames are some other inter frames, the process of the present application can be exited and related techniques can be adopted to perform intra-frame prediction.

[0104] In one illustrative example, on the encoder side, before step 700 of the present application, the intra-frame prediction method further comprises: In order to indicate whether the intraframe prediction method of the embodiment of the present application is to be used for the indicated region, a flag below the frame level and above the CU level (e.g., tile, slice, patch, LCU, etc.) is set, and the flag is written into the codestream according to a grammar, thereby instructing the decoder side to execute the corresponding intraframe prediction method based on the flag.

[0105] The setting of flags at different levels in the embodiment of the present application, on the one hand, flexibly realizes setting the optimal arrangement according to different application scenarios and video contents, and on the other hand, achieves the effect of further saving code rate.For example, if one LCU has multiple CUs, and none of these CUs use the intra-frame prediction method provided in the embodiment of the present application, a flag at the LCU level can be set to indicate that all CUs in the LCU do not use the intra-frame prediction method provided in the embodiment of the present application.That is, it is not necessary to set a flag for each CU separately.That is, the setting of one flag replaces the setting of N flags.

[0106] Correspondingly, on the decoder side, before step 700, the intra-frame prediction method may further include parsing the codestream according to the grammar to obtain the flag.

[0107] In one illustrative example, on the encoder side, before step 700 of the present application, the intra-frame prediction method further comprises: An improved prediction mode is set that is mutually exclusive with the intra prediction method provided in the embodiment of the present application, so as to better determine the intra prediction mode in the intra prediction process.

[0108] At the decoder side, the improved prediction mode mutually exclusive with the set intraframe prediction method provided in the embodiment of the present application is obtained by analyzing the code stream. If the pending block decides to use the intraframe prediction method of the embodiment of the present application, the mutually exclusive improved prediction mode is not used. Or, if the pending block decides to use the mutually exclusive improved prediction mode, the intraframe prediction method of the embodiment of the present application is not used. By setting the mutually exclusive improved prediction mode, it is not necessary to transmit a flag indicating whether the mutually exclusive improved prediction mode is used in the code stream, which saves the transmission of extra flags in the code stream and achieves better compression performance overall.

[0109] In one illustrative example, the mutually exclusive refinement prediction modes may include, for example, IPF, DT, etc.

[0110] For example, the mutually exclusive improved prediction mode is Derived Tree (DT), which is a technology in AVS3, and DT can divide the current CU into rectangular PUs and correspondingly divide it into smaller TUs. In the case of combining with DT, which is adopted in the embodiment of the present application, the intra prediction method provided in the embodiment of the present application can be used for one or several PUs divided by DT, but the complexity increases. For example, the mutually exclusive improved prediction mode is Improved Intra Prediction (IIP), which is a technology in AVS3, and IIP can obtain a predicted value using a more complex filter.

[0111] According to the verification by the inventor of the present application, in the verification process of the intra-frame prediction provided in the embodiment of the present application, when IIP, DT or IPF is used, the amount of calculation or complexity of intra-frame prediction is increased. Therefore, in the present embodiment, by setting a mutually exclusive relationship between these improved prediction modes and the intra-frame prediction of the present application, a good balance between performance and complexity is achieved, thereby better ensuring the applicability of the present application.

[0112] Taking the intraframe prediction method of the present application and IPF as examples, the processing for mutually exclusive cases will be described.

[0113] In one illustrative example, assume that the intra-frame prediction method of the present application is mutually exclusive with IPF. Take the example of first decoding the flag representing whether the currently pending frame in the embodiment of the present application uses the intra-frame prediction method of the embodiment of the present application, and then decoding the flag of IPF, as shown in FIG. 10, the process generally includes the following steps:

[0114] When the intra-frame prediction method of the present application is used for the current block, there is no need to decode the IPF flag. That is, there is no need to transmit the IPF flag in the code stream. When the intra-frame prediction method of the present application is not used for the current block, the IPF flag is decoded to determine whether or not it is necessary to use the IPF. When the IPF is used for the current block, the current block is predicted in combination with the IPF using another intra-frame prediction method. When the IPF is not used for the current block, the current block uses another intra-frame prediction method.

[0115] In one illustrative example, assume that the intra-frame prediction method of the present application is not mutually exclusive with IPF. Take the example of first decoding the flag representing whether the currently pending frame in the embodiment of the present application uses the intra-frame prediction method of the embodiment of the present application, and then decoding the flag of IPF, as shown in FIG. 11, the process generally includes the following steps:

[0116] Regardless of whether the intra-frame prediction method of the present application is used, the IPF flag must be decoded. Moreover, when both the intra-frame prediction method of the present application and the IPF are used, prediction is performed for the current block using the intra-frame prediction method of the present application in combination with the IPF.

[0117] It should be explained that in the embodiment shown in Figure 10 and Figure 11, the intraframe prediction method of the present application is taken as an example whether it is mutually exclusive with one kind of technology. If there is a mutually exclusive relationship between the intraframe prediction method of the present application and this kind of technology and another technology, the process will be more complicated, but the principle is the same. Based on the embodiment shown in Figure 10 and Figure 11 of the present application, those skilled in the art can easily understand it, and will not be described again here.

[0118] In one illustrative example, the embodiment of the present application may further include storing the intra prediction mode information used for intra prediction for use in encoding / decoding neighboring blocks. For example, in MPM mode, it is necessary to refer to the intra prediction mode of neighboring blocks. That is, the encoding / decoding block behind the current frame may use the intra prediction mode of a previously encoded / decoded block, e.g., the neighboring block, based on the neighboring positional relationship. For a chrominance block (encoding unit), it may use the intra prediction mode of a previously encoded / decoded luma block (encoding unit) based on the position. The stored information is used as a reference for the block to be subsequently encoded / decoded. The reason is that the encoding mode information in the same block (encoding unit) can be obtained directly, but the encoding mode information in a different block (encoding unit) cannot be obtained directly. Therefore, it is necessary to store the intra prediction mode information used for intra prediction. In this way, the block to be subsequently encoded / decoded may read these information based on the position.

[0119] In one illustrative example, taking an example of performing intra-frame prediction on a block to be decoded by adopting two different intra-frame prediction modes, the step of storing the intra-frame prediction mode used for the intra-frame prediction includes:

[0120] There is at least one minimum unit that stores one of two different intra-frame prediction modes, and there is at least one minimum unit that stores another of two different intra-frame prediction modes. That is, there are at least two minimum units that store different intra-frame prediction modes. The minimum unit may be a preset fixed-size matrix (e.g., a 4×4 matrix, etc.). Each minimum unit stores one intra-frame prediction mode by itself. In this way, every time a block is coded or decoded, the minimum unit corresponding to the position of the block stores the intra-frame prediction mode of the block.

[0121] For example, as shown in Fig. 12, assuming that intra prediction mode 5 is used for luma prediction of a 16x16 current block, the intra prediction modes of all 4x4 minimum units corresponding to the block are stored as 5. Taking the YUV format as an example, the luma intra prediction mode is generally stored, which may include the luma intra prediction mode of a block containing both luma and chroma components, and may also include the luma intra prediction mode of a block containing only luma components.

[0122] Furthermore, for example, in AVS3, the embodiment of the present application can store two different intraframe prediction modes using logic similar to storing two different motion information in AWP. That is, when only a prediction block determined by one of two intraframe prediction modes is used at a position corresponding to one minimum unit, the intraframe prediction mode of the type is stored in the minimum unit. When only a prediction block determined by another of two intraframe prediction modes is used at a position corresponding to one minimum unit, the other intraframe prediction mode is stored in the minimum unit. When not only a prediction block determined by a first intraframe prediction mode but also a prediction block determined by a second intraframe prediction mode is used at a position corresponding to one minimum unit, one of them can be selected and stored based on a preset judgment method. For example, when the minimum unit is 4×4, a certain point, for example, a point (2, 2), is selected. If the weight of the first type of intraframe prediction mode at this point is equal to or greater than the weight of the second type of intraframe prediction mode, store the first type of intraframe prediction mode; otherwise, store the second type of intraframe prediction mode.Furthermore, for example, the weight of the first type of intraframe prediction mode and the weight of the first type of intraframe prediction mode of all points in the block of the minimum unit are summed, and if the weight sum of the first type of intraframe prediction mode is equal to or greater than the weight sum of the second type of intraframe prediction mode, store the first type of intraframe prediction mode; otherwise, store the second type of intraframe prediction mode.In the embodiment of the present application, a method of storing related information in GPM or AWP is used.In this way, part of the same logic can be reused.

[0123] In one illustrative example, taking two different intra prediction modes as an example to perform intra prediction on the pending block, the step of storing the intra prediction mode used for the intra prediction includes: The method includes a step of selecting and storing the same intra-frame prediction mode for all the minimum units corresponding to the entire block to be processed, thereby reducing the complexity.

[0124] In one illustrative example, the step of selecting and storing the same intra-frame prediction mode for all the minimum units corresponding to the entire block to be processed includes: The method may include a step of determining to store one of the two types of intra-frame prediction modes in all the minimum units of the block to be processed based on the weight matrix derivation mode obtained by analyzing the code stream, or to store another type of intra-frame prediction mode in all the minimum units of the block to be processed. For example, a first type of intra-frame prediction mode is selected for all the weight matrix derivation modes. Furthermore, for example, a second type of intra-frame prediction mode is selected for all the weight matrix derivation modes. Furthermore, for example, a first type of intra-frame prediction mode is selected for all the minimum units of some weight matrix derivation modes, and a second type of intra-frame prediction mode is selected for all the minimum units of other some weight matrix derivation modes.

[0125] Here, the weight matrix derivation mode is a mode for deriving a weight matrix. For one block having a certain width and height, each type of weight matrix derivation mode can derive one weight matrix, and different weight matrix derivation modes derive different weight matrices for blocks of the same size. For example, the AWP of AVS3 has 56 types of weight matrix derivation modes, and the GPM of VVC has 64 types of weight matrix derivation modes.

[0126] In one illustrative example, the step of selecting and storing the same intra-frame prediction mode for all the minimum units corresponding to the entire block to be processed includes: The method may include a step of determining whether to store one of the two intra-frame prediction modes or store another intra-frame prediction mode of the two intra-frame prediction modes in all the minimum units of the block to be processed based on the mode number of the weight matrix derivation mode obtained by analyzing the code stream. In one embodiment, it is possible to determine whether to store the first intra-frame prediction mode or the second intra-frame prediction mode in all the minimum units of the block to be processed by performing a table lookup based on the mode number of the weight matrix derivation mode. Here, taking the derivation mode of the weight matrix same as the AWP as an example in the embodiment of the present application, as shown in Table 1, it is possible to select to store the first intra-frame prediction mode in all the minimum units corresponding to the matrix derivation mode whose mode number in Table 1 is 0, and it is possible to select the second intra-frame prediction mode in all the minimum units corresponding to the matrix derivation mode whose mode number in Table 1 is 1.

[0127] [Table 1]

[0128] One embodiment provides an encoding method, the encoding method comprising: obtaining a target image frame, and dividing the target image frame into blocks to obtain blocks to be processed; performing intra-frame prediction on the waiting block by adopting two or more different intra-frame prediction modes, respectively, to obtain two or more prediction blocks corresponding to the different intra-frame prediction modes; and obtaining a target prediction block of the waiting block based on a weight matrix and the obtained two or more prediction blocks; and performing encoding based on the pending block and the target predicted block to generate a codestream.

[0129] Another embodiment provides a decoding method, the decoding method comprising: parsing the codestream to obtain blocks to be processed and a weight matrix; performing intra-frame prediction on a pending block by adopting two or more different intra-frame prediction modes, respectively, to obtain two or more types of predicted blocks corresponding to the different intra-frame prediction modes; obtaining a target prediction block of the pending block based on the weight matrix and the obtained two or more prediction blocks; performing decoding based on the target predicted block and the pending block to obtain a reconstructed block corresponding to the pending block.

[0130] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, the computer-executable instructions being for implementing an intra-frame prediction method or a decoding method applied on a decoder side according to any one of the above claims.

[0131] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, the computer-executable instructions being for implementing an intra-frame prediction method or encoding method applied on an encoder side according to any one of the above claims.

[0132] An embodiment of the present application provides a decoder including a memory and a processor, the memory storing processor-executable instructions for implementing steps of an intra-frame prediction method or a decoding method applied on a decoder side according to any one of the above claims.

[0133] An embodiment of the present application provides an encoder including a memory and a processor, the memory storing processor-executable instructions for implementing steps of an intra-frame prediction method or encoding method applied on the encoder side according to any one of the above claims.

[0134] Figure 13 is a structural schematic diagram of an intra-frame prediction device in the present application. As shown in Figure 13, the intra-frame prediction device includes at least a prediction module and a combination module.

[0135] The prediction module is configured to perform intra prediction on the pending block by employing two or more different intra prediction modes, respectively, to obtain two or more prediction blocks corresponding to the different intra prediction modes.

[0136] The combination module is configured to derive a target prediction block for the pending block based on the weight matrix and the obtained two or more prediction blocks.

[0137] The intra-frame prediction device provided in the embodiment of the present application may be installed in an encoder or in a decoder.

[0138] The intra-frame prediction device provided in the embodiment of the present application is installed in a decoder, and further includes a decoding module.

[0139] The decoding module is configured to perform decoding on the received codestream to obtain two or more different intra-frame prediction modes, pending blocks, and weight matrices.

[0140] The intra-frame prediction device provided in the embodiment of the present application is installed in an encoder, and further includes a processing module.

[0141] The processing module is configured to perform trials for all or some of the possible combinations of prediction modes and weight matrix derivation modes, calculate a loss cost, select a combination with a smaller loss cost, determine the two or more different intra-frame prediction modes and weight matrices in the combination as two or more different intra-frame prediction modes and weight matrices used for intra-frame prediction, and write information of the determined two or more different intra-frame prediction modes and weight matrix derivation modes into the codestream according to the grammar.

[0142] In one illustrative example, the two or more different intra prediction modes include at least one base intra prediction mode.

[0143] In one illustrative example, two different intra prediction modes are adopted to perform intra prediction on the pending block, and the two different intra prediction modes are both basic intra prediction modes.

[0144] In one illustrative example, the employed base intra prediction mode may further be combined with an improved intra prediction mode to perform predictions for pending blocks.

[0145] In one illustrative example, two different intra-frame prediction modes are adopted to perform intra-frame prediction on a pending block, where the two different intra-frame prediction modes include one basic intra-frame prediction mode and one improved intra-frame prediction mode.

[0146] In the embodiment of the present application, at least two different intra-frame prediction modes are adopted for prediction of pending blocks, in this way prediction can be performed for pending blocks from multiple viewpoints, which is suitable for processing prediction of complex textures, and can improve the quality of intra-frame prediction.

[0147] In one illustrative example, not all points in each weight matrix have the same weight among all possible weight matrices, i.e., there is at least one possible weight matrix that includes at least two distinct weight values ​​among all possible weight matrices.

[0148] In one illustrative example, all possible weight matrices include at least two different weight values.

[0149] In one illustrative example, in all possible weight matrices, there is at least one weight matrix that contains at least two distinct weight values, and there is at least one weight matrix that contains only identical weight values.

[0150] In one illustrative example, taking intra-frame prediction for a pending block by employing two different intra-frame prediction modes, among all possible weight matrices, there is only one weight matrix that contains only two types of weight values, one of which represents that the predicted value of the corresponding point is entirely derived from the value of the corresponding point in the first prediction block, and the other weight value represents that the predicted value of the corresponding point is entirely derived from the value of the corresponding point in the second prediction block.

[0151] In one illustrative example, taking two different intra prediction modes for performing intra prediction on a block to be decoded as an example, in all possible weight matrices, one weight matrix may include multiple types of weight values, and the maximum and minimum values ​​(e.g., 0) of the weight values ​​each represent that the predicted value of the corresponding point is entirely from the value of the corresponding point of the first prediction block and the value of the corresponding point of the second prediction block, and the weight values ​​other than the maximum and minimum represent that the predicted value of the corresponding point is entirely from the value of the corresponding point of the first prediction block and the value of the corresponding point of the second prediction block. Here, the region formed by the weight values ​​other than the maximum and minimum values ​​may be called a transition region.

[0152] In one illustrative example, in all possible weight matrices, if the weight matrix contains only two kinds of weight values, the positions of the weight value changes are represented as one straight line, and if the weight matrix contains multiple kinds of weight values, the positions of the weight values ​​in the transition region that are the same are represented as one straight line. In one embodiment, the straight lines are all horizontal and vertical, or the straight lines are not all horizontal and vertical.

[0153] In one illustrative example, for all possible weight matrices, if the weight matrix contains only two types of weight values, the positions of weight value change are represented as one curve, and if the weight matrix contains multiple types of weight values, the positions where the weight values ​​in the transition region are the same are represented as one curve.

[0154] The diversified weight matrix provided in the embodiment of the present application provides the guarantee for predicting more diverse prediction blocks, and further, the intra-frame prediction method provided in the embodiment of the present application can be applied to more scenes.

[0155] In one illustrative example, in the intra-frame prediction device provided in the embodiment of the present application, the size of the block may include, but is not limited to, the following cases:

[0156] The width of the block is greater than or equal to a first threshold TH1, and the height of the block is greater than or equal to a second threshold TH2, and the values ​​of the first and second thresholds TH1 and TH2 may be 8, 16, 32, 64, 128, etc., and the first threshold TH1 may be equal to the second threshold TH2, e.g., first threshold TH1=second threshold TH2=8. Or, the number of pixels of the block is greater than or equal to a third threshold TH3, and the value of the third threshold TH3 may be 8, 16, 32, 64, 128, etc.

[0157] Alternatively, the width of the block is less than or equal to a fourth threshold TH4 and the height of the block is less than or equal to a fifth threshold TH5, and the values ​​of the fourth and fifth thresholds TH4 and TH5 may be 8, 16, 32, 64, 128, etc., and the fourth threshold TH4 may be equal to the fifth threshold TH5. Alternatively, the number of pixels of the block is less than or equal to a sixth threshold TH6, and the value of the sixth threshold TH6 may be 8, 16, 32, 64, 128, etc.

[0158] The restriction on the block size reduces the impact of the complexity of prediction using multiple prediction modes on the overall system, and the restriction on not applying some block sizes achieves a good balance between compression performance and complexity, thereby better ensuring the applicability of the present application.

[0159] In one illustrative example, with the development of technology, the division of blocks becomes more and more flexible. Besides square blocks, the division method can support blocks with aspect ratios of 1:2, 1:4, 1:8, 2:1, 4:1, 8:1, etc. In the intraframe prediction method provided in the embodiment of the present application, according to the research of the inventors of the present application, the compression performance of some aspect ratio blocks, i.e., some size aspect ratio blocks, such as 1:4 or 4:1 blocks and 1:8 or 8:1 blocks, or 8×32, 8×64, 32×8, 64×8, etc., has not been good or significant so far. In this case, in one embodiment of the present application, the size of the block can be set by setting the aspect ratio of the block. For example, the proportion of width to height is less than or equal to a preset proportion threshold THR, and the proportion of width to height is less than or equal to the proportion threshold THR.

[0160] In one illustrative example, the size of the block and the aspect ratio of the block may be set simultaneously. For example, for the size of the block, if the height of the block is 8 or more, the width of the block is 8 or more, the ratio of the width of the block to the height of the block is 4 or less, and the ratio of the height of the block to the width of the block is 4 or less, the intra-frame prediction method provided in the embodiment of the present application can be used; otherwise, the intra-frame prediction method provided in the embodiment of the present application is not used by default.

[0161] In one illustrative example, taking two different intra-frame prediction modes as an example to perform intra-frame prediction on a block to be decoded, the combination module specifically comprises: Calculating a first product of a matrix corresponding to the first prediction block and a first weighting matrix, and calculating a second product of a matrix corresponding to the second prediction block and a second weighting matrix; Calculating the sum of the first product, the second product, and a preset value; The calculated sum is then normalized to obtain the target prediction block.

[0162] In one illustrative example, the intra-frame prediction device provided in the embodiment of the present application is installed in an encoder, and the processing module further comprises: A frame-level flag is configured to be set to indicate whether the currently pending frame uses the intra-frame prediction method of the present embodiment.

[0163] Correspondingly, the decoding module in the decoder is further configured to determine, based on the flag, whether to continue performing intra-frame prediction on the pending block by employing each of two or more different intra-frame prediction modes obtained by decoding.

[0164] In one illustrative example, the intra-frame prediction device provided in the embodiment of the present application is installed in an encoder, and the processing module further comprises: It is configured to set a flag below the frame level and above the CU level (e.g., tile, slice, patch, LCU, etc.) to indicate whether or not to use the intra-frame prediction method of the embodiment of the present application for the indicated region.

[0165] Correspondingly, the decoding module in the decoder is further configured to determine, based on the flag, whether to continue performing intra-frame prediction on the pending block by employing each of two or more different intra-frame prediction modes obtained by decoding.

[0166] The setting of different levels of flags in the embodiment of the present application, on the one hand, flexibly realizes setting the optimal configuration according to different application scenarios and video contents, and on the other hand, achieves the effect of further saving the code rate.

[0167] In one illustrative example, the intra-frame prediction device provided in the embodiment of the present application is installed in an encoder, and the processing module further comprises: It is configured to set an improved prediction mode that is mutually exclusive with the intra-frame prediction method provided in the embodiment of the present application.

[0168] Correspondingly, the decoding module in the decoder is further configured to analyze the code stream, obtain an improved prediction mode mutually exclusive with the set intraframe prediction method provided in the embodiment of the present application, and not use the mutually exclusive improved prediction mode if the pending block determines to use the intraframe prediction method of the embodiment of the present application, and not use the intraframe prediction method of the embodiment of the present application if the pending block determines to use the mutually exclusive improved prediction mode. By setting to the mutually exclusive improved prediction mode, it is not necessary to transmit a flag of whether to use the mutually exclusive improved prediction mode in the code stream, which saves the transmission of extra flags in the code stream and obtains better compression performance overall.

[0169] By setting a mutually exclusive relationship between these improved prediction modes and the intra-frame prediction of the present application in this embodiment, a good balance between performance and complexity is achieved, thereby better ensuring the applicability of the present application.

[0170] In one illustrative example, the combination module further comprises: The intra-frame prediction mode information used in the intra-frame prediction is stored for use in encoding / decoding adjacent blocks.

[0171] In one illustrative example, taking an example of performing intra-frame prediction on a pending block using two different intra-frame prediction modes, the step of storing the intra-frame prediction mode used for the intra-frame prediction includes:

[0172] There is at least one minimum unit that stores one of the two different intra-frame prediction modes, and there is at least one minimum unit that stores another of the two different intra-frame prediction modes. That is, there are at least two minimum units that store different intra-frame prediction modes.

[0173] In one illustrative example, taking two different intra prediction modes as an example to perform intra prediction on the pending block, the step of storing the intra prediction mode used for the intra prediction includes: The method includes a step of selecting and storing the same intra-frame prediction mode for all minimum units corresponding to the entire block to be processed.

[0174] The decoder provided in the embodiment of the present application is applied to perform intra prediction on the pending block by adopting two or more different intra prediction modes, respectively, to obtain two or more predicted blocks. The obtained two or more predicted blocks are combined according to a weight matrix, thereby obtaining a predicted block for the pending block. The embodiment of the present application determines multiple predicted blocks according to multiple intra prediction modes, realizes prediction processing of complex texture, improves the quality of intra prediction, and thereby improves compression performance.

[0175] Furthermore, by applying the decoder provided in the embodiment of the present application, the diversified weight matrix provides a guarantee for the prediction process of more complex textures, improving the quality of intra-frame prediction, and thereby improving the compression performance. Furthermore, the intra-frame prediction method provided in the embodiment of the present application can be applied to more scenes.

[0176] An embodiment of the present application further provides a decoder including a decoding module, a prediction module, and a combining module.

[0177] The decoding module is configured to perform decoding on the received codestream to obtain two or more different intra-frame prediction modes, pending blocks, and weight matrices.

[0178] The prediction module is configured to perform intra prediction on the pending block by employing two or more different intra prediction modes, respectively, to obtain two or more prediction blocks corresponding to the different intra prediction modes.

[0179] The combination module is configured to derive a target prediction block for the pending block based on the weight matrix and the obtained two or more prediction blocks.

[0180] In one illustrative example, the decryption module further comprises: The method is configured to determine whether to continue performing intra-frame prediction on a pending block by employing each of two or more different intra-frame prediction modes obtained by decoding based on a frame-level flag.

[0181] In one illustrative example, the decoding module is further configured to determine whether to continue performing intra-frame prediction on the pending block using each of two or more different intra-frame prediction modes obtained by decoding based on a flag below the frame level and above the CU level.

[0182] An embodiment of the present application further provides an encoder including a prediction module, a combination module, and a processing module.

[0183] The prediction module is configured to perform intra prediction on the pending block by employing two or more different intra prediction modes, respectively, to obtain two or more prediction blocks corresponding to the different intra prediction modes.

[0184] The combination module is configured to derive a target prediction block for the pending block based on the weight matrix and the obtained two or more prediction blocks.

[0185] The processing module is configured to perform trials for all or some of the possible combinations of prediction modes and weight matrix derivation modes, calculate a loss cost, select a combination with a smaller loss cost, determine the two or more different intra-frame prediction modes and weight matrices in the combination as two or more different intra-frame prediction modes and weight matrices used for intra-frame prediction, and write information of the determined two or more different intra-frame prediction modes and weight matrix derivation modes into the codestream according to the grammar.

[0186] In one illustrative example, the processing module is further configured to set a flag: The flag is a frame level flag for indicating whether the decoder continues to obtain two or more types of prediction blocks corresponding to the different intra-frame prediction modes; And / or the flag is a flag below the frame level and above the coding unit (CU) level to indicate whether the decoder continues to obtain two or more types of prediction blocks corresponding to the different intra-frame prediction modes for the indicated region.

[0187] In one illustrative example, the processing module further comprises: setting a prediction mode mutually exclusive with the intraframe prediction; The method is configured such that, when the intraframe prediction is used for the pending blocks, mutually exclusive prediction modes are not used, or, when the mutually exclusive prediction modes are used for the pending blocks, the intraframe prediction is not used.

[0188] In one illustrative example, the combining module is further configured to store intra-frame prediction mode information used for the intra-frame prediction.

[0189] 14 is a flowchart of another example of an intra-frame prediction method in an embodiment of the present application. As shown in FIG. 14, the intra-frame prediction method includes the following steps:

[0190] In step 1400, intra prediction is performed on the pending block using each of two or more different intra prediction modes.

[0191] In step 1401, after prediction of a preset number of pixel points is completed by prediction of each type of intra-frame prediction mode, a preset number of predicted pixel points of the block to be processed is obtained based on the weight matrix and the pixel points corresponding to each intra-frame prediction mode whose prediction has been completed.

[0192] In step 1402, a target predicted block of the pending block is obtained based on the obtained multiple preset quantities of predicted pixel points.

[0193] The intra prediction method provided in this embodiment is different from the embodiment shown in FIG. 7 in that the processing object in the embodiment shown in FIG. 7 is a block, and the processing object in the embodiment shown in FIG. 14 is a pixel point. Similarly, in the embodiment shown in FIG. 14, intra prediction is performed on a pending block by adopting two or more different intra prediction modes, and after the prediction of a preset number of pixels is completed, a combination is performed on the pixel points corresponding to each intra prediction mode of the completed prediction based on a weight matrix to obtain a preset number of predicted pixel points of the pending block, and finally, the obtained multiple preset numbers of predicted pixel points are combined to obtain a predicted block of the pending block. The embodiment of the present application determines multiple predicted blocks by multiple types of intra prediction modes, realizes prediction processing of complex texture, improves the quality of intra prediction, and thereby improves compression performance.

[0194] Furthermore, the specific implementation of the weight matrix in the embodiment shown in Figure 14 is the same as the embodiment shown in Figure 7. The diversified weight matrix provides a guarantee for the prediction process of more complex textures, improves the quality of intra-frame prediction, and thereby improves compression performance. Furthermore, the intra-frame prediction method provided in the embodiment of the present application can be applied to more scenes.

[0195] In the following, a decoding embodiment will be described by taking the intraframe prediction method provided in the embodiment of the present application to AVS3 as an example. Since AWP technology is used in AVS3, in this embodiment, the intraframe prediction of the present application is called spatial angular weighted prediction (SAWP). In this embodiment, some terms in the AVS3 standard are used, for example, the prediction sample matrix in this embodiment refers to the above prediction block, that is, the "block" can be understood as a "sample matrix". Furthermore, for example, the array in this embodiment is a matrix. In this embodiment, SAWP is taken as an example to act on a luma component, but the embodiment of the present application can be used not only for the luma component but also for chroma components and any other components of any format.

[0196] In one method of this embodiment, the encoder sets a sequence-level flag to determine whether the current sequence to be decoded at the decoder uses SAWP or not. The definition of the sequence header is shown in Table 2.

[0197] [Table 2]

[0198] In Table 2, sawp_enable_flag is a spatial domain angle weighted prediction enable flag and is a binary variable. For example, sawp_enable_flag indicates that the spatial domain angle weighted prediction can be used when the value is 1, and indicates that the spatial domain angle weighted prediction cannot be used when the value is 0.

[0199] In another method in this embodiment, the encoder sets a frame-level flag, which determines whether the decoder uses SAWP for a frame currently waiting to be decoded. For example, the encoder can set the flag to use SAWP for intraframes (e.g., I frames) and not to use SAWP for interframes (e.g., B frames, P frames). For example, the encoder can set the flag to not use SAWP for intraframes and to use SAWP for interframes. For example, the encoder can set the flag to use SAWP for some interframes and not to use SAWP for some interframes.

[0200] In yet another method in this embodiment, the encoder sets a flag below the frame level and above the CU level (e.g., tile, slice, patch, LCU, etc.), which allows the decoder to decide whether to use SAWP for this region.

[0201] It should be noted that in this embodiment, the above flag does not have to be set.

[0202] When the decoder decodes the current CU, if intra-frame prediction is used for the current CU, the decoder decodes the SAWP use flag of the current CU, otherwise, the decoder does not need to decode the SAWP use flag of the current CU. Because the related information of DT and IPF is mutually exclusive with SAWP, if the current CU uses SAWP, the decoder does not need to process the related information of DT and IPF.

[0203] [Table 3]

[0204] where SawpMinSize is the minimum of the length and width, and SawpMaxRatio is the largest aspect ratio.

[0205] In Table 3, sawp_flag represents a spatial domain angle weighted prediction flag and is a binary variable. For example, sawp_flag represents that spatial domain angle weighted prediction is performed when the value is 1, and represents that spatial domain angle weighted prediction is not performed when the value is 0. The value of SawpFlag is equal to the value of sawp_flag. If sawp_flag is not present in the bitstream, the value of SawpFlag is 0.

[0206] The values ​​of SawpEnableFlag in Table 3 are equal to the values ​​of sawp_enable_flag in Table 2. If sawp_enable_flag is not present in the bitstream, the value of SawpEnableFlag is 0.

[0207] In this embodiment, assuming that SAWP is currently used for a CU, a weight matrix derivation mode and two intra-frame prediction modes (two kinds of intra-frame prediction modes are taken as an example in this embodiment) need to be decoded. In this embodiment, the weight matrix derivation mode is taken as an example of reusing the weight matrix derivation mode of AWP, and the decoding of the two intra-frame prediction modes of SAWP is taken as an example of reusing the decoding of the intra-frame prediction modes in the related art.

[0208] [Table 4]

[0209] In Table 4, sawp_idx represents a spatial domain angle weighted prediction mode index for determining the weight matrix of spatial domain angle weighted prediction, and the value of SawpIdx is equal to the value of sawp_idx. If sawp_idx does not exist in the bitstream, the value of SawpIdx is equal to 0.

[0210] In Table 4, intra_luma_pred_mode0 represents a first luminance prediction mode of spatial domain angle weighted prediction for determining a first intra-frame prediction mode of a luminance block of spatial domain angle weighted prediction. intra_luma_pred_mode1 represents a second luminance prediction mode of spatial domain angle weighted prediction for determining a second intra-frame prediction mode of a luminance block of spatial domain angle weighted prediction.

[0211] In one embodiment, the analysis method of sawp_idx may be the same as awp_idx in the related art, the analysis method of intra_luma_pred_mode0 may be the same as intra_luma_pred_mode in the related art, and the analysis method of intra_luma_pred_mode1 may be the same as intra_luma_pred_mode in the related art.

[0212] In one embodiment, the analysis method of intra_luma_pred_mode1 may include the following steps: If intra_luma_pred_mode0 and intra_luma_pred_mode1 both use the most possible mode (MPM), intra_luma_pred_mode1 does not need to further analyze whether it is the first intra prediction mode of MPM or the second intra prediction mode. That is, the second intra prediction mode is determined based on the information of the decoded first intra prediction mode. The reason is that the MPM of AVS3 only has two intra prediction modes, and if intra_luma_pred_mode0 uses one of them, intra_luma_pred_mode1 can use another intra prediction mode by default.

[0213] As shown in Table 5, the binarization method of Intra_luma_pred_mode0 has a value of 0 or 1, which indicates whether or not MPM is used. Specifically, when the first binary symbol in the binary symbol string is "1", it indicates MPM, and when it is "0", it indicates not MPM. If the first binary symbol indicates MPM, the second binary symbol in the binary symbol string indicates which MPM it is.

[0214] [Table 5]

[0215] As for the binarization method of Intra_luma_pred_mode1, as shown in Table 6, a value of 0 or 1 of intra_luma_pred_mode1 indicates whether or not MPM is used. Specifically, if the first binary symbol in the binary symbol string is "1", the second binary symbol is also not necessary. If the value of intra_luma_pred_mode0 is 1, the value of intra_luma_pred_mode1 is 0. If the value of intra_luma_pred_mode0 is 0, the value of intra_luma_pred_mode1 is 1.

[0216] [Table 6]

[0217] Still taking the application of the intra-frame prediction method provided in the embodiment of the present application in AVS3 as an example, another embodiment of the analysis and decoding of the codestream structure is introduced.

[0218] The decoder decodes the current CU, and if intra-frame prediction is used for the current CU, it decodes the DT and IPF usage flags of the current CU, and the unique luma prediction mode intra_luma_pred_mode of each prediction unit in the current intra-frame prediction method.

[0219] If neither DT nor IPF is used for the current CU, the SAWP usage flag for the current CU is decoded. If SAWP is used for the current CU, the weight matrix derivation mode and one intra-frame prediction mode, intra_luma_pred_mode1, are further decoded, and the decoded intra_luma_pred_mode is set to intra_luma_pred_mode0.

[0220] [Table 7]

[0221] IntraLumaPredMode0 and IntraLumaPredMode1 are determined based on intra_luma_pred_mode0 and intra_luma_pred_mode1, respectively, and thereby intra-frame prediction sample matrices predMatrix0 and predMatrix1 are determined.

[0222] Determine the weight matrix SawpWeightArrayY based on SawpIdx, where the input index is SawpIdx.

[0223] A new prediction sample matrix predMatrixSawp is determined based on the two intra-frame prediction sample matrices predMatrix0 and predMatrix1 and the determined weight matrix SawpWeightArrayY.

[0224] The value of the element predMatrixSawp[x][y] in the prediction sample matrix predMatrixSawp for the spatial domain angular weighted prediction mode is ((predMatrix0[x][y] * AwpWeightArrayY[x][y] + predMatrix1[x][y] * ( 8 - AwpWeightArrayY[x][y] ) + 4 ) >> 3).

[0225] After determining the prediction block of SAWP, i.e., the prediction sample matrix predMatrixSawp of the spatial domain angular weighted prediction mode, the subsequent processing can further include decoding the quantized coefficients, determining a residual block by inverse transform and inverse quantization, combining the residual block and the prediction block to generate a reconstructed block, and subsequent in-loop filtering, etc. Specific implementations are not intended to limit the scope of protection of the present application and will not be repeated here.

[0226] The method of storing the intraframe prediction mode of SAWP in this embodiment can use a method similar to the method of storing the motion information of AWP, and it is only necessary to replace the input index with SawpIdx and the output intraframe prediction reference mode (interPredAwpRefMode) with sawpRefMode. If the sawpRefMode of a certain 4×4 block is 0, store IntraLumaPredMode0. Otherwise, the sawpRefMode of the 4×4 block is 1, and store IntraLumaPredMode1.

[0227] The first version of AVS3 only supports 34 intra prediction modes, of which the 34th mode (when the index starts from 0, the index number is 33) is a PCM mode. In the second version of AVS3, more intra prediction modes are added, expanding the intra prediction modes to 66. In order to maintain compatibility with the first version, the second version does not change the decoding method of the conventional intra_luma_pred_mode, but instead proposes that one more flag needs to be added when intra_luma_pred_mode is greater than 1, as shown in Table 8, that is, the intra luminance prediction mode extension flag eipm_pu_flag.

[0228] [Table 8]

[0229] The intra-frame luma prediction mode extension flag eipm_pu_flag is a binary variable. When the value of eipm_pu_flag is 1, it indicates that the intra-frame angular prediction extension mode should be used, and when the value of eipm_pu_flag is 0, it indicates that the intra-frame luma prediction extension mode should not be used. The value of EipmPuFlag is equal to the value of eipm_pu_flag. If eipm_pu_flag is not present in the bitstream, the value of EipmPuFlag is equal to 0.

[0230] Therefore, when supporting the description of the second version of AVS3, the descriptions of eipm_pu_flag, eipm_pu_flag0, and eipm_pu_flag1 should be added after the grammars intra_luma_pred_mode, intra_luma_pred_mode0, and intra_luma_pred_mode1 in the above embodiment, as shown in Table 8. Moreover, IntraLumaPredMode0 is determined based on intra_luma_pred_mode0 and eipm_pu_flag0, and IntraLumaPredMode1 is determined based on intra_luma_pred_mode1 and eipm_pu_flag1.

[0231] In the present embodiment, for embodiments that do not use PCM modes or that use only the 33 or 34 modes prior to the first version, there is no need to transmit eipm_pu_flag.

[0232] Obviously, those skilled in the art will appreciate that each module or step of the present application described above can be implemented in a general-purpose computing device, can be centralized in a single computing device, can be distributed across a network of computing devices, and can optionally be implemented in a computing device executable program code, which can then be stored in a storage device and executed by the computing device. Furthermore, in some cases, the steps shown or described herein may be performed in a different order than described herein, or may be implemented by fabricating each of them as individual integrated circuit modules, or by fabricating multiple of the modules or steps as a single integrated circuit module. In this manner, the present application is not limited to any particular combination of hardware and software.

[0233] The above description is merely the best mode of the present application, and is not intended to limit the present application. Those skilled in the art may find that the present application can have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. within the spirit and principle of the present application should be included within the scope of protection of the present application.

Claims

1. 1. An intra-frame prediction method applied to a decoder, comprising: performing intra-frame prediction on a pending block by employing each of two or more different intra-frame prediction modes to obtain two or more types of predicted blocks corresponding to the different intra-frame prediction modes; obtaining a target prediction block for the pending block based on the weight matrix and the obtained two or more prediction blocks; The two or more different intra-frame prediction modes include a first intra-frame prediction mode, and the first intra-frame prediction mode includes at least one of a direction angle (DC) mode, a plane mode, and an angle prediction mode, and a bilinear mode. Intraframe prediction methods.

2. Before obtaining two or more types of prediction blocks corresponding to different intra-frame prediction modes, the intra-frame prediction method further includes: analyzing a codestream to obtain the two or more different intra-frame prediction modes, the pending blocks, and the weighting matrix; The intraframe prediction method according to claim 1 .

3. Before obtaining two or more types of prediction blocks corresponding to different intra-frame prediction modes, the intra-frame prediction method further includes: Parsing the codestream to obtain a flag; The flag is a frame-level flag for indicating whether to subsequently perform a step of obtaining two or more types of prediction blocks corresponding to the different intra-frame prediction modes; and / or the flag is a flag below the frame level and above the coding unit (CU) level, for indicating whether to subsequently perform a step of obtaining two or more types of prediction blocks corresponding to the different intra-frame prediction modes for the indicated region. The intraframe prediction method according to claim 1 .

4. The intra-frame prediction mode includes two different intra-frame prediction modes. The intraframe prediction method according to claim 1 .

5. The intra-frame prediction modes include two different intra-frame prediction modes, the prediction blocks include a first prediction block and a second prediction block, and the weighting matrix includes a first weighting matrix and a second weighting matrix; The step of obtaining a target prediction block of the pending block based on the weight matrix and the obtained two or more kinds of prediction blocks includes: calculating a first product of the first predicted block and a first weighting matrix, and calculating a second product of the second predicted block and a second weighting matrix; calculating a sum of the first product, the second product, and a preset value; and performing a normalization process on the calculated sum to obtain the target prediction block; the second weight matrix is ​​the difference between a maximum weight value and the first weight matrix; the normalization process includes a step of right-shifting the calculated sum by a preset number of bits to obtain a prediction block of the block to be decoded; The intraframe prediction method according to any one of claims 1 to 3.

6. the maximum weight value is equal to 8, the preset value is equal to 4, and the preset number of bits is equal to 3; The intraframe prediction method according to claim 5 .

7. Among all possible weight matrices, there is at least one weight matrix that includes at least two different weight values. The intraframe prediction method according to claim 5 .

8. Among all the possible weight matrices, one weight matrix includes multiple types of weight values, each of the maximum and minimum weight values ​​representing that the predicted value of the corresponding point is entirely from the value of the corresponding point of the first predictive block and the value of the corresponding point of the second predictive block, and each of the weight values ​​that is not the maximum or minimum represents that the predicted value of the corresponding point is entirely from the value of the corresponding point of the first predictive block and the value of the corresponding point of the second predictive block, and The intraframe prediction method according to claim 7.

9. The width and height of the pending block or the target predicted block are: The width is equal to or greater than a first threshold value TH1, and the height is equal to or greater than a second threshold value TH2, Or, the width is equal to or smaller than the fourth threshold value TH4 and the height is equal to or smaller than the fifth threshold value TH5s, The first threshold TH1 is equal to the second threshold TH2, and the first threshold TH1 is equal to 8; The intraframe prediction method according to claim 1 .

10. The intraframe prediction method further comprises: storing intra-frame prediction mode information used in the intra-frame prediction method; The intra-frame prediction mode includes two different intra-frame prediction modes, The step of storing the intra-frame prediction mode used in the intra-frame prediction method includes: The stored minimum units having different intra-frame prediction modes include at least two minimum units. The intraframe prediction method according to any one of claims 1 to 3.

11. The intra-frame prediction mode includes two different intra-frame prediction modes, determining, based on information of one of the two different intra-frame prediction modes that have been decoded, another intra-frame prediction mode; The first intra-frame prediction mode uses a most probable mode (MPM); The intraframe prediction method according to claim 2 .

12. 10. A decoder, the decoder comprising: a processor, the processor comprising: Decoding the received codestream to obtain two or more different intra-frame prediction modes, pending blocks, and weight matrices; performing intra-frame prediction on a pending block by employing each of two or more different intra-frame prediction modes to obtain two or more types of predicted blocks corresponding to the different intra-frame prediction modes; and obtaining a target prediction block for the pending block based on the weight matrix and the obtained two or more types of prediction blocks; The two or more different intra-frame prediction modes include a first intra-frame prediction mode, and the first intra-frame prediction mode includes at least one of a direction angle (DC) mode, a plane mode, and an angle prediction mode, and a bilinear mode. decoder.

13. 1. An intra-frame prediction method applied to an encoder, comprising: performing intra-frame prediction on a pending block by employing each of two or more different intra-frame prediction modes to obtain two or more types of predicted blocks corresponding to the different intra-frame prediction modes; obtaining a target prediction block for the pending block based on the weight matrix and the obtained two or more prediction blocks; The two or more different intra-frame prediction modes include a first intra-frame prediction mode, and the first intra-frame prediction mode includes at least one of a direction angle (DC) mode, a plane mode, and an angle prediction mode, and a bilinear mode. Intraframe prediction methods.

Images