Method of internal prediction fusion (IPF) (variants) and non-volatile machine-readable storage media

Abstract

FIELD: video encoding and decoding.SUBSTANCE: invention is aimed at an increase in coding performance. The result is achieved by determining whether a constraint condition for using IPF for the current block is satisfied, when the intra-prediction mode selected for the current block includes the corner mode, and performing intra-prediction with respect to the current block using IPF when the constraint condition is not satisfied. By limiting the use of IPF, the case of excessive mode fusion for prediction can be avoided and thus the coding performance can be improved.EFFECT: increase in coding performance.15 cl, 21 dwg, 1 tbl

Description

CROSS-REFERENCE TO ONE OR MORE RELATED APPLICATIONS

[0001] This application claims the priority and benefit of Patent Application No. 202210806435.X, filed on July 8, 2022, in the National Intellectual Property Office of China (CNIPA) and entitled “Method and apparatus for constructing candidate list, video encoding method and apparatus, video decoding method and apparatus, video encoding and decoding system”, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD TO WHICH THE INVENTION RELATES

[0002] Embodiments of the present invention relate to, but are not limited to, video technology, and more particularly to intra-prediction fusion (IPF) methods and a non-transitory computer-readable storage medium.BACKGROUND OF THE INVENTION

[0003] Digital video compression technologies are mainly used to compress large amounts of video data for ease of transmission and storage, etc. Current universal video coding standards, such as H.266 / Versatile Video Coding (VVC), adopt a hybrid block-based coding structure. Each image in a video is divided into square largest coding units (LCUs) of the same size (e.g., 128 x 128 or 64 x 64). Each LCU can be further divided into rectangular coding units (CUs) according to certain rules. Each CU can be divided into a prediction unit (PU), a transform unit (TU), etc. The hybrid coding structure contains modules such as a prediction unit, a transform unit, a quantization unit, an entropy coding unit, and a loop filter. The prediction unit includes an intra-prediction unit and an inter-prediction unit, which are capable of reducing or eliminating the inherent redundancy of the video.Intra-block prediction is performed by using adjacent samples of the block as references, and extra-block prediction is performed by using information about adjacent blocks and reference information from other images in the space as references. Compared to the prediction signal, the residual information is encoded into the bitstream using transformation, quantization, and block-based entropy coding. These methods are described in standards and implemented in various fields related to video compression.

[0004] With the growing popularity of video on the Internet and the increasing demands for video clarity, although existing digital video compression standards save significant amounts of video data, there is still a need to create better digital video compression technologies to reduce the load on bandwidth and traffic when transmitting digital video. SUMMARY OF THE INVENTION

[0005] The following is a summary of the subject matter of the present invention, as described in detail herein. This summary is not intended to limit the scope of legal protection of the claims.

[0006] In a first aspect of the present invention, an intra-prediction fusion (IPF) method is proposed. The method is applied to a decoder, and comprises the following. Whether a constraint condition for using IPF for the current block is satisfied is determined when the intra-prediction mode selected for the current block includes the corner mode. The use of IPF for intra-prediction with respect to the current block is constrained when at least one of the constraint conditions is satisfied.

[0007] In a second aspect of the present invention, an IPF method is proposed. The method is applied to an encoder, and comprises the following. Whether a constraint condition for using IPF for the current block is satisfied is determined when the intra-prediction mode selected for the current block includes the corner mode. The use of IPF for intra-prediction with respect to the current block is restricted when at least one of the constraint conditions is satisfied.

[0008] In a third aspect of the present invention, a non-volatile computer-readable storage medium is provided that stores a computer program and a bit stream, wherein, when executed by a processor, the computer program causes the processor to implement the method according to the second aspect for generating the bit stream.

[0009] Other aspects will become apparent after reading and understanding the graphics and detailed description.BRIEF DESCRIPTION OF GRAPHIC MATERIALS

[0010] The accompanying drawings are intended to aid in understanding the embodiments of the present invention and form part of the patent specification. Together with the embodiments of the present invention, the accompanying drawings are used to explain the technical solutions of the present invention and do not constitute a limitation of the technical solutions of the present invention.

[0011] Fig. 1A is a schematic diagram of an encoding and decoding system according to one embodiment of the present invention.

[0012] Fig. 1B is a diagram showing the structure of an encoding side according to one embodiment of the present invention.

[0013] Fig. 1C is a diagram showing the structure of a decoding side according to one embodiment of the present invention.

[0014] Fig. 2 is a schematic diagram illustrating intra prediction modes according to one embodiment of the present invention.

[0015] Fig. 3 is a schematic diagram illustrating neighboring intra prediction blocks for a current block according to one embodiment of the present invention.

[0016] Fig. 4 is a schematic diagram illustrating a template and a template reference area for a current block according to one embodiment of the present invention.

[0017] Fig. 5 is a schematic diagram illustrating multiple reference lines (MRLs) around a current block according to one embodiment of the present invention.

[0018] Fig. 6 is a flow chart of a video encoding method according to one embodiment of the present invention.

[0019] Fig. 7 is a flow chart of a method for generating a template-based multiple reference line (TMRL) candidate list according to one embodiment of the present invention.

[0020] Fig. 8A is a schematic diagram illustrating a template region and extended reference lines around a current block according to one embodiment of the present invention.

[0021] Fig. 8B is a schematic diagram illustrating a template region and extended reference lines around a current block according to another embodiment of the present invention.

[0022] Fig. 9 is a flow chart of a video decoding method according to one embodiment of the present invention.

[0023] Fig. 10 is a flow chart of an intra prediction fusion (IPF) method according to one embodiment of the present invention.

[0024] Fig. 11 is a flow chart of a video encoding method according to another embodiment of the present invention.

[0025] Fig. 12 is a flow chart of a video decoding method according to another embodiment of the present invention.

[0026] Fig. 13 is a flow chart of a video decoding method according to another embodiment of the present invention.

[0027] Fig. 14-16 are flowcharts illustrating methods for generating a template-based multiple reference line (TMRL) candidate list according to three embodiments of the present invention.

[0028] Fig. 17 is a schematic diagram of an apparatus for generating a TMRL candidate list according to one embodiment of the present invention.

[0029] Fig. 18 is a flow chart of a video encoding method according to another embodiment of the present invention.DETAILED DESCRIPTION

[0030] The present invention has illustrated several embodiments, but the illustration serves as an example and not as a limitation, and it will be obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present invention.

[0031] In the illustration of the present invention, terms such as “as an example” or “for example” are used to indicate examples, illustrations, or explanations. Any embodiment described “as an example” or “for example” in the present invention should not be interpreted as more preferred or advantageous than other embodiments. “And / or” in this text illustrates a relationship of related objects, indicating that three relationships may exist. For example, A and / or B may indicate that: only A exists; A and B exist simultaneously; and only B exists. “Several” or “plurality” means two or more than two. In addition, in order to facilitate a clear illustration of the technical solutions of the embodiments of the present invention, terms such as “first” and “second” are used to distinguish between the same or similar elements with substantially the same functions and effects.Those skilled in the art will appreciate that terms such as "first" and "second" do not limit the number and order of execution, and terms such as "first" and "second" do not limit them so that they are necessarily different.

[0032] When describing representative exemplary embodiments in a patent specification, a method and / or process may be presented as a specific sequence of steps. However, to the extent that the method or process does not rely on a specific order of the steps described herein, the method or process should not be limited to the steps in the specific described order. Those skilled in the art will understand that other sequences of steps are also possible. Therefore, the specific sequence of steps set forth in the patent specification should not be interpreted as limiting the claims. In addition, the claims for the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will understand that these sequences can be changed, and these changes will also remain within the spirit and scope of the embodiments of the present invention.

[0033] The local illumination compensation method, the video encoding method, and the video decoding method proposed in the embodiments of the present invention can be applied to various video encoding standards such as H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC), AVS (Audio and Video Coding Standard), and other standards developed by the Moving Picture Experts Group (MPEC), the Alliance for Open Media (AOM), the Joint Video Education Team (JVET), and extensions of these standards, or other specialized standards.

[0034] Fig. 1A is a schematic diagram of an encoding and decoding system that can be used in embodiments of the present invention. As illustrated in the figure, the system is divided into an encoding side device 1 and a decoding side device 2. Encoding side device 1 is configured to generate a bit stream. Decoding side device 2 can decode the bit stream. Decoding side device 2 can receive the bit stream from encoding side device 1 via channel 3. Channel 3 comprises one or more means or devices that can transmit the bit stream from encoding side device 1 to decoding side device 2. In one example, channel 3 comprises one or more communication means that enable encoding side device 1 to send the bit stream directly to decoding side device 2.The encoding device 1 modulates the bit stream according to a communication standard (such as a wireless communication protocol) and sends the modulated bit stream to the decoding device 2 . The one or more communication means may include wireless and / or wired communication means, and they may form part of a packet network. In another example, the bit stream may also be output from the output interface 15 to a storage device, and the decoding device 2 may read the stored data from the storage device via a stream or download.

[0035] As illustrated in the figure, the encoding side device 1 includes a data source 11, a video encoding device 13, and an output interface 15. The data source 11 includes a video shooting device (for example, a camera), an archive containing previously obtained data, a transmission interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. The video encoding device 13 encodes data from the data source 11 and outputs the encoded data to the output interface 15. The output interface 15 may include at least one of a controller, a modem, or a transmitter. The decoding side device 2 includes an input interface 21, a video decoding device 23, and a display device 25. The input interface 21 includes at least one of a receiver or a modem. The input interface 21 can receive a bit stream via channel 3 or from a storage device. The video decoding device 23 decodes the received bit stream.Display device 25 is configured to display decoded data. Display device 25 may be integrated into other devices of decoding side device 2 or may be provided separately. Display device 25 is not required for the decoding side. In other examples, the decoding side may include other devices or equipment for decoding data.

[0036] Based on the video encoding and decoding system as illustrated in Fig. 1A, various video encoding and decoding methods can be used to implement video compression and decompression.

[0037] Fig. 1B is a schematic diagram of an exemplary video encoding apparatus that can be used in embodiments of the present invention. As illustrated in the figure, the video encoding apparatus 1000 comprises a prediction unit 1100, a partition unit 1101, a residual generation unit 1102 (indicated by a circle with a plus symbol after the partition unit 1101 in the figure), a transform processing unit 1104, a quantization unit 1106, an inverse quantization unit 1108, an inverse transform processing unit 1110, a restoration unit 1112 (indicated by a circle with a plus symbol after the inverse transform processing unit 1110 in the figure), a filtering unit 1113, a decoded image buffer 1114, and an entropy encoding unit 1115. The prediction unit 1100 comprises an inter-prediction unit 1121 and an intra-prediction unit 1126.The decoded image buffer 1114 may also be called an image buffer for decoded images, a buffer for decoded images, a decoded image buffer, etc. The video encoder 20 may also contain more, less, or other functional components not present in the example, such as a transform processing unit 1104, an inverse transform processing unit 1110, etc. may be omitted in some cases.

[0038] The partitioning node 1101 interacts with the prediction node 1100 to partition the received video data into slices, code tree units (CTUs), or other relatively large nodes. The video data received by the partitioning node 1101 may be a video sequence including video frames such as I-frames, P-frames, or B-frames.

[0039] The prediction unit 1100 can split the CTU into coding units (CUs) and perform intra-prediction coding or inter-prediction coding on the CUs. To perform intra-prediction and inter-prediction on the CUs, it can be split into one or more prediction units (PUs).

[0040] The inter prediction unit 1121 can perform inter prediction with respect to the PU and generate prediction data for the PU. The prediction data includes a prediction block for the PU, motion information for the PU, and various syntax elements for the PU. The inter prediction unit 1121 can include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit can be configured to estimate motion to generate a motion vector. The motion compensation unit can be configured to obtain or generate a prediction block according to the motion vector.

[0041] The intra-prediction unit 1126 can perform intra-prediction with respect to the PU and generate prediction data for the PU. The prediction data for the PU may include a prediction block and various syntax elements for the PU.

[0042] The residual generation unit 1102 may generate a residual block for a CU by subtracting from the original block for the CU the prediction blocks for the PUs into which the CU is divided.

[0043] The transform processing unit 1104 can split a CU into one or more transform nodes (TUs), and the splitting for the prediction node and the splitting for the transform node may differ. The residual block associated with the TU is a sub-block obtained by splitting the residual block for the CU. The coefficient block associated with the TU is generated by applying one or more images to the residual block associated with the TU.

[0044] The quantization unit 1106 may quantize one or more coefficients in a coefficient block based on one or more selected quantization parameters (QP). The degree of quantization of the coefficient block may be adjusted by adjusting one or more QPs.

[0045] The inverse quantization unit 1108 and the inverse transform processing unit 1110 may apply inverse quantization and inverse transform to the coefficient block, respectively, to obtain a reconstructed residual block associated with the TU.

[0046] The restoration unit 1112 may add the restored residual block and the prediction block generated by the prediction unit 1100 to generate a restored image.

[0047] The filtering unit 1113 is configured to perform contour filtering on the reconstructed image and store the filtered reconstructed image in the decoded image buffer 1114 as a reference image. The intra-prediction unit 1126 can extract a reference image for a neighboring block for the PU from the decoded image buffer 1114 to perform intra-prediction. The inter-prediction unit 1121 can use the previous reference image buffered in the decoded image buffer 1114 to perform inter-prediction on the PU of the current image.

[0048] The entropy encoding unit 1115 may perform entropy encoding on the received data (for example, syntax elements, blocks of quantized coefficients, motion information, etc.).

[0049] Figure 1C is a schematic diagram of an exemplary video decoding apparatus that can be used in embodiments of the present invention. As illustrated in the figure, the video decoding apparatus 101 comprises an entropy decoding unit 150, a prediction unit 152, an inverse quantization unit 154, an inverse transform processing unit 156, a restoration unit 158 ​​(indicated by a circle with a plus symbol after the inverse transform processing unit 155 in the figure), a filtering unit 159, and a decoded image buffer 160. In other embodiments, the video decoder 30 may comprise more, fewer, or other functional components, for example, the inverse transform processing unit 155 may be omitted in some cases.

[0050] The entropy decoding unit 150 may perform entropy decoding on the received bit stream and extract syntax elements, quantized coefficient blocks, motion information for the PU, etc. Each of the prediction unit 152, the inverse quantization unit 154, the inverse transform processing unit 156, the restoration unit 158, and the filtering unit 159 may perform corresponding operations based on the syntax elements extracted from the bit stream.

[0051] The inverse quantization unit 154 may perform inverse quantization with respect to a block of coefficients associated with the quantized TU.

[0052] The inverse transform processing unit 156 may apply one or more inverse transforms to the block of inversely quantized coefficients to generate a reconstructed residual block for the TU.

[0053] The prediction unit 152 includes an inter-prediction unit 162 and an intra-prediction unit 164. If the PU is encoded using intra-prediction, the intra-prediction unit 164 may determine an intra-prediction mode for the PU based on syntax elements decoded from the bitstream, perform intra-prediction according to the determined intra-prediction mode and reconstructed reference information adjacent to the PU obtained from the decoded image buffer 160, to generate a prediction block for the PU. If the PU is encoded using inter-prediction, the inter-prediction unit 162 may determine one or more reference blocks for the PU based on motion information for the PU and corresponding syntax elements for the PU, and generate a prediction block for the PU based on one or more reference blocks obtained from the decoded image buffer 160.

[0054] The restoration unit 158 ​​may obtain a restored image based on the restored residual block associated with the TU and the prediction block for the PU generated by the prediction unit 152.

[0055] The filtering unit 159 can perform loop filtering on the reconstructed image, and the filtered reconstructed image is stored in the decoded image buffer 160. The decoded image buffer 160 can provide a reference image for subsequent motion compensation, intra-prediction, inter-prediction, etc., and can also output the filtered reconstructed image as decoded video data for presentation on a display device.

[0056] Based on the above-mentioned video encoding apparatus and video decoding apparatus, the following basic encoding and decoding processes can be performed. On the encoding side, an image is divided into blocks, intra-prediction, inter-prediction, or other algorithms are performed on the current block to generate a prediction block for the current block, the prediction block is subtracted from the original block for the current block to obtain a residual block, the residual block is transformed and quantized to obtain a quantization coefficient, and entropy coding is performed on the quantization coefficient to generate a bitstream.On the decoding side, intra-prediction or inter-prediction is performed on the current block to generate a prediction block for the current block. On the other side, the quantization coefficient obtained by decoding the bitstream is inversely quantized and inversely transformed to obtain a residual block. The prediction block and the residual block are added to obtain a reconstructed block. The reconstructed block constitutes a reconstructed image. Loop filtering is performed on the reconstructed image based on the image or block to obtain a decoded image. The encoding side also obtains a decoded image through operations similar to those performed on the decoding side, and the decoded image obtained by the encoding side is also commonly referred to as a reconstructed image.The decoded image can be used as a reference image for inter-prediction of subsequent images. Information determined by the coding side, such as block division information, mode information, and parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc., can be written to the bitstream if necessary. The decoding side determines the same block division information, mode information, and parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc., as the coding side by decoding the bitstream or analyzing existing information, thereby ensuring that the decoded image obtained by the coding side is the same as the decoded image obtained by the decoding side.

[0057] Although the above is an example of a hybrid block-based coding structure, embodiments of the present invention are not limited thereto. As technology advances, one or more modules in the structure and one or more steps in the process may be replaced or optimized.

[0058] In the text, the current block may represent a block-level coding node, such as the current CU or the current PU in the current image.

[0059] When the encoding side performs intra-prediction, the encoding side generally performs intra-prediction on the current block using various angular modes and non-angular modes to obtain a prediction block; it selects the optimal intra-prediction mode for the current block according to the rate-distortion information calculated based on the prediction block and the source block; it encodes the intra-prediction mode and transmits the encoded intra-prediction mode to the decoding code through a bitstream. The decoding side obtains the intra-prediction mode selected for the current block through decoding and performs intra-prediction on the current block according to the intra-prediction mode. In the text, the reference line and intra-prediction mode selected for the current block are also expressed as the reference line and intra-prediction mode selected by the current block.

[0060] In VCC and ECM, for several traditional intra-prediction modes, the reconstructed information around the current block is used to predict the current block. Traditional intra-prediction modes include the planar mode with index 0, the direct current (DC) mode with index 1, and the 65 corner prediction modes with indices 2~66. Figure 2 illustrates the angular direction of the corner modes with indices 2~66. In the text, the corner prediction mode is also referred to as the corner mode for brevity.

[0061] Rectangular prediction blocks have been introduced in VVC. For rectangular prediction blocks, the angular directions of some angular modes with indices 2~66 can be replaced with wider angular directions. As illustrated in the figure, angular modes with indices -14~-1 and 67~80 are obtained through wide-angle replacement, which does not require a flag to indicate the selection of these angular modes. These angular modes are determined by the correspondence between the shape of the current block and the indices (2~66) of the prediction modes selected for the current block.

[0062] In VCC standards, the wide-angle replacement is as follows. The whRatio variable is equal to Abs(Log2(blockWidth) - Log2(blockHeight)).

[0063] For a non-square block, whether or not to perform wide-angle replacement for the prediction corner modes predModeIntra (2 ~ 66) is determined according to whether the following conditions are satisfied, where preModeIntra is expressed by an index. preModeIntra = (preModeIntra + 65) If all of the conditions in the following are satisfied: The width of the current block is greater than the height of the current block; The corner mode index >= 2; The corner mode index < (whRatio > 1) ? (8 + 2 * whRatio) : 8. Otherwise, preModeIntra = (preModeIntra - 67) if all three conditions in the following are satisfied: The height of the current block is greater than the width of the current block; The corner mode index <= 66; The corner mode index > ( whRatio > 1) ? (60 – 2 * whRatio) : 60. After matching in wide-angle mode, each corner internal mode predModeIntra (-14 ~ 80) has an angle of intraPredAngle value.

[0064] Table 1 illustrates the correspondence between preModeIntra and intraPredAngle.

[0065] intraPredAngle in preModeIntra is used for subsequent angle prediction.

[0066] In the text, each predModeIntra corresponds to an intraPredAngle. The angle of each angular prediction model is the angle of the line segment corresponding to the angular prediction model in Fig. 1 in the rectangular coordinate system. For example, for the angular mode with index 34, the intraPredAngle is -32, but the angle is -45°, or 45°, or 135°, which is associated with the 0° direction defined in the rectangular coordinate system.

[0067] The intra prediction modes in the text refer to the traditional angular modes, including planar mode, DC mode, and angular modes, unless otherwise limited.

[0068] If the intra-prediction mode for the current block is directly encoded, 7 bits are required to encode 67 modes, resulting in a very large amount of data. Based on statistical characteristics, it is easy to select the same intra-prediction mode for the current block and the sample region near the current block. Based on this characteristic, HEVC, VVC, and the Enhanced Compression Model (ECM) employ Most Probable Mode (MPM) technology. ECM is a benchmark software based on the VTM-10.0-based benchmark software and has been integrated into various new tools for further coding performance research.

[0069] For MPM, an MPM list is first constructed, and the MPM list is filled with the 6 intra-prediction modes that are most likely to be selected for the current block. If the intra-prediction mode selected for the current block is in the MPM list, only the intra-prediction mode index needs to be encoded (only 3 bits are required). If the intra-prediction mode selected for the current block is not in the MPM list, but is one of the 61 non-MPM modes, the intra-prediction mode is encoded using truncated binary coding (TBC) in the entropy coding stage.

[0070] In VVC, regardless of whether multiple reference lines (MRL) and internal sub-divisions (ISP) are applied, the MPM list contains 6 prediction modes. MPMs in ECM are classified into MPMs and secondary MPMs, with the MPM list having a length of 6 and the secondary MPM list having a length of 16. Among the 6 modes in the MPM list, the planar mode is always entered in the first position of the MPM list, and the remaining 5 positions are filled in the next three stages sequentially until all 5 positions are filled. Additional modes will be automatically adopted as secondary MPMs.

[0071] In step 1, the intra-prediction modes used for prediction blocks at 5 adjacent positions around the current block are input sequentially. As illustrated in Fig. 3, the 5 positions include the upper left (AL), upper (A), upper right (AR), left (L), and lower left (BL) positions of the current block sequentially.

[0072] Step 2 introduces a mode obtained from reconstructed samples around the current block using a gradient histogram.

[0073] In step 3, an angular mode is entered that is close in angle to the angular mode selected in step 1.

[0074] The secondary MPM list may consist of some major angular modes except the intra prediction modes in MPM.

[0075] Since the MPM flag (mpm_flag) is encoded and decoded after the MRL mode, the encoding and decoding of MPM in the ECM depends on the MRL flag. When the MRL mode is not used for the current block, the MPM flag must be decoded to determine whether MPM is used for the current block. When the MRL mode is used for the current block, there is no need to decode the MPM flag, and MPM is used for the current block by default.

[0076] Template-based intra-mode derivation (TIMD) and decoder-side intra-mode derivation (DIMD) are two intra-prediction technologies that are not included in VCC but instead implemented in ECM. Using these two technologies, the decoder side can derive the intra-prediction mode for the current block according to the reconstructed sample values ​​around the current block, thereby eliminating the coding of the intra-prediction mode index, thereby saving bits.

[0077] TIMD is an intra-prediction mode for luminance images. The TIMD mode is generated based on a candidate intra-prediction mode in the MPM list and a template region ("template" for short). In ECM, as illustrated in Fig. 4, the left neighboring region and the upper neighboring region of the current block (such as the current CU) 11 constitute the template region 12 of the current block. The left neighboring region is called the left template region (called the left template), and the upper neighboring region is called the upper template region (called the upper template).

[0078] The reference region 13 of the template is located outside (namely, the left and upper sides) of the template region 12, and the exemplary dimensions and positions of each region are illustrated in the figure. In one example, both the width L1 of the left template and the height L2 of the upper template are 4. The reference region 13 of the template may be a row adjacent to the upper part of the template region or a column adjacent to the left side of the template region.

[0079] For TIMD, it is assumed that the distribution characteristics of the current block and the template region for the current block are consistent. By taking the reconstructed value of the template reference region as the reconstructed value of the reference row, all intra-prediction modes in the MPM and secondary MPM are tested to predict the template region and obtain a prediction result. Then, the difference between the reconstructed value of the template region and the prediction result for each mode is calculated and expressed as the sum of absolute transformed differences (SATD). The intra-prediction mode with the lowest SATD is selected as the optimal intra-prediction mode and used as the TIMD mode for the current block. The decoding side can obtain the TIMD mode in the same way. If TIMD is enabled for a sequence, a flag is required to indicate whether TIMD is used for each current block.If the intra-prediction mode selected for the current block is the TIMD mode, prediction for the current block is performed using the TIMD mode and decoding of the remaining syntax elements related to intra-prediction, such as ISP and MPM, can be skipped, thereby significantly reducing the number of bits to encode the mode.

[0080] After calculating the SATD between the reconstructed pattern area value and the prediction result for each mode (pattern area prediction value), the final TIMD mode to be used can be determined as follows.

[0081] Assuming that mode1 and mode2 are two corner modes for intra prediction in MPM, mode1 is the corner mode with the lowest SATD, which is expressed as cost1, and mode2 is the corner mode with the second lowest SATD, which is expressed as cost2, when cost1 × 2 ≤ cost2, mode1 is used as the TIMD mode of the current block, and when cost1 × 2 > cost2, the prediction mode of weighting the prediction results of mode1 and mode2 is used as the TMID mode of the current block, also known as the TIMD fusion mode.

[0082] The weighing method and weight coefficients are illustrated in the formula below:

[0083] represents the prediction result of the current block using the TIMD merging mode, represents the result of predicting the current block using mode1, represents the prediction result of the current block using mode2, and w1 and w2 represent the weighting coefficients calculated according to cost1 and cost2.

[0084] For DIMD, the reconstructed sample value around the current block is taken as a template, and each 3 x 3 region of the template is scanned using the Sobel operator to calculate gradients in the horizontal and vertical directions. Based on the gradients Dx and Dy obtained in the horizontal and vertical directions, the amplitude value at each position is calculated as Amp = abs(Dx) + abs(Dy), and the angle value at each position is calculated as angular = arctan(Dy / Dx). The angle value at each position in the template corresponds to a traditional angular mode, and the amplitude values ​​of the same angular modes are accumulated to form a histogram of amplitude values ​​and angular modes. In cases where there are two angular modes with the highest and second-highest amplitude values, the prediction values ​​for these two angular modes, as well as for the planar mode, are weighted to obtain the final prediction result of the current block using DIMD.In this case, three intra-prediction modes are fused into a prediction mode that includes a planar mode and two corner modes with the highest and second-highest amplitude values. In the text, this prediction mode is referred to as a DIMD fusion mode. If there are no corner modes with the highest and second-highest amplitude values, then prediction using DIMD is equivalent to prediction using a planar mode.

[0085] In HEVC, intra-prediction is performed by using the top row and left column closest to the current block as references. If the differences between the reconstructed values ​​and the original pixel values ​​of this row and column are relatively large, this can significantly affect the prediction quality of the current block. To address this issue, VVC uses MRL technology. In VVC, in addition to reference line 0 (reference line 0), reference line 1 (reference line 1) and reference line 2 (reference line 2) can also be used as extended reference lines for intra-prediction. To reduce coding complexity, MRL is used only for the non-planar mode in MPM.When the encoding side performs predictions using each angular mode, the encoding side must try all three reference lines and, through rate-distortion optimization, selects the reference line with the lowest rate-distortion cost (RD cost) for the current block. The index of the selected reference line is encoded and transmitted to the decoding side. The decoding side obtains the reference line index through decoding and then determines the reference line selected for the current block based on the reference line index to perform prediction for the current block.

[0086] In the example, as illustrated in Fig. 5, 4 reference lines for the current block are illustrated, including reference line0 221 adjacent to the current block, that is, the reference line with index 0, reference line1 222 with an interval of 1 line from the current block, that is, the reference line with index 1, reference line2 223 with an interval of 2 lines from the current block, that is, the reference line with index 2, and reference line3 224 with an interval of 3 lines from the current block, that is, the reference line with index 3. There may be more reference lines for the current block, that is, there may be a reference line with an index greater than 4 for the current block. Only the reconstructed values ​​of some reference lines can be used for prediction. In the text, the indices of the reference lines are numbered as illustrated in Fig. 5.

[0087] In the text, the reference row is referred to as a "row" for convenience of expression. A reference row actually contains a row and a column. Typically, the reconstructed value of a reference row used for prediction also contains the reconstructed row and column values, which is similar to the usual description of the method in this industry.

[0088]

[0006] = {0, 1, 3, 5, 7, 12}. In the MRL index list, the first position is 0, which is the index of the reference row closest to the current block. Positions 2 through 6 are 1, 3, 5, 7, and 12, which are the indices of 5 extended reference rows, arranged in ascending order of distance from the current block to the extended reference rows.

[0089] If the reference rows selected for the current block are in the MRL index list, multiRefIdx is used to indicate the position of the reference row selected for the current block in the MRL index list, and the selected reference row can be specified by encoding multiRefIdx. The MRL index list {0, 1, 3, 5, 7, 12} is taken as an example, and the indices at positions one through six are from 0 to 5, respectively. Assuming that the reference row with index 0 is selected for the current block, multiRefIdx is 0. Assuming that the reference row with index 7 is selected for the current block, multiRefIdx is 4, and so on for other cases. multiRefIdx can be encoded using model-based context truncated unary encoding and the like, and after encoding, multiple binary numbers can be obtained based on the context, where a binary number can also be called a binary flag, a binary symbol, and a binary bit.The smaller the multiRefIdx value, the shorter the code length and the faster the decoding.

[0090]

[0003] = {0, 1, 3}.

[0091] It should be noted that the same technologies may have the same names in different standards. For example, MPM, i.e., the technology for deriving an MPM list using neighboring blocks for the current block, may have different names. For example, it may be called Adaptive Intra-Mode Coding (AIMC) in AV2 (AVM) and called Frequency-Based Intra-Mode Coding (FIMC) in AVS3 when coding on-screen content. For off-screen content, technologies similar to MPM are always used. Like MRL, the technology of using MRL for intra-prediction is called Multiple Reference Line Selection (MRLS) for intra-prediction in AV2 (AVM). However, this is just a difference in names. Where terms such as MPM and MRL are used in an embodiment, technologies that are essentially similar in other standards should also be covered.

[0092] In one embodiment of the present invention, a template-based multiple reference line intra prediction mode, abbreviated as TMRL mode, is provided, which is a prediction mode for composing a candidate list according to combinations of extended reference lines and intra prediction modes, and encoding and decoding the combination of the extended reference lines and the intra prediction mode.

[0093] The video encoding method of this embodiment is applied to an encoder and includes the following, as illustrated in Fig. 6.

[0094] Step 110, a candidate list of the TMRL mode for the current block is formed, wherein the candidate list is filled with combinations of candidate extended reference lines and candidate intra prediction modes for the current block.

[0095] Step 120, the combination of the reference line and the intra prediction mode is selected for the current block for intra prediction through rate-distortion optimization.

[0096] In the text, reference lines include reference lines with index 0 and extended reference lines. The combination of reference line and intra-prediction mode selected for the current block can be a combination of reference line with index 0 and intra-prediction mode, or a combination of extended reference line and intra-prediction mode.

[0097] Step 130, if the encoding condition for the TMRL mode for the current block is satisfied, the TMRL mode flag for the current block is encoded to indicate that the TMRL mode is used for the current block, and the TMRL mode index for the current block is encoded to indicate the position of the selected combination in the candidate list. The encoding condition at least includes the presence of the selected combination in the candidate list.

[0098] In the text, the candidate list filled with combinations of candidate extended reference lines and candidate intra prediction modes for the current block can also be called the TMRL mode candidate list.

[0099] In the text, the candidate list is filled with combinations of candidate extended reference lines and candidate intra prediction modes for the current block, which means that the combinations in the candidate list must participate in the rate-distortion optimization for the current block, that is, participate in the mode selection process when selecting a prediction mode for the current block through the rate-distortion cost, so that all combinations in the candidate list have a chance to be selected.

[0100] In the text, each of the parameters N, M and K representing a number are positive integers and there is no need for further explanation.

[0101] The TMRL mode candidate list constituted in the embodiments of the present invention is filled with combinations of candidate extended reference lines and candidate intra prediction modes for the current block and is not a list containing only candidate extended reference lines or a list containing only candidate intra prediction modes.In the case where the combination of the reference line and the intra-prediction mode selected for the current block is in the candidate list (in this case, an extended reference line is selected for the current block), that is, when the coding condition, such as the combination of the reference line and the intra-prediction mode selected for the current block is in the list, is satisfied, the TMRL mode flag for the current block is encoded such that it indicates that the TMRL mode is used for the current block, and the TMRL mode index for the current block is encoded such that it indicates the position of the selected combination in the candidate list. The decoding side can determine the extended reference line and the intra-prediction mode selected for the current block based on the TMRL mode flag and the TMRL mode index. The method for encoding and decoding the combination in this embodiment can reduce the coding cost and improve the coding performance.

[0102] The TMRL mode is a mode in which prediction with respect to a template region for the current block is performed based on each of N × M combinations obtained from N extended reference lines and M intra prediction modes for the current block, differences between the reconstructed value of the template region and the prediction values ​​obtained through the prediction are calculated, and the corresponding K combinations are input into a candidate list of the TMRL mode for the current block in ascending order of the differences, where 1 ≤ K ≤ N × M.

[0103] In one example, prediction regarding the template region is performed based on each of 25 combinations of extended reference lines and intra-prediction modes in the ECM. The 25 combinations are stored in ascending order of differences, and the 12 combinations with the lowest differences, that is, the 12 combinations that are most likely to be selected, are entered into the TMRL mode candidate list, so that the TMRL mode index can be encoded using fewer bits. Even if some (for example, extended reference lines with indices 7 and 12) of the extended reference lines {1, 3, 5, 7, 12} are outside the CTU boundaries, the 12 combinations that are most likely to be selected can still be added to the candidate list based on the order of 15 combinations, and thus the bits for encoding the MRL mode index can still be fully and efficiently utilized.Sorting the differences in ascending order ensures that the reference strings and prediction modes most likely to be selected for prediction remain in the candidate list, and that the combinations most likely to be selected are placed at the top of the list, thereby reducing the coding cost. The method for constructing the TMRL candidate list will be illustrated in more detail below.

[0104] In an exemplary embodiment, the encoding condition further includes the following: TIMD is not used for the current block. The method further includes the following. If TIMD is used for the current block, encoding of the TMRL mode flag and the TMRL mode index for the current block is skipped. If TIMD is not used for the current block and the selected combination is not in the candidate list, then the TMRL mode flag for the current block is encoded so as to indicate that the TMRL mode is not used for the current block, and encoding of the TMRL mode index for the current block is skipped.

[0105] This embodiment is based on the case where the TIMD mode is encoded and decoded before the TMRL mode. If the TIMD mode is used for the current block, there is no need to use the TMRL mode, so encoding the TMRL mode flag and the TMRL mode index is skipped. If the TIMD mode is not used for the current block, there are two cases: one is that the combination selected for the current block is in the TMRL mode candidate list, and the other is that the combination selected for the current block is not in the TMRL mode candidate list. If the selected combination is not in the candidate list, the TMRL mode flag must be encoded to indicate that the TMRL mode is not used for the current block, and encoding the TMRL mode index is skipped. If the selected combination is in the candidate list, both the TMRL mode flag and the TMRL mode index must be encoded.

[0106] If TIMD is not used for the current block, the TMRL mode flag and TMRL mode index provided in the embodiment can replace the original index of multiple reference rows multiRefIdx. If TIMD is used for the current block, multiRefIdx can still be used to represent the selected reference row, and multiRefIdx is encoded.

[0107] In one exemplary embodiment, generating a TMRL candidate list for the current block includes the following. The candidate list is generated only when all established conditions for allowing the use of the TMRL mode for the current block are satisfied, wherein the conditions for allowing the use of the TMRL mode include any one or more of the following.

[0108] Condition 1: The current block is a block in the luminance image, that is, the TMRL mode is only used for luminance images.

[0109] Condition 2: The current block is not within the upper bounds of the CTU. If the current block is within the upper bounds of the CTU, there is no available reference row at the upper end of the current block. Therefore, in this embodiment, the condition that the current block is not within the upper bounds of the CTU is adopted as the condition for enabling the TMRL mode.

[0110] Condition 3: MRL is allowed for the current block, that is, TMRL mode is allowed only when MRL mode is allowed.

[0111] Condition 4: The current block size is not larger than the maximum current block size for which TMRL mode can be used, with the maximum size being preset. Larger blocks are typically flat and less likely to contain corner details. TMRL mode can be limited so that it is used for such large blocks.

[0112] Condition 5: The aspect ratio of the current block meets the aspect ratio requirement for the current block to be eligible for TMRL mode. For example, TMRL mode is only eligible for use when the aspect ratio of the current block does not exceed the preset value.

[0113] In one exemplary embodiment, the TMRL mode index is encoded using Golomb-Rice coding. Using Golomb-Rice coding can more accurately classify candidate combinations into categories with different codeword lengths for encoding and decoding, thereby improving coding efficiency.

[0114] In one example embodiment, if the encoding condition for the TMRL mode for the current block is satisfied, then the method further includes: skipping encoding of syntax elements of any one or more of the following modes: MPM mode, ISP mode, multiple transform selection (MTS) mode, low frequency non-separable transform (LFNST) mode, and TIMD mode.

[0115] For example, if TMRL mode is encoded and decoded before TIMD mode, the encoding condition for TMRL mode does not include the clause that TIMD is not used for the current block. If TMRL mode is used for the current block, TIMD is not allowed for the current block, and encoding of TIMD mode syntax elements can be skipped.

[0116] For example, if the TMRL mode is used for the current block, both the reference line and the intra-prediction mode selected for the current block can be designated by the TMRL mode flag and the mode index MRL_IP. In this case, there is no need to encode and decode MPM-related syntax elements.

[0117] For example, in a particular conversion mode, TMRL mode may not be allowed to be used simultaneously with MTS mode and / or LFNST mode.

[0118] In one embodiment, a method for generating a TMRL mode candidate list is provided. The method can be applied to an encoder and can also be applied to a decoder. As illustrated in Fig. 7, the method includes the following.

[0119] Step 210, N × M combinations of extended reference lines and intra prediction modes are obtained according to N extended reference lines and M intra prediction modes for the current block, where N ≥ 1, M ≥ 1, and N × M ≥ 2.

[0120] Step 220, a prediction regarding the template region for the current block is performed according to each of the N × M combinations, and differences between the reconstructed value of the template region and the prediction values ​​obtained through the prediction are calculated.

[0121] The difference in this step can be expressed as the sum of absolute differences (SAD) or the sum of absolute transformed differences (SATD), but is not limited to these. The difference in this step can also be expressed as the sum of squared differences (SSD), mean absolute difference (MAD), mean squared error (MSE), and so on.

[0122] Step 230, K combinations corresponding to the differences are entered into the TMRL mode candidate list for the current block in ascending order of the differences, where 1 ≤ K ≤ N × M.

[0123] The candidate list constructed in the embodiment can realize coding of a combination of an extended reference line and an intra-prediction mode, thereby improving coding efficiency. Furthermore, by using different combinations to predict the template region and sorting the differences, it is possible to select, based on the similarity of the distribution characteristics between the current block and the template region for the current block, K combinations with a relatively high selection probability from N × M combinations. The combinations with a relatively high selection probability are placed at the top of the candidate list, so that the TMRL mode index for the selected combination to be encoded is relatively small, thereby reducing the actual coding cost.

[0124] In one exemplary embodiment, the template region for the current block is in the reference row located closest to the current block, or the template region for the current block is in multiple reference rows located closest to the current block, wherein the N extended reference rows to participate in the combination are extended reference rows located outside the template region. In Fig. 8A, the template region for the current block is in the reference row 30 with index 0, and to construct a candidate list of the TMRLP mode, N extended reference rows that can be used are selected from the specified extended reference rows with indices {1, 3, 5, 7, 12}. If there are more than 13 reference rows between the upper side of the current block and the CTU boundary, then 5 extended reference rows with indices {1, 3, 5, 7, 12} are selected to participate in the combination.If there are 6 to 7 reference rows between the top of the current block and the CTU boundary, then 3 extended reference rows with indices {1, 3, 5} are selected to participate in the combination, and so on.

[0125] In one example, if the pattern area for the current block is at the reference row with index 0, the reference row with index 0 is called the reference row where the pattern area is located, and the reference rows with indices 1 to 3 are called reference rows outside the pattern area. If the pattern area for the current block is at the reference rows with indices 0 and 1, the reference rows where the pattern area is located include extended reference rows, and the reference rows with indices 0 and 1 are the reference rows where the pattern area is located, and the reference rows with indices 2 and 3 are the reference rows that are outside (on the upper side and left side) of the pattern area.

[0126] In Fig. 8A, 5 extended reference rows participating in the combination are illustrated, including reference row 31 with index 1, reference row 33 with index 3, reference row 35 with index 5, reference row 37 with index 7, and reference row 39 with index 12. Unlike Fig. 8A, in the example illustrated in Fig. 8B, the template area 40 for the current block is in two reference lines with indices 0 and 1, and the extended reference lines participating in the combination are 5 extended reference lines, in particular the reference line 42 with index 2, the reference line 43 with index 3, the reference line 45 with index 5, the reference line 47 with index 7, and the reference line 49 with index 12. That is, in the example, N extended reference lines that can be used are selected from the given extended reference lines with indices {2, 3, 5, 7, 12}.There may be many other options for choosing the template region and extended reference rows, such as setting the template region for the current block so that it is within 3 reference rows with indices 0, 1, and 2, setting the template region for the current block so that it is within 4 reference rows with indices 0 through 3, and so on. If the template region is relatively wide, the prediction is relatively more accurate.

[0127] In one example embodiment, the N extended reference strings for the current block are extended reference strings that are outside the template region for the current block and do not extend beyond the CTU boundaries among the N max given extended reference lines, where N maxrepresents the maximum number of extended reference lines that can be used for the TMRL mode. In this embodiment, N extended reference lines used for combination are set such that they are outside the template region for the current block and extend beyond the CTU boundaries. However, if the hardware can support it, extended reference lines extending beyond the CTU boundaries can also be selected for combination.

[0128] In one example of the present invention N max is 5, and the 5 specified extended reference strings are the reference strings with indices {1, 3, 5, 7, 12} or {2, 3, 5, 7, 12}. In another example, N max the given extended reference strings represent N max extended reference rows located closest to the current block, with indices starting at 1. Alternatively, N max the given extended reference strings represent N maxextended reference rows located closest to the current block, with odd indices starting at 1. Alternatively, N max the given extended reference strings represent N max extended reference rows closest to the current block, with even indices starting at 2. Selecting odd reference rows or even reference rows can simplify the operation.

[0129] In one example of an embodiment, M intra prediction modes are selectable only from angular modes, or are selectable only from angular modes and a DC mode, or are selectable from angular modes, a DC mode, and a planar mode.

[0130] In one example, M intra prediction modes are selected in the following ways, with M≥5.

[0131] Step 1: The intra-prediction modes used for the prediction blocks at 5 adjacent positions around the current block are determined, the intra-prediction modes that are selectable are selected sequentially, and the duplicate modes are removed, wherein the 5 adjacent positions are upper left (AL), upper (A), upper right (AR), left (L), and lower left (BL), as illustrated in Fig. 3. If the number of intra-prediction modes selected in step 1 is M, the process ends. If the number of intra-prediction modes selected in step 1 is less than M, and the selected intra-prediction modes include a corner mode, the process proceeds to step 2.

[0132] Step 2, starting from the first angular mode that has been selected, sequentially perform an expansion operation on the angular modes to obtain expanded angular modes, and expanded angular modes that are different from all the selected angular modes are selected until the total number of selected intra prediction modes is equal to M.

[0133] In the embodiment, if no corner mode is selected in step 1 or the total number of selected intra-prediction modes after step 2 is still less than M, then step 3 is performed: intra-prediction modes that are selectable from a set of predetermined intra-prediction modes are determined, and intra-prediction modes that are different from all of the selected intra-prediction modes are selected from the determined intra-prediction modes that are selectable, until the total number of selected intra-prediction modes is equal to M.

[0134] In one example, only angular modes are allowed to be selected in the process of selecting M intra-prediction modes. In another example, only angular modes and the DC mode are allowed to be selected in the process of selecting M intra-prediction modes. In another example, angular modes, the DC mode, and the planar mode are allowed to be selected in the process of selecting M intra-prediction modes. The effect is limited when the planar mode is combined with extended reference lines, and thus the planar mode may not be included in the combination. The situation with the DC mode is similar. However, if the increased computational complexity can be tolerated, the planar mode and the DC mode can also be added to the combination candidate list.

[0135] In one example, the extended operation with respect to the angular modes includes any one or more of the following operations: performing a + / -1 operation with respect to the angular modes; performing a + / -2 operation with respect to the angular modes; and performing a + / -3 operation with respect to the angular modes.

[0136] In one example of an embodiment, M intra-prediction modes include some or all of the intra-prediction modes except for the planar mode in the MPM; or include some or all of the intra-prediction modes except for the planar mode in the MPM and secondary MPMs; or include some or all of the intra-prediction modes except for the planar mode and the DC mode in the MPM; or include some or all of the intra-prediction modes except for the planar mode and the DC mode in the MPM and secondary MPMs; or include some or all of the intra-prediction modes except for the planar mode, the DC mode, and the DIMD mode in the MPM; or include some or all of the intra-prediction modes except for the planar mode, the DC mode, and the DIMD mode in the MPM and secondary MPMs.

[0137] In this example, all intra-prediction modes except the planar mode in MPM are used as the specified intra-prediction modes, or all intra-prediction modes except the planar mode in MPM and secondary MPMs are used as the specified intra-prediction modes. If the reference rows for the current block include all the specified extended reference rows, then all the specified intra-prediction modes are used. If the reference rows for the current block include only some of the specified extended reference rows, then some of the specified intra-prediction modes are used.

[0138] In one exemplary embodiment, M intra-prediction modes are selected as follows. M' intra-prediction modes are selected. According to each reference row for the current block that is outside the template and M' intra-prediction modes, a prediction is made with respect to the template for the current block and the difference between the reconstructed template value and each prediction value obtained through the prediction is calculated, obtaining M' differences. M intra-prediction modes with the lowest differences are selected from the M' intra-prediction modes as M intra-prediction modes to participate in the combination, where M < M'.

[0139] In the example, the template for the current block is used to select M intra-prediction modes from M' intra-prediction modes, but the above template region for the current block is used to select K combinations from N × M combinations. The two templates may be different, but they can also occupy the same region.

[0140] In the example for selecting M' intra-prediction modes, various methods for selecting M intra-prediction modes can be used in the above embodiments of the present invention. For example, M' intra-prediction modes can be directly selected from the MPM list and the secondary MPM list, or selected by step 1 in the method of the above embodiment, or selected by step 1 and step 2 in the method of the above embodiment, or selected by step 1, step 2, and step 3 in the method of the above embodiment, etc.

[0141] In the example, N ≤ N max, 2 ≤ N max ≤ 12; 2 ≤ M ≤ 18; K ≤ K max , 6 ≤ K max ≤ 36, and N max represents the maximum number of extended reference lines allowed to be used for the TMRL_IP mode, and K max represents the maximum number of candidate combinations allowed for use in the TMRL_IP mode. Although ranges of values ​​for the corresponding parameters N, M, and K are provided in this document, these are only examples, and these ranges of values ​​should not be considered limitations of the present invention.

[0142] In one example embodiment, N, M, and K have at least two sets of values, the first set of values ​​is N1, M1, and K1, and the second set of values ​​is N2, M2, and K2, wherein N1≤ N2, M1≤ M2, K1≤ K2, and N1×M1< N2×M2. The first set of values ​​is used to compile a candidate list for the current block of the first size, the second set of values ​​is used to compile a candidate list for the current block of the second size, and the first size is less than the second size. The first size and the second size in this case can represent multiple sizes respectively, for example, the first size can include 4×4, 4×8, 8×8, etc., and the second size can include 16×8, 16×16, 8×16, etc. In the example, different values ​​of N, M, and K are used for the current blocks of different sizes. If the current block size is relatively small, then relatively small values ​​are used to compile the TMRL candidate list.If the current block size is relatively large, then relatively large values ​​are used to generate the TMRL candidate list. This allows for a better balance between operational complexity and performance.

[0143] In one exemplary embodiment, performing a prediction on a template region for the current block according to each of the N×M combinations includes the following. If the current block is on the left border of the image, a prediction on the upper template region for the current block is performed according to each of the N×M combinations, and no prediction is performed on the left template region for the current block. In this embodiment, the operation can be simplified and the time required for the operation can be reduced without affecting performance.

[0144] In one example of the present invention, performing a prediction on a template region of a current block according to each of the N×M combinations includes: performing a prediction according to each of the N×M combinations in the following manner. An original prediction value of the template region is calculated according to a reconstructed value of an extended reference row in the combination and an intra-prediction mode in the combination, wherein the reconstructed value of the extended reference row is the original reconstructed value of the extended reference row or a filtered reconstructed value of the extended reference row. 4-tap filtering or 6-tap filtering is performed on the original prediction value of the template region, and the filtering result is used as the prediction value of the template region obtained through the prediction based on the combination.

[0145] In this embodiment, to predict the template region for the current block, the reconstructed value of the extended reference string may not be filtered, and the original reconstructed value of the extended reference string is used for the calculation. A filter with relatively short taps (e.g., a 4-tap filter) can be used to reduce the computation complexity and speed up the calculation.

[0146] In one exemplary embodiment, performing a prediction on a template region for a current block according to each of N × M combinations, and calculating the difference between the reconstructed value of the template region and the prediction values ​​obtained by the prediction includes the following. Prediction on the entire template region for the current block is performed according to each of K combinations to obtain corresponding K differences, wherein the K differences form a difference set, and the maximum difference in the difference set is denoted as D maxFor each of the remaining combinations, the prediction for the template region on the side of the current block is first performed according to the combination, and the difference D1 between the reconstructed value of the template region on the side of the current block and the prediction value of the template region on the side of the current block is calculated. If D1≥ D max , then the prediction based on the combination is complete. If D1< D max , then the prediction regarding the template region regarding the other side of the current block is performed according to the combination, and the difference D2 is calculated between the reconstructed value of the entire template region for the current block and the prediction value of the entire template region for the current block. If D2< D max , then D2 is added to the set of differences, D max remove from the set of differences and the maximum difference D max in the difference set are updated. If D2≥ D max, then the prediction based on the combination is complete. After performing the prediction based on N × M combinations, the K combinations corresponding to the K differences in the set of differences are taken as the K combinations with the lowest differences.

[0147] In one example, the differences in a difference set can be sorted in ascending order, and when D2 is added to the difference set, the position at which D2 is placed must keep the differences in the difference set in the same ascending order. However, in other examples, the K differences in a difference set can also be sorted after performing a prediction based on all N × M combinations.

[0148] In the example, the combinations can be sorted without performing prediction and difference calculation on the entire pattern area based on all combinations, which can reduce the complexity of the operation and speed up the operation.

[0149] In one exemplary embodiment, K combinations corresponding to the differences are entered into the TMRL mode candidate list for the current block in ascending order of differences as follows. Starting from the first position of the candidate list, K combinations corresponding to the differences are entered into the candidate list in ascending order of differences. The TMRL mode candidate list in the example is filled only with combinations of extended reference rows and intra-prediction modes. Combinations of reference rows with index 0 and intra-prediction modes are designated by other conventional modes, such as MPM.

[0150] In one exemplary embodiment, starting from the first position of the candidate list, K combinations corresponding to the differences are input into the candidate list in ascending order of the differences. The position(s) up to the i-th position in the candidate list are input with the combination(s) of the reference row with index 0 and one or more intra-prediction modes, where i ≥ 2. The candidate list of the TMRL mode in the example is filled not only with combinations of extended reference rows and intra-prediction modes, but also with the combination(s) of the reference row with index 0 and the intra-prediction mode(s). In this case, if the combination selected for the current block is a combination of the reference row with index 0 and the intra-prediction mode, the combination can also be represented by the TMRL mode index. In this case, the TMRL mode index can still be used.

[0151] In one embodiment, a video decoding method associated with the TMRL mode is provided. The method is applied to a decoder and includes the following, as illustrated in Fig. 9.

[0152] Step 310, decode the TMRL mode flag for the current block to determine whether the TMRL mode is used for the current block.

[0153] Step 320, if it is determined that the TMRL mode is used for the current block, then the index of the TMRL mode for the current block is further decoded and a candidate list of the TMRL mode for the current block is generated, wherein the candidate list is filled with combinations of candidate extended reference lines and candidate intra prediction modes for the current block.

[0154] Step 330: Based on the candidate list and the TMRL mode index, determine the combination of extended reference lines and the intra-prediction mode selected for the current block, and perform prediction on the current image according to the selected combination. The TMRL mode index is used to indicate the position of the selected combination of extended reference lines and intra-prediction mode in the candidate list.

[0155] In the present embodiment, after determining that the TMRL mode is used for the current block by decoding the TMRL mode flag, the combinations of extended reference lines and intra-prediction modes are entered into the TMRL mode candidate list, the combination selected for the current block is determined according to the TMRL mode index obtained through decoding and the candidate list, and prediction for the current block is performed based on the selected combination. That is, the TMRL mode index can indicate both the extended reference lines and the intra-prediction mode selected for the current block, and there is no need to use two indices to respectively indicate the extended reference line and the intra-prediction mode, thereby reducing the coding cost.

[0156] In one example of an embodiment, before decoding the TMRL mode flag for the current block, the method further includes the following. If all the conditions for allowing the use of the TMRL mode for the current block are satisfied, then the TMRL mode flag for the current block is decoded, wherein the conditions for allowing the use of the TMRL mode include any one or more of the following: the current block is a block in the luminance image; MRL is allowed for the current block; the current block is not on the upper boundary of the CTU; TIMD is not used for the current block.

[0157] In this example, if one of the above conditions is met, the TMRL mode is not allowed to be used, and decoding for the TMRL mode flag and TMRL mode index can be skipped. However, this is not necessary in other embodiments. For example, if the TMRL mode is encoded and decoded before TIMD, then the use of TIMD for the current block cannot be used as a condition for not allowing the use of TMRL mode. As another example, in the future, if the hardware can support the acquisition of reference lines located outside the CTU boundaries, then the current block, located at the upper boundary of the CTU, will no longer be a condition for not allowing the use of TMRL mode, and so on.

[0158] In one example embodiment, the method further includes the following. In the case where it is determined through decoding that the MRL is allowed for the current block, the current block is not on the upper boundary of the CTU, and TIMD is used for the current block, then the MRL index for the current block is decoded, wherein the MRL index is used to indicate the position of the reference line selected for the current block in the MRL index list. In the example, in the case where TIMD is used for the current block, the TMRL mode is not allowed, but the MRL is still allowed, and thus the reference line selected for the current block can still be determined by decoding the MRL index for the current block, and then the prediction regarding the current block can be performed according to the selected reference line in combination with the TIMD mode selected for the current block.

[0159] In one exemplary embodiment, if it is determined according to the TMRL mode flag that the TMRL mode is used for the current block, the method further comprises: skipping decoding for syntax elements of any one or more of the following modes: MPM mode, ISP mode, MTS mode, LFNST mode, and TIMD mode. If encoding for one or more of the above modes is skipped on the encoding side in the case that the TMRL mode flag is used for the current block, the decoding side accordingly also skips decoding for these modes in the case that the decoding side determines through decoding that the TMRL mode flag is used for the current block.

[0160] In one embodiment, a video decoding method is further provided, specifically including a decoding process for intra-prediction, and the coding side is also described. In this embodiment, the coding side generates a TMRL mode candidate list. If a combination in the candidate list is selected for the current block through mode selection, the TMRL mode syntax elements are encoded and decoded. Combinations of extended reference lines and intra-prediction modes are encoded and decoded.

[0161] In the embodiment, according to N predetermined extended reference lines and M intra prediction modes, a template, i.e., reference line0, is created in the reference line with index 0, and reference can be made to the template region 30 illustrated in Fig. 8A. (x, -1), (-1, y) are coordinates of positions relative to the upper left corner (0, 0) of the current block. In the figure, 5 predetermined extended reference lines with indices {1, 3, 5, 7, 12} are added.

[0162] To construct a TMRL mode candidate list, the SAD between the template prediction values ​​obtained based on N × M combinations and the reconstructed template value is calculated, the corresponding combinations are sorted in ascending order of SAD, and K combinations with relatively small SADs are input into the TMRL mode candidate list in ascending order of SAD, where N × M ≥ 2.

[0163] Constructing a TMRL mode candidate list is an operation that must be performed by both the encoder and decoder. If the coding conditions, such as the fact that the selected combination for the current block is in the candidate list, are satisfied, the coding side encodes a TMRL mode flag to indicate the use of the TMRL mode and also determines the TMRL mode index according to the position of the selected combination in the candidate list. For example, if the selected combination is in the first position, the TMRL mode index is 0; if the selected combination is in the second position, the TMRL mode index is 1, and so on. The TMRL mode index can be encoded using Golomb-Rice coding, but the present invention is not limited thereto.

[0164] This is illustrated by taking N to be 5, M to be 6, and K to be 12 as an example. The indices of the 5 specified extended reference rows are {1, 3, 5, 7, 12}, and 6 intra prediction modes are selected step by step.

[0165] The video decoding method according to the embodiment includes the following.

[0166] Stage 1: Decode syntax elements related to TMRL mode.

[0167] The decoder parses syntax elements related to intra-prediction modes, including syntax elements related to the TIMD mode, MRL mode, and other modes. The TMRL mode proposed in this embodiment can be considered an evolution of the MRL mode, and the syntax elements of the TMRL mode can also be considered part of the syntax elements of the MRL mode. Of course, the two modes can also be considered two different modes.

[0168] If TIMD is used for the current block, the decoding method for MRL mode syntax elements remains unchanged. If TIMD is not used for the current block, TMRL mode syntax elements must be decoded. Syntax elements for the current block are decoded as shown in the table below:

[0169] "cu_tmrl_flag" in the table represents the TMRL mode flag. If "cu_tmrl_flag" is 1, it means that the TMRL mode is used for the current block, meaning that the intra-prediction mode for the current clarity sample is TMRL. If "cu_tmrl_flag" is 0, it means that the TMRL mode is not used for the current block, meaning that the intra-prediction mode for the current clarity sample is a mode other than TMRL.

[0170] "tmrl_idx" in the table represents the TMRL mode index, which indicates the position of the combination of the extended reference row and the intra-prediction mode selected for the current block in the TMRL mode candidate list. Alternatively, "tmrl_idx" can be said to define the index of the selected combination in the candidate list containing sorted TMRL modes (the index indicating the combination's position). "tmrl_idx" can be encoded and decoded using Golomb-Rice coding, which will not be repeated here.

[0171] As shown in the table above, before decoding cu_tmrl_flag, it is first determined whether the following conditions are satisfied: MRL is enabled for the current block (i.e., whether sps_mrl_enabled_flag is 1); the current block is not on the upper boundary of the CTU (i.e., whether y0 % CtbSizeY) > 0); and TIMD is not used for the current block. If these conditions are satisfied, cu_tmrl _flag is decoded. If the other two conditions are satisfied and TIMD is used for the current block, the MRL index intra_luma_ref_idx for the current block is decoded.

[0172] The ISP mode flag intra_subpartitions_mode_flag in the table is decoded after the syntax elements related to the TMRL mode. If the TMRL mode is not used for the current block (!cu_tmrl_flag is true), then intra_subpartitions_mode_flag is decoded. Similarly, if the TMRL mode is not used for the current block (!cu_tmrl_flag is true), then the syntax elements related to the MPM are decoded.

[0173] Step 2: Compile a TMRL mode candidate list, and determine the extended reference line and intra-prediction mode selected for the current block according to the TMRL mode index and the candidate list. After the analysis step, before performing prediction on the current block, if the TMRL mode is used for the current block, compose a TMRL mode candidate list, and determine the extended reference line and intra-prediction mode selected for the current block according to the TMRL mode index and the candidate list.

[0174] To construct a TMRL mode candidate list, candidate extended reference lines and candidate intra prediction modes must first be defined.

[0175] ■ Define candidate extended reference strings

[0176] Candidate extended reference lines are selected from the given extended reference lines. Which of the given extended reference lines can be used is determined according to the position of the current block in the image. In principle, the upper reference lines that can be used for the current block must not exceed the upper boundary of the CTU. In one example, among the extended reference lines with indices {1, 3, 5, 7, 12}, extended reference lines that do not exceed the CTU boundary are added to the candidate extended reference lines. To achieve better coding performance or reduce complexity, more or fewer extended reference lines can be used.

[0177] ■ Determine candidate intra prediction modes

[0178] In this embodiment, the TMRL mode is not associated with the MPM (it may be associated with the MPM in other embodiments), but a candidate list of intra-prediction modes is compiled, and the intra-prediction modes used for combination are selected from the candidate list. The method for obtaining the candidate list is as follows.

[0179] First, from the 67 traditional prediction modes, planar mode and DC mode are excluded, or only planar mode is excluded and DC mode is retained. One or more excluded modes are not added to the candidate list, i.e., they are not used as intra-prediction modes for combination in TMRL modes.

[0180] In the embodiment, the length of the prediction mode candidate list to be constructed is 6. Non-repeating intra-prediction modes are selected sequentially according to the intra-prediction modes used for prediction blocks at 5 adjacent positions around the current block to fill the prediction mode candidate list. Then, an expansion operation is performed on the angular modes entered into the list. Specifically, a + / -1 operation is performed on the angular modes sequentially, and non-repeating expanded angular modes are sequentially selected and entered into the prediction mode candidate list. If the number of modes entered into the candidate list reaches 6, the filling is stopped.

[0181] The specific process of + / -1 operation is illustrated in the table below:

[0182] The above "angle mode - 1" refers to the angle mode obtained by subtracting 1 from the index of the input angle mode. For example, if the input angle mode is mode 3, then "angle mode + 1" represents mode 2. The above "angle mode + 1" refers to the angle mode obtained by adding 1 to the index of the input angle mode. For example, if the input angle mode is mode 3, then "angle mode + 1" represents mode 4.

[0183] If the "angle mode - 1" is less than the angle mode 2, for example, if the "angle mode - 1" is angle mode 1 (the indices of the angle modes are numbered from 2, and the angle mode 1 does not exist), then the angle mode in the direction opposite to the "angle mode - 1" is selected. Suppose there are 65 angle modes, in this case, the angle mode in the opposite direction is the 66 angle mode. If the "angle mode + 1" is greater than the 66 angle mode, then the angle mode in the opposite direction to the "angle mode + 1" is selected in a similar way. For example, if the input angle mode is the 66 mode, then the "angle mode + 1" does not exist, in this case, the angle mode in the direction opposite to the "angle mode + 1" is the 2 angle mode.

[0184] Regarding the expansion with respect to angular modes, the + / -1 operation is performed with respect to angular modes. In other embodiments, the + / -1 operation can be expanded to the "+ / -X" operation. Suppose X is 3, the "+ / -1," "+ / -2," and "+ / -3" operations can be performed with respect to angular modes until the list of candidate prediction modes is full.

[0185] If the prediction mode candidate list is still not full after performing the expansion operation on the angular modes that have been input into the list, the prediction mode candidate list is filled with non-repeating modes in the set of specified modes until the prediction mode candidate list is full. The mode set contains some angular modes selected according to statistical rules as follows: mpm_default[] = {DC_IDX, VER_IDX, HOR_IDX, VER_IDX - 4, VER_IDX + 4, 14, 22, 42, 58, 10, 26, 38, 62, 6, 30, 34, 66, 2, 48, 52, 16}, where DC_IDX represents the DC mode, VER_IDX represents the vertical mode, HOR_IDX represents the horizontal mode, and the remaining numbers represent the angular modes corresponding to the numbers.

[0186] In an embodiment, the length of the candidate mode list is 6. For performance purposes, more angle modes can be tried, i.e., the length is set to a value greater than 6, or to reduce complexity, fewer modes can be tried, i.e., the length is set to a value less than 6.

[0187] In this embodiment, when determining candidate intra-prediction modes, the planar mode and DC mode may be excluded, or only the planar mode may be excluded. However, without considering complexity, two modes may not be excluded; that is, all of the planar mode, DC mode, and all angular modes may be used as candidate intra-prediction modes for participation in combination with extended reference lines.

[0188] ■ Compile the TMRL regime candidate list

[0189] After determining the candidate extended reference lines and the candidate intra prediction modes, all combinations of the extended reference lines in the list of the extended reference lines and the prediction modes in the candidate prediction mode list can be tried one by one, and prediction with respect to the template region in the reference line 0 (the template as illustrated in the figure below) is performed based on each combination, as illustrated in Fig. 8A. The difference between the reconstructed value of the template region and the prediction value obtained by predicting based on each combination is calculated, and K combinations corresponding to the lowest differences are input into the candidate list of the TMRL mode in ascending order of differences.

[0190] In the prediction process of the embodiment, the use of the TMRL mode is limited if the current block is in the first row of the CTU. If the current block is on the left edge of the image, the TMRL mode can still be used. In this case, since the reference row 0 on the left is already outside the image edge, no prediction is performed on the left template; instead, prediction is performed only on the upper region of the template.

[0191] The pattern region prediction process can be fully coordinated with other conventional intra-angle prediction processes. That is, the reconstructed reference line value is first filtered and then used as the initial prediction value of the pattern region. After the pattern region prediction is performed, 4- or 6-tap filtering is performed on the initial prediction result based on the filtered reconstructed reference line value and the intra-prediction mode in combination, and the filtered initial prediction result is then used as the prediction value obtained through the prediction. Considering the complexity of the operation, filtering on the reconstructed reference line value can be omitted, and a filter with a relatively short tap can be used.In an embodiment, to obtain a prediction value of a template region by prediction, the reconstructed pixel value of the reference row is not filtered, and the result of the original prediction is filtered by a 4-tap interpolation filter with a precision of 1 / 32 under a non-integer angle.

[0192] Finally, a prediction is made on the template region based on the angle and reference row in the current combination and filter. The SAD between the predicted template region values ​​obtained through prediction and the reconstructed template region value is calculated and sorted in ascending SAD order, and the 12 combinations with the lowest SADs are selected and entered into the TMRL mode candidate list.

[0193] A fast algorithm can be used in the sorting process. In the above process, 5 reference rows and 6 prediction modes with a total of 30 combinations need to be tried, but only 12 combinations with the lowest SAD need to be selected. In this embodiment, after completing the prediction based on the first 12 combinations and obtaining the corresponding SAD, only the 12 combinations with the lowest differences (also called costs) need to be retained and used for updating, starting from the 13th combination. Starting from the 13th combination, prediction is performed only for the upper region of the template, and the corresponding SAD is calculated. If the SAD calculated based on the upper template is greater than the highest difference corresponding to the combination among the 12 combinations with the lowest differences, then the prediction and difference calculation for the left region of the template can be skipped. For details, please refer to the above embodiment.

[0194] Step 3, according to the compiled TMRL mode candidate list and the TMRL mode index obtained through decoding, determine a combination of the extended reference line and the intra prediction mode selected for the current block, and perform intra prediction with respect to the current block according to the selected combination.

[0195] If TMRL mode is used, refIdx and predModeIntra determine the mode used for intra prediction, and they are determined according to the TMRL mode index "tmrl_idx" and the TMRL mode candidate list.

[0196] For the ECM-4.0 reference software, using the method described in the embodiment and the settings of N = 5 (the 5 extended reference lines are reference lines 1, 3, 5, 7, 12, respectively), M = 6 (6 specified prediction modes, not including the planar mode and the DC mode), and K = 12 (only 12 combinations with small SADs are selected from all combinations), the measurement results under the AI ​​configuration are as follows:

[0197] By setting N=5 (5 extended reference lines represent reference lines 1, 3, 5, 7, 12, respectively), M=8 (8 set prediction modes) and K=16, the measurement results under AI configuration are as follows:

[0198] By setting N=5 (5 extended reference lines represent reference lines 1, 3, 5, 7, 12, respectively), M=12 (12 set prediction modes) and K=24, the measurement results under AI configuration are as follows:

[0199] By setting N=5 (5 extended reference lines are reference lines 1, 3, 5, 7, 12, respectively), M=8 (8 specified prediction modes, not including planar mode and DC mode) and K=16, the measurement results under AI configuration are as follows:

[0200] In the table, the meanings of various parameters are as follows.

[0201] EncT: Encoding time 10X% means that when integrating the reference string sorting technology, the encoding time is 10X% compared with the encoding time before integration, which means the encoding time is increased by X%.

[0202] DecT: Decoding time 10X% means that when integrating the reference line sorting technology, the decoding time is 10X% compared with the decoding time before integration, which means the decoding time is increased by X%.

[0203] ClassA1 and ClassA2 are video test sequences with a resolution of 3840 x 2160, ClassB is a test sequence with a resolution of 1920 x 1080, ClassC is a test sequence with a resolution of 832 x 480, ClassD is a test sequence with a resolution of 416 x 240, and ClassE is a test sequence with a resolution of 1280 x 720; ClassF represents several sequences of screen content with different resolutions.

[0204] Y, U, and V represent the three color components. The columns containing Y, U, and V represent the Bjøntegaard-Delta speed (BD speed) of the test results for Y, U, and V. The lower the value, the better the encoding performance.

[0205] "All inside" is the test configuration of the "all inside" configuration.

[0206] It can be seen that using the TMRL mode for intra prediction encoding and decoding of this embodiment can significantly improve the encoding performance.

[0207] In this embodiment, a template area of ​​1 row and 1 column is used for extended reference lines or prediction modes, and the extended reference lines or prediction modes are sorted and filtered in ascending order of SAD. For extended reference lines, if all extended reference lines (including reference line 1) are sorted, only a template of 1 row and 1 column can be used. However, for prediction mode selection, similar to the TIMD mode, more reference lines can be used to obtain more accurate results in order to select more suitable prediction modes. In other embodiments, the method for determining prediction modes in the candidate list can also be changed to the TMRL mode.

[0208] For example, when it is necessary to construct a candidate list for a prediction mode of length 6, a list of length greater than 6 can first be constructed according to the same construction and filling methods as in the embodiment, and then 4 rows and 4 columns closest to the current block are used as a template, and prediction according to the template is performed using the 5th reference row and the intra prediction mode in the prediction mode candidate list, and the differences (SAD or SATD) between the prediction values ​​obtained by prediction and the reconstructed template value are calculated and sorted in ascending order, and 6 intra prediction modes with small differences are selected as intra prediction modes in the constructed candidate list of the TMRL mode of length 6. In addition, the length of the TMRL mode candidate list of 6 is only an example, and the length can be adjusted depending on the situation.

[0209] In this embodiment, 65 corner modes are used as an example, but in other embodiments, the corner modes can be expanded to 129 corner modes or more to achieve better performance. When the corner modes are expanded to more corner modes, the number of intra-prediction filters must also be increased accordingly. For example, with 129 corner modes, filtering with a precision of 1 / 64 is used.

[0210] In one embodiment, an intra-prediction fusion (IPF) technology is further proposed. IPF allows weighting the prediction results obtained by using the corner mode with two adjacent reference lines to obtain the final prediction result for the current block. The weighting method is as follows: . is the result obtained by performing a prediction on the current block using the reference row with index a (reference row a) and the corner mode, represents the result obtained by performing a prediction on the current block using the reference row with index a+1 (reference row a+1) and the corner mode, is the result of merger prediction, represents the weighting factor for , represents the weighting factor for , is 3 / 4 and is 1 / 4.

[0211] When IPF is used for the current block, the merge prediction result is taken as the final prediction result for the current block.

[0212] In the embodiment, a flag indicating whether the IPF is used or not (activated or not) is not set, but by default, the IPF is activated for the corner mode selected for the current block when the following conditions are met: the selected corner mode for the current block is a corner mode with a non-integer slope; the product of the width and height of the current block is greater than 16; and the ISP mode is not used for the current block.

[0213] In other words, IPF is not allowed to be used when at least one of the constraint conditions is satisfied: the corner mode selected for the current block is an integer slope corner mode; the product of the width and height of the current block is less than or equal to 16; or the ISP mode is used for the current block.

[0214] When the IntraPredAngle of an angle mode is 0 when divided by 32, the angle mode is an integer-inclination angle mode. The correspondence between IntraPredAngle and angle modes can be seen in Table 1 above.

[0215] When the current mode satisfies the IPF usage condition, the intra-prediction mode selected for the current block can be either DIMD fusion (fusion of planar mode and two angular modes) selected via DIMD, or TIMD fusion (selected via TIMD). From a hardware implementation perspective, it is better to minimize fusion. When DIMD fusion is selected, there are already three intra-prediction results for the current block that need to be fused. Using IPF additionally will cause excessive fusion, increasing the complexity of the prediction phase.

[0216] In one embodiment of the present invention, an IPF method is proposed. The method can be applied to an encoder and can also be applied to a decoder. As illustrated in Fig. 10, the method includes the following.

[0217] Step 410, whether one or more constraint conditions for using the IPF for the current block are satisfied or not is determined when one or more intra prediction modes selected for the current block include the corner mode.

[0218] Step 420, the use of IPF for intra prediction with respect to the current block is restricted when at least one of the restriction conditions is satisfied.

[0219] At this stage, the restriction on using IPF for intra-prediction with respect to the current block may be that IPF is not allowed, or it may be that in the case where the selected intra-prediction modes include multiple angular modes that satisfy the usage condition for IPF, IPF is allowed to be performed only on some angular modes. In one example, an angular mode that satisfies the usage condition for IPF refers to an angular mode with a non-integer slope or an angular mode with an angle that is neither -45°, 0°, 45°, 90°, nor 135°.

[0220] In the text, the case where IPF is not allowed when the constraint condition is satisfied is a sufficient condition for not using IPF to predict the current block. Conversely, the case where IPF is allowed when the constraint condition is not satisfied is a necessary condition for IPF to predict the current block. However, even if other constraint conditions are met, IPF may still not be allowed.

[0221] In one example embodiment of the present invention, the one or more constraint conditions include the following mode number constraint condition: using the IPF will result in the need to combine more than N intra-prediction modes for prediction with respect to the current block, where N is an integer greater than or equal to 3.

[0222] In one example embodiment, under the condition that N is 3, determining whether one or more restriction conditions on using the IPF for the current block are satisfied when one or more intra-prediction modes selected for the current block include a corner mode, and restricting the use of the IPF for intra-prediction with respect to the current block when at least one of the one or more restriction conditions is satisfied includes the following. In the case where the TIMD merge mode is selected for the current block and two corner modes satisfying the use condition for the IPF are used for TIMD merge, it is determined that the restriction condition on the number of modes is satisfied, and it is possible to perform the IPF for only one of the two corner modes to perform intra-prediction with respect to the current block.In the case where the TIMD merging mode is selected for the current block, and only one of the two intra-prediction modes used for the TIMD merging is a corner mode satisfying the usage condition for the IPF, it is determined that the mode number limitation condition is not satisfied, and an opportunity is provided to perform the IPF for the corner mode to perform intra-prediction with respect to the current block.

[0223] In one example embodiment of the present invention, the one or more constraint conditions include a mode number constraint condition: using the IPF will result in the need to combine more than M angular modes for prediction with respect to the current block, where M is an integer greater than or equal to 2.

[0224] In one example embodiment, under the condition that M is 3, determining whether one or more restriction conditions on using the IPF for the current block are satisfied when one or more intra-prediction modes selected for the current block include a corner mode, and restricting the use of the IPF for intra-prediction with respect to the current block when at least one of the one or more restriction conditions is satisfied includes the following. In the case where the TIMD merge mode is selected for the current block and two corner modes satisfying the use condition for the IPF are used for TIMD merge, it is determined that the restriction condition on the number of modes is satisfied, and it is possible to perform the IPF for only one of the two corner modes to perform intra-prediction with respect to the current block.For example, the IPF is limited to the corner mode with the lowest cost or the second-lowest cost across two corner modes. If the TIMD fusion mode is selected for the current block, and only one of the two intra-prediction modes used for TIMD fusion is a corner mode that satisfies the IPF usage condition, it is determined that the mode limitation condition is not satisfied, and the IPF is limited to the corner mode for performing intra-prediction on the current block.

[0225] In one example embodiment, under the condition that M is 3, determining whether one or more restriction conditions on using the IPF for the current block are satisfied when one or more intra-prediction modes selected for the current block include a corner mode, and restricting the use of the IPF for intra-prediction with respect to the current block when at least one of the one or more restriction conditions is satisfied includes the following. In the case where the DIMD merge mode is selected for the current block and two corner modes satisfying the use condition for the IPF are used for DIMD merge, it is determined that the restriction condition on the number of modes is satisfied, and it is possible to perform the IPF for only one of the two corner modes to perform intra-prediction with respect to the current block.For example, the IPF is limited to executing only the corner mode with the highest cost or the second-highest cost in two corner modes. If the DIMD merging mode is selected for the current block and only one of the two intra-prediction modes used for DIMD merging is a corner mode that satisfies the IPF usage condition, it is determined that the mode limitation condition is not satisfied, and the IPF is limited to executing the corner mode that satisfies the IPF usage condition to perform intra-prediction on the current block.

[0226] In one example embodiment, under the condition that M is 2, determining whether one or more restriction conditions on using IPF for the current block are satisfied when one or more intra-prediction modes selected for the current block include a corner mode, and restricting the use of IPF for intra-prediction with respect to the current block when at least one of the one or more restriction conditions is satisfied includes the following. In the case where the TIMD merging mode is selected for the current block and two corner modes satisfying the use condition for IPF are used for TIMD merging, it is determined that the restriction condition on the number of modes is satisfied, and the possibility of executing IPF for intra-prediction with respect to the current block is rejected.In the case where the TIMD merging mode is selected for the current block and only one of the two intra-prediction modes used for TIMD merging is a corner mode satisfying the usage condition for IPF, it is determined that the limitation condition for the number of modes is not satisfied, and an opportunity is provided to perform IPF for the corner mode satisfying the usage condition of IPF to perform intra-prediction with respect to the current block.

[0227] In the above embodiments, the capability of executing IPF for the corner mode for performing intra-prediction with respect to the current block includes the following. A weighted sum of two or more prediction results is taken as the final prediction result of the current block, where the two or more prediction results comprise: a prediction result obtained by performing prediction with respect to the current block according to the first reference row selected for the current block and the corner mode, and a prediction result obtained by performing prediction with respect to the current block according to a second reference row different from the first reference row and the corner mode. For example, the second reference row may be a reference row adjacent to the first reference row or a reference row with index 0.

[0228] In an exemplary embodiment of the present invention, one or more constraint conditions include any one or more of the following. Constraint Condition 1: The TIMD merge mode is selected for the current block. Constraint Condition 2: The DIMD merge mode is selected for the current block. Constraint Condition 3: The MRL is used for the current block. Constraint Condition 4: The index of the reference row selected for the current block is greater than or equal to K, where K is an integer greater than or equal to 3. Constraint Condition 5: The width of the current block is less than or equal to the set value. Constraint Condition 5: The height of the current block is less than or equal to the set value. Constraint Condition 7: The product of the width and height of the current block is less than or equal to the set value. Constraint Condition 8: The ISP mode is used for the current block.Constraint Condition 9: The angle of the angular mode selected for the current block is -45°, 0°, 45°, 90°, or 135°, or the angular mode selected for the current block is an angular mode with integer bias. Constraint Condition 10: The current frame to which the current block belongs is an interframe. Constraint Condition 11: The current image to which the current block belongs is a chroma image, i.e., the IPF is used only for the luma image and is not used for the chroma image. IPF is not allowed to be performed for intra-prediction on the current block when at least one of constraint conditions 1–11 is satisfied.

[0229] In an exemplary embodiment, one or more constraint conditions include constraint condition 4, where K is 3 or 5 or 7 or 12.

[0230] In the above embodiments of the present invention, the use of IPF is limited, for example, the number of intra-prediction modes used for fusion is limited, the number of corner modes used for fusion is limited, and the use of IPF together with prediction modes in which multiple modes are fused, such as the TIMD mode and the DIMD mode, is limited, so that excessive fusion during prediction can be avoided, and excessive fusion may lead to an inappropriate increase in the complexity of the prediction phase.

[0231] In one embodiment, the current block size may also be limited, and IPF is used only when the current block size exceeds a certain threshold. This is because when the current block size is smaller than the threshold, the current block typically contains more textures, limiting the performance improvement achieved through merge prediction. In one embodiment, IPF is not allowed when the index of the reference row selected for the current block is greater than or equal to K. That is, IPF is not allowed when the extended reference row selected for the current block is relatively far from the current block, since in this case, the performance improvement achieved through IPF is limited.

[0232] In this embodiment, the computational complexity can be reduced by limiting the simultaneous use of IPF and MRL. In this embodiment, IPF is not allowed when the current frame is interframe (e.g., B-frame or P-frame), which helps reduce the cost of interframe coding.

[0233] In an exemplary embodiment of the present invention, the method further includes the following. Intra-prediction is performed with respect to the current block using the IPF when none of one or more constraint conditions are satisfied. Intra-prediction is performed with respect to the current block using the IPF as follows. The weighted sum of the first prediction result and the second prediction result is taken as the final prediction result of the current block when only one intra-prediction mode is selected for the current block, and the intra-prediction mode is the corner mode.The first prediction result is obtained by performing a prediction with respect to the current block in accordance with the first reference line selected for the current block and the first angular mode selected for the current block, and the second prediction result is obtained by performing a prediction with respect to the current block in accordance with the second reference line and the first angular mode, wherein the second reference line is an adjacent line for the first reference line or a reference line with index 0.

[0234] It should be noted that using IPF for intra-prediction on the current block when none of the above constraint conditions are satisfied does not imply that IPF is permitted only when all constraint conditions are not satisfied. For example, when the constraint on the number of modes is satisfied, IPF can still be performed for a specific corner mode that satisfies the IPF usage condition.

[0235] In one embodiment, the weighted sum of the first prediction result and the second prediction result is calculated according to the following formula: (formula 1) or (formula 2). - the first result of the prediction, - the second result of the prediction, – the final prediction result for the current block, – weighting factor for , And – weighting factor for . + = (1 << shift), offset=1 << (shift - 1), shift >= 1, And are established parameters.

[0236] By using the above calculation algorithms, the decimal fraction can be avoided, thereby improving the calculation efficiency.

[0237] In one exemplary embodiment of the present invention, the method further comprises performing intra-prediction on the current block using IPF when none of the constraint conditions are satisfied. Performing intra-prediction on the current block using IPF comprises the following. A weighted sum of three or more prediction results is taken as the final prediction result of the current block when the intra-prediction mode selected for the current block is the fusion mode.Three or more prediction results include: several prediction results obtained by performing a prediction with respect to the current block according to the first reference line selected for the current block and each intra prediction mode in the merging mode, and one or more prediction results obtained by performing a prediction with respect to the current block according to the second reference line and one or more corner modes for which the possibility of performing IPF in the merging mode is provided, where the second reference line is a neighboring line for the first reference line or the reference line with index 0.

[0238] In the above embodiments of the present invention, the weight assigned to the first prediction result is greater than the weight assigned to the second prediction result, for calculating the weighted sum of the first prediction result and the second prediction result. For example, when the first reference row is the reference row with index 1 and the second reference row is the reference row with index 0, the first prediction result is assigned a weight of 3 / 4, and the second prediction result is assigned a weight of 1 / 4.

[0239] In the above embodiments of the present invention, the second reference row is adjacent to the first reference row and is closer to the current block than the first reference row when the index of the first reference row is greater than or equal to K, where K is an integer greater than or equal to 1.

[0240] For example, the second reference row may be determined in at least one of the following ways: when the index of the first reference row is 1, it is determined that the index of the second reference row is 0. When the index of the first reference row is 3, it is determined that the index of the second reference row is 2. When the index of the first reference row is 5, it is determined that the index of the second reference row is 4. When the index of the first reference row is 7, it is determined that the index of the second reference row is 6. When the index of the first reference row is 12, it is determined that the index of the second reference row is 11.

[0241] When the index of the first reference row is greater than or equal to K, the reference row adjacent to the first reference row and located closer to the current block than the first reference row is set as the second reference row, so that the prediction accuracy with respect to the current block using the second reference row can be increased, thereby improving the accuracy of the prediction result obtained by merging.

[0242] In one exemplary embodiment of the present invention, the IPF can be used together with the TMRL mode. That is, in the case where an extended reference row and a corner mode in the candidate list of the TMRL mode are selected for the current block, and none of one or more constraint conditions are satisfied, a first prediction result is calculated by performing a prediction with respect to the current block according to the extended reference row and the corner mode, a second prediction result is calculated by performing a prediction with respect to the current block according to another reference row and the corner mode, and a weighted sum of the first prediction result and the second prediction result is taken as the final prediction result of the current block, where the other reference row is the reference row with index 0 or an adjacent row of the extended reference row.

[0243] In one embodiment of the present invention, a video encoding method is further provided. The method is applied to an encoder and includes the following, as illustrated in Fig. 11.

[0244] Step 510, the reference line and the intra prediction mode selected for the current block are determined by mode selection.

[0245] Step 520, the prediction value of the current block is obtained by performing prediction on the current block according to the IPF method described in any embodiment of the present invention. In this step, in the case where intra-prediction on the current block is performed using IPF, the prediction value of the current block is obtained in accordance with the final prediction result of the current block.

[0246] Step 530, the remainder of the current block is determined according to the original value of the current block and the prediction value of the current block.

[0247] In the video encoding method of the present invention, prediction with respect to the current block is performed using the IPF method of any embodiment of the present invention, and thus various effects achieved by using the IPF can be achieved.

[0248] In one embodiment of the present invention, a video decoding method is further provided. The method is applied to a decoder and includes the following, as illustrated in Fig. 12.

[0249] Step 610, the reference line and the intra prediction mode selected for the current block are determined by decoding the bitstream.

[0250] Step 620, the prediction value of the current block is obtained by performing intra-prediction on the current block according to the IPF method described in any embodiment of the present invention. In this step, if intra-prediction on the current block is performed using IPF, the prediction value of the current block is obtained according to the final prediction result of the current block.

[0251] Step 630, the recovered value of the current block is determined according to the prediction value of the current block.

[0252] In the video encoding method of the present invention, prediction with respect to the current block is performed using the IPF method of any embodiment of the present invention, and thus various effects achieved by using the IPF can be achieved.

[0253] The intra-prediction method based on the TMRL mode in the above-described embodiments is a prediction mode in which a candidate list is constructed according to combinations of extended reference lines and intra-prediction modes, and the combination of the extended reference line and the intra-prediction mode is decoded and encoded. However, for the template-based or sort-based method, a single reference line is used for prediction. A single reference line typically contains noise, which can affect the prediction accuracy.

[0254] Therefore, in one embodiment of the present invention, a method for using IPF based on the TMRL mode is provided. IPF is used based on the TMRL mode, that is, IPF is performed with respect to a selected angular mode. In other words, the final prediction result of the current block is obtained by weighting the result obtained by performing prediction on the current block according to the selected reference row and angular mode and the result obtained by performing prediction on the current block according to another reference row and angular mode. The prediction results obtained using different reference rows are merged, which improves the prediction accuracy.

[0255] IPF is used based on the TMRL mode, so there is no need to set a flag indicating whether IPF is used for the current block, and the encoder and decoder can make the determination according to the agreed conditions.

[0256] In one embodiment of the present invention, the IPF can be used together with the TMRL mode. That is, in the case where an extended reference row and a corner mode in the TMRL mode candidate list are selected for the current block, and none of one or more constraint conditions are satisfied, a first prediction result is calculated by performing a prediction with respect to the current block according to the extended reference row and the corner mode, a second prediction result is calculated by performing a prediction with respect to the current block according to another reference row and the corner mode, and a weighted sum of the first prediction result and the second prediction result is taken as the final prediction result of the current block, where the other reference row is the reference row with index 0 or a neighboring row for the extended reference row.

[0257] In the embodiment, the method in the above-mentioned embodiments can be used to generate a TMRL mode candidate list and perform encoding and decoding. The difference is that in the embodiment, to generate a prediction value for the current block, the prediction value generated according to the selected corner mode and the selected reference row is weighted against the prediction value generated according to the selected corner mode and another reference row to obtain the final prediction value of the current block.

[0258] In an embodiment, there is an IPF restriction based on the TMRL mode. For example, IPF is performed only when the selected reference line is close to the current block. In particular, merging is allowed only if the reference line is reference line 1; or reference line 1 or 3; or reference line 1, 3, or 5; or reference line 1, 3, 5, or 7; or reference line 1, 3, 5, 7, or 12. In addition, the use of IPF may also be restricted when certain angle modes are used for the current block, for example, IPF is not allowed when the angle of the angle mode is a horizontal angle, a vertical angle, -45°, 45°, or 135°. Moreover, if the size of the current block is less than or equal to a certain value, IPF may also be invalid, since smaller blocks usually contain more textures.

[0259] For TMRL mode-based IPF, a flag may be added to the TMRL mode syntax elements to indicate whether IPF is used or not.

[0260] In one embodiment of the present invention, a video decoding method is further provided. The method is applied to a decoder and includes the following, as illustrated in Fig. 13.

[0261] Step 710, the TMRL mode index and the TMRL merge flag for the current block are further decoded when the TMRL mode is determined to be used for the current block through decoding.

[0262] Step 720: A TMRL mode candidate list is generated for the current block, and the extended reference line and intra-prediction mode selected for the current block are determined according to the candidate list and the TMRL mode index. In this step, the method in the above embodiments can be used to generate the candidate list and determine the extended reference line and intra-prediction mode selected for the current block.

[0263] Step 730. When the TMRL merge flag indicates the use of IPF, the weighted sum of the first prediction result and the second prediction result is taken as the final prediction result of the current block. In this step, the weighted sum of the first prediction result and the second prediction result can be obtained using Formula 1 and Formula 2.

[0264] The first prediction result is obtained by performing prediction on the current block according to an extended reference row and the intra-prediction mode. The second prediction result is obtained by performing prediction on the current block according to another reference row and the intra-prediction mode. The intra-prediction mode is the corner mode. The weighting of the two prediction results can be performed using the formula in the above embodiments.

[0265] In the embodiment, when the TMRL merge flag indicates not using the IPF, the final prediction result of the current block is obtained by performing a prediction on the current block in accordance with the extended reference line and the intra prediction mode.

[0266] In the embodiment, based on the TMRL mode, the TMRL merge flag is set to indicate the use of IPF. On the encoding side, the TMRL merge flag can be encoded according to whether one or more restriction conditions that are the same or similar to those presented in the above embodiments are satisfied. If it is determined that IPF will be used, the value of the TMRL merge flag is set to 1. If it is determined that IPF will not be used, the value of the TMRL merge flag is set to 0. On the decoding side, the decoding side does not need to evaluate these restriction conditions again, and it can determine, according to the TMRM merge flag, whether IPF is performed in the corner mode selected in the case of using the TMRL mode. This approach simplifies the processing at the decoding stage and increases the flexibility of setting the use condition for IPF.In addition, using IPF based on TMRL mode can improve the prediction accuracy and enhance the video coding performance.

[0267] In one example of an embodiment, the other reference row is a reference row adjacent to the extended reference row and located on the inner side of the extended reference row, or a reference row adjacent to the extended reference row and located on the outer side of the extended reference row, or a reference row with index 0. The inner side refers to the side close to the current block, and the outer side refers to the side distant from the current block.

[0268] In one exemplary embodiment, a condition for decoding the TMRL merge flag may be added. That is, further decoding of the TMRL mode index and the TMRL merge flag for the current block includes the following. Whether the size of the current block satisfies the set condition is determined after decoding the TMRL mode index for the current block, the TMRL merge flag for the current block is decoded when the set condition is satisfied, and decoding of the TMRL merge flag for the current block is skipped when the set condition is not satisfied.

[0269] The set condition includes any one or more of the following conditions: the width of the current block is greater than the set value; the height of the current block is greater than the set value; the product of the width and height of the current block is greater than the set value; or the current frame to which the current block belongs is not an outer frame, that is, the ability to perform merge mode for inner blocks in B-frames and P-frames is not provided.

[0270] Because smaller blocks typically contain more texture, the improvement achieved by IPF is limited. Therefore, in this embodiment, the condition that the current block size is larger than a set value is used to determine whether to decode the TMRL merge flag. When the current block size is not larger than the set value, the TMRL merge flag cannot be encoded or decoded to improve coding efficiency.

[0271] In one example of the present invention, a candidate list of the TMRL mode for the current block is constituted as follows. N × M combinations of extended reference lines and intra-prediction modes are constituted according to the N extended reference lines and M intra-prediction modes for the current block, where N≥ 1, M≥ 1, and N×M≥ 2. Prediction with respect to the template region for the current block is performed according to each of the N × M combinations, and differences between the reconstructed value of the template region and the prediction values ​​obtained by the prediction are calculated. K combinations corresponding to the differences are entered into the candidate list of the TMRL mode for the current block in ascending order of the differences, where 1 ≤ K ≤ N × M. The M intra-prediction modes are selected from angular modes other than angular modes with angles of -45°, 0°, 45°, 90°, or 135°.

[0272] In this embodiment, to generate a candidate list of the TMRL mode, the M intra-prediction modes participating in the combination are limited to angle modes other than certain angle modes with specific angles. This approach facilitates the combination of the TMRL mode and the IPF mode to improve prediction accuracy.

[0273] In one embodiment, a video decoding method is further provided. To parse CU-level syntax elements, it is necessary to additionally decode tmrl_fusion_flag, which is the TMRL fusion flag.

[0274] An example is illustrated in the following table.

[0275] sps_mrl_enabled_flag in the table above is a sequence flag, where sps_mrl_enabled_flag equal to 1 indicates that MRL can be used for the current sequence, and sps_mrl_enabled_flag equal to 1 indicates that MRL is not used for the current sequence. (y0 % CtbSizeY) > 0 indicates that the CU is not located in the first row of the CTU. tmrl_fusion_flag equal to 1 indicates that the fusion mode will be used for subsequent TMRL modes; otherwise, the fusion mode will not be used.

[0276] In another embodiment, whether or not to decode tmrl_fusion_flag may further depend on the size of the current coding block. For example, as shown in the following table, when the coding block satisfies the conditions cbWidth > N and cbHeight > M, tmrl_fusion_flag is decoded.

[0277] In another embodiment, for example, as shown in the following table, when the coding block satisfies cbWidth *cbHeight > L, tmrl_fusion_flag is decoded.

[0278] M, N, and L in the table represent the specified size values ​​and are all positive integers. M can be equal to N.

[0279] In an embodiment, to construct a candidate list of the TMRL mode according to the template-based sorting, an exhaustive traversal of N×M combinations of N available reference lines and M available intra-prediction modes is performed. Prediction values ​​for the template region are generated, and the differences between the prediction values ​​and the reconstructed value of the template region are calculated. The top K combinations are selected in ascending order of the differences, forming a candidate list of {reference line, prediction mode} combinations. The difference can be, for example, the sum of absolute differences (SAD). An embodiment can limit the use of IPF to only certain modes. To construct the candidate list, one or more of the following intra-prediction modes can be excluded: planar mode, DC mode, angular mode with a horizontal angle, angular mode with a vertical angle, angular mode with an angle of -45°, angular mode with an angle of 45 degrees, and angular mode with an angle of 135 degrees.

[0280] The fusion in the prediction stage should be based on the value of tmrl_fusion_flag. When the value of tmrl_fusion_flag is 1, the selected intra prediction mode and the selected reference row generate the prediction signal p. a , the selected intra prediction mode and reference line 0 generate a prediction signal p b , and the merger of p a and p b generates the final prediction signal. Otherwise, p a is directly used as the final prediction signal.

[0281] Accordingly, embodiments of the present invention further provide a video encoding method. The method is applied to an encoder and includes the following, as illustrated in Fig. 18.

[0282] Step 1110, a candidate list of the TMRL mode for the current block is formed, wherein the candidate list is filled with combinations of candidate extended reference lines and candidate intra prediction modes for the current block.

[0283] Step 1120, the combination of the reference line and the intra prediction mode for the current block is selected through rate-distortion optimization.

[0284] Step 1130, when the coding condition for the TMRL mode for the current block is satisfied, the TMRL mode flag for the current block is coded to indicate the use of the TMRL mode for the current block, and the TMRL mode index for the current block is coded to indicate the position of the selected combination in the candidate list.

[0285] Step 1140, the TMRL merge flag for the current block is encoded to indicate the use of IPF for the current block or to indicate the non-use of IPF for the current block. The encoding condition at least includes the presence of the selected combination in the candidate list.

[0286] The portion of the video encoding method in the embodiment that is not related to merging may be the same as in the previously described embodiment of the video encoding method using TMRL without merging. For example, the setting of the encoding condition in step 1130 may be the same, and so on.

[0287] In one example embodiment, encoding the TMRL merge flag for the current block to indicate the use of IPF for the current block or the non-use of IPF for the current block includes the following. In the case where a combination containing a corner mode in the candidate list is selected for the current block, encoding the TMRL merge flag for the current block to indicate the non-use of IPF when at least one of one or more specified constraint conditions is satisfied, and encoding the TMRL merge flag for the current block to indicate the use of IPF when none of one or more specified constraint conditions is satisfied.

[0288] The set one or more constraint conditions further includes any one or more of the following constraint conditions: the TIMD merge mode is selected for the current block; the DIMD mode is selected for the current block; the index of the reference line selected for the current block is greater than or equal to K, where K is an integer greater than or equal to 3; the width of the current block is less than or equal to the set value; the height of the current block is less than or equal to the set value; the product of the width and height of the current block is less than or equal to the set value; the ISP mode is used for the current block; the angle of the angular mode selected for the current block is -45°, 0°, 45°, 90°, or 135°, or the angular mode selected for the current block is an angular mode with an integer slope; the current frame to which the current block belongs is an intermediate frame; or the current picture to which the current block belongs is a chroma picture.

[0289] In one example of an embodiment, to generate a candidate list of the TMRL mode for the current block, the intra prediction modes to participate in the combination do not include angular modes with angles of -45°, 0°, 45°, 90°, or 135°.

[0290] On the encoder side, when a combination in the TMRL mode candidate list is selected for the current block and an IPF is used for the current block, prediction for the current block is performed based on this combination using the IPF. The prediction results obtained from the combination and those obtained using the corner mode in the combination and another reference row are weighted to obtain a prediction value for the current block, so that the reconstructed value of the current block can be determined.

[0291] In order to use TMRL-based IPF, the merge prediction mode obtained from the combination of the extended reference row and the corner mode can also be included as a combination in the candidate list to generate the TMRL mode candidate list.

[0292] In one embodiment of the present invention, a method for constructing a candidate list of the MRL mode is provided. The method can be applied to an encoder and can also be applied to a decoder. As illustrated in Fig. 14, the method includes the following.

[0293] Step 810, N × M original combinations of extended reference lines and corner modes are obtained according to N extended reference lines and M intra prediction modes for the current block.

[0294] Step 820, a prediction regarding a template region for the current block is performed according to each of the N × M original combinations, and differences between the reconstructed value of the template region and the prediction values ​​obtained through the prediction are calculated.

[0295] Step 830, for each of the source combinations containing predetermined corner modes in the K source combinations with the lowest differences, it is determined whether merging is necessary for the source combination, the merging combination corresponding to the source combination is entered into a candidate list of the TMRL mode if merging is necessary for the source combination, and the source combination is entered into the candidate list if merging is not necessary for the source combination, wherein K, N and M are set positive integers, and 1 ≤ K ≤ N × M.

[0296] In an embodiment, for each of the source combinations containing predetermined corner modes in K source combinations with the smallest differences, determining whether merging is necessary for the source combination and adding the merging combination corresponding to the source combination to the TMRL mode candidate list when merging is necessary for the source combination includes the following. Prediction with respect to the template region for the current block is performed according to the merging combination corresponding to the source combination, calculating the difference between the reconstructed value of the template region and the prediction value obtained by the prediction, and determining that merging is necessary when the difference corresponding to the source combination is greater than the difference corresponding to the merging combination, and determining that merging is not necessary when the difference corresponding to the source combination is less than or equal to the difference corresponding to the merging combination.In order to perform a prediction with respect to the current block according to the merging combination, the weighted sum of the result of the first prediction and the result of the second prediction is taken as the prediction value of the current block, wherein the result of the first prediction is obtained by performing a prediction with respect to the current block according to the extended reference row and the angular mode in the original combination, the result of the second prediction is obtained by performing a prediction with respect to the current block according to the second reference row and the angular mode in the original combination, and the second reference row is an adjacent row for the first reference row or a reference row with index 0. The predetermined angular modes contain all angular modes or contain angular modes other than angular modes with an integer slope.

[0297] In the embodiment, the weighted sum of the first prediction result and the second prediction result may be calculated using formula 1 or formula 2.

[0298] In an embodiment, K combinations are selected for inclusion in the candidate list by sorting the non-merging modes. Then, it is further determined whether merging of the reference line in each of the K combinations is necessary. The determination method is based on the difference between the prediction signal generated in the case of the merging mode for the reference line in the currently selected combination and the reconstructed signal, and the difference between the prediction signal generated in the case of the non-merging mode for the reference line in the currently selected combination and the reconstructed signal. If the difference in the case of the merging mode is smaller, merging is necessary for the combination; otherwise, merging is not necessary for the combination. In this embodiment, it is only necessary to predict the template region according to 5 x M + K combinations and calculate the differences for the 5 x M + K combinations.

[0299] In another embodiment of the present invention, a method for constructing an MRL mode candidate list is further provided. The method can be applied to an encoder and can also be applied to a decoder. As illustrated in Fig. 15, the method includes the following.

[0300] Step 910, N × M original combinations of extended reference lines and intra prediction modes are obtained according to N extended reference lines and M intra prediction modes for the current block.

[0301] Step 920, merging processing is performed with respect to N × M source combinations, wherein the merging processing includes: for each of the source combinations containing predetermined corner modes, replacing the source combination with a corresponding merging combination when a set condition is satisfied.

[0302] Step 930, prediction regarding the template region for the current block is performed according to each of the N × M combinations obtained after the fusion processing, and differences between the reconstructed value of the template region and the prediction values ​​obtained through the prediction are calculated.

[0303] Step 940, K combinations corresponding to the differences are entered into the TMRL mode candidate list for the current block in ascending order of the differences, where K, N and M are set positive integers, and 1 ≤ K ≤ N × M. For each original combination, performing a prediction with respect to the current block according to the merging combination corresponding to the original combination includes taking a weighted sum of the first prediction result and the second prediction result as a prediction value of the current block, wherein the first prediction result is obtained by performing a prediction with respect to the current block according to the extended reference row and the corner mode in the original combination, the second prediction result is obtained by performing a prediction with respect to the current block according to the second reference row and the corner mode in the original combination, and the second reference row is an adjacent row for the first reference row or a reference row with index 0.

[0304] In one example embodiment, the established condition includes any one or more of the following conditions: the size of the current block is greater than N × M, where N and M are established positive integers; and the intra prediction mode selected for the current block is a corner mode with a non-integer bias, wherein the predetermined corner modes contain all corner modes or contain corner modes other than corner modes with an integer bias.

[0305] In the embodiment, the weighted sum of the first prediction result and the second prediction result may be calculated using formula 1 or formula 2.

[0306] In this embodiment, it is not necessary to sort the merging modes for the reference rows and the non-merging modes for the reference rows by sorting the difference costs for the template. When one or more of the following conditions are satisfied, the merging modes for the reference rows are used to participate in the sorting for the template and generate a list of combinations of {reference row, prediction mode}; otherwise, the merging mode is not used. The following conditions include: the size of the current block is greater than N x M; the current intra-prediction mode is not a planar mode, a DC mode, an angular mode with a horizontal angle, an angular mode with a vertical angle, an angular mode with an angle of -45 degrees, an angular mode with an angle of 45 degrees, or an angular mode with an angle of 135 degrees.

[0307] In the prediction stage, when TMRL is used for the current block, the selected reference row and prediction mode are determined according to tmrl_idex obtained through decoding and the candidate list generated according to the template-based sorting. Whether to use the adjacent reference row to generate the fusion prediction result is determined according to the same conditions. That is, IPF is performed with respect to the selected angular mode when one or more of the following conditions are satisfied: the size of the current block is greater than N x M; the current intra prediction mode is not the planar mode, DC mode, angular mode with a horizontal angle, angular mode with a vertical angle, angular mode with an angle of -45 degrees, angular mode with an angle of 45 degrees, or angular mode with an angle of 135 degrees.

[0308] In one embodiment of the present invention, a method for generating MRL mode candidate lists is further provided. The method can be applied to an encoder and can also be applied to a decoder. As illustrated in Fig. 16, the method includes the following.

[0309] Step 1010, N × M original combinations of extended reference lines and corner modes are obtained according to N extended reference lines and M intra prediction modes for the current block.

[0310] Step 1020, corresponding fusion combinations are obtained according to original combinations containing predetermined corner modes in N × M original combinations.

[0311] Step 1030, prediction regarding the template region for the current block is performed according to each of the N × M original combinations and each of the obtained fusion combinations, and differences between the reconstructed value of the template region and the prediction values ​​obtained through the prediction are calculated.

[0312] Step 1040, K combinations corresponding to the differences are entered into the TMRL mode candidate list for the current block in ascending order of the differences, where K, N, and M are set positive integers, and 1 ≤ K ≤ N × M.

[0313] In an embodiment, obtaining corresponding fusion combinations according to source combinations containing predetermined angular modes in N × M source combinations includes the following. For each of the source combinations containing predetermined angular modes, a fusion combination is obtained according to the source combination.In order to perform a prediction with respect to the current block according to the merging combination, the weighted sum of the result of the first prediction and the result of the second prediction is taken as the prediction value of the current block, wherein the result of the first prediction is obtained by performing a prediction with respect to the current block according to the extended reference row and the angular mode in the original combination, the result of the second prediction is obtained by performing a prediction with respect to the current block according to the second reference row and the angular mode in the original combination, and the second reference row is an adjacent row for the first reference row or a reference row with index 0. The predetermined angular modes contain all angular modes or contain angular modes other than angular modes with an integer slope.

[0314] In an embodiment where all the intra prediction modes to participate in a combination are corner modes, it is necessary to predict the template region according to the 5 x M x 2 combinations and calculate the differences for the 5 x M x 2 combinations.

[0315] In the embodiment, the weighted sum of the first prediction result and the second prediction result may be calculated using formula 1 or formula 2.

[0316] In the above embodiments of the present invention, taking the reference row with indexes {1, 3, 5, 7, 12} as the reference rows to participate in the combination as an example, the fusion combination corresponding to the original combination may be, for example, as follows. When the original combination consists of the reference row with index 1 and an angle mode, the corresponding fusion combination may be a combination of the reference rows with indices 1 and 2 and the angle mode. To perform a prediction on the current block based on the fusion combination, the prediction result obtained according to the reference row with index 1 and the angle mode and the prediction result obtained according to the reference row with index 2 and the angle mode are weighted together to obtain the final prediction result of the current block.In another example, if an adjacent reference row located on the inside is selected as the second reference row, the corresponding merge combination may be a combination of reference rows with indices 0 and 1 and a corner mode.

[0317] When the TMRL mode candidate list construction method in the above embodiments is applied to video encoding and decoding, if a fusion combination in the candidate list is selected for the current block, fusion prediction is performed on the current block according to the fusion combination prediction method. If an original combination in the candidate list is selected for the current block, prediction is performed on the current block according to the prediction method for the original combination.

[0318] In one embodiment of the present invention, a bitstream is further provided, wherein the bitstream is generated by a video encoding method according to any embodiment of the present invention.

[0319] In one embodiment of the present invention, an apparatus for generating an MRL candidate list is further provided. As illustrated in Fig. 17, the apparatus comprises a processor 71 and a memory 73 in which a computer program is stored, wherein the processor 71, when executing the computer program, can implement the method for generating an MRL candidate list as described in any embodiment of the present invention.

[0320] In one embodiment of the present invention, an IPF apparatus is further provided. The apparatus comprises a processor and a memory device on which a computer program is stored, wherein the processor, when executing the computer program, can implement the IPF method according to any embodiment of the present invention.

[0321] In one embodiment of the present invention, a video decoding apparatus is further provided. The apparatus comprises a processor and a memory device in which a computer program is stored, wherein the processor, when executing the computer program, can implement the video decoding method as described in any embodiment of the present invention.

[0322] In one embodiment of the present invention, a video encoding apparatus is further provided. The apparatus comprises a processor and a memory device on which a computer program is stored, wherein the processor, when executing the computer program, can implement the video encoding method as described in any embodiment of the present invention.

[0323] In one embodiment of the present invention, a video encoding and decoding system is further provided, which comprises a video encoding apparatus described in any embodiment of the present invention and a video decoding apparatus described in any embodiment of the present invention.

[0324] In one embodiment of the present invention, a non-transitory computer-readable storage medium is further provided, wherein a computer program is stored on the computer-readable storage medium. The computer program, when executed by the processor, implements the IPF method as described in any embodiment of the present invention, the method for generating an MRL candidate list as described in any embodiment of the present invention, or implements the video decoding method as described in any embodiment of the present invention, or implements the video encoding method as described in any embodiment of the present invention.

[0325] The processor according to the above-mentioned embodiments of the present invention may be a general-purpose processor, which includes a central processing unit (CPU), a network processor (abbreviated NP), a microprocessor, etc., or other conventional processors, etc. The processor may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), discrete logic devices or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or other equivalent integrated or discrete logic circuits, or a combination of the above devices.That is, the processor of the above-mentioned embodiments may be any processing device or combination of devices that can implement the various methods, steps, and logical flow charts disclosed in the embodiments of the present invention. If the embodiments of the present invention are partially implemented in software, the instructions for the software may be stored on a suitable non-volatile computer-readable storage medium, and one or more hardware processors may be used to execute the instructions for implementing the methods of the embodiments of the present invention. The term "processor" as used herein may refer to the above-mentioned structure or any other structure suitable for implementing the technology described herein.

[0326] In one or more of the above-described exemplary embodiments, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions in the form of one or more instructions or codes may be stored on or transmitted via a computer-readable medium and executed by a hardware processor. A computer-readable medium may be a tangible medium, such as a storage medium, or any communication medium that facilitates the transfer of a computer program from one place to another, for example, in accordance with a communication protocol. Thus, a computer-readable medium may typically be a non-transitory tangible computer-readable storage medium or a communication medium, such as a signal or a carrier wave.A data storage medium may be any accessible medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in the present invention. A computer program product may comprise a machine-readable medium.

[0327] By way of example, but not limitation, such computer-readable storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disc ROM (CD-ROM), or other optical disk storage devices, magnetic disk storage devices, or other magnetic data storage devices, flash memory, or any other medium that can be used to store desired program codes in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection may also be called a computer-readable medium.For example, if instructions are transmitted from a website, server, or other remote source via coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technology such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair cable, DSL, or wireless technology such as infrared, radio, and microwave are included in the definition of storage medium. However, it should be understood that machine-readable storage media and data storage media do not include a connection carrying a signal, signals, or other transient media, but refer to a non-transitory tangible machine-readable storage medium. As used herein, disks and optical disks include compact discs, laser optical discs, optical discs, digital versatile discs (DVDs), floppy disks, or Blu-ray discs, etc., while disks typically reproduce data magnetically, optical disks use lasers to optically reproduce data. Combinations of the above would also be included in the scope of machine-readable media.

[0328] In some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding or included in a combined codec. Also, the methods may be implemented entirely within one or more circuits or logic elements.

[0329] The technical solutions of the embodiments of the present invention can be implemented in a wide variety of devices or equipment, including wireless mobile phones, integrated circuits (ICs), or a group of ICs (e.g., chipsets). Various components, modules, or blocks are described in the embodiments of the present invention to highlight the functional aspects of devices configured to perform the described methods, but they do not necessarily have to be implemented by different hardware blocks. Instead, as described above, in the encoder and decoder, the various blocks can be combined into a hardware block or provided by a set of functionally compatible hardware blocks (including one or more processors, as described above) in combination with suitable software and / or firmware.

Claims

1. An intra prediction fusion (IPF) method, wherein the method is applied to a decoder and comprises: determining whether a constraint condition for using the IPF for the current block is satisfied when the intra prediction mode selected for the current block includes the corner mode; and Performing intra-prediction on the current block using IPF when the constraint condition is not satisfied.

2. The method according to paragraph 1, characterized in that The constraint condition includes a mode limitation condition: the use of IPF, which results in the need to merge more than M angular modes for prediction with respect to the current block, where M is an integer greater than or equal to 2.

3. The method according to paragraph 2, characterized in that provided that M is 2, determining whether a constraint condition for using the IPF for the current block is satisfied when the intra prediction mode selected for the current block includes the corner mode, and constraining the use of the IPF for intra prediction with respect to the current block when at least one of the constraint conditions is satisfied, includes: in the case where the template-based intra-mode derivation (TIMD) fusion mode is selected for the current block, and two corner modes are used for the TIMD fusion, determining that the mode number limitation condition is satisfied and prohibiting the execution of IPF for intra prediction with respect to the current block; and in a case where the TIMD merging mode is selected for the current block, and only one of the two intra-prediction modes used for TIMD merging is a corner mode satisfying the usage condition for IPF, determining that the mode number limitation condition is not satisfied, and allowing IPF execution for the corner mode satisfying the usage condition for IPF to perform intra-prediction with respect to the current block.

4. The method according to paragraph 1, characterized in that a limitation condition includes any one or more of the following: constraint condition 1: TIMD merge mode is selected for the current block; Constraint condition 2: The decoder-side internal mode merging (DIMD) mode is selected for the current block; Constraint condition 3: Multiple reference rows (MRL) are used for the current block; constraint condition 4: the index of the reference row selected for the current block is greater than or equal to K, where K is an integer greater than or equal to 3; constraint condition 5: the width of the current block is less than or equal to the set value; constraint condition 5: the height of the current block is less than or equal to the set value; constraint condition 7: the product of the width and height of the current block is less than or equal to the set value; constraint condition 8: Internal Sub-Partitioning (ISP) mode is used for the current block; constraint condition 9: the angle of the angular mode selected for the current block is -45°, 0°, 45°, 90°, or 135°, or the angular mode selected for the current block is an integer slope angular mode; constraint condition 10: the current frame to which the current block belongs is an outer frame; and constraint condition 11: the current image to which the current block belongs is not a brightness image; wherein IPF is not allowed for intra-prediction with respect to the current block when at least one of the constraint conditions 1-11 is satisfied.

5. The method according to paragraph 4, characterized in that the constraint condition includes constraint condition 4, where K is equal to 3, or 5, or 7, or 12.

6. The method according to claim 1, characterized in that performing intra-prediction with respect to the current block using IPF includes: accepting a weighted sum of the first prediction result and the second prediction result as a final prediction result of the current block, when only one intra-prediction mode is selected for the current block, and the intra-prediction mode is an angular mode, wherein the first prediction result is obtained by performing a prediction with respect to the current block according to the first reference line selected for the current block and the first angular mode selected for the current block, and the second prediction result is obtained by performing a prediction with respect to the current block according to the second reference line and the first angular mode, wherein the second reference line is an adjacent line for the first reference line or a reference line with index 0.

7. The method according to claim 1, characterized in that performing intra-prediction with respect to the current block using IPF includes: Taking a weighted sum of three or more prediction results as a final prediction result of the current block, when the intra-prediction mode selected for the current block is a merge mode, wherein the three or more prediction results comprise: several prediction results obtained by performing a prediction with respect to the current block according to a first reference line selected for the current block and each intra-prediction mode in the merge mode, and one or more prediction results obtained by performing a prediction with respect to the current block according to a second reference line and one or more corner modes for which the IPF is enabled in the merge mode, wherein the second reference line is a neighboring line for the first reference line or a reference line with index 0.

8. The method according to paragraph 6 or 7, characterized in that the second reference row is adjacent to the first reference row and is closer to the current block than the first reference row when the index of the first reference row is greater than or equal to K, where K is an integer greater than or equal to 1.

9. The method according to paragraph 6 or 7, characterized in that the weight assigned to the first prediction result is greater than the weight assigned to the second prediction result to calculate the weighted sum of the first prediction result and the second prediction result.

10. The method according to paragraph 6, characterized in that determining whether a constraint condition for using IPF for a current block is satisfied when an intra-prediction mode selected for the current block includes a corner mode, and performing intra-prediction with respect to the current block using IPF when the constraint condition is not satisfied, includes: In a case where an extended reference row and a corner mode in a template-based multiple reference row mode candidate list (TMRL) are selected for the current block and the constraint condition is not satisfied, calculating a first prediction result by performing a prediction with respect to the current block according to the extended reference row and the corner mode, and calculating a second prediction result by performing a prediction with respect to the current block according to another reference row and the corner mode, and taking a weighted sum of the first prediction result and the second prediction result as the final prediction result of the current block, wherein the other reference row is a reference row with an index of 0 or an adjacent row for the extended reference row.

11. An intra prediction fusion (IPF) method, wherein the method is applied to an encoder and comprises: determining whether a constraint condition for using the IPF for the current block is satisfied when the intra prediction mode selected for the current block includes the corner mode; and Performing intra-prediction on the current block using IPF when the constraint condition is not satisfied.

12. The method according to paragraph 11, characterized in that a limitation condition includes any one or more of the following: constraint condition 1: TIMD merge mode is selected for the current block; Constraint condition 2: The decoder-side internal mode merging (DIMD) mode is selected for the current block; Constraint condition 3: Multiple reference rows (MRL) are used for the current block; constraint condition 4: the index of the reference row selected for the current block is greater than or equal to K, where K is an integer greater than or equal to 3; constraint condition 5: the width of the current block is less than or equal to the set value; constraint condition 5: the height of the current block is less than or equal to the set value; constraint condition 7: the product of the width and height of the current block is less than or equal to the set value; constraint condition 8: Internal Sub-Partitioning (ISP) mode is used for the current block; constraint condition 9: the angle of the angular mode selected for the current block is -45°, 0°, 45°, 90°, or 135°, or the angular mode selected for the current block is an integer slope angular mode; constraint condition 10: the current frame to which the current block belongs is an outer frame; and constraint condition 11: the current image to which the current block belongs is not a brightness image; wherein IPF is not allowed for intra-prediction with respect to the current block when at least one of the constraint conditions 1-11 is satisfied.

13. The method according to claim 11, characterized in that performing intra-prediction with respect to the current block using IPF includes: accepting a weighted sum of the first prediction result and the second prediction result as a final prediction result of the current block, when only one intra-prediction mode is selected for the current block, and the intra-prediction mode is an angular mode, wherein the first prediction result is obtained by performing a prediction with respect to the current block according to the first reference line selected for the current block and the first angular mode selected for the current block, and the second prediction result is obtained by performing a prediction with respect to the current block according to the second reference line and the first angular mode, wherein the second reference line is an adjacent line for the first reference line or a reference line with index 0.

14. The method according to paragraph 13, characterized in that determining whether a constraint condition for using IPF for a current block is satisfied when an intra-prediction mode selected for the current block includes a corner mode, and performing intra-prediction with respect to the current block using IPF when the constraint condition is not satisfied, includes: In a case where an extended reference row and a corner mode in a template-based multiple reference row mode candidate list (TMRL) are selected for the current block and the constraint condition is not satisfied, calculating a first prediction result by performing a prediction with respect to the current block according to the extended reference row and the corner mode, and calculating a second prediction result by performing a prediction with respect to the current block according to another reference row and the corner mode, and taking a weighted sum of the first prediction result and the second prediction result as the final prediction result of the current block, wherein the other reference row is a reference row with an index of 0 or an adjacent row for the extended reference row.

15. A non-volatile machine-readable data carrier storing a computer program and a bit stream, wherein when executed by a processor, the computer program causes the processor to implement the method according to any of paragraphs 11-14 for generating the bit stream.

Images