Video coding and decoding method, device and system

Abstract

A video encoding and decoding method, apparatus and system, using a TMRL permission usage flag, determine whether to permit encoding or decoding of a TMRL mode syntax element of a current block by a value of the TMRL permission usage flag when decoding the current block. According to the embodiment of the invention, whether the TMRL mode is used or not is represented through the high-level syntax elements, so that the mode selection of video coding and decoding is more flexible, and better adaptability is achieved.

Description

Video encoding and decoding method, device and system Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, video technology, and more specifically, to a video encoding and decoding method, apparatus, and system. Background Art

[0002] Digital video compression technology primarily compresses large amounts of digital video data for easier transmission and storage. Currently popular video codec standards, such as H.266 / Versatile Video Coding (VVC), employ a block-based hybrid coding framework. Each video frame is divided into square largest coding units (LCUs) of equal size (e.g., 128x128, 64x64, etc.). Each LCU can be divided into rectangular coding units (CUs) based on a specific rule. Coding units may also be divided into prediction units (PUs) and transform units (TUs). The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-frame prediction and inter-frame prediction, which are used to reduce or remove inherent redundancy in the video. Intra-frame blocks are predicted using neighboring pixels as reference, while inter-frame blocks use information from spatially adjacent blocks and reference information from other frames. In contrast to the prediction signal, the residual information is transformed, quantized, and entropy-encoded into a bitstream on a block-by-block basis. These techniques are described in standards and implemented in various fields related to video compression.

[0003] With the surge in Internet videos and people's increasing demand for video clarity, although existing digital video compression standards can save a lot of video data, there is still a need to pursue better digital video compression technology to reduce the bandwidth and traffic pressure of digital video transmission.

[0004] SUMMARY OF THE INVENTION

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] An embodiment of the present disclosure provides a video decoding method, applied to a decoder, comprising:

[0007] Determine the value of the template-based multiple reference line intra prediction TMRL permission flag by decoding;

[0008] When decoding the current block:

[0009] When the value of the TMRL usage permission flag indicates that the TMRL mode is permitted, decoding of the TMRL mode syntax element of the current block is permitted;

[0010] When the value of the TMRL usage permission flag indicates that the TMRL mode is not allowed to be used, decoding of the TMRL mode syntax element of the current block is skipped.

[0011] An embodiment of the present disclosure further provides a video encoding method, applied to an encoder, comprising:

[0012] Determine the value of the template-based multiple reference line intra prediction TMRL allowed flag;

[0013] When encoding the current block:

[0014] When the value of the TMRL usage permission flag indicates that the TMRL mode is permitted, encoding the TMRL mode syntax element of the current block is permitted;

[0015] When the value of the TMRL usage permission flag indicates that the TMRL mode is not allowed to be used, encoding of the TMRL mode syntax element of the current block is skipped.

[0016] An embodiment of the present disclosure further provides a code stream, wherein the code stream is generated by the video encoding method described in any embodiment of the present disclosure.

[0017] An embodiment of the present disclosure further provides a video decoding device, comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method described in any embodiment of the present disclosure when executing the computer program.

[0018] An embodiment of the present disclosure further provides a video encoding device, including a processor and a memory storing a computer program, wherein the processor can implement the video encoding method described in any embodiment of the present disclosure when executing the computer program.

[0019] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0020] An embodiment of the present disclosure further provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, can implement the video decoding method described in any embodiment of the present disclosure, or implement the video encoding method described in any embodiment of the present disclosure.

[0021] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

[0022] Summary of the Figures

[0023] The accompanying drawings are used to provide an understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure and do not constitute a limitation to the technical solutions of the present disclosure.

[0024] FIG1A is a schematic diagram of a coding and decoding system according to an embodiment of the present disclosure;

[0025] FIG1B is a framework diagram of an encoding end according to an embodiment of the present disclosure;

[0026] FIG1C is a framework diagram of a decoding end according to an embodiment of the present disclosure;

[0027] FIG2 is a schematic diagram of an intra-frame prediction mode according to an embodiment of the present disclosure;

[0028] FIG3 is a schematic diagram of adjacent intra-frame prediction blocks of a current block according to an embodiment of the present disclosure;

[0029] FIG4 is a schematic diagram of a template of a current block and a template reference area according to an embodiment of the present disclosure;

[0030] FIG5 is a schematic diagram of multiple reference rows around a current block according to an embodiment of the present disclosure;

[0031] FIG6 is a flowchart of a video encoding method according to an embodiment of the present disclosure;

[0032] FIG7 is a flowchart of a method for constructing a TMRL mode candidate list according to an embodiment of the present disclosure;

[0033] FIG8A is a schematic diagram of a template area around a current block and an extended reference row according to an embodiment of the present disclosure;

[0034] FIG8B is a schematic diagram of a template area around a current block and an extended reference row according to another embodiment of the present disclosure;

[0035] FIG9 is a flowchart of a video decoding method according to an embodiment of the present disclosure;

[0036] FIG10 is a flowchart of a video decoding method according to another embodiment of the present disclosure;

[0037] FIG11 is a flowchart of a video encoding method according to another embodiment of the present disclosure;

[0038] FIG12 is a schematic diagram of a video decoding apparatus according to an embodiment of the present disclosure.

[0039] Details

[0040] The present disclosure describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure.

[0041] In the description of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment described as "exemplary" or "for example" in the present disclosure should not be interpreted as being more preferred or advantageous than other embodiments. "And / or" in this article is a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "Multiple" refers to two or more than two. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present disclosure, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0042] When describing representative exemplary embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art can readily understand that these sequences can vary and still remain within the spirit and scope of the disclosed embodiments.

[0043] The video coding and decoding method proposed in the embodiments of the present disclosure can be applied to various video coding and decoding standards, such as: H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC), AVS (Audio Video Coding Standard), and other standards developed by MPEG (Moving Picture Experts Group), AOM (Alliance for Open Media), JVET (Joint Video Experts Team) and extensions of these standards, or any other customized standards.

[0044] FIG1A is a block diagram of a video encoding and decoding system that can be used in an embodiment of the present disclosure. As shown in the figure, the system is divided into an encoding end device 1 and a decoding end device 2. The encoding end device 1 generates a code stream. The decoding end device 2 can decode the code stream. The decoding end device 2 can receive the code stream from the encoding end device 1 via a link 3. The link 3 includes one or more media or devices capable of moving the code stream from the encoding end device 1 to the decoding end device 2. In one example, the link 3 includes one or more communication media that enable the encoding end device 1 to send the code stream directly to the decoding end device 2. The encoding end device 1 modulates the code stream according to a communication standard (e.g., a wireless communication protocol) and sends the modulated code stream to the decoding end device 2. The one or more communication media may include wireless and / or wired communication media and may form part of a packet network. In another example, the code stream can also be output from the output interface 15 to a storage device, and the decoding end device 2 can read the stored data from the storage device via streaming or downloading.

[0045] As shown in the figure, the encoding end device 1 includes a data source 11, a video encoding device 13 and an output interface 15. The data source 11 includes a video capture device (such as a camera), an archive containing previously captured data, an input interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. The video encoding device 13 encodes the data from the data source 11 and outputs it to the output interface 15. The output interface 15 may include at least one of a regulator, a modem and a transmitter. The decoding end device 2 includes an input interface 21, a video decoding device 23 and a display device 25. The input interface 21 includes at least one of a receiver and a modem. The input interface 21 can receive a code stream via the link 3 or from a storage device. The video decoding device 23 decodes the received code stream. The display device 25 is used to display the decoded data. The display device 25 can be integrated with other devices of the decoding end device 2 or set separately. The display device 25 is optional for the decoding end. In other examples, the decoding end may include other devices or equipment that apply the decoded data.

[0046] Based on the video coding and decoding system shown in FIG. 1A , various video coding and decoding methods can be used to implement video compression and decompression.

[0047] FIG1B is a block diagram of an exemplary video encoding device that can be used in embodiments of the present disclosure. As shown, the video encoding device 1000 includes a prediction unit 1100, a partitioning unit 1101, a residual generation unit 1102 (indicated by the circle with a plus sign after the partitioning unit 1101 in the figure), a transform processing unit 1104, a quantization unit 1106, an inverse quantization unit 1108, an inverse transform processing unit 1110, a reconstruction unit 1112 (indicated by the circle with a plus sign after the inverse transform processing unit 1110 in the figure), a filter unit 1113, a decoded image buffer 1114, and an entropy coding unit 1115. The prediction unit 1100 includes an inter-frame prediction unit 1121 and an intra-frame prediction unit 1126. The decoded image buffer 1114 may also be referred to as a decoded image buffer, a decoded picture buffer, or a decoded picture buffer. The video encoder 20 may also include more, fewer, or different functional components than shown in this example. For example, the transform processing unit 1104 and the inverse transform processing unit 1110 may be eliminated in some cases.

[0048] The division unit 1101 cooperates with the prediction unit 1100 to divide the received video data into slices, coding tree units (CTUs) or other larger units. The video data received by the division unit 1101 can be a video sequence including video frames such as I frames, P frames or B frames.

[0049] The prediction unit 1100 can divide a CTU into coding units (CUs) and perform intra-frame prediction coding or inter-frame prediction coding on the CU. When performing intra-frame prediction and inter-frame prediction on the CU, the CU can be divided into one or more prediction units (PUs).

[0050] The inter-frame prediction unit 1121 can perform inter-frame prediction on the PU and generate prediction data for the PU, which includes the prediction block of the PU, the motion information of the PU, and various syntax elements. The inter-frame prediction unit 1121 can include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit can be used to estimate motion to generate a motion vector, and the motion compensation unit can be used to obtain or generate a prediction block based on the motion vector.

[0051] The intra-prediction unit 1126 may perform intra prediction on the PU and generate prediction data for the PU. The prediction data for the PU may include a prediction block and various syntax elements for the PU.

[0052] The residual generation unit 1102 may subtract the prediction blocks of the PUs into which the CU is partitioned from the original blocks of the CU to generate the residual blocks of the CU.

[0053] The transform processing unit 1104 may divide a CU into one or more transform units (TUs). The division of a prediction unit (PU) and a transform unit (TU) may be different. A TU-associated residual block is a sub-block obtained by dividing the residual block of the CU. One or more transforms are applied to the TU-associated residual block to generate a TU-associated coefficient block.

[0054] The quantization unit 1106 may quantize the coefficients in the coefficient block based on the selected quantization parameter. The degree of quantization of the coefficient block may be adjusted by adjusting the quantization parameter (QP: Quantizer Parameter).

[0055] The inverse quantization unit 1108 and the inverse transform unit 1110 may apply inverse quantization and inverse transform to the coefficient block, respectively, to obtain a reconstructed residual block associated with the TU.

[0056] The reconstruction unit 1112 may add the reconstructed residual block and the prediction block generated by the prediction unit 1100 to generate a reconstructed image.

[0057] Filter unit 1113 performs loop filtering on the reconstructed image and stores the filtered reconstructed image in decoded image buffer 1114 as a reference image. Intra-frame prediction unit 1126 extracts reference images of blocks adjacent to the PU from decoded image buffer 1114 to perform intra-frame prediction. Inter-frame prediction unit 1121 can use the reference image of the previous frame cached in decoded image buffer 1114 to perform inter-frame prediction on the PU of the current frame image.

[0058] The entropy coding unit 1115 may perform entropy coding operations on received data (eg, syntax elements, quantized coefficient blocks, motion information, etc.).

[0059] FIG1C is a block diagram of an exemplary video decoding device that can be used in embodiments of the present disclosure. As shown, the video decoding device 101 includes an entropy decoding unit 150, a prediction unit 152, an inverse quantization unit 154, an inverse transform unit 156, a reconstruction unit 158 ​​(represented by the circle with a plus sign after the inverse transform unit 155), a filter unit 159, and a decoded image buffer 160. In other embodiments, the video decoder 30 may include more, fewer, or different functional components. For example, the inverse transform unit 155 may be eliminated in some cases.

[0060] Entropy decoding unit 150 may perform entropy decoding on the received bitstream to extract syntax elements, quantized coefficient blocks, motion information for PUs, etc. Prediction unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 159 may each perform corresponding operations based on the syntax elements extracted from the bitstream.

[0061] The inverse quantization unit 154 may perform inverse quantization on the coefficient block associated with the quantized TU.

[0062] Inverse transform processing unit 156 may apply one or more inverse transforms to the inverse quantized coefficient block to generate a reconstructed residual block for the TU.

[0063] Prediction unit 152 includes inter-prediction unit 162 and intra-prediction unit 164. If a PU is encoded using intra prediction, intra prediction unit 164 may determine an intra prediction mode for the PU based on syntax elements decoded from the codestream, perform intra prediction based on the determined intra prediction mode and reconstructed reference information of the PU's neighbors obtained from decoded picture buffer 160, and generate a prediction block for the PU. If a PU is encoded using inter prediction, inter prediction unit 162 may determine one or more reference blocks for the PU based on the motion information of the PU and corresponding syntax elements, and generate a prediction block for the PU based on the reference blocks obtained from decoded picture buffer 160.

[0064] The reconstruction unit 158 ​​may obtain a reconstructed image based on the reconstructed residual block associated with the TU and the prediction block of the PU generated by the prediction unit 152 .

[0065] The filter unit 159 may perform loop filtering on the reconstructed image, and the filtered reconstructed image is stored in the decoded image buffer 160. The decoded image buffer 160 may provide a reference image for subsequent motion compensation, intra-frame prediction, inter-frame prediction, etc., and may also output the filtered reconstructed image as decoded video data for presentation on a display device.

[0066] Based on the above-described video encoding and decoding devices, the following basic encoding and decoding process can be performed. On the encoding side, a frame of an image is divided into blocks. Intra-frame prediction, inter-frame prediction, or other algorithms are performed on the current block to generate a predicted block for the current block. The predicted block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain quantized coefficients, and the quantized coefficients are entropy encoded to generate a bitstream. On the decoding side, intra-frame prediction or inter-frame prediction is performed on the current block to generate a predicted block for the current block. The quantized coefficients obtained from the decoded bitstream are then inversely quantized and inversely transformed to obtain a residual block. The predicted block and residual block are added together to obtain a reconstructed block. The reconstructed block forms a reconstructed image. The reconstructed image is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoding side also performs similar operations as the decoding side to obtain a decoded image, which is often referred to as a reconstructed image. The decoded image obtained by the encoding side can serve as a reference frame for inter-frame prediction of subsequent frames. The block division information determined by the encoding side, as well as information and parameters for prediction, transform, quantization, entropy coding, and loop filtering modes, can be written into the bitstream if necessary. The decoding end determines the same block division information, prediction, transformation, quantization, entropy coding, loop filtering and other mode information and parameter information as the encoding end by decoding the code stream or analyzing the existing information, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end.

[0067] Although the above is an example of a block-based hybrid coding framework, the embodiments of the present disclosure are not limited thereto. With the development of technology, one or more modules in the framework and one or more steps in the process can be replaced or optimized.

[0068] Herein, the current block may be a block-level coding unit such as a current coding unit (CU) or a current prediction unit (PU) in a current image.

[0069] When performing intra-frame prediction, the encoder typically uses various angular and non-angular modes to predict the current block to obtain a predicted block. Based on the rate-distortion information calculated between the predicted block and the original block, the encoder selects the optimal intra-frame prediction mode for the current block, encodes this intra-frame prediction mode, and transmits it to the decoder via the bitstream. The decoder decodes the selected intra-frame prediction mode and performs intra-frame prediction on the current block using this intra-frame prediction mode. In this article, the reference row and intra-frame prediction mode selected for the current block are also referred to as the reference row and intra-frame prediction mode selected for the current block.

[0070] In VVC and ECM, various traditional intra-frame prediction modes use reconstructed information around the current block to predict the current block. These include: planar mode (mode index 0), mean mode (DC mode, mode index 1), and 65 angular prediction modes (mode indices 2 to 66). Figure 2 shows the angular directions of the angular prediction modes with mode indices 2 to 66. Herein, the angular prediction modes are referred to as angular modes.

[0071] Since rectangular prediction blocks are introduced in VVC, for rectangular blocks, the angle directions of some angle modes with indices 2 to 66 can be replaced with wider angle directions. As shown in the figure, the angle modes with indices -14 to -1, 67 to 80 are angle modes obtained by wide angle replacement. The selection of these angle modes does not need to be represented by a flag bit, but is obtained through the correspondence between the shape of the current block and the prediction mode index (2 to 66) selected for the current block. After completing the matching of the wide angle mode, each angle prediction mode predModeIntra (-14 to 80) will have an angle value intraPredAngle, as shown in the following table. The angle value of the angle mode will be used for subsequent angle prediction.

[0072] Table 1. Correspondence between angle prediction mode and angle value

[0073] predModeIntra-14-13-12-11-10-9-8-7-6-5-4-3-2-1234intraPredAngle5123412561711281028673645751453935322926predModeIntra567891011121 31415161718192021intraPredAngle232018161412108643210-1-2-3predModeIntra2223242526272829303132333435363738intraPredAngle-4-6-8-10 -12-14-16-18-20-23-26-29-32-29-26-23-20predModeIntra3940414243444546474849505152535455intraPredAngle-18-16-14-12-10-8-6-4-3-2-10 12346predModeIntra5657585960616263646566676869707172intraPredAngle810121416182023262932353945515764predModeIntra7374757677787980 intraPredAngle7386102128171256341512

[0074] In this article, each angle mode predModeIntra corresponds to an angle, and the angle of each angle prediction mode is the angle of the line segment corresponding to the angle prediction mode in Figure 1 in the rectangular coordinate system. For example, the angle mode with index number 34, the angle value intraPredAngle is -32, and the angle is -45° or 45° or 135°, which is related to the 0° direction defined by the rectangular coordinate system.

[0075] Unless otherwise specified, the intra-frame prediction mode mentioned herein refers to a traditional intra-frame prediction mode including a Planar mode, a DC mode, and an angular mode.

[0076] Based on statistical properties, pixels closer to the current block are more likely to use the same intra-frame prediction mode as the current block. This characteristic has led to the adoption of the most probable mode (MPM) technique in HEVC, VVC, and the Enhanced Compression Model (ECM). ECM is based on the VTM-10.0 reference software and integrates various new tools to further enhance codec performance.

[0077] MPM first constructs an MPM list and populates it with the six intra prediction modes most likely to be selected by the current block. If the intra prediction mode selected by the current block is in the MPM list, only its index number (only 3 bits) needs to be encoded. If the intra prediction mode selected by the current block is not in the MPM list but is one of the 61 non-MPM modes, the intra prediction mode is encoded using Truncated Binary Code (TBC) during the entropy coding stage.

[0078] In VVC, regardless of whether multiple reference lines (MRL) and intra sub-block partitioning (ISP) are applied, the MPM list has 6 prediction modes. The MPM in ECM is divided into MPM and Secondary MPM. MPM and Secondary MPM use lists of length 6 and length 16 respectively. Among the 6 modes in the MPM list, the Planar mode is always filled in the first position in the MPM, and the filling of the remaining 5 positions is carried out in sequence according to the set steps until the 5 positions are filled. The extra modes will automatically enter the Secondary MPM. The Secondary MPM list can be composed of some main angle modes other than the intra prediction mode in the MPM. Since the encoding and decoding order of the MPM flag (mpm_flag) is after the MRL mode, the encoding and decoding of the MPM in ECM depends on the MRL flag. When the current block does not use the MRL mode, the MPM flag needs to be decoded to determine whether the current block uses the MPM. When the current block uses the MRL mode, there is no need to decode the MPM flag, and the current block uses the MPM by default.

[0079] Template-based intra mode derivation (TIMD) and decoder-side intra mode derivation (DIMD) are two intra prediction techniques for luma frames adopted into the ECM reference software. These two technologies derive the intra prediction mode for the current block at the decoder based on the reconstructed pixel values ​​around the current block, thereby saving the bits required to encode the intra prediction mode index.

[0080] In ECM, as shown in Figure 4, the left adjacent area and the upper adjacent area of ​​the current block (such as the current CU) 11 constitute the template area 12 of the current block. The left adjacent area is called the left template area (referred to as the left template), and the upper adjacent area is called the upper template area or the upper template area (referred to as the upper template). A template reference area 13 is provided outside the template area 12 (referring to the left and upper sides), and the exemplary sizes and positions of each area are shown in the figure. In one example, the width L1 of the left template and the height L2 of the upper template are both 4. The template reference area 13 can be a row adjacent to the upper side of the template area or a column adjacent to the left side.

[0081] TIMD assumes that the distribution characteristics of the current block and the template area of ​​the current block are consistent, and uses the reconstructed value of the template reference area as the reconstructed value of the reference row. It traverses all intra-frame prediction modes in MPM and Secondary MPM to predict the template area and obtain the prediction result. Then, the sum of absolute transformed differences (SATD) between the reconstructed value on the template area and the prediction result of each mode (the prediction value on the template area) is calculated, and the TIMD mode of the current block is determined. The decoding end derives the TIMD mode in the same way. If the sequence allows the use of TIMD, and the intra-frame prediction mode selected by the current block is the TIMD mode, only one flag bit is required to indicate that the current block is predicted using the TIMD mode. The decoding of the remaining syntax elements related to intra-frame prediction, such as ISP, MPM, etc., can be skipped, thereby significantly reducing the coding bits.

[0082] After obtaining the SATD between the reconstructed value of the template area and the prediction result of each mode, the TIMD mode can be determined according to the following method: Assume that mode1 and mode2 are two angle modes used for intra prediction in MPM, mode1 is the angle mode with the smallest SATD, and its SATD is cost1; mode2 is the angle mode with the second smallest SATD, and its SATD is cost2:

[0083] When cost1×2≤cost2, mode1 is used as the TIMD mode of the current block;

[0084] When cost1×2>cost2, the prediction mode obtained by weighting the prediction results of mode1 and mode2 is used as the TMID mode of the current block, also known as the TIMD fusion mode.

[0085] DIMD uses the reconstructed pixel values ​​around the current block as a template. The Sobel operator is used to scan each 3x3 region on the template and calculate the horizontal and vertical gradients. The horizontal and vertical gradients Dx and Dy are then calculated. Based on Dx and Dy, the amplitude value Amp = abs(Dx) + abs(Dy) and the angle value angular = arctan(Dy / Dx) are calculated at each location. The angle value at each location on the template is mapped to a traditional angle pattern, and the amplitude values ​​for the same angle pattern are accumulated to form a histogram of the amplitude and angle patterns. When the two angle patterns with the highest and second highest amplitude values ​​exist, the prediction values ​​of the two angle patterns and the planar mode are weighted to obtain the final prediction result for the current block using DIMD. This prediction mode combines three intra-frame prediction modes: the planar mode and the two angle patterns with the highest and second highest amplitude values. This is referred to as the DIMD fusion mode in this paper. When the two angle patterns with the highest and second highest amplitude values ​​do not exist, the prediction using DIMD is equivalent to the prediction using the planar mode.

[0086] In HEVC, intra prediction uses the row and column immediately above and to the left of the current block as reference rows. If the reconstructed values ​​of these rows and columns differ significantly from the original pixel values, the prediction quality of the current block can be significantly affected. To address this issue, VVC adopts the MRL technique. In addition to using reference row index 0 (Reference line 0), reference row index 1 (Reference line 1) and reference row index 2 (Reference line 2) can also be used as extended reference rows for intra prediction. To reduce coding complexity, MRL is only used in non-planar modes within the MPM. When predicting the current block based on each angular mode, the encoder tries all three reference rows and selects the one with the lowest rate-distortion cost (RD cost). The index of the selected reference row is encoded and sent to the decoder. The decoder decodes the reference row index and uses it to determine the selected reference row for the current block, which is then used for prediction.

[0087] FIG5 shows four reference lines for the current block, namely reference line 0 221, reference line 1 222, reference line 2 223, and reference line 3 224. The current block may have more reference lines, and during prediction, only reconstructed values ​​from a portion of the reference lines may be used.

[0088] In ECM, the MRL mode can use more reference rows. The indices of multiple candidate reference rows are entered into a list. This is called an MRL index list, MRL list, candidate reference row list, or reference row index list. If TIMD is not used in the current block, the MRL index list is 6 in length and can hold the indices of six reference rows. The indices and their order are fixed: 0, 1, 3, 5, 7, and 12, respectively. The first position is filled with the index. When using MRL, the MRL index is encoded to represent the position of the reference row selected for the current block in the MRL index list. Taking the MRL index list {0, 1, 3, 5, 7, 12} as an example, the MRL indices corresponding to indices 0, 1, 3, 5, 7, and 12 in the table are 0, 1, 2, 3, 4, and 5, respectively. The MRL index can be encoded using a unary truncated code based on the context model. After encoding, a binary identifier based on the context model is obtained. The binary identifier can also be called a binary identifier bit, a binary symbol, or a binary bit. The smaller the MRL index value, the shorter the code length and the faster the decoding.

[0089] When the MRL mode and TIMD mode are used simultaneously, the length of the MRL index list is 3, which can be filled with the indexes of 3 reference rows and the order between the indexes is fixed, expressed as {0, 1, 3}.

[0090] In this article, although the reference row is called a "row", this is for convenience of expression. A reference row actually includes one row and one column. Generally, the reconstructed value of the reference row used in prediction also includes the reconstructed value of one row and one column, which is the same as the usual description method in the industry.

[0091] The same technology may have different names in different standards. For example, a technology like MPM, which uses blocks around the current block to derive a list of most likely modes, is called Adaptive Intra Mode Coding (AIMC) in AV2 (AVM), and is called Frequency-based Intra Mode Coding (FIMC) in AVS3 for screen content coding. In non-screen content coding, technologies similar to MPM are always enabled. And a technology like MRL, which uses multiple reference lines for intra prediction, is called Multiple Reference Line Selection for Intra Prediction (MRLS) in AV2 (AVM). However, this is merely a difference in name; when this embodiment uses terms such as MPM and MRL, it should also cover these essentially identical technologies in other standards.

[0092] One embodiment provides an intra prediction fusion (IPF) technology. IPF uses two reference lines and the angle mode selected for the current block to form two combinations of reference lines and intra prediction modes. The current block is predicted based on the two combinations. The two prediction results are weighted and used as the final prediction result for the current block, as follows:

[0093] p fusion =w a ×p a +w b ×p b

[0094] Among them, p a is the result of predicting the current block using the combination of the reference row indexed by a and the angle mode, p b is the result of predicting the current block using the reference row with index b and the angle mode, p fusion is the prediction result after fusion of the current block; w a When p is weighted a The weight, w b When p is weighted b In one example, b=a+1, w a is 3 / 4,w b When the current block uses IPF, the above fused prediction result is used as the final prediction result of the current block, that is, the prediction value of the current block.

[0095] In one example, IPF turns on the angle mode selected for the current block by default when all of the following conditions are met: the angle mode selected for the current block is not an integer slope angle mode; the width multiplied by the height of the current block is greater than 16; the current block does not select the intra-frame sub-block division ISP mode. Among them, when the angle value (intraPredAngle) of the angle mode is divided by 32 with a remainder of 0, the angle mode is an integer slope angle mode. The correspondence between intraPredAngle and the angle mode can be seen in Table 1 above. In other examples, IPF can also add other restrictions such as mode quantity restrictions. For example, if the use of IPF will cause the prediction of the current block to require the fusion of more than three intra-frame prediction modes, or if the use of IPF will cause the prediction of the current block to require the fusion of more than two angle modes, IPF is not allowed to be used.

[0096] One embodiment provides a template-based multiple reference line intra-intra prediction (TMRL) mode. The TMRL mode constructs a candidate list based on a combination of extended reference lines and intra-prediction modes, and then encodes and decodes the combination. The combined encoding and decoding approach of the TMRL mode can reduce coding overhead and improve coding performance.

[0097] The video encoding method of this embodiment is applied to an encoder, as shown in FIG6 , and includes:

[0098] Step 110 , constructing a candidate list of the TMRL mode of the current block, wherein the candidate list is filled with combinations of extended reference rows and intra prediction modes of the current block candidates;

[0099] Step 120 , through rate-distortion optimization, the current block selects a combination of a reference row and an intra prediction mode for intra prediction;

[0100] In this article, the reference lines include the reference line with index 0 and the extended reference line. The combination of the reference line and the intra-frame prediction mode selected by the current block may be a combination of the reference line with index 0 and an intra-frame prediction mode, or a combination of an extended reference line and an intra-frame prediction mode.

[0101] Step 130: When the coding condition of the TMRL mode of the current block is satisfied, encode the TMRL mode flag of the current block to indicate that the current block uses the TMRL mode, and encode the TMRL mode index of the current block to indicate the position of the selected combination in the candidate list;

[0102] The encoding conditions include: the selected combination is in the candidate list (in this case, the current block selects the extended reference row), and may also include any one or more of the following: Condition 1, the current block is a block in the luma frame; that is, the TMRL mode is only used for luma frames; Condition 2, the current block is not located at the upper boundary of the coding tree unit (CTU); Condition 3, the sequence in which the current block is located allows the use of MRL; Condition 4, the size of the current block meets the size requirements for using the TMRL mode; Condition 5, the aspect ratio of the current block meets the aspect ratio requirements for the current block using the TMRL mode. The TMRL mode index can use the Golomb-Rice coding method to more reasonably classify the candidate combinations into categories with different codeword lengths for encoding and decoding, thereby improving coding efficiency.

[0103] When the current block uses the TMRL mode, the MPM mode, intra-frame sub-block partitioning ISP mode, multi-transform selection MTS mode, low-frequency non-divisible transform LFNST mode, etc. can be skipped. The TMRL mode flag and TMRL mode index can simultaneously indicate the reference row and intra-frame prediction mode selected by the current block, and there is no need to encode or decode the MPM-related syntax elements.

[0104] In this paper, the candidate list is filled with the combination of the extended reference row and the intra prediction mode of the current block candidate, which means that the combination in the candidate list needs to participate in the rate-distortion optimization of the current block, that is, the mode selection process of selecting the current block prediction mode through the rate-distortion cost.

[0105] In one example of this embodiment, when the current block uses TIMD, encoding of the TMRL mode flag and TMRL mode index for the current block is skipped. When the current block does not use TIMD but the selected combination is not in the candidate list, encoding of the TMRL mode flag for the current block indicates that the TMRL mode is not used, and encoding of the TMRL mode index is skipped. When the current block does not use TIMD, the TMRL mode flag and TMRL mode index provided in this embodiment can replace the original MRL index.

[0106] One embodiment provides a method for constructing a TMRL mode candidate list, which can be applied to both an encoder and a decoder. As shown in FIG7 , the method includes:

[0107] Step 210 , obtaining N×M combinations of extended reference lines and intra prediction modes according to the N extended reference lines and M intra prediction modes of the current block, where N≥1, M≥1, and N×M≥2;

[0108] Step 220 , predicting the template region of the current block according to the N×M combinations, and calculating the error between the reconstructed value of the template region and the predicted value;

[0109] In addition to SATD and the sum of absolute difference (SAD), the error in this step can also be expressed by the sum of squared differences (SSD), mean absolute difference (MAD), mean squared error (MSE), etc.

[0110] Step 230 , filling K combinations corresponding to the errors into a candidate list of the TMRL mode of the current block in order from small to large, 1≤K≤N×M.

[0111] The candidate list created in this embodiment can implement combined coding of extended reference lines and intra-frame prediction modes, improving coding efficiency. Placing combinations with a high likelihood of being selected at the top of the candidate list reduces the TMRL mode index of the selected combination during coding, reducing coding overhead.

[0112] The template area of ​​the current block can be set on one or more reference lines that are closest to the current block. The N extended reference lines involved in the combination are extended reference lines that are located outside the template area and do not exceed the CTU boundary. In one example, when constructing the candidate list of the TMRL mode, all extended reference lines that do not exceed the CTU boundary are selected from the extended reference lines with predefined indexes {1, 3, 5, 7, 12}, where N≤5. The template area of ​​the current block is set on the reference line with an index of 0 (that is, the reference line with an index of 0 is called the reference line where the template area is located), and the other reference lines are called reference lines located outside the template area. Figure 8A shows 5 extended reference lines involved in the combination: reference line 31 with an index of 1, reference line 33 with an index of 3, reference line 35 with an index of 5, reference line 37 with an index of 7, and reference line 39 with an index of 12. In another example shown in FIG8B , the template area 40 of the current block is set on two reference lines with indexes 0 and 1, and the extended reference lines participating in the combination are all extended reference lines that do not exceed the CTU boundary among the reference line 42 with index 2, the reference line 43 with index 3, the reference line 45 with index 5, the reference line 47 with index 7, and the reference line 49 with index 12.

[0113] In one example of this embodiment, the M intra-frame prediction modes participating in the combination are allowed to be selected only from the angular mode, or only from the angular mode and the DC mode, or from the angular mode, the DC mode, and the planar mode. They can be selected from the intra-frame prediction modes selected by the MPM, or the MPM and the second MPM, or they can be selected step by step according to a predetermined rule.

[0114] An embodiment provides a video decoding method related to the TMRL mode, which is applied to a decoder, as shown in FIG9 , and includes:

[0115] Step 310 , decoding the multi-reference row intra prediction TMRL mode flag of the current block to determine whether the current block uses the TMRL mode;

[0116] Step 320: If it is determined that the current block uses the TMRL mode, continue decoding the TMRL mode index of the current block and construct a candidate list of the TMRL mode of the current block, wherein the candidate list is filled with combinations of extended reference rows and intra prediction modes of the current block candidates;

[0117] Step 330: Determine the combination of the extended reference row and the intra-frame prediction mode selected for the current block based on the candidate list and the TMRL mode index, and predict the current frame based on the selected combination; wherein the TMRL mode index is used to indicate the position of the combination of the selected extended reference row and the intra-frame prediction mode in the candidate list.

[0118] This embodiment uses the TMRL mode index to simultaneously indicate the selected extended reference row and intra prediction mode for the current block, eliminating the need for two indexes. This reduces coding overhead. If the TMRL mode flag determines that the current block uses TMRL mode, decoding of syntax elements related to MPM mode, ISP mode, MTS mode, and LFNST mode can be skipped.

[0119] In an example of this embodiment, the video decoding method of this example includes:

[0120] Step 1: The decoder decodes syntax elements related to the TMRL mode, which in this example include syntax elements related to the TIMD, MRL and other modes.

[0121] The syntax related to current block decoding is shown in the following table:

[0122]

[0123]

[0124] The "cu_tmrl_flag" in the table is the TMRL mode flag. A flag equal to 1 indicates that the current block uses the TMRL mode, which defines the intra-frame prediction type of the current luma sample as the TMRL mode. A flag equal to 0 indicates that the current block does not use the TMRL mode, which defines the intra-frame prediction mode type of the current luma sample as not the TMRL mode. The "tmrl_idx" in the table is the TMRL mode index, which indicates the position of the combination of the extended reference row and intra-frame prediction mode selected for the current block in the candidate list of the TMRL mode. It can also be said to define the index of the selected combination in the candidate list sorted by the TMRL mode (indicator of the position of the combination).

[0125] The TMRL mode can be regarded as an evolution of the MRL mode, or as an integral part of the MRL mode. In this example, when the current block uses the TIMD mode, the decoding method of the MRL mode syntax elements remains unchanged. When the current block does not use the TIMD mode, the syntax elements of the TMRL mode need to be decoded. As shown in the table above, before decoding cu_tmrl_flag, first determine whether the following conditions are met: the current block allows the use of MRL (that is, whether sps_mrl_enabled_flag is 1), the current block is not located at the upper boundary of the CTU (that is, whether (y0%CtbSizeY)>0), and the current block does not use TIMD. When these conditions are met, decode cu_tmrl_flag. If the other two conditions are met and the current block uses TIMD, decode the multi-reference row index intra_luma_ref_idx of the current block.

[0126] Step 2: If the current block uses the TMRL mode, a candidate list of the TMRL mode needs to be constructed. The construction of the candidate list of the TMRL mode is an operation that both the encoder and the decoder need to perform.

[0127] ■ Determine candidate extended reference lines. Candidate extended reference lines are selected from predefined extended reference lines. Based on the position of the current block in the image, which of the predefined extended reference lines can be used is determined. This example constructs a template based on the reference line indexed 0. For N=5, all extended reference lines indexed {1, 3, 5, 7, 12} that do not exceed the CTU boundary are added to the candidate extended reference lines.

[0128] ■Determine the candidate intra-frame prediction modes. In this example, M=6, that is, the number of candidate intra-frame prediction modes is 6. First, among the 67 traditional prediction modes, remove the Planar mode and the DC mode, or only remove the Planar mode and retain the DC mode, and then select 6 intra-frame prediction modes to participate in the combination step by step: The first step is to select non-repeated intra-frame prediction modes in sequence from the intra-frame prediction modes used by the prediction blocks at the 5 neighboring positions around the current block, as shown in Figure 3. The second step is to perform an expansion operation on the angle mode selected in the first step, and perform addition and subtraction operations on each angle mode in sequence; select the non-repeating angle modes obtained by expansion, and end until the number of selected modes is 6. If 6 intra-frame prediction modes have not been selected after the expansion operation, or if no angle mode is selected in the first step, proceed to the third step: select non-repeating modes from the predefined mode set until the number of selected modes is 6.

[0129] ■ Construct a candidate list for the TMRL mode. After determining the candidate extended reference lines and intra-frame prediction modes, try all combinations in the extended reference line list and candidate list one by one. Predict these combinations on the template area where reference line 0 is located. Calculate the error between the reconstructed value of the template area and the predicted value obtained by each combination. The K combinations with the smallest error are added to the TMRL mode candidate list in ascending order of error. In this example, K = 12.

[0130] Step 3: Determine the combination of the extended reference row and intra prediction mode selected for the current block based on the constructed TMRL mode candidate list and the decoded TMRL mode index, and perform intra prediction on the current block based on the selected combination. When the current block uses the TMRL mode, the index refIdx of the reference row (in this case, the extended reference row) selected by the current block and the intra prediction mode (represented by the variable predModeIntra) are included in the determined combination.

[0131] Using the TMRL mode can save codewords. However, template-based techniques typically increase decoder complexity. Furthermore, using the TMRL mode requires the codec to support this complex computation, which not all codecs can do. To address this issue, high-level syntax elements can be used to indicate whether to use the TMRL mode, making mode selection more flexible and adaptable for video codecs.

[0132] The high-level syntax elements in this article refer to syntax elements at the sequence level, picture level, slice level, etc. that have a constraining effect on the block-level intra-frame prediction syntax elements.

[0133] An embodiment of the present disclosure provides a video decoding method that controls the use of the TMRL mode through high-level syntax. As shown in Figure 10, the video decoding method of this embodiment includes:

[0134] Step 410: Determine the value of the TMRL use permission flag by decoding; as the name indicates, the TMRL use permission flag is used to indicate whether the TMRL mode is allowed to be used;

[0135] In this step, the value of the TMRL allowed use flag is determined by decoding, which can be determined by decoding the TMRL allowed use flag itself, or by decoding other syntax elements related to TMRL to determine the value of the TMRL allowed use flag, or by decoding other syntax elements related to TMRL and the TMRL allowed use flag to determine the value of the TMRL allowed use flag.

[0136] Step 420: When decoding the current block, if the value of the TMRL use permission flag indicates that the TMRL mode is allowed to be used, the TMRL mode syntax elements of the current block are allowed to be decoded; if the value of the TMRL use permission flag indicates that the TMRL mode is not allowed to be used, the decoding of the TMRL mode syntax elements of the current block is skipped.

[0137] For identifiers such as the TMRL use permission flag, the MRL use permission flag, and the template use permission flag, a value of 0 generally indicates that the corresponding mode is not allowed to be used, and a value of 1 indicates that the corresponding mode is allowed to be used. However, the present disclosure is not limited to this, and it is also possible to use 1 to indicate that the mode is not allowed to be used and 0 to indicate that the mode is allowed to be used.

[0138] This embodiment increases the flexibility and adaptability of the use of the TMRL mode by setting a TMRL use permission flag to indicate whether the TMRL mode is allowed to be used; and correct decoding of the TMRL mode syntax elements of the current block can be achieved according to the value of the TMRL use permission flag.

[0139] In an exemplary embodiment of the present disclosure, the TMRL allows the use of a flag that is a sequence-level identifier. However, in other embodiments, the TMRL allows the use of a flag that is an image-level or slice-level identifier.

[0140] In an exemplary embodiment of the present disclosure, the TMRL use permission flag can be decoded independently without relying on other flags. In this case, the value of the TMRL use permission flag is obtained by decoding the TMRL use permission flag.

[0141] In the standard text of VVC, in order to determine whether to open the MRL mode (also known as MRL technology, MRL tool) for the current block, sps_mrl_enabled_flag is used to indicate the sequence-level MRL permission flag. The relevant syntax is shown in the following table:

[0142] Sequence parameter set RBSP syntax

[0143]

[0144] Among them, sps_mrl_enabled_flag is 0, which means that the use of MRL is not allowed, and sps_mrl_enabled_flag is 1, which means that the use of MRL is allowed.

[0145] This embodiment adds a sequence-level TMRL enable flag sps_tmrl_enabled_flag in a form without dependency to control whether the TMRL mode is allowed to be used. The relevant syntax is shown in the following table:

[0146] Sequence parameter set RBSP syntax

[0147]

[0148] In the above table, TMRL allows the use of the flag sps_tmrl_enabled_flag and MRL allows the use of the flag sps_mrl_enabled_flag to be decoded independently, and there is no dependency.

[0149] In an exemplary embodiment of the present disclosure, the TMRL mode uses an extended reference line, and a reference line with an index of 0 is generally used when making a mode selection for intra-frame prediction. Therefore, when the TMRL mode is allowed to be used, intra-frame prediction of the current block involves the use of multiple reference lines. When doing high-level syntax design, the decoding of the TMRL allowable use flag can depend on the MRL allowable use flag. At this time, the determination of the value of the TMRL allowable use flag by decoding includes: decoding the MRL allowable use flag to obtain the value of the MRL allowable use flag: when the value of the MRL allowable use flag indicates that the use of MRL is not allowed, skipping the decoding of the TMRL allowable use flag, and the default value of the TMRL allowable use flag is a value indicating that the TMRL mode is not allowed; and when the MRL allowable use flag indicates that the use of MRL is allowed, decoding the TMRL allowable use flag to obtain the value of the TMRL allowable use flag.

[0150] This embodiment adds a sequence-level TMRL permission flag in the form of a dependency relationship to control whether the TMRL mode is allowed. The relevant syntax is shown in the following table:

[0151] Sequence parameter set RBSP syntax

[0152]

[0153]

[0154] According to the above table, during decoding, the MRL usage permission flag sps_mrl_enabled_flag is decoded first. When sps_mrl_enabled_flag at the sequence level is 1, indicating that the use of MRL is allowed, the TMRL usage permission flag sps_tmrl_enabled_flag is decoded to determine the value of sps_tmrl_enabled_flag.

[0155] In this embodiment, the TMRL allowed use flag is an identifier at the sequence level, and the MRL allowed use flag is an identifier at the sequence level. In another embodiment, the TMRL allowed use flag may be an identifier at the image level, and the MRL allowed use flag may be an identifier at the sequence level or the image level. In yet another embodiment, the TMRL allowed use flag is an identifier at the slice level, and the MRL allowed use flag may be an identifier at the sequence level or the image level or the slice level. In these embodiments, when the TMRL allowed use flag and the MRL allowed use flag are identifiers of the same level, the decoding order of the MRL allowed use flag is before the TMRL allowed use flag.

[0156] There are a variety of template-based coding and decoding tools in ECM, such as DIMD, TIMD and TMRL, all of which use templates as prediction tools. There can also be a unified identifier to control all template-based technologies. To this end, an embodiment of the present disclosure introduces a sequence-level identifier: the template use permission flag, which is represented by sps_tm_enabled_flag in one example. In this article, as its name implies, the template use permission flag is used to indicate whether the use of the template is allowed, that is, whether the use of template-based coding and decoding tools is allowed. When the value of the template use permission flag indicates that the use of the template is not allowed, template-based coding and decoding tools such as DIMD, TIMD and TMRL cannot be used.

[0157] In an exemplary embodiment of the present disclosure. In this embodiment, the decoding of the TMRL use permission flag depends on the template use permission flag. The method of determining the value of the TMRL use permission flag by decoding includes: decoding the template use permission flag to obtain the value of the template use permission flag; when the value of the template use permission flag indicates that the template is not allowed to be used, skipping the decoding of the TMRL use permission flag, and defaulting the value of the TMRL use permission flag to a value indicating that the TMRL mode is not allowed to be used; when the template use permission flag indicates that the template is allowed to be used, decoding the TMRL use permission flag again to obtain the value of the TMRL use permission flag.

[0158] This embodiment adds a sequence-level TMRL permission flag in the form of a dependency relationship to control whether the TMRL mode is allowed. The relevant syntax is shown in the following table:

[0159] Sequence parameter set RBSP syntax

[0160]

[0161] According to the above table, during decoding, the template enable flag sps_tm_enabled_flag is decoded first. When sps_tm_enabled_flag at the sequence level is 1, indicating that the template is allowed to be used, the TMRL enable flag sps_tmrl_enabled_flag is decoded to determine the value of sps_tmrl_enabled_flag.

[0162] In this embodiment, the TMRL allowed use flag is a sequence-level identifier, and the template allowed use flag is a sequence-level identifier. In another embodiment, the TMRL allowed use flag is an image-level identifier, and the template allowed use flag may be a sequence-level or image-level identifier. In yet another embodiment, the TMRL allowed use flag is a slice-level identifier, and the template allowed use flag may be a sequence-level, image-level, or slice-level identifier. In these embodiments, when the TMRL allowed use flag and the template allowed use flag are identifiers of the same level, the template allowed use flag is decoded before the TMRL allowed use flag.

[0163] In an exemplary embodiment of the present disclosure, the decoding of the TMRL use permission flag depends on the MRL use permission flag and the template use permission flag. The method of determining the value of the TMRL use permission flag by decoding includes: decoding the MRL use permission flag and the template use permission flag to obtain the value of the MRL use permission flag and the value of the template use permission flag; when the value of the MRL use permission flag indicates that the use of the MRL is allowed, and the value of the template use permission flag indicates that the use of the template is allowed, then decoding the TMRL use permission flag to obtain the value of the TMRL use permission flag; when the value of the MRL use permission flag indicates that the use of the MRL is not allowed, or the value of the template use permission flag indicates that the use of the template is not allowed, skipping the decoding of the TMRL use permission flag, and defaulting the value of the TMRL use permission flag to the value indicating that the use of the TMRL mode is not allowed.

[0164] The dependency relationship of this embodiment can be expressed using the syntax in the following table:

[0165] Sequence parameter set RBSP syntax

[0166]

[0167] According to the above table, when decoding, first decode the template enable flag sps_tm_enabled_flag and the MRL enable flag sps_mrl_enabled_flag. When the sps_tm_enabled_flag and sps_mrl_enabled_flag at the sequence level are both 1, that is, the use of the template and MRL is allowed, then decode the TMRL enable flag sps_tmrl_enabled_flag to determine the value of sps_tmrl_enabled_flag.

[0168] In this embodiment, the TMRL allowed use flag is a sequence-level identifier, and the MRL allowed use flag and the template allowed use flag are sequence-level identifiers. In another embodiment, the TMRL allowed use flag is an image-level identifier, and the MRL allowed use flag and the template allowed use flag can be sequence-level or image-level identifiers. In yet another embodiment, the TMRL allowed use flag is a slice-level identifier, and the MRL allowed use flag and the template allowed use flag can be sequence-level or image-level or slice-level identifiers. In these embodiments, when the TMRL allowed use flag, the MRL allowed use flag and the template allowed use flag are identifiers of the same level, the decoding order of the MRL allowed use flag and the template allowed use flag is before the TMRL allowed use flag.

[0169] In ECM, there is a type of syntax element called general constraints information (GCI). GCI includes a series of flags that are used to restrict whether there are some sequence-level identifiers in the code stream. In one example, when a certain flag of GCI is greater than 0 (such as 1), it means that the sequence-level identifier corresponding to the flag in the code stream is restricted and does not need to be decoded. The sequence-level identifier is, for example, the identifier of a certain codec tool (such as MRL, TMRL, DIMD, TIMD, etc.), and if the flag is equal to 0, it means that no restriction is imposed on the sequence-level identifier corresponding to the flag in the code stream, and the sequence-level identifier needs to be decoded.

[0170] In an exemplary embodiment of the present disclosure, whether the TMRL mode is restricted is indicated by a flag bit set in the GCI. In this embodiment, the TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag by decoding includes: decoding the GCI, and determining whether the TMRL mode is restricted based on the value of the flag bit in the GCI used to indicate whether the TMRL mode is restricted; if the TMRL mode is restricted, skipping decoding the TMRL use permission flag and determining that the value of the TMRL use permission flag is a value indicating that the TMRL mode is not allowed to be used; and if the TMRL mode is not restricted, continuing to decode the sequence-level identifier to determine the value of the TMRL use permission flag.

[0171] This embodiment introduces the sequence-level TMRL enabled flag sps_tmrl_enabled_flag, and when the decoding of sps_tmrl_enabled_flag does not depend on the value of the sequence-level MRL enabled flag sps_mrl_enabled_flag, the GCI can be adjusted accordingly to add a flag bit to indicate whether the TMRL mode is restricted, as shown in the following syntax table:

[0172] GCI syntax (General constraints information syntax)

[0173]

[0174] As shown in the table above, this embodiment adds a gci_no_tmrl_constraint_flag to indicate whether the TMRL mode is restricted, that is, whether the value of the TMRL enabled flag, sps_tmrl_enabled_flag, is restricted. When gci_no_tmrl_constraint_flag is 1, there is no need to decode sps_tmrl_enabled_flag, and the default sps_tmrl_enabled_flag is 0, indicating that the TMRL mode is not permitted. When gci_no_tmrl_constraint_flag is 0, there is no restriction on the value of sps_tmrl_enabled_flag, and the relevant sequence-level identifier must be decoded to determine the value of sps_tmrl_enabled_flag.

[0175] The gci_no_mrl_constraint_flag in the table above is used to indicate whether the MRL mode is restricted, that is, whether the value of the MRL enable flag sps_mrl_enabled_flag is restricted. When gci_no_mrl_constraint_flag is 1, the value of sps_mrl_enabled_flag defaults to 0. When gci_no_mrl_constraint_flag is 0, it indicates that there is no restriction on the value of sps_mrl_enabled_flag, and the value of sps_mrl_enabled_flag can be determined through decoding.

[0176] In an exemplary embodiment of the present disclosure, whether the MRL mode is restricted is indicated by a flag bit set in the GCI, and when the MRL mode is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL use permission flag is a sequence-level identifier, and the method of determining the value of the TMRL use permission flag by decoding includes: decoding the GCI, and determining whether the MRL mode is restricted based on the value of the flag bit in the GCI used to indicate whether the MRL mode is restricted; if the MRL mode is restricted, skipping the decoding of the TMRL use permission flag, and determining that the value of the TMRL use permission flag is a value indicating that the TMRL mode is not allowed to be used; and if the MRL mode is not restricted, continuing to decode the sequence-level identifier to determine the value of the TMRL use permission flag.

[0177] Referring to the syntax table of the previous embodiment, when the decoding of the TMRL enable flag sps_tmrl_enabled_flag depends on the value of the MRL enable flag sps_mrl_enabled_flag, whether the MRL mode is restricted and whether the TMRL mode is restricted are determined simultaneously based on the flag gci_no_mrl_constraints_flag in the GCI indicating whether the MRL mode is restricted. When gci_no_mrl_constraints_flag is 1, sps_tmrl_enabled_flag is defaulted to 0, and there is no need to decode sps_tmrl_enabled_flag. When gci_no_mrl_constraints_flag is 0, the value of sps_tmrl_enabled_flag is determined by decoding the identifier of the relevant sequence level.

[0178] In an exemplary embodiment of the present disclosure, a flag set in the GCI indicates whether template use is restricted. When template use is restricted, template-based tools (including the TMRL mode) are also restricted. In this embodiment, the TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag by decoding includes: decoding the GCI, and determining whether template use is restricted based on the value of the flag bit in the GCI used to indicate whether template use is restricted; if template use is restricted, skipping decoding the TMRL use permission flag and determining that the value of the TMRL use permission flag is a value indicating that the TMRL mode is not allowed to be used; and if template use is not restricted, continuing to decode the sequence-level identifier to determine the value of the TMRL use permission flag.

[0179] In this embodiment, when the decoding of sps_tmrl_enabled_flag depends on the value of the template usage enable flag sps_tm_enabled_flag at the sequence level, gci_no_tm_constraints_flag is used to simultaneously determine whether template usage and TMRL mode are restricted, as shown in the following syntax table:

[0180] General constraints information syntax

[0181]

[0182] As shown in the table above, a flag bit, gci_no_tm_constraints_flag, is set in GCI to indicate whether the MRL mode is restricted. When gci_no_tm_constraints_flag is 1, the default sps_tmrl_enabled_flag is 0, and there is no need to decode sps_tmrl_enabled_flag. When gci_no_tm_constraints_flag is 0, the value of sps_tmrl_enabled_flag must be determined by decoding the relevant sequence-level identifier.

[0183] In an exemplary embodiment of the present disclosure, two flags set in the GCI respectively indicate whether the MRL mode is restricted and whether the use of the template is restricted. When the MRL mode or the use of the template is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL allowed use flag is a sequence-level identifier, and the method of determining the value of the TMRL allowed use flag by decoding includes: decoding the GCI, determining whether the MRL mode is restricted according to the value of the flag used to indicate whether the MRL mode is restricted in the GCI, and determining whether the use of the template is restricted according to the value of the flag used to indicate whether the use of the template is restricted in the GCI; in the case that the MRL mode is restricted or the use of the template is restricted, skipping the decoding of the TMRL allowed use flag, and determining that the value of the TMRL allowed use flag is a value indicating that the use of the TMRL mode is not allowed; in the case that neither the MRL mode nor the use of the template is restricted, continuing to decode the sequence-level identifier to determine the value of the TMRL allowed use flag.

[0184] In the above embodiment, when continuing to decode the sequence-level identifier to determine the value of the TMRL permission flag, the TMRL permission flag can be independently decoded according to the above embodiment, or decoded relying on the decoding method of the MRL permission flag and / or the template permission flag.

[0185] The TMRL allowed use flag in the above embodiment is an identifier at the sequence level. In other embodiments, when the TMRL allowed use flag is an identifier at the picture level or slice level, a corresponding flag bit may also be set in the GCI to indicate whether the TMRL mode is restricted (that is, to indicate whether there is a TMRL allowed use flag at the picture level or slice level in the code stream). If the corresponding flag bit indicates that the TMRL mode is restricted, the value of the TMRL allowed use flag is determined to be a value indicating that the TMRL mode is not allowed to be used, and the decoding of the TMRL allowed use flag is skipped. If the corresponding flag bit indicates that the TMRL mode is not restricted, when the TMRL allowed use flag is an identifier at the picture level, the sequence level and picture level identifiers may be continued to be decoded to determine the value of the TMRL allowed use flag; when the TMRL allowed use flag is an identifier at the slice level, the sequence level, picture level, and slice level identifiers may be continued to be decoded to determine the value of the TMRL allowed use flag.

[0186] In an exemplary embodiment of the present disclosure, a TMRL permission flag is used, combined with whether the current block uses the TIMD mode, to control the switching between the traditional MRL mode and the evolved TMRL mode for decoding. Whether the current block uses the TIMD mode can be determined by decoding the TIMD mode flag (intra_timd_flag). The method also includes: determining whether the current block uses the TIMD mode; if the current block uses the TIMD mode, decoding the MRL index of the current block and skipping the decoding of the TMRL mode syntax element of the current block; if the current block does not use the TIMD mode, decoding the TMRL mode syntax element of the current block and skipping the decoding of the MRL index of the current block.

[0187] In an exemplary embodiment of the present disclosure, the decoding of the TMRL mode syntax element of the current block includes: decoding the TMRL mode flag of the current block to obtain the value of the TMRL mode flag, where the TMRL mode flag is used to indicate whether the current block uses the TMRL mode; determining whether the current block uses the TMRL mode based on the value of the TMRL mode flag: if it is determined that the TMRL mode is used, decoding the TMRL mode index of the current block; if the TMRL mode is not used, skipping the decoding of the TMRL mode index of the current block. After decoding the TMRL mode index of the current block, this embodiment further includes: constructing a candidate list of the TMRL mode of the current block; determining a combination of the extended reference row and the intra-frame prediction mode selected for the current block based on the TMRL mode index and the candidate list; and predicting the current block based on the selected combination to obtain a prediction value of the current block.

[0188] The changes in CU-level syntax elements related to this embodiment are shown in the following table:

[0189]

[0190]

[0191]

[0192] Different from the syntax table of the aforementioned embodiment, whether the block-level TMRL mode flag cu_tmrl_flag is decoded in this embodiment no longer depends solely on whether the MRL mode is allowed (sps_mrl_enabled_flag is 1), the current block is not at the upper edge of the CTU (((y0%CtbSizeY)>0) and the current block does not use the TIMD mode (intra_timd_flag is 0, i.e., intra_timd_flag), but also depends on whether the TMRL mode is allowed (sps_tmrl_enabled_flag is 1). Whether to decode cu_ The tmrl_flag may also depend on other modes that cannot be used in combination with the TMRL mode not being used. When the MRL mode is enabled (sps_mrl_enabled_flag is 1), the current block is not at the upper edge of the CTU (((y0%CtbSizeY)>0), and the current block uses TIMD mode (intra_timd_flag is 0) and the TMRL mode is not allowed (sps_tmrl_enabled_flag is 0), the multiple reference row index intra_luma_ref_idx is decoded.

[0193] In this embodiment, the combination of the extended reference line and the intra-frame prediction mode can be the original combination of the extended reference line and the intra-frame prediction mode, or it can be the corresponding fused combination obtained by performing IPF on the original combination including the predetermined angle mode. For example, if an original combination includes an extended reference line and a predetermined angle mode, the fused combination corresponding to the original combination refers to the combination of the extended reference line and the predetermined angle mode, and another reference line and the predetermined angle mode. When predicting the current block based on the fused combination, the current block is predicted based on the extended reference line and the predetermined angle mode to obtain a first prediction result, and the current block is predicted based on the other reference line and the predetermined angle mode to obtain a second prediction result, and the weighted sum of the first prediction result and the second prediction result is used as the prediction value of the current block.

[0194] The predetermined angle pattern may include all angle patterns, or some angle patterns, for example, angle patterns other than integer slope patterns, or angle patterns other than angle patterns of -45°, 0°, 45°, 90°, and 135°. Another reference row in the fused combination may be an adjacent row (either an inner adjacent row or an outer adjacent row) of the extended reference row in the corresponding original combination, or a reference row with an index of 0.

[0195] For example, if the reference behavior indexes of the extended reference behaviors involved in the combination are {1, 3, 5, 7, 12}, and the original combination is a combination of a reference row with index 1 and a certain angle mode, the corresponding fused combination includes the combination of the reference row with index 1 and the angle mode, and the combination of the reference row with index 2 (or index 0) and the angle mode. When predicting the current block based on this fused combination, the prediction result of the reference row with index 1 and the angle mode, and the prediction result of the reference row with index 2 (or index 0) and the angle mode are weighted to obtain the prediction value of the current block.

[0196] In an example of this embodiment, constructing a candidate list of TMRL modes for the current block includes:

[0197] Obtain N×M original combinations of extended reference lines and intra prediction modes according to the N extended reference lines and M intra prediction modes of the current block;

[0198] Performing intra-frame prediction fusion IPF according to the original combination including the predetermined angle mode in the N×M original combinations to obtain a corresponding fusion combination;

[0199] Predicting the template area of ​​the current block according to the N×M original combinations and the obtained fusion combination, and calculating the error between the reconstructed value of the template area and the predicted value;

[0200] In order of the errors from small to large, K combinations corresponding to the errors are filled into a candidate list of the TMRL mode of the current block, where K, N, and M are positive integers and 1≤K≤N×M.

[0201] In another example of this embodiment, constructing a candidate list of TMRL modes for the current block includes:

[0202] Obtain N×M original combinations of extended reference lines and intra prediction modes according to the N extended reference lines and M intra prediction modes of the current block;

[0203] Predicting the template area of ​​the current block respectively according to the N×M original combinations, and calculating the error between the reconstructed value of the template area and the predicted value;

[0204] For each combination of the K original combinations with the smallest errors including the predetermined angle mode, determine whether fusion is required; if fusion is required, perform IPF on the original combination to obtain the corresponding fusion combination, and fill the fusion combination into the candidate list; if fusion is not required, fill the original combination into the candidate list, K, N, M are positive integers, 1≤K≤N×M.

[0205] In this example, when determining whether fusion is required for each combination, the template area of ​​the current block is predicted according to the fusion combination corresponding to the original combination, and the error between the reconstructed value of the template area and the predicted value is calculated. When the error corresponding to the original combination is greater than the error corresponding to the fusion combination, it is determined that fusion is required; when the error corresponding to the original combination is less than or equal to the error corresponding to the fusion combination, it is determined that fusion is not required.

[0206] In another example of this embodiment, constructing a candidate list of TMRL modes for the current block includes:

[0207] Obtain N×M original combinations of extended reference lines and intra prediction modes according to the N extended reference lines and M intra prediction modes of the current block;

[0208] Performing a fusion process on the N×M original combinations, the fusion process comprising: for each original combination including a predetermined angle pattern in the N×M original combinations, when a set condition is met, replacing the original combination with a corresponding fusion combination obtained by performing IPF on the original combination;

[0209] Predicting the template area of ​​the current block according to the N×M combinations after the fusion process, and calculating the error between the reconstructed value of the template area and the predicted value;

[0210] In order of the errors from small to large, K combinations corresponding to the errors are filled into a candidate list of the TMRL mode of the current block; K, N, M are positive integers, 1≤K≤N×M.

[0211] In this example, the set conditions include any one or more of the following conditions: the size of the current block is greater than NxM, where N and M are positive integers; and the intra prediction mode selected by the current block is not an angle mode with an integer slope. However, the conditions that can be set in this example are not limited to these.

[0212] The above three examples of this embodiment provide three ways of combining IPF in constructing the candidate list of TMRL, and the IPF method can be used to improve the coding efficiency of the TMRL mode.

[0213] In the above example of combining IPF of this embodiment, when predicting the current block according to the selected combination to obtain the predicted value of the current block, the following steps are included:

[0214] When the selected combination is the original combination, predicting the current block according to the extended reference row and the intra prediction mode in the original combination to obtain a prediction value of the current block;

[0215] When the selected combination is a fusion combination, the weighted sum of the first prediction result and the second prediction result is used as the prediction value of the current block; wherein, the first prediction result is the result of predicting the current block according to the extended reference row and the predetermined angle pattern in the original combination corresponding to the fusion combination, and the second prediction result is the result of predicting the current block according to another reference row and the predetermined angle pattern in the original combination corresponding to the fusion combination. As described above, the another reference row may be adjacent to the extended reference row in the original combination corresponding to the fusion combination, or be the reference row with index 0.

[0216] In this example, the weighted sum of the first prediction result and the second prediction result can be calculated according to the following formula:

[0217] p fusion =(w a p a +w b p b )>>shift; or

[0218] p<000002"=(w a p a +w b p b +offset)>>shift <00004"Wherein, p a is the first prediction result, p b is the second prediction result, p fusion is the final prediction result of the current block, w a is the weight of p<"w b is the weight of p b ;

[0220] Wherein, w a +w b =(1<<shift), offset=1<<(shift - 1), shift≥1, shift and offset are set parameters.

[0221] Generally, the sum of the two weights used for weighting the two prediction results is 1. To avoid decimal operations, w a and w b in the above formula are the weights amplified by left shift operation. Therefore, w a +w b =(1<<shift). The size of the prediction value is restored by right shift operation in the formula. ">>" in the formula is the right shift symbol, and "<<" is the left shift symbol. The number on the right side of the symbol is the number of bits for right shift or left shift.

[0222] An embodiment of the present disclosure further provides a video encoding method, which is applied to an encoder, as shown in FIG11 , and includes:

[0223] Step 510, determining the value of the TMRL permission flag;

[0224] Step 520, when encoding the current block: when the value of the TMRL use permission flag indicates that the TMRL mode is allowed to be used, the encoding of the TMRL mode syntax elements of the current block is allowed; when the value of the TMRL use permission flag indicates that the TMRL mode is not allowed to be used, the encoding of the TMRL mode syntax elements of the current block is skipped.

[0225] This embodiment controls the use of the TMRL mode and the encoding of syntax elements through the value of the TMRL use permission flag, which can improve the flexibility and adaptability of the use of the TMRL mode. As an example, when the hardware does not support the TMRL mode, the value of the TMRL use permission flag can be set to 0, indicating that the TMRL mode is not allowed to be used. If the hardware supports the TMRL mode, the value of the TMRL use permission flag can be set to 1, indicating that the TMRL mode is allowed to be used.

[0226] In an exemplary embodiment of the present disclosure, the TMRL allowed use flag is a sequence-level identifier. In other embodiments, the TMRL allowed use flag may be an image-level or slice-level identifier.

[0227] In an exemplary embodiment of the present disclosure, the TMRL use permission flag can be encoded separately and does not depend on other flags. In this case, the value of the TMRL use permission flag can be determined based on configuration information or set conditions, and the TMRL use permission flag can be encoded. The above configuration information can be recorded in a configuration file. The above set conditions can be conditions set in addition to other flags, such as image size, image quality requirements, transmission bandwidth, available computing resources, etc. The TMRL use permission flag is a 1-bit identifier with a value of 0 or 1. During encoding, the value of the TMRL use permission flag can be directly written into the code stream.

[0228] In an exemplary embodiment of the present disclosure, the encoding of the TMRL use permission flag depends on the MRL use permission flag. The determination of the value of the TMRL use permission flag includes: determining the value of the MRL use permission flag; in the case where the value of the MRL use permission flag indicates that the use of MRL is not allowed, skipping the encoding of the TMRL use permission flag, and defaulting the value of the TMRL use permission flag to a value indicating that the use of the TMRL mode is not allowed; in the case where the MRL use permission flag indicates that the use of MRL is allowed, determining the value of the TMRL use permission flag according to the configuration information or set conditions, and encoding the TMRL use permission flag. In this embodiment, the value of the MRL use permission flag can be determined according to the configuration information or set conditions.

[0229] In this embodiment, the TMRL allowed use flag is an identifier at the sequence level, and the MRL allowed use flag is an identifier at the sequence level. In another embodiment, the TMRL allowed use flag is an identifier at the image level, and the MRL allowed use flag may be an identifier at the sequence level or the image level. In yet another embodiment, the TMRL allowed use flag is an identifier at the slice level, and the MRL allowed use flag may be an identifier at the sequence level or the image level or the slice level. In these embodiments, when the TMRL allowed use flag and the MRL allowed use flag are identifiers of the same level, the coding order of the MRL allowed use flag is located before the TMRL allowed use flag.

[0230] In an exemplary embodiment of the present disclosure, the encoding of the TMRL use permission flag depends on the template use permission flag. Determining the value of the TMRL use permission flag includes: determining the value of the template use permission flag; if the value of the template use permission flag indicates that the template is not allowed to be used, skipping the encoding of the TMRL use permission flag and defaulting the value of the TMRL use permission flag to a value indicating that the TMRL mode is not allowed to be used; if the template use permission flag indicates that the template is allowed to be used, determining the value of the TMRL use permission flag based on configuration information or set conditions, and encoding the TMRL use permission flag. The value of the template use permission flag can be determined based on configuration information or set conditions.

[0231] In this embodiment, the TMRL allowed use flag is a sequence-level identifier, and the template allowed use flag is a sequence-level identifier. In another embodiment, the TMRL allowed use flag is an image-level identifier, and the template allowed use flag may be a sequence-level or image-level identifier. In yet another embodiment, the TMRL allowed use flag is a slice-level identifier, and the template allowed use flag may be a sequence-level, image-level, or slice-level identifier. In these embodiments, when the TMRL allowed use flag and the template allowed use flag are identifiers of the same level, the template allowed use flag is encoded before the TMRL allowed use flag.

[0232] In an exemplary embodiment of the present disclosure, the encoding of the TMRL use permission flag depends on the template use permission flag and the template use permission flag. The determination of the value of the TMRL use permission flag includes: determining the value of the MRL use permission flag and the value of the template use permission flag; when the value of the MRL use permission flag indicates that the use of the MRL is allowed, and the value of the template use permission flag indicates that the use of the template is allowed, then determining the value of the TMRL use permission flag according to the configuration information or the set conditions, and encoding the TMRL use permission flag; when the value of the MRL use permission flag indicates that the use of the MRL is not allowed, or the value of the template use permission flag indicates that the use of the template is not allowed, skipping the encoding of the TMRL use permission flag, and the default value of the TMRL use permission flag is a value indicating that the use of the TMRL mode is not allowed.

[0233] In this embodiment, the TMRL allowed use flag is a sequence-level identifier, and the MRL allowed use flag and the template allowed use flag are sequence-level identifiers. In another embodiment, the TMRL allowed use flag is an image-level identifier, and the MRL allowed use flag and the template allowed use flag can be sequence-level or image-level identifiers. In yet another embodiment, the TMRL allowed use flag is a slice-level identifier, and the MRL allowed use flag and the template allowed use flag can be sequence-level or image-level or slice-level identifiers. In these embodiments, when the TMRL allowed use flag, the MRL allowed use flag and the template allowed use flag are identifiers of the same level, the coding order of the MRL allowed use flag and the template allowed use flag is before the TMRL allowed use flag.

[0234] When the TMRL usage permission flag is a sequence-level identifier, the use of the TMRL mode can be restricted by setting a flag in the GCI. When the value of the flag indicates that the MRL mode is restricted, the TMRL usage permission flag does not need to be encoded.

[0235] In an exemplary embodiment of the present disclosure, a flag is set in the GCI to indicate whether the TMRL mode is restricted, and whether the TMRL mode is restricted does not depend on other flags in the GCI. In this embodiment, the TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag includes: determining whether the TMRL mode is restricted based on the value of the flag in the GCI used to indicate whether the TMRL mode is restricted; if the TMRL mode is restricted, skipping encoding the TMRL use permission flag and determining the value of the TMRL use permission flag to be a value indicating that the TMRL mode is not allowed to be used; and if the TMRL mode is not restricted, continuing to encode the sequence-level identifier to determine the value of the TMRL use permission flag.

[0236] In an exemplary embodiment of the present disclosure, a flag is set in the GCI to indicate whether the MRL mode is restricted, and when the MRL mode is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag includes: determining whether the MRL mode is restricted based on the value of the flag bit in the GCI used to indicate whether the MRL mode is restricted; when the MRL mode is restricted, skipping encoding the TMRL use permission flag and determining the value of the TMRL use permission flag to be a value indicating that the TMRL mode is not allowed to be used; and when the MRL mode is not restricted, continuing to encode the sequence-level identifier to determine the value of the TMRL use permission flag.

[0237] In an exemplary embodiment of the present disclosure, a flag is set in the GCI to indicate whether template use is restricted, and when the MRL mode is restricted or template use is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag includes: determining whether template use is restricted based on the value of the flag bit in the GCI used to indicate whether template use is restricted; if template use is restricted, skipping encoding the TMRL use permission flag and determining the value of the TMRL use permission flag to be a value indicating that the TMRL mode is not allowed to be used; if template use is not restricted, continuing to encode the sequence-level identifier to determine the value of the TMRL use permission flag.

[0238] In an exemplary embodiment of the present disclosure, two flags are set in the GCI to respectively indicate whether the MRL mode is restricted and whether the use of the template is restricted. When the MRL mode is restricted or the use of the template is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL use permission flag is a sequence-level identifier. Determining the value of the TMRL use permission flag includes: determining whether the MRL mode is restricted based on the value of the flag used to indicate whether the MRL mode is restricted in the GCI, and determining whether the use of the template is restricted based on the value of the flag used to indicate whether the use of the template is restricted in the GCI; in the case where the MRL mode is restricted or the use of the template is restricted, skipping the encoding of the TMRL use permission flag and determining the value of the TMRL use permission flag to be a value indicating that the use of the TMRL mode is not permitted; in the case where the MRL mode is not restricted and the use of the template is not restricted, continuing to encode the sequence-level identifier to determine the value of the TMRL use permission flag.

[0239] In the above embodiment, when continuing to encode the sequence-level identifier to determine the value of the TMRL permission flag, the method of independently encoding the TMRL permission flag in the above embodiment can be adopted to determine the value of the TMRL permission flag according to the configuration information or set conditions; or the TMRL permission flag can be encoded using an encoding method that depends on the MRL permission flag and / or the template permission flag to determine the value of the TMRL permission flag.

[0240] In the above embodiment, the TMRL allowed use flag is a sequence-level identifier. In other embodiments, when the TMRL allowed use flag is an image-level or slice-level identifier, a corresponding flag bit may be set in the GCI to indicate whether the TMRL mode is restricted (that is, to indicate whether there is an image-level or slice-level TMRL allowed use flag in the code stream). If the corresponding flag bit indicates that the TMRL mode is restricted, the value of the TMRL allowed use flag is determined to be a value indicating that the TMRL mode is not allowed to be used, and the encoding of the TMRL allowed use flag is skipped. If the corresponding flag bit indicates that the TMRL mode is not restricted, when the TMRL allowed use flag is an image-level identifier, the sequence-level and image-level identifiers may be continued to be encoded to determine the value of the TMRL allowed use flag; when the TMRL allowed use flag is a slice-level identifier, the sequence-level, image-level, and slice-level identifiers may be continued to be encoded to determine the value of the TMRL allowed use flag.

[0241] In an exemplary embodiment of the present disclosure, a TMRL permission flag is used, combined with whether the current block uses the TIMD mode, to control the switching between the encoding of the traditional MRL mode and the evolved TMRL mode. Whether the current block uses the TIMD mode can be determined by the value of the TIMD mode flag (intra_timd_flag). In this embodiment, when the value of the TMRL permission flag indicates that the TMRL mode is allowed to be used, the method further includes: determining whether the current block uses the TIMD mode; if the current block uses the TIMD mode, encoding the MRL index of the current block and skipping the encoding of the TMRL mode syntax element of the current block; if the current block does not use the TIMD mode, encoding the TMRL mode syntax element of the current block and skipping the encoding of the MRL index of the current block.

[0242] In an exemplary embodiment of the present disclosure, the encoding of the TMRL mode syntax element of the current block includes: constructing a candidate list of the TMRL mode of the current block; selecting a combination of a reference row and an intra-frame prediction mode for the current block through rate-distortion optimization; and, when the encoding condition of the TMRL mode of the current block is met, encoding the TMRL mode flag of the current block to indicate that the current block uses the TMRL mode, and encoding the TMRL mode index of the current block to indicate the position of the selected combination in the candidate list; wherein the encoding condition includes at least: the selected combination is in the candidate list (at this time, the selected reference row is an extended reference row).

[0243] In an example of this embodiment, the encoding conditions also include any one or more of the following: the current block is a block in a luma frame; the current block is not located at the upper boundary of the coding tree unit CTU; the current block allows the use of multiple reference lines MRL; the current block allows the use of templates; the size of the current block meets the size requirements for the current block using the TMRL mode; the aspect ratio of the current block meets the requirements for the aspect ratio of the current block using the TMRL mode.

[0244] In an example of this embodiment, when constructing the candidate list of TMRL, the original combination of the extended reference line and the angle mode can be subjected to IPF to obtain the corresponding fused combination, and the fused combination is filled into the candidate list. At this time, the combination of the reference line and the intra-frame prediction mode can be a combination of the reference line with index 0 and the intra-frame prediction mode, the original combination of the extended reference line and the intra-frame prediction mode, or the fused combination of the extended reference line and the intra-frame prediction mode obtained by performing IPF on the original combination. The method of filling the fused combination into the candidate list can adopt any method of using IPF to construct the candidate list of the TMRL mode of the current block in the aforementioned embodiment, and will not be repeated here.

[0245] An embodiment of the present disclosure further provides a code stream, wherein the code stream is generated by the video encoding method described in any embodiment of the present disclosure.

[0246] An embodiment of the present disclosure further provides a video decoding device, as shown in FIG12 , comprising a processor 71 and a memory 73 storing a computer program, wherein the processor 71 can implement the video decoding method described in any embodiment of this document when executing the computer program.

[0247] An embodiment of the present disclosure further provides a video encoding device, including a processor and a memory storing a computer program, wherein the processor can implement the video encoding method described in any embodiment of the present disclosure when executing the computer program.

[0248] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0249] An embodiment of the present disclosure further provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the video decoding method described in any embodiment of the present disclosure, or implements the video encoding method described in any embodiment of the present disclosure.

[0250] The processor of the above-mentioned embodiment of the present disclosure may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a microprocessor, etc., or other conventional processors; the processor may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), discrete logic or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or other equivalent integrated or discrete logic circuits, or a combination of the above devices. That is, the processor of the above-mentioned embodiment may be any processing device or device combination that implements the various methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure. If the embodiments of the present disclosure are partially implemented in software, the instructions for the software may be stored in a suitable non-volatile computer-readable storage medium, and one or more processors may be used to execute the instructions in hardware to implement the methods of the embodiments of the present disclosure. The term "processor" used herein may refer to the above-mentioned structure or any other structure suitable for implementing the technology described herein.

[0251] In one or more exemplary embodiments above, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium that facilitates the transfer of a computer program from one place to another, such as according to a communication protocol. In this way, a computer-readable medium may generally correspond to a non-transitory tangible computer-readable storage medium or a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in this disclosure. A computer program product may include a computer-readable medium.

[0252] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may also be referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient (transient) media, but rather refer to non-transient tangible storage media. As used herein, disk and optical disk include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, or Blu-ray disc, among others, where disks typically reproduce data magnetically, while optical discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0253] In some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Also, the techniques may be fully implemented in one or more circuits or logic elements.

[0254] The technical solutions of the embodiments of the present disclosure can be implemented in a wide variety of devices or equipment, including wireless mobile phones, integrated circuits (ICs), or a group of ICs (e.g., chipsets). Various components, modules, or units are described in the embodiments of the present disclosure to emphasize the functional aspects of the devices configured to perform the described techniques, but they do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units can be combined in a codec hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

Claims

1. A video decoding method, applied to a decoder, comprising: Determine the value of the template-based multiple reference line intra prediction TMRL permission flag by decoding; When decoding the current block: When the value of the TMRL usage permission flag indicates that the TMRL mode is permitted, decoding of the TMRL mode syntax element of the current block is permitted; When the value of the TMRL usage permission flag indicates that the TMRL mode is not allowed to be used, decoding of the TMRL mode syntax element of the current block is skipped.

2. The method according to claim 1, wherein The TMRL allows the use of flags that are sequence-level identifiers, picture-level identifiers, or slice-level identifiers.

3. The method according to claim 1, wherein Determining the value of the TMRL use permission flag by decoding includes: obtaining the value of the TMRL use permission flag by decoding the TMRL use permission flag.

4. The method according to claim 1, wherein Determining the value of the TMRL permission flag by decoding includes: Decoding the MRL permission flag to obtain a value of the MRL permission flag; In the case where the value of the MRL use permission flag indicates that the use of MRL is not allowed, the decoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is defaulted to a value indicating that the use of the TMRL mode is not allowed; In the case that the MRL use permission flag indicates that the use of MRL is permitted, the TMRL use permission flag is decoded to obtain the value of the TMRL use permission flag.

5. The method according to claim 1, wherein Determining the value of the TMRL permission flag by decoding includes: Decoding a template use permission flag to obtain a value of the template use permission flag; In the case where the value of the template use permission flag indicates that the template is not allowed to be used, the decoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is defaulted to a value indicating that the TMRL mode is not allowed to be used; In the case that the template use permission flag indicates that the template is allowed to be used, the TMRL use permission flag is decoded to obtain the value of the TMRL use permission flag.

6. The method of claim 1, wherein: Determining the value of the TMRL permission flag by decoding includes: Decoding the MRL use permission flag and the template use permission flag to obtain the value of the MRL use permission flag and the value of the template use permission flag; If the value of the MRL use permission flag indicates that the MRL is allowed to be used, and the value of the template use permission flag indicates that the template is allowed to be used, then decoding the TMRL use permission flag to obtain the value of the TMRL use permission flag; When the value of the MRL use permission flag indicates that the use of MRL is not allowed, or the value of the template use permission flag indicates that the use of template is not allowed, the decoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is defaulted to a value indicating that the use of TMRL mode is not allowed.

7. The method of claim 1, wherein: The TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag by decoding includes: Decoding the general restriction information GCI, and determining whether the TMRL mode is restricted according to the value of the flag bit in the GCI indicating whether the TMRL mode is restricted; In the case where the TMRL mode is restricted, decoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is determined to be a value indicating that the TMRL mode is not allowed to be used; In the case where the TMRL mode is not restricted, the sequence level identifier is decoded to determine the value of the TMRL usage permission flag.

8. The method of claim 1, wherein: The TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag by decoding includes: Decoding the GCI and determining whether the MRL mode is restricted based on the value of the flag bit in the GCI indicating whether the MRL is restricted; In the case where the MRL mode is restricted, the decoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is determined to be a value indicating that the TMRL mode is not allowed to be used; In the case where the MRL mode is not restricted, the sequence level identifier is decoded to determine the value of the TMRL allowed flag.

9. The method of claim 1, wherein: The TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag by decoding includes: Decoding the GCI, and determining whether the use of the template is restricted according to the value of the flag bit in the GCI indicating whether the use of the template is restricted; In the case where the use of the template is restricted, the decoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is determined to be a value indicating that the use of the TMRL mode is not permitted; In the case where template usage is not restricted, the sequence-level identifier is decoded to determine the value of the TMRL usage permission flag.

10. The method of claim 1, wherein: The TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag by decoding includes: Decoding the GCI, determining whether the MRL mode is restricted based on a value of a flag bit in the GCI indicating whether the MRL mode is restricted, and determining whether template usage is restricted based on a value of a flag bit in the GCI indicating whether template usage is restricted; In the case where the MRL mode is restricted or the template use is restricted, the decoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is determined to be a value indicating that the TMRL mode is not allowed to be used; In the case where the MRL mode is not restricted and the template usage is not restricted, the sequence level identifier is decoded to determine the value of the TMRL usage permission flag.

11. The method of claim 1, wherein: In a case where the value of the TMRL use permission flag indicates that the TMRL mode is allowed to be used, the method further includes: Determining whether the current block uses a template-based intra prediction mode derived TIMD mode; When the current block uses the TIMD mode, decode the multi-reference row MRL index of the current block and skip decoding the TMRL mode syntax elements of the current block; When the current block does not use the TIMD mode, the TMRL mode syntax element of the current block is decoded and the decoding of the MRL index of the current block is skipped.

12. The method of claim 1, wherein: The decoding of the TMRL mode syntax elements of the current block includes: Decoding a TMRL mode flag of a current block to obtain a value of the TMRL mode flag; the TMRL mode flag is used to indicate whether the current block uses the TMRL mode; determining whether the current block uses the TMRL mode according to the value of the TMRL mode flag; if it is determined that the TMRL mode is used, decoding the TMRL mode index of the current block; if it is determined that the TMRL mode is not used, skipping decoding the TMRL mode index of the current block; After decoding the TMRL mode index of the current block, the method further includes: constructing a candidate list of the TMRL mode of the current block; determining a combination of an extended reference row and an intra-frame prediction mode selected for the current block based on the TMRL mode index and the candidate list; and predicting the current block based on the selected combination to obtain a prediction value of the current block.

13. The method of claim 12, wherein: The candidate list of the TMRL mode of the current block is constructed, including: Obtain N×M original combinations of extended reference lines and intra prediction modes according to the N extended reference lines and M intra prediction modes of the current block; Performing intra-frame prediction fusion IPF according to the original combination including the predetermined angle mode in the N×M original combinations to obtain a corresponding fusion combination; Predicting the template area of ​​the current block according to the N×M original combinations and the obtained fusion combination, and calculating the error between the reconstructed value of the template area and the predicted value; In order of the errors from small to large, K combinations corresponding to the errors are filled into a candidate list of the TMRL mode of the current block, where K, N, and M are positive integers and 1≤K≤N×M.

14. The method of claim 12, wherein: The candidate list of the TMRL mode of the current block is constructed, including: Obtain N×M original combinations of extended reference lines and intra prediction modes according to the N extended reference lines and M intra prediction modes of the current block; Predicting the template area of ​​the current block respectively according to the N×M original combinations, and calculating the error between the reconstructed value of the template area and the predicted value; For each original combination including the predetermined angle pattern in the K original combinations with the smallest error, determine whether fusion is required; if fusion is required, perform IPF on the original combination to obtain a corresponding fusion combination, and fill the fusion combination into the candidate list; if fusion is not required, fill the original combination into the candidate list, K, N, M are positive integers, 1≤K≤N×M.

15. The method of claim 12, wherein: The determining whether fusion is required for each original combination of the predetermined angle pattern among the K original combinations with the smallest errors includes: The template area of ​​the current block is predicted according to the fusion combination corresponding to the original combination, and the error between the reconstructed value of the template area and the predicted value is calculated. When the error corresponding to the original combination is greater than the error corresponding to the fusion combination, it is determined that fusion is required; when the error corresponding to the original combination is less than or equal to the error corresponding to the fusion combination, it is determined that fusion is not required.

16. The method of claim 12, wherein: The candidate list of the TMRL mode of the current block is constructed, including: Obtain N×M original combinations of extended reference lines and intra prediction modes according to the N extended reference lines and M intra prediction modes of the current block; Performing a fusion process on the N×M original combinations, the fusion process comprising: for each original combination including a predetermined angle pattern in the N×M original combinations, when a set condition is met, replacing the original combination with a corresponding fusion combination obtained by performing IPF on the original combination; Predicting the template area of ​​the current block according to the N×M combinations after the fusion process, and calculating the error between the reconstructed value of the template area and the predicted value; In order of the errors from small to large, K combinations corresponding to the errors are filled into a candidate list of the TMRL mode of the current block; K, N, M are positive integers, 1≤K≤N×M.

17. The method of claim 16, wherein: The setting conditions include any one or more of the following conditions: The size of the current block is greater than NxM, where N and M are positive integers; The intra prediction mode selected for the current block is not an integer slope angle mode.

18. The method according to any one of claims 13 to 17, wherein: The step of predicting the current block according to the selected combination to obtain a predicted value of the current block includes: When the selected combination is the original combination, predicting the current block according to the extended reference row and the intra prediction mode in the original combination to obtain a prediction value of the current block; When the selected combination is a fusion combination, the weighted sum of the first prediction result and the second prediction result is used as the prediction value of the current block; wherein, the first prediction result is the result of predicting the current block based on the extended reference row and the predetermined angle pattern in the original combination corresponding to the fusion combination, and the second prediction result is the result of predicting the current block based on another reference row and the predetermined angle pattern in the original combination corresponding to the fusion combination.

19. The method of claim 18, wherein: The predetermined angle pattern includes all angle patterns, or includes other angle patterns except the angle pattern with integer slope; The another reference row is adjacent to the extended reference row in the original combination corresponding to the fused combination, or the another reference row is a reference row with an index of 0.

20. The method of claim 18, wherein: The weighted sum of the first prediction result and the second prediction result is calculated according to the following formula: p fusion =(w a p a +w b p b )>>shift or p fusion =(w a p a +w b p b +offset)>>shift Among them, p a is the first prediction result, p b For the second prediction result, p fusion The final prediction result of the current block, w a For p a The weight, w b For p b The weight of where, w a + w b = (1 << shift), offset = 1 << (shift - 1), shift ≥ 1, and shift and offset are set parameters.

21. A video encoding method, applied to an encoder, comprising: Determine the value of the template-based multiple reference line intra prediction TMRL allowed flag; When encoding the current block: When the value of the TMRL usage permission flag indicates that the TMRL mode is permitted, encoding the TMRL mode syntax element of the current block is permitted; When the value of the TMRL usage permission flag indicates that the TMRL mode is not allowed to be used, encoding of the TMRL mode syntax element of the current block is skipped.

22. The method of claim 21, wherein: The TMRL allows the use of flags that are sequence-level identifiers, picture-level identifiers, or slice-level identifiers.

23. The method of claim 21, wherein: The determining the value of the TMRL use permission flag includes: determining the value of the TMRL use permission flag according to configuration information or setting conditions, and encoding the TMRL use permission flag.

24. The method of claim 21, wherein: The determining the value of the TMRL usage permission flag includes: determining the value of the multiple reference line MRL usage permission flag; When the value of the MRL use permission flag indicates that the MRL is not allowed to be used, the encoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is defaulted to a value indicating that the TMRL mode is not allowed to be used; In the case where the MRL use permission flag indicates that the use of MRL is permitted, the value of the TMRL use permission flag is determined according to configuration information or setting conditions, and the TMRL use permission flag is encoded.

25. The method of claim 21, wherein The determining the value of the TMRL use permission flag includes: determining the value of the template use permission flag; When the value of the template use permission flag indicates that the template is not allowed to be used, the encoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is defaulted to a value indicating that the TMRL mode is not allowed to be used; In the case where the template use permission flag indicates that the template is allowed to be used, the value of the TMRL use permission flag is determined according to configuration information or setting conditions, and the TMRL use permission flag is encoded.

26. The method of claim 21, wherein: Determining the value of the TMRL permission flag includes: Determine the value of the MRL allow-use flag and the value of the template allow-use flag; If the value of the MRL use permission flag indicates that the MRL is allowed to be used, and the value of the template use permission flag indicates that the template is allowed to be used, determining the value of the TMRL use permission flag according to configuration information or set conditions, and encoding the TMRL use permission flag; When the value of the MRL use permission flag indicates that the use of MRL is not allowed, or the value of the template use permission flag indicates that the use of template is not allowed, the encoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is defaulted to a value indicating that the TMRL mode is not allowed.

27. The method of claim 21, wherein: The TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag includes: Determining whether the TMRL mode is restricted according to the value of the flag bit in the GCI indicating whether the TMRL mode is restricted; In the case where the TMRL mode is restricted, skipping encoding of the TMRL use permission flag, and determining that the value of the TMRL use permission flag is a value indicating that the TMRL mode is not allowed to be used; In the case where the TMRL mode is not restricted, the sequence level identifier is continued to be encoded to determine the value of the TMRL allowed use flag.

28. The method of claim 21, wherein The TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag includes: Determining whether the MRL mode is restricted according to the value of the flag bit in the GCI used to indicate whether the MRL is restricted; In the case where the MRL mode is restricted, the encoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is determined to be a value indicating that the TMRL mode is not allowed to be used; In the case where the MRL mode is not restricted, the sequence level identifier is continued to be encoded to determine the value of the TMRL allowed use flag.

29. The method of claim 21, wherein: The TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag includes: Determining whether the use of the template is restricted according to the value of the flag bit in the GCI used to indicate whether the use of the template is restricted; In the case where the use of the template is restricted, the encoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is determined to be a value indicating that the use of the TMRL mode is not allowed; In the case where template usage is not restricted, the sequence-level identifier is continued to be encoded to determine the value of the TMRL allowed usage flag.

30. The method of claim 21, wherein The TMRL use permission flag is a sequence-level identifier, and determining the value of the TMRL use permission flag includes: Determining whether the MRL mode is restricted based on a value of a flag bit in the GCI indicating whether the MRL mode is restricted, and determining whether template usage is restricted based on a value of a flag bit in the GCI indicating whether template usage is restricted; In the case where the MRL mode is restricted or the use of the template is restricted, the encoding of the TMRL use permission flag is skipped, and the value of the TMRL use permission flag is determined to be a value indicating that the TMRL mode is not allowed to be used; In the case where the MRL mode is not restricted and the template usage is not restricted, the sequence level identifier is continued to be encoded to determine the value of the TMRL allowed usage flag.

31. The method of claim 21, wherein: In a case where the value of the TMRL use permission flag indicates that the TMRL mode is allowed to be used, the method further includes: Determining whether the current block uses a template-based intra prediction mode derived TIMD mode; When the current block uses the TIMD mode, encode the MRL index of the current block and skip encoding the TMRL mode syntax elements of the current block; When the current block does not use the TIMD mode, the TMRL mode syntax element of the current block is encoded, and the encoding of the MRL index of the current block is skipped.

32. The method of claim 21 or 31, wherein: The TMRL mode syntax elements for encoding the current block include: Construct a candidate list of TMRL modes for the current block; Select a reference row and intra prediction mode combination for the current block through rate-distortion optimization; When the encoding condition of the TMRL mode of the current block is satisfied, encoding the TMRL mode flag of the current block to indicate that the current block uses the TMRL mode, and encoding the TMRL mode index of the current block to indicate the position of the combination selected by the current block in the candidate list; The encoding condition at least includes: the selected combination is in the candidate list.

33. The method of claim 32, wherein: The encoding conditions also include any one or more of the following: The current block is a block in the luma frame; The current block is not located at the upper boundary of the coding tree unit CTU; The current block allows the use of multiple reference lines MRL; The current block allows the use of templates; The size of the current block meets the size requirement of the current block using the TMRL mode; The aspect ratio of the current block meets the requirement of the TMRL mode for the aspect ratio of the current block.

34. A code stream, wherein The code stream is generated by the video encoding method according to any one of claims 21 to 33.

35. A video decoding device comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it is capable of implementing the video decoding method according to any one of claims 1 to 20.

36. A video encoding device comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it can implement the video encoding method according to any one of claims 21 to 33.

37. A video encoding and decoding system, wherein: It includes the video encoding device as claimed in claim 36 and the video decoding device as claimed in claim 35.

38. A non-transitory computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, it can implement the video decoding method according to any one of claims 1 to 20, or implement the video encoding method according to any one of claims 21 to 33.

Images