Video encoding and decoding method, apparatus and system

Abstract

A method, apparatus, and system for video encoding and decoding utilizes a TMRL availability flag, and when decoding a current block, determines whether encoding or decoding is permitted for a TMRL mode syntax element of the current block according to the value of the TMRL availability flag. By using a high-level syntax element to indicate whether a TMRL mode is used, embodiments of the present invention provide greater flexibility and adaptability in video encoding and decoding mode selection.

Description

[Technical Field]

[0001] FIELD OF THE INVENTION Embodiments of the present invention relate to, but are not limited to, video technology, and more particularly to methods, apparatus and systems for video encoding and decoding. [Background technology]

[0002] Digital video compression technology primarily compresses massive amounts of digital video data for transmission, storage, and other purposes. Current general-purpose video encoding and decoding standards, such as H.266 / Versatile Video Coding (VVC), all employ a block-based hybrid coding framework. Each frame in a video is divided into square largest coding units (LCUs) of the same size (e.g., 128x128, 64x64, etc.). Each LCU can be divided into rectangular coding units (CUs) based on a rule. The coding units may be further divided into prediction units (PUs), transform units (TUs), and other modules. The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and loop filter. The prediction module includes intra-prediction and inter-prediction, which are configured to reduce or remove redundancy within the video. Intrablock prediction is performed using surrounding pixels as a reference, while interblock prediction is performed using spatially neighboring blocks and reference information in other frames. Residual information from the predicted signal is converted into a bitstream by block-by-block transformation, quantization, and entropy coding. These techniques are described in standards and are implemented in various fields related to video compression.

[0003] Although existing digital video compression standards have already achieved significant savings in video data with the proliferation of video on the Internet and the increasing demand for high-definition video, better digital video compression techniques are still needed to reduce the bandwidth and traffic pressure required for digital video transmission. Summary of the Invention

[0004] The following is a general description of the subject matter detailed in this specification, and is not intended to limit the scope of protection of the claims.

[0005] One embodiment of the present invention provides a video decoding method applied to a decoder, the method comprising: determining a value of a template-based multi-reference row intra-prediction TMRL availability flag by decoding; When decrypting the current block, If the value of the TMRL available flag indicates that TMRL mode use is permitted, then permit decoding of the TMRL mode syntax element of the current block; If the value of the TMRL available flag indicates that the use of TMRL mode is not permitted, then decoding of the TMRL mode syntax element of the current block is skipped.

[0006] An embodiment of the present invention further provides a method of video encoding applied to an encoder, the method comprising: determining a value of a template-based multi-reference row intra-prediction TMRL available flag; When encoding the current block, If the value of the TMRL available flag indicates that TMRL mode use is permitted, then permit encoding of the TMRL mode syntax element of the current block; If the value of the TMRL available flag indicates that the use of the TMRL mode is not permitted, the encoding of the TMRL mode syntax element of the current block is skipped.

[0007] An embodiment of the present invention further provides a bitstream, said bitstream being generated by the video encoding method according to any embodiment of the present invention.

[0008] An embodiment of the present invention further provides a video decoding apparatus, the apparatus including a processor and a memory in which a computer program is stored, the processor being capable of implementing the video decoding method according to any embodiment of the present invention when executing the computer program.

[0009] An embodiment of the present invention further provides a video encoding apparatus, the apparatus including a processor and a memory in which a computer program is stored, the processor being capable of implementing the video encoding method according to any embodiment of the present invention when executing the computer program.

[0010] An embodiment of the present invention further provides a video encoding and decoding system, which includes a video encoding apparatus according to any embodiment of the present invention and a video decoding apparatus according to any embodiment of the present invention.

[0011] An embodiment of the present invention further provides a non-transitory computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, can realize a video decoding method according to any embodiment of the present invention or a video encoding method according to any embodiment of the present invention.

[0012] Other aspects will be appreciated upon reading and understanding the drawings and detailed description. [Brief explanation of the drawings]

[0013] The drawings are intended to provide an understanding of the embodiments of the present invention, constitute a part of the specification, and interpret the technical means of the present invention together with the embodiments of the present invention, but do not limit the technical means of the present invention. [Figure 1A] 1 is a schematic diagram of an encoding and decoding system according to an embodiment of the present invention; [Figure 1B] FIG. 1 is a framework diagram of the encoding side according to an embodiment of the present invention. [Figure 1C] FIG. 10 is a framework diagram on the decoding side according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram of an intra prediction mode according to an embodiment of the present invention. [Figure 3] 2 is a schematic diagram of intra-predicted blocks neighboring a current block according to one embodiment of the present invention; [Figure 4] 3 is a schematic diagram of a template and template reference area of ​​a current block according to one embodiment of the present invention; [Figure 5] FIG. 2 is a schematic diagram of multiple reference rows in the vicinity of a current block according to one embodiment of the present invention; [Figure 6] 2 is a flowchart of a video encoding method according to an embodiment of the present invention; [Figure 7] 1 is a flowchart of a method for building a candidate list in a TMRL mode according to an embodiment of the present invention. [Figure 8A] FIG. 2 is a schematic diagram of a template region and extended reference rows near a current block according to one embodiment of the present invention; [Figure 8B] FIG. 10 is a schematic diagram of a template region and extended reference rows near a current block according to another embodiment of the present invention; [Figure 9] 3 is a flowchart of a video decoding method according to an embodiment of the present invention. [Figure 10] 4 is a flowchart of a video decoding method according to another embodiment of the present invention. [Figure 11] 4 is a flowchart of a video encoding method according to another embodiment of the present invention. [Figure 12] 1 is a schematic diagram of a video decoding apparatus according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0014] Although multiple examples are described in the present invention, the descriptions are illustrative and not restrictive, and it will be apparent to those skilled in the art that there are many more examples and embodiments that are covered by the examples described in the present invention.

[0015] In the description of the present invention, words such as "exemplary" or "for example" are intended to represent an example, illustration, or interpretation. Any embodiment described "exemplary" or "for example" in the present invention should not be construed as being preferred or superior to other embodiments. In this specification, "and / or" describes a relationship between related objects and indicates the existence of three types of relationships. For example, A and / or B can represent three cases: the presence of only A, the simultaneous presence of A and B, and the presence of only B. "Plural" means two or more than two. Furthermore, to clearly describe technical means according to embodiments of the present invention, terms such as "first" and "second" are used to distinguish between identical or similar objects having substantially the same functions and actions. As will be understood by those skilled in the art, the terms "first" and "second" do not imply limitations on the number or order of execution, and the terms "first" and "second" do not necessarily refer to different objects.

[0016] In describing representative illustrative embodiments, the specification may have previously presented a method and / or process as a particular sequence of steps. However, if the method or process does not rely on a particular order of steps described herein, the method or process is not limited to the particular order of steps described. As one of ordinary skill in the art will recognize, other order of steps are possible. Therefore, the particular order of steps described in the specification should not be construed as a limitation on the claims. Furthermore, method and / or process claims should not be limited to performing their steps in the order described; as one of ordinary skill in the art will recognize, these orders may vary within the spirit and scope of the embodiments of the present invention.

[0017] The video encoding and decoding method according to an embodiment of the present invention can be applied to various video encoding and decoding standards, such as H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC), AVS (Audio Video coding Standard), and other standards established by MPEG (Moving Picture Experts Group), AOM (Alliance for Open Media), JVET (Joint Video Experts Team), extensions of these standards, or any other customized standards.

[0018] FIG. 1A is a block diagram of a video encoding and decoding system applicable to an embodiment of the present invention. As shown in the figure, the system is divided into an encoding device 1 and a decoding device 2. The encoding device 1 generates a bitstream, and the decoding device 2 can decode the bitstream. The decoding device 2 can receive the bitstream from the encoding device 1 via a link 3. The link 3 includes one or more media or devices that can transfer the bitstream from the encoding device 1 to the decoding device 2. In one example, the link 3 includes one or more communication media that allow the encoding device 1 to directly transmit the bitstream to the decoding device 2. The encoding device 1 modulates the bitstream based on a communication standard (e.g., a wireless communication protocol) and transmits the modulated bitstream to the decoding device 2. The one or more communication media may include wireless and / or wired communication media and may form part of a packet network. In another example, the bitstream may be output to a storage device via an output interface 15, and the decoding device 2 can read the stored data from the storage device by streaming or downloading.

[0019] As shown in the figure, the encoding device 1 includes a data source 11, a video encoding device 13, and an output interface 15. The data source 11 may include a video capture device (e.g., a camera), an archive containing previously captured data, a feed interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. The video encoding device 13 encodes data from the data source 11 and then outputs it to the output interface 15, which may include at least one of a regulator, a modem, and a transmitter. The decoding device 2 includes an input interface 21, a video decoding device 23, and a display device 25. The input interface 21 includes at least one of a receiver and a modem. The input interface 21 can receive a bitstream via link 3 or from a storage device. The video decoding device 23 performs encoding on the received bitstream. The display device 25 is configured to display decoded data. The display device 25 may be integrated with other devices in the decoding device 2 or located independently; the display device 25 is optional for the decoding side. In other examples, the decryption side may include other equipment or devices to which the decrypted data is applied.

[0020] Based on the video encoding and decoding system according to FIG. 1A, various video encoding and decoding methods can be used to realize video compression and decompression.

[0021] 1B is a block diagram of an exemplary video encoding device applicable to embodiments of the present invention. As shown in the drawing, the video encoding device 1000 includes a prediction unit 1100, a partition unit 1101, a residual generation unit 1102 (indicated in the drawing by a circle with a plus sign after the partition unit 1101), a transform processing unit 1104, a quantization unit 1106, an inverse quantization unit 1108, an inverse transform processing unit 1110, a reconstruction unit 1112 (indicated in the drawing by a circle with a plus sign after the inverse transform processing unit 1110), a filter unit 1113, a decoded picture buffer 1114, and an entropy coding unit 1115. Here, the prediction unit 1100 includes an inter prediction unit 1121 and an intra prediction unit 1126, and the decoded picture buffer 1114 may also be referred to as a decoded picture buffer, a decoded image buffer, a decoded picture buffer, or the like. Video encoder 20 may include more, fewer, or different functional components than these examples, eg, transform processing unit 1104, inverse transform processing unit 1110, etc. may be revoked in some cases.

[0022] The partitioning unit 1101 cooperates with the prediction unit 1100 to partition the received video data into slices, coding tree units (CTUs), or other relatively large units. The video data received by the partitioning unit 1101 may be a video sequence including video frames such as I-frames, P-frames, or B-frames.

[0023] The prediction unit 1100 divides the CTU into coding units (CUs) and can perform intra-prediction coding or inter-prediction coding on the CUs. When performing intra-prediction and inter-prediction on a CU, the CU can be divided into one or more prediction units (PUs).

[0024] The inter prediction unit 1121 may perform inter prediction on the PU to generate prediction data for the PU, where the prediction data includes a prediction block for the PU, motion information for the PU, and various syntax elements. The inter prediction unit 1121 may include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit may be configured to perform motion estimation to generate a motion vector, and the motion compensation unit may be configured to obtain or generate a prediction block based on the motion vector.

[0025] The intra prediction unit 1126 may perform intra prediction on the PU to generate prediction data for the PU. The prediction data for the PU may include a prediction block for the PU and various syntax elements.

[0026] The residual generation unit 1102 subtracts the predicted block of the PU formed by dividing the CU based on the original block of the CU to generate a residual block of the CU.

[0027] The transform processing unit 1104 may divide a CU into one or more transform units (TUs), and the prediction unit and the transform unit may be divided differently. A residual block associated with a TU is a sub-block obtained by dividing the residual block of the CU. A coefficient block associated with the TU is generated by applying one or more types of transform to the residual block associated with the TU.

[0028] The quantization unit 1106 can quantize the coefficients in the coefficient block based on the selected quantization parameter, and can adjust the degree of quantization for the coefficient block by adjusting the quantizer parameter (QP).

[0029] The inverse quantization unit 1108 and the inverse transform processing unit 1110 may apply inverse quantization and inverse transform, respectively, to the coefficient block to generate a reconstructed residual block associated with the TU.

[0030] The reconstruction unit 1112 adds the reconstructed residual block and the prediction block generated by the prediction unit 1100 to generate a reconstructed image.

[0031] The filter unit 1113 performs loop filtering on the reconstructed image and stores the filtered reconstructed image as a reference image in the decoded picture buffer 1114. The intra prediction unit 1126 can extract reference images of blocks near the PU from the decoded picture buffer 1114 to perform intra prediction. The inter prediction unit 1121 can perform inter prediction on the PU of the image of the current frame using the reference image of the previous frame buffered in the decoded picture buffer 1114.

[0032] The entropy coding unit 1115 may perform entropy coding operations on the received data (eg, syntax elements, quantized coefficient blocks, motion information, etc.).

[0033] 1C is a block diagram of an exemplary video decoding apparatus applicable to embodiments of the present invention. As shown in the figure, video decoding apparatus 101 includes entropy decoding unit 150, prediction unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158 ​​(denoted in the figure by a circle with a plus sign after inverse transform processing unit 155), filter unit 159, and decoded picture buffer 160. In other embodiments, video decoder 30 may include more or fewer functional components, or may include different functional components, e.g., in some cases, inverse transform processing unit 155 may be omitted.

[0034] The entropy decoding unit 150 can perform entropy decoding on the received bitstream to extract syntax elements, quantized coefficient blocks, and motion information of PUs, etc. The prediction unit 152, the inverse quantization unit 154, the inverse transform processing unit 156, the reconstruction unit 158, and the filter unit 159 can all perform corresponding operations based on the syntax elements extracted from the bitstream.

[0035] Inverse quantization unit 154 may perform inverse quantization on coefficient blocks associated with the quantized TUs.

[0036] Inverse transform processing unit 156 may apply one or more inverse transforms to the dequantized coefficient blocks to generate reconstructed residual blocks of the TUs.

[0037] The prediction unit 152 includes an inter prediction unit 162 and an intra prediction unit 164. If intra prediction coding is used for the PU, the intra prediction unit 164 may determine an intra prediction mode of the PU based on syntax elements decoded from the bitstream, and perform intra prediction based on the determined intra prediction mode and configured reference information of the PU's neighbors obtained from the decoded picture buffer 160 to generate a predictive block of the PU. If inter prediction coding is used for the PU, the inter prediction unit 162 may determine one or more reference blocks of the PU based on the motion information of the PU and corresponding syntax elements, and generate a predictive block of the PU based on the reference block obtained from the decoded picture buffer 160.

[0038] The reconstruction unit 158 ​​may obtain a reconstructed image based on the reconstructed residual block associated with the TU and the prediction block of the PU generated by the prediction unit 152.

[0039] Filter unit 159 may perform loop filtering on the reconstructed image, and the filtered reconstructed image is stored in decoded picture buffer 160. Decoded picture buffer 160 may provide reference images for subsequent motion compensation, intra-prediction, and inter-prediction, and may output the filtered reconstructed image as decoded video data for display on a display device.

[0040] The above-described video encoding device and video decoding device can perform the following basic encoding and decoding flows. On the encoding side, an image of one frame is divided into blocks, and intra prediction, inter prediction, or other algorithms are performed on the current block to generate a predicted block of the current block. The predicted block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain quantized coefficients. The quantized coefficients are then entropy coded to generate a bitstream. On the decoding side, intra prediction or inter prediction is performed on the current block to generate a predicted block of the current block. Meanwhile, the quantized coefficients obtained by decoding the bitstream are inversely quantized and inversely transformed to obtain a residual block. The predicted block and the residual block are added to obtain a reconstructed block. A reconstructed image is constructed using the reconstructed block. A loop filter is applied to the reconstructed image based on the image or block to obtain a decoded image. Similarly, the encoding side obtains a decoded image through operations similar to those on the decoding side. The decoded image obtained by the encoding side is usually also called a reconstructed image. The decoded image can be used as a reference frame for inter prediction of a subsequent frame. The mode information and parameter information such as block partition information, prediction, transform, quantization, entropy coding, loop filter, etc. determined by the encoding side may be written into the bitstream as needed. The decoding side determines the same mode information and parameter information such as block partition information, prediction, transform, quantization, entropy coding, loop filter, etc. as the encoding side by decoding the bitstream or analyzing existing information, thereby ensuring that the decoded image obtained at the encoding side is the same as the decoded image obtained at the decoding side.

[0041] Although the above example illustrates a block-based hybrid coding framework, embodiments of the present invention are not limited thereto. As technology evolves, one or more modules in the framework and one or more steps in the process may be replaced or improved.

[0042] In this specification, the current block may be a unit of block-level encoding and decoding, such as a current coding unit (current CU) or a current prediction unit (current PU) in a current image.

[0043] When the encoding side performs intra prediction, it usually performs prediction on the current block using various angular modes and non-angular modes to obtain a predicted block, selects the most appropriate intra prediction mode for the current block based on rate-distortion information calculated from the predicted block and the original block, encodes the intra prediction mode, and transmits it to the decoding side as a bitstream. The decoding side obtains the intra prediction mode selected for the current block by decoding, and performs intra prediction on the current block according to the intra prediction mode. In this specification, the reference row and intra prediction mode selected for the current block are also referred to as the reference row and intra prediction mode selected by the current block.

[0044] In VVC and ECM, many traditional intra prediction modes predict a current block using already reconstructed information about the current block's neighbors. These intra prediction modes include planar mode (i.e., Planar mode, with mode index 0), average mode (i.e., DC mode, with mode index 1), and 65 angular prediction modes (with mode indexes 2 to 66). Figure 2 shows the angular directions of angular prediction modes with mode indexes 2 to 66. In this specification, angular prediction modes are also abbreviated as angular modes.

[0045] Since rectangular prediction blocks are introduced in VVC, for rectangular blocks, the angular directions of some of the angle modes with indexes 2 to 66 can be replaced with wider angle directions. As shown in the drawing, the angle modes with indexes -14 to -1 and 67 to 80 are angle modes obtained by replacing the wide angles. The selection of these angle modes does not need to be indicated by flag bits, but instead is determined by the correspondence between the shape of the current block and the index (2 to 66) of the prediction mode selected by the current block. After wide angle mode matching is completed, each angle prediction mode predModeIntra (-14 to 80) has one angle value intraPredAngle, as shown in the table below. The angle value of the angle mode is used for subsequent angle prediction.

[0046] [Table 1]

[0047] In this specification, every angle mode predModeIntra corresponds to one angle, and the angle of each angle prediction mode is the angle in the Cartesian coordinate system of the line segment corresponding to the angle prediction mode in Fig. 1. For example, the angle mode with index number 34 has an angle value intraPredAngle of -32, and the angle is -45°, 45° or 135°, which is related to the 0° direction defined by the Cartesian coordinate system.

[0048] In this specification, unless otherwise specified, the intra prediction mode refers to a traditional intra prediction mode including a planar mode, a DC mode, and an angular mode.

[0049] According to statistical characteristics, the closer a pixel region is to the current block, the more likely it is to select the same intra prediction mode as the current block. Based on this characteristic, HEVC, VVC, and the Enhanced Compression Model (ECM) all adopt the most probable mode (MPM) technology. ECM is reference software that integrates various new tools based on the VTM-10.0 reference software to further improve encoding and decoding performance.

[0050] The MPM constructs an MPM list and then inserts into it the six intra-prediction modes most likely to be selected by the current block. If the intra-prediction mode selected by the current block is in the MPM list, only its index number (which requires only 3 bits) needs to be coded. If the intra-prediction mode selected by the current block is not in the MPM list but is among 61 non-MPM modes, the intra-prediction mode is coded using a truncated binary code (TBC) in the entropy coding step.

[0051] In VVC, regardless of whether multiple reference lines (MRL) and intra sub-partitions (ISP) are applied, the MPM list always has six prediction modes. The MPM in the ECM also includes an MPM and a secondary MPM (Secondary MPM), which use lists of length 6 and 16, respectively. Of the six modes in the MPM list, the planar mode is always filled in the first position of the MPM, and the remaining five positions are filled in order according to a set step until all five positions are filled. The remaining modes are automatically placed in the secondary MPM. The secondary MPM list can be configured with several other primary angle modes except for the intra prediction modes in the MPM. Because the encoding and decoding order of the MPM flag (mpm_flag) comes after the MRL mode, the encoding and decoding of the MPM in the ECM must rely on the MRL flag bit. When the current block does not use MRL mode, the MPM flag needs to be decoded to determine whether the current block uses MPM or not; on the other hand, when the current block uses MRL mode, there is no need to decode the MPM flag, and it is determined by default that the current block uses MPM.

[0052] Template-based intra mode derivation (TIMD) and decoder-side intra mode derivation (DIMD) are two intra prediction techniques for luma frames delivered in the ECM reference software. These techniques derive the intra prediction mode of the current block at the decoding side based on the reconstructed pixel values ​​of the current block's neighbors, thus saving bits for encoding the intra prediction mode index.

[0053] In ECM, as shown in FIG. 4, the left-side adjacent region and the upper-side adjacent region of a current block (e.g., a current CU) 11 constitute the template region 12 of the current block. Here, the left-side adjacent region is referred to as the left template region (abbreviated as left template), and the upper-side adjacent region is referred to as the upper template region or the upper template region (abbreviated as upper template). A template reference region 13 is disposed outside (referring to the left and upper sides) of the template region 12, and exemplary dimensions and positions of each region are shown in the drawing. In one example, the width L1 of the left template and the height L2 of the upper template are both 4. The template reference region 13 may be a row adjacent to the top or a column adjacent to the left of the template region.

[0054] TIMD assumes that the distribution characteristics of the current block and its template region are consistent. Using the reconstructed values ​​of the template reference region as the reconstructed values ​​of the reference row, all intra prediction modes in MPM and Secondary MPM are traversed to predict the template region to obtain a prediction result. Next, the sum of absolute transformed differences (SATD) between the reconstructed values ​​in the template region and the prediction results of each mode (predicted values ​​in the template region) is calculated, and the TIMD mode of the current block is determined. The decoding side derives the TIMD mode using the same method. If the sequence allows the use of TIMD and the intra prediction mode selected by the current block is TIMD mode, only one flag bit is required to indicate that the current block uses TIMD mode. Decoding of other syntax elements related to intra prediction (e.g., ISP, MPM, etc.) can be skipped, thereby significantly reducing coding bits.

[0055] After calculating the SATD between the reconstruction value of the template region and the prediction results of each mode, the TIMD mode can be determined in the following manner: Assuming that mode1 and mode2 are two angle modes adopted for intra prediction in MPM, mode1 is the angle mode with the smallest SATD and its SATD is cost1, and mode2 is the angle mode with the second smallest SATD and its SATD is cost2.

[0056] If cost1×2≦cost2, let mode1 be the TIMD mode of the current block.

[0057] If cost1×2>cost2, the prediction mode obtained by weighting the prediction results of mode1 and mode2 is set as the TMID mode of the current block, which is also called the TIMD fusion mode.

[0058] DIMD uses the reconstructed pixel values ​​of the current block's neighborhood as a template, scans each 3x3 region of the template using a Sobel operator, calculates horizontal and vertical gradients, and calculates the amplitude value Amp = abs(Dx) + abs(Dy) and angle value angular = arctan(Dy / Dx) at each position based on the horizontal and vertical gradients Dx and Dy. Based on the angle values ​​at each position in the template, a traditional angle mode is mapped, and the amplitude values ​​of the same angle mode are added to obtain a histogram of amplitude values ​​and angle modes. If two angle modes exist, one with the highest amplitude value and the other with the second highest amplitude value, the predicted values ​​of the two angle modes with the highest and second highest amplitude values ​​and the planar mode are weighted to obtain the final prediction result when DIMD is used for the current block. This prediction mode combines three intra-prediction modes (planar mode and two angle modes with the highest and second highest amplitude values). This prediction mode is referred to as DIMD fusion mode in this specification. If there are no angular modes with the highest and second highest amplitude values, the prediction when DIMD is used is the same as the prediction when planar modes are used.

[0059] In HEVC, intra prediction is performed using the row above and the column to the left of the current block as reference rows. If the difference between the reconstructed values ​​of the row and column and the original pixel values ​​is large, the prediction quality of the current block is also significantly affected. To solve this problem, VVC employs MRL technology, which allows intra prediction to be performed using a reference row with index 0 (Reference line 0) as well as a reference row with index 1 (Reference line 1) and a reference row with index 2 (Reference line 2) as extended reference rows. To reduce coding complexity, MRL is only used in non-planar modes in MPM. When the encoder performs prediction for the current block based on each angle mode, it tries all three reference rows and selects the reference row with the lowest rate-distortion cost (RD cost). The index of the finally selected reference row is coded and transmitted to the decoder. The decoder performs decoding to obtain the reference row index, and determines the reference row selected by the current block based on the reference row index to use for prediction of the current block.

[0060] 5 shows four reference lines of the current block, which are reference line 0 221 with index 0, reference line 1 222 with index 1, reference line 2 223 with index 2, and reference line 3 224 with index 3. The current block may have more reference lines, and prediction may be performed using only the reconstructed values ​​of the reference lines.

[0061] In ECM, MRL mode can use more reference rows, and multiple candidate reference row indices are stored in a single list. This is called the MRL index list, MRL list, candidate reference row list, or reference row index list. If the current block does not use TIMD, the MRL index list is six in length and can store six reference row indices. These six reference row indices and their order are constant: 0, 1, 3, 5, 7, and 12, respectively. The index stored in the first position is the index. When MRL is used, the MRL index is encoded to represent the position in the MRL index list of the reference row selected by the current block. For example, if the MRL index list is {0, 1, 3, 5, 7, 12}, the MRL indices corresponding to indices 0, 1, 3, 5, 7, and 12 in the table are 0, 1, 2, 3, 4, and 5, respectively. The MRL index can be encoded using a truncated unary code based on the contest model, and a binary flag based on the contest model is obtained through encoding. The binary flag may also be referred to as a binary flag bit, a binary code, a binary bit, etc. The smaller the value of the MRL index, the shorter the code length and the faster the decoding.

[0062] When the MRL mode and the TIMD mode are used simultaneously, the length of the MRL index list is 3, three reference row indexes can be inserted, and the order between the indexes is constant and represented as {0, 1, 3}.

[0063] In this specification, for convenience of description, a reference row is referred to as a "row", but in reality, one reference row includes one row and one column. Generally, the reconstructed value of a reference row used in prediction also includes the reconstructed value of one row and one column. This is the same as the term commonly used in the industry.

[0064] The same technique may have different names in different standards. For example, a technique such as MPM, which derives a list of the most likely modes using blocks neighboring a current block, is called Adaptive Intra Mode Coding (AIMC) in AV2 (AVM), and is called Frequency-based Intra Mode Coding (FIMC) in AVS3 for screen content coding. In non-screen content coding, a technique similar to MPM is always enabled. On the other hand, a technique for performing intra prediction using multiple reference lines, such as MRL, is called Multiple Reference Line Selection for Intra Prediction (MRLS) applied to intra prediction in AV2 (AVM). However, these are merely differences in names, and it should be understood that the terms MPM, MRL, etc. used in this embodiment also cover substantially equivalent techniques in other standards.

[0065] One embodiment provides an intra prediction fusion (IPF) technology. IPF uses two reference rows and the selected angle mode of a current block to construct two combinations of reference rows and intra prediction modes, and predicts the current block based on each of the two combinations. The two obtained prediction results are weighted to determine the final prediction result of the current block, as follows: pfusion=wa×pa+wb×pb

[0066] Here, pa is the prediction result of the current block using the combination of the reference row with index a and the angle mode, pb is the prediction result of the current block using the combination of the reference row with index b and the angle mode, pfusion is the prediction result of the current block after fusion, wa is the weight of pa during weighting, and wb is the weight of pb during weighting. In one example, b=a+1, wa is 3 / 4, and wb is 1 / 4. If the current block uses IPF, the above-mentioned prediction result after fusion is determined as the final prediction result of the current block, i.e., the prediction value of the current block.

[0067] In one example, when all of the following conditions are met: the angle mode selected by the current block is not an integer slope angle mode, the product of the width and height of the current block is greater than 16, and the current block does not select the intra sub-block partition ISP mode, the IPF is enabled by default for any angle mode selected by the current block. Here, when the remainder obtained when the angle value (intraPredAngle) of the angle mode is divided by 32 is 0, the angle mode is an integer slope angle mode, and the correspondence between intraPredAngle and the angle mode is as shown in Table 1 above. In another example, the IPF may add other restrictions (e.g., a restriction on the number of modes). For example, if the use of the IPF requires three or more intra prediction modes to be combined for prediction of the current block, or if the use of the IPF requires two or more angle modes to be combined for prediction of the current block, the use of the IPF is not permitted.

[0068] One embodiment provides a template-based multiple reference line & intra_intra prediction mode (abbreviated as TMRL mode). The TMRL mode is an intra prediction mode that builds a candidate list based on a combination of an extended reference line and an intra prediction mode, and performs encoding and decoding for the combination of the extended reference line and the intra prediction mode. The combined encoding and decoding method of the TMRL mode can reduce coding costs and improve coding performance.

[0069] The video encoding method according to this embodiment is applied to an encoder, and as shown in FIG. 6, the method includes the following steps:

[0070] In step 110, a candidate list of TMRL modes for the current block is constructed, and the candidate list is filled with candidate combinations of extended reference rows and intra prediction modes for the current block.

[0071] In step 120, rate-distortion optimization selects one combination of reference row and intra-prediction mode for the current block to use for intra-prediction.

[0072] In this specification, the reference row includes a reference row with index 0 and an extended reference row, and the combination of the reference row and intra prediction mode selected by the current block may be a combination of the reference row with index 0 and one type of intra prediction mode, or a combination of one extended reference row and one type of intra prediction mode.

[0073] In step 130, if the coding condition for the TMRL mode of the current block is met, the TMRL mode flag of the current block is coded to indicate that the current block uses a TMRL mode, and the TMRL mode index of the current block is coded to indicate the position of the selected combination in the candidate list.

[0074] Here, the above coding conditions include that the selected combination is in the candidate list (in this case, the row selected by the current block is the extended reference row), and may further include one or more of the following conditions 1 to 5: Condition 1: The current block is a block in a luma frame, i.e., the TMRL mode is only used for luma frames; Condition 2: The current block is not located at the upper boundary of a coding tree unit (CTU); Condition 3: The sequence in which the current block is located allows the use of MRL; Condition 4: The dimensions of the current block meet the dimension requirements for using the TMRL mode; Condition 5: The aspect ratio of the current block meets the requirements that the aspect ratio of the current block must meet for using the TMRL mode. The TMRL mode index uses the Golomb-Rice coding method to more rationally classify candidate combinations into categories with different codeword lengths before encoding and decoding, thereby improving coding efficiency.

[0075] If the current block uses the TMRL mode, the MPM mode, the intra sub-block partitioned ISP mode, the multi-transform selection MTS mode, the low band non-separable transform LFNST mode, etc. may be skipped. The TMRL mode flag and the TMRL mode index can simultaneously indicate the reference row and intra prediction mode selected by the current block, eliminating the need to encode and decode MPM-related syntax elements.

[0076] In this specification, the candidate list contains candidate combinations of the extended reference row and the intra prediction mode of the current block, which means that the combinations in the candidate list need to participate in the rate-distortion optimization of the current block, i.e., the mode selection process of the prediction mode of the current block is selected according to the rate-distortion cost.

[0077] In one example of this embodiment, if the current block uses TIMD, coding of the TMRL mode flag and TMRL mode index of the current block is skipped. If the current block does not use TIMD but the selected combination is not in the candidate list, the TMRL mode flag of the current block is coded to indicate that TMRL mode is not used, and coding of the TMRL mode index is skipped. If the current block does not use TIMD, the TMRL mode flag and TMRL mode index according to this embodiment can replace the original MRL index.

[0078] One embodiment provides a method for constructing a TMRL mode candidate list, which may be applied to an encoder or a decoder. As shown in Figure 7, the method includes the following steps:

[0079] In step 210, according to the N extended reference rows and M intra prediction modes of the current block, N×M combinations of extended reference rows and intra prediction modes are obtained, where N≧1, M≧1, and N×M≧2.

[0080] In step 220, the template region of the current block is predicted based on each of the NxM combinations, and the difference between the reconstructed value of the template region and the predicted value is calculated.

[0081] In this step, the difference can be expressed not only by SATD and the sum of absolute difference (SAD), but also by the sum of squared difference (SSD), mean absolute difference (MAD), mean squared error (MSE), etc.

[0082] In step 230, K combinations corresponding to the differences are filled into the candidate list of the TMRL mode of the current block in ascending order of the differences, where 1≦K≦N×M.

[0083] In this embodiment, the constructed candidate list can realize coding for combinations of extended reference rows and intra prediction modes, thereby improving coding efficiency. By arranging combinations that are likely to be selected at the beginning of the candidate list, the TMRL mode index of the combination selected during coding can be made smaller, thereby reducing coding costs.

[0084] The template region of the current block can be located in one or more reference rows closest to the current block. The N extended reference rows participating in the combination are extended reference rows located outside the template region and not crossing the CTU boundary. In one example, when constructing a candidate list for the TMRL mode, all extended reference rows that do not cross the CTU boundary are selected from the predefined extended reference rows with indices {1, 3, 5, 7, 12}. N≦5. The template region of the current block is located in the reference row with index 0 (i.e., the reference row with index 0 is called the reference row where the template region is located), and the other reference rows are called reference rows located outside the template region. Figure 8A shows five extended reference rows participating in the combination (reference row 31 with index 1, reference row 33 with index 3, reference row 35 with index 5, reference row 37 with index 7, and reference row 39 with index 12). On the other hand, in another example shown in FIG. 8B, the template area 40 of the current block is arranged in two reference rows with indexes 0 and 1, and the extended reference rows participating in the combination are all extended reference rows that do not cross the CTU boundary among the reference row 42 with index 2, the reference row 43 with index 3, the reference row 45 with index 5, the reference row 47 with index 7, and the reference row 49 with index 12.

[0085] In one example of this embodiment, the M intra prediction modes participating in the combination must be selected from angular modes, or from among angular modes and DC modes, or can be selected from among angular modes, DC modes, and planar modes. They may also be selected from intra prediction modes selected by the MPM, or the MPM and the second MPM, or may be selected sequentially according to a predetermined rule.

[0086] One embodiment provides a video decoding method related to TMRL mode, which is applied to a decoder, and as shown in Figure 9, the method includes the following steps:

[0087] In step 310, the multi-reference row intra prediction TMRL mode flag of the current block is decoded to determine whether the current block uses the TMRL mode.

[0088] If it is determined in step 320 that the current block uses a TMRL mode, the TMRL mode index of the current block is subsequently decoded to build a candidate list of the TMRL mode of the current block, which is filled with candidate combinations of the extended reference row and the intra-prediction mode of the current block.

[0089] In step 330, a combination of the extension reference row and the intra prediction mode selected by the current block is determined based on the candidate list and the TMRL mode index, and prediction is performed for the current frame according to the selected combination, where the TMRL mode index is used to represent the position of the selected combination of the extension reference row and the intra prediction mode in the candidate list.

[0090] In this embodiment, the TMRL mode index can simultaneously indicate the extended reference row and intra prediction mode selected by the current block, eliminating the need for two indexes and reducing coding costs.If it is determined based on the TMRL mode flag that the current block uses the TMRL mode, decoding of syntax elements related to the MPM mode, ISP mode, MTS mode, and LFNST mode can be skipped.

[0091] In one example of this embodiment, the video decoding method according to this example includes the following steps.

[0092] In step 1, the decoder decodes syntax elements associated with the TMRL mode. Examples include the associated syntax elements for modes such as TIMD, MRL, etc.

[0093] The syntax related to decoding the current block is shown in Table 2 below.

[0094] [Table 2(1)] [Table 2(2)]

[0095] "cu_tmrl_flag" in Table 2 is the TMRL mode flag, and when it is equal to 1, it indicates that the current block uses the TMRL mode, i.e., it is defined that the intra prediction type of the current luma sample is the TMRL mode. "cu_tmrl_flag" is equal to 0, it indicates that the current block does not use the TMRL mode, i.e., it is defined that the intra prediction mode type of the current luma sample is not the TMRL mode. "tmrl_idx" in Table 2 is the TMRL mode index, and it indicates the position in the TMRL mode candidate list of the combination of the extended reference row and intra prediction mode selected by the current block. In other words, it defines the index (index indicating the location of the combination) of the selected combination in the candidate list ordered by TMRL mode.

[0096] TMRL mode can be considered an evolution of MRL mode and may be considered a component of MRL mode. In this example, if the current block uses TIMD mode, the decoding method for the MRL mode syntax element remains unchanged. If the current block does not use TIMD mode, the TMRL mode syntax element needs to be decoded. As shown in Table 1, before decoding cu_tmrl_flag, it is first determined whether the following conditions are met: the current block enables the use of MRL (i.e., whether sps_mrl_enabled_flag is 1), the current block is not located at the upper boundary of the CTU (i.e., whether (y0%CtbSizeY)>0 is true), and the current block does not use TIMD. If these conditions are met, cu_tmrl_flag is decoded. If the other two conditions are met but the current block uses TIMD, the multi-reference row index intra_luma_ref_idx of the current block is decoded.

[0097] In step 2, if the current block uses a TMRL mode, a candidate list of TMRL modes must be constructed. Constructing a candidate list of TMRL modes is an operation that must be performed by both the encoder and the decoder.

[0098] ■ Determine candidate extended reference rows. Candidate extended reference rows are selected from predefined extended reference rows. Based on the position of the current block in the image, usable ones of the predefined extended reference rows are determined. In this example, a template is constructed at the reference row with index 0, and N=5. Of the extended reference rows with indexes {1, 3, 5, 7, 12}, all extended reference rows that do not cross the CTU boundary are included as extended reference row candidates.

[0099] ■ Determine candidate intra prediction modes. In this example, M=6, meaning the number of candidate intra prediction modes is six. First, from the 67 traditional prediction modes, either planar mode and DC mode are eliminated, or only planar mode is eliminated and DC mode is reserved. Next, six intra prediction modes to participate in the combination are selected through the following steps. In the first step, non-overlapping intra prediction modes are selected sequentially from among the intra prediction modes used by the prediction blocks at five adjacent positions near the current block (as shown in FIG. 3). In the second step, an extension operation is performed on the angle modes selected in the first step, and a plus-one / minus-one operation is performed on each angle mode in order to select non-overlapping angle modes obtained by the extension. The process ends when the number of selected modes reaches six. If six intra prediction modes cannot be selected after the extension operation or if no angle mode was selected in the first step, the process proceeds to the third step. In the third step, non-overlapping modes are selected from a predefined mode set until the number of selected modes reaches six.

[0100] ■ Construct a candidate list for TMRL modes. After the candidate extended reference lines and intra-prediction modes are determined, all combinations in the extended reference line list and candidate list are tried one by one, and predictions are made for each of these combinations in the template region in the row where reference line 0 is located. The difference between the reconstructed value of the template region and the predicted value predicted by each combination is calculated, and the K combinations with the smallest corresponding differences are entered into the candidate list for TMRL modes in ascending order of difference. In this example, K=12.

[0101] In step 3, based on the constructed TMRL mode candidate list and the TMRL mode index obtained by decoding, a combination of the extended reference row selected by the current block and the intra prediction mode is determined, and intra prediction is performed on the current block based on the selected combination. If the current block uses a TMRL mode, the index refIdx of the reference row selected by the current block (in this case, the extended reference row) and the intra prediction mode (represented by the variable predModeIntra) are included in the determined combination.

[0102] Using the TMRL mode can save codewords. However, template-based techniques generally increase the complexity of the decoder. In addition, using the TMRL mode requires the encoder / decoder to support the complex calculations, which is not possible for all encoders / decoders. In light of this, whether the TMRL mode is used or not can be expressed as a high-level syntax element, making the mode selection of video encoding and decoding more flexible and providing better adaptability.

[0103] In this specification, a high-level syntax element refers to a syntax element having a level such as a sequence level, an image level, or a slice level that acts as a restriction on a block-level intra prediction syntax element.

[0104] An embodiment of the present invention provides a video decoding method, which controls the use of TMRL modes through high-level syntax. As shown in Figure 10, the video decoding method according to this embodiment includes the following steps:

[0105] In step 410, the value of the TMRL available flag is determined by decoding. As the name suggests, the TMRL available flag is used to indicate whether the use of a TMRL mode is permitted.

[0106] In this step, determining the value of the TMRL available flag by decoding may be determining the value of the TMRL available flag by decoding the TMRL available flag itself, or determining the value of the TMRL available flag by decoding other TMRL related syntax elements, or determining the value of the TMRL available flag by decoding other TMRL related syntax elements and the TMRL available flag.

[0107] In step 420, when decoding the current block, if the value of the TMRL availability flag indicates that use of TMRL mode is permitted, then decoding is permitted for the TMRL mode syntax elements of the current block, and if the value of the TMRL availability flag indicates that use of TMRL mode is not permitted, then decoding is skipped for the TMRL mode syntax elements of the current block.

[0108] Identifiers such as the TMRL available flag, MRL available flag, and template available flag usually indicate that use of the corresponding mode is not permitted when the value is 0, and that use of the corresponding mode is permitted when the value is 1. However, the present invention is not limited to this, and it is also possible for 1 to indicate that the mode is not permitted and 0 to indicate that the mode is permitted.

[0109] In this embodiment, the TMRL availability flag is set to indicate the availability of the TMRL mode, thereby enhancing the flexibility and adaptability of the use of the TMRL mode, and the value of the TMRL availability flag can be used to realize accurate decoding of the TMRL mode syntax element of the current block.

[0110] In one exemplary embodiment of the present invention, the TMRL availability flag is a sequence-level identifier, while in other embodiments, the TMRL availability flag may be a picture-level or slice-level identifier.

[0111] In an exemplary embodiment of the present invention, the TMRL available flag can be decoded independently without relying on other flags, and the value of the TMRL available flag is obtained by decoding the TMRL available flag.

[0112] In the standard text of VVC, sps_mrl_enabled_flag represents the sequence-level MRL availability flag to determine whether to enable MRL mode (also called MRL technology or MRL tool) for the current block. The relevant syntax is shown in Table 2.

[0113] [Table 3]

[0114] Here, when sps_mrl_enabled_flag is 0, it indicates that the use of the MRL is not permitted, and when sps_mrl_enabled_flag is 1, it indicates that the use of the MRL is permitted.

[0115] In this embodiment, the sequence level TMRL availability flag sps_tmrl_enabled_flag is added without any request relationship to control the permission to use the TMRL mode. The associated syntax is shown in Table 4.

[0116] [Table 4]

[0117] In Table 4, the TMRL enabled flag sps_tmrl_enabled_flag and the MRL enabled flag sps_mrl_enabled_flag are decoded individually, and there is no request relationship.

[0118] In an exemplary embodiment of the present invention, a TMRL mode uses an extended reference row, and a reference row with index 0 is typically used when selecting a mode for intra prediction. Therefore, if the use of a TMRL mode is permitted, performing intra prediction on a current block involves the use of multiple reference rows. When designing a high-level syntax, decoding of the TMRL available flag can be relied upon by the MRL available flag. In this case, determining the value of the TMRL available flag by decoding includes decoding the MRL available flag to obtain the value of the MRL available flag. If the value of the MRL available flag indicates that the use of MRL is not permitted, decoding of the TMRL available flag is skipped, and the value of the TMRL available flag is determined by default to be a value indicating that the use of a TMRL mode is not permitted. If the MRL available flag indicates that the use of MRL is permitted, the TMRL available flag is decoded to obtain the value of the TMRL available flag.

[0119] In this embodiment, the TMRL availability flag at one sequence level is incremented in a request-related manner to control the permission to use the TMRL mode. The relevant syntax is shown in Table 5.

[0120] [Table 5]

[0121] According to Table 5, when decoding, the MRL availability flag sps_mrl_enabled_flag is decoded first, and once it is determined that the sequence level sps_mrl_enabled_flag is 1 (i.e., MRL use is permitted), the TMRL availability flag sps_tmrl_enabled_flag is decoded to determine the value of sps_tmrl_enabled_flag.

[0122] In this embodiment, the TMRL available flag is a sequence-level identifier, and the MRL available flag is a sequence-level identifier. In another embodiment, the TMRL available flag may be an image-level identifier, and the MRL available flag may be a sequence-level or image-level identifier. In another embodiment, the TMRL available flag is a slice-level identifier, and the MRL available flag may be a sequence-level, image-level, or slice-level identifier. In these embodiments, if the TMRL available flag and the MRL available flag are the same level identifier, the decoding order of the MRL available flag is before the TMRL available flag.

[0123] In ECM, there are multiple types of template-based encoding and decoding tools, such as DIMD, TIMD, and TMRL, all of which use templates as prediction tools. Alternatively, a single unified indicator may be provided to control all template-based techniques. In light of this, one embodiment of the present invention introduces a sequence-level identifier, a template usage enable flag, denoted by sps_tm_enabled_flag in one example. As its name implies, the template usage enable flag is used herein to indicate whether template usage is enabled, i.e., whether the use of template-based encoding and decoding tools is enabled. If the value of the template usage enable flag indicates that template usage is not enabled, all template-based encoding and decoding tools, such as DIMD, TIMD, and TMRL, are disabled.

[0124] In an exemplary embodiment of the present invention, decoding of the TMRL available flag relies on the template available flag. The aforementioned determining the value of the TMRL available flag by decoding includes decoding the template available flag to obtain the value of the template available flag. If the value of the template available flag indicates that template use is not permitted, decoding of the TMRL available flag is skipped, and the value of the TMRL available flag is determined by default to be a value indicating that TMRL mode use is not permitted. If the template available flag indicates that template use is permitted, the TMRL available flag is decoded to obtain the value of the TMRL available flag.

[0125] In this embodiment, the TMRL availability flag at one sequence level is incremented in a request-related manner to control the TMRL mode usage permission. The relevant syntax is shown in Table 6.

[0126] [Table 6]

[0127] According to Table 6, when decoding, the template enable flag sps_tm_enabled_flag is decoded first, and once it is determined that the sequence level sps_tm_enabled_flag is 1 (i.e., template use is permitted), the TMRL enable flag sps_tmrl_enabled_flag is decoded to determine the value of sps_tmrl_enabled_flag.

[0128] In this embodiment, the TMRL available flag is a sequence-level identifier, and the template available flag is a sequence-level identifier. In another embodiment, the TMRL available flag is an image-level identifier, and the template available flag may be a sequence-level or image-level identifier. In another embodiment, the TMRL available flag is a slice-level identifier, and the template available flag may be a sequence-level, image-level, or slice-level identifier. In these embodiments, if the TMRL available flag and the template available flag are the same level identifier, the template available flag comes before the TMRL available flag in the decoding order.

[0129] In an exemplary embodiment of the present invention, decoding the TMRL available flag relies on the MRL available flag and the template available flag. The step of determining the value of the TMRL available flag by decoding includes decoding the MRL available flag and the template available flag to obtain the values ​​of the MRL available flag and the template available flag. If the value of the MRL available flag indicates that use of MRL is permitted and the value of the template available flag indicates that use of templates is permitted, the TMRL available flag is decoded to obtain the value of the TMRL available flag. If the value of the MRL available flag indicates that use of MRL is not permitted or if the value of the template available flag indicates that use of templates is not permitted, decoding the TMRL available flag is skipped, and the value of the TMRL available flag is determined by default to be a value indicating that use of TMRL mode is not permitted.

[0130] The request relationship according to this embodiment can be expressed by the syntax shown in Table 7.

[0131] [Table 7]

[0132] According to Table 7, when decoding, the template availability flag sps_tm_enabled_flag and the MRL availability flag sps_mrl_enabled_fla are decoded first, and once it is determined that the sequence level sps_tm_enabled_flag and sps_mrl_enabled_flag are both 1 (i.e., the use of the template and MRL is permitted), the TMRL availability flag sps_tmrl_enabled_flag is decoded to determine the value of sps_tmrl_enabled_flag.

[0133] In this embodiment, the TMRL available flag is a sequence-level identifier, and the MRL available flag and the template available flag are sequence-level identifiers. In another embodiment, the TMRL available flag is an image-level identifier, and the MRL available flag and the template available flag may be sequence-level or image-level identifiers. In another embodiment, the TMRL available flag is a slice-level identifier, and the MRL available flag and the template available flag may be sequence-level, image-level, or slice-level identifiers. In these embodiments, when the TMRL available flag, the MRL available flag, and the template available flag are the same level identifiers, the MRL available flag and the template available flag are before the TMRL available flag in the decoding order.

[0134] In ECM, one type of syntax element is called General Constraints Information (GCI), which includes a series of flag bits that are used to restrict whether some sequence-level identifiers are present in the bitstream. In one example, when a flag bit in the GCI is greater than 0 (e.g., 1), it means that the sequence-level identifier in the bitstream corresponding to the flag bit is restricted and does not need to be decoded. The sequence-level identifier is, for example, the identifier of a certain encoding and decoding tool (e.g., MRL, TMRL, DIMD, TIMD, etc.). When the flag bit is equal to 0, it means that the sequence-level identifier in the bitstream corresponding to the flag bit is not restricted and needs to be decoded.

[0135] In an exemplary embodiment of the present invention, a flag bit provided in the GCI indicates whether the TMRL mode is restricted. In this embodiment, the TMRL availability flag is a sequence-level identifier, and determining the value of the TMRL availability flag by decoding includes: decoding the GCI and determining whether the TMRL mode is restricted based on the value of a flag bit in the GCI that indicates whether the TMRL mode is restricted; if the TMRL mode is restricted, skipping decoding the TMRL availability flag and determining that the value of the TMRL availability flag is a value that indicates that use of the TMRL mode is not permitted; and if the TMRL mode is not restricted, continuing to decode the sequence-level identifier to determine the value of the TMRL availability flag.

[0136] This embodiment retrieves the sequence-level TMRL enabled flag sps_tmrl_enabled_flag, and if the decoding of sps_tmrl_enabled_flag does not depend on the value of the sequence-level MRL enabled flag sps_mrl_enabled_flag, it makes a corresponding adjustment to the GCI to add one flag bit to indicate whether the TMRL mode is restricted, as shown in the syntax table below.

[0137] [Table 8]

[0138] As shown in Table 8, in this embodiment, one gci_no_tmrl_constraint_flag is added to indicate whether or not there is a restriction on the TMRL mode, i.e., whether or not there is a restriction on the value of the TMRL availability flag sps_tmrl_enabled_flag. If gci_no_tmrl_constraint_flag is 1, there is no need to decode sps_tmrl_enabled_flag, and sps_tmrl_enabled_flag is determined to be 0 by default, which indicates that use of the TMRL mode is not permitted. If gci_no_tmrl_constraint_flag is 0, this indicates that there is no restriction on the value of sps_tmrl_enabled_flag, and the associated sequence level identifier must be decoded to determine the value of sps_tmrl_enabled_flag.

[0139] gci_no_mrl_constraint_flag in Table 8 indicates whether the MRL mode is restricted, i.e., whether the value of the MRL availability flag sps_mrl_enabled_flag is restricted. When gci_no_mrl_constraint_flag is 1, the value of sps_mrl_enabled_flag is determined to be 0 by default, and when gci_no_mrl_constraint_flag is 0, the value of sps_mrl_enabled_flag is not restricted and the value of sps_mrl_enabled_flag can be determined by decoding.

[0140] In an exemplary embodiment of the present invention, a flag bit arranged in the GCI indicates whether the MRL mode is restricted. However, if the MRL mode is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL availability flag is a sequence-level identifier, and determining the value of the TMRL availability flag by decoding includes: decoding the GCI and determining whether the MRL mode is restricted based on the value of the flag bit in the GCI indicating whether the MRL mode is restricted; if the MRL mode is restricted, skipping decoding the TMRL availability flag and determining that the value of the TMRL availability flag is a value indicating that use of the TMRL mode is not permitted; and if the MRL mode is not restricted, continuing to decode the sequence-level identifier to determine the value of the TMRL availability flag.

[0141] Referring to the syntax table according to the previous embodiment, when decoding the TMRL enable flag sps_tmrl_enabled_flag depends on the value of the MRL enable flag sps_mrl_enabled_flag, whether the MRL mode is restricted and whether the TMRL mode is restricted are determined simultaneously based on the flag bit gci_no_mrl_constraints_flag, which indicates whether the MRL mode is restricted in the GCI. If gci_no_mrl_constraints_flag is 1, sps_tmrl_enabled_flag is determined to be 0 by default, and there is no need to decode sps_tmrl_enabled_flag. If gci_no_mrl_constraints_flag is 0, the value of sps_tmrl_enabled_flag is determined by decoding the associated sequence level identifier.

[0142] In an exemplary embodiment of the present invention, a flag bit arranged in the GCI indicates whether template use is restricted. If template use is restricted, tools (including TMRL modes) based on the template are also restricted. In this embodiment, the TMRL availability flag is a sequence-level identifier, and determining the value of the TMRL availability flag by decoding includes: decoding the GCI and determining whether template use is restricted based on the value of the flag bit in the GCI indicating whether template use is restricted; if template use is restricted, skipping decoding the TMRL availability flag and determining that the value of the TMRL availability flag is a value indicating that TMRL mode use is not permitted; and if template use is not restricted, continuing to decode the sequence-level identifier to determine the value of the TMRL availability flag.

[0143] In this embodiment, when the decoding of sps_tmrl_enabled_flag depends on the value of the sequence-level template enable flag sps_tm_enabled_flag, the gci_no_tm_constraints_flag is used to simultaneously determine whether the template usage and TMRL mode are restricted, as shown in the syntax table below.

[0144] [Table 9]

[0145] As shown in Table 9, the GCI has one flag bit, gci_no_tm_constraints_flag, to indicate whether the MRL mode is restricted. If gci_no_tm_constraints_flag is 1, sps_tmrl_enabled_flag is determined to be 0 by default, and there is no need to decode sps_tmrl_enabled_flag. If gci_no_tm_constraints_flag is 0, the value of sps_tmrl_enabled_flag must be determined by decoding the associated sequence-level identifier.

[0146] In an exemplary embodiment of the present invention, two flag bits arranged in the GCI indicate whether the MRL mode is restricted and whether the use of a template is restricted, respectively. If the use of the MRL mode or the template is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL availability flag is a sequence-level identifier, and determining the value of the TMRL availability flag by decoding includes: decoding the GCI, determining whether the MRL mode is restricted based on the value of the flag bit in the GCI indicating whether the MRL mode is restricted, and determining whether the use of a template is restricted based on the value of the flag bit in the GCI indicating whether the use of a template is restricted; if the MRL mode is restricted or the use of a template is restricted, skipping decoding the TMRL availability flag and determining that the value of the TMRL availability flag is a value indicating that the use of the TMRL mode is not permitted; and if neither the MRL mode nor the use of the template is restricted, continuing to decode the sequence-level identifier to determine the value of the TMRL availability flag.

[0147] In the above-described embodiment, when the sequence-level identifier is subsequently decoded to determine the value of the TMRL available flag, the decoding can be performed according to the above-described embodiment by independently decoding the TMRL available flag or by relying on the MRL available flag and / or the template available flag.

[0148] In the above-described embodiment, the TMRL availability flag is a sequence-level identifier. In other embodiments, when the TMRL availability flag is a picture-level or slice-level identifier, a corresponding flag bit may be placed in the GCI to indicate whether or not there is a restriction on the TMRL mode (i.e., whether or not there is a picture-level or slice-level TMRL availability flag in the bitstream). If the corresponding flag bit indicates that the TMRL mode is restricted, the value of the TMRL availability flag is determined to be a value indicating that use of the TMRL mode is not permitted, and decoding of the TMRL availability flag is skipped. If the corresponding flag bit indicates that the TMRL mode is not restricted, and if the TMRL availability flag is an image-level identifier, the sequence-level and picture-level identifiers can be subsequently decoded to determine the value of the TMRL availability flag. If the TMRL availability flag is a slice-level identifier, the sequence-level, picture-level, and slice-level identifiers can be subsequently decoded to determine the value of the TMRL availability flag.

[0149] In an exemplary embodiment of the present invention, the TMRL availability flag is used to control decoding for switching between the traditional MRL mode and the evolved TMRL mode, taking into account whether the current block uses TIMD mode. Whether the current block uses TIMD mode can be determined by decoding a TIMD mode flag (intra_timd_flag). The method further includes determining whether the current block uses TIMD mode, and if the current block uses TIMD mode, decoding an MRL index of the current block and skipping decoding for the TMRL mode syntax element of the current block, and if the current block does not use TIMD mode, decoding the TMRL mode syntax element of the current block and skipping decoding for the MRL index of the current block.

[0150] In an exemplary embodiment of the present invention, the decoding of the TMRL mode syntax element of the current block includes: decoding a TMRL mode flag of the current block to obtain a value of the TMRL mode flag, where the TMRL mode flag indicates whether the current block uses the TMRL mode; determining whether the current block uses the TMRL mode based on the value of the TMRL mode flag; decoding a TMRL mode index of the current block if it is determined that the TMRL mode is used; and skipping decoding the TMRL mode index of the current block if the TMRL mode is not used. This embodiment further includes, after decoding the TMRL mode index of the current block, building a candidate list of the TMRL mode of the current block; determining a combination of an extended reference row and an intra prediction mode selected by the current block based on the TMRL mode index and the candidate list; and performing prediction on the current block based on the selected combination to obtain a predicted value of the current block.

[0151] In this embodiment, the changes to the relevant CU-level syntax elements are as shown in the table below.

[0152] [Table 10(1)] [Table 10(2)]

[0153] The difference from the syntax table according to the above-described embodiment is that in this embodiment, whether to decode the block-level TMRL mode flag cu_tmrl_flag is not only requested if the use of MRL mode (sps_mrl_enabled_flag is 1) is permitted, the current block is not located at the upper boundary of the CTU ((y0%CtbSizeY)>0), and the current block is not in TIMD mode (intra_timd_flag is 0, i.e., !intra_timd_flag), but also if the use of TMRL mode is permitted (sps_tmrl_enabled_flag is 1). Whether or not to decode g can also be determined by checking that other modes that cannot be used in combination with TMRL mode are not being used. If any one of the following conditions is met: the use of MRL mode is enabled (sps_mrl_enabled_flag is 1), the current block is not located at the upper boundary of the CTU (((y0%CtbSizeY)>0), the current block uses TIMD mode (intra_timd_flag is 0), or the use of TMRL mode is not enabled (sps_tmrl_enabled_flag is 0), the multi-reference row index intra_luma_ref_idx is decoded.

[0154] In this embodiment, the combination of the extended reference row and the intra prediction mode may be the original combination of the extended reference row and the intra prediction mode, or may be a corresponding fusion combination obtained by performing IPF on the original combination including a predetermined angle mode. For example, an original combination includes an extended reference row and a predetermined angle mode, in which case the fusion combination corresponding to the original combination is the combination of the extended reference row, the predetermined angle mode, and another reference row and the predetermined angle mode. Predicting the current block based on the fusion combination includes predicting the current block based on the extended reference row and the predetermined angle mode to obtain a first prediction result, predicting the current block based on the other reference row and the predetermined angle mode to obtain a second prediction result, and determining the weighted sum of the first prediction result and the second prediction result as the predicted value of the current block.

[0155] The above-mentioned predetermined angle mode may include a full angle mode or a partial angle mode, for example, including angle modes other than integer tilt angle modes, or angle modes other than angle modes of -45°, 0°, 45°, 90°, and 135°. Another reference row in the fused combination may be an adjacent row (which may be an inner adjacent row or an outer adjacent row) corresponding to the extended reference row in the original combination, or may be a reference row with an index of 0.

[0156] For example, if the extended reference row participating in the combination is a reference row with index {1,3,5,7,12}, and the original combination is a combination of a reference row with index 1 and one angle mode, the corresponding fused combination includes a combination of a reference row with index 1 and the angle mode, and a combination of a reference row with index 2 (or index 0) and the angle mode. When predicting a current block based on the fused combination, a prediction result of the reference row with index 1 and the angle mode and a prediction result of the reference row with index 2 (or index 0) and the angle mode are weighted to obtain a prediction value of the current block.

[0157] In one example of this embodiment, constructing the candidate list of TMRL modes for the current block includes: Obtaining N×M kinds of original combinations of the extension reference rows and the intra prediction modes according to the N extension reference rows and the M intra prediction modes of the current block; performing intra prediction fusion IPF on original combinations including a predetermined angle mode among the N×M kinds of original combinations to obtain corresponding fusion combinations; Predicting a template region of a current block based on each of the N×M original combinations and the obtained fusion combinations, and calculating a difference between a reconstructed value of the template region and a predicted value obtained by prediction; and filling K combinations corresponding to the differences into a candidate list of TMRL modes of the current block according to the ascending order of the differences, where K, N, and M are positive integers, and 1≦K≦N×M.

[0158] In another example of this embodiment, constructing the candidate list of the TMRL mode of the current block includes: Obtaining N×M kinds of original combinations of the extension reference rows and the intra prediction modes according to the N extension reference rows and the M intra prediction modes of the current block; performing prediction for a template region of the current block based on each of the N×M original combinations, and calculating a difference between the reconstructed value of the template region and a predicted value obtained by the prediction; determining whether each of the K element combinations with the smallest difference, including a predetermined angle mode, needs to be merged, and if it needs to be merged, performing IPF on the element combination to obtain a corresponding merged combination, and adding the merged combination to the candidate list; if it does not need to be merged, adding the element combination to the candidate list, where K, N, and M are positive integers, and 1≦K≦N×M.

[0159] In this example, when determining whether or not each combination needs to be merged, a prediction is made for the template region of the current block based on the fusion combination corresponding to the original combination, and the difference between the reconstructed value of the template region and the predicted value obtained by prediction is calculated. If the difference corresponding to the original combination is greater than the difference corresponding to the fusion combination, it is determined that fusion is necessary; if the difference corresponding to the original combination is equal to or less than the difference corresponding to the fusion combination, it is determined that fusion is not necessary.

[0160] In another example of this embodiment, constructing the candidate list of TMRL modes for the current block includes: Obtaining N×M kinds of original combinations of the extension reference rows and the intra prediction modes according to the N extension reference rows and the M intra prediction modes of the current block; performing a fusion process on the N×M kinds of element combinations, the fusion process including, when a predetermined condition is satisfied for each of the element combinations including a predetermined angle mode among the N×M kinds of element combinations, replacing the element combination with a corresponding fused combination obtained by performing IPF on the element combination; performing prediction for a template region of the current block based on each of the N×M combinations obtained by the fusion process, and calculating a difference between the reconstructed value of the template region and the predicted value obtained by the prediction; and populating a candidate list of TMRL modes of the current block with K combinations corresponding to the differences in ascending order of the differences, where K, N, and M are positive integers, and 1≦K≦N×M.

[0161] In this example, the predetermined condition includes any one or more of the following conditions: the dimensions of the current block are greater than N×M (N and M are positive integers), and the intra prediction mode selected by the current block is not an integer gradient angle mode, but the predetermined condition in this example is not limited thereto.

[0162] The above three examples of this embodiment provide three methods for combining IPF in constructing a candidate list for TMRL, and the coding efficiency of the TMRL mode can be improved by using the IPF method.

[0163] In the above-described example of combining IPFs in this embodiment, performing prediction on the current block based on the selected combination to obtain a predicted value of the current block is as follows: If the selected combination is the original combination, performing prediction on the current block according to the extended reference row and the intra prediction mode in the original combination to obtain a predicted value of the current block; If the selected combination is a fusion combination, determining a weighted sum of the first prediction result and the second prediction result as a predicted value of the current block, where the first prediction result is a result of predicting the current block based on an extended reference row and a predetermined angle mode in an original combination corresponding to the fusion combination, and the second prediction result is a result of predicting the current block based on another reference row and a predetermined angle mode in an original combination corresponding to the fusion combination. As mentioned above, the another reference row may be adjacent to the extended reference row in the original combination corresponding to the fusion combination, or may be a reference row with an index of 0.

[0164] In this example, the weighted sum of the first prediction result and the second prediction result can be calculated by the following formula:

[0165] pfusion=(wapa+wbpb)≫shift; or pfusion=(wapa+wbpb+offset)≫shift where pa is the first prediction result, pb is the second prediction result, pfusion is the final prediction result of the current block, wa is the weight of pa, and wb is the weight of pb.

[0166] Here, wa+wb=(1<<shift), offset=1<<(shift-1), shift≧1, and shift and offset are set parameters.

[0167] Generally, the sum of the two weights used to weight the two prediction results is 1. To avoid decimal arithmetic, wa and wb in the above formula are the weights after being expanded by a left shift operation, so wa+wb=(1<<shift). In the formula, the magnitude of the prediction value is restored by a right shift operation. In the formula, ">" is the right shift sign, "<<" is the left shift sign, and the right side of the sign is the number of digits for the right or left shift.

[0168] An embodiment of the present invention further provides a video encoding method, which is applied in an encoder, and includes the following steps, as shown in FIG.

[0169] In step 510, the value of the TMRL available flag is determined.

[0170] In step 520, in encoding the current block, if the value of the TMRL availability flag indicates that the use of the TMRL mode is permitted, the TMRL mode syntax element of the current block is encoded, and if the value of the TMRL availability flag indicates that the use of the TMRL mode is not permitted, the encoding of the TMRL mode syntax element of the current block is skipped.

[0171] In this embodiment, the value of the TMRL availability flag controls the use of TMRL modes and the encoding of syntax elements, thereby increasing the flexibility and adaptability of TMRL mode use. As one example, if the hardware does not support the TMRL mode, the value of the TMRL availability flag can be set to 0 to indicate that the use of the TMRL mode is not permitted. If the hardware supports the TMRL mode, the value of the TMRL availability flag can be set to 1 to indicate that the use of the TMRL mode is permitted.

[0172] In one exemplary embodiment of the present invention, the TMRL availability flag is a sequence-level identifier. In other embodiments, the TMRL availability flag may be a picture-level or slice-level identifier.

[0173] In an exemplary embodiment of the present invention, the TMRL available flag can be coded independently, without relying on other flags. In this case, the value of the TMRL available flag can be determined based on configuration information or predetermined conditions, and then the TMRL available flag can be coded. The above-mentioned configuration information can be recorded in a configuration file. The above-mentioned predetermined conditions can be conditions set outside of other flags, such as image size, image quality requirements, transmission bandwidth, available computing resources, etc. The TMRL available flag is a 1-bit identifier with a value of 0 or 1. When coding, the value of the TMRL available flag can be written directly into the bitstream.

[0174] In an exemplary embodiment of the present invention, encoding of the TMRL available flag relies on the MRL available flag. Determining the value of the TMRL available flag includes determining the value of the MRL available flag, and if the value of the MRL available flag indicates that use of the MRL is not permitted, skipping encoding of the TMRL available flag and defaulting the value of the TMRL available flag to a value indicating that use of the TMRL mode is not permitted, and if the MRL available flag indicates that use of the MRL is permitted, determining the value of the TMRL available flag based on configuration information or predetermined conditions, and encoding the TMRL available flag. In this embodiment, the value of the MRL available flag is determined based on configuration information or predetermined conditions.

[0175] In this embodiment, the TMRL available flag is a sequence-level identifier, and the MRL available flag is a sequence-level identifier. In another embodiment, the TMRL available flag is an image-level identifier, and the MRL available flag may be a sequence-level or image-level identifier. In another embodiment, the TMRL available flag is a slice-level identifier, and the MRL available flag may be a sequence-level, image-level, or slice-level identifier. In these embodiments, if the TMRL available flag and the MRL available flag are the same level identifier, the coding order of the MRL available flag is before the TMRL available flag.

[0176] In an exemplary embodiment of the present invention, encoding of the TMRL available flag relies on the template available flag. Determining the value of the TMRL available flag includes determining the value of the template available flag, and if the value of the template available flag indicates that use of the template is not permitted, skipping encoding of the TMRL available flag and determining the value of the TMRL available flag to be a value indicating that use of the TMRL mode is not permitted by default, and if the template available flag indicates that use of the template is permitted, determining the value of the TMRL available flag based on configuration information or a predetermined condition, and encoding the TMRL available flag. The value of the template available flag can be determined based on the configuration information or a predetermined condition.

[0177] In this embodiment, the TMRL available flag is a sequence-level identifier, and the template available flag is a sequence-level identifier. In another embodiment, the TMRL available flag is an image-level identifier, and the template available flag may be a sequence-level or image-level identifier. In another embodiment, the TMRL available flag is a slice-level identifier, and the template available flag may be a sequence-level, image-level, or slice-level identifier. In these embodiments, if the TMRL available flag and the template available flag are the same level identifier, the template available flag is coded before the TMRL available flag.

[0178] In an exemplary embodiment of the present invention, encoding of the TMRL available flag depends on the template available flag and the template available flag. Determining the value of the TMRL available flag includes determining the value of the MRL available flag and the value of the template available flag; if the value of the MRL available flag indicates that use of the MRL is permitted and the value of the template available flag indicates that use of the template is permitted, determining the value of the TMRL available flag based on configuration information or a predetermined condition and encoding the TMRL available flag; if the value of the MRL available flag indicates that use of the MRL is not permitted or the value of the template available flag indicates that use of the template is not permitted, skipping encoding of the TMRL available flag and determining the value of the TMRL available flag to be a value indicating that use of the TMRL mode is not permitted by default.

[0179] In this embodiment, the TMRL available flag is a sequence-level identifier, and the MRL available flag and the template available flag are sequence-level identifiers. In another embodiment, the TMRL available flag is an image-level identifier, and the MRL available flag and the template available flag may be sequence-level or image-level identifiers. In another embodiment, the TMRL available flag is a slice-level identifier, and the MRL available flag and the template available flag may be sequence-level, image-level, or slice-level identifiers. In these embodiments, when the TMRL available flag, the MRL available flag, and the template available flag are the same level identifiers, the MRL available flag and the template available flag are coded before the TMRL available flag.

[0180] If the TMRL availability flag is a sequence level identifier, the use of the TMRL mode can be restricted by a flag bit placed in the GCI, and if the value of the flag bit indicates that the MRL mode is restricted, there is no need to encode the TMRL availability flag.

[0181] In one exemplary embodiment of the present invention, a flag bit is provided in the GCI to indicate whether a TMRL mode is restricted. Whether a TMRL mode is restricted does not depend on other flag bits in the GCI. In this embodiment, the TMRL availability flag is a sequence-level identifier, and determining the value of the TMRL availability flag includes determining whether a TMRL mode is restricted based on the value of the flag bit in the GCI for indicating whether a TMRL mode is restricted; if the TMRL mode is restricted, skipping encoding the TMRL availability flag and determining that the value of the TMRL availability flag is a value indicating that use of the TMRL mode is not permitted; and if the TMRL mode is not restricted, continuing to encode the sequence-level identifier to determine the value of the TMRL availability flag.

[0182] In one exemplary embodiment of the present invention, a flag bit is provided in the GCI to indicate whether the MRL mode is restricted. However, if the MRL mode is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL availability flag is a sequence-level identifier, and determining the value of the TMRL availability flag includes determining whether the MRL mode is restricted based on the value of the flag bit in the GCI that indicates whether the MRL mode is restricted; if the MRL mode is restricted, skipping encoding of the TMRL availability flag and determining that the value of the TMRL availability flag is a value that indicates that use of the TMRL mode is not permitted; and if the MRL mode is not restricted, continuing to encode the sequence-level identifier to determine the value of the TMRL availability flag.

[0183] In an exemplary embodiment of the present invention, a flag bit is provided in the GCI to indicate whether template use is restricted. If the MRL mode is restricted or the template use is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL availability flag is a sequence-level identifier, and determining the value of the TMRL availability flag includes determining whether template use is restricted based on the value of the flag bit in the GCI indicating whether template use is restricted; if template use is restricted, skipping encoding the TMRL availability flag and determining the value of the TMRL availability flag to indicate that TMRL mode use is not permitted; and if template use is not restricted, continuing to encode the sequence-level identifier to determine the value of the TMRL availability flag.

[0184] In an exemplary embodiment of the present invention, two flag bits are provided in the GCI to indicate whether the MRL mode is restricted and whether the use of a template is restricted, respectively. If the MRL mode is restricted or the use of a template is restricted, the TMRL mode is also restricted. In this embodiment, the TMRL availability flag is a sequence-level identifier, and determining the value of the TMRL availability flag includes: determining whether the MRL mode is restricted based on the value of the flag bit in the GCI indicating whether the MRL mode is restricted; determining whether the use of a template is restricted based on the value of the flag bit in the GCI indicating whether the use of a template is restricted; if the MRL mode is restricted or the use of a template is restricted, skipping encoding the TMRL availability flag and determining that the value of the TMRL availability flag is a value indicating that the use of the TMRL mode is not permitted; and if the MRL mode is not restricted and the use of a template is not restricted, continuing to encode the sequence-level identifier to determine the value of the TMRL availability flag.

[0185] In the above-described embodiments, when the sequence-level identifier is subsequently encoded to determine the value of the TMRL available flag, the method of independently encoding the TMRL available flag according to the above-described embodiments can be adopted to determine the value of the TMRL available flag based on configuration information or predetermined conditions, or the TMRL available flag can be encoded using an encoding method that relies on the MRL available flag and / or template available flag to determine the value of the TMRL available flag.

[0186] In the above-described embodiment, the TMRL availability flag is a sequence-level identifier. In other embodiments, if the TMRL availability flag is a picture-level or slice-level identifier, a corresponding flag bit can be provided in the GCI to indicate whether the TMRL mode is restricted (i.e., whether a picture-level or slice-level TMRL availability flag is present in the bitstream). If the corresponding flag bit indicates that the TMRL mode is restricted, the value of the TMRL availability flag is determined to be a value indicating that use of the TMRL mode is not permitted, and encoding of the TMRL availability flag is skipped. If the corresponding flag bit indicates that the TMRL mode is not restricted, if the TMRL availability flag is a picture-level identifier, the value of the TMRL availability flag can be determined by subsequently encoding sequence-level and picture-level identifiers; if the TMRL availability flag is a slice-level identifier, the value of the TMRL availability flag can be determined by subsequently encoding sequence-level, picture-level, and slice-level identifiers.

[0187] In an exemplary embodiment of the present invention, a TMRL availability flag is used to control encoding of switching between the traditional MRL mode and the evolved TMRL mode, in conjunction with whether the current block uses the TIMD mode. Whether the current block uses the TIMD mode can be determined by the value of a TIMD mode flag (intra_timd_flag). In this embodiment, if the value of the TMRL availability flag indicates that use of the TMRL mode is permitted, the method further includes determining whether the current block uses the TIMD mode, and if the current block uses the TIMD mode, encoding an MRL index of the current block and skipping encoding of the TMRL mode syntax element of the current block; and if the current block does not use the TIMD mode, encoding the TMRL mode syntax element of the current block and skipping encoding of the MRL index of the current block.

[0188] In an exemplary embodiment of the present invention, encoding the TMRL mode syntax element of the current block includes: building a candidate list of the TMRL mode of the current block; selecting a combination of a reference row and an intra-prediction mode for the current block through rate-distortion optimization; and, if a coding condition of the TMRL mode of the current block is satisfied, encoding a TMRL mode flag of the current block to indicate that the current block uses a TMRL mode and encoding a TMRL mode index of the current block to indicate a position of the selected combination in the candidate list, where the coding condition includes at least that the selected combination is in the candidate list (where the selected reference row is an extended reference row).

[0189] In one example of this embodiment, the encoding conditions further include one or more of: the current block is a block in a luminance frame; the current block is not located at the upper boundary of a coding tree unit CTU; the current block allows the use of a multi-reference row MRL; the current block allows the use of a template; the dimensions of the current block satisfy the dimension requirements for the current block that must be satisfied in order to use the TMRL mode; and the aspect ratio of the current block satisfy the requirement for the aspect ratio of the current block that must be satisfied in order to use the TMRL mode.

[0190] In one example of this embodiment, when constructing a TMRL candidate list, IPF can be performed on the original combination of the extended reference row and the angle mode to obtain a corresponding fusion combination, and the fusion combination can be inserted into the candidate list. In this case, the above-mentioned combination of the reference row and the intra prediction mode can be a combination of the reference row with index 0 and the intra prediction mode, the original combination of the extended reference row and the intra prediction mode, or the fusion combination of the extended reference row and the intra prediction mode obtained by performing IPF on the original combination. The method of inserting the fusion combination into the candidate list can be any method for constructing a candidate list of the TMRL mode of the current block using IPF according to the above-mentioned embodiment, and redundant description will be omitted here.

[0191] An embodiment of the present invention further provides a bitstream, said bitstream being generated by the video encoding method according to any embodiment of the present invention.

[0192] An embodiment of the present invention further provides a video decoding apparatus, which, as shown in FIG. 12, includes a processor 71 and a memory 73 storing a computer program, and when the processor 71 executes the computer program, can realize the video decoding method according to any embodiment of the present invention.

[0193] An embodiment of the present invention further provides a video encoding apparatus, the apparatus including a processor and a memory in which a computer program is stored, the processor being capable of implementing the video encoding method according to any embodiment of the present invention when executing the computer program.

[0194] An embodiment of the present invention further provides a video encoding and decoding system, which includes a video encoding apparatus according to any embodiment of the present invention and a video decoding apparatus according to any embodiment of the present invention.

[0195] An embodiment of the present invention further provides a non-transitory computer-readable storage medium, the computer-readable storage medium storing a computer program, which, when executed by a processor, realizes the video decoding method according to any of the embodiments of the present invention, or realizes the video encoding method according to any of the embodiments of the present invention.

[0196] The processor according to the above-described embodiments of the present invention may be a general-purpose processor, including a central processor (CPU), a network processor (abbreviated as NP), a microprocessor, or other general processor. The processor may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a discrete logic or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or other equivalent integrated or discrete logic circuit, or a combination of the above-described devices. That is, the processor in the above-described embodiments is any processor component or combination of devices capable of implementing the methods, steps, and logic block diagrams according to the embodiments of the present invention. When the embodiments of the present invention are implemented in part by software, the instructions used in the software may be stored in a suitable non-transitory computer-readable storage medium, and the instructions may be executed in hardware by one or more processors to implement the methods according to the embodiments of the present invention. As used herein, the term "processor" may refer to the above-described structure or any other structure applicable to the technology described herein.

[0197] In one or more exemplary embodiments described above, the functions described may be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, the functions may be stored on or transmitted by a computer-readable medium in the form of one or more instructions or code and executed by a processing unit of the hardware. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium, such as a data storage medium, or a communication medium, which may be any medium that facilitates the transmission of a computer program from one place to another, for example, based on a communication protocol. Thus, the computer-readable medium may generally correspond to a non-transitory tangible computer-readable storage medium or a communication medium, such as a signal or carrier wave. The data storage medium may be any available medium that can be stored and retrieved by one or more computers or one or more processors to retrieve instructions, code, and / or data architecture for implementing the techniques of the present invention. A computer program product may include a computer-readable medium.

[0198] By way of non-limiting example, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical storage, magnetic disk or other magnetic storage, cache memory, or any other medium that can store desired program code in the form of instructions or data structures and that can be written to or retrieved by a computer. Additionally, any connection may be considered a computer-readable medium; for example, transmission of instructions from a website, server, or other long-distance resource via coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio waves, and microwaves, also includes within the definition of medium. However, computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory (momentary) media, but rather cover non-transitory, tangible storage media. As used herein, magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), soft magnetic disks, Blu-ray disks, etc. Among them, magnetic disks generally reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0199] In some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules that are deployed for encoding and decoding, or may be incorporated into a combined encoder / decoder, or the techniques may be implemented entirely in one or more circuit or logic elements.

[0200] The technical means according to the embodiments of the present invention can be widely implemented in various types of apparatuses or devices, including a wireless mobile phone, an integrated circuit (IC), or a set of ICs (e.g., a chip group). In the embodiments of the present invention, each component, module, or unit is described to emphasize the functional aspects of the apparatus of the technology described herein by being arranged therein, but it is not necessarily limited to being realized by different hardware units. As mentioned above, each unit can be combined in an encoder / decoder hardware unit, or can be provided by a set of interoperating hardware units (including the one or more processors mentioned above) combined with appropriate software and / or firmware.

Claims

1. 1. A method of video decoding, applied to a decoder, comprising: determining a value of a template-based multi-reference row intra-prediction TMRL available flag by decoding; When decrypting the current block, If the value of the TMRL available flag indicates that use of the TMRL mode is permitted, then permit decoding of the TMRL mode syntax element of the current block; If the value of the TMRL available flag indicates that the use of TMRL mode is not permitted, skip decoding the TMRL mode syntax element of the current block. Video decoding methods.

2. The TMRL availability flag is a sequence level identifier, or a picture level identifier, or a slice level identifier. The method of claim 1.

3. The above-mentioned determining the value of the TMRL available flag by decoding includes: and obtaining a value of the TMRL availability flag by decoding the TMRL availability flag. The method of claim 1.

4. The above-mentioned determining the value of the TMRL available flag by decoding includes: obtaining a value of the MRL availability flag by decoding the MRL availability flag; If the value of the MRL available flag indicates that the use of MRL is not permitted, skipping decoding for the TMRL available flag and determining by default that the value of the TMRL available flag is a value indicating that the use of TMRL mode is not permitted; If the MRL available flag indicates that use of the MRL is permitted, decoding the TMRL available flag to obtain a value of the TMRL available flag. The method of claim 1.

5. The above-mentioned determining the value of the TMRL available flag by decoding includes: obtaining a value of the template available flag by decoding the template available flag; If the value of the template available flag indicates that the use of a template is not permitted, skipping decoding for the TMRL available flag and determining by default that the value of the TMRL available flag is a value indicating that the use of a TMRL mode is not permitted; If the template availability flag indicates that use of the template is permitted, decoding the TMRL availability flag to obtain a value of the TMRL availability flag. The method of claim 1.

6. The above-mentioned determining the value of the TMRL available flag by decoding includes: Decoding an MRL available flag and a template available flag to obtain a value of the MRL available flag and a value of the template available flag; If the value of the MRL available flag indicates that use of the MRL is permitted and the value of the template available flag indicates that use of the template is permitted, decode the TMRL available flag to obtain the value of the TMRL available flag; If the value of the MRL available flag indicates that use of MRL is not permitted, or if the value of the template available flag indicates that use of a template is not permitted, skipping decoding for the TMRL available flag and determining by default that the value of the TMRL available flag is a value indicating that use of a TMRL mode is not permitted. The method of claim 1.

7. The TMRL availability flag is a sequence level identifier, and determining the value of the TMRL availability flag by decoding includes: Decoding the general restriction information GCI and determining whether the TMRL mode is restricted based on a value of a flag bit in the GCI for indicating whether the TMRL mode is restricted; If the TMRL mode is restricted, skip decoding the TMRL available flag and determine that the value of the TMRL available flag is a value indicating that the use of the TMRL mode is not permitted; If the TMRL mode is not restricted, then continuing to decode the sequence level identifier to determine the value of the TMRL available flag. The method of claim 1.

8. The TMRL availability flag is a sequence level identifier, and determining the value of the TMRL availability flag by decoding includes: Decoding the GCI and determining whether the MRL mode is restricted based on a value of a flag bit in the GCI indicating whether the MRL mode is restricted; If the MRL mode is restricted, skip decoding the TMRL available flag and determine that the value of the TMRL available flag is a value indicating that the use of the TMRL mode is not permitted; If the MRL mode is not restricted, then continuing to decode the sequence level identifier to determine the value of the TMRL available flag. The method of claim 1.

9. The TMRL availability flag is a sequence level identifier, and determining the value of the TMRL availability flag by decoding includes: decoding the GCI and determining whether the template is restricted in use based on a value of a flag bit in the GCI that indicates whether the template is restricted in use; If the use of the template is restricted, skipping decoding for the TMRL available flag and determining that the value of the TMRL available flag is a value indicating that the use of the TMRL mode is not permitted; If the template usage is not restricted, then subsequently decoding the sequence level identifier to determine the value of the TMRL availability flag. The method of claim 1.

10. The TMRL availability flag is a sequence level identifier, and determining the value of the TMRL availability flag by decoding includes: decoding the GCI, determining whether the MRL mode is restricted based on a value of a flag bit in the GCI indicating whether the MRL mode is restricted, and determining whether the template usage is restricted based on a value of a flag bit in the GCI indicating whether the template usage is restricted; If the MRL mode is restricted or the use of the template is restricted, skip decoding the TMRL available flag and determine that the value of the TMRL available flag is a value indicating that the use of the TMRL mode is not permitted; If the MRL mode is unrestricted and the template usage is unrestricted, then continuing to decode the sequence level identifier to determine the value of the TMRL available flag. The method of claim 1.

11. If the value of the TMRL available flag indicates that use of the TMRL mode is permitted, the method Determining whether the current block derives a TIMD mode based on the intra-prediction mode of the template; If the current block uses TIMD mode, decoding the multi-reference row MRL index of the current block and skipping decoding for the TMRL mode syntax element of the current block; If the current block does not use TIMD mode, decoding the TMRL mode syntax element of the current block and skipping decoding for the MRL index of the current block. The method of claim 1.

12. Decoding the TMRL mode syntax element of the current block includes: Decoding a TMRL mode flag of a current block to obtain a value of the TMRL mode flag, the TMRL mode flag indicating whether the current block uses a TMRL mode; determining whether a current block uses a TMRL mode based on a value of the TMRL mode flag, and if it is determined that the current block uses the TMRL mode, decoding a TMRL mode index of the current block; and if it does not use the TMRL mode, skipping decoding the TMRL mode index of the current block; After decoding the TMRL mode index of the current block, the method further comprises: Constructing a candidate list for the TMRL mode of the current block; determining a combination of an extended reference row and an intra prediction mode selected by the current block based on the TMRL mode index and the candidate list; and performing a prediction on the current block based on the selected combination to obtain a predicted value of the current block. The method of claim 1.

13. The construction of the candidate list for the TMRL mode of the current block includes: Obtaining N×M kinds of original combinations of the extension reference rows and the intra prediction modes according to the N extension reference rows and the M intra prediction modes of the current block; performing intra-prediction fusion IPF on original combinations including a predetermined angle mode among the N×M kinds of original combinations to obtain corresponding fusion combinations; performing prediction on a template region of a current block based on each of the N×M kinds of original combinations and the obtained fused combinations, and calculating a difference between a reconstructed value of the template region and a predicted value obtained by prediction; Filling the K combinations corresponding to the differences into a candidate list of a TMRL mode of the current block according to an ascending order of the differences; Here, K, N, and M are positive integers, and 1≦K≦N×M. The method of claim 12.

14. The construction of the candidate list for the TMRL mode of the current block includes: Obtaining N×M kinds of original combinations of the extension reference rows and the intra prediction modes according to the N extension reference rows and the M intra prediction modes of the current block; performing predictions for a template region of the current block based on each of the N×M original combinations, and calculating a difference between the reconstructed value of the template region and a predicted value obtained by the predictions; determining whether each of the combinations including the predetermined angle mode among the K kinds of original combinations with the smallest difference needs to be merged; if it is necessary to merge, performing IPF on the original combination to obtain a corresponding merged combination, and adding the merged combination to the candidate list; if it is not necessary to merge, adding the original combination to the candidate list; Here, K, N, and M are positive integers, and 1≦K≦N×M. The method of claim 12.

15. Determining whether or not each of the combinations including the predetermined angle mode among the K kinds of original combinations with the smallest difference needs to be merged is performed by: Performing prediction on a template region of the current block according to the fusion combination corresponding to the original combination, calculating a difference between the reconstructed value of the template region and the predicted value obtained by prediction, and determining that fusion is required if the difference corresponding to the original combination is greater than the difference corresponding to the fusion combination; If the difference corresponding to the original combination is equal to or less than the difference corresponding to the fused combination, determining that there is no need to fuse. The method of claim 12.

16. The construction of the candidate list for the TMRL mode of the current block includes: Obtaining N×M kinds of original combinations of the extension reference rows and the intra prediction modes according to the N extension reference rows and the M intra prediction modes of the current block; performing a fusion process on the N×M kinds of element combinations, the fusion process including, for each of the element combinations including a predetermined angle mode among the N×M kinds of element combinations, replacing the element combination with a corresponding fused combination obtained by performing IPF on the element combination when a predetermined condition is satisfied; performing prediction for a template region of the current block based on each of the NxM combinations obtained by the fusion process, and calculating a difference between the reconstructed value of the template region and the predicted value obtained by the prediction; Filling the K combinations corresponding to the differences into a candidate list of a TMRL mode of the current block according to an ascending order of the differences; Here, K, N, and M are positive integers, and 1≦K≦N×M. The method of claim 12.

17. The predetermined condition is: The dimensions of the current block are greater than N×M, where N and M are positive integers; The intra prediction mode selected by the current block is not an integer gradient angle mode.

17. The method of claim 16.

18. The step of predicting the current block based on the selected combination to obtain a predicted value of the current block includes: If the selected combination is the original combination, performing prediction on the current block according to the extended reference row and the intra prediction mode in the original combination to obtain a predicted value of the current block; If the selected combination is a fusion combination, determining a weighted sum of the first prediction result and the second prediction result as a prediction value of the current block; Here, the first prediction result is a result of prediction on the current block based on an extended reference row and a predetermined angle mode in an original combination corresponding to the fusion combination, and the second prediction result is a result of prediction on the current block based on another reference row and a predetermined angle mode in an original combination corresponding to the fusion combination.

18. The method of any one of claims 13 to 17.

19. The predetermined angle mode includes a full angle mode or an angle mode other than an integer tilt angle mode; The other one reference row is adjacent to the extended reference row in the original combination corresponding to the fusion combination, or the other one reference row is a reference row with an index of 0.

20. The method of claim 18.

20. The weighted sum of the first prediction result and the second prediction result is calculated by the following formula: pfusion=(wapa+wbpb)≫shift Or, pfusion=(wapa+wbpb+offset)≫shift where pa is the first prediction result, pb is the second prediction result, pfusion is the final prediction result of the current block, wa is the weight of pa, and wb is the weight of pb. Here, wa+wb=(1<<shift), offset=1<<(shift-1), shift≧1, and shift and offset are set parameters.

20. The method of claim 18.

21. 1. A method of video coding, applied in an encoder, comprising: determining a value of a template-based multi-reference row intra-prediction TMRL available flag; When encoding the current block, If the value of the TMRL available flag indicates that use of the TMRL mode is permitted, then permitting encoding of a TMRL mode syntax element of the current block; If the value of the TMRL available flag indicates that the use of the TMRL mode is not permitted, skip encoding the TMRL mode syntax element of the current block. A method of video coding.

22. The TMRL availability flag is a sequence level identifier, or a picture level identifier, or a slice level identifier.

22. The method of claim 21.

23. Determining the value of the TMRL available flag includes: determining a value of the TMRL available flag based on configuration information or a predetermined condition, and encoding the TMRL available flag.

22. The method of claim 21.

24. Determining the value of the TMRL available flag includes: determining a value of a multi-reference row MRL available flag; If the value of the MRL available flag indicates that the use of MRL is not permitted, skip encoding the TMRL available flag and determine by default that the value of the TMRL available flag is a value indicating that the use of TMRL mode is not permitted; If the MRL available flag indicates that the use of the MRL is permitted, determining the value of the TMRL available flag based on configuration information or a predetermined condition, and encoding the TMRL available flag.

22. The method of claim 21.

25. Determining the value of the TMRL available flag includes: determining a value of a template availability flag; If the value of the template available flag indicates that the use of a template is not permitted, skipping encoding of the TMRL available flag and determining by default that the value of the TMRL available flag is a value indicating that the use of a TMRL mode is not permitted; If the template availability flag indicates that the template is permitted to be used, determining the value of the TMRL availability flag based on configuration information or a predetermined condition, and encoding the TMRL availability flag.

22. The method of claim 21.

26. Determining the value of the TMRL available flag includes: determining a value of an MRL available flag and a value of a template available flag; If the value of the MRL available flag indicates that use of the MRL is permitted and the value of the template available flag indicates that use of the template is permitted, determining the value of the TMRL available flag based on configuration information or a predetermined condition, and encoding the TMRL available flag; If the value of the MRL available flag indicates that use of MRL is not permitted, or if the value of the template available flag indicates that use of a template is not permitted, skipping encoding of the TMRL available flag and determining by default that the value of the TMRL available flag is a value indicating that use of a TMRL mode is not permitted.

22. The method of claim 21.

27. The TMRL availability flag is a sequence level identifier, and determining the value of the TMRL availability flag includes: Determining whether the TMRL mode is restricted based on a value of a flag bit in the GCI for indicating whether the TMRL mode is restricted; If the TMRL mode is restricted, skip encoding the TMRL available flag, and determine that the value of the TMRL available flag is a value indicating that the TMRL mode is not permitted to be used; If the TMRL mode is not restricted, then continue encoding the sequence level identifier to determine the value of the TMRL available flag.

22. The method of claim 21.

28. The TMRL availability flag is a sequence level identifier, and determining the value of the TMRL availability flag includes: determining whether the MRL mode is restricted based on a value of a flag bit in the GCI indicating whether the MRL is restricted; If the MRL mode is restricted, skip encoding the TMRL available flag, and determine that the value of the TMRL available flag is a value indicating that the use of the TMRL mode is not permitted; If the MRL mode is not restricted, then continuing to encode the sequence level identifier to determine the value of the TMRL available flag.

22. The method of claim 21.

29. The TMRL availability flag is a sequence level identifier, and determining the value of the TMRL availability flag includes: determining whether the use of the template is restricted based on a value of a flag bit in the GCI that indicates whether the use of the template is restricted; If the use of the template is restricted, skip encoding the TMRL available flag and determine that the value of the TMRL available flag is a value indicating that the use of the TMRL mode is not permitted; If the template usage is not restricted, then subsequently encoding the sequence level identifier to determine the value of the TMRL availability flag.

22. The method of claim 21.

30. The TMRL availability flag is a sequence level identifier, and determining the value of the TMRL availability flag includes: determining whether the MRL mode is restricted based on a value of a flag bit in the GCI indicating whether the MRL mode is restricted, and determining whether the use of the template is restricted based on a value of a flag bit in the GCI indicating whether the use of the template is restricted; If the MRL mode is restricted or the use of the template is restricted, skip encoding the TMRL available flag, and determine that the value of the TMRL available flag is a value indicating that the use of the TMRL mode is not permitted; If the MRL mode is unrestricted and the template usage is unrestricted, then continuing to encode the sequence level identifier to determine the value of the TMRL available flag.

22. The method of claim 21.

31. If the value of the TMRL available flag indicates that use of the TMRL mode is permitted, the method Determining whether the current block uses a template-based intra-prediction mode to derive a TIMD mode; If the current block uses TIMD mode, encoding the MRL index of the current block and skipping encoding for the TMRL mode syntax element of the current block; If the current block does not use TIMD mode, encoding the TMRL mode syntax element of the current block and skipping encoding the MRL index of the current block.

22. The method of claim 21.

32. The encoding of the TMRL mode syntax element of the current block comprises: Constructing a candidate list for the TMRL mode of the current block; selecting a combination of reference row and intra prediction mode for the current block by rate-distortion optimization; If the encoding condition of the TMRL mode of the current block is satisfied, encoding a TMRL mode flag of the current block to indicate that the current block uses the TMRL mode, and encoding a TMRL mode index of the current block to indicate the position in the candidate list of the combination selected by the current block; Here, the encoding condition includes at least that the selected combination is in the candidate list.

32. The method of claim 21 or 31.

33. The encoding condition is: the current block is a block in a luminance frame; the current block is not located at the upper boundary of a coding tree unit (CTU); The current block allows the use of multi-reference rows (MRLs), The current block allows the use of templates, The dimensions of the current block satisfy the dimension requirements for the current block that must be met in order to use the TMRL mode; and the aspect ratio of the current block satisfies a requirement for the aspect ratio of the current block to be satisfied in order to use the TMRL mode.

33. The method of claim 32.

34. A bitstream comprising: The bitstream is generated by a method of video coding according to any one of claims 21 to 33. Bitstream.

35. 1. An apparatus for video decoding, comprising: a processor and a memory in which a computer program is stored, The processor, when executing the computer program, is capable of implementing the method for video decoding according to any one of claims 1 to 20. Video decoding device.

36. 1. A video encoding apparatus, comprising: a processor and a memory in which a computer program is stored, The processor, when executing the computer program, is capable of implementing the video coding method of any one of claims 21 to 33. Video coding device.

37. A video encoding device according to claim 36 and a video decoding device according to claim 35. Video encoding and decoding system.

38. 1. A non-transitory computer-readable storage medium, comprising: The computer-readable storage medium has stored thereon a computer program, which, when executed by a processor, is capable of implementing the video decoding method of any one of claims 1 to 20 or the video encoding method of any one of claims 21 to 33. A non-transitory computer-readable storage medium.

Images