Multi-reference line index list sorting method, video encoding and decoding method, device and system

Abstract

The invention discloses a multi-reference-line index list sorting method, a video encoding and decoding method, a device and a system, and the method comprises the steps: carrying out the prediction of a template region according to at least a part of reference lines, located at the outer side of the template region, of a current block, and calculating an error between a reconstruction value of the template region and a predicted value obtained through prediction; according to the sequence of the errors from small to large, the indexes of the reference lines corresponding to the errors are filled into an MRL index list of the current block, and the MRL index list sorted based on the current block is obtained; during encoding and decoding, the MRL index list of the current block can be ranked, and the multi-reference line index of the current block can be decoded or encoded according to the ranked MRL index list. The embodiment of the invention can save the coding overhead and improve the coding performance.

Description

Multi-reference row index list sorting method, video encoding and decoding method, device and system Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, video technology, and more specifically, to a multi-reference row index list sorting method, a video encoding and decoding method, a device, and a system. Background Art

[0002] Digital video compression technology primarily compresses large amounts of digital video data for easier transmission and storage. Currently popular video codec standards, such as H.266 / Versatile Video Coding (VVC), employ a block-based hybrid coding framework. Each video frame is divided into square largest coding units (LCUs) of equal size (e.g., 128x128, 64x64, etc.). Each LCU can be divided into rectangular coding units (CUs) based on a specific rule. Coding units may also be divided into prediction units (PUs) and transform units (TUs). The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-frame prediction and inter-frame prediction, which are used to reduce or remove inherent redundancy in the video. Intra-frame blocks are predicted using neighboring pixels as reference, while inter-frame blocks use information from spatially adjacent blocks and reference information from other frames. In contrast to the prediction signal, the residual information is transformed, quantized, and entropy-encoded into a bitstream on a block-by-block basis. These techniques are described in standards and implemented in various fields related to video compression.

[0003] With the surge in Internet videos and people's increasing demand for video clarity, although existing digital video compression standards can save a lot of video data, there is still a need to pursue better digital video compression technology to reduce the bandwidth and traffic pressure of digital video transmission.

[0004] SUMMARY OF THE INVENTION

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] An embodiment of the present disclosure provides a method for sorting a multi-reference row index list, including:

[0007] Predicting the template region respectively according to at least part of the reference rows of the current block located outside the template region, and calculating an error between a reconstructed value of the template region and a predicted value obtained by prediction;

[0008] In order of the errors from small to large, the indexes of the reference rows corresponding to the errors are filled into the MRL index list of the current block to obtain a sorted MRL index list based on the current block.

[0009] An embodiment of the present disclosure further provides a video decoding method, including:

[0010] Decode the intra prediction related syntax elements of the current block in the code stream, and if the current block allows the use of multiple reference lines (MRL), continue decoding the multiple reference line index of the current block; the multiple reference line index is used to indicate the position of the index of the reference line selected by the current block in the MRL index list;

[0011] If it is determined according to the decoding result that the MRL index list of the current block is allowed to be sorted, the MRL index list of the current block is sorted according to the method described in any embodiment of the present disclosure to obtain a sorted MRL index list based on the current block;

[0012] The reference row selected by the current block is determined according to the multiple reference row indices and the MRL index list sorted based on the current block, and intra-frame prediction is performed on the current block according to the reference row selected by the current block.

[0013] An embodiment of the present disclosure further provides a video encoding method, including:

[0014] Perform intra-frame prediction on the current block to determine the reference row selected for the current block;

[0015] If it is determined that the MRL index list of the current block is allowed to be sorted, the MRL index list of the current block is sorted according to the method described in any embodiment of the present disclosure to obtain a sorted MRL index list based on the current block;

[0016] According to the position of the index of the reference row selected by the current block in the MRL index list sorted based on the current block, multiple reference row indexes of the current block are determined and encoded.

[0017] An embodiment of the present disclosure also provides a code stream, wherein the code stream includes multiple reference row indexes of a current block, and the multiple reference row indexes are used to indicate the position of the index of the reference row selected by the current block in an MRL index list, and the MRL index list is an MRL index list sorted based on the current block, or a set MRL index list.

[0018] An embodiment of the present disclosure further provides a device for sorting a multi-reference row index list, comprising a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement the multi-reference row index list sorting method as described in any embodiment of the present disclosure.

[0019] An embodiment of the present disclosure further provides a video decoding device, comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method as described in any embodiment of the present disclosure when executing the computer program.

[0020] An embodiment of the present disclosure further provides a video encoding device, including a processor and a memory storing a computer program, wherein the processor can implement the video encoding method as described in any embodiment of the present disclosure when executing the computer program.

[0021] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0022] An embodiment of the present disclosure also provides a non-volatile computer-readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor, implements the multi-reference row index list sorting method as described in any embodiment of the present disclosure, or implements the video decoding method as described in any embodiment of the present disclosure, or implements the video encoding method as described in any embodiment of the present disclosure.

[0023] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

[0024] Summary of the Figures

[0025] The accompanying drawings are used to provide an understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure and do not constitute a limitation to the technical solutions of the present disclosure.

[0026] FIG1A is a schematic diagram of a coding and decoding system according to an embodiment of the present disclosure;

[0027] FIG1B is a framework diagram of an encoding end according to an embodiment of the present disclosure;

[0028] FIG1C is a framework diagram of a decoding end according to an embodiment of the present disclosure;

[0029] FIG2 is a schematic diagram of an intra-frame prediction mode according to an embodiment of the present disclosure;

[0030] FIG3 is a schematic diagram of adjacent intra-frame prediction blocks of a current block according to an embodiment of the present disclosure;

[0031] FIG4 is a schematic diagram of a template area and a template reference area of ​​a current block according to an embodiment of the present disclosure;

[0032] FIG5 is a schematic diagram of multiple reference rows around a current block according to an embodiment of the present disclosure;

[0033] FIG6 is a flowchart of a method for sorting a multi-reference row index list according to an embodiment of the present disclosure;

[0034] FIG7 is a flowchart of a video decoding method according to an embodiment of the present disclosure;

[0035] FIG8 is a flowchart of a video encoding method according to an embodiment of the present disclosure;

[0036] FIG9 is a schematic diagram of a device for sorting a multi-reference row index list according to an embodiment of the present disclosure;

[0037] FIG10 is a schematic diagram of a template and multiple reference rows around a current block according to an embodiment of the present disclosure.

[0038] Details

[0039] The present disclosure describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure.

[0040] In the description of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment described as "exemplary" or "for example" in the present disclosure should not be interpreted as being more preferred or advantageous than other embodiments. "And / or" in this article is a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "Multiple" refers to two or more than two. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present disclosure, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0041] When describing representative exemplary embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art can readily understand that these sequences can vary and still remain within the spirit and scope of the disclosed embodiments.

[0042] The local illumination compensation method and video coding and decoding method proposed in the embodiments of the present disclosure can be applied to various video coding and decoding standards, such as: H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC), AVS (Audio Video coding Standard), and other standards developed by MPEG (Moving Picture Experts Group), AOM (Alliance for Open Media), JVET (Joint Video Experts Team) and extensions of these standards, or any other customized standards.

[0043] FIG1A is a block diagram of a video encoding and decoding system that can be used in an embodiment of the present disclosure. As shown in the figure, the system is divided into an encoding end device 1 and a decoding end device 2. The encoding end device 1 generates a code stream. The decoding end device 2 can decode the code stream. The decoding end device 2 can receive the code stream from the encoding end device 1 via a link 3. The link 3 includes one or more media or devices capable of moving the code stream from the encoding end device 1 to the decoding end device 2. In one example, the link 3 includes one or more communication media that enable the encoding end device 1 to send the code stream directly to the decoding end device 2. The encoding end device 1 modulates the code stream according to a communication standard (e.g., a wireless communication protocol) and sends the modulated code stream to the decoding end device 2. The one or more communication media may include wireless and / or wired communication media and may form part of a packet network. In another example, the code stream can also be output from the output interface 15 to a storage device, and the decoding end device 2 can read the stored data from the storage device via streaming or downloading.

[0044] As shown in the figure, the encoding end device 1 includes a data source 11, a video encoding device 13 and an output interface 15. The data source 11 includes a video capture device (such as a camera), an archive containing previously captured data, an input interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. The video encoding device 13 encodes the data from the data source 11 and outputs it to the output interface 15. The output interface 15 may include at least one of a regulator, a modem and a transmitter. The decoding end device 2 includes an input interface 21, a video decoding device 23 and a display device 25. The input interface 21 includes at least one of a receiver and a modem. The input interface 21 can receive a code stream via the link 3 or from a storage device. The video decoding device 23 decodes the received code stream. The display device 25 is used to display the decoded data. The display device 25 can be integrated with other devices of the decoding end device 2 or set separately. The display device 25 is optional for the decoding end. In other examples, the decoding end may include other devices or equipment that apply the decoded data.

[0045] Based on the video coding and decoding system shown in FIG. 1A , various video coding and decoding methods can be used to implement video compression and decompression.

[0046] FIG1B is a block diagram of an exemplary video encoding device that can be used in embodiments of the present disclosure. As shown, the video encoding device 1000 includes a prediction unit 1100, a partitioning unit 1101, a residual generation unit 1102 (indicated by the circle with a plus sign after the partitioning unit 1101 in the figure), a transform processing unit 1104, a quantization unit 1106, an inverse quantization unit 1108, an inverse transform processing unit 1110, a reconstruction unit 1112 (indicated by the circle with a plus sign after the inverse transform processing unit 1110 in the figure), a filter unit 1113, a decoded image buffer 1114, and an entropy coding unit 1115. The prediction unit 1100 includes an inter-frame prediction unit 1121 and an intra-frame prediction unit 1126. The decoded image buffer 1114 may also be referred to as a decoded image buffer, a decoded picture buffer, or a decoded picture buffer. The video encoder 20 may also include more, fewer, or different functional components than shown in this example. For example, the transform processing unit 1104 and the inverse transform processing unit 1110 may be eliminated in some cases.

[0047] The division unit 1101 cooperates with the prediction unit 1100 to divide the received video data into slices, coding tree units (CTUs) or other larger units. The video data received by the division unit 1101 can be a video sequence including video frames such as I frames, P frames or B frames.

[0048] The prediction unit 1100 can divide a CTU into coding units (CUs) and perform intra-frame prediction coding or inter-frame prediction coding on the CU. When performing intra-frame prediction and inter-frame prediction on the CU, the CU can be divided into one or more prediction units (PUs).

[0049] The inter-frame prediction unit 1121 can perform inter-frame prediction on the PU and generate prediction data for the PU, which includes the prediction block of the PU, the motion information of the PU, and various syntax elements. The inter-frame prediction unit 1121 can include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit can be used to estimate motion to generate a motion vector, and the motion compensation unit can be used to obtain or generate a prediction block based on the motion vector.

[0050] The intra-prediction unit 1126 may perform intra prediction on the PU and generate prediction data for the PU. The prediction data for the PU may include a prediction block and various syntax elements for the PU.

[0051] The residual generation unit 1102 may subtract the prediction blocks of the PUs into which the CU is partitioned from the original blocks of the CU to generate the residual blocks of the CU.

[0052] The transform processing unit 1104 may divide a CU into one or more transform units (TUs). The division of a prediction unit (PU) and a transform unit (TU) may be different. A TU-associated residual block is a sub-block obtained by dividing the residual block of the CU. One or more transforms are applied to the TU-associated residual block to generate a TU-associated coefficient block.

[0053] The quantization unit 1106 may quantize the coefficients in the coefficient block based on the selected quantization parameter. The degree of quantization of the coefficient block may be adjusted by adjusting the quantization parameter (QP: Quantizer Parameter).

[0054] The inverse quantization unit 1108 and the inverse transform unit 1110 may apply inverse quantization and inverse transform to the coefficient block, respectively, to obtain a reconstructed residual block associated with the TU.

[0055] The reconstruction unit 1112 may add the reconstructed residual block and the prediction block generated by the prediction unit 1100 to generate a reconstructed image.

[0056] Filter unit 1113 performs loop filtering on the reconstructed image and stores the filtered reconstructed image in decoded image buffer 1114 as a reference image. Intra-frame prediction unit 1126 extracts reference images of blocks adjacent to the PU from decoded image buffer 1114 to perform intra-frame prediction. Inter-frame prediction unit 1121 can use the reference image of the previous frame cached in decoded image buffer 1114 to perform inter-frame prediction on the PU of the current frame image.

[0057] The entropy coding unit 1115 may perform entropy coding operations on received data (eg, syntax elements, quantized coefficient blocks, motion information, etc.).

[0058] FIG1C is a block diagram of an exemplary video decoding device that can be used in embodiments of the present disclosure. As shown, the video decoding device 101 includes an entropy decoding unit 150, a prediction unit 152, an inverse quantization unit 154, an inverse transform unit 156, a reconstruction unit 158 ​​(represented by the circle with a plus sign after the inverse transform unit 155), a filter unit 159, and a decoded image buffer 160. In other embodiments, the video decoder 30 may include more, fewer, or different functional components. For example, the inverse transform unit 155 may be eliminated in some cases.

[0059] Entropy decoding unit 150 may perform entropy decoding on the received bitstream to extract syntax elements, quantized coefficient blocks, motion information for PUs, etc. Prediction unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 159 may each perform corresponding operations based on the syntax elements extracted from the bitstream.

[0060] The inverse quantization unit 154 may perform inverse quantization on the coefficient block associated with the quantized TU.

[0061] Inverse transform processing unit 156 may apply one or more inverse transforms to the inverse quantized coefficient block to generate a reconstructed residual block for the TU.

[0062] Prediction unit 152 includes inter-prediction unit 162 and intra-prediction unit 164. If a PU is encoded using intra prediction, intra prediction unit 164 may determine an intra prediction mode for the PU based on syntax elements decoded from the codestream, perform intra prediction based on the determined intra prediction mode and reconstructed reference information of the PU's neighbors obtained from decoded picture buffer 160, and generate a prediction block for the PU. If a PU is encoded using inter prediction, inter prediction unit 162 may determine one or more reference blocks for the PU based on the motion information of the PU and corresponding syntax elements, and generate a prediction block for the PU based on the reference blocks obtained from decoded picture buffer 160.

[0063] The reconstruction unit 158 ​​may obtain a reconstructed image based on the reconstructed residual block associated with the TU and the prediction block of the PU generated by the prediction unit 152 .

[0064] The filter unit 159 may perform loop filtering on the reconstructed image, and the filtered reconstructed image is stored in the decoded image buffer 160. The decoded image buffer 160 may provide a reference image for subsequent motion compensation, intra-frame prediction, inter-frame prediction, etc., and may also output the filtered reconstructed image as decoded video data for presentation on a display device.

[0065] Based on the above-described video encoding and decoding devices, the following basic encoding and decoding process can be performed. On the encoding side, a frame of an image is divided into blocks. Intra-frame prediction, inter-frame prediction, or other algorithms are performed on the current block to generate a predicted block for the current block. The predicted block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain quantized coefficients, and the quantized coefficients are entropy encoded to generate a bitstream. On the decoding side, intra-frame prediction or inter-frame prediction is performed on the current block to generate a predicted block for the current block. The quantized coefficients obtained from the decoded bitstream are then inversely quantized and inversely transformed to obtain a residual block. The predicted block and residual block are added together to obtain a reconstructed block. The reconstructed block forms a reconstructed image. The reconstructed image is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoding side also performs similar operations as the decoding side to obtain a decoded image, which is often referred to as a reconstructed image. The decoded image obtained by the encoding side can serve as a reference frame for inter-frame prediction of subsequent frames. The block division information determined by the encoding side, as well as information and parameters for prediction, transform, quantization, entropy coding, and loop filtering modes, can be written into the bitstream if necessary. The decoding end determines the same block division information, prediction, transformation, quantization, entropy coding, loop filtering and other mode information and parameter information as the encoding end by decoding the code stream or analyzing the existing information, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end.

[0066] Although the above is an example of a block-based hybrid coding framework, the embodiments of the present disclosure are not limited thereto. With the development of technology, one or more modules in the framework and one or more steps in the process can be replaced or optimized.

[0067] Herein, the current block may be a block-level coding unit such as a current coding unit (CU) or a current prediction unit (PU) in the current image.

[0068] During intra-frame prediction, the encoder typically uses various angular and non-angular modes to predict the current block and obtain a predicted block. Based on the rate-distortion information calculated between the predicted block and the original block, the encoder selects the optimal prediction mode for the current block, encodes this prediction mode, and transmits it to the decoder via the bitstream. The decoder decodes the prediction mode and uses it to perform intra-frame prediction for the current block. Throughout the development of digital video codec standards, non-angular modes have remained relatively stable, including the mean mode (DC mode) and the planar mode (Planar mode). Angular modes have continued to increase with the evolution of digital video codec standards. For example, the H series of international digital video coding standards, H.264 / AVC, includes only eight traditional angular prediction modes and one traditional non-angular prediction mode. H.265 / HEVC expands this to 33 traditional angular prediction modes and two traditional non-angular prediction modes. In H.266 / VVC, the traditional intra-frame prediction modes are Planar mode, DC mode, and 65 angular modes, as shown in Figure 2. The DC mode is suitable for large flat areas, and its prediction value is obtained by calculating the average value of the reference pixels on the left and / or above. The Planar mode is suitable for pixel gradients, that is, for areas where pixel values ​​change slowly.

[0069] If the prediction block mode is directly encoded, 67 modes require 7 bits to encode, resulting in a very large amount of data. Based on statistical characteristics, pixel regions closer to the current block are more likely to select the same intra-frame prediction mode as the current block. This characteristic has led to the adoption of the most probable mode (MPM) technology in HEVC, VVC, and the Enhanced Compression Model (ECM). ECM is based on the VTM-10.0 reference software and integrates various new tools to further explore codec performance.

[0070] MPM technology first constructs an MPM list, which is populated with the intra prediction modes most likely to be selected by the current block. If the prediction mode selected by the current block is in the MPM list, only its index number (only 3 bits) is encoded. If the prediction mode selected by the current block is not in the MPM list but is one of the 61 non-MPM modes, the prediction mode is encoded using Truncated Binary Code (TBC) during the entropy coding stage.

[0071] In VVC, regardless of whether multiple reference lines (MRL) and intra sub-partitions (ISP) are used, the MPM list contains six prediction modes. The MPM in ECM is divided into MPM and Secondary MPM, with the MPM and Secondary MPM lists of length 6 and 16, respectively. Of the six modes in the MPM list, the Planar mode always appears first. The remaining five positions are filled using the following three steps, at which point the remaining modes automatically enter the Secondary MPM. The first step is to select the prediction mode for the five adjacent prediction blocks surrounding the current block, as shown in Figure 3. These include the prediction modes for the prediction blocks to the upper left (AL), upper (A), upper right (AR), left (L), and lower left (BL) of the current block. The second step is to reconstruct pixels surrounding the current block using a mode derived from a gradient histogram. The third step is to select an angle mode close to the angle mode selected in the first step. The Secondary MP list consists of some main angle modes other than the modes in the MPM.

[0072] Since the encoding and decoding order of the MPM flag is after the MRL mode, the encoding and decoding of the MPM in the ECM needs to depend on the MRL flag. The following table is an example:

[0073]

[0074]

[0075] Template-based intra mode derivation (TIMD) is an intra-frame prediction mode for luma frames. The TIMD mode is generated by the candidate modes and templates in the MPM list. In ECM, as shown in Figure 4, the left adjacent area and the upper adjacent area of ​​the current block (shown as Current CU) 11 constitute the template area 12 of the current block, and the template reference area 13 is set outside the template area 12 (i.e., on the left and top sides). The exemplary sizes and positions of each area are shown in the figure. The template reference area 13 can be a row adjacent to the template area above or a column adjacent to the left.

[0076] TIMD assumes that the distribution characteristics of the current block and the template area of ​​the current block are consistent. Using the reconstructed value of the template reference area as the reference row, it traverses all modes in the MPM and Secondary MPM to predict the template area and obtain the prediction result. Then, the sum of absolute transformed differences (SATD) between the reconstructed value on the template area and the prediction result of each mode is calculated, and the mode with the smallest SATD, i.e. the optimal mode, is selected as the TIMD mode for the current block. The decoding end can derive the TIMD mode using the same derivation method. If the sequence allows the use of TIMD, each current block requires a flag bit to indicate whether TIMD is used. If the intra-frame prediction mode selected for the current block is the TIMD mode, the current block is predicted using the TIMD mode, and the decoding process of the remaining luminance frames and intra-frame prediction-related syntax elements such as ISP and MPM can be skipped, thereby significantly reducing the coding bits of the mode.

[0077] After obtaining the SATD between the reconstructed value on the template area and the prediction result of each mode, the final TIMD mode to be used can be determined according to the following method:

[0078] Assume that mode1 and mode2 are two angle modes for intra-frame prediction in MPM, mode1 is the angle mode with the smallest SATD, and its SATD is cost1; mode2 is the angle mode with the second smallest SATD, and its SATD is cost2:

[0079] When cost1×2≤cost2, mode1 is used as the TIMD mode of the current block;

[0080] When cost1×2>cost2, the prediction mode obtained by weighting the prediction results of mode1 and mode2 is used as the TMID mode of the current block, also known as the TIMD fusion mode.

[0081] The weighting method and weights are shown in the following formula:

[0082] Pred=Pred mode1 ×w1+Pred mode2 ×w2

[0083]

[0084] Among them, Pred is the prediction result of the current block using TIMD fusion mode, Pred mode1 Use the prediction result of mode1 for the current block, Pred mode2The prediction result of mode2 is used for the current block, and w1 and w2 are the weights calculated according to cost1 and cost2.

[0085] In HEVC, intra prediction uses the row and column immediately above and to the left of the current block as references. If the reconstructed values ​​of these rows and columns differ significantly from the original pixel values, the prediction quality of the current block will be significantly affected. To address this issue, VVC adopts the MRL intra prediction technique. In addition to using the nearest row of reference pixels (reference line 0), VVC can also use reference lines 1 and 2 as extended reference lines for intra prediction. To reduce coding complexity, MRL is only used in non-planar modes within the MPM. When predicting for each angular mode, the encoder tries all three reference lines and selects the one with the lowest rate-distortion cost (RD cost). The index of this selected reference line is the same as the index of the selected intra prediction mode and is also encoded and sent to the decoder. The decoder decodes the reference line index and then selects the corresponding reference line for prediction based on the reference line index.

[0086] Figure 5 shows an example of four reference lines: reference line 0 221, adjacent to the current block (i.e., reference line index 0); reference line 1 222, one line away from the current block (i.e., reference line index 1); reference line 2 223, two lines away from the current block (i.e., reference line index 2); and reference line 3 224, three lines away from the current block (i.e., reference line index 3). During prediction, only the reconstructed values ​​of a portion of the reference lines can be used. In one example, the area within reference line 0 is used as the template area for the current block. Reference line 0 can be referred to as the reference line within the template area, and reference lines 1 through 3 can be referred to as reference lines outside the template area.

[0087] In this article, the indexes of the reference rows are numbered in the manner shown in FIG5 . Except for the reference row with index 0, the other reference rows are extended reference rows.

[0088] In this article, although the reference row is called a "row", this is for convenience of expression. A reference row actually includes one row and one column. The reconstructed value of the reference row used in prediction also includes the reconstructed value of one row and one column. This is the same as the common description method in the industry.

[0089] In ECM, decoding methods related to multiple reference row indices are shown in Tables 1 and 2. Table 1 is used when the current block does not use TIMD, and Table 2 is used when the current block uses TIMD.

[0090] Table 1

[0091]

[0092]

[0093] In ECM, the MRL tool can use more reference lines. In order to encode the reference line selected for the current block (i.e., the reference line selected for the current block), the indices of multiple candidate reference lines are filled into a list. In this article, this list is called a multiple reference line index list, abbreviated as an MRL index list, but it can also be called a multiple reference line list, a candidate reference line list, a reference line index list, etc. When the current block does not use TIMD, the length of the multiple reference line index list is 6, that is, there are 6 positions in total that can be filled with the indices of 6 reference lines. The indices of these 6 reference lines and their order are fixed, expressed as MULTI_REF_LINE_IDX[6] = {0, 1, 3, 5, 7, 12}. In this multiple reference line index list, the index filled in the first position is 0, which is the index of the reference line closest to the current block. The indices filled in the second to sixth positions are 1, 3, 5, 7, and 12, respectively, which are the indices of the 5 extended reference lines from the nearest to the farthest distance from the current block.

[0094] When encoding the reference row selected by the current block, due to the existence of the multi-reference row index list, the index of the reference row is not encoded directly. Instead, the index of the position of the reference row index in the multi-reference row index list is encoded. This is called the multi-reference row index (multiRefIdx). The multi-reference row index is used to indicate the position of the reference row index selected by the current block in the multi-reference row index list. Taking the MRL index list {0,1,3,5,7,12} as an example, the indices of the first to sixth positions are 0 to 5, respectively. If the index of the reference row selected by the current block is 7, and index 7 is the fifth position in the multi-reference row index list, the multi-reference row index is 4.

[0095] The multi-reference row index (multiRefIdx) can be encoded using a unary truncation code based on a context model, but is not limited thereto. The smaller the value of the multi-reference row index, the shorter the code length and the faster the decoding.

[0096] The MRL mode can also be used simultaneously with the TIMD mode. When using TIMD, the length of the MRL index list is 3, which can be filled with the indexes of 3 reference lines and the order between the indexes is fixed, expressed as MULTI_REF_LINE_IDX[3]={0,1,3}, where 0 is the index of the reference line of the template area, and 1 and 3 are the indexes of the extended reference lines.

[0097] The encoding and decoding methods of multiple reference row indexes are shown in Table 2:

[0098] Table 2

[0099]

[0100]

[0101] For ease of understanding, the above text divides MRL into two tables for example when TIMD is used and when TIMD is not used. The above two tables can also be combined into one table.

[0102] In different standards, the technique of deriving a list of most probable prediction modes (MPM) using blocks surrounding the current block may have different names. For example, in AV2 (AVM), it is called Adaptive Intra Mode Coding (AIMC), and in AVS3, in the case of screen content coding, it is called Frequency-based Intra Mode Coding (FIMC). In the case of non-screen content coding, this technique similar to MPM is always enabled. The Multiple Reference Line (MRL) technique for intra prediction using multiple reference lines is called Multiple Reference Line Selection for intra prediction (MRLS) in AV2 (AVM). This is merely a difference in name; while this embodiment uses MPM and MRL, it also covers these essentially identical techniques in other standards.

[0103] Compared to VVC, ECM expands the number of reference row candidates for MRL, but only brings a 0.09% improvement in coding performance for intra-frame coding. Research has shown that this is due to the increase in the number of MRL candidate index rows, which requires more identification positions for encoding the MRL mode, resulting in an increased additional burden. As can be seen from the MRL decoding methods shown in Tables 1 and 2 above, the multi-reference row index list consists of indexes in a specified fixed order. The closer the index of the selected reference row is to the front of the table, the lower the cost of encoding the multi-reference row index.

[0104] An embodiment of the present disclosure provides a method for sorting a multi-reference row index list, which can be used on both the encoding side and the decoding side. As shown in FIG6 , the method includes:

[0105] Step 110, predicting the template region according to at least part of the reference rows of the current block located outside the template region, and calculating an error between a reconstructed value of the template region and a predicted value obtained by prediction;

[0106] The above error can be expressed by absolute error and SAD, or by error transform absolute value and SATD, but is not limited thereto, and can also be expressed by the sum of squared differences (SSD), mean absolute difference (MAD), mean square error (MSE), etc.

[0107] Step 120 , filling the index of the reference row corresponding to the error into the MRL index list of the current block in the order of the error from small to large, to obtain a sorted MRL index list based on the current block.

[0108] In the previous embodiments, the MRL index list is set by the encoding and decoding system, and the indexes and order of the reference rows filled in the MRL index list are fixed. The embodiment of the present disclosure uses a template-based approach to predict the template area according to at least part of the reference rows of the current block located outside the template area, and calculates the error between the reconstructed value of the template area and the predicted value obtained by prediction, and fills the index of the reference row corresponding to the error into the MRL index list of the current block in order from small to large, to obtain an MRL index list sorted based on the current block. Therefore, what is constructed according to the embodiment of the present disclosure is a block-level MRL index list, in which the indexes and order of the reference rows are determined based on the distribution characteristics of the template area of ​​the current block, so that the reference rows selected by the current block are more likely to appear in the MRL index list, and are also more likely to appear in the front position in the MRL index list, that is, the values ​​of the multiple reference row indices that need to be encoded are more likely to be relatively small values, which reduces the code length, occupies less bandwidth, and also makes the decoding speed faster, thereby improving the encoding performance.

[0109] In an exemplary embodiment of the present disclosure, the template area is predicted separately based on at least part of the reference lines of the current block outside the template area, including: for each reference line in the at least part of the reference lines, the template area is predicted using the reconstructed value of the reference line and the intra-frame prediction mode selected by the current block. When the method of this embodiment is used on the encoding end, after the intra-frame prediction mode selection is completed and an intra-frame prediction mode (here referring to traditional intra-frame prediction modes such as average mode, plane mode and angle mode) is selected for the current block, the template area is predicted based on the intra-frame prediction mode selected by the current block, combined with each reference line involved in the sorting. When the method of this embodiment is used on the decoding end, after decoding the intra-frame prediction related syntax elements of the current block and determining the intra-frame prediction mode selected by the current block, the above prediction is performed again, and after obtaining the MRL index list sorted based on the current block, the reference row selected by the current block can be determined based on the multiple reference row indexes obtained by decoding.

[0110] In an exemplary embodiment of the present disclosure, the index of the reference row corresponding to the error is added to the MRL index list of the current block, starting from the K1th position of the MRL index list and filling in the index of the reference row corresponding to the error position by position, up to the K2th position, where 2≤K1≤K2≤M, where M is the length of the MRL index list. The index actually added to the MRL index list may be less than or equal to M.

[0111] In the MRL index list of the current block, the reference row where the template area is located can be filled in, such as reference row 21 shown in Figure 5, whose index is 0; the reference rows outside the template area can also be filled in, such as reference rows 22, 23 and 24 shown in Figure 5, and the indexes of these three reference rows are 1, 2, and 3 respectively. When sorting the MRL index list of the current block, the reference row where the template area is located is not used for prediction, so the indexes of the reference rows where the template area is located do not participate in the sorting, and these reference rows are more likely to be selected. In this embodiment, they are directly filled in one or more positions at the front of the MRL index list, and the indexes of the reference rows participating in the sorting are filled in position by position starting from the K1th position. K1 can be 2 or 3, but is not limited to this.

[0112] In an exemplary embodiment of the present disclosure, the method further includes: in the MRL index list sorted based on the current block, from the first position to the (K1 - 1)th position that does not participate in the sorting, the indexes of the reference rows where the template area of the current block is located are filled. Among them, the index of the reference row filled in the first position is 0. In this embodiment, when the template area is limited to only the nearest row and column of the current block, only the index 0 of the nearest reference row to the current block needs to be filled. However, in other embodiments, the template area can also be extended, such as extended to the second row above the current block and the second column on the left, or a larger area, that is, set on multiple reference rows nearest to the current block. At this time, the indexes of more reference rows can be filled into the MRL index list. However, not all the indexes of the reference rows overlapping with the template area need to be filled into the MRL index list, and a part of them can also be selected to be filled into the MRL index list.

[0113] In an exemplary embodiment of the present disclosure, the operation of filling the index of the reference row corresponding to the error into the MRL index list of the current block starts from the first position of the MRL index list and fills the indexes of the reference rows corresponding to the error one by one, and at most can be filled to the K2th position, where 1 ≤ K2 ≤ M, and M is the length of the MRL index list. Different from the foregoing embodiments, in this embodiment, it is not necessary to fill the index of the reference row where the template area of the current block is located in the MRL index list. For example, assuming that there is only one row of the reference row where the template area is located, and the index of the reference row selected by the current block is 0, that is, when not using the extended reference row, the index of the selected reference row does not need to be represented by the multi-reference row index, and the MRL usage flag can be set to false to represent it. Therefore, the index 0 does not need to be filled into the MRL index list, and the indexes of the reference rows participating in the sorting can be filled from the first position of the MRL index list in ascending order of the error.

[0114] In an exemplary embodiment of the present disclosure, when K2 < M, starting from the (K2 + 1)th position to the Mth position of the MRL index list, the indexes of M - K2 reference rows are filled one by one in the order from near to far from the current block. The M - K2 reference rows are located outside the template area and are specified by the system, and the indexes are different from the indexes already filled into the MRL index list.

[0115] In this embodiment, the index of the reference row corresponding to the error is no longer filled in the order of error from small to large at one or more positions at the end of the MRL index list, but is filled in with the designated reference row outside the template area. For example, the length M of the MRL index list is 6, and the index of the reference row sorted according to the error is only filled in the second and third positions. In an example, the set MRL index list is {0,1,3,5,7,12}, and 1,3,5,7,12 are the indexes of the designated reference rows. This embodiment makes predictions based on the reference rows with indexes 1 to 12, assuming that the two rows with the smallest errors are the reference rows with indexes 5 and 7, because these two reference rows were originally the designated reference rows. Therefore, the MRL index list sorted based on the current block is {0,5,7,1,3,12}, that is, the indexes 1,3,12 of the reference rows specified in the set MRL index list are filled in the positions at the end of the sorted MRL index list. If the two rows with the smallest error are the reference rows with indices 4 and 6, since these two rows are not designated reference rows, the sorted MRL index list is {0, 4, 6, 1, 3, 5}, that is, the indices of the designated reference rows are filled in position by position in the order from nearest to farthest from the current block.

[0116] In an exemplary embodiment of the present disclosure, the template area of ​​the current block is set on a reference line closest to the current block, that is, on a reference line with an index of 0; or, the template area of ​​the current block is set on multiple reference lines closest to the current block; such as being set on two reference lines with indexes of 0 and 1, or being set on three reference lines with indexes of 0, 1, and 2, or being set on four reference lines with indexes of 0, 1, 2, and 3, and so on.

[0117] In an exemplary embodiment of the present disclosure, at least part of the reference rows of the current block located outside the template area includes:

[0118] 1) N' reference rows closest to the current block outside the template area; or

[0119] 2) odd-numbered rows among the N' reference rows closest to the current block outside the template area; or

[0120] 3) an even-numbered row among the N' reference rows closest to the current block outside the template area; or

[0121] 4) N ​​reference lines selected from the N' reference lines closest to the current block outside the template area by comparing the errors between adjacent lines, N ≤ N'; or

[0122] 5) N reference rows specified among the N' reference rows closest to the current block outside the template area;

[0123] Where N'≤Nmax , N max The maximum number of reference rows allowed to participate in the sort.

[0124] When sorting, it is first necessary to determine the number of reference rows for reference sorting and which reference rows to select. This embodiment provides multiple selection methods.

[0125] In this embodiment, it is assumed that the current block uses TIMD and the N max and N set when the current block does not use TIMD max Both are 12, the template area is in the reference row closest to the current block, that is, the reference row with index 0, and the reference row of the current block is not allowed to exceed the upper boundary of the CTU. Then the first method mentioned above is to select the N' reference rows closest to the current block outside the template area for sorting. If the number of reference rows between the outside of the template area and the upper boundary of the CTU is greater than or equal to 12 rows, then N'=12, that is, the 12 reference rows closest to the current block outside the template area are selected for sorting. If there are not 12 reference rows, N' is equal to the number of reference rows between the outside of the template area and the upper boundary of the CTU. The second and third methods are to select the odd and even rows from the reference rows determined by the first method, respectively, to simplify the calculation.

[0126] In an example of this embodiment, the fourth method of this embodiment, namely selecting N reference rows from the N' reference rows closest to the current block outside the template region by comparing the errors between adjacent rows, can be implemented as follows: selecting the reference row closest to the current block outside the template region as the first reference row, comparing the newly selected reference row with its adjacent reference rows one by one, and selecting the adjacent reference row closest to the reference row and with an error between the two reference rows greater than a set threshold as a new reference row, and repeating this process until N' reference rows have been compared and selected. The error refers to the error between the reconstructed values ​​of two reference rows (the two reference rows involved in the comparison), or refers to the error between the predicted values ​​obtained by predicting the template region separately with the reconstructed values ​​of the two reference rows as a reference.

[0127] The following example illustrates the comparison process. For ease of illustration, reference rows outside the template region are numbered in descending order of distance from the current block. The reference row closest to the current block is called the first reference row outside the template region, referred to as the first reference row. The reference row next closest to the current block is called the second reference row outside the template region, referred to as the second reference row, and so on. Furthermore, assume that the number of reference rows between the template region and the upper boundary of the CTU is greater than 12.

[0128] The comparison process is as follows: after selecting the first reference row outside the template area, compare the error between the first reference row and its adjacent row, i.e., the second reference row. If the error is less than the set threshold, continue to compare the error between the first reference row and the third reference row. If the error is greater than the set threshold, the third reference row is used as a newly selected row; then compare the error between the third reference row and its adjacent row (i.e., the fourth reference row). If the error is greater than the set threshold, the fourth reference row is used as a newly selected row; then compare the error between the fourth reference row and its adjacent row (i.e., the fifth reference row), and so on, until the 12th reference row is compared. If the error between the 12th reference row and the most recently selected reference row is greater than the set threshold, the 12th reference row is selected for sorting, otherwise the 12th reference row is not selected.

[0129] As can be seen, when using the above four methods, the MRL index list sorting method of the disclosed embodiment is not limited to sorting itself, but can also select reference rows for sorting, thereby also selecting reference rows whose indexes are populated into the MRL index list. This helps find the optimal reference rows and improves video encoding efficiency.

[0130] However, the present disclosure can also use designated reference rows for sorting. In the fifth method described above, the reference rows used for sorting are N designated reference rows from the N' reference rows closest to the current block outside the template area, for example, reference rows with designated indices 1, 3, 5, 7, and 12. This is equivalent to sorting the reference rows in the set MRL index list, without introducing any new reference rows.

[0131] In an example of this embodiment, the N' reference rows closest to the current block outside the template area are reference rows that do not exceed the upper boundary of the coding tree unit CTU where the current block is located;

[0132] In one example of this embodiment, the current block uses the template-based intra mode to derive the TIMD. max and N set when the current block does not use TIMD max are greater than or equal to 3. In one example, N is set when the current block uses TIMD. max is 12, and the current block does not use TIMD. max It can also be set to 12. max The specific value of is not limited in this disclosure, and can be 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and values ​​greater than 15, such as 18, 24, 36, etc.

[0133] The present disclosure also provides a video decoding method, as shown in FIG7 , including:

[0134] Step 210: Decode the intra prediction related syntax elements of the current block in the code stream, and if the current block allows the use of multiple reference lines (MRL), continue decoding the multiple reference line index of the current block;

[0135] As mentioned above, the multiple reference row index is used to indicate the position of the index of the reference row selected by the current block in the MRL index list.

[0136] Step 220: If it is determined according to the decoding result that the MRL index list of the current block is allowed to be sorted, the MRL index list of the current block is sorted according to the method described in any embodiment of the present disclosure to obtain a sorted MRL index list based on the current block;

[0137] Step 230 : Determine the reference row selected for the current block according to the multiple reference row indices and the MRL index list sorted based on the current block, and perform intra prediction on the current block according to the reference row selected for the current block.

[0138] During the decoding process of this embodiment, when it is determined that the MRL index list of the current block is allowed to be sorted, the MRL index list of the current block is first sorted according to the method of the above-mentioned embodiment of the present disclosure to obtain the sorted MRL index list based on the current block; then, based on the multiple reference row indices obtained by decoding and the sorted MRL index list based on the current block, the reference row selected for the current block is determined, and then intra-frame prediction is performed on the current block based on the reference row selected for the current block. This embodiment can use the MRL index list sorted based on the current block to decode the multiple reference row indices, so that the reference row selected by the current block is more likely to be located at the front of the MRL index list, thereby saving encoding overhead and improving encoding performance.

[0139] In an exemplary embodiment of the present disclosure, when decoding the intra-frame prediction-related syntax elements of the current block in the code stream, the intra-frame prediction mode selected by the current block is decoded for sorting the MRL index list of the current block; when intra-frame prediction is performed on the current block based on the reference row selected by the current block, intra-frame prediction is performed on the current block based on the reference row selected by the current block and the intra-frame prediction mode selected by the current block.

[0140] In an exemplary embodiment of the present disclosure, the method further includes: if it is determined based on the decoding result that sorting of the MRL index list for the current block is not permitted, determining a selected reference row for the current block based on the multiple reference row index and a set MRL index list, and performing intra prediction on the current block based on the selected reference row for the current block. In this embodiment, if it is determined that sorting of the MRL index list for the current block is not permitted, the set MRL index list is used to determine the reference row pointed to by the multiple reference row index.

[0141] In an example of this embodiment, when it is determined that the current block does not use TIMD, the set MRL index list is {0, 1, 3, 5, 7, 12}; when it is determined that the current block uses TIMD, the set MRL index list is {0, 1, 3} or {0, 1, 2}.

[0142] Whether to allow sorting of the MRL index list of the current block can be determined according to the decoded multiple reference row index, or can be directly indicated by using a flag.

[0143] In an exemplary embodiment of the present disclosure, the method further includes: when it is determined that the MRL index list of the current block is allowed to be sorted, setting a flag of the current block indicating that the MRL index list is allowed to be sorted to true.

[0144] In another exemplary embodiment of the present disclosure, any one or more of the following conditions are used as sufficient conditions for not allowing the MRL index list of the current block to be sorted:

[0145] The current block does not use multiple reference lines MRL;

[0146] The multi-reference row index indicates a position in the MRL index list that does not participate in the sorting, such as the first position;

[0147] The current frame is a non-luminance frame;

[0148] The current block is located at the upper boundary of the coding tree unit CTU;

[0149] The intra prediction mode selected by the current block is a specified mode, and the specified mode includes any one or more of the following modes: planar mode, TIMD fusion mode with one mode being a planar mode, average mode, horizontal angle mode, vertical angle mode, and wide angle mode;

[0150] The current block size does not meet the set conditions;

[0151] The aspect ratio of the current block does not meet the set conditions.

[0152] If none of the conditions (not limited to the conditions listed above) for not allowing the sorting of the MRL index list of the current block are met, it is determined that the sorting of the MRL index list of the current block is allowed.

[0153] In one example, either the current block using TIMD or the current block not using TIMD can be used as a sufficient condition for not allowing sorting of the MRL index list of the current block. That is, in this case, sorting of the MRL index list is only allowed when the current block uses TIMD or the current block does not use TIMD.

[0154] In an exemplary embodiment of the present disclosure, when the current block uses MRL, the method further includes: decoding syntax elements of some transform modes of the current block that can be used simultaneously with MRL, and skipping syntax elements of some transform modes that cannot be used simultaneously with MRL.

[0155] This embodiment allows skipping of some transform modes when the current block uses MRL. The MRL mode is positioned relatively late among all intra-frame prediction modes. For example, the determination of whether MRL is used requires confirmation that the current mode is not DIMD, BDPCM, MIP, or TIMD. In VVC, the residual signal can use both primary and secondary transforms, making the transformed residual easier to encode. The primary transform includes DCT2 and DST7. The secondary transform further transforms the residual after the primary transform using DCT2. In ECM, the primary transform modes have also been further expanded. Considering that selecting different modes requires additional bits to represent, which incurs codeword overhead, and that traversing these transform modes at the encoder end increases encoding time, when multiple reference line mode is selected for intra-frame prediction, the primary transform can be restricted to using only DCT2 mode with the secondary transform, or only primary transform, or only DCT2. This allows the encoding of the corresponding transform mode index to be skipped based on whether the MRL mode has been selected.

[0156] In an exemplary embodiment of the present disclosure, when the current block uses MRL, the method further includes: during decoding of the intra prediction mode selected for the current block, allowing decoding of syntax elements related to the second MPM. This embodiment allows MRL and the second MPM mode to be used simultaneously.

[0157] In an exemplary embodiment of the present disclosure, decoding the multiple reference row indexes of the current block includes: decoding the multiple reference row indexes of the current block according to different MRL index lists when the current block uses TIMD and when the current block does not use TIMD.

[0158] An embodiment of the present disclosure further provides a video encoding method, as shown in FIG8 , including:

[0159] Step 310 , performing intra-frame prediction on the current block to determine a reference row selected for the current block;

[0160] Step 320: If it is determined that the MRL index list of the current block is allowed to be sorted, sort the MRL index list of the current block according to the method described in any one of claims 1 to 11 to obtain a sorted MRL index list based on the current block;

[0161] Step 330 : Determine and encode multiple reference row indexes of the current block according to the position of the index of the reference row selected by the current block in the MRL index list sorted based on the current block.

[0162] During the encoding process of this embodiment, when it is determined that the MRL index list of the current block is allowed to be sorted, the MRL index list of the current block is first sorted according to the method of the above-mentioned embodiment of the present disclosure to obtain the MRL index list after sorting the current block; then, based on the position of the index of the reference row selected by the current block in the MRL index list after sorting the current block, the multiple reference row indexes of the current block are determined and encoded. This embodiment can use the MRL index list based on the sorting of the current block to encode the multiple reference row indexes, so that the reference row selected by the current block is more likely to be located at the front of the MRL index list, thereby saving the encoding overhead of the multiple reference row indexes and improving the encoding performance.

[0163] In an exemplary embodiment of the present disclosure, before sorting the MRL index list of the current block, the method further includes: determining the intra prediction mode selected for the current block.

[0164] In an exemplary embodiment of the present disclosure, the method further includes: if it is determined that sorting of the MRL index list for the current block is not permitted, determining and encoding multiple reference row indexes for the current block based on the position of the index of the selected reference row for the current block in the set MRL index list. In this embodiment, if it is determined that sorting of the MRL index list for the current block is not permitted, the position of the selected reference row in the set MRL index list is used to determine the multiple reference row indexes for the current block.

[0165] In an exemplary embodiment of the present disclosure, the method further includes: if it is determined that sorting of the MRL index list of the current block is permitted, setting a flag for the current block indicating that sorting of the MRL index list is permitted to true. In this embodiment, whether sorting of the MRL index list of the current block is permitted is directly indicated using a flag.

[0166] In an exemplary embodiment of the present disclosure, any one or more of the following conditions are used as sufficient conditions for not allowing the MRL index list of the current block to be sorted:

[0167] The current block does not use MRL;

[0168] The reference row selected by the current block is located in the template area of ​​the current block;

[0169] The current frame is a non-luminance frame;

[0170] The current block is located at the upper boundary of the coding tree unit CTU;

[0171] The intra prediction mode selected by the current block is a specified mode, and the specified mode includes any one or more of the following modes: planar mode, TIMD fusion mode with one mode being a planar mode, average mode, horizontal angle mode, vertical angle mode, and wide angle mode;

[0172] The current block size does not meet the set conditions;

[0173] The aspect ratio of the current block does not meet the set conditions;

[0174] The current block uses TIMD or the current block does not use TIMD.

[0175] When none of the conditions (not limited to the above ones) for not allowing the sorting of the MRL index list of the current block is satisfied, it is determined that the sorting of the MRL index list of the current block is allowed.

[0176] In an exemplary embodiment of the present disclosure, different MRL index lists are used when the current block uses TIMD and when the current block does not use TIMD.

[0177] The determining and encoding of multiple reference row indexes of the current block includes:

[0178] When the current block uses TIMD, the multiple reference row indexes of the current block are determined and encoded according to the position of the index of the reference row selected by the current block in the first MRL index list;

[0179] When the current block does not use TIMD, the multiple reference row indexes of the current block are determined and encoded according to the position of the index of the reference row selected by the current block in the second MRL index list;

[0180] The length of the second MRL index list is greater than or equal to the length of the first MRL index list.

[0181] In an exemplary embodiment of the present disclosure, when the current block uses MRL, the method further includes: encoding syntax elements of some transform modes of the current block that can be used simultaneously with MRL, and skipping encoding of some transform modes that cannot be used simultaneously with MRL.

[0182] In an exemplary embodiment of the present disclosure, when the current block uses MRL and the mode selected by the current block is not the TIMD mode, the method further includes: after encoding the multiple reference row indexes, continuing to encode the intra-frame prediction mode selected by the current block, and during the process of encoding the intra-frame prediction mode selected by the current block, allowing encoding of syntax elements related to the second MPM.

[0183] An embodiment of the present disclosure further provides a bitstream, wherein the bitstream includes multiple reference row indexes for a current block, the multiple reference row indexes being used to indicate the positions of the indexes of reference rows selected for the current block in an MRL index list, the MRL index list being a sorted MRL index list based on the current block, or a preset MRL index list. The bitstream of this embodiment can be generated by the video encoding method described in any embodiment of the present disclosure.

[0184] An embodiment of the present disclosure further provides a multi-reference row index list sorting device, as shown in FIG9 , comprising a processor 71 and a memory 73 storing a computer program, wherein the processor 71 can implement the multi-reference row index list sorting method as described in any embodiment of the present disclosure when executing the computer program.

[0185] An embodiment of the present disclosure further provides a video decoding device, see FIG9 , comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method as described in any embodiment of the present disclosure when executing the computer program.

[0186] An embodiment of the present disclosure further provides a video encoding device, see FIG9 , comprising a processor and a memory storing a computer program, wherein the processor can implement the video encoding method as described in any embodiment of the present disclosure when executing the computer program.

[0187] The processors of the above embodiments of the present disclosure may be general-purpose processors, including central processing units (CPUs), network processors (NPs), microprocessors, etc., or other conventional processors; the processors may also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components; or combinations of the above devices. That is, the processors of the above embodiments may be any processing device or combination of devices that implement the various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. If the embodiments of the present disclosure are partially implemented in software, the instructions for the software may be stored in a suitable non-volatile computer-readable storage medium, and one or more processors may be used to execute the instructions in hardware to implement the methods of the embodiments of the present disclosure.

[0188] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0189] An embodiment of the present disclosure also provides a non-volatile computer-readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor, implements the multi-reference row index list sorting method as described in any embodiment of the present disclosure, or implements the video decoding method as described in any embodiment of the present disclosure, or implements the video encoding method as described in any embodiment of the present disclosure.

[0190] An embodiment of the present disclosure provides a video decoding method for sorting a multi-reference row index list of a current block during the decoding process. This embodiment, based on template region sorting, is an operation that both the encoder and the decoder need to perform. It acts on the position of a selected reference row in the MRL index list represented by a multi-reference row index from the MRL index list, and then determines the selected reference row for the current block in combination with the MRL index list sorted based on the current block, and then uses the determined reference row and the selected intra-frame prediction mode (traditional prediction mode) for prediction.

[0191] This embodiment uses a specific example to illustrate how to sort the MRL index list using the sum of absolute differences (SAD) between the predicted values ​​(pred) and the reconstructed values ​​(reco) within the template region. (x, -1) and (-1, y) are the coordinates relative to the top-left corner (0, 0) of the current block. In this embodiment, the template region of one row and one column of the current block is shown in the hatched area in Figure 10. This figure shows the six reference rows outside the current block's template region.

[0192] The method of this embodiment includes:

[0193] first step

[0194] The decoder parses the intra prediction mode, confirms whether the MRL-related syntax elements parsed by the decoder are used on the current block or not, and decodes the multiple reference row index of the current block.

[0195] The syntax table related to this embodiment is shown in the following table:

[0196]

[0197]

[0198] In the table, intra_luma_mrl_tab_idx defines the multi-reference row index when the current block does not use TIMD (i.e., intra_timd_flag), to indicate the position of the index of the reference row selected by the current block in the sorted MRL index list, and its value range is 0 to 5. intra_timd_luma_ref_idx defines the multi-reference row index when the current block uses TIMD, to indicate the position of the index of the reference row selected by the current block in the set MRL index list, and its value range is 0 to 2. In both cases, the multi-reference row index can use truncated unary code encoding and decoding based on the context model.

[0199] In this example, when the current block uses TIMD, the set MRL index list is not sorted. However, in other examples, when the current block uses TIMD, a sorted MRL index list for the current block can also be generated. When sorting the MRL index list when the current block uses TIMD, only the reference rows in the set MRL index list with the index filled in can be sorted, or more reference rows can be introduced, such as the same reference rows as when the current block does not use TIMD to participate in the sorting. At this time, a syntax element similar to intra_luma_mrl_tab_idx can be used to replace intra_timd_luma_ref_idx.

[0200] The timd in the table of this article is the same concept as TIMD in the text, and other concepts are similar.

[0201] Step 2

[0202] After the decoding phase, before predicting the current block, if the multi-reference row index of the current block is not 0, that is, the index of the selected reference row is not 0, the pre-set MRL index list is not used. Instead, the MRL index list of the current block needs to be sorted. This sorting is performed only for the extended reference rows (i.e., the reference rows with an index not equal to 0 in this embodiment).

[0203] The sorting process is as follows.

[0204] First, determine the maximum number of extended reference lines allowed for the current block's position in the image. In principle, the upper reference lines that can be used by the current block should not exceed the upper CTU boundary. In ECM, the maximum size of a CTU can be 256x256. When the y coordinate of the upper left corner of the current block in the image is taken as the remainder of 256, the remainder is the number of reference lines within the upper CTU boundary. At the same time, it is set that MRL can use a maximum of 13 reference lines (reference lines 0 to 12) in non-TIMD mode. The reference lines that can participate in the sorting are reference lines 1 to N, that is, reference lines with indexes 1 to N. Where N = min(12, (x%256)-1)). In other embodiments, if the hardware supports it, the reference lines used can also exceed the CTU boundary.

[0205] Reference lines 1 through N are outside the template area of ​​reference line 0. For each of these reference lines, use the selected intra prediction mode to predict the template area. Since the template area of ​​reference line 0 has already been reconstructed, the SAD can be calculated based on the reconstructed value and the predicted value. Sort the N SAD values ​​in ascending order, and enter the indices of the M reference lines with the smallest calculated SADs into the MRL index list.

[0206] Specifically, the MRL index list is 6 in length when the current block does not use TIMD and 3 in length when it does. The fill values ​​set for these two lists are {0, 1, 3, 5, 7, 12} and {0, 1, 3}, respectively, representing reference lines with indices 0, 1, 3, 5, 7, 12 and 0, 1, 3. This embodiment sorts reference lines 1 through N, so the index 0 in the first position of the MRL index list remains unchanged. Subsequent positions are filled with the sorted, expanded reference lines. If the current block does not use TIMD, a maximum of 5 indexes can be filled in; if the current block uses TIMD, a maximum of 2 indexes can be filled in.

[0207] During the sorting process, it should be considered that in some cases, two adjacent reference rows to be sorted may have similar reconstruction values. In this case, the prediction results of these two rows are similar, and one of the rows can be skipped to remove redundancy. There are two skipping methods:

[0208] Method 1: Calculate the SAD between the reconstructed values ​​at corresponding positions in two adjacent reference rows. When the SAD is less than a preset threshold, the two reference rows are considered too similar and one of them should be skipped. If the adjacent reference row is skipped, the selected reference row is then compared with the next closest reference row for a threshold comparison.

[0209] Method 2: Use two reference rows to predict the template area respectively, calculate the SAD value between the prediction results on the template area, and when the SAD value is less than a preset threshold, it is considered that the two rows are too similar and one of them should be skipped.

[0210] The above skipping method can skip any one of the two rows, but considering that the reference row closer to the current block is usually more suitable as a reference row, this embodiment skips the reference row farther away.

[0211] Because MRL, TIMD, and MPM can be used simultaneously in ECM, the intra-frame prediction mode sometimes chosen under TIMD is the TIMD fusion mode. The combined use of MRL and Planar modes generally does not provide additional performance gains. Therefore, when MRL is selected, if the intra-frame prediction mode is Planar or one of the TIMD fusion modes is Planar, then sorting can be omitted. Besides Planar mode, sorting can also be omitted for certain angles or modes, such as DC mode, wide angle mode, horizontal and vertical angle modes, and so on. Wide angle mode is equivalent to not sorting for rectangular prediction blocks.

[0212] When no sorting is performed, the initial reference row list such as {1, 3, 5, 7, 12} is directly used without reordering the reference rows in the list based on SAD.

[0213] This embodiment can skip some transform modes. The current MRL encoding is at a relatively late position among all intra-frame modes. In ECM, it is necessary to determine whether the current mode is MRL after determining that it is not DIMD, BDPCM, MIP, or TIMD. Considering that the subsequent transform mode encoding also needs to use more identification bits to identify the primary transform and the secondary transform, in order to reduce the coding overhead in this regard, MRL can also be restricted to be used in combination with some transform modes, for example, it can only be used with the primary transform of DCT2 or it can only be used with the primary transform and the secondary transform of DCT2. For example, when MRL is restricted to be used only with DCT2 under the primary transform, the syntax element of the decoded transform mode should check whether MRL has been selected. If it is selected, the primary transform mode will not be decoded.

[0214] The corresponding syntax is as follows:

[0215]

[0216]

[0217] In the column filled with "intra_luma_ref_tab_idx!=0&&" in the above table, some conditions are defined to determine whether to decode mts_idx (the transform mode index selected by a transform), including that the multiple reference row index is not 0.

[0218] In another example, when MRL is not used together with the secondary transform under the restriction of primary transform, the syntax element of the decoded transform mode should check whether MRL has been selected. If it is selected, the secondary transform mode will not be decoded. The corresponding syntax is as follows:

[0219]

[0220]

[0221] In the column filled with "intra_luma_ref_tab_idx!=0&&" in the table above, some conditions are defined to determine whether to decode lfnst_idx (the transform mode index selected by the secondary transform), including the multi-reference row index is not 0

[0222] Step 3

[0223] The reference row is determined according to the reordered multiple reference row index list and the decoded list index, and the current block is predicted.

[0224] This embodiment uses the error between the predicted and reconstructed values ​​of the template region to sort the multi-reference row index list in ascending order during intra-frame prediction, resulting in a multi-reference row index list sorted based on the current block. This allows reference rows that are more likely to be selected for prediction to be placed higher in the multi-reference row index list, saving coding overhead. During the sorting process, this embodiment uses threshold setting to skip potentially duplicate reference rows to reduce redundancy and improve sorting efficiency. In this embodiment, the reference row containing the template region and reference rows smaller than that reference row are excluded from the sorting and retain their original position in the candidate list.

[0225] Based on this embodiment, the number of reference rows that can be sorted can be increased. Although this embodiment uses a maximum of 12 extended reference rows for sorting, in practice, if complexity is not a concern, this restriction can be removed so that more than 12 reference rows can be used for sorting, thereby achieving better encoding and decoding performance.

[0226] In addition, the size of the template area can be increased. The template area is not limited to the reference line closest to the current block, but can continue to expand, for example, to the right of the upper template area, below the left template area, and to the gray area intersecting the upper left corner. It can also continue to expand above the upper template area and to the left of the left template area. This will use more reference line areas. For example, if the area of ​​reference line 1 is further used as the template area, reference line 1 itself cannot participate in the sorting.

[0227] On the ECM-4.0 reference software, using the method described in this embodiment to reorder the MRL index list when the current block does not use TIMD can further bring the following performance improvements on the reference software:

[0228]

[0229] By using the above method of this embodiment, the MRL index lists of the current block are sorted both when TIMD is used and when TIMD is not used, which can further bring the following performance improvement over the reference software:

[0230]

[0231] The above test also uses reference lines 1 to 12 for sorting when the current block uses TIMD. Only the two reference lines line_a and line_b with the smallest SAD are retained and filled into the sorted MRL index list: {0, line_a, line_b}.

[0232] The parameters in the table are as follows:

[0233] EncT: Encoding Time, 10X% means that after the reference row sorting technology is integrated, the encoding time is 10X% compared with before integration, which means that the encoding time increases by X%.

[0234] DecT: Decoding Time, 10X% means that after the reference row sorting technology is integrated, the decoding time is 10X% compared with before integration, which means that the decoding time increases by X%.

[0235] ClassA1 and ClassA2 are test video sequences with a resolution of 3840x2160, ClassB is a test sequence with a resolution of 1920x1080, ClassC is 832x480, ClassD is 416x240, and ClassE is 1280x720; ClassF is a screen content sequence with several different resolutions.

[0236] All intra indicates the test configuration of all intra-frame configurations.

[0237] In one or more exemplary embodiments above, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium that facilitates the transfer of a computer program from one place to another, such as according to a communication protocol. In this way, a computer-readable medium may generally correspond to a non-transitory tangible computer-readable storage medium or a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in this disclosure. A computer program product may include a computer-readable medium.

[0238] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may also be referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient (transient) media, but rather refer to non-transient tangible storage media. As used herein, disk and optical disk include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, or Blu-ray disc, among others, where disks typically reproduce data magnetically, while optical discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0239] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0240] The technical solutions of the embodiments of the present disclosure can be implemented in a wide variety of devices or equipment, including wireless mobile phones, integrated circuits (ICs), or a group of ICs (e.g., chipsets). Various components, modules, or units are described in the embodiments of the present disclosure to emphasize the functional aspects of the devices configured to perform the described techniques, but they do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units can be combined in a codec hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

Claims

1. A method for sorting a multi-reference line index list, comprising: Predicting the template region respectively according to at least some reference lines outside the template region of the current block, and calculating the error between the reconstruction value of the template region and the predicted value obtained by prediction; Filling the indexes of the reference lines corresponding to the errors into the multi-reference line (MRL) index list of the current block in ascending order of the errors, to obtain the sorted MRL index list based on the current block.

2. The method according to claim 1, wherein: The predicting the template region respectively according to at least some reference lines outside the template region of the current block includes: for each of the at least some reference lines, using the reconstruction value of the reference line and the intra-frame prediction mode selected for the current block to predict the template region.

3. The method according to claim 1, wherein: The filling the indexes of the reference lines corresponding to the errors into the MRL index list of the current block starts from the K1-th position of the MRL index list and fills the indexes of the reference lines corresponding to the errors into each position one by one, and at most can be filled to the K2-th position, where 2 ≤ K1 ≤ K2 ≤ M, and M is the length of the MRL index list.

4. The method according to claim 3, wherein: The method further includes: in the sorted MRL index list based on the current block, filling the indexes of the reference lines where the template region of the current block is located from the 1st position to the K1-1 non-sorting positions, where the index of the reference line filled in the 1st position is 0.

5. The method according to claim 1, wherein: The filling the indexes of the reference lines corresponding to the errors into the MRL index list of the current block starts from the 1st position of the MRL index list and fills the indexes of the reference lines corresponding to the errors into each position one by one, and at most can be filled to the K2-th position, where 1 ≤ K2 ≤ M, and M is the length of the MRL index list.

6. The method according to claim 3 or 5, wherein: The method further includes: when K2 < M, starting from the K2+1-th position to the M-th position of the MRL index list, filling the indexes of M-K2 reference lines into each position in the order from near to far from the current block, the M-K2 reference lines are located outside the template region and are specified by the system, and the indexes are different from the indexes already filled in the MRL index list.

7. The method according to claim 1, wherein: The at least some reference lines outside the template region of the current block include: The N' reference lines closest to the current block outside the template region; or The odd-numbered lines among the N' reference lines closest to the current block outside the template region; or The even-numbered lines among the N' reference lines closest to the current block outside the template region; or Among the N' reference lines closest to the current block outside the template region, N reference lines selected by comparing the errors between adjacent lines, N ≤ N'; or N specified reference lines among the N' reference lines closest to the current block outside the template region; Where N'≤N max , N max The maximum number of reference rows allowed to participate in the sort.

8. The method according to claim 7, wherein: The N' reference rows closest to the current block outside the template area are reference rows that do not exceed the upper boundary of the coding tree unit CTU where the current block is located; N is set when the current block uses template-based intra mode to derive TIMD max and N set when the current block does not use TIMD max Are both greater than or equal to 3.

9. The method of claim 7, wherein: The method of selecting N reference rows by comparing the errors between adjacent reference rows is as follows: A reference row closest to the current block outside the template area is selected as the first reference row, the newly selected reference row is compared with its adjacent reference rows one by one, and an adjacent reference row closest to the reference row and with an error between the two reference rows greater than a set threshold is selected as a new reference row, and this cycle is repeated until N' reference rows have been compared and selected; wherein the error refers to the error between the reconstructed values ​​of the two reference rows, or refers to the error between the predicted values ​​obtained by predicting the template area respectively with the reconstructed values ​​of the two reference rows as reference.

10. The method of claim 1 or 9, wherein: The error is expressed as absolute error and SAD, or as error transform absolute value and SATD.

11. The method of claim 1 , wherein: The template area of ​​the current block is set on a reference line closest to the current block: or The template area of ​​the current block is set on a plurality of reference lines closest to the current block.

12. A video decoding method, comprising: Decode the intra prediction related syntax elements of the current block in the code stream, and if the current block allows the use of multiple reference lines (MRL), continue decoding the multiple reference line index of the current block, where the multiple reference line index indicates the position of the index of the reference line selected by the current block in the MRL index list; When it is determined according to the decoding result that the MRL index list of the current block is allowed to be sorted, sorting the MRL index list of the current block according to the method according to any one of claims 1 to 11 to obtain a sorted MRL index list based on the current block; The reference row selected by the current block is determined according to the multiple reference row indices and the MRL index list sorted based on the current block, and intra-frame prediction is performed on the current block according to the reference row selected by the current block.

13. The method of claim 12, wherein: The method also includes: when it is determined according to the decoding result that sorting of the MRL index list of the current block is not allowed, determining the reference row selected by the current block according to the multiple reference row indices and the set MRL index list, and performing intra-frame prediction on the current block according to the reference row selected by the current block.

14. The method of claim 12, wherein: The sufficient condition for allowing the MRL index list of the current block to be sorted includes: a flag indicating that the current block allows the MRL index list to be sorted is true.

15. The method of claim 13, wherein: Any one or more of the following conditions is sufficient to disallow sorting of the MRL index list for the current block: The current block does not use multiple reference lines MRL; The multi-reference row index indicates the position in the MRL index list that does not participate in the sorting; The current frame is a non-luminance frame; The current block is located at the upper boundary of the coding tree unit CTU; The intra prediction mode selected by the current block is a specified mode, and the specified mode includes any one or more of the following modes: planar mode, TIMD fusion mode with one mode being a planar mode, average mode, horizontal angle mode, vertical angle mode, and wide angle mode; The current block size does not meet the set conditions; The aspect ratio of the current block does not meet the set conditions; The current block uses TIMD or the current block does not use TIMD.

16. The method of claim 12, wherein: In the case where the current block uses MRL, the method further includes: Decoding syntax elements of some transform modes that can be used with MRL for the current block, and skipping syntax elements of some transform modes that cannot be used with MRL; and / or During decoding of the intra prediction mode selected for the current block, syntax elements related to the second most probable mode (MPM) are allowed to be decoded.

17. The method of claim 12, wherein: The intra-frame prediction related syntax elements of the current block in the decoded code stream include: decoding the intra-frame prediction mode selected by the current block to be used for sorting the MRL index list of the current block; The performing intra-frame prediction on the current block according to the reference line selected by the current block includes: performing intra-frame prediction on the current block according to the reference line selected by the current block and the intra-frame prediction mode selected by the current block.

18. The method of claim 17, wherein: The decoding of the multiple reference row indexes of the current block includes: decoding the multiple reference row indexes of the current block according to different MRL index lists when the current block uses TIMD and when the current block does not use TIMD.

19. The method of claim 13, wherein: When it is determined that the current block does not use TIMD, the set MRL index list is {0, 1, 3, 5, 7, 12}; When it is determined that the current block uses TIMD, the set MRL index list is {0, 1, 3} or {0, 1, 2}.

20. A video encoding method, comprising: Perform intra-frame prediction on the current block to determine the reference row selected for the current block; If it is determined that sorting of the MRL index list of the current block is allowed, sorting the MRL index list of the current block according to any one of the methods described in claims 1 to 11 to obtain a sorted MRL index list based on the current block; According to the position of the index of the reference row selected by the current block in the MRL index list sorted based on the current block, multiple reference row indexes of the current block are determined and encoded.

21. The method of claim 20, wherein: The method further includes: determining and encoding multiple reference row indexes of the current block according to the position of the index of the reference row selected by the current block in the set MRL index list when determining that sorting of the MRL index list of the current block is not allowed.

22. The method of claim 20, wherein: The method further includes: in a case where it is determined that the MRL index list of the current block is allowed to be sorted, setting a flag of the current block indicating that the MRL index list is allowed to be sorted to true.

23. The method of claim 21, wherein: Any one or more of the following conditions is sufficient to disallow sorting of the MRL index list for the current block: The current block does not use multiple reference lines MRL; The reference row selected by the current block is located in the template area of ​​the current block; The current frame is a non-luminance frame; The current block is located at the upper boundary of the coding tree unit CTU; The intra prediction mode selected by the current block is a specified mode, and the specified mode includes any one or more of the following modes: planar mode, TIMD fusion mode with one mode being a planar mode, average mode, horizontal angle mode, vertical angle mode, and wide angle mode; The current block size does not meet the set conditions; The aspect ratio of the current block does not meet the set conditions; The current block uses TIMD or the current block does not use TIMD; If none of the conditions for not allowing the sorting of the MRL index list of the current block is met, it is determined that the sorting of the MRL index list of the current block is allowed.

24. The method of claim 20, wherein: The method further comprises: The determining and encoding of multiple reference row indexes of the current block includes: When the current block uses TIMD, the multiple reference row indexes of the current block are determined and encoded according to the position of the index of the reference row selected by the current block in the first MRL index list; When the current block does not use TIMD, the multiple reference row indexes of the current block are determined and encoded according to the position of the index of the reference row selected by the current block in the second MRL index list; wherein the length of the second MRL index list is greater than or equal to the length of the first MRL index list.

25. The method of claim 20, wherein: In the case that the current block uses MRL, the method further includes: encoding syntax elements of some transform modes of the current block that can be used simultaneously with MRL, and skipping encoding of some transform modes that cannot be used simultaneously with MRL.

26. The method of claim 20, wherein: When the current block uses MRL and the mode selected for the current block is not the TIMD mode, the method further includes: after encoding the multiple reference row indexes, continuing to encode the intra-frame prediction mode selected for the current block, and in the process of encoding the intra-frame prediction mode selected for the current block, allowing encoding of syntax elements related to the second MPM.

27. The method of claim 20, wherein: Before sorting the MRL index list of the current block according to the method according to any one of claims 1 to 11, the method further comprises: determining the intra prediction mode selected for the current block.

28. A code stream, wherein The code stream includes multiple reference row indexes of the current block, and the multiple reference row indexes are used to indicate the position of the index of the reference row selected by the current block in the MRL index list. The MRL index list is an MRL index list sorted based on the current block, or a set MRL index list.

29. The code stream of claim 28, wherein: The code stream is generated by the video encoding method according to any one of claims 20 to 27.

30. A device for sorting a multi-reference row index list, comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it is capable of implementing the method for sorting a multi-reference row index list as claimed in any one of claims 1 to 11.

31. A video decoding device comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it is capable of implementing the video decoding method according to any one of claims 12 to 19.

32. A video encoding device comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it can implement the video encoding method according to any one of claims 20 to 27.

33. A video encoding and decoding system, wherein: It includes the video encoding device as claimed in claim 32 and the video decoding device as claimed in claim 31.

34. A non-transitory computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, it can implement the multi-reference row index list sorting method as described in any one of claims 1 to 11, or implement the video decoding method as described in any one of claims 12 to 19, or implement the video encoding method as described in any one of claims 20 to 27.

Images