Method for constructing a candidate list, video encoding method and apparatus, video decoding method and apparatus, and video coding system

Abstract

A method for constructing a candidate list in a multi-reference line intra prediction mode, a video encoding method and apparatus, a video decoding method and apparatus, and a video encoding system are provided. A template region is predicted based on each of N×M combinations of N extended reference lines and M intra prediction modes, and K combinations are entered into the candidate list in the TMRL_IP mode in ascending order of the error between the reconstructed value and the predicted value of the template region. When the encoding side selects a combination in the candidate list for the current block, it encodes the TMRL_IP mode flag and the TMRL_IP mode index, and the decoding side decodes the TMRL_IP mode flag and the TMRL_IP mode index to determine the extended reference line and the intra prediction mode selected for the current block. In the embodiments of the present application, by encoding and decoding the combination of the extended reference line and the intra prediction mode, the coding efficiency and the coding performance can be improved.

Description

Technical Field

[0001] Reference to Related Applications This application claims the priority of PCT International Application No. PCT / CN2022 / 103946, filed on July 5, 2022, with the invention title of "Method for Constructing Candidate List, Video Encoding Method and Apparatus, Video Decoding Method and Apparatus, and Video Encoding System", and all the contents of the PCT International Application are incorporated herein by reference.

[0002] Embodiments of this application relate to video technology, but are not limited thereto. More specifically, they relate to a method for constructing a candidate list in a multi-reference line intra prediction mode, a video encoding method and apparatus, a video decoding method and apparatus, and a video encoding system.

Background Art

[0003] Digital video compression technology is mainly used to compress huge digital video data to facilitate its transmission, storage, etc. Currently, in common video coding standards such as H.266 / Multi-View Video Coding (VVC), a block-based hybrid coding framework is used. Each image (frame) in a video is divided into square largest coding units (LCUs) of the same size (e.g., 128×128, 64×64, etc.). Each largest coding unit can also be divided into rectangular coding units (CUs) based on rules. The coding unit may be further divided into a prediction unit (PU), a transform unit (TU), etc. The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filter. The prediction module includes intra prediction and inter prediction to reduce or remove redundancy within the video. For intra blocks, samples around the block are used as references for prediction, and for inter blocks, information of spatially adjacent blocks or reference information in another image is referred to. Compared with the predicted signal, the residual information is formed into a bitstream by block-based transform, quantization, and entropy coding. These technologies are described in the standards and are implemented in various fields related to video compression.

[0004] With the rapid increase in Internet video and the growing demand for video resolution, although a large amount of video data can be saved by existing digital video compression standards, there is still a need for better digital video compression technology to reduce the bandwidth and traffic burden of digital video transmission.

Summary of the Invention

[0005] The following content is an overview of the subject matter to be described in detail in this specification. This overview is not intended to limit the scope of protection of the claims.

[0006] One embodiment of the present application provides a method for constructing a candidate list in a multi-reference line intra prediction (MRL_IP) mode. The method for constructing a candidate list in the MRL_IP mode is obtaining N×M combinations of extended reference lines and intra prediction modes based on N extended reference lines and M intra prediction modes of the current block, where N≧1, M≧1, and N×M≧2 are satisfied; predicting a template region of the current block based on each of the N×M combinations, and calculating an error between a reconstructed value of the template region and a predicted value obtained by the prediction; entering K combinations corresponding to the errors in ascending order of the errors into a candidate list of the template-based multi-reference line intra prediction (TMRL_IP) mode of the current block, where 1≦K≦N×M is satisfied.

[0007] One embodiment of the present application further provides a video decoding method. The video decoding method is decoding a multi-reference line intra prediction (MRL_IP) mode flag of the current block to determine whether to use the MRL_IP mode for the current block; when it is determined to use the MRL_IP mode for the current block, subsequently decoding an MRL_IP mode index of the current block and constructing a candidate list of the MRL_IP mode of the current block, where the candidate list is filled with candidate combinations of extended reference lines and intra prediction modes for the current block; Based on the candidate list and the MRL_IP mode index, determining a combination of an extended reference line and an intra prediction mode selected for the current block, and predicting the current image based on the selected combination. The TMRL_IP mode index is used to indicate the position of the combination of the selected extended reference line and the intra prediction mode in the candidate list.

[0008] An embodiment of the present application further provides a video encoding method. The video encoding method includes constructing a candidate list of the multi-reference line intra prediction (MRL_IP) mode for the current block, where the candidate list is filled with candidate combinations of an extended reference line and an intra prediction mode for the current block; selecting, through rate-distortion optimization, a combination of a reference line and an intra prediction mode used for intra prediction for the current block; when the encoding condition of the MRL_IP mode of the current block is satisfied, encoding the MRL_IP mode flag of the current block to indicate using the MRL_IP mode for the current block, and encoding the MRL_IP mode index of the current block to indicate the position of the selected combination in the candidate list. The encoding condition includes at least that the selected combination is in the candidate list.

[0009] One embodiment of the present application further provides a bitstream. The bitstream includes block-level syntax elements used for intra prediction. The syntax elements include a multi-reference line intra prediction (MRL_IP) mode flag and an MRL_IP mode index for the current block. The MRL_IP mode flag is used to indicate whether to use the MRL_IP mode for the current block, and the MRL_IP mode index is used to indicate the position of the combination of the extended reference line and the intra prediction mode selected for the current block in the candidate list of the MRL_IP mode.

[0010] One embodiment of the present application further provides an apparatus for constructing a candidate list of the multi-reference line intra prediction mode. The apparatus for constructing the candidate list of the MRL_IP mode includes a processor and a memory storing a computer program. When the processor executes the computer program, the method for constructing the candidate list of the MRL_IP mode described in any one of the embodiments of the present application can be realized.

[0011] One embodiment of the present application further provides a video decoding apparatus. The video decoding apparatus includes a processor and a memory storing a computer program. When the processor executes the computer program, the video decoding method described in any one of the embodiments of the present application can be realized.

[0012] One embodiment of the present application further provides a video encoding apparatus. The video encoding apparatus includes a processor and a memory storing a computer program. When the processor executes the computer program, the video encoding method described in any one of the embodiments of the present application can be realized.

[0013] One embodiment of the present application further provides a video coding system. The video coding system includes a video encoding device described in any one of the embodiments of the present application and a video decoding device described in any one of the embodiments of the present application.

[0014] One embodiment of the present application further provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement a method for constructing a candidate list in a multi-reference line intra prediction (MRL_IP) mode described in any one of the embodiments of the present application, or can implement a video decoding method described in any one of the embodiments of the present application, or can implement a video encoding method described in any one of the embodiments of the present application.

[0015] After reading and understanding the drawings and the detailed description, other aspects can be understood.

Brief Description of the Drawings

[0016] The drawings are provided for understanding the embodiments of the present application, form a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation of the technical solutions of the present application.

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0017] In the present application, a plurality of embodiments are described, but the description is exemplary and not restrictive. Also, it is obvious to those skilled in the art that more examples and embodiments can be obtained within the scope included in the embodiments described in the present application.

[0018] In the description of this application, terms such as "exemplary" or "for example" mean "by way of example, illustration, explanation". In this application, any embodiment described "exemplarily" or "for example" should not be construed as being superior to other embodiments. The term "and / or" in this specification is used to explain the relevant relationship of the relevant objects and indicates that there are three types of relationships. For example, in the case of A and / or B, it indicates three situations: only A exists, A and B exist simultaneously, and only B exists. "Plurality" means two or more. Also, in order to clearly explain the technical solutions of the embodiments of this application, terms such as "first", "second", etc. are used to distinguish the same or similar ones with substantially the same functions and roles. Those skilled in the art can understand the following. Terms such as "first", "second", etc. do not limit the number or execution order, and terms such as "first", "second", etc. do not necessarily limit that they must be different.

[0019] In the description of representative exemplary embodiments, this specification may present a method and / or process as a specific sequence of steps. However, the method or process does not depend on the specific sequence of steps described in this specification, and the method or process should not be limited to the specific sequence of steps described. As understood by those skilled in the art, other orders of steps are also possible. Therefore, the specific order of steps described in the specification should not be construed as a limitation of the scope of the claims. Furthermore, claims corresponding to a method and / or process should not be limited to performing the steps in the described order. Those skilled in the art can easily understand that these orders may change and still fall within the spirit and scope of the embodiments of this application even if they change.

[0020] The local illumination compensation (LIC) method and video coding method according to the embodiments of the present application can be applied to various video coding standards. Examples include H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC), Audio Video Coding Standard (AVS), Moving Picture Experts Group (MPEG), standards created by the Alliance for Open Media (AOM), the Joint Video Experts Team (JVET), and extensions of these standards, or any other customized standards, etc.

[0021] FIG. 1A is a block diagram showing a video coding system that can be used in an embodiment of the present application. As shown in the figure, the system is divided into an encoding-side device 1 and a decoding-side device 2. The encoding-side device 1 generates a bitstream. The decoding-side device 2 can decode the bitstream. The decoding-side device 2 can receive the bitstream from the encoding-side device 1 via link 3. Link 3 includes one or more media or devices that can move the bitstream from the encoding-side device 1 to the decoding-side device 2. In one example, link 3 includes one or more communication media that enable the encoding-side device 1 to directly transmit the bitstream to the decoding-side device 2. The encoding-side device 1 can modulate the bitstream according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated bitstream to the decoding-side device 2. The one or more communication media can include wireless communication media and / or wired communication media and can form part of a packet-based network. In another example, the bitstream can also be output from the output interface 15 to a storage device. The decoding-side device 2 can read the data stored in the storage device via streaming or downloading.

[0022] As shown in FIG. 1A, the encoding-side device 1 includes a data source 11, a video encoding device 13, and an output interface 15. The data source 11 includes a video capture device (e.g., a camera), an archive containing previously captured data, a feed interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. The video encoding device 13 can encode the data from the data source 11 and output it to the output interface 15. The output interface 15 can include at least one of a regulator, a modem, and a transmitter. The decoding-side device 2 includes an input interface 21, a video decoding device 23, and a display device 25. The input interface 21 includes at least one of a receiver and a modem. The input interface 21 can receive a bitstream via link 3 or from a storage device. The video decoding device 23 decodes the received bitstream. The display device 25 is used to display the decoded data. The display device 25 may be integrated with other components of the decoding-side device 2 or may be provided separately. The decoding side may not include the display device 25. In other examples, the decoding side may include other devices or apparatuses to which the decoded data is applicable.

[0023] Based on the video coding system shown in FIG. 1A, various video coding methods can be used to achieve video compression and decompression.

[0024] FIG. 1B is a block diagram of an exemplary video encoding apparatus that can be used in an embodiment of the present application. As shown in the figure, the video encoding apparatus 1000 includes a prediction unit 1100, a splitting unit 1101, a residual generation unit 1102 (shown as a circle with a plus sign following the splitting unit 1101 in the figure), a conversion processing unit 1104, a quantization unit 1106, an inverse quantization unit 1108, an inverse conversion processing unit 1110, a reconstruction unit 1112 (shown as a circle with a plus sign following the inverse conversion processing unit 1110 in the figure), a filter unit 1113, a buffer 1114 for decoded images, and an entropy encoding unit 1115. The prediction unit 1100 includes an inter prediction unit 1121 and an intra prediction unit 1126. The buffer 1114 for decoded images may be referred to as a buffer for decoded images, a buffer for decoding images, a buffer for decoded images, etc. The video encoder 20 may also include more, fewer, or different functional components compared to this example. For example, in some cases, the conversion processing unit 1104, the inverse conversion processing unit 1110, etc. may not be included.

[0025] The splitting unit 1101, in cooperation with the prediction unit 1100, splits the received video data into slices, coding tree units (CTUs), or other relatively large units. The video data received by the splitting unit 1101 may be a video sequence including video frames such as I-frames, P-frames, and B-frames.

[0026] The prediction unit 1100 can split a CTU into coding units (CUs) and perform intra prediction coding or inter prediction coding on the CUs. When performing intra prediction and inter prediction on a CU, the CU can be split into one or more prediction units (PUs).

[0027] The inter prediction unit 1121 can perform inter prediction on the PU to generate prediction data of the PU. The prediction data includes the predicted block of the PU, the motion information of the PU, and various syntax elements. The inter prediction unit 1121 can include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit can be used for motion estimation to generate a motion vector, and the motion compensation unit can be used to obtain or generate a predicted block based on the motion vector.

[0028] The intra prediction unit 1126 can perform intra prediction on the PU to generate prediction data of the PU. The prediction data of the PU can include the predicted block of the PU and various syntax elements.

[0029] The residual generation unit 1102 can subtract the predicted block of the PU obtained by dividing the CU from the original block of the CU to generate the residual block of the CU.

[0030] The transform processing unit 1104 can divide the CU into one or more transform units (TUs). The division of the prediction unit and the division of the transform unit may be different. The residual block related to the TU is a sub-block obtained by dividing the residual block of the CU. By applying one or more transforms to the residual block related to the TU, the coefficient block related to the TU is generated.

[0031] The quantization unit 1106 can quantize the coefficients in the coefficient block based on the selected quantization parameter (QP). The degree of quantization of the coefficient block can be adjusted by adjusting the QP.

[0032] The inverse quantization unit 1108 and the inverse transform processing unit 1110 can obtain a reconstructed residual block related to the TU by applying inverse quantization and inverse transform to the coefficient block, respectively.

[0033] The reconstruction unit 1112 can generate a reconstructed image by adding the reconstructed residual block and the prediction block generated by the prediction unit 1100.

[0034] The filter unit 1113 performs in-loop filtering on the reconstructed image and stores the filtered reconstructed image in the buffer 1114 of the decoded image as a reference image. The intra prediction unit 1126 can extract a reference image of a block adjacent to the PU from the buffer 1114 of the decoded image and perform intra prediction. The inter prediction unit 1121 can perform inter prediction on the PU of the current image by using the reference image of the previous image cached in the buffer 1114 of the decoded image.

[0035] The entropy encoding unit 1115 performs an entropy encoding operation on the received data (e.g., syntax elements, quantized coefficient blocks, motion information, etc.).

[0036] FIG. 1C is a block diagram of an exemplary video decoding apparatus that can be used in an embodiment of the present application. As shown in the figure, the video decoding apparatus 101 includes an entropy decoding unit 150, a prediction unit 152, an inverse quantization unit 154, an inverse transform processing unit 156, a reconstruction unit 158 (shown as a circled + following the inverse transform processing unit 155 in the figure), a filter unit 159, and a buffer 160 for the decoded image. In other embodiments, the video decoder 30 may include more, fewer, or different functional components. For example, in some cases, it may not include the inverse transform processing unit 155, etc.

[0037] The entropy decoding unit 150 can perform entropy decoding on the received bitstream to extract information such as syntax elements, quantized coefficient blocks, and motion information of PUs. The prediction unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 159 can all execute corresponding operations based on the syntax elements extracted from the bitstream.

[0038] The inverse quantization unit 154 can inverse-quantize the coefficient blocks related to the quantized TUs.

[0039] The inverse transform processing unit 156 can apply one or more inverse transforms to the inverse-quantized coefficient blocks to generate the reconstruction residual blocks of the TUs.

[0040] The prediction unit 152 includes an inter prediction unit 162 and an intra prediction unit 164. When intra prediction coding is used for a PU, the intra prediction unit 164 determines the intra prediction mode of the PU based on the syntax elements decoded from the bitstream, and based on the determined intra prediction mode and the reconstructed reference information adjacent to the PU obtained from the decoded image buffer 160, executes intra prediction to generate the prediction block of the PU. When inter prediction coding is used for a PU, the inter prediction unit 162 determines one or more reference blocks of the PU based on the motion information of the PU and the corresponding syntax elements, and based on the reference blocks obtained from the decoded image buffer 160, generates the prediction block of the PU.

[0041] The reconstruction unit 158 can obtain a reconstructed image based on the reconstruction residual blocks related to the TUs and the prediction blocks of the PUs generated by the prediction unit 152.

[0042] The filter unit 159 can perform in-loop filtering on the reconstructed image. The filtered reconstructed image is stored in the buffer 160 of the decoded image. The buffer 160 of the decoded image can provide a reference image used for subsequent motion compensation, intra prediction, inter prediction, etc., and can also output the filtered reconstructed image as decoded video data and display it on a display device.

[0043] Based on the above video encoding device and video decoding device, the following basic coding process can be executed. On the encoding side, one image is divided into blocks, and for the current block, an intra prediction or an inter prediction is performed or other algorithms are used to generate a predicted block of the current block. The predicted block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain quantization coefficients, and the quantization coefficients are entropy-coded to generate a bitstream. On the decoding side, an intra prediction or an inter prediction is performed on the current block to generate a predicted block of the current block. On the other hand, the quantization coefficients obtained by analyzing the bitstream are inverse quantized and inverse transformed to obtain a residual block. The predicted block and the residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image. The reconstructed image is loop-filtered based on the image or blocks to obtain the decoded image. On the encoding side, in order to obtain the decoded image, a process similar to that on the decoding side is performed. The decoded image obtained on the encoding side is usually also called the reconstructed image. The decoded image can be used as a reference image for inter prediction for subsequent images. The block division information, prediction, transformation, quantization, entropy coding, loop filtering, and other mode information and parameter information determined on the encoding side are written into the bitstream as needed. The decoding side decodes the bitstream or analyzes the existing information to determine the same block division information, prediction, transformation, quantization, entropy coding, loop filtering, and other mode information and parameter information as on the encoding side. Thereby, it is ensured that the decoded image obtained on the encoding side is the same as the decoded image obtained on the decoding side.

[0044] The above is an example of a block-based hybrid coding framework, but the embodiments of the present application are not limited thereto. With the development of technology, one or more modules within the framework and one or more steps within the process can be replaced or optimized.

[0045] In this specification, the current block can be a block-level coding unit such as the current coding unit (CU) in the current image, the current prediction unit (PU), and the like.

[0046] On the encoding side, in intra prediction, usually, the current block is predicted through various angular modes and non-angular modes to obtain a predicted block. Based on the rate-distortion information calculated from the predicted block and the original block, the optimal intra prediction mode for the current block is selected, and this intra prediction mode is encoded and transmitted to the decoding side through the bitstream. On the decoding side, the intra prediction mode selected for the current block is obtained by decoding, and the intra prediction of the current block is performed according to this intra prediction mode. Through the development of previous digital video coding standards, the non-angular modes are relatively stable, including the average value mode (i.e., DC mode) and the planar mode (i.e., Planar mode). The angular modes have been increasing with the evolution of digital video coding standards. Taking the H series of international digital video coding standards as an example, the H.264 / AVC standard has only 8 traditional angular prediction modes and 1 traditional non-angular prediction mode. The H.265 / HEVC has been extended to 33 traditional angular prediction modes and 2 traditional non-angular prediction modes. In H.266 / VVC, the traditional intra prediction modes include, as shown in FIG. 2, the Planar mode, the DC mode, and 65 angular modes. Among them, the DC mode is suitable for large flat regions, and its predicted value is obtained by calculating the average value of the left and / or upper reference samples. The Planar mode is suitable when the pixels change gradually, that is, it is suitable for regions where the pixel values change slowly.

[0047] If the intra prediction mode of the current block is directly encoded, 7 bits are required to encode 67 modes, resulting in a large amount of data. From the statistical characteristics, the closer to the current block, the higher the tendency for the intra prediction mode selected for this sample region to be the same as the intra prediction mode selected for the current block. Based on this characteristic, the most probable mode (MPM) technique is adopted in HEVC, VVC, and the enhanced compression model (ECM). ECM is a reference software that can further extract coding performance by integrating various new tools based on the VTM-10.0 reference software.

[0048] For MPM, first, an MPM list is created, and the MPM list contains the six intra prediction modes that are most likely to be selected for the current block. If the intra prediction mode selected for the current block is in the MPM list, only its index (only 3 bits are required) needs to be encoded. If the intra prediction mode selected for the current block is not in the MPM list and is within the 61 non-MPM (non-Most Probable Mode) modes, this intra prediction mode is encoded using a truncated binary code (TBC) at the entropy encoding stage.

[0049] In VVC, regardless of whether Multiple Reference Line (MRL) and Intra Sub-Partitions (ISP) are applied, the MPM list has six prediction modes. The MPM in ECM is divided into MPM and Secondary MPM. The length of the MPM list is 6, and the length of the Secondary MPM list is 16. Among the six modes included in the MPM list, the Planar mode is always entered at the first position in the MPM list, and the remaining five positions are filled by the following three steps until all five positions are filled. The extra modes are automatically entered into the Secondary MPM.

[0050] In the first step, the intra prediction modes used for the prediction blocks at five neighboring positions around the current block are filled in order. As shown in Figure 3, the five positions sequentially include the positions of the upper left (AL), upper (A), upper right (AR), left (L), and lower left (BL) of the current block.

[0051] In the second step, the mode derived based on the reconstructed samples and gradient histogram around the current block is filled.

[0052] In the third step, the angle mode close to the angle of the angle mode selected in the first step is filled.

[0053] The Secondary MPM list can be composed of some main angle modes other than the intra prediction modes in MPM.

[0054] The MPM flag (mpm_flag) is encoded and decoded after the MRL mode. Therefore, the encoding and decoding of MPM in the ECM depend on the MRL flag. If the MRL mode is not used for the current block, it is necessary to decode the MPM flag to determine whether to use MPM for the current block. If the MRL mode is used for the current block, there is no need to decode the MPM flag, and it is default to use MPM for the current block.

[0055] The following table shows examples of syntax elements related to MPM.

[0056]

Table a

[0057] Template-based intra mode derivation (TIMD) is an intra prediction mode for luminance frames. The TIMD mode is generated based on the candidate intra prediction modes in the MPM list and the template area (abbreviated as template). In the ECM, as shown in FIG. 4, the left adjacent area and the upper adjacent area of the current block (for example, the current CU) 11 constitute the template area 12 of the current block. The left adjacent area therein is called the left template area (abbreviated as left template), and the upper adjacent area is called the upper template area or the upper side template area (abbreviated as upper template).

[0058] As shown in the figure, a template reference area 13 is provided outside the template area 12 (meaning the left side and the upper side), and the exemplary sizes and positions of each area are shown. As an example, both the width L1 of the left template and the height L2 of the upper template are 4. The template reference area 13 may be the adjacent row above the template area or the adjacent column on the left side.

[0059] In TIMD, it is assumed that the distribution characteristics of the current block and the template area of the current block are the same. The reconstructed value of the template reference area is used as the reconstructed value of the reference line. All intra prediction modes in MPM and Secondary MPM are traversed to predict the template area and obtain the prediction result. Then, the error between the reconstructed value of the template area and the prediction result of each mode is calculated and expressed by the sum of absolute transformed differences (SATD). The intra prediction mode that minimizes SATD, that is, the optimal intra prediction mode, is selected, and this intra prediction mode is set as the TIMD mode of the current block. On the decoding side, the TIMD mode can be derived by a similar derivation method. If TIMD can be used for the sequence, each current block requires a flag indicating whether to use TIMD. If the intra prediction mode selected for the current block is the TIMD mode, prediction is performed on the current block using the TIMD mode, and the decoding of the remaining syntax elements related to intra prediction such as ISP and MPM can be skipped, thereby significantly reducing the coding bits of the mode.

[0060] JPEG2025521793000003.jpg77154

[0061] The weighting method and weights are shown by the following formula.

Number

Number

[0062] JPEG2025521793000006.jpg26154

[0063] In HEVC, for intra prediction, prediction is performed with reference to the uppermost row and the leftmost column of the current block. If the error between the reconstructed values of this row and column and the original sample values is large, the prediction quality of the current block will also be significantly affected. To solve this problem, VVC adopts the MRL intra prediction technique, which not only uses the reference sample of the nearest 1 line, i.e., the reference line with index 0 (Reference line0), but also, for intra prediction, the reference line with index 1 (Reference line1) and the reference line with index 2 (Reference line2) can be used as extended reference lines. To reduce the coding complexity, MRL is used only for the non-planar mode in MPM. On the encoding side, when predicting using each angular mode, all three of these reference lines are tried, and through rate-distortion optimization, the one reference line with the smallest rate-distortion cost (RD Cost) for the current block is selected, and the index of the selected reference line is encoded and sent to the decoding side. The decoding side decodes to obtain the index of the reference line, determines the reference line selected for the current block according to the index of the reference line, and uses it for the prediction of the current block.

[0064] In the example shown in FIG. 5, four reference lines of the current block are illustrated, including a reference line 0 (reference line0) 221 adjacent to the current block, that is, a reference line with an index of 0, a reference line 1 (reference line1) 222 separated from the current block by one line, that is, a reference line with an index of 1, a reference line 2 (reference line2) 223 separated from the current block by two lines, that is, a reference line with an index of 2, and a reference line 3 (reference line3) 224 separated from the current block by three lines, that is, a reference line with an index of 3. The current block may have more reference lines, that is, reference lines with an index of 4 or more. When predicting, it is also possible to use only the reconstructed values of some reference lines. In this specification, the index of the reference line is numbered according to the method shown in FIG. 5.

[0065] In this specification, the reference line is referred to as a "line". Actually, one reference line includes one row and one column. Generally, the reconstructed value of the reference line used when predicting also includes the reconstructed values of one row and one column, which is the same as the usual description method in the industry.

[0066] As an example, when the template area of the current block is set to the reference line with index 0, the reference line with index 0 is called the reference line where the template area is located, and the reference lines with indices 1 to 3 are called the reference lines located outside the template area. When the template area of the current block is set to the reference lines with indices 0 and 1, the reference lines with indices 0 and 1 are the reference lines where the template area is located, and the reference lines with indices 2 and 3 are the reference lines located outside the template area. In this specification, the reference lines of the current block include the reference line with index 0 and the reference lines with indices greater than 0, and all the reference lines with indices greater than 0 are called extended reference lines. When the reference line where the template area is located is the reference line with index 0, all the extended reference lines are the reference lines outside the template area. When there are multiple reference lines where the template area is located, the reference lines where the template area is located include the extended reference lines.

[0067] In the ECM, the decoding methods related to the multi-reference lines are shown in Table 1 and Table 2 below. Table 1 is used when TIMD is not used for the current block, and Table 2 is used when TIMD is used for the current block.

[0068]

Table 1

[0069] For the MRL index, it can be encoded using a context model-based truncated unary code. After encoding, multiple bins of the context model are obtained. Bins are also called binary flags, binary symbols, binary bits, etc.

[0070] In ECM, more reference lines can be used in the MRL mode. To encode the reference lines selected for the current block, the indices of multiple candidate reference lines are listed in a list. In this specification, this list is called the multi-reference line index list (abbreviated as the MRL index list), but it can also be called the multi-reference line list, candidate reference line list, reference line index list, etc. When TIMD is not used for the current block, the length of the MRL index list is 6, that is, there are 6 positions in total, and the indices of 6 reference lines can be listed. The indices and their order of these 6 reference lines are fixed, and are 0, 1, 3, 5, 7, 12 in sequence, and can be expressed by the formula MULTI_REF_LINE_IDX[6] = { 0, 1, 3, 5, 7, 12}. In this multi-reference line index list, the index entered in the first position is 0, which is the index of the reference line closest to the current block, and the indices entered from the second to the sixth positions are 1, 3, 5, 7, 12 respectively, which are the indices of 5 extended reference lines arranged in the order from the one closest to the current block to the one farthest from it.

[0071] If the reference line selected for the current block is in the MRL index list, the multi-reference line index (multiRefIdx) indicates the position of the reference line selected for the current block in the MRL index list, and the selected reference line can be represented by encoding the multi-reference line index. Taking the MRL index list being {0, 1, 3, 5, 7, 12} as an example, the indexes from the first position to the sixth position are 0 to 5 respectively. Assuming that the reference line selected for the current block is the reference line with index 0, since the reference line with index 0 is in the first position of the MRL index list, the multi-reference line index is 0. Assuming that the reference line selected for the current block is the reference line with index 7, since the reference line with index 7 is in the fifth position of the MRL index list, the multi-reference line index is 4, and other situations are the same.

[0072] The multi-reference line index can be encoded using a method such as a context model-based truncated single-term code. The smaller the value of the multi-reference line index, the smaller the code length and the faster the decoding.

[0073] The MRL mode can also be used simultaneously with the TIMD mode. When using the TIMD mode, the length of the MRL index list is 3, the indexes of three reference lines can be filled in, and the order of the indexes is fixed, which can be represented by the formula MULTI_REF_LINE_IDX[3] = {0, 1, 3}. Accordingly, the coding method of the multi-reference line is shown in Table 2.

[0074]

Table 2

[0075] For easier understanding, the above content has been exemplified in two separate tables, showing the cases of using TIMD simultaneously with MRL and not using it. However, the two tables can also be combined into one.

[0076] Techniques for deriving the list of the most probable modes using the blocks around the current block, such as MPM, may have different names depending on the standard. For example, in AV2 (AVM), it is called Adaptive Intra Mode Coding (AIMC). In AVS3, for screen content coding, it is called Frequency-based Intra Mode Coding (FIMC). For non-screen content coding, techniques like MPM are always used. Techniques for intra prediction using multiple reference lines like MRL are called Multiple reference line selection for intra prediction (MRLS) in AV2 (AVM). However, this is just a difference in names, and the use of terms such as MPM and MRL in this embodiment is assumed to cover these substantially identical techniques in other standards as well.

[0077] In ECM, when the selected mode is not TIMD, the use of MRL technology is bound to the use of the non-planar mode in MPM. In the most common case, five extended reference lines and five intra prediction modes in MPM are combined, resulting in a total of 25 combinations. For these 25 combinations, in the most common case, three equally probable bins (syntax elements of MPM) and six bins based on the context model (one is a syntax element of MPM and five are syntax elements of MRL) need to be used, requiring a total of nine bins and resulting in a high coding cost.

[0078] When the distance from above the current block to the upper boundary of the CTU is less than 13 lines, the reference line with index 12 cannot be used. At this time, there are a total of 20 combinations of the 5 intra prediction modes in the MPM and the 4 available extended reference lines in the MRL. When the distance from above the current block to the upper boundary of the CTU is less than 8 lines, the reference line with index 7 cannot be used. At this time, there are a total of 15 combinations of the 5 intra prediction modes in the MPM and the 3 available extended reference lines in the MRL. When the distance from above the current coding block to the upper boundary of the CTU is less than 6 lines, the reference line with index 5 cannot be used. At this time, there are a total of 10 combinations of the 5 intra prediction modes in the MPM and the 4 available extended reference lines in the MRL.

[0079] That is, the bins for coding support 25 combinations of the extended reference lines in the MRL and the intra prediction modes in the MPM. However, when the number of reference lines above the current block is insufficient, there are combinations that cannot be used. To support these combinations, extra bins are required, and the 9 bins used in the MPM and MRL are not fully utilized, so it is necessary to improve the coding performance.

[0080] When performing intra prediction on the current block, it is necessary to combine the extended reference lines and the intra prediction modes. However, when coding, all of the above methods separately construct the MRL index list and the MPM list, and based on the reference line and the intra prediction mode selected for the current block, separately determine the multi-reference line index and the MPM index for the current block, and separately encode and decode the multi-reference line index and the MPM index, so that the reference line and the intra prediction mode selected for the current block can be determined on the decoding side.

[0081] Unlike the above method, embodiments of the present application provide a multi-reference line intra prediction mode, which is abbreviated as the MRL_IP (Multiple reference line&intra prediction) mode, and is a method for encoding and decoding a combination of an extended reference line and an intra prediction mode.

[0082] The video encoding method according to the embodiment of the present application is shown in FIG. 6 and includes steps 110 to 130.

[0083] Step 110: Construct a candidate list for the multi-reference line intra prediction (MRL_IP) mode of the current block, where the candidate list contains candidate combinations of the extended reference line and the intra prediction mode for the current block.

[0084] Step 120: Through rate-distortion optimization, select a combination of the reference line and the intra prediction mode used for intra prediction for the current block.

[0085] The reference lines include the reference line with index 0 and the extended reference line. One combination of the reference line and the intra prediction mode selected for the current block may be a combination of the reference line with index 0 and one intra prediction mode, or a combination of one extended reference line and one intra prediction mode.

[0086] Step 130: When the encoding condition of the MRL_IP mode of the current block is satisfied, encode the MRL_IP mode flag of the current block to indicate that the MRL_IP mode is used for the current block, and encode the MRL_IP mode index of the current block to indicate the position of the selected combination in the candidate list.

[0087] Here, the encoding condition includes at least that the selected combination is in the candidate list.

[0088] In this specification, what is entered in the candidate list is a combination of the extended reference line of the current block and the candidate intra prediction modes. This means that the combinations in the candidate list need to participate in the rate-distortion optimization of the current block, that is, they need to participate in the mode selection process for selecting the prediction mode of the current block through the rate-distortion cost. As a result, all combinations in the candidate list may be selected.

[0089] In this specification, the parameters N, M, and K indicating the number are all positive integers and do not need to be described separately.

[0090] What is entered in the candidate list of the MRL_IP mode constructed according to the embodiment of the present application is a combination of the extended reference line of the current block and the candidate intra prediction modes, and it is no longer a list of single candidate extended reference lines or a list of candidate intra prediction modes. When encoding conditions such as the combination of the reference line and the intra prediction mode selected for the current block is in the candidate list (at this time, an extended reference line is selected for the current block), that is, the selected combination is one of the combinations in the candidate list, are satisfied, using the MRL_IP mode for the current block is indicated by encoding the MRL_IP mode flag of the current block, and the position of the selected combination in the candidate list is indicated by encoding the MRL_IP mode index of the current block. The decoding side can determine the extended reference line and the intra prediction mode selected for the current block based on the MRL_IP mode flag and the MRL_IP mode index. By the combination encoding and decoding method of this embodiment, the coding cost can be reduced and the coding performance can be improved.

[0091] In an exemplary embodiment of the present application, the candidate list of the MRL_IP mode is a candidate list of the template-based multi-reference line intra prediction mode constructed according to the method of the embodiment of the present application. The "template-based multi-reference line intra prediction mode" is referred to as the TMRL_IP mode in this specification and is abbreviated as the TMRL mode. The TMRL_IP mode is an MRL_IP mode that performs prediction and ordering based on a template for a combination. The MRL_IP mode is a mode in which the extended reference line and the candidate combination of the intra prediction mode of the current block are entered into the candidate list for coding and decoding the current block, and is not limited to the TMRL_IP mode.

[0092] In the TMRL_IP mode, according to each of the N×M combinations obtained based on the N extended reference lines and M intra prediction modes of the current block, the template region of the current block is predicted, and the error between the reconstructed value of the template region and the predicted value obtained by prediction is calculated. In ascending order of the errors, the corresponding K combinations are entered into the candidate list of the TMRL_IP mode of the current block, where 1≦K≦N×M is satisfied and K is a set value. In one example, the template region is predicted based on each of the 25 combinations of the extended reference lines and the intra prediction modes in the ECM, the 25 combinations are sorted in ascending order of the errors, and the 12 combinations with the smallest error, that is, the combinations with the highest possibility of being selected, are entered into the candidate list of the TMRL_IP mode. Thereby, the number of coding bits for encoding the TMRL_IP mode index can be reduced. Even when a part of the extended reference lines {1, 3, 5, 7, 12} (for example, the extended reference lines with indexes 7 and 12) is outside the CTU boundary, based on the order of the 15 combinations, the 12 combinations with the highest possibility of being selected can be entered into the candidate list, and the coding bits of the MRL_IP mode index can still be fully and effectively utilized. By sorting in ascending order of the errors, the reference lines and prediction modes with a higher possibility of being selected for prediction are left in the candidate list, and the combinations with a higher possibility of being selected are arranged in the previous positions in the list, so that the coding cost is reduced.

[0093] The creation of the candidate list for the TMRL_IP mode will be described in detail below.

[0094] In another exemplary embodiment of the present application, the candidate list of the multi-reference line intra prediction (MRL_IP) mode for the current block is not sorted based on a template, but is created in another way. For example, based on the statistical results of a large amount of sample data, the combination of the reference line with the highest selection probability and the intra prediction mode is entered into the candidate list of the MRL_IP mode. It is not essential that only the combination of the extended reference line and the intra prediction mode can be entered into the candidate list of the MRL_IP mode. In addition, the combination of the reference line with index 0 and other intra prediction modes can also be entered into the candidate list. Even when the selected combination is the combination of the reference line with index 0 and the intra prediction mode in the candidate list, the selected combination can be indicated by encoding the MRL_IP mode index.

[0095] In an exemplary embodiment of the present application, the encoding condition further includes not using template-based intra mode derivation (TIMD) for the current block. The video encoding method further includes, when using TIMD for the current block, skipping the encoding of the MRL_IP mode flag and the MRL_IP mode index of the current block; and when not using TIMD for the current block but the selected combination is not in the candidate list, encoding the MRL_IP mode flag of the current block to indicate not using the MRL_IP mode for the current block and skipping the encoding of the MRL_IP mode index of the current block.

[0096] This embodiment is based on encoding and decoding the TIMD mode before the MRL_IP mode. When using the TIMD mode for the current block, there is no need to use the MRL_IP mode, so the encoding of the MRL_IP mode flag and the MRL_IP mode index is skipped. When not using the TIMD mode for the current block, there are two cases: the situation where the combination selected for the current block is in the candidate list of the MRL_IP mode, and the situation where the selected combination is not in the candidate list of the MRL_IP mode. When the selected combination is not in the candidate list, it is necessary to encode the MRL_IP mode flag to indicate that the MRL_IP mode is not used for the current block, and also skip the encoding of the MRL_IP mode index. When the selected combination is in the candidate list, it is necessary to encode both the MRL_IP mode flag and the MRL_IP mode index.

[0097] When not using TIMD for the current block, the MRL_IP mode flag and the MRL_IP mode index provided by this embodiment can replace the original multi-reference line index multiRefIdx. When using TIMD for the current block, it is still possible to use the multi-reference line index to indicate the selected reference line, and the multi-reference line index can be encoded.

[0098] In an exemplary embodiment of the present application, constructing the candidate list of the MRL_IP mode for the current block includes constructing the candidate list only when all the set conditions for using the MRL_IP mode for the current block are satisfied. The conditions for using the MRL_IP mode include any one or more of the following. Condition 1: The current block is a block in the luminance frame, that is, the MRL_IP mode is used only for the luminance frame (i.e., the luminance image), but the present application is not limited to this. Condition 2: The current block is not located at the upper boundary of the coding tree unit (CTU). If it is located at the upper boundary of the CTU, since there is no available reference line above the current block, in this embodiment, not being located at the upper boundary of the CTU is a condition for being able to use the MRL_IP mode. Condition 3: MRL can be used for the current block, that is, the MRL_IP mode can be used only when the MRL mode can be used. Condition 4: The size of the current block is not larger than the maximum size of the current block for which the MRL_IP mode can be used. This maximum size may be preset. Generally, larger blocks are flatter and have relatively fewer angular details. For such large blocks, the use of the MRL_IP mode can be restricted. Step 5: The aspect ratio of the current block meets the requirement for using the MRL_IP mode for the aspect ratio of the current block. For example, the use of the MRL_IP mode is permitted only when the aspect ratio of the current block is not larger than the preset value.

[0099] In an exemplary embodiment of the present application, the Golomb-Rice coding method is used to encode the MRL_IP mode index. By using the Golomb-Rice coding, candidate combinations can be reasonably grouped into categories with different codeword lengths for encoding and decoding, improving the coding efficiency.

[0100] JPEG2025521793000009.jpg13154

Number

[0101] For the prefix, each bin is encoded and decoded using a context model-based truncated unary code, and the values of the prefix and the output bits are shown in the following table.

[0102]

Table b

[0103] For each suffix, each bin is encoded and decoded using an equiprobable truncated binary code, and the suffix value and output bits are shown in the following table.

[0104]

Table c

[0105] When the value of K is different, according to the difference in the magnitude of the value of K, the calculation methods of the prefix and the suffix can be adjusted by, for example, setting different divisors. In the above example, a context model-based truncated unary code is used for the prefix, and an equiprobable truncated binary code is used for the suffix. However, as another example, an equiprobable truncated unary code can be used for the prefix, and a context model-based truncated binary code can be used for the suffix, etc., and the present application is not particularly limited.

[0106] In an exemplary embodiment of the present application, when the encoding conditions of the MRL_IP mode of the current block are satisfied, the video encoding method further includes skipping the encoding of the syntax elements of any one or more of the MPM mode, the intra subpartition split (ISP) mode, the multi-conversion selection (MTS) mode, the low-frequency non-separable transform (LFNST) mode, and the TIMD mode.

[0107] For example, when encoding and decoding the MRL_IP mode before the TIMD mode, the encoding conditions of the MRL_IP mode do not include not using TIMD for the current block. When using the MRL_IP mode for the current block, TIMD cannot be used for the current block, and the encoding of the syntax elements of the TIMD mode can be skipped.

[0108] For example, when using the MRL_IP mode for the current block, the MRL_IP mode flag and the MRL_IP mode index can simultaneously indicate the reference line and the intra prediction mode selected for the current block, and there is no need to encode and decode the syntax elements related to MPM.

[0109] For example, in a specific conversion mode, it can be restricted that the MRL_IP mode is not used simultaneously with the Multiple Transform Selection (MTS) mode and / or the Low Frequency Non-separable Transform (LFNST) mode.

[0110] One embodiment of the present application provides a method for constructing a candidate list of the TMRL_IP mode, as shown in FIG. 7. This method includes steps 210 to 230.

[0111] Step 210: Based on N extended reference lines and M intra prediction modes of the current block, obtain N×M combinations of the extended reference lines and the intra prediction modes, where N≧1, M≧1, and N×M≧2 are satisfied.

[0112] Step 220: Predict the template area of the current block based on each of the N×M combinations, and calculate the error between the reconstructed value of the template area and the predicted value obtained by prediction.

[0113] The error in this step can be represented by, for example, the Sum of Absolute Difference (SAD), or the Sum of Absolute Transformed Difference (SATD), but is not limited to these, and can also be represented by the Sum of Squared Difference (SSD), the Mean Absolute Difference (MAD), the Mean Squared Error (MSE), etc.

[0114] Step 230: Enter K combinations corresponding to the error in ascending order of error into the candidate list of the template-based multi-reference line intra prediction (TMRL_IP) mode for the current block, where 1 ≤ K ≤ N × M is satisfied.

[0115] In this embodiment, when constructing the candidate list for the TMRL_IP mode for the current block, N × M combinations are obtained based on N extended reference lines and M intra prediction modes, the template area of the current block is predicted according to each of the N × M combinations, the error between the reconstructed value and the predicted value of the template area is calculated, and the corresponding K combinations are entered into the candidate list in ascending order of the error. The candidate list created in this embodiment can realize the encoding of the combination of the extended reference line and the intra prediction mode, and can improve the encoding efficiency. Also, by predicting the template area using different combinations and ordering the errors, based on the similarity of the distribution characteristics between the current block and the template area of the current block, K combinations with a high probability of being selected from the N × M combinations can be selected, and by arranging the combinations with a high probability of being selected at the previous positions in the candidate list, the TMRL_IP mode index of the combination selected during encoding becomes smaller, and the actual encoding cost can be reduced.

[0116] In an exemplary embodiment of the present application, the template region of the current block is set to the one reference line closest to the current block, or the template region of the current block is set to a plurality of reference lines closest to the current block, and the N extended reference lines are extended reference lines located outside the template region. In FIG. 8A, the template region of the current block is set to the reference line 30 with index 0. When constructing the candidate list in TMRL_IP mode, N available extended reference lines are selected from the pre-defined extended reference lines with indices {1, 3, 5, 7, 12}. If there are more than 13 reference lines between the upper part of the current block and the CTU boundary, 5 extended reference lines with indices {1, 3, 5, 7, 12} are selected to form combinations. If there are 6 or 7 reference lines between the upper part of the current block and the CTU boundary, 3 extended reference lines with indices {1, 3, 5} are selected to form combinations.

[0117] FIG. 8A shows five extended reference lines participating in the combination, including reference line 31 with index 1, reference line 33 with index 3, reference line 35 with index 5, reference line 37 with index 7, and reference line 39 with index 12. Different from FIG. 8A, in the example shown in FIG. 8B, the template area 40 of the current block is set to two reference lines with indices 0 and 1, and the extended reference lines participating in the combination are five extended reference lines, namely, reference line 42 with index 2, reference line 43 with index 3, reference line 45 with index 5, reference line 47 with index 7, and reference line 49 with index 12. That is, in this example, N available extended reference lines are selected from the pre-defined extended reference lines with indices {2, 3, 5, 7, 12}. There are also many other options for the selection of the template area and the extended reference lines. For example, the template area of the current block can be set to three reference lines with indices 0, 1, and 2, or the template area of the current block can be set to four reference lines with indices 0 to 3. When the template area is relatively large, the prediction is relatively accurate.

[0118] In an exemplary embodiment of the present application, the N extended reference lines of the current block are extended reference lines that are located outside the template area of the current block and do not exceed the boundary of the coding tree unit (CTU) among the pre-defined N max extended reference lines. N max is the maximum number of extended reference lines that can be used in the TMRL_IP mode. In this embodiment, the N extended reference lines used in the combination are restricted to the area outside the template area of the current block and not exceeding the CTU boundary. However, if the hardware can support it, extended reference lines exceeding the CTU boundary may be used in the combination.

[0119] In an exemplary embodiment of the present application, N maxIt is 5, and the five pre-defined extended reference lines are reference lines indexed by {1, 3, 5, 7, 12} or {2, 3, 5, 7, 12}. In another exemplary embodiment of the present application, the pre-defined N max extended reference lines are the N extended reference lines starting from index 1 and closest to the current block, or the N extended reference lines starting from index 1 and having odd indices and closest to the current block, or the N extended reference lines starting from index 2 and having even indices and closest to the current block. Selecting odd or even reference lines simplifies the calculation. max extended reference lines are the N extended reference lines starting from index 1 and closest to the current block, or the N extended reference lines starting from index 1 and having odd indices and closest to the current block, or the N extended reference lines starting from index 2 and having even indices and closest to the current block. max extended reference lines are the N extended reference lines starting from index 1 and closest to the current block, or the N extended reference lines starting from index 1 and having odd indices and closest to the current block, or the N extended reference lines starting from index 2 and having even indices and closest to the current block. max extended reference lines are the N extended reference lines starting from index 1 and closest to the current block, or the N extended reference lines starting from index 1 and having odd indices and closest to the current block, or the N extended reference lines starting from index 2 and having even indices and closest to the current block. Selecting odd or even reference lines simplifies the calculation.

[0120] In an exemplary embodiment of the present application, the M intra prediction modes can be selected only from the angular mode, or only from the angular mode and the DC mode, or from the angular mode, the DC mode, and the planar mode.

[0121] In an exemplary embodiment of the present application, the M intra prediction modes are selected by the following process. First step: Determine the intra prediction modes used for the prediction blocks at a plurality of neighboring positions around the current block, select the selectable intra prediction modes among them in order, and delete the duplicate modes. When the number of intra prediction modes selected in the first step becomes M, end. If the number of intra prediction modes selected in the first step is less than M and includes the angular mode, execute the second step. Second step: Perform an extension operation on the selected first angular mode in order until the total number of selected intra prediction modes becomes M to obtain the extended angular modes, and select the extended angular modes different from all the selected angular modes.

[0122] In an example of this embodiment, in the first step, for each of the current block located at the upper boundary of the CTU and the current block not located at the upper boundary of the CTU, the predicted blocks at a plurality of neighboring positions around the current block include some or all of the predicted blocks such as the predicted block on the left side of the current block, the predicted block above, the predicted block in the upper left, the predicted block in the lower left, and the predicted block in the upper right. As shown in FIG. 3, the predicted block on the left side of the current block, the predicted block above, the predicted block in the upper left, the predicted block in the lower left, and the predicted block in the upper right are the predicted blocks respectively indicated by L, A, AL, BL, and AR in the figure. M≧5 is satisfied.

[0123] According to this example, even for the current block located at the upper boundary of the CTU, M intra prediction modes participating in the combination can be selected from among the intra prediction modes used for the predicted blocks at neighboring positions such as above, upper left, and upper right of it. That is, the access restriction for the predicted blocks at these positions is released. As a result, when constructing the candidate list for the current block located at the upper boundary of the CTU, there are more diverse intra prediction modes participating in the combination. Thereby, the diversity of the combinations in the constructed candidate list can be increased, and the coding performance can be improved.

[0124] In an example of this embodiment, the plurality of neighboring positions around the current block include five neighboring positions around the current block, and as shown in FIG. 3, the five neighboring positions are respectively the upper left, the upper, the upper right, the left, and the lower left.

[0125] In an example of this embodiment, when the number of intra prediction modes selected based on the intra prediction modes used for the predicted blocks at a plurality of neighboring positions around the current block is less than M, the first step further includes the following method, that is, Based on the reconstruction values around the current block, two intra prediction modes are derived using the gradient histogram of decoder-side intra mode derivation (DIMD), and an intra prediction mode different from any of the selected intra prediction modes is selected from the two derived intra prediction modes, when the selected intra prediction mode does not include the DC mode, selecting the DC mode, including continuously selecting the intra prediction modes participating in the combination by at least one of the above methods.

[0126] In this example, the intra prediction modes participating in the combination can be selected from the intra prediction modes derived from DIMD and / or the DC mode. By this method, the diversity of the intra prediction modes participating in the combination can be increased, and thus the diversity of the combinations in the constructed candidate list can be increased, so that the coding performance can be improved.

[0127] In an example of this embodiment, in the second step, for all the angular modes selected in the first step, starting from the first selected angular mode, the following processing is performed in order in the set order, that is, an expansion operation of the angular mode is performed to obtain the expanded angular mode, and if the expanded angular mode is different from any of the selected angular modes, the expanded angular mode is selected, and when the total number of the selected intra prediction modes reaches M, the process of ending is performed. The number of all the angular modes selected in the first step is 1 or more and 5 or less. That is, when all the intra prediction modes selected in the first step are angular modes, it is also possible to perform the expansion operation one by one for all the angular modes without reducing the number. Thereby, the possibility of the expanded angular mode can be increased, and the coding performance can be improved.

[0128] In an example of this embodiment, the process of selecting M intra prediction modes further includes, when the angular mode is not selected in the first step, or when the total number of intra prediction modes selected in the second step is still less than M, executing a third step. In the third step, the selectable intra prediction modes from a pre-defined set of intra prediction modes are determined, and until the total number of selected intra prediction modes reaches M, intra prediction modes different from any of the selected intra prediction modes are sequentially selected from the determined selectable intra prediction modes.

[0129] In an example of this embodiment, only the angular mode can be selected in the process of selecting M intra prediction modes. In another example of this embodiment, only the angular mode and the DC mode can be selected in the process of selecting M intra prediction modes. In still another example of this embodiment, the angular mode, the DC mode, and the Planar mode can be selected in the process of selecting M intra prediction modes. Since the coupling effect with the extended reference line is limited for the Planar mode, it may be excluded from the combination. The same is true for the DC mode. However, if the increase in computational complexity can be supported, the Planar mode and the DC mode can also be added to the candidate list and participate in the combination.

[0130] In an example of this embodiment, the extension operation of the angular mode includes any one or more of the operations of adding 1 and subtracting 1 to / from the angular mode, adding 2 and subtracting 2 to / from the angular mode, adding 3 and subtracting 3 to / from the angular mode, and adding 4 and subtracting 4 to / from the angular mode.

[0131] In an exemplary embodiment of the present application, among the M intra prediction modes, some or all of the intra prediction modes other than the Planar mode in the most probable mode (MPM) are used, or some or all of the intra prediction modes other than the Planar mode among the MPM and the second MPM are used, or some or all of the intra prediction modes other than the Planar mode and the DC mode among the MPM are used, or some or all of the intra prediction modes other than the Planar mode and the DC mode among the MPM and the second MPM are used, or some or all of the intra prediction modes other than the Planar mode, the DC mode, and the DIMD mode among the MPM are used, or some or all of the intra prediction modes other than the Planar mode, the DC mode, and the DIMD mode among the MPM and the second MPM are used.

[0132] In this embodiment, all of the intra prediction modes other than the Planar mode among the MPM are used as pre-defined intra prediction modes, or all of the intra prediction modes other than the Planar mode among the MPM and the second MPM are used as pre-defined intra prediction modes. When the reference lines of the current block include all of the pre-defined extended reference lines, all of the pre-defined intra prediction modes are used. When the reference lines of the current block include some of the pre-defined extended reference lines, some of the pre-defined intra prediction modes are used.

[0133] In an exemplary embodiment of the present application, the M intra prediction modes are selected in the following manner. Select M' intra prediction modes. Based on the reference lines located outside the template of the current block and the M' intra prediction modes, predict the template of the current block respectively, and calculate the errors between the reconstructed values of the template and the predicted values obtained by prediction to calculate M' errors. As the M intra prediction modes participating in the combination, select M intra prediction modes corresponding to the minimum errors from the M' intra prediction modes, where M < M' is satisfied.

[0134] The template of the current block in this embodiment is used to select M intra prediction modes from M' intra prediction modes. The aforementioned template area of the current block is used to select K combinations from N×M combinations. They may be different from each other, or they may occupy the same area.

[0135] When selecting M' intra prediction modes in this embodiment, various methods for selecting M intra prediction modes in the above embodiments of the present application can be used. For example, a method of directly selecting from the lists of MPM and the second MPM, or a method of selecting through the first step in the method of the foregoing embodiment, or a method of selecting through the first step and the second step, or a method of selecting through the first step, the second step, and the third step, etc. can be used.

[0136] In an exemplary embodiment of the present application, N≦N max 、2≦N max ≦12, 2≦M≦18, K≦K max 、6≦K max ≦36, N max is the maximum number of extended reference lines available for use in the TMRL_IP mode, and K max is the maximum number of candidate combinations available for use in the TMRL_IP mode. Here, the value ranges of the related parameters N, M, and K are shown, but these are merely exemplary and should not be regarded as limitations of the present application. In other embodiments of the present application, values outside these value ranges of this embodiment may be taken.

[0137] In an example of this embodiment, N = 5, M = 10, and K = 20 are satisfied. In another example, N = 5, M = 6, and K = 12 are satisfied. In yet another example, N = 5, M = 8, and K = 16 are satisfied. In yet another example, N = 5, M = 12, and K = 24 are satisfied.

[0138] In an exemplary embodiment of the present application, N, M, and K have at least two sets of values. The first set of values is N1, M1, K1, the second set of values is N2, M2, K2, N1≤N2, M1≤M2, K1≤K2, and N1×M1<N2×M2 are satisfied. The first set of values is used when constructing a candidate list for the current block having a first size, and the second set of values is used when constructing a candidate list for the current block having a second size. The first size is smaller than the second size. Here, the first size and the second size can each represent various sizes. For example, the first size can include 4×4, 4×8, 8×8, etc., and the second size can include 16×8, 16×16, 8×16, etc.

[0139] In this embodiment, different N, M, and K are used for current blocks of different sizes. When the size of the current block is relatively small, smaller values are used to construct the candidate list in the TMRL_IP mode. When the size of the current block is relatively large, larger values are used to construct the candidate list in the TMRL_IP mode. Thereby, a better balance between the complexity of the operation and the performance can be achieved.

[0140] In an exemplary embodiment of the present application, predicting the template area of the current block based on each of the N×M combinations means that when the current block is located at the left boundary of the picture, predicting the upper template area of the current block based on each of the N×M combinations and not predicting the left template area of the current block. According to this embodiment, the operation can be simplified and the time required for the operation can be shortened without affecting the performance.

[0141] In an exemplary embodiment of the present application, predicting the template area of the current block based on each of the N×M combinations includes predicting in the following manner based on each of the N×M combinations. An initial predicted value of the template area is calculated based on the reconstructed value of the extended reference line in the combination and the intra prediction mode in this combination, and the reconstructed value of the extended reference line is the original reconstructed value of the extended reference line or the reconstructed value after filtering. 4-tap filtering or 6-tap filtering is performed on the initial predicted value of the template area, and the filtering result is used as the predicted value of the template area obtained by prediction based on this combination.

[0142] In this embodiment, when predicting the template area of the current block, the original reconstructed value can be used in the calculation without filtering the reconstructed value of the extended reference line, and in order to reduce the complexity of the calculation and speed up the calculation, a filter with a relatively short number of taps (for example, a 4-tap filter) can be used.

[0143] In an exemplary embodiment of the present application, predicting the template area of the current block based on each of the N×M combinations and calculating the error between the reconstructed value of the template area and the predicted value obtained by prediction includes the following. Predict the entire template area of the current block based on each of the K combinations to obtain an error set consisting of the corresponding K errors, and record the maximum error in the error set as D max For each of the remaining combinations, first predict the template area on one side of the current block based on this combination, calculate the error D1 between the reconstructed value and the predicted value of the template area on this one side of the current block, and if D1≧D max then complete the prediction based on this combination, and if D1<D max then also predict the template area on the other side of the current block based on this combination, calculate the error D2 between the reconstructed value and the predicted value of the entire template area of the current block, and if D2<D maxIf so, add D2 to the error set, delete D from the error set, update the maximum error D within the error set, and if D2≥D, complete the prediction based on this combination. After completing the prediction based on N×M combinations, set the K combinations corresponding to the K errors in the error set as the K combinations corresponding to the minimum error. max in the error set, max and update the maximum error D in the error set. If D2≥D, max complete the prediction based on this combination. After completing the prediction based on N×M combinations, set the K combinations corresponding to the K errors in the error set as the K combinations corresponding to the minimum error.

[0144] In one example, the errors in the error set can be sorted in ascending order, and when adding D2 to the error set, insert D at a position where the errors in the error set can still be sorted in ascending order. However, in other examples, after completing the prediction based on N×M combinations, the K errors in the error set can also be sorted.

[0145] In this embodiment, without performing prediction and error calculation for the entire template area based on all combinations, the sorting of combinations can be completed, reducing the complexity and speeding up the operation.

[0146] In an exemplary embodiment of the present application, entering the K combinations corresponding to the errors in ascending order of error into the candidate list of the TMRL_IP mode of the current block includes entering the K combinations corresponding to the errors in ascending order of error into the candidate list starting from the first position of the candidate list. Only combinations of the extended reference line and the intra prediction mode are entered in the candidate list of the TMRL_IP mode of this embodiment. The combination of the reference line with index 0 and the intra prediction mode is indicated by other conventional modes such as MPM.

[0147] In another exemplary embodiment of the present application, starting from the i-th position in the candidate list, K combinations corresponding to the errors are entered into the candidate list in ascending order of the errors. Before the i-th position in the candidate list, a combination of a reference line with index 0 and one or more intra prediction modes is entered, and i ≥ 2 is satisfied. In this embodiment, in the candidate list of the TMRL_IP mode, not only combinations of extended reference lines and intra prediction modes but also combinations of a reference line with index 0 and intra prediction modes are entered. In this case, when the combination selected for the current block is a combination of a reference line with index 0 and an intra prediction mode, this can also be indicated by the TMRL_IP mode index. The TMRL_IP mode flag at this time can still be used.

[0148] One embodiment of the present application provides a video decoding method as shown in FIG. 9. This method includes steps 310 to 330.

[0149] Step 310: Decode the multi-reference line intra prediction (MRL_IP) mode flag of the current block to determine whether to use the MRL_IP mode for the current block.

[0150] Step 320: If it is determined to use the MRL_IP mode for the current block, subsequently decode the MRL_IP mode index of the current block, construct a candidate list for the MRL_IP mode of the current block, and the candidate list contains candidate combinations of extended reference lines and intra prediction modes for the current block.

[0151] Step 330: Based on the candidate list and the MRL_IP mode index, determine the combination of the extended reference line and the intra prediction mode selected for the current block, and predict the current image based on the selected combination.

[0152] Here, the TMRL_IP mode index is used to indicate the position in the candidate list of the combination of the selected extended reference line and the intra prediction mode.

[0153] In this embodiment, after decoding the MRL_IP mode flag and determining to use the MRL_IP mode for the current block, the combination of the extended reference line and the intra prediction mode is entered into the candidate list of the MRL_IP mode, and the combination selected for the current block is determined based on the MRL_IP mode index obtained by decoding and the candidate list, and prediction is performed. That is, the MRL_IP mode index can simultaneously indicate the extended reference line and the intra prediction mode selected for the current block without using two indexes. The coding cost can be reduced.

[0154] In an exemplary embodiment of the present application, the candidate list of the MRL_IP mode is the candidate list of the TMRL_IP mode constructed according to the method of the embodiment of the present application. However, the MRL_IP mode constructed in other embodiments of the present application is not limited to the TMRL_IP mode. For example, based on the statistical results of a large amount of sample data, the combination of the reference line with the highest selected probability and the intra prediction mode is entered into the candidate list of the MRL_IP mode. Only the combination of the extended reference line and the intra prediction mode can be entered into the candidate list of the MRL_IP mode, but the combination of the reference line with index 0 and other intra prediction modes can also be entered.

[0155] In this embodiment, when constructing the candidate list of the TMRL_IP mode, K may be a set value. However, in this embodiment, K is determined based on the TMRL_IP mode index. The TMRL_IP mode index is such that when the combination of the selected extended reference line and the intra prediction mode is at the L-th position in the candidate list, K = L, where L is a positive integer. For example, if the TMRL_IP mode index is 5, assuming the indexes are numbered from 0, it indicates that the selected combination is at the 6th position in the candidate list, and K = 6. Different from the decoding side, the encoding side needs to determine the TMRL_IP mode index based on the position of the selected combination in the candidate list and use the set value of K. In contrast, on the decoding side, after decoding the obtained TMRL_IP mode index, the constructed candidate list only needs to include the combination at the position indicated by the TMRL_IP mode index. By reducing the number of combinations entered in the candidate list of the TMRL_IP mode of the current block, the operation of constructing the candidate list can be simplified, and computing resources can be saved.

[0156] In an exemplary embodiment of the present application, before decoding the MRL_IP mode flag of the current block, the video decoding method further includes decoding the MRL_IP mode flag of the current block when all conditions for using the MRL_IP mode for the current block are satisfied. The conditions for using the MRL_IP mode include any one or more of the following, namely: The current block is a block in the luminance frame; The MRL can be used for the current block; The current block is not located at the upper boundary of the coding tree unit (CTU); The template-based intra mode derivation (TIMD) is not used for the current block; including any one or more of the above conditions.

[0157] In this embodiment, when one of the above conditions is satisfied, the use of the MRL_IP mode is not permitted, and the decoding of the MRL_IP mode flag and the MRL_IP mode index can be skipped. However, this may not be the case in other embodiments. For example, when encoding and decoding the MRL_IP mode before TIMD, the use of TIMD for the current block is not made a condition for not being able to use the MRL_IP mode. As another example, in the future, when the hardware becomes able to support the acquisition of reference lines outside the CTU boundary, the fact that the current block is located at the upper boundary of the CTU will no longer be a condition for not being able to use the MRL_IP mode, and so on.

[0158] In an exemplary embodiment of the present application, the video decoding method further includes decoding the MRL index of the current block when the MRL can be used for the current block, the current block is not located at the upper boundary of the CTU, and it is determined that TIMD is to be used for the current block. The MRL index is used to indicate the position in the MRL index list of the reference line selected for the current block. In this embodiment, although the MRL_IP mode cannot be used when TIMD is used for the current block, the MRL can still be used. Therefore, the reference line selected for the current block can still be determined by decoding the multi-reference line index of the current block, and the current block can be predicted in combination with the TIMD mode selected for the current block.

[0159] In an exemplary embodiment of the present application, when it is determined to use the MRL_IP mode for the current block based on the MRL_IP mode flag, the video decoding method further includes skipping the decoding of syntax elements of any one or more of the MPM mode, ISP mode, MTS mode, LFNST mode, and TIMD mode. Corresponding to the encoding side, when encoding and using the MRL_IP mode flag for the current block, when one or more of the above modes are skipped, the decoding side also skips the decoding of these modes when it is determined to use the MRL_IP mode flag for the current block by decoding.

[0160] One embodiment of the present application further provides a video decoding method, specifically related to the decoding process of intra prediction. The encoding process of intra prediction on the encoding side will also be described.

[0161] In this embodiment, by constructing a candidate list for the TMRL_IP mode on the encoding side and encoding and decoding the syntax elements of the TMRL_IP mode when a combination within the candidate list is selected for the current block by mode selection, and encoding and decoding the combination of the extended reference line and the intra prediction mode (conventional intra prediction modes including the DC mode, Planar mode, and multiple angular modes), the coding efficiency can be improved.

[0162] In this embodiment, based on N pre-defined extended reference lines and M intra prediction modes, a template is constructed at the position of the reference line with index 0, that is, reference line0. (x, -1) and (-1, y) are the coordinates with respect to the (0, 0) position of the upper left corner of the current block respectively. The template area of one row and one column of the current block in this embodiment can refer to the template area 30 shown in FIG. 8A. In this figure, 5 pre-defined extended reference lines with indices {1, 3, 5, 7, 12} are also provided.

[0163] When constructing the candidate list of the TMRL_IP mode, calculate the SAD between the predicted value and the reconstructed value of the template based on N×M combinations, sort the corresponding combinations in ascending order of SAD, and enter K combinations with relatively small SAD among the N×M combinations into the candidate list of the TMRL_IP mode in ascending order of SAD, where N×M≥2 is satisfied.

[0164] The construction of the candidate list of the TMRL_IP mode is an operation that needs to be performed by both the encoder and the decoder. On the encoding side, if encoding conditions such as the combination selected for the current block being in the candidate list are satisfied, it indicates that the TMRL_IP mode is used by encoding the TMRL_IP mode flag. Also, the TMRL_IP mode index is determined based on the position of the selected combination in the candidate list. For example, if it is in the first position, the TMRL_IP mode index is 0, and if it is in the second position, the TMRL_IP mode index is determined to be 1. The TMRL_IP mode index may be encoded using the Golomb-Rice coding method, but is not limited thereto.

[0165] Hereinafter, an example with N = 5, M = 6, and K = 12 will be used for explanation. The indices of the 5 pre-defined extended reference lines are {1, 3, 5, 7, 12}, and 6 intra prediction modes are selected step by step.

[0166] The video decoding method of this embodiment includes the content of steps 1 to 3.

[0167] Step 1: Decode the syntax elements related to the TMRL_IP mode.

[0168] The related syntax elements of the intra prediction mode analyzed by the decoder include the related syntax elements of modes such as TIMD and MRL. The TMRL_IP mode provided in this embodiment can be considered as an evolved form of the MRL mode, and the syntax elements of the TMRL_IP mode can also be considered as part of the syntax elements of the MRL mode. Of course, both can also be considered as two different modes.

[0169] When the TIMD mode is used for the current block, there is no change in the decoding method for the syntax elements of the MRL mode. When the TIMD mode is not used for the current block, it is necessary to decode the syntax elements of the TMRL_IP mode. As shown in the following table, decode the related syntax for the current block.

[0170]

Table d

[0171] The "cu_tmrl_flag" in the table, i.e., the TMRL_IP mode flag. When "cu_tmrl_flag" is equal to 1, it indicates that the TMRL_IP mode is used for the current block, that is, it is defined that the type of the intra prediction mode of the current luminance sample is the template-based multi-reference line intra prediction mode. When "cu_tmrl_flag" is equal to 0, it indicates that the TMRL_IP mode is not used for the current block, that is, it is defined that the type of the intra prediction mode of the current luminance sample is not the template-based multi-reference line intra prediction mode.

[0172] The "tmrl_idx" in the table, i.e., the TMRL_IP mode index, is the position in the candidate list of the TMRL_IP mode of the combination of the extended reference line and the intra prediction mode selected for the current block, and it can also be said to define the index of the selected combination (the index indicating the position of the combination) in the ordered candidate list of the TMRL_IP mode. "tmrl_idx" can be encoded and decoded using the Golomb-Rice coding method, which will not be repeated here.

[0173] As can be seen from the above table, before decoding cu_tmrl_flag, first determine whether the following conditions are met: whether MRL can be used for the current block (i.e., whether sps_mrl_enabled_flag is equal to 1), whether the current block is not located at the upper boundary of the CTU (i.e., whether (y0 % CtbSizeY) > 0 holds), and whether TIMD is not used for the current block. If these conditions are met, then decode cu_tmrl_flag. If the other two conditions are met but TIMD is used for the current block, then decode the multi-reference line index intra_luma_ref_idx of the current block.

[0174] The ISP mode flag (intra_subpartitions_mode_flag) in the table is decoded after the syntax elements related to the TMRL_IP mode. If the TMRL_IP mode is not used for the current block (!cu_tmrl_flag holds), the intra_subpartitions_mode_flag is decoded.

[0175] Step 2: Construct a candidate list for the TMRL_IP mode, and based on the TMRL_IP mode index and the candidate list, determine the extended reference line and the intra prediction mode selected for the current block.

[0176] After the analysis stage and before predicting the current block, if the TMRL_IP mode is used for the current block, it is necessary to construct a candidate list for the TMRL_IP mode, and based on the TMRL_IP mode index and the candidate list, determine the extended reference line and the intra prediction mode selected for the current block.

[0177] To construct a candidate list for the TMRL_IP mode, first, it is necessary to determine the candidate extended reference line and the candidate intra prediction mode.

[0178] ■Determine the candidate extended reference line

[0179] The candidate extended reference line is selected from the pre-defined extended reference lines. Based on the position of the current block in the image, determine the available extended reference lines among the pre-defined extended reference lines. In principle, the upper reference line available for the current block should not exceed the upper boundary of the CTU. In one example, all the extended reference lines with indices {1, 3, 5, 7, 12} that do not exceed the CTU boundary are added to the candidate extended reference lines. Also, more or fewer extended reference lines can be used to improve coding performance or reduce complexity.

[0180] ■Determine the candidate intra prediction mode

[0181] In this embodiment, instead of binding the TMRL_IP mode and the MPM (which may be bound in other embodiments), a candidate list of intra prediction modes is constructed, and the intra prediction mode used for combination is selected from this candidate list. The candidate list is derived by the following method.

[0182] First, from the conventional 67 prediction modes, the Planar mode and the DC mode are removed, or only the Planar mode is removed and the DC mode is left. The removed modes are not added to the candidate list, that is, among the TMRL_IP modes, they are not used as the intra prediction modes participating in the combination.

[0183] In this embodiment, the length of the candidate prediction mode list to be constructed is 6. When constructing the candidate prediction mode list, first, based on the intra prediction modes used for the prediction blocks at the five neighboring positions around the current block, non-overlapping intra prediction modes are selected in order and added to the candidate prediction mode list. As an example, access the prediction blocks (abbreviated as blocks) at the neighboring positions around the current block in the following order, obtain their intra prediction modes, and make selections.

[0184] Access the left block, determine whether the block is an intra-coded block, and if so and the intra prediction mode of the block is not the Planar mode, select the intra prediction mode of the block and add it to the candidate list.

[0185] Access the upper block, determine whether the block is an intra-coded block, and if so, and the intra prediction mode of the block is not the Planar mode and does not overlap with the existing intra prediction modes in the candidate list (that is, is different from all existing intra prediction modes), select the intra prediction mode of the block and add it to the candidate list.

[0186] Access the left block, determine whether the block is an inter-coding block, and if so, and if the intra prediction mode of the block is not the planar mode and does not overlap with the existing intra prediction modes in the candidate list, select the intra prediction mode of the block and add it to the candidate list.

[0187] Access the upper block, determine whether the block is an inter-coding block, and if so, and if the intra prediction mode of the block is not the planar mode and does not overlap with the existing intra prediction modes in the candidate list, select the intra prediction mode of the block and add it to the candidate list.

[0188] Access the upper-left block, determine whether the block is an intra-coding block, and if so, and if the intra prediction mode of the block is not the planar mode and does not overlap with the existing intra prediction modes in the candidate list, select the intra prediction mode of the block and add it to the candidate list.

[0189] Access the lower-left block, determine whether the block is an intra-coding block, and if so, and if the intra prediction mode of the block is not the planar mode and does not overlap with the existing intra prediction modes in the candidate list, select the intra prediction mode of the block and add it to the candidate list.

[0190] Access the upper-right block, determine whether the block is an intra-coding block, and if so, and if the intra prediction mode of the block is not the planar mode and does not overlap with the existing intra prediction modes in the candidate list, select the intra prediction mode of the block and add it to the candidate list.

[0191] Access the upper-left block, determine whether the block is an inter-coded block, and if so, and if the intra prediction mode of the block is not the planar mode and does not overlap with the existing intra prediction modes in the candidate list, select the intra prediction mode of the block and add it to the candidate list.

[0192] Access the lower-left block, determine whether the block is an inter-coded block, and if so, and if the intra prediction mode of the block is not the planar mode and does not overlap with the existing intra prediction modes in the candidate list, select the intra prediction mode of the block and add it to the candidate list.

[0193] Access the upper-right block, determine whether the block is an inter-coded block, and if so, and if the intra prediction mode of the block is not the planar mode and does not overlap with the existing intra prediction modes in the candidate list, select the intra prediction mode of the block and add it to the candidate list.

[0194] The current block in the embodiments of the present application (including other embodiments) may be a block of an inter-coded image or a block of an intra-coded image. If the current block is a block of an inter-coded image and after analysis, it is an intra-coded block (a block predicted using the intra prediction mode), the predicted block at the neighboring position of this intra-coded block may be an inter-coded block, and the intra prediction mode of this inter-coded block may be transmitted from the reference block of this inter-coded block.

[0195] In this embodiment, even for the current block at the upper boundary of the CTU, access is made to the prediction blocks at five positions adjacent to the current block in the above order, the intra prediction modes thereof are obtained and selection is performed, and access to the prediction blocks at the upper left, upper, and upper right is not skipped.

[0196] Based on the intra prediction modes used for the prediction blocks at the five adjacent positions of the current block, non-overlapping intra prediction modes are sequentially selected and entered into the candidate prediction mode list, and then an expansion operation for the angular mode may be directly performed, or one or two of the following selection operations may be first performed. Type 1: Based on the reconstruction values around the current block, two intra prediction modes are derived using the gradient histogram of DIMD, and a mode that does not overlap with the existing modes in the list is selected from the two derived intra prediction modes and entered into the candidate list. Type 2: If the DC mode is not present among the already selected intra prediction modes in the candidate list, the DC mode is selected and entered into the candidate list.

[0197] Next, an expansion operation for the angular mode is performed on the modes entered in the list. Specifically, in order, for all the selected angular modes, an operation of adding 1 and an operation of subtracting 1 are performed, non-overlapping expanded angular modes are selected and sequentially added to the candidate prediction mode list, and when the number of modes entered in the candidate list reaches 6, the filling may be stopped.

[0198] The specific contents of the operation of adding 1 and the operation of subtracting 1 are shown in the following table.

[0199]

Table e

[0200] The above "angle mode -1" refers to the angle mode obtained by subtracting 1 from the index of the entered angle mode. For example, when the entered angle mode is mode 3, "angle mode -1" becomes angle mode 2. The above "angle mode +1" refers to the angle mode obtained by adding 1 to the index of the entered angle mode. For example, when the entered angle mode is mode 3, "angle mode +1" becomes angle mode 4.

[0201] And when "angle mode -1" is smaller than angle mode 2, for example, when obtaining "angle mode -1" which is "angle mode 1" (the index of the angle mode is numbered from 2 and "angle mode 1" does not exist), select the angle mode in the opposite direction of "angle mode -1". Assuming there are a total of 65 types of angle modes, at this time, the angle mode in the opposite direction of "angle mode -1" is namely angle mode 66. And when "angle mode +1" is larger than angle mode 66, similarly, select the angle mode in the opposite direction of "angle mode +1". For example, when the entered angle mode is angle mode 66 and "angle mode +1" does not exist, the angle mode in the opposite direction of the selected "angle mode +1" at this time is namely angle mode 2.

[0202] The above extension of the angle mode is to perform an operation of adding 1 and an operation of subtracting 1 to the angle mode. In other embodiments, it is also possible to extend the operation of adding 1 and the operation of subtracting 1 to the operation of adding X and the operation of subtracting X from the operation of adding 1 and the operation of subtracting 1. Assuming X = 3, until the candidate prediction mode list is filled, it is possible to perform an operation of adding 1 and an operation of subtracting 1, an operation of adding 2 and an operation of subtracting 2, and an operation of adding 3 and an operation of subtracting 3 to the angle mode. Specifically, it is shown in the following table. In the table, delta = 1, 2,... X - 1.

[0203]

Table f

[0204] In another embodiment of the present application, the extension operations for the angle mode include an operation of adding 1 and an operation of subtracting 1, an operation of adding 2 and an operation of subtracting 2, an operation of adding 3 and an operation of subtracting 3, and an operation of adding 4 and an operation of subtracting 4. Assuming that there are a total of 4 selected angle modes in the first step, an exemplary setting order may be as follows: perform an operation of adding 1 and an operation of subtracting 1 on the selected first angle mode (abbreviated as the first angle mode), perform an operation of adding 1 and an operation of subtracting 1 on the selected second angle mode (abbreviated as the second angle mode), perform an operation of adding 1 and an operation of subtracting 1 on the selected third angle mode (abbreviated as the third angle mode), perform an operation of adding 1 and an operation of subtracting 1 on the selected fourth angle mode (referred to as the fourth angle mode), perform an operation of adding 2 and an operation of subtracting 2 on the first angle mode, perform an operation of adding 2 and an operation of subtracting 2 on the second angle mode, perform an operation of adding 2 and an operation of subtracting 2 on the third angle mode, perform an operation of adding 2 and an operation of subtracting 2 on the fourth angle mode, then continue to perform an operation of adding 3 and an operation of subtracting 3 on the first angle mode to the fourth angle mode, and further perform an operation of adding 4 and an operation of subtracting 4 on the first angle mode to the fourth angle mode. In the angle extension of this embodiment, every time the extension operation of one angle mode is completed, the two obtained extended angle modes are compared with the selected angle mode, an angle mode different from any of the selected angle modes is selected, and the total number of the selected intra prediction modes is updated. When the total number reaches M, the construction of the candidate list of the intra prediction modes is terminated, that is, the process of selecting M intra prediction modes is terminated.

[0205] After performing an extended operation of the angle mode on the modes entered in the list, if the candidate prediction mode list is not yet filled, fill the list with non-duplicate modes from the pre-defined mode set until the candidate prediction mode list is filled. The mode set includes several angle modes selected according to the following statistical laws.

[0206] mpm_default[] = { DC_IDX, VER_IDX, HOR_IDX, VER_IDX - 4, VER_IDX + 4, 14, 22, 42, 58, 10, 26, 38, 62, 6, 30, 34, 66, 2, 48, 52, 16};

[0207] Here, DC_IDX represents the DC mode, VER_IDX represents the vertical mode, HOR_IDX represents the horizontal mode, and the remaining numbers represent the angle modes corresponding to those numbers.

[0208] In this embodiment, the length of the candidate prediction mode list is set to 6. However, from the perspective of performance, it is also possible to set the length to a value greater than 6 in order to try more angle modes. For example, it can be set to 8, 10, 12, 14, etc. Also, in order to reduce complexity, fewer modes can be tried and the length can be set to a value less than 6.

[0209] In this embodiment, when determining the candidate intra prediction mode, the Planar mode and the DC mode are excluded, or only the Planar mode is excluded. However, if complexity is not considered, these two modes do not have to be excluded. That is, the Planar mode, the DC mode, and all angle modes may participate in the combination with the extended reference line as candidate intra prediction modes.

[0210] Compared with the case of directly using the previous N intra prediction modes in the MPM list and the second MPM list, in this embodiment, the candidate list of intra prediction modes constructed by the above-described steps cancels the access restriction on the upper prediction block at the upper boundary of the CTU, and enhances the diversity of the candidate list constructed for the current block located at the upper boundary of the CTU. Furthermore, by increasing the DIMD mode and the DC mode, the diversity of the intra prediction modes in the candidate list can be further enhanced.

[0211] ■Construct the candidate list of the TMRL_IP mode

[0212] After determining the candidate extended reference line and the intra prediction mode, all combinations in the extended reference line list and the candidate prediction mode list can be tried one by one. Based on each of these combinations, referring to FIG. 8A, predict the template area of the line where reference line0 is located (Template, shown in the next figure), calculate the error between the reconstructed value of the template area and the predicted value obtained from the prediction based on each combination, and record the K combinations with the smallest errors in the candidate list of the TMRL_IP mode in ascending order of the errors.

[0213] In the prediction process of this embodiment, the use of the TMRL_IP mode is restricted only when the current block occupies the first row of the CTU. When the current block is located at the left boundary of the image, the TMRL_IP mode can still be used. In this case, since the left reference line0 is already outside the image boundary, the left template is not used for prediction, that is, only the upper template area is predicted.

[0214] The prediction process for the template area can be exactly the same as other normal intra - angle prediction processes. That is, first, the reconstructed value of the reference line is filtered, and the result is used as the initial prediction value of the template area. After predicting the template area based on the intra - prediction mode in combination with the reconstructed value of the filtered reference line, 4 - tap or 6 - tap filtering is performed on the initial prediction result, and the result is used as the predicted value obtained by prediction. Note that considering the complexity of the operation, the step of filtering the reconstructed value of the reference line may be omitted, or a filter with a relatively short number of taps may be used. In this embodiment, when obtaining the predicted value of the template area by prediction, instead of filtering the reconstructed value of the pixels of the reference line, 4 - tap interpolation filtering with 1 / 32 precision at a non - integer angle is performed on the initial prediction result, and the 4 - tap filter is as follows.

[0215]

Table g

[0216] An appropriate filter is selected based on the position deviation between the lines and the samples of the lines at the current angle. The deviation iFact is derived as follows:

[0217] predModeIntra is obtained based on the correspondence between the traditional angle and the wide - angle, and intraPredAngle is obtained by looking up the table based on predModeIntra. The table is as follows.

[0218]

Table h

[0219] iFact is determined by intraPredAngle and the index refIdx of the selected reference line.

[0220] iFact = ( ( x + 1 + refIdx ) * intraPredAngle ) & 31

[0221] Finally, based on the angle and reference line in the current combination, as well as the filter, predict the template region. Calculate the SAD between the predicted value of the template region obtained from the prediction and the reconstructed value of the template region, sort them in ascending order of SAD, select K combinations with the minimum SAD, and enter them into the candidate list of the TMRL_IP mode.

[0222] In the sorting process, a fast algorithm can be used. In the above process, it is necessary to try a total of 30 combinations of 5 reference lines and 6 prediction modes, but only select K combinations with the minimum SAD among them. In this embodiment, after completing the prediction based on the first K combinations and obtaining the corresponding SAD, from the (K + 1)-th combination, maintain and update the K combinations with the minimum error (which may also be called cost). From the (K + 1)-th combination, only predict the upper template region and calculate the corresponding SAD. If the SAD calculated based on the upper template is already greater than the largest one among the K combinations with the minimum error, the prediction and error calculation of the left template region can be skipped. Specifically, reference can be made to the previous embodiment.

[0223] The value of K in this embodiment is the same as that on the encoding side, that is, K = 12. However, in another embodiment of the present application, K is determined according to the TMRL_IP mode index. When the TMRL_IP mode index indicates that the combination of the selected extended reference line and the intra prediction mode is in the L-th position in the candidate list (that is, the selected one is the L-th combination in the candidate list), K = L. Representing the value of the TMRL_IP mode index by L', when 0 ≦ L' < K is satisfied, L = L' + 1. When L is smaller than 12 (the value of K set on the encoding side), instead of selecting the 12 combinations with the lowest cost from among the N×M combinations, L combinations with the lowest cost may be selected. By the decoding side determining the value of K based on the TMRL_IP mode index, the complexity of decoder ordering can be reduced.

[0224] Step 3: Based on the constructed candidate list of the TMRL_IP mode and the TMRL_IP mode index obtained by decoding, determine the combination of the extended reference line and the intra prediction mode selected for the current block, and perform intra prediction on the current block based on the selected combination.

[0225] When using the TMRL_IP mode, the reference line index refIdx and the variable predModeIntra define the mode used for intra prediction, and both are determined based on the TMRL_IP mode index "tmrl_idx" and the candidate list of the TMRL_IP mode.

[0226] In the ECM-4.0 reference software, using the method described in this embodiment, with the settings of N = 5 (the five extended reference lines are 1, 3, 5, 7, 12 respectively), M = 6 (six predefined prediction modes excluding the Planar mode and the DC mode), and K = 12 (selecting only the first 12 combinations with small SAD in all combinations), the measurement results in the AI layout are as follows.

[0227]

Table i

[0228] With the settings of N = 5 (the five extended reference lines are 1, 3, 5, 7, and 12 respectively), M = 8 (eight predefined prediction modes), and K = 16, the measurement results in the AI layout are as follows.

[0229]

Table j

[0230] With the settings of N = 5 (the five extended reference lines are 1, 3, 5, 7, and 12 respectively), M = 12 (twelve predefined prediction modes), and K = 24, the measurement results in the AI layout are as follows.

[0231]

Table k

[0232] With the settings of N = 5 (the five extended reference lines are 1, 3, 5, 7, and 12 respectively), M = 8 (eight predefined prediction modes that do not include the Planar mode but include the DC mode), and K = 16, the measurement results in the AI layout are as follows.

[0233]

Table l

[0234] The meanings of the parameters in the table are as follows.

[0235] EncT: Encoding Time, that is, the encoding time. 10X% means that when the reference line sorting technology is integrated, the encoding time is 10X% compared to before integration, which means the encoding time has increased by X%.

[0236] DecT: Decoding Time, which is the decoding time. 10X% means that when the reference line sorting technology is integrated, the decoding time is 10X% compared to before integration, indicating that the decoding time has increased by X%.

[0237] Class A1 and Class A2 are test video sequences with a resolution of 3840×2160, Class B is a test sequence with a resolution of 1920×1080, Class C is a test sequence with a resolution of 832×480, Class D is a test sequence with a resolution of 416×240, Class E is a test sequence with a resolution of 1280×720, and Class F is a sequence of several screen contents with different resolutions.

[0238] JPEG2025521793000023.jpg19155

[0239] All intra indicates a test configuration where all are intra frames.

[0240] With the settings of N = 5 (the five extended reference lines are 1, 3, 5, 7, 12 respectively), M = 10 (10 predefined prediction modes), and K = 20, and when constructing the candidate list, combinations are made directly using the previous M prediction modes in the MPM list and the second MPM list. The test results in the All intra and Random access configurations are shown in Table A below.

[0241]

Table A

[0242] On one hand, with the same settings of N = 5, M = 10, and K = 20, a candidate list is constructed using the method of this embodiment (to select the intra prediction mode for the current block at the upper boundary of the CTU, access to neighboring prediction blocks at the upper, upper-right, and upper-left positions is permitted, and before angle expansion, a selection is made between the mode derived based on DIMD and the DC mode, and an expansion operation is performed for all selected angle modes, etc., including such processes), and the test results in the AI configuration are shown in Table B below.

[0243]

Table B

[0244] As can be seen by comparison, in the "overall" row, by the method of directly using and combining the previous M prediction modes in the MPM list and the second MPM list, compared with the reference software, in the all intra configuration, BD-rates of 0.08% / 0.01% / 0.04% for Y / U / V can be saved, and in the random access configuration, BD-rates of 0.03% / 0.08% / 0.13% for Y / U / V can be saved. In contrast, by constructing the candidate list using the method of this embodiment, compared with the reference software, in the all intra configuration, BD-rates of 0.11% / 0.10% / 0.07% for Y / U / V can be saved, and in the random access configuration, BD-rates of 0.05% / -0.05% / 0.06% for Y / U / V can be saved. The overall effect is improved.

[0245] As can be seen from this, by performing intra prediction coding using the TMRL_IP mode of this embodiment, the coding performance can be significantly improved.

[0246] In this embodiment, a template area of one row and one column is used for both the extended reference line and the prediction model, and sorting and selection are performed in ascending order of SAD. For the extended reference line, if all the extended reference lines (including reference line1) are sorted, only a template of one row and one column can be used. However, for the selection of the prediction mode, in order to select a more appropriate prediction mode, more reference lines can be used as in the case of the TIMD mode, and thus more accurate results can be obtained. In other embodiments, the method of determining the candidate prediction mode list based on the TMRL_IP mode can also be changed.

[0247] For example, when it is necessary to construct a candidate prediction mode list with a length of 6, according to the same construction method and filling method as in this embodiment, first construct a list with a length greater than 6, and then use the 4 rows and 4 columns closest to the current block as a template. Use the 5th reference line and the intra prediction mode in the candidate prediction mode list to predict the template, calculate the error (SAD or SATD) between the predicted value obtained from the prediction and the reconstructed value of the template, sort them in ascending order of the error, and select 6 intra prediction modes with small errors and use them as the intra prediction modes in the candidate list of the TMRL_IP mode with a length of 6 to be constructed. Also, the length of the candidate list of the TMRL_IP mode being 6 is just an example, and the numerical value can be adjusted according to the situation.

[0248] This embodiment has exemplified 65 types of angle modes. However, in other embodiments, it is possible to extend the angle to 129 types or more in order to obtain better performance. When extending to more angles, it is also necessary to appropriately increase the number of filters for intra prediction. For example, in the case of 129 types of angles, filtering with 1 / 64 accuracy is used.

[0249] One embodiment of the present application further provides a bitstream. The bitstream includes block-level syntax elements used for intra prediction, and the syntax elements include a multi-reference line intra prediction (MRL_IP) mode flag and an MRL_IP mode index for the current block. The MRL_IP mode flag is used to indicate whether to use the MRL_IP mode for the current block, and the MRL_IP mode index is used to indicate the position of the combination of the extended reference line and the intra prediction mode selected for the current block in the candidate list of the MRL_IP mode. The bitstream of this embodiment can be generated by the video encoding method of any one embodiment of the present application.

[0250] In addition, one embodiment of the present application further provides an apparatus for constructing a candidate list of the multi-reference line intra prediction mode. As shown in FIG. 10, the apparatus for constructing the candidate list of the MRL_IP mode includes a processor 71 and a memory 73 storing a computer program. When the processor 71 executes the computer program, the method for constructing the candidate list of the MRL_IP mode described in any one embodiment of the present application can be realized.

[0251] One embodiment of the present application further provides a video decoding apparatus. As shown in FIG. 10, the video decoding apparatus includes a processor and a memory storing a computer program. When the processor executes the computer program, the video decoding method described in any one embodiment of the present application can be realized.

[0252] One embodiment of the present application further provides a video encoding apparatus. As shown in FIG. 10, the video encoding apparatus includes a processor and a memory storing a computer program. When the processor executes the computer program, the video encoding method described in any one embodiment of the present application can be realized.

[0253] The processor in the above embodiment of the present application can be a general-purpose processor including a central processing unit (CPU), a network processor (NP), a microprocessor, etc., or can be other ordinary processors, etc. The processor can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), discrete logic or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or can be other equivalently integrated or discrete logic circuits, or can be a combination of the above components. That is, the processor in the above embodiment can be any processing component or combination of components in the various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. When the embodiment of the present application is implemented partially by software, the instructions used in the software can be stored in a suitable non-volatile computer-readable storage medium, and the method of the embodiment of the present application can be implemented by executing the instructions in hardware by one or more processors. The term "processor" as used herein can refer to the above-described structure or any other structure suitable for implementing the technology described herein.

[0254] One embodiment of the present application further provides a video coding system. The video coding system includes a video encoding device described in any one of the embodiments of the present application and a video decoding device described in any one of the embodiments of the present application.

[0255] One embodiment of the present application further provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, can implement the method for constructing a candidate list of the multi-reference line intra prediction (MRL_IP) mode described in any one embodiment of the present application, or can implement the video decoding method described in any one embodiment of the present application, or can implement the video encoding method described in any one embodiment of the present application.

[0256] In one or more of the above exemplary embodiments, the functions described can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, the functions can be stored on a computer-readable medium as one or more instructions or codes, or can be transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium includes a computer-readable medium that is a tangible medium such as a data storage medium, or can include any communication medium that facilitates transmission of a computer program from one place to another, for example, according to a communication protocol. Thus, the computer-readable medium can typically be a non-transitory tangible computer-readable storage medium, or a communication medium such as a signal or a carrier. The data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to search for instructions, codes, and / or data structures for implementing the techniques described in the present application. A computer program product can include a computer-readable medium.

[0257] By way of non-limiting example, such a computer-readable storage medium can include random access memory (RAM), read only memory (ROM), electrically erasable programmable ROM (EEPROM), compact disk ROM (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can store the desired program code in the form of instructions or data structures and is accessible by a computer. Also, any connection can be referred to as a computer-readable storage medium. By way of example, when transmitting instructions from a website, server or other remote source using coaxial cable, fiber optic cable, twisted-pair cabling, digital subscriber line (DSL), or wireless technologies such as infrared, radio, microwave, etc., the coaxial cable, fiber optic cable, twisted-pair cabling, DSL, or wireless technologies such as infrared, radio, microwave, etc. are included in the definition of the medium. However, computer-readable storage media and data storage media do not include connections, carriers, signals, or other transient media, and are non-transitory tangible storage media. As used herein, magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, or Blu-ray disks, etc. Magnetic disks typically reproduce data magnetically, and optical disks reproduce data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0258] In some aspects, the functions described herein can be provided within dedicated hardware and / or software modules configured to be used for encoding and decoding and can also be incorporated into an integrated encoder-decoder. Also, the techniques described herein can be fully implemented in one or more circuits or logic elements.

[0259] The technical solution of the embodiments of the present application can be implemented in various devices or apparatuses including a wireless mobile phone, an integrated circuit (IC), or a set of ICs (e.g., a chipset). In the embodiments of the present application, various components, modules, or units are used to emphasize the functions of a device configured to execute the described techniques. It is not necessarily realized by different hardware units. As described above, the various units may be combined in the hardware units of the coding, or may be provided in combination with a suitable software and / or firmware in an aggregate of interoperable hardware units (including one or more of the processors described above).

Claims

1. A method for constructing a candidate list for a multi-reference line intra prediction (MRL_IP) mode, comprising: obtaining N×M combinations of extended reference lines and intra prediction modes based on N extended reference lines and M intra prediction modes of a current block, where N≧1, M≧1, and N×M≧2 are satisfied; predicting a template region of the current block based on each of the N×M combinations, and calculating an error between a reconstructed value of the template region and a predicted value obtained by the prediction; entering K combinations corresponding to the error in ascending order of the error into a candidate list for a template-based multi-reference line intra prediction (TMR_LIP) mode of the current block, where 1≦K≦N×M is satisfied; including; A method for constructing a candidate list for a multi-reference line intra prediction mode, characterized by the above.

2. The template region of the current block is set to one reference line closest to the current block, or the template region of the current block is set to a plurality of reference lines closest to the current block, where the N extended reference lines are extended reference lines located outside the template region. The construction method according to claim 1, characterized by the above.

3. The error is represented by a sum of absolute differences (SAD) or a sum of absolute transformed differences (SATD). The construction method according to claim 1, characterized by the above.

4. The N extended reference lines of the current block are N ｍａｘ of the pre-defined N extended reference lines, which are located outside the template area of the current block and do not exceed the boundary of the coding tree unit (CTU). N ｍａｘ is the maximum number of extended reference lines that can be used in the TMRL_IP mode, The construction method according to claim 1, characterized by the above.

5. the pre-defined N ｍａｘ individual extended reference lines are Five extended reference lines with indices {1, 3, 5, 7, 12}, or Five extended reference lines with indices {2, 3, 5, 7, 12}, or The index starts from 1 and is the N ｍａｘ extended reference lines closest to the current block, or ｍａｘ or The index starts from 1 and the index is odd, and the N ｍａｘ extended reference lines closest to the current block, or The index starts from 2 and the index is even, and the N ｍａｘ extended reference lines closest to the current block The construction method according to claim 4, characterized by the above.

6. The M intra prediction modes are selected by the following process, that is, in the first step, determine the intra prediction modes used for prediction blocks at a plurality of neighboring positions around the current block, select the selectable intra prediction modes in order, and delete the overlapping modes; when the number of intra prediction modes selected in the first step becomes M, end. If the number of intra prediction modes selected in the first step is less than M and includes an angular mode, execute the second step. In the second step, until the total number of selected intra prediction modes reaches M, perform an expansion operation on the angle modes in order from the first selected angle mode to obtain the expanded angle modes, and select an expanded angle mode that is different from all the selected angle modes. The construction method according to claim 1, characterized in that.

7. In the first step, for each of the current block located at the upper boundary of the CTU and the current block not located at the upper boundary of the CTU, the prediction blocks at a plurality of neighboring positions around the current block include some or all of the prediction blocks such as the prediction block on the left side of the current block, the prediction block above, the prediction block in the upper left, the prediction block in the lower left, and the prediction block in the upper right, and M≥5 is satisfied. The construction method according to claim 6, characterized in that.

8. If the number of intra prediction modes selected based on the intra prediction modes used for the prediction blocks at a plurality of neighboring positions around the current block is less than M, the first step further includes the following method, that is, Based on the reconstruction values around the current block, use the gradient histogram of decoder-side intra mode derivation (DIMD) to derive two intra prediction modes, and select an intra prediction mode that is different from any of the selected intra prediction modes from the two derived intra prediction modes; If the selected intra prediction mode does not include the DC mode, select the DC mode; Continuing to select the intra prediction modes participating in the combination by at least one of the above methods. The construction method according to claim 6 or 7, characterized in that.

9. In the second step, for all the angle modes selected in the first step, perform the following processing in order in the set order from the first selected angle mode, that is, Perform an expansion operation on the angle mode to obtain the expanded angle mode. If the expanded angle mode is different from any of the selected angle modes, select the expanded angle mode, and end when the total number of selected intra prediction modes reaches M. The number of all the angle modes selected in the first step is 1 or more and 5 or less. The construction method according to claim 6, characterized in that.

10. The process of selecting the M intra prediction modes further includes: When the angular mode is not selected in the first step, or when the total number of intra prediction modes selected in the second step is still less than M, execute a third step. In the third step, determine the selectable intra prediction modes from a pre-defined set of intra prediction modes, and sequentially select intra prediction modes different from any of the selected intra prediction modes from the determined selectable intra prediction modes until the total number of selected intra prediction modes becomes M. The construction method according to claim 6, characterized in that.

11. The selectable intra prediction modes include only the angular mode, or only the angular mode and the DC mode, or the angular mode, the DC mode, and the Planar mode. The construction method according to claim 6 or 10, characterized in that.

12. The expansion operation of the angular mode is: An operation of adding 1 to and subtracting 1 from the angular mode, An operation of adding 2 to and subtracting 2 from the angular mode, An operation of adding 3 to and subtracting 3 from the angular mode, An operation of adding 4 to and subtracting 4 from the angular mode, including any one or more of the above operations. The construction method according to claim 6, characterized in that.

13. As the M intra prediction modes, use some or all of the intra prediction modes other than the Planar mode among the most probable modes (MPMs), or As the M intra prediction modes, use some or all of the intra prediction modes other than the Planar mode among the MPM and the second MPM, or As the M intra prediction modes, use some or all of the intra prediction modes other than the Planar mode and the DC mode among the MPMs, or As the M intra prediction modes, use some or all of the intra prediction modes other than the Planar mode and the DC mode among the MPM and the second MPM, or As the M intra prediction modes, use some or all of the intra prediction modes other than the Planar mode, the DC mode, and the DIMD mode among the MPMs, or As the M intra prediction modes, some or all of the intra prediction modes other than the Planar mode, DC mode, and DIMD mode among the MPM and the second MPM are used. The construction method according to claim 1, characterized in that.

14. The M intra prediction modes are selected by the following method, that is, selecting M' intra prediction modes; predicting the template of the current block based on the reference lines located outside the template of the current block and the M' intra prediction modes, respectively, and calculating M' errors by calculating the error between the reconstructed value of the template and the predicted value obtained by prediction; selecting, as the M intra prediction modes participating in the combination, M intra prediction modes corresponding to the minimum error from the M' intra prediction modes, where M < M' is satisfied; selected by the method of The construction method according to claim 1, characterized in that.

15. N ≤ N ｍａｘ 、2 ≤ N ｍａｘ ≤ 12、2 ≤ M ≤ 18、K ≤ K ｍａｘ 、6 ≤ K ｍａｘ ≤ 36、N ｍａｘ is the maximum number of extended reference lines that can be used in the TMR_LP mode, and K ｍａｘ is the maximum number of candidate combinations that can be used in the TMR_LP mode. The construction method according to claim 1, characterized in that.

16. N = 5, M = 10, K = 20 is satisfied, or N = 5, M = 6, K = 12 is satisfied, or N = 5, M = 8, K = 16 is satisfied, or N = 5, M = 12, K = 24 is satisfied, The construction method according to claim 1, characterized in that.

17. The N, M, and K have at least two sets of values, and the first set of values is N １ , M １ , K １ , and the second set of values is N ２ , M ２ , K ２ , where N １ ≤ N ２ , M １ ≤ M ２ , K １ ≤ K ２ , and N １ × M １ < N ２ × M ２ is satisfied The first set of values is used when constructing the candidate list for the current block having the first size, the second set of values is used when constructing the candidate list for the current block having the second size, and the first size is smaller than the second size. The construction method according to claim 1, characterized in that.

18. Predicting the template area of the current block based on each of the N×M combinations described above includes predicting the upper template area of the current block based on each of the N×M combinations when the current block is located at the left boundary of the image, and not predicting the left template area of the current block. The construction method according to claim 1, characterized in that.

19. Predicting the template area of the current block based on each of the N×M combinations described above includes predicting in the following manner based on each of the N×M combinations, that is, Calculating an initial prediction value of the template region based on a reconstructed value of an extended reference line in the combination and an intra prediction mode in the combination, where the reconstructed value of the extended reference line is an original reconstructed value of the extended reference line or a reconstructed value after filtering, and calculating; Performing 4-tap filtering or 6-tap filtering on the initial prediction value of the template region, and using the filtering result as the predicted value of the template region obtained by predicting based on the combination; Including predicting in such a manner; The construction method according to claim 1, characterized in that.

20. Predicting a template region of a current block based on each of the N×M combinations described above, and calculating an error between a reconstructed value of the template region and a predicted value obtained by prediction, Predict the entire template area of the current block based on each of the K combinations to obtain an error set consisting of the corresponding K errors, and record the maximum error D in the error set ｍａｘ as such, and For each of the remaining combinations, first, predict the template region on one side of the current block based on the combination, and calculate the error D between the reconstruction value and the predicted value of the template region on the one side of the current block １ If D １ ≥ D ｍａｘ then complete the prediction based on the combination. If D １ < D ｍａｘ then also predict the template region on the other side of the current block based on the combination, and calculate the error D between the reconstruction value and the predicted value of the entire template region of the current block ２ If D ２ < D ｍａｘ then add D ２ to the error set, delete D ｍａｘ from the error set, update the maximum error D ｍａｘ in the error set. If D ２ ≥ D ｍａｘ then complete the prediction based on the combination After completing the prediction based on the N×M combinations, setting K combinations corresponding to K errors in the error set as K combinations corresponding to the minimum error; Including; The construction method according to claim 1, characterized in that.

21. Entering the K combinations corresponding to the errors in ascending order of the errors into a candidate list of the TMRL_IP mode of the current block described above, Entering the K combinations corresponding to the errors in ascending order of the errors into the candidate list starting from the first position of the candidate list, or Entering the K combinations corresponding to the errors in ascending order of the errors into the candidate list starting from the i-th position of the candidate list, where a combination of a reference line with an index of 0 and one or more intra prediction modes is entered before the i-th position of the candidate list, and i≧2 is satisfied, and entering; Including; The construction method according to claim 1, characterized in that.

22. Decoding a multi-reference line intra prediction (MRL_IP) mode flag of a current block to determine whether to use the MRL_IP mode for the current block, When it is determined to use the MRL_IP mode for the current block, subsequently decode the MRL_IP mode index of the current block and construct a candidate list for the MRL_IP mode of the current block, wherein the candidate list is filled with candidate combinations of extended reference lines and intra prediction modes for the current block, and construct; Based on the candidate list and the MRL_IP mode index, determine a combination of an extended reference line and an intra prediction mode selected for the current block, and predict the current image based on the selected combination; A video decoding method comprising: The MRL_IP mode index is used to indicate the position of the selected combination of the extended reference line and the intra prediction mode in the candidate list. A video decoding method characterized by the above.

23. The candidate list of the MRL_IP mode is a candidate list of a template-based multi-reference line intra prediction (TMR_LIP) mode constructed based on the method according to any one of claims 1 to 21, K is a set value, or K is determined based on the TMR_LIP mode index, and when the TMR_LIP mode index indicates that the selected combination of the extended reference line and the intra prediction mode is at the L-th position in the candidate list, K = L, and L is a positive integer. The video decoding method according to claim 18, characterized by the above.

24. Before decoding the MRL_IP mode flag of the current block as described above, the video decoding method further includes decoding the MRL_IP mode flag of the current block when all conditions for using the MRL_IP mode for the current block are satisfied. The conditions for using the MRL_IP mode include any one or more of the following, that is, The current block is a block in the luminance frame; It is possible to use MRL for the current block; The current block is not located at the upper boundary of the coding tree unit (CTU); Do not use template-based intra mode derivation (TIMD) for the current block; Including any one or more of the above conditions. The video decoding method according to claim 22 or 23, characterized in that

25. wherein the method can use MRL for the current block, and when it is determined that the current block is not located at the upper boundary of the CTU and TIMD is used for the current block, further includes decoding the MRL index of the current block, wherein the MRL index is used to indicate a position in the MRL index list of the reference line selected for the current block, The video decoding method according to claim 22, characterized in that

26. Based on the MRL_IP mode flag, when it is determined to use the MRL_IP mode for the current block, the method further includes skipping the decoding of syntax elements of any one or more of the MPM mode, intra sub-partition split (ISP) mode, multi-conversion selection (MTS) mode, low-frequency non-separable transform (LFNST) mode, and TIMD mode, The video decoding method according to claim 22, characterized in that

27. Constructing a candidate list for the multi-reference line intra prediction (MRL_IP) mode of the current block, wherein the candidate list is filled with a combination of an extended reference line and an intra prediction mode candidate for the current block, Selecting, through rate distortion optimization, a combination of a reference line and an intra prediction mode used for intra prediction for the current block, When the encoding condition of the MRL_IP mode of the current block is satisfied, encoding the MRL_IP mode flag of the current block to indicate using the MRL_IP mode for the current block, and encoding the MRL_IP mode index of the current block to indicate the position of the selected combination in the candidate list, A video encoding method including wherein the encoding condition includes at least that the selected combination is in the candidate list, The video encoding method, characterized in that

28. The candidate list of the MRL_IP mode is a candidate list of the template-based multi-reference line intra prediction (TMRL_IP) mode constructed based on the method according to any one of claims 1 to 21, and K is a set value. The video encoding method according to claim 27, characterized in that.

29. The encoding condition further includes not using template-based intra mode derivation (TIMD) for the current block. The method further includes When using TIMD for the current block, skipping the encoding of the MRL_IP mode flag and MRL_IP mode index of the current block; When not using TIMD for the current block but the selected combination is not in the candidate list, encoding the MRL_IP mode flag of the current block to indicate not using the MRL_IP mode for the current block, and skipping the encoding of the MRL_IP mode index of the current block; including The video encoding method according to claim 27, characterized in that.

30. Constructing the candidate list of the MRL_IP mode of the current block as described above includes constructing the candidate list only when all the set conditions for using the MRL_IP mode for the current block are satisfied. The conditions for using the MRL_IP mode include any one or more of the following, that is, The current block is a block in the luminance frame; The current block is not located at the upper boundary of the coding tree unit (CTU); MRL can be used for the current block; The size of the current block is not larger than the maximum size of the current block for which the MRL_IP mode can be used; The aspect ratio of the current block satisfies the requirement for using the MRL_IP mode for the aspect ratio of the current block; including any one or more of the above conditions. The video encoding method according to claim 27, characterized in that.

31. Encoding the MRL_IP mode index using the Golomb-Rice coding method. The video encoding method according to claim 27, characterized in that.

32. When the encoding conditions of the MRL_IP mode of the current block are satisfied, the method further includes skipping the encoding of syntax elements in any one or more of the MPM mode, the intra-subpartition splitting (ISP) mode, the multi-conversion selection (MTS) mode, the low-frequency non-separable conversion (LFNSST) mode, and the TIMD mode. The video encoding method according to claim 27, characterized in that.

33. A bitstream, The bitstream includes block-level syntax elements used for intra prediction, the syntax elements include a multi-reference line intra prediction (MRL_IP) mode flag and an MRL_IP mode index for the current block, the MRL_IP mode flag is used to indicate whether to use the MRL_IP mode for the current block, and the MRL_IP mode index is used to indicate the position of the combination of the extended reference line and the intra prediction mode selected for the current block in the candidate list of the MRL_IP mode. A bitstream characterized by that.

34. A bitstream, The bitstream is generated by the video encoding method according to any one of claims 27 to 32. The bitstream according to claim 33, characterized in that.

35. An apparatus for constructing a candidate list of the multi-reference line intra prediction (MRL_IP) mode, Comprising a processor and a memory storing a computer program, When the processor executes the computer program, the method for constructing a candidate list of the MRL_IP mode according to any one of claims 1 to 21 can be realized. An apparatus for constructing a candidate list of the multi-reference line intra prediction mode, characterized in that.

36. A video decoding apparatus, Comprising a processor and a memory storing a computer program, When the processor executes the computer program, the video decoding method according to any one of claims 22 to 26 can be realized. A video decoding apparatus, characterized in that.

37. A video encoding apparatus, A processor and a memory storing a computer program, When the processor executes the computer program, the video encoding method according to any one of claims 27 to 32 can be realized. A video encoding apparatus characterized by the above.

38. Including the video encoding apparatus according to claim 37 and the video decoding apparatus according to claim 36. A video coding system characterized by the above.

39. A non-transitory computer-readable storage medium, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for constructing a candidate list of the multi-reference line intra prediction (MRL_IP) mode according to any one of claims 1 to 21 can be realized, or the video decoding method according to any one of claims 22 to 26 can be realized, or the video encoding method according to any one of claims 27 to 32 can be realized. A non-transitory computer-readable storage medium characterized by the above.

Images