Intra Prediction Fusion Method, Video Encoding Method and Apparatus, Video Decoding Method and Apparatus, and Video Coding System

Abstract

An intra prediction fusion method, a video encoding method and apparatus, a video decoding method and apparatus, and a video coding system are provided. When the intra prediction mode selected for the current block includes an angular mode, this method includes determining whether a constraint condition for using IPF for the current block is satisfied, and when at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block. By restricting the use of IPF, it is possible to avoid fusing too many modes during prediction and improve coding performance. In the present application, a video encoding method and apparatus, a video decoding method and apparatus, and a video coding system using this intra prediction fusion method are further provided.

Description

Technical Field

[0001] Reference to Related Applications This application claims the priority of Chinese Patent Application No. 202210806435.X, filed on July 8, 2022, with the invention title of "Method for Constructing Candidate List, Video Encoding Method and Apparatus, Video Decoding Method and Apparatus, and Video Encoding System", and all of its contents are incorporated herein by reference in their entirety.

[0002] Embodiments of the present application relate to, but are not limited to, video technology. More specifically, they relate to an intra prediction fusion method, a video encoding method and apparatus, a video decoding method and apparatus, and a video encoding system.

Background Art

[0003] Digital video compression technology is mainly used to compress huge digital video data to facilitate its transmission, storage, etc. Currently, in common video coding standards such as H.266 / Versatile Video Coding (VVC), a block-based hybrid coding framework is used. Each image (frame) in a video is divided into square largest coding units (LCUs) of the same size (e.g., 128×128, 64×64, etc.). Each largest coding unit can also be divided into rectangular coding units (CUs) based on rules. The coding unit may be further divided into a prediction unit (PU), a transform unit (TU), etc. The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filter. The prediction module includes intra prediction and inter prediction to reduce or remove redundancy within the video. For an intra block, samples around the block are used as references for prediction, and for an inter block, information of spatially adjacent blocks or reference information in another image is referred to. Compared with the predicted signal, the residual information is formed into a bitstream by block-based transformation, quantization, and entropy coding. These technologies are described in the standards and are implemented in various fields related to video compression.

[0004] With the rapid increase in Internet video and the growing demand for video resolution, despite the fact that a large amount of video data can be saved by existing digital video compression standards, there is still a need for better digital video compression technology to reduce the bandwidth and traffic burden of digital video transmission.

Summary of the Invention

[0005] The following content is an overview of the subject matter to be described in detail in this specification. This overview is not intended to limit the scope of protection of the claims.

[0006] One embodiment of the present application provides an intra prediction fusion method. When the intra prediction mode selected for the current block includes an angular mode, the intra prediction fusion method includes determining whether the constraint conditions for using IPF for the current block are satisfied, and when at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block.

[0007] One embodiment of the present application further provides a video decoding method. The video decoding method includes decoding a bitstream to determine a reference line and an intra prediction mode selected for the current block, predicting the current block based on the intra prediction fusion method described in any one of the embodiments of the present application to obtain a predicted value of the current block, and determining a reconstructed value of the current block based on the predicted value of the current block.

[0008] One embodiment of the present application further provides a video encoding method. The video encoding method includes determining a reference line and an intra prediction mode selected for the current block in mode selection, predicting the current block based on the intra prediction fusion method described in any one of the embodiments of the present application to obtain a predicted value of the current block, and determining a residual of the current block based on the original value and the predicted value of the current block.

[0009] One embodiment of the present application is a video Decoding method and further provides. The video Decoding method is If it is determined by decoding that the template-based multi-reference line intra prediction (TMRL) mode is used for the current block, continue to decode the TMRL mode index and TMRL fusion flag of the current block, and construct a candidate list for the TMRL mode for the current block, and determine the extended reference line and intra prediction mode selected for the current block based on the candidate list and the TMRL mode index, and if the TMRL fusion flag indicates that intra prediction fusion (IPF) is used, include taking the weighted sum of the first prediction result and the second prediction result as the final prediction result of the current block, where the first prediction result is obtained by predicting the current block based on the extended reference line and the intra prediction mode, and the second prediction result is obtained by predicting the current block based on other reference lines and the intra prediction mode, and this intra prediction mode is the angular mode.

[0010] One embodiment of the present application further provides a video encoding method. The video encoding method includes constructing a candidate list for the template-based multi-reference line intra prediction (TMRL) mode of the current block, where the candidate list is filled with candidate combinations of the extended reference line and the intra prediction mode for the current block, selecting one combination of the reference line and the intra prediction mode for the current block through rate-distortion optimization, if the encoding conditions of the TMRL mode of the current block are satisfied, encoding the TMRL mode flag of the current block to indicate that the TMRL mode is used for the current block, and encoding the TMRL mode index of the current block to indicate the position of the selected combination in the candidate list, encoding the TMRL fusion flag of the current block to indicate whether intra prediction fusion (IPF) is used for the current block. The encoding conditions include at least that the selected combination is in the candidate list.

[0011] One embodiment of the present application further provides a method for constructing a candidate list for the multi-reference line (MRL) intra prediction mode. The method for constructing a candidate list for the MRL intra prediction mode includes: Based on N extended reference lines and M intra prediction modes of the current block, obtaining N×M original combinations of the extended reference lines and the angular modes; Based on the original combinations including a pre-determined angular mode among the N×M original combinations, obtaining corresponding fused combinations; Predicting a template region of the current block based on each of the N×M original combinations and the obtained fused combinations, and calculating an error between the reconstructed value of the template region and the predicted value obtained by prediction; Entering K combinations corresponding to the errors in ascending order of the errors into the candidate list of the template-based multi-reference line intra prediction (TMRL) mode of the current block, where K, N, and M are set positive integers and 1≦K≦N×M is satisfied.

[0012] One embodiment of the present application further provides a method for constructing a candidate list for the multi-reference line (MRL) intra prediction mode. The method for constructing a candidate list for the MRL intra prediction mode includes: Based on N extended reference lines and M intra prediction modes of the current block, obtaining N×M original combinations of the extended reference lines and the intra prediction modes; Predicting a template region of the current block based on each of the N×M original combinations, and calculating an error between the reconstructed value of the template region and the predicted value obtained by prediction; For each of the original combinations including the predetermined angular mode in the K original combinations with the smallest errors, determine whether fusion is necessary. If fusion is necessary, enter the fusion combination corresponding to the original combination into the candidate list of the template-based multi-reference line intra prediction (TMRL) mode of the current block. If fusion is not necessary, enter the original combination into the candidate list, where K, N, and M are set positive integers and 1 ≦ K ≦ N × M is satisfied, and the entering is included.

[0013] One embodiment of the present application further provides a method for constructing a candidate list of the multi-reference line (MRL) intra prediction mode. The method for constructing the candidate list of the MRL intra prediction mode includes Based on N extended reference lines and M intra prediction modes of the current block, obtaining N × M original combinations of the extended reference lines and the intra prediction modes, Performing a fusion process on the N × M original combinations, where the fusion process includes replacing the original combination with the corresponding fusion combination when the set conditions are satisfied for each of the original combinations including the predetermined angular mode, and performing, Predicting the template area of the current block based on each of the N × M combinations after performing the fusion process, and calculating the error between the reconstructed value of the template area and the predicted value obtained by the prediction, Entering K combinations corresponding to the errors in ascending order of the errors into the candidate list of the template-based multi-reference line intra prediction (TMRL) mode of the current block, and including. K, N, and M are set positive integers, 1 ≦ K ≦ N × M is satisfied, When performing a prediction on the current block based on the fused combination corresponding to the original combination, the weighted sum of the first prediction result and the second prediction result is used as the predicted value of the current block. The first prediction result is the prediction result obtained by predicting the current block based on the extended reference line and the angle mode in the original combination. The second prediction result is the prediction result obtained by predicting the current block based on the angle mode and the second reference line in the original combination. The second reference line is an adjacent line of the first reference line or a reference line with an index of 0.

[0014] One embodiment of the present application further provides a bitstream. The bitstream is generated by the video encoding method described in any one of the embodiments of the present application.

[0015] One embodiment of the present application further provides an intra prediction fusion device. The intra prediction fusion device includes a processor and a memory storing a computer program. When the processor executes the computer program, the intra prediction fusion method described in any one of the embodiments of the present application can be realized.

[0016] One embodiment of the present application further provides a device for constructing a candidate list of the multi-reference line intra prediction mode. The device for constructing a candidate list of the multi-reference line intra prediction mode includes a processor and a memory storing a computer program. When the processor executes the computer program, the method for constructing a candidate list of the multi-reference line intra prediction mode described in any one of the embodiments of the present application can be realized.

[0017] One embodiment of the present application further provides a video decoding device. The video decoding device includes a processor and a memory storing a computer program. When the processor executes the computer program, the video decoding method described in any one of the embodiments of the present application can be realized.

[0018] One embodiment of the present application further provides a video encoding device. The video encoding device includes a processor and a memory storing a computer program. When the processor executes the computer program, the video encoding method according to any one of the embodiments of the present application can be realized.

[0019] One embodiment of the present application further provides a video coding system. The video coding system includes the video encoding device according to any one of the embodiments of the present application and the video decoding device according to any one of the embodiments of the present application.

[0020] One embodiment of the present application further provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the intra prediction fusion method according to any one of the embodiments of the present application can be realized, or the method for constructing a candidate list of the multi-reference line intra prediction mode according to any one of the embodiments of the present application can be realized, or the video decoding method according to any one of the embodiments of the present application can be realized, or the video encoding method according to any one of the embodiments of the present application can be realized.

[0021] After reading and understanding the drawings and the detailed description, other aspects can be understood.

Brief Description of the Drawings

[0022] The drawings are provided to understand the embodiments of the present application, form a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation of the technical solutions of the present application.

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Modes for Carrying Out the Invention

[0023] In the present application, a plurality of embodiments are described, but the descriptions are illustrative and not limiting. Also, it is obvious to those skilled in the art that more examples and embodiments can be obtained within the scope included in the embodiments described in the present application.

[0024] In the description of the present application, terms such as "exemplary" or "for example" mean "by way of example, illustration, explanation". In the present application, no embodiment described "exemplarily" or "for example" should be construed as being superior to other embodiments. The term "and / or" in this specification is used to explain the relationship of related objects and indicates that there are three types of relationships. For example, in the case of A and / or B, it indicates three situations: only A exists, A and B exist simultaneously, and only B exists. "Plurality" means two or more. Also, in order to clearly explain the technical solutions of the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same or similar ones with substantially the same functions and roles. Those skilled in the art can understand the following. Terms such as "first" and "second" do not limit the number or the execution order, and terms such as "first" and "second" do not necessarily limit that they must be different.

[0025] In the description of representative exemplary embodiments, this specification may present a method and / or process as a particular sequence of steps. However, the method or process does not depend on the particular sequence of steps described herein, and the method or process should not be limited to the particular sequence of steps described. As will be understood by those skilled in the art, other orders of steps are possible. Accordingly, the particular order of steps described in the specification should not be construed as a limitation of the claims. Furthermore, claims corresponding to the method and / or process should not be limited to performing the steps in the order described. Those skilled in the art can easily understand that these orders may vary and still fall within the spirit and scope of the embodiments of the present application even if they vary.

[0026] According to the embodiments of the present application Intra prediction fusion (IPF) The method, the video coding method, can be applied to various video coding standards. Examples include H.264 / Advanced Video Coding (AVC), H.265 / high efficiency video coding (HEVC), H.266 / Versatile Video Coding (VVC), audio video Coding standard (AVS), Moving Picture Experts Group (MPEG), standards created by the Alliance for Open Media (AOM), the joint video Experts team (JVET), and extensions of these standards, or any other customized standards, etc.

[0027] FIG. 1A is a block diagram showing a video coding system that can be used in an embodiment of the present application. As shown in the figure, the system is divided into an encoding-side device 1 and a decoding-side device 2. The encoding-side device 1 generates a bitstream. The decoding-side device 2 can decode the bitstream. The decoding-side device 2 can receive the bitstream from the encoding-side device 1 via a link 3. The link 3 includes one or more media or devices that can move the bitstream from the encoding-side device 1 to the decoding-side device 2. In one example, the link 3 includes one or more communication media that enable the encoding-side device 1 to directly transmit the bitstream to the decoding-side device 2. The encoding-side device 1 can modulate the bitstream according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated bitstream to the decoding-side device 2. The one or more communication media can include a wireless communication media and / or a wired communication media and can form part of a packet network. In another example, the bitstream can also be output from an output interface 15 to a storage device. The decoding-side device 2 can read the data stored in the storage device via streaming or downloading.

[0028] As shown in FIG. 1A, the encoding-side device 1 includes a data source 11, a video encoding device 13, and an output interface 15. The data source 11 includes a video capture device (e.g., a camera), an archive containing previously captured data, a feed interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. The video encoding device 13 can encode the data from the data source 11 and output it to the output interface 15. The output interface 15 can include at least one of a regulator, a modem, and a transmitter. The decoding-side device 2 includes an input interface 21, a video decoding device 23, and a display device 25. The input interface 21 includes at least one of a receiver and a modem. The input interface 21 can receive a bitstream via link 3 or from a storage device. The video decoding device 23 decodes the received bitstream. The display device 25 is used to display the decoded data. The display device 25 may be integrated with other components of the decoding-side device 2 or provided separately. The decoding side may not include the display device 25. In other examples, the decoding side may include other devices or apparatuses to which the decoded data is applicable.

[0029] Based on the video coding system shown in FIG. 1A, various video coding methods can be used to achieve video compression and decompression.

[0030] Figure 1B is a block diagram of an exemplary video encoding device that can be used in an embodiment of the present application. As shown in the figure, the video encoding device 1000 includes a prediction unit 1100, a splitting unit 1101, a residual generation unit 1102 (indicated by a circled plus following the splitting unit 1101 in the figure), a conversion processing unit 1104, a quantization unit 1106, an inverse quantization unit 1108, an inverse conversion processing unit 1110, a reconstruction unit 1112 (indicated by a circled plus following the inverse conversion processing unit 1110 in the figure), a filter unit 1113, a buffer 1114 for decoded images, and an entropy encoding unit 1115. The prediction unit 1100 includes an inter-prediction unit 1121 and an intra-prediction unit 1126. The buffer 1114 for decoded images may be referred to as a buffer for decoded images, a buffer for decoding images, a buffer for decoded images, etc. Video Encoding device 1000 may also include more, fewer, or different functional components compared to this example. For example, in some cases, the conversion processing unit 1104, the inverse conversion processing unit 1110, etc. may not be included.

[0031] The splitting unit 1101, in cooperation with the prediction unit 1100, splits the received video data into slices, coding tree units (CTUs), or other relatively large units. The video data received by the splitting unit 1101 may be a video sequence including video frames such as I-frames, P-frames, and B-frames.

[0032] The prediction unit 1100 can split a CTU into coding units (CUs) and perform intra-prediction coding or inter-prediction coding on the CUs. When performing intra-prediction and inter-prediction on a CU, the CU can be split into one or more prediction units (PUs).

[0033] The inter prediction unit 1121 can perform inter prediction on a PU to generate prediction data for the PU. The prediction data includes the predicted block of the PU, the motion information of the PU, and various syntax elements. The inter prediction unit 1121 can include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit can be used for motion estimation to generate a motion vector, and the motion compensation unit can be used to obtain or generate a predicted block based on the motion vector.

[0034] The intra prediction unit 1126 can perform intra prediction on a PU to generate prediction data for the PU. The prediction data of the PU can include the predicted block of the PU and various syntax elements.

[0035] The residual generation unit 1102 can subtract the predicted block of the PU obtained by splitting the CU from the original block of the CU to generate a residual block of the CU.

[0036] The transform processing unit 1104 can split a CU into one or more transform units (TUs). The splitting of the prediction unit and the splitting of the transform unit may be different. The residual block related to a TU is a sub-block obtained by splitting the residual block of the CU. By applying one or more transforms to the residual block related to the TU, a coefficient block related to the TU is generated.

[0037] The quantization unit 1106 can quantize the coefficients in the coefficient block based on the selected quantization parameter (QP). The degree of quantization of the coefficient block can be adjusted by adjusting the QP.

[0038] The inverse quantization unit 1108 and the inverse transform processing unit 1110 can obtain a reconstructed residual block related to the TU by applying inverse quantization and inverse transform to the coefficient block, respectively.

[0039] The reconstruction unit 1112 can generate a reconstructed image by adding the reconstructed residual block and the prediction block generated by the prediction unit 1100.

[0040] The filter unit 1113 performs in-loop filtering on the reconstructed image and stores the filtered reconstructed image in the buffer 1114 of the decoded image as a reference image. The intra prediction unit 1126 can extract a reference image of a block adjacent to the PU from the buffer 1114 of the decoded image and perform intra prediction. The inter prediction unit 1121 can perform inter prediction on the PU of the current image by using the reference image of the previous frame cached in the buffer 1114 of the decoded image.

[0041] The entropy encoding unit 1115 performs an entropy encoding operation on the received data (e.g., syntax elements, quantized coefficient blocks, motion information, etc.).

[0042] FIG. 1C is a block diagram of an exemplary video decoding apparatus that can be used in an embodiment of the present application. As shown in the figure, the video decoding apparatus 101 includes an entropy decoding unit 150, a prediction unit 152, an inverse quantization unit 154, an inverse transform processing unit 155 and a reconstruction unit 158 (shown as a circle with a + following the inverse transform processing unit 155 in the figure), a filter unit 159, and a buffer 160 of the decoded image. In other embodiments, the video Decoding device 101 may include more, fewer, or different functional components. For example, in some cases, it may not include the inverse transform processing unit 155, etc.

[0043] The entropy decoding unit 150 can perform entropy decoding on the received bit stream to extract information such as syntax elements, quantized coefficient blocks, and motion information of PUs. The prediction unit 152, inverse quantization unit 154, inverse transform processing unit 155 , reconstruction unit 158, and filter unit 159 can all execute corresponding operations based on the syntax elements extracted from the bit stream.

[0044] The inverse quantization unit 154 can inverse-quantize the coefficient blocks related to the quantized TUs.

[0045] Inverse transform processing unit 155 can apply one or more inverse transforms to the inverse-quantized coefficient blocks to generate the reconstruction residual blocks of the TUs.

[0046] The prediction unit 152 includes an inter prediction unit 162 and an intra prediction unit 164. When intra prediction coding is used for a PU, the intra prediction unit 164 determines the intra prediction mode of the PU based on the syntax elements decoded from the bit stream, and based on the determined intra prediction mode and the reconstructed reference information adjacent to the PU obtained from the decoded image buffer 160, can execute intra prediction to generate the prediction block of the PU. When inter prediction coding is used for a PU, the inter prediction unit 162 determines one or more reference blocks of the PU based on the motion information of the PU and the corresponding syntax elements, and can generate the prediction block of the PU based on the reference blocks obtained from the decoded image buffer 160.

[0047] The reconstruction unit 158 can obtain a reconstructed image based on the reconstruction residual blocks related to the TUs and the prediction blocks of the PUs generated by the prediction unit 152.

[0048] The filter unit 159 can perform in-loop filtering on the reconstructed image. The filtered reconstructed image is stored in the buffer 160 of the decoded image. The buffer 160 of the decoded image can provide a reference image used for subsequent motion compensation, intra prediction, inter prediction, etc., and can also output the filtered reconstructed image as decoded video data for display on a display device.

[0049] Based on the above video encoding device and video decoding device, the following basic coding process can be executed. On the encoding side, one image is divided into blocks, and for the current block, an intra prediction or an inter prediction is performed or other algorithms are used to generate a predicted block of the current block. The predicted block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain quantization coefficients. The quantization coefficients are entropy coded to generate a bitstream. On the decoding side, an intra prediction or an inter prediction is performed on the current block to generate a predicted block of the current block. On the other hand, the quantization coefficients obtained by decoding the bitstream are inverse quantized and inverse transformed to obtain a residual block. The predicted block and the residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image. The reconstructed image is loop-filtered based on the image or blocks to obtain the decoded image. On the encoding side, in order to obtain the decoded image, similar processing to that on the decoding side is performed. The decoded image obtained on the encoding side is usually also called the reconstructed image. The decoded image can be used as a reference image for inter prediction for subsequent images. The block partitioning information, prediction, transformation, quantization, entropy coding, loop filtering, and other mode information and parameter information determined on the encoding side are written into the bitstream as needed. The decoding side decodes the bitstream or analyzes the existing information to determine the same block partitioning information, prediction, transformation, quantization, entropy coding, loop filtering, and other mode information and parameter information as on the encoding side. Thereby, it is ensured that the decoded image obtained on the encoding side is the same as the decoded image obtained on the decoding side.

[0050] The above is an example of a block-based hybrid coding framework, but the embodiments of the present application are not limited thereto. With the development of technology, one or more modules within the framework and one or more steps within the process can be replaced or optimized.

[0051] In this specification, the current block can be a block-level coding unit such as the current coding unit (current CU) in the current image, the current prediction unit (current PU), etc.

[0052] On the encoding side, in intra prediction, usually, the current block is predicted through various angular modes and non-angular modes to obtain a predicted block. Based on the rate-distortion information calculated from the predicted block and the original block, the optimal intra prediction mode for the current block is selected, and this intra prediction mode is encoded and transmitted to the decoding side through the bitstream. On the decoding side, the intra prediction mode selected for the current block is obtained by decoding, and the intra prediction of the current block is performed according to this intra prediction mode. In this specification, the reference line and intra prediction mode selected for the current block are also described as the reference line and intra prediction mode selected for the current block.

[0053] In VVC and ECM, multiple types of conventional intra prediction modes predict the current block using the reconstructed information around the current block. It includes a planar mode (i.e., Planar mode, mode index is 0), an average value mode (i.e., DC mode, mode index is 1), and 65 angular prediction modes (mode indices are 2 to 66). Figure 2 shows the angular directions of the angular modes with mode indices from 2 to 66. In the text, the angular prediction mode is also abbreviated as the angular mode.

[0054] Since rectangular prediction blocks are introduced in VVC, for rectangular blocks, among the angular modes with indices from 2 to 66, the angular directions of some angular modes can be replaced with wider angular directions. As shown in the figure, the angular modes with indices from -14 to -1 and 67 to 80 are the angular modes obtained by wide-angle replacement. The selection of these angular modes does not need to be represented by a flag and is obtained based on the correspondence between the shape of the current block and the index (2 to 66) of the prediction mode selected for the current block.

[0055] In the VVC standard, the method of wide-angle replacement is as follows.

[0056] The variable whRatio is Abs( Log2( blockWidth ) - Log2( blockHeight ) ). For non-square prediction blocks, wide-angle replacement is performed based on whether the following conditions are satisfied for the angular prediction mode predModeIntra (2 to 66). predModeIntra is indicated by an index. When all of the following three conditions are satisfied, predModeIntra is equal to ( predModeIntra + 65 ). The width of the current block is larger than the height. The angular mode index is 2 or more. The angular mode index is ( whRatio > 1 )? ( 8 + 2 * whRatio ) : less than 8. Otherwise, when all of the following three conditions are satisfied, predModeIntra is equal to (( predModeIntra - 67 ). The height of the current block is larger than the width. The angular mode index is 66 or less. The angular mode index is ( whRatio > 1 )? ( 60 - 2 * whRatio ) : larger than 60. When the matching in the wide-angle mode is completed, there is one angle value intraPredAngle for each angle prediction mode predModeIntra (-14 to 80).

[0057] JPEG2025521765000055.jpg101153

[0058] The angle value of the angle mode is used for subsequent angle prediction.

[0059] JPEG2025521765000056.jpg27170

[0060] As used in this specification, the intra prediction mode, unless otherwise limited, refers to the conventional intra prediction modes including the Planar mode, DC mode, and angle mode.

[0061] Encoding the intra prediction mode of the current block directly requires 7 bits to encode 67 modes, resulting in a large data volume. From statistical characteristics, the closer the current block is, the higher the tendency that the intra prediction mode selected for this sample region is the same as the intra prediction mode selected for the current block. Based on this characteristic, the Most Probable Mode (MPM) technology is adopted in HEVC, VVC, and the Enhanced Compression Model (ECM). ECM is a reference software that can further extract coding performance by integrating various new tools based on the VTM-10.0 reference software.

[0062] For MPM, first create an MPM list, which contains the six intra prediction modes that are most likely to be selected for the current block. If the intra prediction mode selected for the current block is in the MPM list, only its index (only 3 bits are required) needs to be encoded. If the intra prediction mode selected for the current block is not in the MPM list and is within the 61 non-MPM (non-MPM) modes, this intra prediction mode is encoded using a Truncated Binary Code (TBC) truncated at the entropy encoding stage.

[0063] In VVC, regardless of the application of Multiple Reference Line (MRL) and Intra Sub-Partitions (ISP), the MPM list has six prediction modes. The MPM in ECM is divided into MPM and Secondary MPM. The length of the MPM list is 6, and the length of the Secondary MPM list is 16. Among the six modes included in the MPM list, the Planar mode is always entered in the first position in the MPM list, and the remaining five positions are filled by the following three steps until all five positions are filled. The extra modes are automatically entered into the Secondary MPM.

[0064] In the first step, fill in the intra prediction modes used for the prediction blocks at the five neighboring positions around the current block in order. As shown in Figure 3, the five positions sequentially include the positions of the upper left (AL), upper (A), upper right (AR), left (L), and lower left (BL) of the current block.

[0065] In the second step, fill in the mode derived using the gradient histogram based on the reconstructed samples around the current block.

[0066] In the third step, fill an angular mode close to the angle of the angular mode selected in the first step.

[0067] The Secondary MPM list can be composed of some other main angular modes in addition to the intra prediction modes in MPM.

[0068] Since the MPM flag (mpm_flag) is encoded and decoded after the MRL mode, the encoding and decoding of MPM in the ECM depend on the MRL flag. If the MRL mode is not used for the current block, it is necessary to decode the MPM flag to determine whether to use MPM for the current block. If the MRL mode is used for the current block, it is not necessary to decode the MPM flag, and it is default to use MPM for the current block.

[0069] Template based intra mode derivation (TIMD) and Decoder-side intra mode derivation (DIMD) are two intra prediction techniques not in the VVC standard but adopted in the ECM reference software. With these two techniques, the intra prediction mode of the current block can be derived based on the reconstructed sample values around the current block on the decoder side, and as a result, it plays a role in omitting the encoding of the index of the intra prediction mode and saving bits.

[0070] TIMD is an intra prediction mode for the luminance frame. The TIMD mode is generated based on the candidate intra prediction modes in the MPM list and the template region (abbreviated as template). In ECM, as shown in FIG. 4, the left adjacent region and the upper adjacent region of the current block (for example, the current CU) 11 constitute the template region 12 of the current block. The left adjacent region therein is called the left template region (abbreviated as left template), and the upper adjacent region is called the upper template region or the upper side template region (abbreviated as upper template).

[0071] As shown in the figure, a template reference region 13 is provided outside (meaning the left side and the upper side) of the template region 12, and the exemplary sizes and positions of each region are shown. As an example, both the width L1 of the left template and the height L2 of the upper template are 4. The template reference region 13 may be an adjacent row above the template region, or an adjacent column on the left side.

[0072] In TIMD, it is assumed that the distribution characteristics of the current block and the template area of the current block are the same. The reconstructed value of the template reference area is used as the reconstructed value of the reference line. All intra prediction modes in MPM and Secondary MPM are traversed to predict the template area and obtain the prediction result. Then, the error between the reconstructed value of the template area and the prediction result of each mode is calculated and represented by the sum of absolute transformed differences (SATD). The intra prediction mode that minimizes SATD, that is, the optimal intra prediction mode, is selected, and this intra prediction mode is set as the TIMD mode of the current block. On the decoding side, the TIMD mode can be derived by a similar derivation method. If TIMD can be used for the sequence, each current block requires a flag indicating whether to use TIMD. If the intra prediction mode selected for the current block is the TIMD mode, the current block is predicted using the TIMD mode, and the decoding of the remaining syntax elements related to intra prediction such as ISP and MPM can be skipped, thereby significantly reducing the coding bits of the mode.

[0073] JPEG2025521765000057.jpg64170

[0074] The weighting method and weights are shown by the following formula.

Number

Number

[0075] JPEG2025521765000060.jpg22170

[0076] In DIMD, using the reconstructed sample values around the current block as a template, the Sobel operator scans each 3×3 region on the template to calculate the gradients in the horizontal and vertical directions, obtaining the horizontal and vertical gradients Dx and Dy. Based on Dx and Dy, the amplitude value Amp = abs(Dx)+abs(Dy) and the angle value angular = arctan(Dy / Dx) at each position are obtained. The angle value at each position on the template corresponds to the conventional angle mode, and the amplitude values in the same angle mode are accumulated to obtain the histogram of the amplitude value and the angle mode. When there are two angle modes, namely the angle mode with the highest amplitude value and the second highest angle mode, by weighting the angle mode with the highest amplitude value, the second highest angle mode, and the predicted value of the planar mode, the final prediction result when using DIMD for the current block can be obtained. In the prediction mode at this time, three intra prediction modes, namely the planar mode, the angle mode with the highest amplitude value, and the angle mode with the second highest amplitude value, are fused. This is referred to as the DIMD fusion mode in this specification. When there is no angle mode with the highest amplitude value and the second highest angle mode, the prediction using DIMD is equivalent to the planar mode prediction.

[0077] In HEVC, for intra prediction, prediction is performed by referring to the uppermost row and the leftmost column closest to the current block. When the error between the reconstructed values of this row and column and the original sample values is large, the prediction quality of the current block is also greatly affected. To solve this problem, VVC adopts the multiple reference line (MRL) intra prediction technique. Not only the reference line with index 0 (Reference line0) is used, but also the reference line with index 1 (Reference line1) and the reference line with index 2 (Reference line2) can be used as extended reference lines for intra prediction. To reduce the coding complexity, MRL is used only for the non-planar mode in MPM. On the encoding side, when predicting using each angular mode, all three of these reference lines are tried, and through rate-distortion optimization, one reference line with the smallest rate-distortion cost (RD Cost) for the current block is selected, and the index of the selected reference line is encoded and sent to the decoding side. The decoding side decodes to obtain the index of the reference line, determines the reference line selected for the current block according to the index of the reference line, and uses it for the prediction of the current block.

[0078] In the example shown in FIG. 5, four reference lines of the current block are illustrated, including a reference line 0 (reference line0) 221 adjacent to the current block, that is, a reference line with an index of 0, a reference line 1 (reference line1) 222 separated from the current block by one line, that is, a reference line with an index of 1, a reference line 2 (reference line2) 223 separated from the current block by two lines, that is, a reference line with an index of 2, and a reference line 3 (reference line3) 224 separated from the current block by three lines, that is, a reference line with an index of 3. The current block may have more reference lines, that is, reference lines with an index of 4 or more. When predicting, it is also possible to use only the reconstructed values of some reference lines. In this specification, the indices of the reference lines are numbered according to the method shown in FIG. 5.

[0079] In this specification, the reference line is referred to as a "line". Actually, one reference line includes one row and one column. Generally, the reconstructed value of the reference line used when predicting also includes the reconstructed values of one row and one column, which is the same as the usual description method in the industry.

[0080] In ECM, more reference lines can be used in the MRL mode. To encode the reference lines selected for the current block, the indices of multiple candidate reference lines are listed in a list. This list is called the multi-reference line index list (abbreviated as the MRL index list), and can also be called the multi-reference line list, candidate reference line list, reference line index list, etc. When TIMD is not used for the current block, the length of the MRL index list is 6, that is, there are 6 positions in total, and the indices of 6 reference lines can be listed. The indices and their order of these 6 reference lines are fixed, which are 0, 1, 3, 5, 7, 12 in order, and can be expressed by the formula MULTI_REF_LINE_IDX[6] = { 0, 1, 3, 5, 7, 12}. In this MRL index list, the index entered in the first position is 0, which is the index of the reference line closest to the current block, and the indices entered from the second to the sixth positions are 1, 3, 5, 7, 12 respectively, which are the indices of 5 extended reference lines arranged in the order from the one closest to the current block to the one farthest from it.

[0081] If the reference line selected for the current block is in the MRL index list, the multi-reference line index (multiRefIdx) indicates the position of the reference line selected for the current block in the MRL index list, and the selected reference line can be represented by encoding the MRL index. Taking the MRL index list as {0, 1, 3, 5, 7, 12} as an example, the MRL indexes corresponding to the positions from the 1st to the 6th are 0 to 5 respectively. Assuming that the reference line selected for the current block is the reference line with index 0, the MRL index is 0. Assuming that the reference line selected for the current block is the reference line with index 7, the MRL index is 4, and the same applies to other situations. For the MRL index, it can be encoded using a context model-based truncated unary code. After encoding, multiple context model-based bins are obtained. Bins are also called binary flags, binary symbols, binary bits, etc. The smaller the value of the MRL index, the shorter the code length and the faster the decoding.

[0082] JPEG2025521765000061.jpg22170

[0083] Note that the same technology may have different names in different standards. For example, a technology like MPM that derives a list of the most accurate modes using blocks around the current block is called Adaptive Intra Mode Coding (AIMC) in AV2 (AVM). In AVS3, for screen content coding, it is called Frequency-based Intra Mode Coding (FIMC). For non-screen content coding, a technology like MPM is always used. A technology like MRL that performs intra prediction using multiple reference lines is called Multiple reference line selection for intra prediction (MRLS) in AV2 (AVM). However, this is only a difference in name, and the use of terms such as MPM and MRL in this embodiment is assumed to cover these substantially identical technologies in other standards as well.

[0084] One embodiment provides a template-based multiple reference line intra prediction (Multiple reference line & intra_ intra prediction) mode, which is abbreviated as the TMRL mode. The TMRL mode is a prediction mode that constructs a candidate list based on a combination of extended reference lines and intra prediction modes, and encodes and decodes a combination of extended reference lines and intra prediction modes.

[0085] The video encoding method of this embodiment is applied to an encoder and includes steps 110 to 130 as shown in FIG. 6. Step 110: Construct a candidate list for the TMRL mode of the current block, where the candidate list is filled with candidate combinations of extended reference lines and intra prediction modes for the current block. Step 120: Through rate distortion optimization, select one combination of a reference line and an intra prediction mode for the current block to be used for intra prediction. In this specification, the reference lines include a reference line with index 0 and an extended reference line. The combination of the reference line and the intra prediction mode selected for the current block may be a combination of the reference line with index 0 and one intra prediction mode, or a combination of one extended reference line and one intra prediction mode. Step 130: If the encoding condition of the TMRL mode for the current block is satisfied, indicate to use the TMRL mode for the current block by encoding the TMRL mode flag of the current block, and indicate the position of the selected combination in the candidate list by encoding the TMRL mode index of the current block. Here, the encoding condition includes at least that the selected combination is in the candidate list.

[0086] In this specification, the candidate list filled with the candidate combinations of the extended reference line and the intra prediction mode of the current block can also be called the candidate list of the TMRL mode.

[0087] In this specification, what is entered in the candidate list is the candidate combination of the extended reference line and the intra prediction mode of the current block. This means that the combinations in the candidate list need to participate in the rate distortion optimization of the current block, that is, they need to participate in the mode selection process of selecting the prediction mode of the current block through the rate distortion cost. As a result, all combinations in the candidate list may be selected.

[0088] In this specification, the parameters N, M, and K indicating the number are all positive integers and do not need to be described separately.

[0089] What is entered into the candidate list in the TMRL mode constructed according to this embodiment is the extended reference line of the current block and the candidate combination of intra prediction modes, and it is no longer a list of single candidate extended reference lines or a list of candidate intra prediction modes. When the combination of the reference line and the intra prediction mode selected for the current block is in the candidate list (at this time, the extended reference line is selected for the current block), that is, when the encoding conditions such as the selected combination being one of the combinations in this candidate list are satisfied, it is indicated that the TMRL mode is used for the current block by encoding the TMRL mode flag of the current block, and the position of the selected combination in the candidate list is indicated by encoding the TMRL mode index of the current block. The decoding side can determine the extended reference line and the intra prediction mode selected for the current block based on the TMRL mode flag and the TMRL mode index. By the combination encoding and decoding method of this embodiment, the coding cost can be reduced and the coding performance can be improved.

[0090] In the TMRL mode, according to each of the N×M combinations obtained based on the N extended reference lines and M intra prediction modes of the current block, the template area of the current block is predicted, and the error between the reconstructed value of the template area and the predicted value obtained by prediction is calculated. The corresponding K combinations are entered into the candidate list of the TMRL mode of the current block in ascending order of the error, where 1≦K≦N×M is satisfied.

[0091] In one example, a template region is predicted based on each of 25 combinations of an extended reference line and an intra prediction mode in the ECM, the 25 combinations are sorted in ascending order of error, and 12 combinations with the minimum error, that is, the most likely to be selected, are entered into the candidate list for the TMRL mode. Thereby, the number of coding bits for encoding the TMRL mode index can be reduced. Even when a part of the extended reference lines {1, 3, 5, 7, 12} (for example, the extended reference lines with indexes 7 and 12) is outside the CTU boundary, based on the order of 15 combinations, 12 combinations with the highest selection probability can also be entered into the candidate list, and the coding bits of the TMRL mode index can still be fully and effectively utilized. By sorting in ascending order of error, reference lines and prediction modes more likely to be selected for prediction remain in the candidate list, and combinations more likely to be selected are arranged in the previous positions in the list, so that the coding cost is reduced. The creation of the candidate list for the TMRL mode will be described in detail below.

[0092] In one example of this embodiment, the encoding condition further includes not using TIMD for the current block. When using TIMD for the current block, the video encoding method skips the encoding of the TMRL mode flag and the TMRL mode index of the current block, and when not using TIMD for the current block but the selected combination is not in the candidate list, it encodes the TMRL mode flag of the current block to indicate not using the TMRL mode for the current block and skips the encoding of the TMRL mode index of the current block.

[0093] This embodiment is the case where encoding and decoding are performed based on the TIMD mode before the TMRL mode. When using the TIMD mode for the current block, since there is no need to use the TMRL mode, the encoding of the TMRL mode flag and the TMRL mode index is skipped. When not using the TIMD mode for the current block, there are two cases: the situation where the combination selected for the current block is in the candidate list of the TMRL mode, and the situation where the selected combination is not in the candidate list of the TMRL mode. When the selected combination is not in the candidate list, it is necessary to encode the TMRL mode flag to indicate that the TMRL mode is not used for the current block, and the encoding of the TMRL mode index is skipped. When the selected combination is in the candidate list, it is necessary to encode both the TMRL mode flag and the TMRL mode index.

[0094] When not using TIMD for the current block, the TMRL mode flag and the TMRL mode index provided by this embodiment can replace the original multi-reference line index multiRefIdx. When using TIMD for the current block, it is still possible to use the multi-reference line index to indicate the selected reference line, and the multi-reference line index can be encoded.

[0095] In an example of this embodiment, constructing the candidate list of the TMRL mode for the current block includes constructing the candidate list only when all the set conditions for using the TMRL mode for the current block are satisfied. The conditions for using the TMRL mode include any one or more of the following. Condition 1: The current block is a block in the luminance frame, that is, the TMRL mode is used only for the luminance frame (i.e., the luminance image). Condition 2: The current block is not located at the upper boundary of the coding tree unit (CTU). If it is located at the upper boundary of the CTU, there is no available reference line above the current block. Therefore, in this embodiment, not being located at the upper boundary of the CTU is a condition for using the TMRL mode. Condition 3: The TMRL mode can be used only when the MRL can be used for the current block, that is, when the MRL mode can be used. Condition 4: The size of the current block is not larger than the maximum size of the current block for which the TMRL mode can be used. This maximum size may be preset. Generally, larger blocks are flatter and have relatively fewer angular details. For such large blocks, the use of the TMRL mode can be restricted. Condition 5: The aspect ratio of the current block meets the requirement for using the TMRL mode for the aspect ratio of the current block. For example, the use of the TMRL mode is permitted only when the aspect ratio of the current block is not larger than a preset value.

[0096] In an example of this embodiment, the Golomb-Rice coding method is used to encode the TMRL mode index. By using the Golomb-Rice coding, candidate combinations can be more reasonably grouped into categories with different codeword lengths for encoding and decoding, improving the coding efficiency.

[0097] In an example of this embodiment, when the encoding condition of the TMRL mode of the current block is satisfied, the video encoding method further includes skipping the encoding of the syntax elements of any one or more of the MPM mode, intra subpartition split (ISP) mode, multi-conversion selection (MTS) mode, low-frequency non-separable transform (LFNST) mode, and TIMD mode.

[0098] For example, when encoding and decoding based on the TMRL mode before the TIMD mode, the encoding conditions of the TMRL mode do not include not using TIMD for the current block. When using the TMRL mode for the current block, TIMD cannot be used for the current block, and the encoding of the syntax elements of the TIMD mode can be skipped.

[0099] For example, when using the TMRL mode for the current block, the TMRL mode flag and the TMRL mode index can simultaneously indicate the reference line and the intra prediction mode selected for the current block, and there is no need to encode and decode the syntax elements related to MPM.

[0100] For example, in a specific conversion mode, it can be restricted that the TMRL mode is not used simultaneously with the Multiple Transform Selection (MTS) mode and / or the Low Frequency Non-separable Transform (LFNST) mode.

[0101] One embodiment provides a method for constructing a candidate list of the TMRL mode. This method may be applied to an encoder or a decoder. As shown in FIG. 7, this method includes the following content. Step 210: Based on N extended reference lines and M intra prediction modes of the current block, obtain N×M combinations of the extended reference lines and the intra prediction modes, where N≧1, M≧1, and N×M≧2 are satisfied. Step 220: Predict the template region of the current block based on each of the N×M combinations, and calculate the error between the reconstructed value of the template region and the predicted value obtained by prediction. The error in this step can be represented by, but not limited to, the Sum of Absolute Difference (SAD), the Sum of Absolute Transformed Difference (SATD), the Sum of Squared Difference (SSD), the Mean Absolute Difference (MAD), the Mean Squared Error (MSE), etc. Step 230: Enter K combinations corresponding to the error in ascending order of the error into the candidate list of the TMRL mode of the current block, where 1 ≤ K ≤ N × M is satisfied.

[0102] With the candidate list created in this embodiment, the encoding of the combination of the extended reference line and the intra prediction mode can be realized, and the encoding efficiency can be improved. Also, by predicting and ordering the errors for the template areas using different combinations, based on the similarity of the distribution characteristics between the current block and the template area of the current block, K combinations with relatively high selection probabilities can be selected from the N × M combinations, and by arranging the combinations with high selection probabilities at the previous positions in the candidate list, the TMRL mode index of the combination selected during encoding can be reduced, and the actual encoding cost can be reduced.

[0103] In an example of this embodiment, the template area of the current block is set to the reference line closest to the current block, or the template area of the current block is set to a plurality of reference lines closest to the current block, and the N extended reference lines participating in the combination are the extended reference lines located outside the template area. In FIG. 8A, the template area of the current block is set to the reference line 30 with an index of 0. When constructing the candidate list in TMRL mode, N available extended reference lines are selected from the pre-defined extended reference lines with indices {1, 3, 5, 7, 12}. If there are 13 or more reference lines between the upper part of the current block and the CTU boundary, 5 extended reference lines with indices {1, 3, 5, 7, 12} are selected to form a combination. If there are 6 or 7 reference lines between the upper part of the current block and the CTU boundary, 3 extended reference lines with indices {1, 3, 5} are selected to form a combination.

[0104] In this example, when the template area of the current block is set to the reference line with an index of 0, the reference line with an index of 0 is called the reference line where the template area is located, and the reference lines with indices from 1 to 3 are called the reference lines located outside the template area. When the template area of the current block is set to the reference lines with indices of 0 and 1, the reference lines where the template area is located include extended reference lines, the reference lines with indices of 0 and 1 are the reference lines where the template area is located, and the reference lines with indices of 2 and 3 are the reference lines located outside (above and to the left) of the template area.

[0105] FIG. 8A shows five extended reference lines participating in the combination, namely reference line 31 with index 1, reference line 33 with index 3, reference line 35 with index 5, reference line 37 with index 7, and reference line 39 with index 12. Different from FIG. 8A, in the example shown in FIG. 8B, the template area 40 of the current block is set to two reference lines with indices 0 and 1, and the extended reference lines participating in the combination are five extended reference lines, namely reference line 42 with index 2, reference line 43 with index 3, reference line 45 with index 5, reference line 47 with index 7, and reference line 49 with index 12. That is, in this example, N available extended reference lines are selected from the pre-defined extended reference lines with indices {2, 3, 5, 7, 12}. There are also many other options for the selection of the template area and the extended reference lines. For example, the template area of the current block can be set to three reference lines with indices 0, 1, 2, or the template area of the current block can be set to four reference lines with indices 0 to 3. When the template area is relatively large, the prediction is relatively accurate.

[0106] In one example of this embodiment, the N extended reference lines of the current block are the extended reference lines that are located outside the template area of the current block and do not exceed the boundary of the coding tree unit (CTU) among the pre-defined N max extended reference lines. N max is the maximum number of extended reference lines that can be used in the TMRL mode. In this embodiment, the N extended reference lines used in the combination are restricted to the area outside the template area of the current block and not exceeding the CTU boundary. However, if the hardware can support it, extended reference lines exceeding the CTU boundary can also be selected to participate in the combination.

[0107] In one example of this embodiment, N maxIt is equal to 5, and the five pre-defined extended reference lines are reference lines indexed by {1, 3, 5, 7, 12} or {2, 3, 5, 7, 12}. In another example, the pre-defined N max extended reference lines are the N max extended reference lines starting from index 1 and closest to the current block, or the N max extended reference lines starting from index 1 and having odd indices and closest to the current block, or the N max extended reference lines starting from index 2 and having even indices and closest to the current block. Selecting odd or even reference lines simplifies the calculation.

[0108] In an example of this embodiment, the M intra prediction modes can be selected only from the angle mode, or only from the angle mode and the DC mode, or from the angle mode, the DC mode, and the planar mode.

[0109] In one example, the M intra prediction modes are selected by the following method. M ≥ 5 is satisfied. First step: Determine the intra prediction modes used for the prediction blocks at five neighboring positions around the current block, select the available intra prediction modes among them in order, and delete the duplicate modes. The five neighboring positions are, as shown in FIG. 3, the upper left, upper, upper right, left, and lower left respectively. When the number of intra prediction modes selected in the first step becomes M, end. If the number of intra prediction modes selected in the first step is less than M and includes the angle mode, execute the second step. Second step: Perform an extension operation on the selected first angle mode in order until the total number of selected intra prediction modes becomes M to obtain the extended angle modes, and select the extended angle modes different from all the selected angle modes.

[0110] In this embodiment, when the angular mode is not selected in the first step, or when the total number of intra prediction modes selected in the second step is still less than M, the third step is executed. In the third step, the selectable intra prediction modes from the pre-defined set of intra prediction modes are determined, and until the total number of selected intra prediction modes reaches M, intra prediction modes different from any of the selected intra prediction modes are sequentially selected from the determined selectable intra prediction modes. This further includes the above steps.

[0111] In one example, only the angular mode can be selected in the process of selecting M intra prediction modes. In another example, only the angular mode and the DC mode can be selected in the process of selecting M intra prediction modes. In yet another example, the angular mode, the DC mode, and the Planar mode can be selected in the process of selecting M intra prediction modes. Since the combination effect with the extended reference line of the Planar mode is limited, it may be excluded from the combination. The same is true for the DC mode. However, if the increase in computational complexity can be supported, the Planar mode and the DC mode can also be added to the candidate list and participate in the combination.

[0112] In one example, the extension operation of the angular mode includes any one or more of the operations of adding 1 to and subtracting 1 from the angular mode, adding 2 to and subtracting 2 from the angular mode, and adding 3 to and subtracting 3 from the angular mode.

[0113] In an example of this embodiment, as the M intra prediction modes, some or all of the intra prediction modes other than the Planar mode among the most probable modes (MPMs) are used, or some or all of the intra prediction modes other than the Planar mode among the MPMs and the second MPMs are used, or some or all of the intra prediction modes other than the Planar mode and the DC mode among the MPMs are used, or some or all of the intra prediction modes other than the Planar mode and the DC mode among the MPMs and the second MPMs are used, or some or all of the intra prediction modes other than the Planar mode, the DC mode, and the DIMD mode among the MPMs are used, or some or all of the intra prediction modes other than the Planar mode, the DC mode, and the DIMD mode among the MPMs and the second MPMs are used.

[0114] In this example, all of the intra prediction modes other than the Planar mode among the MPMs are used as pre - defined intra prediction modes, or all of the intra prediction modes other than the Planar mode among the MPMs and the second MPMs are used as pre - defined intra prediction modes. When the reference lines of the current block include all of the pre - defined extended reference lines, all of the pre - defined intra prediction modes are used. When the reference lines of the current block include some of the pre - defined extended reference lines, some of the pre - defined intra prediction modes are used.

[0115] In an example of this embodiment, the M intra prediction modes are selected by the following method. Select M' intra prediction modes. Based on the reference lines located outside the template of the current block and the M' intra prediction modes, predict the template of the current block respectively, and calculate the errors between the reconstructed values of the template and the predicted values obtained by prediction to calculate M' errors. As the M intra prediction modes participating in the combination, select M intra prediction modes corresponding to the minimum errors from the M' intra prediction modes, where M < M' is satisfied.

[0116] The template of the current block in this example is used to select M intra prediction modes from M' intra prediction modes, and the aforementioned template area of the current block is used to select K combinations from N×M combinations. Both may be different or may occupy the same area.

[0117] When selecting M' intra prediction modes in this example, various methods for selecting M intra prediction modes in the above embodiment can be used. For example, a method of directly selecting from the lists of MPM and the second MPM, or a method of selecting through the first step in the method of the aforementioned embodiment, or a method of selecting through the first step and the second step, or a method of selecting through the first step, the second step, and the third step, etc. can be used.

[0118] In an example of this embodiment, N≦N max , 2≦N max ≦12, 2≦M≦18, K≦K max , 6≦K max ≦36, N max is the maximum number of extended reference lines that can be used in the TMRL mode, and K max is the maximum number of candidate combinations that can be used in the TMRL mode. Here, the value ranges of the related parameters N, M, and K are shown, but this is merely exemplary.

[0119] In an example of this embodiment, N, M, and K have at least two sets of values. The first set of values is N1, M1, K1, the second set of values is N2, M2, K2, and N1≤N2, M1≤M2, K1≤K2, and N1×M1<N2×M2 are satisfied. The first set of values is used when constructing a candidate list for the current block having the first size, and the second set of values is used when constructing a candidate list for the current block having the second size. The first size is smaller than the second size. Here, the first size and the second size can each represent various sizes. For example, the first size can include 4×4, 4×8, 8×8, etc., and the second size can include 16×8, 16×16, 8×16, etc. In this example, different N, M, and K are used for current blocks of different sizes. When the size of the current block is relatively small, smaller values are used to construct the candidate list in the TMRL mode. When the size of the current block is relatively large, larger values are used to construct the candidate list in the TMRL mode. Thereby, a better balance between the computational complexity and performance can be achieved.

[0120] In an example of this embodiment, predicting the template area of the current block based on each of the N×M combinations includes, when the current block is located at the left boundary of the picture, predicting the upper template area of the current block based on each of the N×M combinations and not predicting the left template area of the current block. According to this embodiment, the operation can be simplified and the time required for the operation can be shortened without affecting the performance.

[0121] In one example of this embodiment, predicting the template area of the current block based on each of the N×M combinations includes predicting in the following manner based on each of the N×M combinations. An initial prediction value of the template area is calculated based on the reconstruction value of the extended reference line in the combination and the intra prediction mode in this combination. The reconstruction value of the extended reference line is the original reconstruction value of the extended reference line or the reconstruction value after filtering. 4-tap filtering or 6-tap filtering is performed on the initial prediction value of the template area, and the filtering result is used as the predicted value of the template area obtained by prediction based on this combination.

[0122] In this embodiment, when predicting the template area of the current block, the original reconstruction value can be used in the calculation without filtering the reconstruction value of the extended reference line. Also, in order to reduce the complexity of the calculation and speed up the calculation, a filter with a relatively short number of taps (for example, a 4-tap filter) can be used.

[0123] In one example of this embodiment, predicting the template area of the current block based on each of the N×M combinations and calculating the error between the reconstruction value of the template area and the predicted value obtained by prediction includes the following. Predict the entire template area of the current block based on each of the K combinations to obtain an error set consisting of the corresponding K errors, and record the maximum error D in the error set. For each of the remaining combinations, first predict one side of the template area of the current block based on this combination, calculate the error D1 between the reconstruction value and the predicted value of this one side of the template area of the current block. If D1≧D max , complete the prediction based on this combination. If D1<D max , then, also predict the other side of the template area of the current block based on this combination, calculate the error D2 between the reconstruction value and the predicted value of the entire template area of the current block. If D2<D max , then, also predict the other side of the template area of the current block based on this combination, calculate the error D2 between the reconstruction value and the predicted value of the entire template area of the current block. If D2<D maxIf so, add D2 to the error set, delete D from the error set, update the maximum error D within the error set, and if D2≥D, complete the prediction based on this combination. After completing the prediction based on N×M combinations, set the K combinations corresponding to the K errors in the error set as the K combinations corresponding to the minimum error. max in the error set, and max update D. If D2≥D max complete the prediction based on this combination. After completing the prediction based on N×M combinations, set the K combinations corresponding to the K errors in the error set as the K combinations corresponding to the minimum error.

[0124] In one example, the errors in the error set can be arranged in ascending order, and when adding D2 to the error set, insert D2 at a position where the errors in the error set can still be arranged in ascending order. However, in other examples, after completing the prediction based on N×M combinations, the K errors in the error set can also be sorted.

[0125] In this example, without performing prediction and error calculation for the entire template area based on all combinations, the sorting of combinations can be completed, reducing the computational complexity and speeding up the operation.

[0126] In an example of this embodiment, entering the K combinations corresponding to the errors in ascending order of error into the candidate list of the TMRL mode of the current block includes entering the K combinations corresponding to the errors in ascending order of error into the candidate list starting from the first position of the candidate list. Only combinations of the extended reference line and the intra prediction mode are entered in the candidate list of the TMRL mode in this example. Combinations of the reference line with index 0 and the intra prediction mode are indicated by other conventional modes such as MPM.

[0127] In an example of this embodiment, from the i-th position of the candidate list, K combinations corresponding to the errors are entered into the candidate list in ascending order of the errors. Before the i-th position of the candidate list, a combination of a reference line with index 0 and one or more intra prediction modes is entered, and i ≥ 2 is satisfied. In this example, in the candidate list of the TMRL mode, not only combinations of the extended reference line and the intra prediction mode but also combinations of the reference line with index 0 and the intra prediction mode are entered. In this case, when the combination selected for the current block is a combination of the reference line with index 0 and the intra prediction mode, it can also be indicated by the TMRL mode index. The TMRL mode flag at this time can still be used.

[0128] One embodiment of the present application provides a video decoding method related to the TMRL mode. This method is applied to a decoder and includes steps 310 to 330 as shown in FIG. 9. Step 310: Decode the multi-reference line intra prediction (TMRL) mode flag of the current block to determine whether to use the TMRL mode for the current block. Step 320: If it is determined to use the TMRL mode for the current block, subsequently decode the TMRL mode index of the current block, construct a candidate list for the TMRL mode of the current block, and enter candidate combinations of the extended reference line and the intra prediction mode for the current block into the candidate list. Step 330: Based on the candidate list and the TMRL mode index, determine the combination of the extended reference line and the intra prediction mode selected for the current block, and predict the current image based on the selected combination. Here, the TMRL mode index is used to indicate the position in the candidate list of the selected combination of the extended reference line and the intra prediction mode.

[0129] In this embodiment, after decoding the TMRL mode flag and determining to use the TMRL mode for the current block, a combination of an extended reference line and an intra prediction mode is entered into the candidate list of the TMRL mode, and the combination selected for the current block is determined based on the decoded TMRL mode index and the candidate list, and prediction is performed. That is, the TMRL mode index can simultaneously indicate the extended reference line and the intra prediction mode selected for the current block without using two indexes. The coding cost can be reduced.

[0130] In an example of this embodiment, before decoding the TMRL mode flag of the current block, the video decoding method further includes decoding the TMRL mode flag of the current block when all conditions for using the TMRL mode for the current block are satisfied. The conditions for using the TMRL mode include any one or more of the following, that is, the current block is a block in the luminance frame, MRL can be used for the current block, the current block is not located at the upper boundary of the coding tree unit (CTU), template-based intra mode derivation (TIMD) is not used for the current block, including any one or more of these conditions.

[0131] In this example, when one of the above conditions is met, the use of the TMRL mode is not permitted, and the decoding of the TMRL mode flag and the TMRL mode index can be skipped. However, this may not be the case in other embodiments. For example, when encoding and decoding based on the TMRL mode before TIMD, the use of TIMD for the current block is not made a condition where the TMRL mode cannot be used. As another example, in the future, when the hardware becomes able to support the acquisition of reference lines outside the CTU boundary, the fact that the current block is located at the upper boundary of the CTU will no longer be a condition where the TMRL mode cannot be used, etc.

[0132] In an example of this embodiment, the video decoding method further includes decoding the MRL index of the current block when the MRL can be used for the current block, the current block is not located at the upper boundary of the CTU, and it is determined that TIMD is to be used for the current block. The MRL index is used to indicate the position in the MRL index list of the reference line selected for the current block. In this example, although the use of the TMRL mode is not permitted when TIMD is used for the current block, the MRL can still be used. Therefore, by still decoding the multi-reference line index of the current block, the reference line selected for the current block can be determined, and in combination with the TIMD mode selected for the current block, the current block can be predicted.

[0133] In an example of this embodiment, when it is determined to use the TMRL mode for the current block based on the TMRL mode flag, the video decoding method further includes skipping the decoding of syntax elements of any one or more of the MPM mode, ISP mode, MTS mode, LFNST mode, and TIMD mode. Corresponding to the encoding side, when encoding and using the TMRL mode flag for the current block, when one or more of the above modes are skipped, the decoding side also skips the decoding of these modes when it is determined to use the TMRL mode flag for the current block by decoding.

[0134] One embodiment further provides a video decoding method, which mainly relates to the decoding process of intra prediction. The encoding side will also be described. In this embodiment, on the encoding side, a candidate list for the TMRL mode is constructed. When a combination in the candidate list is selected for the current block by mode selection, the syntax elements of the TMRL mode are encoded and decoded, and the combination of the extended reference line and the intra prediction mode is encoded and decoded.

[0135] In this embodiment, based on N pre-defined extended reference lines and M intra prediction modes, a template is constructed at the position of the reference line with index 0, that is, reference line0. The template area 30 shown in FIG. 8A can be referred to. (x, -1), (-1, y) are the coordinates with respect to the (0, 0) position of the upper left corner of the current block, respectively. In this figure, 5 pre-defined extended reference lines with indices {1, 3, 5, 7, 12} are also provided.

[0136] When constructing the candidate list for the TMRL mode, calculate the sum of absolute differences (SAD) between the predicted value and the reconstructed value of the template based on N×M combinations, sort the corresponding combinations in ascending order of SAD, and enter the K combinations with relatively small SAD among the N×M combinations into the candidate list for the TMRL mode in ascending order of SAD, where N×M≥2 is satisfied.

[0137] The construction of the candidate list for the TMRL mode is an operation that needs to be performed by both the encoder and the decoder. On the encoding side, if encoding conditions such as the combination selected for the current block being in the candidate list are satisfied, it indicates using the TMRL mode by encoding the TMRL mode flag. Also, the TMRL mode index is determined based on the position of the selected combination in the candidate list. For example, if it is in the first position, the TMRL mode index is 0, and if it is in the second position, the TMRL mode index is determined to be 1. The TMRL mode index may be encoded using the Golomb-Rice coding method, but is not limited to this.

[0138] Hereinafter, an example with N = 5, M = 6, and K = 12 will be used for explanation. The indexes of the five pre-defined extended reference lines are {1, 3, 5, 7, 12}, and six intra prediction modes are selected step by step.

[0139] The video decoding method of this embodiment includes the content of steps 1 to 3.

[0140] Step 1: Decode the syntax elements related to the TMRL mode. The related syntax elements of the intra prediction mode analyzed by the decoder include the related syntax elements of modes such as TIMD and MRL. The TMRL mode provided in this embodiment can be considered as an evolved form of the MRL mode, and the syntax elements of the TMRL mode can also be considered as a part of the syntax elements of the MRL mode. Of course, both can also be considered as two different modes.

[0141] When the TIMD mode is used for the current block, there is no change in the decoding method for the syntax elements of the MRL mode. When the TIMD mode is not used for the current block, it is necessary to decode the syntax elements of the TMRL mode. As shown in the following table, decode the related syntax for the current block.

[0142]

Table 2

[0143] "cu_tmrl_flag" in the table is the TMRL mode flag. When "cu_tmrl_flag" is equal to 1, it indicates that the TMRL mode is used for the current block, that is, it is defined that the type of the intra prediction mode of the current luminance sample is the template-based multi-reference line intra prediction mode. When "cu_tmrl_flag" is equal to 0, it indicates that the TMRL mode is not used for the current block, that is, it is defined that the type of the intra prediction mode of the current luminance sample is not the template-based multi-reference line intra prediction mode.

[0144] The "tmrl_idx" in the table, i.e., the TMRL mode index, indicates the position of the combination of the extended reference line and the intra prediction mode selected for the current block in the candidate list of TMRL modes, and can also be said to define the index of the selected combination (the index indicating the position of the combination) in the ordered candidate list of TMRL modes. "tmrl_idx" can be encoded and decoded using the Golomb-Rice coding method, which will not be repeated here.

[0145] As can be seen from the above table, before decoding cu_tmrl_flag, it is first necessary to determine whether the following conditions are met: MRL can be used for the current block (i.e., whether sps_mrl_enabled_flag is equal to 1), the current block is not located at the upper boundary of the CTU (i.e., whether (y0 % CtbSizeY) > 0 holds), and TIMD is not used for the current block. If these conditions are met, cu_tmrl_flag is decoded. If the other two conditions are met but TIMD is used for the current block, the multi-reference line index intra_luma_ref_idx of the current block is decoded.

[0146] The ISP mode flag (intra_subpartitions_mode_flag) in the table is decoded after the syntax elements related to the TMRL mode. If the TMRL mode is not used for the current block (!cu_tmrl_flag holds), intra_subpartitions_mode_flag is decoded. Similarly, if the TMRL mode is not used for the current block (!cu_tmrl_flag holds), the syntax elements related to MPM are decoded.

[0147] Step 2: Construct the candidate list of TMRL modes, and based on the TMRL mode index and the candidate list, determine the extended reference line and the intra prediction mode selected for the current block. After the parsing stage and before predicting the current block, if the TMRL mode is used for the current block, it is necessary to construct a candidate list for the TMRL mode. Based on the TMRL mode index and the candidate list, determine the extended reference line and the intra prediction mode selected for the current block.

[0148] To construct the candidate list for the TMRL mode, first, it is necessary to determine the candidate extended reference line and the candidate intra prediction mode.

[0149] JPEG2025521765000064.jpg671

[0150] The candidate extended reference line is selected from the pre-defined extended reference lines. Based on the position of the current block in the image, determine the available extended reference lines among the pre-defined extended reference lines. In principle, the upper reference line available for the current block should not exceed the upper boundary of the CTU. In one example, all the extended reference lines with indices {1, 3, 5, 7, 12} that do not exceed the CTU boundary are added to the candidate extended reference lines. Also, more or fewer extended reference lines can be used to improve coding performance or reduce complexity.

[0151] JPEG2025521765000065.jpg679

[0152] In this embodiment, instead of binding the TMRL mode and the MPM (it may also be bound in other embodiments), construct a candidate list for the intra prediction mode and select the intra prediction mode used for the combination from this candidate list. The candidate list is derived in the following way.

[0153] First, from the conventional 67 types of prediction modes, the Planar mode and the DC mode are removed, or only the Planar mode is removed and the DC mode is left. The removed modes are not added to the candidate list, that is, among the TMRL modes, they are not used as the intra prediction modes participating in the combination.

[0154] In this embodiment, the length of the candidate prediction mode list to be constructed is 6. First, non-overlapping intra prediction modes are sequentially selected from the intra prediction modes used in the prediction blocks at five neighboring positions around the current block and added to the candidate prediction mode list. Next, an extension operation of the angle mode is performed on the modes entered in the list. Specifically, for the angle mode, an operation of adding 1 and an operation of subtracting 1 can be performed, and non-overlapping extended angle modes are selected and sequentially added to the candidate prediction mode list. When the number of modes entered in the candidate list reaches 6, the filling is stopped.

[0155] The specific contents of the operation of adding 1 and the operation of subtracting 1 are shown in the following table.

[0156]

Table 3

[0157] The above-mentioned "angle mode - 1" refers to the angle mode obtained by subtracting 1 from the index of the entered angle mode. For example, when the entered angle mode is mode 3, "angle mode - 1" is angle mode 2. The above-mentioned "angle mode + 1" refers to the angle mode obtained by adding 1 to the index of the entered angle mode. For example, when the entered angle mode is mode 3, "angle mode + 1" is angle mode 4.

[0158] When "angle mode -1" is smaller than angle mode 2, for example, when obtaining "angle mode 1" from "angle mode -1" (the index of the angle mode is numbered starting from 2 and "angle mode 1" does not exist), select the angle mode in the opposite direction of "angle mode -1". Assuming there are a total of 65 types of angle modes, at this time, the angle mode in the opposite direction of "angle mode -1" is namely angle mode 66. And when "angle mode +1" is larger than angle mode 66, similarly, select the angle mode in the opposite direction of "angle mode +1". For example, if the entered angle mode is angle mode 66 and "angle mode +1" does not exist, the angle mode in the opposite direction of "angle mode +1" selected at this time is namely angle mode 2.

[0159] The above extension of the angle mode is to perform an operation of adding 1 and an operation of subtracting 1 to the angle mode. In other embodiments, it is also possible to extend the operation of adding 1 and the operation of subtracting 1 to the operation of adding X and the operation of subtracting X from the operation of adding 1 and the operation of subtracting 1. Assuming X = 3, until the candidate prediction mode list is filled, it is possible to perform the operation of adding 1 and the operation of subtracting 1, the operation of adding 2 and the operation of subtracting 2, and the operation of adding 3 and the operation of subtracting 3 to the angle mode.

[0160] After performing the extension operation of the angle mode on the mode entered in the list, if the candidate prediction mode list is not yet filled, use the non-duplicate modes in the pre-defined mode set to fill the list until the candidate prediction mode list is filled. The mode set includes several angle modes selected according to the following statistical laws. mpm_default[] = { DC_IDX, VER_IDX, HOR_IDX, VER_IDX - 4, VER_IDX + 4, 14, 22, 42, 58, 10, 26, 38, 62, 6, 30, 34, 66, 2, 48, 52, 16}; Here, DC_IDX represents the DC mode, VER_IDX represents the vertical mode, HOR_IDX represents the horizontal mode, and the remaining numbers represent the angle modes corresponding to those numbers.

[0161] In this embodiment, the length of the candidate prediction mode list is set to 6. However, from the perspective of performance, in order to try more angle modes, it is also possible to set the length to a value greater than 6. Also, in order to reduce complexity, fewer modes can be tried and the length can be set to a value less than 6.

[0162] In this embodiment, when determining the candidate intra prediction mode, the Planar mode and the DC mode are excluded, or only the Planar mode is excluded. However, if complexity is not considered, these two modes do not have to be excluded. That is, the Planar mode, the DC mode, and all angle modes may participate in the combination with the extended reference line as candidate intra prediction modes.

[0163] JPEG2025521765000067.jpg683

[0164] After determining the candidate extended reference line and the intra prediction mode, all combinations in the extended reference line list and the candidate prediction mode list can be tried one by one. Based on each of these combinations, referring to FIG. 8A, the template (shown in the next figure) area of the line where reference line0 is located is predicted, the error between the reconstructed value of the template area and the predicted value obtained from the prediction based on each combination is calculated, and the K combinations with the smallest errors are entered into the candidate list of the TMRL mode in ascending order of the errors.

[0165] In the prediction process of this embodiment, the use of the TMRL mode is restricted only when the current block occupies the first row of the CTU. When the current block is located at the left boundary of the image, the TMRL mode can still be used. In this case, since the left reference line0 is already outside the image boundary, the left template is not used for prediction, that is, only the upper template area is predicted.

[0166] The prediction process on the template area can be exactly the same as other normal intra-angle prediction processes. That is, first, the reconstructed value of the reference line is filtered, and the result is used as the initial prediction value of the template area. After predicting the template area based on the intra-prediction mode in combination with the reconstructed value of the filtered reference line, 4-tap or 6-tap filtering is performed on the initial prediction result, and the result is used as the predicted value obtained by prediction. Note that, considering the computational complexity, the step of filtering the reconstructed value of the reference line may be omitted, or a filter with relatively short taps may be used. In this embodiment, when obtaining the predicted value of the template area by prediction, instead of filtering the reconstructed value of the pixels of the reference line, 4-tap interpolation filtering with 1 / 32 precision at a non-integer angle is performed on the initial prediction result.

[0167] Finally, based on the angle, reference line, and filter in the current combination, the template area is predicted. The SAD between the predicted value of the template area obtained from the prediction and the reconstructed value of the template area is calculated, sorted in ascending order of SAD, and 12 combinations with the minimum SAD are selected and entered into the candidate list of the TMRL mode.

[0168] In the sorting process, a high-speed algorithm can be used. In the above process, although it is necessary to try a total of 30 combinations of 5 reference lines and 6 prediction modes, only 12 combinations with the smallest SAD among them need to be selected. In this embodiment, after completing the prediction based on the previous 12 combinations and obtaining the corresponding SAD, from the 13th combination, it is only necessary to maintain and update the 12 combinations with the minimum error (which may also be called cost). From the 13th combination, only the upper template area is predicted and the corresponding SAD is calculated. If the SAD calculated based on the upper template is already larger than the combination with the largest error among the 12 combinations with the minimum error, the prediction and error calculation of the left template area can be skipped. Specifically, reference can be made to the previous embodiment.

[0169] Step 3: Based on the candidate list of the constructed TMRL mode and the TMRL mode index obtained by decoding, determine the combination of the extended reference line and the intra prediction mode selected for the current block, and perform intra prediction on the current block based on the selected combination.

[0170] JPEG2025521765000068.jpg17170

[0171] In the ECM-4.0 reference software, using the method described in this embodiment, with the settings of N = 5 (the 5 extended reference lines are 1, 3, 5, 7, and 12 respectively), M = 6 (6 predefined prediction modes excluding the Planar mode and the DC mode), and K = 12 (only selecting the previous 12 combinations with small SAD in all combinations), the measurement results in the AI layout are as follows.

[0172]

Table 4

[0173] N = 5 (the five extended reference lines are 1, 3, 5, 7, and 12 respectively), M = 8 (the eight predefined prediction modes), K = 16. In this setting, the measurement results in the AI layout are as follows.

[0174]

Table 5

[0175] N = 5 (the five extended reference lines are 1, 3, 5, 7, and 12 respectively), M = 12 (the twelve predefined prediction modes), K = 24. In this setting, the measurement results in the AI layout are as follows.

[0176]

Table 6

[0177] N = 5 (the five extended reference lines are 1, 3, 5, 7, and 12 respectively), M = 8 (the eight predefined prediction modes that do not include the Planar mode but include the DC mode), K = 16. In this setting, the measurement results in the AI layout are as follows.

[0178]

Table 7

[0179] The meanings of the parameters in the table are as follows.

[0180] EncT: Encoding Time, that is, the encoding time. 10X% means that when the reference line sorting technology is integrated, the encoding time is 10X% compared to before integration, which means the encoding time has increased by X%.

[0181] DecT: Decoding Time, i.e., the decoding time. 10X% means that when the reference line sorting technology is integrated, the decoding time is 10X% compared to before integration, which means the decoding time has increased by X%.

[0182] Class A1 and Class A2 are test video sequences with a resolution of 3840×2160, Class B is a test sequence with a resolution of 1920×1080, Class C is a test sequence with a resolution of 832×480, Class D is a test sequence with a resolution of 416×240, Class E is a test sequence with a resolution of 1280×720, and Class F is a sequence of several screen contents with different resolutions.

[0183] JPEG2025521765000073.jpg16170

[0184] All intra indicates a test configuration where all are intra frames.

[0185] As can be seen from this, by performing intra prediction coding using the TMRL mode of this embodiment, the coding performance can be significantly improved.

[0186] In this embodiment, for both the extended reference line and the prediction mode, a template area of one row and one column is used, and sorting and selection are performed in ascending order of SAD. For the extended reference line, if all extended reference lines (including reference line1) are sorted, only a template of one row and one column can be used. However, for the selection of the prediction mode, in order to select a more appropriate prediction mode, more reference lines can be used as in the case of the TIMD mode, thereby obtaining more accurate results. In other embodiments, the method of determining the candidate prediction mode list based on the TMRL mode can also be changed.

[0187] For example, when it is necessary to construct a candidate prediction mode list with a length of 6, according to the same construction method and filling method as in this embodiment, first construct a list with a length greater than 6. Then, use the 4 rows and 4 columns closest to the current block as a template, predict the template using the 5th reference line and the intra prediction mode in the candidate prediction mode list, calculate the error (SAD or SATD) between the predicted value obtained from the prediction and the reconstructed value of the template, sort them in ascending order of the error, and select 6 intra prediction modes with small errors to be the intra prediction modes in the candidate list of the TMRL mode with a length of 6 to be constructed. Also, the length of the candidate list of the TMRL mode being 6 is just an example, and the numerical value can be adjusted according to the situation.

[0188] This embodiment has exemplified 65 types of angle modes. However, in other embodiments, it is possible to expand the angles to 129 types or more in order to obtain better performance. When expanding to more angles, it is also necessary to appropriately increase the number of filters for intra prediction. For example, in the case of 129 types of angles, filtering with 1 / 64 precision is used.

[0189] In one embodiment, an intra prediction fusion (IPF) technique is further provided. In IPF, it is permitted to weight the prediction results obtained using two adjacent reference lines in a certain angle mode to obtain the final prediction result of the current block. The method of weighting is as follows.

Equation

[0190] JPEG2025521765000075.jpg32170

[0191] When using IPF for the current block, the predicted result after the above fusion is used as the final predicted result of the current block.

[0192] In this embodiment, a flag indicating whether IPF is used (i.e., activated) is not set. By default, when the following conditions are met, that is, the angle mode selected for the current block is not an integer slope angle mode, the product of the width and height of the current block is greater than 16, the ISP mode is not selected for the current block, when the above conditions are met, IPF is activated for the angle mode selected for the current block.

[0193] In other words, when at least one of the following constraint conditions is satisfied, that is, the angle mode selected for the current block is an integer slope angle mode, the product of the width and height of the current block is 16 or less, the ISP mode is selected for the current block, when at least one of the above constraint conditions is satisfied, the use of IPF is not permitted.

[0194] Here, when the remainder of dividing the angle value (intraPredAngle) of the angle mode by 32 is 0, this angle mode is an integer slope angle mode, and the correspondence between intraPredAngle and the angle mode is shown in Table 1 above.

[0195] When the current mode meets the usage conditions of IPF, the intra prediction mode selected for the current block may be the DIMD fusion mode (fused based on the planar mode and two angle modes) selected using DIMD, or may also be the TIMD fusion mode selected using TIMD. From the perspective of hardware implementation, the less the fusion, the better. When the DIMD fusion mode is selected, three intra prediction results have already been used for the fusion of the current block. If IPF is further used, too much fusion will increase the complexity of the prediction stage.

[0196] One embodiment of the present application provides an intra prediction fusion method applicable to both an encoder and a decoder. As shown in FIG. 10, this method includes the following content. In step 410, when the selected intra prediction mode for the current block includes an angular mode, it is determined whether the constraint condition for using IPF for the current block is satisfied. Step 420 In, when at least one of the constraint conditions is satisfied, the use of IPF is restricted when performing intra prediction for the current block. In this step, restricting the use of IPF when performing intra prediction for the current block may mean not permitting the use of IPF. When the selected intra prediction mode includes a plurality of angular modes that satisfy the use conditions of IPF, it may also mean permitting IPF only for some of those angular modes. In one example, the angular mode that satisfies the use conditions of IPF refers to an angular mode that is not an angular mode with an integer slope, or an angular mode whose angle is other than -45°, 0°, 45°, 90°, 135°.

[0197] In this text, not permitting the use of IPF when a certain constraint condition is satisfied is a sufficient condition for not using IPF when predicting the current block. On the other hand, permitting the use of IPF when a certain constraint condition is not satisfied is a necessary condition for using IPF when predicting the current block. If other constraint conditions are satisfied, there may still be a possibility of not permitting the use of IPF.

[0198] In an exemplary embodiment of the present application, the constraint condition includes the following mode number constraint condition. The mode number constraint condition means that in order to predict the current block by using IPF, it is necessary to fuse N or more intra prediction modes, and N is an integer of 3 or more.

[0199] In an example of this embodiment, when N is 3, if the intra prediction mode selected for the current block includes the angular mode, it is determined whether the constraint condition for using IPF for the current block is satisfied. When at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block includes the following. When the TIMD fusion mode is selected for the current block and two angular modes that satisfy the usage conditions of IPF are used for TIMD fusion, it is determined that the mode number constraint condition is satisfied, and when performing intra prediction for the current block, it is permitted to perform IPF for only one of the two angular modes. When the TIMD fusion mode is selected for the current block and only one of the two intra prediction modes used for TIMD fusion is an angular mode that satisfies the usage conditions of IPF, it is determined that the mode number constraint condition is not satisfied, and when performing intra prediction for the current block, it is permitted to perform IPF for this angular mode.

[0200] In an exemplary embodiment of the present application, the constraint condition includes the following mode number constraint condition. The mode number constraint condition is that in order to predict the current block by using IPF, it is necessary to fuse M or more angular modes, and M is an integer of 2 or more.

[0201] In an example of this embodiment, when M is 3, if the intra prediction mode selected for the current block includes the angular mode, it is determined whether the constraint condition for using IPF for the current block is satisfied. When at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block includes the following. When the TIMD fusion mode is selected for the current block and two angular modes that satisfy the usage conditions of IPF are used for TIMD fusion, it is determined that the mode number constraint condition holds, and when performing intra prediction for the current block, it is permitted to perform IPF for only one of the two angular modes. For example, it is permitted to perform IPF only for the angular mode with the smallest cost or the second smallest cost among the two angular modes. When the TIMD fusion mode is selected for the current block and only one of the two intra prediction modes used for TIMD fusion is an angular mode that satisfies the usage conditions of IPF, it is determined that the mode number constraint condition does not hold, and when performing intra prediction for the current block, it is permitted to perform IPF for this angular mode.

[0202] In an example of this embodiment, when M is 3, if the intra prediction mode selected for the current block includes an angular mode, it is determined whether the constraint condition for using IPF for the current block holds. If at least one of the constraint conditions holds, restricting the use of IPF when performing intra prediction for the current block includes the following content. When the DIMD fusion mode is selected for the current block and two angular modes that satisfy the usage conditions of IPF are used for DIMD fusion, it is determined that the mode number constraint condition holds, and when performing intra prediction for the current block, it is permitted to perform IPF for only one of the two angular modes. For example, IPF is permitted only for the angular mode with the highest amplitude value or the second highest amplitude value among the two angular modes. When the DIMD fusion mode is selected for the current block and only one of the two angular modes used for DIMD fusion is an angular mode that satisfies the usage conditions of IPF, it is determined that the mode number constraint condition does not hold, and when performing intra prediction for the current block, it is permitted to perform IPF for the angular mode that satisfies the usage conditions of IPF.

[0203] In an example of this embodiment, when M is 2, if the intra prediction mode selected for the current block includes the angular mode, it is determined whether the constraint condition for using IPF for the current block is satisfied. If at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block includes the following. When the TIMD fusion mode is selected for the current block and two angular modes are used for TIMD fusion, it is determined that the mode number constraint condition is satisfied, and the use of IPF is not permitted when performing intra prediction for the current block. When the TIMD fusion mode is selected for the current block and only one of the two intra prediction modes used for TIMD fusion is the angular mode and this angular mode satisfies the IPF usage condition, it is determined that the mode number constraint condition is not satisfied, and it is permitted to perform IPF for the angular mode that satisfies the IPF usage condition when performing intra prediction for the current block.

[0204] In the above embodiment, permitting the execution of IPF for one angular mode when performing intra prediction for the current block includes using the weighted sum of two or more prediction results as the final prediction result of the current block. The two or more prediction results include the prediction result obtained by predicting the current block based on the first reference line selected for the current block and this angular mode, and the prediction result obtained by predicting the current block based on the second reference line different from the first reference line and this angular mode. For example, the second reference line may be a reference line adjacent to the first reference line or a reference line with index 0.

[0205] In an exemplary embodiment of the present application, the constraint condition includes any one or more of the following, that is, Constraint condition 1: The TIMD fusion mode is selected for the current block. Constraint condition 2: The DIMD fusion mode is selected for the current block. Constraint 3: The MRL is used for the current block. Constraint 4: The index of the reference line selected for the current block is K or more, and K is an integer of 3 or more. Constraint 5: The width of the current block is equal to or less than the set value. Constraint 6 : The height of the current block is equal to or less than the set value. Constraint 7: The product of the width and height of the current block is equal to or less than the set value. Constraint 8: The ISP mode is used for the current block. Constraint 9: The angle of the angle mode selected for the current block is one of -45°, 0°, 45°, 90°, 135°, or the angle mode selected for the current block is an angle mode with an integer inclination. Constraint 10: The current frame to which the current block belongs is an inter-frame. Constraint 11: The current image to which the current block belongs is a chroma image, that is, the IPF is used only for the luma image, and the IPF is not used for the chroma image. including any one or more of the following conditions, When at least one of Constraints 1 to 11 is satisfied, the use of IPF is not permitted when performing intra prediction for the current block.

[0206] In an example of this embodiment, the constraint condition includes Constraint 4, and K is 3, or 5, or 7, or 12.

[0207] In the above embodiment of the present application, the use of IPF is restricted. For example, in this embodiment, the number of intra prediction modes used for fusion can be restricted, the number of angle modes used for fusion can be restricted, and the simultaneous use of IPF and multiple prediction modes such as TIMD and DIMD, which may be fused, can be restricted. By these means, it is possible to avoid inappropriate increase in the complexity of the prediction stage due to excessive fusion during prediction.

[0208] In this embodiment, the size of the current block can be restricted, and IPF is used when the size of the current block is larger than the set value. This is because when the current block is smaller than a certain size, there are usually more textures, and the improvement in performance by using fusion prediction is limited. In this embodiment, when the index of the reference line selected for the current block is K or more, the use of IPF is not permitted, that is, when the extended reference line selected for the current block is relatively far from the current block, the use of IPF is not permitted, and in this case, the improvement in performance by using IPF is limited.

[0209] In this embodiment, by restricting the simultaneous use of IPF and MRL, the computational complexity can be reduced. In this embodiment, by not permitting the use of IPF when the current frame is an inter-frame such as a B frame or a P frame, the cost of inter-frame coding can be reduced.

[0210] In an exemplary embodiment of the present application, when not all of the constraint conditions are satisfied, the method further includes using IPF when performing intra prediction on the current block. Using IPF when performing intra prediction on the current block includes setting the weighted sum of the first prediction result and the second prediction result as the final prediction result of the current block when the intra prediction mode selected for the current block is only one and is the angular mode. The first prediction result is a prediction result obtained by predicting the current block based on the first reference line selected for the current block and the first angular mode selected for the current block, the second prediction result is a prediction result obtained by predicting the current block based on the second reference line and the first angular mode, and the second reference line is an adjacent line of the first reference line or a reference line whose index is 0.

[0211] If not all of the constraints are satisfied, IPF is used when performing intra prediction on the current block. This does not mean that the use of IPF is permitted only when not all of the constraints are satisfied. For example, even if the mode number constraint is satisfied, IPF can be executed based on some angular modes that satisfy the usage conditions of IPF.

[0212] In an example of this embodiment, the weighted sum of the first prediction result and the second prediction result is calculated according to Equation 4 or Equation 5.

Equation

Equation

[0213] The above calculation method can avoid the appearance of decimals during the operation and can improve the efficiency of the calculation method.

[0214] In an exemplary embodiment of the present application, the above method further includes using IPF when performing intra prediction on the current block when not all of the constraints are satisfied. Using IPF when performing intra prediction on the current block includes, when the intra prediction mode selected for the current block is a merge mode, using the weighted sum of three or more prediction results as the final prediction result of the current block. The three or more prediction results include a plurality of prediction results obtained by predicting the current block based on each of the first reference line selected for the current block and the intra prediction mode in the merge mode, and one or more prediction results obtained by predicting the current block based on one or more angular modes for which IPF is permitted in the merge mode and the second reference line. The second reference line is an adjacent line of the first reference line or a reference line with an index of 0.

[0215] In the above-described embodiment of the present application, when calculating the weighted sum of the first prediction result and the second prediction result, the weight given to the first prediction result is larger than the weight given to the second prediction result. For example, when the first reference line is the reference line with an index of 1 and the second reference line is the reference line with an index of 0, the weight of the first prediction result is set to 3 / 4, and the weight of the second prediction result is set to 1 / 4.

[0216] In the above-described embodiment of the present application, when the index of the first reference line is K or more, the second reference line is adjacent to the first reference line and closer to the current block than the first reference line, and K is an integer of 1 or more.

[0217] For example, the second reference line can be determined based on at least one of the following methods. When the index of the first reference line is 1, it is determined that the index of the second reference line is 0. When the index of the first reference line is 3, it is determined that the index of the second reference line is 2. When the index of the first reference line is 5, it is determined that the index of the second reference line is 4. When the index of the first reference line is 7, it is determined that the index of the second reference line is 6. When the index of the first reference line is 12, it is determined that the index of the second reference line is 11.

[0218] When the index of the first reference line is K or more, by setting the second reference line to a reference line adjacent to the first reference line and closer to the current block than the first reference line, the accuracy of the prediction of the current block based on the second reference line can be improved, and thus the accuracy of the prediction result after fusion can be improved.

[0219] In an exemplary embodiment of the present application, IPF can be used simultaneously with the TMRL mode. That is, when one extended reference line and one angular mode in the candidate list of the template-based multiple reference line & intra prediction (TMRL) mode for the current block are selected and not all of the constraint conditions are satisfied, a first prediction result obtained by predicting the current block based on this extended reference line and this angular mode, and a second prediction result obtained by predicting the current block based on another reference line and this angular mode are calculated, and the weighted sum of the first prediction result and the second prediction result is used as the final prediction result of the current block. The above-mentioned another reference line is a reference line with an index of 0 or an adjacent line of this extended reference line.

[0220] One embodiment of the present application further provides a video encoding method applied to an encoder. As shown in FIG. 11, this method includes the following contents. In step 510, determine the reference line and the intra prediction mode selected for the current block in mode selection. In step 520, predict the current block based on the intra prediction fusion method described in any embodiment of the present application to obtain a predicted value of the current block. In this step, when using IPF when performing intra prediction on the current block, obtain the predicted value of the current block based on the final prediction result of the current block. In step 530, determine the residual of the current block based on the original value and the predicted value of the current block.

[0221] In the video encoding method of this embodiment, by predicting the current block using the intra prediction fusion method of any embodiment of the present application, various effects of this intra prediction fusion method can be obtained.

[0222] One embodiment of the present application further provides a video decoding method applicable to a decoder. As shown in FIG. 12, this method includes the following content. In step 610, decode the bitstream to determine the selected reference line and the intra prediction mode for the current block. In step 620, predict the current block based on the intra prediction fusion method described in any embodiment of the present application to obtain a predicted value of the current block. In this step, when using IPF when performing intra prediction on the current block, obtain the predicted value of the current block based on the final prediction result of the current block. In step 630, determine the reconstructed value of the current block based on the predicted value of the current block.

[0223] The video of this embodiment Decoding In the method, by predicting the current block using the intra prediction fusion method of any embodiment of the present application, various effects of this intra prediction fusion method can be obtained.

[0224] The intra prediction method of the template-based multi-reference line intra prediction (TMRL) mode used in the above embodiment is a prediction mode that constructs a candidate list based on a combination of an extended reference line and an intra prediction mode, and encodes and decodes the combination of the extended reference line and the intra prediction mode. However, regardless of whether it is based on a template or whether it is ordered, prediction is performed using one reference line. Usually, one reference line contains noise, which affects the accuracy of prediction.

[0225] Therefore, one embodiment of the present application proposes a method of using IPF based on the TMRL mode. Using IPF based on the TMRL mode, that is, performing IPF for the selected angle mode, that is, weighting the result of predicting the current block based on the selected reference line and its angle mode and the result of predicting the current block based on another reference line and its angle mode to obtain the final prediction result of the current block. By fusing the prediction results of different reference lines, the accuracy of the prediction is improved.

[0226] When using IPF based on the TMRL mode, it is not necessary to set a flag indicating whether to use IPF for the current block, and the encoder and decoder can make a judgment based on the agreed conditions.

[0227] In one embodiment of the present application, when IPF can be used simultaneously with the TMRL mode, if one extended reference line and one angle mode in the candidate list of the TMRL mode are selected for the current block and not all of the constraint conditions are satisfied, the first prediction result of predicting the current block based on this extended reference line and this angle mode and the second prediction result of predicting the current block based on another reference line and this angle mode are calculated, and the weighted sum of the first prediction result and the second prediction result is used as the final prediction result of the current block. This other reference line is the reference line with an index of 0 or the adjacent line of this extended reference line.

[0228] In this embodiment, the method of the foregoing embodiment can be used both when constructing the candidate list of the TMRL mode and when encoding and decoding. The difference is that in this embodiment, at the stage of generating the predicted value of the current block, the predicted value generated based on the selected angle mode and the selected reference line and the predicted value generated based on the selected angle mode and another reference line are weighted to obtain the final predicted value of the current block.

[0229] In this embodiment, IPF based on the TMRL mode is restricted. For example, IPF is performed only when the selected reference line is one of several reference lines close to the current block. Specifically, fusion is permitted only when the reference line is reference line 1, or the reference lines are reference lines 1 and 3, or the reference lines are reference lines 1, 3, and 5, or the reference lines are reference lines 1, 3, 5, and 7, or the reference lines are reference lines 1, 3, 5, 7, and 12. Also, when a certain angle mode is used for the current block, it can be limited not to use IPF. For example, when the angle of the angle mode is a horizontal angle, a vertical angle, -45 degrees, 45 degrees, or 135 degrees, IPF cannot be used. Also, when the size of the current block is below a certain size, usually, smaller blocks have more textures, so it may not be necessary to permit the use of IPF, i.e., fusion prediction.

[0230] When using IPF based on the TMRL mode, in the syntax element of the TMRL mode, a flag indicating whether to use IPF can also be added.

[0231] One embodiment of the present application provides a video decoding method applied to a decoder. As shown in FIG. 13, this method includes steps 710 to 730. In step 710, when it is determined by decoding that a template-based multi-reference line intra prediction (TMRL) mode is used for the current block, the TMRL mode index and the TMRL fusion flag of the current block are continuously decoded. In step 720, a candidate list for the TMRL mode is constructed for the current block, and based on the candidate list and the TMRL mode index, the selected extended reference line and the intra prediction mode for the current block are determined. In this step, using the same method as in the foregoing embodiments, a candidate list can be constructed, and an extended reference line and an intra prediction mode selected for the current block can be determined. In step 730, when the TMRL fusion flag indicates that intra prediction fusion (IPF) is used, the weighted sum of the first prediction result and the second prediction result is taken as the final prediction result of the current block. In this step, the weighted sum of the first prediction result and the second prediction result can be calculated from the foregoing formulas 1 and 2.

[0232] The first prediction result is obtained by predicting the current block based on the extended reference line and the intra prediction mode, and the second prediction result is obtained by predicting the current block based on other reference lines and the intra prediction mode, and this intra prediction mode is an angular mode. When weighting the two prediction results, it can be calculated using the formulas of the foregoing embodiments.

[0233] In this embodiment, when the TMRL fusion flag indicates that IPF is not used, the current block is predicted based on the extended reference line and the intra prediction mode to obtain the final prediction result of the current block.

[0234] In this embodiment, based on the TMRL mode, by setting a TMRL fusion flag, it indicates whether to use IPF. On the encoding side, the TMRL fusion flag can be encoded based on whether the same or similar constraint conditions as those in the foregoing embodiments are satisfied. If it is determined to use IPF, the TMRL fusion flag is set to 1; if it is determined not to use IPF, the TMRL fusion flag is set to 0. On the other hand, on the decoding side, it is not necessary to make a judgment in combination with these constraint conditions. Based on this TMRL fusion flag, it is possible to determine whether to perform IPF on the angular mode selected when using the TMRL mode. The processing on the decoding side can be simplified, and the flexibility of setting the IPF usage conditions can be enhanced. By using IPF based on the TMRL mode, the prediction accuracy can be improved, and the performance of video encoding can be enhanced.

[0235] In an example of this embodiment, the above other reference lines are reference lines adjacent to and inside the extended reference line thereof, or reference lines adjacent to and outside the extended reference line thereof, or reference lines with an index of 0.

[0236] In an example of this embodiment, the conditions for decoding the TMRL fusion flag can be increased. That is, continuously decoding the TMRL mode index and the TMRL fusion flag of the current block includes, after decoding the TMRL mode index of the current block, determining whether the size of the current block satisfies the set conditions. If the set conditions are satisfied, decoding the TMRL fusion flag of the current block; if the set conditions are not satisfied, skipping the decoding of the TMRL fusion flag of the current block.

[0237] The set conditions include any one or more of the following, that is, the width of the current block is greater than the set value, the height of the current block is greater than the set value, The product of the width and height of the current block is greater than the set value. It is prohibited that the current frame to which the current block belongs is not an inter-frame, that is, in B-frames and P-frames, there is an intra-block using the merge mode. It includes one or more of the following conditions.

[0238] Generally, small blocks have more texture, and the improvement using IPF is limited. Therefore, in this embodiment, the size of the current block being larger than the set value is used as a condition for decoding the TMRL merge flag. If the size of the current block is not larger than the set value, it may not be necessary to encode and decode the TMRL merge flag to improve coding efficiency.

[0239] In an example of this embodiment, constructing a candidate list for the TMRL mode for the current block is obtaining N×M combinations of extended reference lines and intra prediction modes based on N extended reference lines and M intra prediction modes of the current block, where N≧1, M≧1, and N×M≧2 are satisfied, and predicting the template region of the current block based on each of the N×M combinations, and calculating the error between the reconstructed value of the template region and the predicted value obtained by the prediction, and entering K combinations corresponding to the errors in ascending order of the errors into the candidate list of the template-based multi-reference line intra prediction (TMRL) mode of the current block, where 1≦K≦N×M is satisfied, and The M intra prediction modes are selected from angle modes other than the angle modes with angles of -45°, 0°, 45°, 90°, and 135°.

[0240] In this embodiment, when constructing the candidate list in the TMRL mode, restricting the M intra prediction modes participating in the combination to other angular modes excluding some specific angles is advantageous for linking with the IPF and can improve the prediction accuracy.

[0241] One embodiment of the present application provides a video decoding method. At the stage of analyzing the syntax element at the CU level, it is necessary to additionally decode the flag tmrl_fusion_flag, that is, the TMRL fusion flag.

[0242] For example, it is as shown in the following table.

[0243]

Table 8

[0244] The sps_mrl_enabled_flag in the above table is a sequence-level flag. When it is 1, it indicates that MRL can be used for the current sequence. 0 When it is, it indicates not to use MRL. (y0 % CtbSizeY) > 0 indicates that the current CU is not located in the first row of the CTU. When tmrl_fusion_flag is 1, it indicates to use the fusion mode in the subsequent TMRL mode, and otherwise, not to use the fusion mode.

[0245] In another embodiment, whether to decode tmrl_fusion_flag can also be determined based on the size of the current coding block (Cb). For example, as shown in the following table, when the width cbWidth of the coding block > N and the height cbHeight of the coding block > M, tmrl_fusion_flag is decoded.

[0246]

Table 9

[0247] In yet another embodiment, for the coding block, when cbWidth * cbHeight > L is satisfied, the tmrl_fusion_flag is decoded as shown in the following table.

[0248]

Table 10

[0249] M, N, and L in the table are set values of sizes, all of which are positive integers, and M may be equal to N.

[0250] In this embodiment, when constructing the candidate list in the TMRL mode by ordering based on a template, for the N reference lines and M intra prediction modes that can be used, N×M combinations are traversed, the predicted value of the template region is generated, the error between the predicted value of the template region and the reconstructed value is obtained, the top K are selected in ascending order of the magnitude of the error, and a candidate list of {reference line, prediction mode} combinations is constructed. The error value can be, for example, SAD. In this embodiment, it is possible to restrict the use of IPF to only some modes. When constructing the candidate list, one or more of the intra prediction modes such as PLANAR, DC, the angular mode with a horizontal angle, the angular mode with a vertical angle, the angular mode with an angle of -45 degrees, the angular mode with an angle of 45 degrees, and the angular mode with an angle of 135 degrees can be excluded.

[0251] Fusion in the prediction stage needs to be based on the value of tmrl_fusion_flag. When tmrl_fusion_flag is 1, the prediction signal p a is generated by the selected intra prediction mode and the selected reference line, and the prediction signal p b is generated by the selected intra prediction mode and reference line reference line 0, and p a and p b are fused to generate the final prediction signal. Otherwise, pa Use it directly as the final prediction signal.

[0252] Embodiments of the present application further provide a video encoding method applied to an encoder. As shown in FIG. 18, the method includes the following content. In step 1110, construct a candidate list for the template-based multi-reference line intra prediction (TMRL) mode of the current block. The candidate list is filled with candidate combinations of extended reference lines and intra prediction modes for the current block. In step 1120, through rate-distortion optimization, select one combination of a reference line and an intra prediction mode for the current block. In step 1130, if the encoding conditions for the TMRL mode of the current block are satisfied, indicate the use of the TMRL mode for the current block by encoding the TMRL mode flag of the current block, and indicate the position of the selected combination in the candidate list by encoding the TMRL mode index of the current block. In step 1140, encode the TMRL fusion flag of the current block to indicate whether intra prediction fusion (IPF) is used for the current block. The encoding conditions include at least that the selected combination is in the candidate list.

[0253] The part of the video encoding method of this embodiment that is not related to fusion may be the same as the foregoing embodiment of the video encoding method that uses TMRL but does not perform fusion. For example, the setting of the encoding conditions in step 1130 may be the same.

[0254] In an example of this embodiment, encoding the TMRL fusion flag of the current block to indicate whether intra prediction fusion (IPF) is used for the current block includes: when a combination including an angular mode in the candidate list is selected for the current block, encoding the TMRL fusion flag of the current block to indicate that IPF is not used when at least one of the set constraints is satisfied, and encoding the TMRL fusion flag of the current block to indicate that IPF is used when all of the set constraints are not satisfied.

[0255] The set constraints further include any one or more of the following, that is, the TIMD fusion mode is selected for the current block, the DIMD mode is selected for the current block, the index of the reference line selected for the current block is K or more, where K is an integer of 3 or more, the width of the current block is less than or equal to a set value, the height of the current block is less than or equal to a set value, the product of the width and height of the current block is less than or equal to a set value, the intra sub - partition splitting (ISP) mode is used for the current block, the angle of the angular mode selected for the current block is one of - 45°, 0°, 45°, 90°, 135°, or the angular mode selected for the current block is an angular mode with an integer inclination, the current frame to which the current block belongs is an inter - frame, the current image to which the current block belongs is a color difference image, and further includes any one or more of the above conditions.

[0256] In an example of this embodiment, when constructing a candidate list of the TMRL mode for the current block, the intra prediction modes participating in the combination do not include the angular modes with angles of -45°, 0°, 45°, 90°, and 135°.

[0257] On the encoding side, when a combination within the candidate list of the TMRL mode is selected for the current block and the IPF is used, the current block is predicted using the IPF based on that combination, and the result based on that combination and the prediction result based on another reference line of the angular mode within that combination are weighted to obtain the predicted value of the current block. Consequently, the reconstructed value of the current block can be determined.

[0258] In order to use the IPF based on the TMRL, when constructing the candidate list of the TMRL, the fusion prediction mode obtained based on the combination of the extended reference line and the angular mode can also be set as one combination within the candidate list.

[0259] One embodiment of the present application provides a method for constructing a candidate list of the multi-reference line intra prediction mode, which can be applied to both an encoder and a decoder. As shown in FIG. 14, this method includes the following content. In step 810, based on N extended reference lines and M intra prediction modes of the current block, N×M original combinations of the extended reference line and the intra prediction mode are obtained. In step 820, the template area of the current block is predicted based on each of the N×M original combinations, and the error between the reconstructed value of the template area and the predicted value obtained by the prediction is calculated. In step 830, for each of the original combinations that include a pre-determined angle mode among the K original combinations with the smallest error, determine whether fusion is necessary. If fusion is necessary, enter the fusion combination corresponding to the original combination into the candidate list of the template-based multi-reference line intra prediction (TMRL) mode of the current block. If fusion is not necessary, enter the original combination into the candidate list. K, N, and M are positive integers that are set, and 1 ≤ K ≤ N × M is satisfied.

[0260] In this embodiment, for each of the original combinations that include a pre-determined angle mode among the K original combinations with the smallest error, determining whether fusion is necessary and, if fusion is necessary, entering the fusion combination corresponding to the original combination into the candidate list of the TMRL mode of the current block is including predicting the template area of the current block based on the fusion combination corresponding to the original combination, calculating the error between the reconstructed value of the template area and the predicted value obtained by prediction, and determining that fusion is necessary if the error corresponding to the original combination is greater than the error corresponding to the fusion combination, and determining that fusion is not necessary if the error corresponding to the original combination is less than or equal to the error corresponding to the fusion combination. When performing prediction on the current block based on the fusion combination, the weighted sum of the first prediction result and the second prediction result is used as the predicted value of the current block. The first prediction result is the prediction result obtained by predicting the current block based on the extended reference line and the angle mode in the original combination. The second prediction result is the prediction result obtained by predicting the current block based on the angle mode and the second reference line in the original combination. The second reference line is an adjacent line of the first reference line or a reference line with an index of 0. The pre-determined angle modes include all angle modes or angle modes other than the angle modes with integer slopes.

[0261] The weighted sum of the first prediction result and the second prediction result of this embodiment can be calculated according to the above formula one or formula two.

[0262] In this embodiment, the modes without fusion are sorted to select K combinations to construct a candidate list. Next, it is determined one by one whether each reference line in the K combinations needs to be fused. The determination method is based on the magnitude of the error between the prediction signal and the reconstructed signal generated in the fusion mode and the non-fusion mode according to the reference line in the currently selected combination. If the error is smaller when using the fusion mode, fusion is used for this combination; otherwise, fusion is not used. In this embodiment, it is only necessary to perform prediction and error value calculation on the template area based on (5×M + K) combinations.

[0263] Another embodiment of the present application provides a method for constructing a candidate list of a multi-reference line intra prediction mode that can be applied to either an encoder or a decoder. As shown in FIG. 15, this method includes the following content. In step 910, based on the N extended reference lines and M intra prediction modes of the current block, N×M original combinations of the extended reference lines and the intra prediction modes are obtained. In step 920, a fusion process is performed on the N×M original combinations. The fusion process includes replacing the original combination with the corresponding fusion combination when the set condition is satisfied for each of the original combinations including the pre-determined angular mode. In step 930, the template area of the current block is predicted based on each of the N×M combinations after the fusion process is performed, and the error between the reconstruction value of the template area and the predicted value obtained by the prediction is calculated. In step 940, enter K combinations corresponding to the errors in ascending order of error in the candidate list of the template-based multi-reference line intra prediction (TMRL) mode of the current block. K, N, and M are positive integers that are set, and 1 ≤ K ≤ N × M is satisfied. When performing a prediction on the current block based on the combined combination corresponding to the original combination, the weighted sum of the first prediction result and the second prediction result is used as the predicted value of the current block. The first prediction result is the prediction result obtained by predicting the current block based on the extended reference line and the angle mode in the original combination. The second prediction result is the prediction result obtained by predicting the current block based on the angle mode and the second reference line in the original combination. The second reference line is an adjacent line to the first reference line or a reference line with an index of 0.

[0264] In an example of this embodiment, the setting conditions include any one or more of the following conditions. The size of the current block is larger than N × M, where N and M are positive integers that are set. The intra prediction mode selected for the current block is not an angle mode with an integer slope. All pre-determined angle modes include all angle modes, or include angle modes other than the angle mode with an integer slope.

[0265] The weighted sum of the first prediction result and the second prediction result in this embodiment can be calculated according to the above formula one or formula two.

[0266] In this embodiment, based on the ordering of the error costs on the template, it is not necessary to order the fusion mode and the non-fusion mode for the reference lines simultaneously. When one or more of the following conditions are satisfied, use the fusion mode for adjacent reference lines to participate in the ordering for the template and generate a list of combinations {reference line, prediction mode}. Otherwise, do not use the fusion mode. The conditions are as follows. The size of the current block is larger than N×M. The current intra prediction mode is not PLANAR, DC, horizontal angle, vertical angle, -45 degrees, 45 degrees, or 135 degrees angle mode.

[0267] In the prediction stage, when using TMRL for the current block, based on the decoded tmrl_idx and the candidate list generated according to the template-based ordering, determine the selected reference line and prediction mode, and based on the same conditions, determine whether to generate a fused prediction result using adjacent reference lines. That is, perform IPF for the selected angle mode when one or more of the following conditions are met. The size of the current block is larger than N×M. The current intra prediction mode is not PLANAR, DC, horizontal angle, vertical angle, -45 degrees, 45 degrees, or 135 degrees angle mode.

[0268] Another embodiment of the present application provides a method for constructing a candidate list of a multi-reference line intra prediction mode that can be applied to both an encoder and a decoder. As shown in FIG. 16, this method includes the following content. In step 1010, based on the N extended reference lines and M intra prediction modes of the current block, obtain N×M original combinations of extended reference lines and angle modes. In step 1020, based on the original combinations including the pre-determined angle mode among the N×M original combinations, obtain the corresponding fused combinations. In step 1030, predict the template region of the current block based on each of the N×M original combinations and the obtained fused combinations, and calculate the error between the reconstructed value of the template region and the predicted value obtained by prediction. In step 1040, enter K combinations corresponding to the errors in ascending order of error into the candidate list of the template-based multi-reference line intra prediction (TMRL) mode of the current block. K, N, and M are set positive integers, and 1 ≤ K ≤ N × M is satisfied.

[0269] This embodiment includes obtaining a corresponding fusion combination based on an original combination including a pre-determined angle mode among N × M original combinations, which includes obtaining one fusion combination based on each of the original combinations including the pre-determined angle mode. When performing prediction on the current block based on the fusion combination, the weighted sum of the first prediction result and the second prediction result is used as the predicted value of the current block. The first prediction result is a prediction result obtained by predicting the current block based on the extended reference line and the angle mode in the original combination, and the second prediction result is a prediction result obtained by predicting the current block based on the second reference line and the angle mode in the original combination. The second reference line is an adjacent line of the first reference line or a reference line with an index of 0. The pre-determined angle mode includes all angle modes, or angle modes other than the angle modes with integer slopes.

[0270] In this embodiment, when all the intra prediction modes participating in the combination are angle modes, it is necessary to perform prediction and error value calculation on the template area based on 5 × M × 2 combinations.

[0271] The weighted sum of the first prediction result and the second prediction result of this embodiment can be calculated according to the above formula one or formula two.

[0272] In the above embodiment of the present application, it is exemplified that the reference lines participating in the combination are reference lines with indexes {1, 3, 5, 7, 12}. For example, if the original combination is a combination of a reference line with an index of 1 and a certain angle mode, the fused combination corresponding to the original combination can be a combination of two reference lines with indexes of 1 and 2 and that angle mode. When predicting the current block based on this fused combination, weights are assigned to the prediction results of the reference line with an index of 1 and its angle mode, and the prediction results of the reference line with an index of 2 and its angle mode to obtain the final prediction result of the current block. In another example, the inner adjacent reference line is selected as the second reference line, and then the corresponding fused combination can be a combination of two reference lines with indexes of 0 and 1 and its angle mode.

[0273] When applying the method for constructing the candidate list of the TMRL mode according to the above embodiment to video encoding and video decoding, if the fused combination in the candidate list is selected for the current block, a fused prediction is performed on the current block according to the prediction method of the fused combination, and if the original combination in the candidate list is selected, the current block is predicted according to the prediction method of the original combination, and IPF is not used.

[0274] One embodiment of the present application further provides a bitstream generated by the video encoding method described in any embodiment of the present application.

[0275] Furthermore, one embodiment of the present application further provides an apparatus for constructing a candidate list for the multi-reference line intra prediction mode. As shown in FIG. 17, the apparatus for constructing a candidate list for the multi-reference line intra prediction mode includes a processor 71 and a memory 73 storing a computer program. When the processor 71 executes the computer program, the method for constructing a candidate list for the multi-reference line intra prediction mode described in any one of the embodiments of this specification can be realized.

[0276] One embodiment of the present application further provides an intra prediction fusion apparatus. The intra prediction fusion apparatus includes a processor and a memory storing a computer program. When the processor executes the computer program, the intra prediction fusion method described in any one of the embodiments of the present application can be realized.

[0277] One embodiment of the present application further provides a video decoding apparatus. The video decoding apparatus includes a processor and a memory storing a computer program. When the processor executes the computer program, the video decoding method described in any one of the embodiments of the present application can be realized.

[0278] One embodiment of the present application further provides a video encoding apparatus. The video encoding apparatus includes a processor and a memory storing a computer program. When the processor executes the computer program, the video encoding method described in any one of the embodiments of the present application can be realized.

[0279] One embodiment of the present application further provides a video coding system. The video coding system includes the video encoding apparatus described in any one of the embodiments of the present application and the video decoding apparatus described in any one of the embodiments of the present application.

[0280] One embodiment of the present application further provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement the intra prediction fusion method described in any one of the embodiments of the present application, or can implement the method for constructing a candidate list of the multi-reference line intra prediction mode described in any one of the embodiments of the present application, or can implement the video decoding method described in any one of the embodiments of the present application, or can implement the video encoding method described in any one of the embodiments of the present application.

[0281] The processor in the above embodiment of the present application can be a general-purpose processor including a central processing unit (CPU), a network processor (NP), a microprocessor, etc., or can be other ordinary processors, etc. The processor can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), discrete logic or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or other equivalently integrated or discrete logic circuits, or can be a combination of the above components. That is, the processor in the above embodiment can be any processing component or combination of components that realizes various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. When the embodiment of the present application is implemented partially in software, the instructions used in the software can be stored in a suitable non-volatile computer-readable storage medium, and the method of the embodiment of the present application can be implemented by executing the instructions in hardware by one or more processors. The term "processor" as used herein can refer to the above-described structure or any other structure suitable for implementing the technology described herein.

[0282] In one or more of the above exemplary embodiments, the functions described can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, the functions can be stored on a computer-readable medium as one or more instructions or codes, or can be transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium includes computer-readable media such as tangible media like data storage media, or can include any communication media that facilitates the transmission of a computer program from one place to another, for example, in accordance with a communication protocol. Thus, the computer-readable medium can typically be a non-transitory tangible computer-readable storage medium or a communication medium such as a signal or a carrier. The data storage media can be any available medium that can be accessed by one or more computers or one or more processors to search for instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product can include a computer-readable medium.

[0283] By way of non-limiting example, such computer-readable storage media can include random access memory (RAM), read only memory (ROM), electrically erasable programmable ROM (EEPROM), compact disk ROM (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can store the desired program code in the form of instructions or data structures and is accessible by a computer. Also, any connection can be referred to as a computer-readable storage media. For example, when transmitting instructions from a website, server or other remote source using coaxial cable, fiber optic cable, twisted-pair cabling, digital subscriber line (DSL), or wireless technologies such as infrared, radio, microwave, etc., coaxial cable, fiber optic cable, twisted-pair cabling, DSL, or wireless technologies such as infrared, radio, microwave, etc. are included in the definition of the media. However, computer-readable storage media and data storage media do not include connections, carriers, signals, or other transient (ephemeral) media, and are non-transitory tangible storage media. As used herein, magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, or Blu-ray disks, etc. Magnetic disks typically reproduce data magnetically, and optical disks reproduce data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0284] In some aspects, the functions described herein can be provided within dedicated hardware and / or software modules configured to be used for encoding and decoding, or can also be incorporated into an integrated encoder-decoder. Also, the techniques described herein can be fully realized in one or more circuits or logic elements.

[0285] The technical solution of the embodiments of the present application can be implemented in various devices or apparatuses including a wireless mobile phone, an integrated circuit (IC), or a set of ICs (e.g., a chipset). In the embodiments of the present application, various components, modules, or units are used to emphasize the functions of a device configured to execute the described techniques. It is not necessarily realized by different hardware units. As described above, the various units may be combined in hardware units such as an encoder and a decoder, or may be provided in combination with a collection of interoperable hardware units (including one or more of the processors described above) and appropriate software and / or firmware.

Claims

1. An intra prediction fusion (IPF) method, comprising: when the selected intra prediction mode for the current block includes an angular mode, determining whether a constraint condition for using IPF for the current block is satisfied; when at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block; including an intra prediction fusion method characterized by the above.

2. The constraint condition includes a mode number constraint condition, and the mode number constraint condition means that it is necessary to fuse N or more intra prediction modes to predict the current block by using IPF, where N is an integer of 3 or more. The method according to claim 1, characterized by the above.

3. When N = 3, when the selected intra prediction mode for the current block includes an angular mode, determining whether a constraint condition for using IPF for the current block is satisfied; when at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block includes: when a template-based intra mode derivation (TIMD) fusion mode is selected for the current block and two angular modes that satisfy the usage conditions of IPF are used for TIMD fusion, determining that the mode number constraint condition is satisfied, and when performing intra prediction for the current block, permitting the execution of IPF for only one of the two angular modes; when a TIMD fusion mode is selected for the current block and only one of the two intra prediction modes used for TIMD fusion is an angular mode that satisfies the usage conditions of IPF, determining that the mode number constraint condition is not satisfied, and when performing intra prediction for the current block, permitting the execution of IPF for the angular mode; including The method according to claim 2, characterized by the above.

4. The constraint condition includes a mode number constraint condition, and the mode number constraint condition means that it is necessary to fuse M or more angular modes to predict the current block by using IPF, where M is an integer of 2 or more. The method according to claim 1, characterized in that...

5. When M is 3, if the intra prediction mode selected for the current block includes the angular mode, determine whether the constraint condition for using IPF for the current block is satisfied. If at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block is... When the TIMD fusion mode is selected for the current block and two angular modes that satisfy the usage conditions of IPF are used for TIMD fusion, determine that the mode number constraint condition is satisfied, and permit performing IPF for only one of the two angular modes when performing intra prediction for the current block; When the TIMD fusion mode is selected for the current block and only one of the two intra prediction modes used for TIMD fusion is an angular mode that satisfies the usage conditions of IPF, determine that the mode number constraint condition is not satisfied, and permit performing IPF for the angular mode when performing intra prediction for the current block; including The method according to claim 4, characterized in that...

6. Permitting performing IPF for only one of the two angular modes includes permitting performing IPF for only the angular mode with the smallest cost or the second smallest cost among the two angular modes. The method according to claim 3 or 5, characterized in that...

7. When M is 3, if the intra prediction mode selected for the current block includes the angular mode, determine whether the constraint condition for using IPF for the current block is satisfied. If at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction for the current block is... When a decoder-side intra mode derivation (DIMD) fusion mode is selected for the current block and two angular modes that satisfy the usage conditions of IPF are used for DIMD fusion, it is determined that the mode number constraint condition is satisfied, and when performing intra prediction on the current block, it is permitted to perform IPF only on one of the two angular modes. When a DIMD fusion mode is selected for the current block and only one of the two angular modes used for DIMD fusion is an angular mode that satisfies the usage conditions of IPF, it is determined that the mode number constraint condition is not satisfied, and when performing intra prediction on the current block, it is permitted to perform IPF on the angular mode that satisfies the usage conditions of IPF. including The method according to claim 4, characterized in that.

8. When M is 2, if the selected intra prediction mode for the current block includes an angular mode, it is determined whether the constraint condition for using IPF for the current block is satisfied. When at least one of the constraint conditions is satisfied, restricting the use of IPF when performing intra prediction on the current block is When a TIMD fusion mode is selected for the current block and two angular modes are used for TIMD fusion, it is determined that the mode number constraint condition is satisfied, and the use of IPF is not permitted when performing intra prediction on the current block. When a TIMD fusion mode is selected for the current block and only one of the two intra prediction modes used for TIMD fusion is an angular mode and the angular mode satisfies the usage conditions of IPF, it is determined that the mode number constraint condition is not satisfied, and when performing intra prediction on the current block, it is permitted to perform IPF on the angular mode that satisfies the usage conditions of IPF. including The method according to claim 4, characterized in that.

9. The angular mode that satisfies the usage conditions of IPF is not an angular mode with an integer slope. The method according to claim 3, 5, 7 or 8, characterized in that.

10. Permitting execution of IPF for one angular mode when performing intra prediction on the aforementioned current block includes using a weighted sum of two or more prediction results as the final prediction result of the current block. The two or more prediction results include a prediction result obtained by predicting the current block based on a first reference line selected for the current block and the angular mode, and a prediction result obtained by predicting the current block based on a second reference line different from the first reference line and the angular mode. The method according to claim 3, 5, 7 or 8, characterized in that.

11. The constraint conditions include any one or more of the following, that is, Constraint condition 1: The TIMD fusion mode is selected for the current block. Constraint condition 2: The DIMD fusion mode is selected for the current block. Constraint condition 3: A multiple reference line (MRL) is used for the current block. Constraint condition 4: The index of the reference line selected for the current block is K or more, and K is an integer of 3 or more. Constraint condition 5: The width of the current block is less than or equal to a set value. Constraint condition 5: The height of the current block is less than or equal to a set value. Constraint condition 7: The product of the width and height of the current block is less than or equal to a set value. Constraint condition 8: The intra sub - partitions (ISP) mode is used for the current block. Constraint condition 9: The angle of the angular mode selected for the current block is any one of - 45°, 0°, 45°, 90°, 135°, or the angular mode selected for the current block is an angular mode with an integer inclination. Constraint condition 10: The current frame to which the current block belongs is an inter - frame. Constraint condition 11: The current image to which the current block belongs is not a luminance image. including any one or more of the above conditions. When at least one of the constraint conditions 1 to 11 is satisfied, the use of IPF is not permitted when performing intra prediction on the current block. The method according to claim 1, characterized in that.

12. The constraint condition includes constraint condition 4, and K is 3, or 5, or 7, or 12. The method according to claim 11, characterized in that.

13. The method further includes using IPF when performing intra prediction on the current block when not all of the constraint conditions are satisfied, The above-mentioned use of IPF when performing intra prediction on the current block means that when the selected intra prediction mode for the current block is only one and is an angular mode, taking the weighted sum of the first prediction result and the second prediction result as the final prediction result of the current block, The first prediction result is a prediction result obtained by predicting the current block based on the first reference line selected for the current block and the first angular mode selected for the current block, the second prediction result is a prediction result obtained by predicting the current block based on the second reference line and the first angular mode, and the second reference line is an adjacent line of the first reference line or a reference line with an index of 0. The method according to claim 1, characterized in that.

14. The method further includes using IPF when performing intra prediction on the current block when not all of the constraint conditions are satisfied, The above-mentioned use of IPF when performing intra prediction on the current block means that when the selected intra prediction mode for the current block is a fusion mode, taking the weighted sum of three or more prediction results as the final prediction result of the current block, The three or more prediction results include a plurality of prediction results obtained by predicting the current block based on the first reference line selected for the current block and each of the intra prediction modes in the fusion mode, and one or more prediction results obtained by predicting the current block based on one or more angular modes permitted by the IPF in the fusion mode and the second reference line, The second reference line is an adjacent line of the first reference line or a reference line with an index of 0. The method according to claim 1, characterized in that.

15. When the index of the first reference line is K or more, the second reference line is adjacent to the first reference line and closer to the current block than the first reference line, and K is an integer of 1 or more. The method according to claim 13 or 14, characterized in that...

16. When calculating the weighted sum of the first prediction result and the second prediction result, the weight given to the first prediction result is greater than the weight given to the second prediction result. The method according to claim 13 or 14, characterized in that...

17.

18. When the intra prediction mode selected for the current block includes the angular mode, determine whether the constraint conditions for using intra prediction fusion (IPF) for the current block are satisfied. If not all of the constraint conditions are satisfied, using IPF when performing intra prediction for the current block... One extended reference line and one angular mode are selected from the candidate list of the template-based multiple reference line & intra intra prediction (TMRLL) mode for the current block, and if not all of the constraint conditions are satisfied, calculate a first prediction result obtained by predicting the current block based on the extended reference line and the angular mode, and a second prediction result obtained by predicting the current block based on another reference line and the angular mode, and use the weighted sum of the first prediction result and the second prediction result as the final prediction result of the current block, where... The other reference line is a reference line with an index of 0 or an adjacent line of the extended reference line. The method according to claim 13, characterized in that...

19. A video decoding method, comprising: Decoding a bitstream to determine a reference line and an intra prediction mode selected for the current block; Predicting the current block based on the intra prediction fusion method according to any one of claims 1 to 18 to obtain a predicted value of the current block; Determining a reconstructed value of the current block based on the predicted value of the current block; Including A video decoding method, characterized in that...

20. A video encoding method, comprising: Determining a reference line and an intra prediction mode selected for the current block in mode selection; Predicting the current block based on the intra prediction fusion method according to any one of claims 1 to 18 to obtain a predicted value of the current block; Determining a residual of a current block based on an original value and a predicted value of the current block; including A video encoding method characterized by the above.

21. A video decoding method, comprising: When it is determined by decoding that a template-based multi-reference line intra prediction (TMPL) mode is used for a current block, subsequently decoding a TMPL mode index and a TMPL fusion flag of the current block; Constructing a candidate list for the TMPL mode for the current block, and determining an extended reference line and an intra prediction mode selected for the current block based on the candidate list and the TMPL mode index; When the TMPL fusion flag indicates that intra prediction fusion (IPF) is used, taking a weighted sum of a first prediction result and a second prediction result as a final prediction result of the current block; The first prediction result is obtained by predicting the current block based on the extended reference line and the intra prediction mode, and the second prediction result is obtained by predicting the current block based on another reference line and the intra prediction mode, and this intra prediction mode is an angular mode. A video decoding method characterized by the above.

22. The other reference line is a reference line adjacent to and inside the extended reference line, or a reference line adjacent to and outside the extended reference line, or a reference line with an index of 0. The method according to claim 21, characterized by the above.

23. The method further includes, when the TMPL fusion flag indicates that IPF is not used, predicting the current block based on the extended reference line and the intra prediction mode to obtain a final prediction result of the current block. The method according to claim 21, characterized by the above.

24. Decoding the aforementioned TMRL mode index and TMRL fusion flag of the current block continuously includes, after decoding the TMRL mode index of the current block, determining whether the size of the current block meets the set conditions. If the set conditions are met, decoding the TMRL fusion flag of the current block; if the set conditions are not met, skipping the decoding of the TMRL fusion flag of the current block. The set conditions include any one or more of the following, that is, the width of the current block is greater than the set value; the height of the current block is greater than the set value; the product of the width and height of the current block is greater than the set value; the current frame to which the current block belongs is not an inter-frame; including any one or more of the above conditions, The method according to claim 21, characterized in that.

25. Constructing a candidate list for the TMRL mode for the current block as described above includes obtaining N×M combinations of extended reference lines and intra prediction modes based on N extended reference lines and M intra prediction modes of the current block, where N≥1, M≥1, and N×M≥2 are satisfied; predicting the template area of the current block based on each of the N×M combinations, and calculating the error between the reconstructed value of the template area and the predicted value obtained by prediction; entering K combinations corresponding to the errors in ascending order of the errors into the candidate list of the template-based multi-reference line intra prediction (TMRL) mode for the current block, where 1≤K≤N×M is satisfied; The M intra prediction modes are selected from angle modes other than the angle modes with angles of -45°, 0°, 45°, 90°, and 135°. The method according to claim 21, characterized in that.

26.

27. Constructing a candidate list for the template-based multi-reference line intra prediction (TMRL) mode of the current block, where the candidate list is filled with candidate combinations of extended reference lines and intra prediction modes for the current block; selecting one combination of a reference line and an intra prediction mode for the current block through rate distortion optimization; When the encoding conditions of the TMRL mode of the current block are satisfied, use the TMRL mode for the current block by encoding the TMRL mode flag of the current block, and indicate the position of the selected combination in the candidate list by encoding the TMRL mode index of the current block, Encode the TMRL merge flag of the current block to indicate whether intra prediction merge (IPF) is used for the current block, A video encoding method including The encoding conditions include at least that the selected combination is in the candidate list, A video encoding method characterized by the above.

28. Encoding the TMRL merge flag of the current block as described above to indicate whether intra prediction merge (IPF) is used for the current block means For the current block, when a combination including an angular mode in the candidate list is selected, when at least one of the set constraint conditions is satisfied, encode the TMRL merge flag of the current block to indicate that IPF is not used, and when all of the set constraint conditions are not satisfied, encode the TMRL merge flag of the current block to indicate that IPF is used, including The set constraint conditions further include any one or more of the following, that is, The TIMD merge mode is selected for the current block, The DIMD mode is selected for the current block, The index of the reference line selected for the current block is K or more, and K is an integer of 3 or more, The width of the current block is equal to or less than a set value, The height of the current block is equal to or less than a set value, The product of the width and height of the current block is equal to or less than a set value, The intra subpartition split (ISP) mode is used for the current block, The angle of the angular mode selected for the current block is any one of -45°, 0°, 45°, 90°, 135°, or the angular mode selected for the current block is an angular mode with an integer inclination, The current frame to which the current block belongs is an interframe, The current image to which the current block belongs is a chrominance difference image, further comprising any one or more of the following conditions: The method according to claim 27, characterized in that.

29. When constructing the candidate list of the TMRL mode of the current block, the intra prediction modes participating in the combination do not include the angular modes with angles of -45°, 0°, 45°, 90°, and 135°. The method according to claim 27, characterized in that.

30. A method for constructing a candidate list of the multi-reference line (MRL) intra prediction mode, comprising: Based on N extended reference lines and M intra prediction modes of the current block, obtaining N×M original combinations of the extended reference lines and the angular modes; Based on the original combinations including the pre-determined angular modes among the N×M original combinations, obtaining the corresponding fusion combinations; Predicting the template region of the current block based on each of the N×M original combinations and the obtained fusion combinations, and calculating the error between the reconstruction value of the template region and the predicted value obtained by prediction; Entering K combinations corresponding to the errors in ascending order of the errors into the candidate list of the template-based multi-reference line intra prediction (TMRL) mode of the current block, where K, N, and M are set positive integers and 1≦K≦N×M is satisfied; including A method for constructing a candidate list of the multi-reference line intra prediction mode, characterized in that.

31. The obtaining of the corresponding fusion combinations based on the original combinations including the pre-determined angular modes among the N×M original combinations described above is: including obtaining one fusion combination based on each of the original combinations including the pre-determined angular modes. When performing a prediction on the current block based on the fusion combination, the weighted sum of the first prediction result and the second prediction result is used as the predicted value of the current block. The first prediction result is a prediction result obtained by predicting the current block based on the extended reference line and the angle mode in the original combination. The second prediction result is a prediction result obtained by predicting the current block based on the second reference line and the angle mode in the original combination. The second reference line is an adjacent line of the first reference line or a reference line with an index of 0. The pre-determined angle modes include all angle modes, or angle modes other than those with an integer slope. The method according to claim 30, characterized in that.

32.

33. A method for constructing a candidate list of the multi-reference line (MRL) intra prediction mode, comprising: Based on N extended reference lines of the current block and M intra prediction modes, obtaining N×M original combinations of the extended reference lines and the intra prediction modes; Predicting the template region of the current block based on each of the N×M original combinations, and calculating the error between the reconstruction value of the template region and the predicted value obtained by the prediction; For each of the original combinations including the pre-determined angle modes in the K original combinations with the smallest error, determining whether fusion is required. If fusion is required, entering the fusion combination corresponding to the original combination into the candidate list of the template-based multi-reference line intra prediction (TMLR) mode of the current block. If fusion is not required, entering the original combination into the candidate list, where K, N, and M are positive integers set, and 1≤K≤N×M is satisfied. Including A method for constructing a candidate list of the multi-reference line intra prediction mode, characterized in that.

34. For each of the original combinations of the K original combinations with the smallest error as described above, which include a pre-determined angle mode, determine whether fusion is necessary. If fusion is necessary, enter the fusion combination corresponding to the original combination into the candidate list of the TMRL mode of the current block. Predict the template area of the current block based on the fusion combination corresponding to the original combination, calculate the error between the reconstructed value of the template area and the predicted value obtained by prediction. If the error corresponding to the original combination is greater than the error corresponding to the fusion combination, determine that fusion is necessary. If the error corresponding to the original combination is less than or equal to the error corresponding to the fusion combination, determine that fusion is not necessary. This includes When performing a prediction on the current block based on the fusion combination, use the weighted sum of the first prediction result and the second prediction result as the predicted value of the current block. The first prediction result is the prediction result obtained by predicting the current block based on the extended reference line and the angle mode in the original combination. The second prediction result is the prediction result obtained by predicting the current block based on the angle mode and the second reference line in the original combination. The second reference line is the adjacent line of the first reference line or the reference line with an index of 0. The pre-determined angle modes include all angle modes, or angle modes other than integer-slope angle modes. The method according to claim 33, characterized in that.

35.

36. Based on the N extended reference lines and M intra prediction modes of the current block, obtain N×M original combinations of the extended reference lines and the intra prediction modes. Performing a fusion process on the N×M original combinations, where the fusion process includes replacing the original combination with the corresponding fusion combination when the set conditions are met for each of the original combinations including the pre-determined angle mode. Predicting a template region of a current block based on each of N×M combinations for which a fusion process has been performed, and calculating an error between a reconstruction value of the template region and a predicted value obtained by prediction; Entering K combinations corresponding to the error in ascending order of the error in a candidate list for a template-based multi-reference line intra prediction (TMRL) mode of the current block; A method for constructing a candidate list for a multi-reference line (MRL) intra prediction mode, including: K, N, and M are positive integers set, and 1≤K≤N×M is satisfied; When performing prediction on a current block based on a fusion combination corresponding to the original combination, a weighted sum of a first prediction result and a second prediction result is used as a predicted value of the current block, where the first prediction result is a prediction result obtained by predicting the current block based on an extended reference line and an angle mode in the original combination, and the second prediction result is a prediction result obtained by predicting the current block based on an angle mode and a second reference line in the original combination, and the second reference line is an adjacent line of the first reference line or a reference line with an index of 0; A method for constructing a candidate list for a multi-reference line intra prediction mode, characterized by the above.

37. The setting conditions are: The size of the current block is larger than N×M, and N and M are positive integers set; The intra prediction mode selected for the current block is not an angle mode with an integer slope; All the pre-determined angle modes include all angle modes, or include angle modes other than the angle mode with an integer slope; Including any one or more of the above conditions; The method according to claim 36, characterized by the above.

38.

39. A bitstream, The bitstream is generated by the video encoding method according to any one of claims 20, 27 to 29; A bitstream, characterized by the above.

40. An intra prediction fusion device, Including a processor and a memory storing a computer program; When the processor executes the computer program, the intra prediction fusion method according to any one of claims 1 to 18 can be realized; An intra prediction fusion device, characterized by the above.

41. An apparatus for constructing a candidate list for a multi-reference line (MRL) intra prediction mode, comprising: a processor and a memory storing a computer program, wherein when the processor executes the computer program, the method for constructing a candidate list for the MRL intra prediction mode according to any one of Claims 30 to 38 can be realized, characterized in that it is an apparatus for constructing a candidate list for a multi-reference line intra prediction mode.

42. A video decoding apparatus, comprising: a processor and a memory storing a computer program, wherein when the processor executes the computer program, the video decoding method according to any one of Claims 19, 21 to 26 can be realized, characterized in that it is a video decoding apparatus.

43. A video encoding apparatus, comprising: a processor and a memory storing a computer program, wherein when the processor executes the computer program, the video encoding method according to any one of Claims 20, 27 to 29 can be realized, characterized in that it is a video encoding apparatus.

44. A video coding system, comprising: the video encoding apparatus according to Claim 36 and the video decoding apparatus according to Claim 35, characterized in that it is a video coding system.

45. A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the intra prediction fusion method according to any one of Claims 1 to 18 can be realized, or the video decoding method according to any one of Claims 19, 21 to 26 can be realized, or the video encoding method according to any one of Claims 20, 27 to 29 can be realized, or the method for constructing a candidate list for a multi-reference line (MRL) intra prediction mode according to any one of Claims 30 to 38 can be realized, characterized in that it is a non-transitory computer-readable storage medium.

Images