Encoding method and device, decoding method and device, encoder, decoder, bitstream, storage medium

Abstract

Embodiments of the present invention disclose an encoding method and apparatus, a decoding method and apparatus, an encoder, a decoder, a bitstream, and a storage medium, wherein the encoding method in the method is applied to an encoder, and the method includes: determining a template for a current block and one or more prediction templates for the template of the current block; determining weights of the one or more prediction templates based on the template for the current block and the one or more prediction templates; determining one or more first prediction blocks for the current block based on prediction parameters of the current block; and fusing the one or more first prediction blocks based on the weights of the one or more prediction templates to obtain a second prediction block for the current block.

Description

[Technical Field]

[0001] Embodiments of the present invention relate to video encoding and decoding techniques, including, but not limited to, encoding methods and apparatuses, decoding methods and apparatuses, encoders, decoders, bitstreams, storage media, and the like. [Background technology]

[0002] In the field of video coding and decoding, how to improve the video compression efficiency is of great importance. In the process of digitizing images and videos, a large amount of data redundancy occurs, which makes video compression technology possible. sample There is a strong spatial correlation between the current block and the already coded and decoded blocks. sample and inside the current block sample Inter-prediction is a prediction method based on spatial correlation between adjacent frames, which reduces spatial redundancy in video coding based on the prediction results. Since there is also strong similarity between adjacent frames in a video, inter-prediction methods are used in video coding and decoding technology to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0003] Therefore, research to further improve the accuracy of intra- and inter-prediction is beneficial to improving video encoding and decoding performance. Summary of the Invention [Problem to be solved by the invention]

[0004] The encoding method and apparatus, decoding method and apparatus, encoder, decoder, bitstream, and storage medium according to embodiments of the present invention can improve the accuracy of intra-prediction and / or inter-prediction, thereby improving video encoding and decoding performance. [Means for solving the problem]

[0005] The encoding method and apparatus, decoding method and apparatus, encoder, decoder, bitstream, and storage medium according to the embodiments of the present invention are realized as follows.

[0006] According to one aspect of an embodiment of the present invention, there is provided an encoding method applicable to an encoder, the method including: determining a template of a current block and one or more prediction templates for the template of the current block; determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; determining one or more first prediction blocks of the current block based on prediction parameters of the current block; and fusing the one or more first prediction blocks based on the weights of the one or more prediction templates to obtain a second prediction block of the current block.

[0007] According to another aspect of an embodiment of the present invention, there is provided a decoding method applicable to a decoder, the method including: determining a template of a current block and one or more prediction templates for the template of the current block; determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; determining one or more first prediction blocks of the current block based on prediction parameters of the current block; fusing the one or more first prediction blocks based on the weights of the one or more prediction templates to obtain second prediction blocks of the current block; and determining a reconstructed value of the current block based on the second prediction blocks.

[0008] According to one aspect of an embodiment of the present invention, there is provided an encoding device applicable to an encoder, the device including: a first determination module configured to determine a template for a current block and one or more prediction templates for the template for the current block; a second determination module configured to determine weights of the one or more prediction templates based on the template for the current block and the one or more prediction templates; a third determination module configured to determine one or more first prediction blocks for the current block based on prediction parameters of the current block; and a first fusion module configured to fuse the one or more first prediction blocks based on the weights of the one or more prediction templates to obtain a second prediction block for the current block.

[0009] According to one aspect of an embodiment of the present invention, an encoder is provided, the encoder including a first memory and a first processor, wherein the first memory is used to store a computer program executable on the first processor, and the first processor is used to execute a method according to an embodiment of the present invention when the computer program is executed.

[0010] According to an aspect of an embodiment of the present invention, there is provided a decoding device applicable to a decoder, the device including: a fourth determination module configured to determine a template for a current block and one or more prediction templates for the template of the current block; a fifth determination module configured to determine weights of the one or more prediction templates based on the template for the current block and the one or more prediction templates; a sixth determination module configured to determine one or more first prediction blocks for the current block based on prediction parameters of the current block; a second fusion module configured to fuse the one or more first prediction blocks based on weights of the one or more prediction templates to obtain a second prediction block for the current block; and a seventh determination module configured to determine a reconstructed value of the current block based on the second prediction blocks.

[0011] According to one aspect of an embodiment of the present invention, a decoder is provided, the decoder including a second memory and a second processor, wherein the second memory is used to store a computer program operable on the second processor, and the second processor, when running the computer program, is used to execute the decoding method described in the embodiment of the present invention.

[0012] According to one aspect of an embodiment of the present invention, a bitstream is provided, the bitstream being generated by a residual block determined based on a second predicted block of a current block, wherein the second predicted block is obtained by the encoding method.

[0013] According to one aspect of an embodiment of the present invention, there is provided an electronic device including a processor suitable for executing a computer program and a computer-readable storage medium, the computer program storing thereon being configured to implement a method according to an embodiment of the present invention when executed by the processor.

[0014] According to one aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored that, when executed, implements a method according to an embodiment of the present invention.

[0015] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. [Brief explanation of the drawings]

[0016] The drawings herein are incorporated into the specification and constitute a part of this specification, and these drawings illustrate embodiments suitable for the present invention, and are used together with the description to explain the technical solutions of the present invention. The drawings in the following description are merely some embodiments of the present invention, and it is obvious that those skilled in the art can derive other drawings based on these drawings without performing any creative work.

[0017] The flowcharts shown in the drawings are merely illustrative and do not necessarily include all the contents or operations / steps, and do not necessarily have to be performed in the order described. For example, some operations / steps can be further decomposed, and some operations / steps can be integrated or partially integrated, and may be changed according to actual circumstances.

[0018] [Figure 1] FIG. 1 is a schematic diagram of the basic flow of a video encoder. [Figure 2] FIG. 1 is a schematic diagram of the basic flow of a video decoder. [Figure 3] 1 is a schematic diagram of a network architecture of an encoding / decoding system according to an embodiment of the present invention; [Figure 4] FIG. 2 is a schematic diagram illustrating the implementation flow of an encoding method according to an embodiment of the present invention. [Figure 5] 3 is a schematic diagram of a template of a current block and a reference area of ​​the template of the current block according to an embodiment of the present invention; FIG. [Figure 6]FIG. 1 is a schematic diagram of an implementation flow of a method for determining a prediction template according to an embodiment of the present invention. [Figure 7] 1 is a schematic diagram of the HoG principle according to an embodiment of the present invention; [Figure 8] FIG. 10 is a schematic diagram of a histogram according to an embodiment of the present invention. [Figure 9] FIG. 1 is a schematic diagram of an implementation flow of a method for determining a prediction template according to an embodiment of the present invention. [Figure 10A] FIG. 1 is a schematic diagram of an implementation flow of a method for determining a prediction template according to an embodiment of the present invention. [Figure 10B] FIG. 10 is a schematic diagram of a reference row of a template of a current block according to an embodiment of the present invention; [Figure 11] FIG. 1 is a schematic diagram of an implementation flow of a method for determining a prediction template according to an embodiment of the present invention. [Figure 12] FIG. 2 is a schematic diagram illustrating an implementation flow of a decoding method according to an embodiment of the present invention. [Figure 13] FIG. 1 is a schematic diagram of the prediction flow of DIMD. [Figure 14] FIG. 1 is a schematic diagram of the prediction flow of TIMD. [Figure 15] Schematic diagram of the TMRL prediction flow. [Figure 16] FIG. 10 is a schematic diagram of an inter unidirectional weighted prediction flow. [Figure 17] FIG. 1 is a schematic diagram of a bidirectional weighted prediction flow. [Figure 18] FIG. 1 is a schematic diagram of an inter-weighted prediction flow. [Figure 19] FIG. 1 is a schematic diagram of a minMSE-based DIPF prediction flow according to an embodiment of the present invention. [Figure 20] FIG. 10 is a schematic diagram of a specific implementation flow of step 1904 according to an embodiment of the present invention. [Figure 21] FIG. 1 is a schematic diagram of a TIPF prediction flow according to an embodiment of the present invention. [Figure 22] FIG. 2 is a schematic diagram of a specific implementation flow of step 2104 according to an embodiment of the present invention. [Figure 23]FIG. 2 is a schematic diagram of adjacent blocks of a current block; [Figure 24] FIG. 2 is a schematic diagram of a TMRLF prediction flow according to an embodiment of the present invention. [Figure 25] FIG. 1 is a schematic diagram of a prediction flow of IWPF according to an embodiment of the present invention. [Figure 26] 1 is a schematic diagram of a neighboring template curT of a current block and a neighboring template refT of a reference block; [Figure 27] 1 is a structural schematic diagram of an encoding device according to an embodiment of the present invention; [Figure 28] 1 is a structural schematic diagram of a decoding device according to an embodiment of the present invention; [Figure 29] 1 is a structural schematic diagram of an encoder according to an embodiment of the present invention; [Figure 30] FIG. 2 is a structural schematic diagram of a decoder according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0019] In order to clarify the objectives, technical solutions and advantages of the embodiments of the present invention, the specific technical solutions of the present invention will be described in more detail below with reference to the drawings in the embodiments of the present invention. The following examples are for illustrating the present invention but are not intended to limit the scope of the present invention.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0021] In the following description, "some embodiments," "the present embodiments," "embodiments of the present invention," and cited examples describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same or different subsets of all possible embodiments, and may be combined with each other as long as they do not conflict.

[0022] The terms "first, second, third" and the like appearing in the embodiments of the present invention are merely intended to indicate and distinguish the objects being described, and do not distinguish between sequential orders, nor do they indicate any particular limitations on the number of devices in the embodiments of the present invention, and do not limit the embodiments of the present invention.

[0023] Most video coding and decoding standards employ a block-based hybrid coding framework. Each image, subimage, or frame in a video is divided into square largest coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 128x128 or 64x64). Each LCU or CTU is then divided into rectangular coding units (CUs) according to a set of rules. A coding unit may be further divided into prediction units (PUs) and / or transform units (TUs). The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filters. The prediction module includes intra-prediction and inter-prediction. Inter-prediction includes motion estimation and motion compensation. The coding of adjacent pixels in a frame of a video is performed. sample Since there is a strong correlation between them, in video coding and decoding technology, intra prediction methods are used to predict adjacent sample Because there is a strong similarity between adjacent frames in a video, in video encoding and decoding technology, inter prediction methods are used to eliminate temporal redundancy between adjacent frames to improve encoding and decoding efficiency.

[0024] The basic flow of a video encoder and decoder is shown in Figure 1. On the encoder side, as shown in Figure 1, an image 101 of one frame is divided into blocks, a prediction block of the current block is generated using intra prediction or inter prediction for the current block, the prediction block is subtracted from the original block of the current block to obtain a residual block, the residual block is transformed and quantized to obtain a quantization coefficient matrix, and the quantization coefficient matrix is ​​entropy coded and output as a bitstream. On the decoder side (not shown), a prediction block of the current block is generated using intra prediction or inter prediction for the current block, the bitstream is analyzed to obtain a quantization coefficient matrix, the quantization coefficient matrix is ​​inversely quantized and inversely transformed to obtain a residual block, and the prediction block and the residual block are added to obtain a reconstructed block. A reconstructed image is constructed from the reconstructed blocks, and in-loop filtering is performed on the reconstructed image based on the image or block to obtain a decoded image. On the encoder side, a decoded image must be obtained using operations similar to those on the decoder side. On the encoder side, the obtained decoded image may be used as a reference image for inter prediction for a subsequent frame. Mode or parameter information, such as block partition information, prediction, transform, quantization, entropy coding, and in-loop filtering, determined by the encoder side must be included in the output bitstream, if necessary. The decoder side, as shown in FIG. 2, determines the same mode or parameter information, such as block partition information, prediction, transform, quantization, entropy coding, and in-loop filtering, as the encoder side through analysis based on analysis and existing information, ensuring that the decoded image obtained by the encoder side is identical to the decoded image obtained by the decoder side. The decoded image obtained by the encoder side is usually also called a reconstructed image. During prediction, the current block can be divided into prediction units, and during transformation, the current block can be divided into transform units, although the division of the prediction units and the transform units may be different.

[0025] The above is a basic flow of a video encoder and decoder in a block-based hybrid coding framework. As technology develops, some modules or steps of the framework or flow may be optimized. The encoding method and decoding method according to an embodiment of the present invention can be applied to this basic flow of a video encoder and decoder in a block-based hybrid coding framework, but are not limited to this framework and flow. Those skilled in the art will understand that with the evolution of encoders and decoders and the emergence of new business scenarios, the methods according to embodiments of the present invention can be similarly applied to similar technical problems.

[0026] The current block may be a current coding unit (CU) or a current prediction unit (PU), etc.

[0027] Furthermore, an embodiment of the present invention further provides a network architecture of an encoding / decoding system including an encoder and a decoder, of which FIG. 3 shows a schematic diagram of the network architecture of an encoding / decoding system according to an embodiment of the present invention. As shown in FIG. 3, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, in which the electronic devices 13 to 1N can perform video interaction via the communication network 01. In the implementation process, the electronic device may be a device with various video encoding and decoding functions, for example, a smartphone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital telephone, a video telephone, a television, a sensing device, a server, etc., and is not specifically limited in the embodiment of the present invention. Here, the decoder or encoder described in the embodiment of the present invention may be the above-mentioned electronic device.

[0028] It should be noted that the method of the embodiment of the present invention is mainly applied to the intra prediction and / or inter prediction module as shown in Fig. 1 and the intra prediction and / or inter prediction module as shown in Fig. 2. That is, the embodiment of the present invention may be applied to an encoder, a decoder, or even to an encoder and a decoder simultaneously, but is not specifically limited in the embodiment of the present invention.

[0029] Furthermore, when applied to an intra-prediction and / or inter-prediction module on the encoder side, the "current block" is specifically a coded block on which intra-prediction and / or inter-prediction is to be performed, and when applied to an intra-prediction and / or inter-prediction module on the decoder side, the "current block" is specifically a decoded block on which intra-prediction and / or inter-prediction is to be performed.

[0030] An embodiment of the present invention provides an encoding method applied to an encoder. FIG. 4 is a schematic diagram of the implementation flow of the encoding method according to an embodiment of the present invention. As shown in FIG. 4, the method includes steps 401 to 404 as follows:

[0031] In step 401, a template for a current block and one or more prediction templates for the template for the current block are determined.

[0032] In step 402, weights of the one or more prediction templates are determined based on the template of the current block and the one or more prediction templates.

[0033] In step 403, one or more first prediction blocks of the current block are determined based on the prediction parameters of the current block.

[0034] In step 404, the one or more first prediction blocks are fused according to the weights of the one or more prediction templates to obtain a second prediction block of the current block.

[0035] In an embodiment of the present invention, weights of the one or more prediction templates are determined based on a template of a current block and one or more prediction templates of the template of the current block, and one or more first prediction blocks of the current block are blended based on the weights to obtain a second prediction block of the current block. In this way, compared with a method of blending one or more first prediction blocks of a current block based on fixed weights, the weight determination method according to an embodiment of the present invention is based on a template and one or more prediction templates of the template of the current block, and therefore has strong adaptability, that is, the weights change with changes in the current block, and the obtained weights are more suited to the actual situation of the current block, which is beneficial to improving the prediction accuracy of the current block and enhancing coding performance.

[0036] Further alternative embodiments and related terms for each of the above steps will be described below.

[0037] In step 401, a template for a current block and one or more prediction templates for the template for the current block are determined.

[0038] In embodiments of the present invention, the method for determining the template for the current block, the size and shape, and the relative position of the template and the current block are not limited. In some embodiments, the template for the current block may be an area adjacent to or non-adjacent to the current block. The height and width of the template for the current block to be obtained are L2 and L1, respectively. In DIMD, L1 = L2 = 3, but L1 and L2 are not limited to being equal to 3. The encoder may adaptively determine the size of the template based on the block size of the current block, or may transmit the template to the decoder in the form of a bitstream, and the decoder may obtain the template size by decoding the bitstream.

[0039] In an embodiment of the present invention, the shape of the template of the current block may be various, and may be L-shaped or straight-line shaped; the L-shaped template may include an upper left, upper, and left template, or may include an upper left, upper, upper right, left, and / or lower left template; and the template of the current block may include only an upper template or only a left template, or may include only an upper template and a left template.

[0040] It can be understood that the prediction template of the template of the current block may be understood as a predicted block / predicted value / reference block / reference value of the template of the current block. In some embodiments, the template of the current block can be predicted based on the prediction parameters of the template of the current block to obtain prediction templates of some or all of the templates of the current block, where the prediction parameters of the template of the current block are used to indicate one or more intra prediction modes and / or one or more motion parameters, and the motion parameters include MVs and / or reference frame indexes.

[0041] Furthermore, in some embodiments, the template of the current block can be predicted based on the prediction parameters of the template of the current block and the reference region / reference pixel / reference sample of the template of the current block, to obtain a prediction template for part or all of the template of the current block.

[0042] The reference area of ​​the template for the current block may include the neighboring areas of the upper left, upper, upper right, left, and / or lower left of the template. In some embodiments, if one or more of the upper left, upper, upper right, left, and / or lower left areas of the template are available / have already been encoded and reconstructed / have already been reconstructed, these available areas are selected as the reference area of ​​the template for the current block; if they are not available, these areas are not selected. Of course, the reference area of ​​the template may also include pixels / samples of areas not adjacent to it, for example, the reference pixels / samples of the second row from the top of the template. For example, the template for the current block and the reference area of ​​the template for the current block are shown in FIG. 5.

[0043] In embodiments of the present invention, there may be various methods for determining prediction templates, and the encoder may obtain some or all of the one or more prediction templates through one or more prediction template determination methods as described below. For example, the encoder may obtain one or more prediction templates for the template of the current block through at least one of the following embodiments 1 to 5.

[0044] In the first embodiment, as shown in FIG. 6, the encoder can determine the prediction template using steps 601 to 603 as follows.

[0045] In step 601, a histogram of oriented gradients (HoG) is calculated for the samples of the template of the current block to obtain the gradient direction and gradient magnitude of the corresponding samples.

[0046] That is, the encoder may calculate the gradient direction and gradient magnitude of the template samples of the current block based on the predefined horizontal and vertical filters, where the sizes of the horizontal and vertical filters are not limited and may be 2x2, 3x3, 4x4, etc., and the sizes of the horizontal and vertical filters may be determined based on the size of the template of the current block. For example, the horizontal filter may include a sober filter, and the vertical filter may include a vertical sober filter.

[0047] In step 602, an angular mode is determined based on the gradient direction and gradient strength.

[0048] In step 603, a template of the current block is predicted based on the angle mode to obtain the predicted template.

[0049] The angle mode determined here is used to perform intra prediction on the template of the current block to obtain a prediction template for the template of the current block.

[0050] In some embodiments, the encoder can convert the gradient directions into predefined candidate angle modes and determine the angle mode from the converted candidate angle modes based on the gradient magnitudes. Of course, in other embodiments, the encoder can directly determine the angle mode based on the gradient directions and gradient magnitudes without converting the gradient directions into candidate angle modes.

[0051] Furthermore, in some embodiments, the encoder may determine the angle modes by obtaining N angle modes based on the candidate angle modes corresponding to the N largest gradient strengths. For example, the encoder may determine the candidate angle modes corresponding to the N largest gradient strengths as the N angle modes, where N is any value greater than 0, e.g., N=2, 3, or 4, etc. In some embodiments, the encoder may write the value of N into the bitstream.

[0052] And / or in some other embodiments, the encoder may determine the angular modes as follows: obtain one or more of the angular modes based on candidate angular modes corresponding to gradient strengths greater than or equal to a first threshold. For example, the encoder may determine candidate angular modes corresponding to gradient strengths greater than or equal to a first threshold as the angular modes.

[0053] And / or in some other embodiments, the encoder may determine the one or more angular modes as follows: obtain the one or more angular modes based on candidate angular modes corresponding to second gradient strengths whose difference values ​​from at least one first gradient strength are less than or equal to a second threshold. For example, the encoder may determine as the angular modes candidate angular modes corresponding to second gradient strengths whose difference values ​​from at least one first gradient strength are less than or equal to a second threshold.

[0054] In some embodiments of the present invention, the relationship between the first gradient strength and the second gradient strength is not limited. In some embodiments, the magnitude of the first gradient strength is adjacent to the magnitude of the second gradient strength. For example, the magnitude of the first gradient strength is next to the magnitude of the second gradient strength.

[0055] To facilitate understanding of the above steps 601 and 602, the following exemplary description will be given, but this does not limit the technical solutions of the embodiments of the present invention. For example, as shown in FIG. 7, the encoder uses a 3x3 horizontal sober filter and a vertical sober filter to calculate the horizontal gradient Gx and vertical gradient Gy of the central pixel / sample of the template of the current block, respectively. Then, it calculates the angle of the corresponding pixel / sample by atan(Gy / Gx). It converts the angle into one of 65 candidate angle modes IPM in VVC, and the sum of the absolute values ​​of Gx and Gy is the cumulative intensity value / gradient strength of the candidate angle mode. The above process can be repeated for each central pixel / sample (or even for some central pixels / samples) of the template of the current block to obtain a candidate angle mode histogram. As shown in FIG. 8, the abscissa of the histogram is the index of the candidate angle mode, and the ordinate of the histogram is the corresponding gradient strength of the candidate angle mode.

[0056] In the candidate angle mode histogram, the first N candidate angle modes with the largest gradient magnitudes are selected as intra prediction modes (i.e., the angle modes for performing intra prediction). In an embodiment of the present invention, the first N candidate angle modes with the largest gradient magnitudes may be directly selected as intra prediction modes, or a first threshold may be set and a corresponding candidate angle mode may be selected as an intra prediction mode only if the gradient magnitude is greater than or equal to the first threshold, or a second threshold may be set and a candidate angle mode corresponding to the gradient magnitude may be selected as an intra prediction mode only if the difference between adjacent gradient magnitudes is less than or equal to the second threshold.

[0057] The encoding method according to the first embodiment is an improvement over the decoder side intra mode derivation (DIMD) prediction method.

[0058] In the second embodiment, as shown in FIG. 9, the encoder can determine the prediction template using steps 901 and 902 as follows.

[0059] In step 901, a template of the current block is predicted based on a candidate prediction mode in a mode list to obtain a candidate prediction template.

[0060] In some embodiments, the candidate prediction modes in the mode list of the encoder include: the angular modes obtained using the steps described in Example 1; A prediction mode in the Most Probable Mode (MPM) list; The prediction modes in the MPM list after the Planar mode has been removed, and An angle mode obtained by expanding an angle mode in the MPM list whose angle is greater than or equal to a fourth threshold; and an angle mode obtained by expanding an angle mode in the MPM list whose angle mode is less than or equal to a fifth threshold.

[0061] Furthermore, in some embodiments, in the MPM list, a DC mode is after or before an angular mode constructed based on neighboring blocks of the current block.

[0062] Furthermore, in some embodiments, in the mode list, the N angular modes are after or before an angular mode constructed based on neighboring blocks of the current block.

[0063] In some embodiments, the encoder may implement step 901 as follows: A candidate prediction template is obtained by predicting a template for the current block based on sample values ​​of a reference region of the template for the current block and the candidate prediction mode. The reference region includes a non-adjacent region and / or an adjacent region of the template for the current block. Furthermore, in some embodiments, the reference region is a region / reconstructed region that has already been encoded and decoded. For example, in some embodiments, the reference region includes the upper left region, upper region, upper right region, left region, and / or lower left region of the template for the current block.

[0064] In step 902, a candidate prediction template is obtained based on the sample value error between the candidate prediction template and the template of the current block.

[0065] In some embodiments, the prediction module can be determined as follows: obtain N prediction templates based on candidate prediction templates corresponding to the N smallest sample value errors, and / or obtain one or more prediction templates based on candidate prediction templates corresponding to sample value errors less than or equal to a third threshold, where N is any value greater than 0, for example, N=2, 3, 4, etc. In some embodiments, the encoder can write the value of N into the bitstream.

[0066] In some embodiments, the sample error type is not limited, and represents the error between the sample values ​​of the candidate prediction template and the sample values ​​of the template of the current block. In some embodiments, the sample error type is at least one of the following: sum of absolute transform differences (SATD), sum of absolute differences (SAD), mean absolute deviation (MAD), mean absolute error (MAE), normalized correlation coefficient (NCC), mean squared error (MSE), and sum of squared errors (SSE).

[0067] It should be noted that the encoding method according to the second embodiment is an improvement over the template-based intra mode derivation and fusion (TIMD) prediction method.

[0068] In the third embodiment, as shown in FIG. 10A, the encoder can determine the prediction template using steps 1001 to 1003 as follows.

[0069] In step 1001, a candidate prediction mode in the mode list and a reference row in the reference row list of the template of the current block are combined to obtain a combination list.

[0070] In template-based multiple reference line intra prediction (TMRL), the template of the current block is the pixel / sample of the first row and the first column adjacent to the current block. In addition, the template of the current block is the pixel / sample of the first M rows and the first M columns adjacent to the current block. sample where M is any value greater than 1. For example, as shown in FIG. 10B, the templates of the current block are the left-neighboring region and the top-neighboring region of the current block, and the reference rows are reference rows 1, 2, and 3, etc., as shown in FIG. 10B.

[0071] In some embodiments, the Reference Row List of the current block includes indices indicating the reference rows, and the embodiments of the present invention do not limit which reference row indices are included in the Reference Row List. For example, in some embodiments, the Reference Row indices included in the Reference Row List are {1,3,5,7,12}, {1,4,5,7,10,12}, or {1,2,3,7,10,11}.

[0072] In addition, for the current block, the extended reference rows used by TMRL are {1, 3, 5, 7, 12}, which are a total of five reference rows. The extended MPM list has a total of 10 candidate prediction modes. Therefore, a TMRL combination list can be constructed. The combination list has a total of 5x10=50 combinations, and each combination includes an index of one candidate prediction mode and an index of at least one reference row. The encoder can predict the template of the current block based on the candidate prediction mode and at least one reference row indicated by a combination to obtain a candidate prediction template.

[0073] In an embodiment of the present invention, the mode list described in step 1001 can be understood by referring to the above description of the mode list, that is, the candidate prediction modes in the mode list can be determined based on the above method, and will not be described again here.

[0074] In step 1002, a template of the current block is predicted based on the candidate prediction mode and reference row indicated by the combination in the combination list to obtain a candidate prediction template.

[0075] In step 1003, a candidate prediction template is obtained based on the sample value error between the candidate prediction template and the template of the current block.

[0076] In some embodiments, the prediction module can be determined as follows: obtain N prediction templates based on candidate prediction templates corresponding to the N smallest sample value errors, and / or obtain one or more prediction templates based on candidate prediction templates corresponding to sample value errors less than or equal to a third threshold, where N is any value greater than 0, for example, N=2, 3, 4, etc. In some embodiments, the encoder can write the value of N into the bitstream.

[0077] In some embodiments, the sample error type is not limited, and represents the error between the sample values ​​of the candidate prediction template and the sample values ​​of the template of the current block. In some embodiments, the sample error type is at least one of the following: sum of absolute transform differences (SATD), sum of absolute differences (SAD), mean absolute deviation (MAD), mean absolute error (MAE), normalized correlation coefficient (NCC), mean squared error (MSE), and sum of squared errors (SSE).

[0078] The encoding method according to the third embodiment may be an improvement over TMRL.

[0079] In the fourth embodiment, as shown in FIG. 11, the encoder can determine the prediction template using steps 1101 and 1102 as follows.

[0080] In step 1101, at least one candidate MV is determined.

[0081] In some embodiments, the encoder can obtain at least one candidate MV based on the Merge candidate list or the AMVP candidate list, and in some embodiments, the encoder can obtain the first N candidate MVs in the Merge candidate list or the AMVP candidate list.

[0082] In an embodiment of the present invention, the candidate MVs in the Merge candidate list or the AMVP candidate list are MVs subdivided by template matching, DMVR and / or Multipass DMVR.

[0083] In some embodiments, the encoder may further write related information of at least one candidate MV into the bitstream, so that the decoder does not need to perform step 1101.

[0084] In step 1102, motion compensation is performed on the template of the current block based on the candidate MV and the reference frame of the current block to obtain the prediction template.

[0085] In some embodiments, the reference frame of the current block is a forward reference frame or a backward reference frame of the current block.

[0086] In an embodiment of the present invention, the prediction template participating in the determination of the weight may or may not include a first offset term, and correspondingly, the first prediction block participating in the determination of the second prediction block may or may not include a second offset term.

[0087] In an embodiment of the present invention, the prediction template participating in the determination of the weight may or may not include a first nonlinear term, and correspondingly, the first prediction block participating in the determination of the second prediction block may or may not include a second nonlinear term.

[0088] In some embodiments, the first offset term is a template generated based on the sample value bit depth of the template of the current block, i.e., the one or more prediction templates further include a template generated based on the sample value bit depth of the template of the current block; and correspondingly, the second offset term is a block generated based on the sample value bit depth of the current block, i.e., the one or more first prediction blocks further include a block generated based on the sample value bit depth of the current block.

[0089] In some embodiments, the first nonlinear term is a prediction template obtained by performing a transformation on the candidate prediction template having the smallest sample value error, i.e., the one or more prediction templates further include a prediction template obtained by performing a transformation (e.g., a squaring calculation) on the candidate prediction template having the smallest sample value error; correspondingly, the second nonlinear term is a prediction block obtained by performing the transformation (e.g., a squaring calculation) on the first prediction block corresponding to the candidate prediction template having the smallest sample value error, i.e., the one or more first prediction blocks further include a prediction block obtained by performing the transformation on the first prediction block corresponding to the candidate prediction template having the smallest sample value error.

[0090] In addition, the first predicted block corresponding to the candidate prediction template with the smallest sample value error is the first predicted block obtained by predicting the current block based on the prediction mode for obtaining the candidate prediction template with the smallest sample value error.

[0091] In some embodiments, the prediction template participating in determining the weight may include a third offset term, where the third offset term is a prediction template obtained by predicting a template of the current block based on a specific prediction mode or a specific MV, and the first prediction block participating in determining the second prediction block may include a fourth offset term, where the fourth offset term is a prediction block obtained by predicting the current block based on the specific prediction mode or the specific MV. Based on this, in some embodiments, the encoder needs to further perform a mode / MV overlap check, and in response to determining that the specific prediction mode is different from the intra prediction mode / MV indicated by the prediction parameters, predict the template of the current block based on the specific prediction mode / the specific MV to obtain a first prediction module.

[0092] Illustratively, in some embodiments, the specific prediction mode includes a planar mode and / or a DC mode.

[0093] In an embodiment of the present invention, the one or more prediction templates further include a prediction template obtained by predicting the template of the current block based on an intermediate prediction mode before each fusion operation in the Chroma Fusion, OBMC, MHP, and / or SGPM methods.

[0094] In some embodiments, if the one or more prediction templates are less than N, the prediction templates are replenished to N by adding at least one of the first offset term, the third offset term, the first nonlinear term, and a first preset value, and correspondingly, the first prediction blocks are replenished to N by adding at least one of the second offset term, the fourth offset term, the second nonlinear term, and a second preset value.

[0095] Illustratively, the first preset value is equal to zero.

[0096] In Example 5, the encoder can determine a prediction template using the following steps: determining at least one candidate BV; and determining a prediction template for the template of the current block based on the candidate BV, the current block, and the current image.

[0097] In some embodiments, the encoder may derive at least one candidate BV based on a Merge candidate list, an AMVP candidate list, and / or a preset value. Further, in some embodiments, the encoder may obtain the first N candidate BVs in the Merge candidate list or the AMVP candidate list. Illustratively, the preset value is equal to 0.

[0098] IBC (Intra Block Copy) significantly improves the compression efficiency of screen content coding and is therefore used in all screen content coding applications, from HEVC to VVC. Unlike camera-captured content, screen content is computer-generated. It is noise-free, contains text, computer graphics, and other elements, and has clear boundaries. Screen content contains a large amount of duplicated content. IBC can be considered an application of inter-prediction to intra-prediction. As mentioned above, inter-prediction uses a reference block in a reference picture as the prediction block for the current block; the reference picture is not the current picture. IBC identifies a block from an already coded, decoded, or reconstructed portion of the current picture as the prediction block for the current block. IBC is also known as intra-picture block compensation or current picture referencing (CPR).

[0099] IBC represents the positional difference between a current block and a reference block using a block vector (BV), which is similar to the MV in inter prediction. The encoder determines the best matching block for the current block within a search range using a block matching method and encodes the BV. There are several methods for encoding the BV, and they will not be described here. IBC may be considered a type of intra prediction method, or a separate prediction method independent of intra prediction and inter prediction.

[0100] In step 402, weights of the one or more prediction templates are determined based on the template of the current block and the one or more prediction templates.

[0101] In some embodiments, the encoder may implement step 402 as follows: determine weights for the one or more prediction templates based on the template of the current block and the one or more prediction templates, so that a sample value error between the template of the current block and a target prediction value of the template of the current block is minimized, where the target prediction value is equal to a weighted sum of sample values ​​of the one or more prediction templates.

[0102] There is no limitation on the type of sampled error, and the sampled error type can be one of MSE, SATD, SAD, MAD, MAE, NCC, and SSE.

[0103] Furthermore, in some embodiments, the encoder may determine an autocorrelation matrix of the prediction template based on sample values ​​of the prediction template, determine a cross-correlation vector between the prediction template and the template of the current block based on sample values ​​of the prediction template and sample values ​​of the template of the current block, and determine weights for the one or more prediction templates based on the autocorrelation matrix and the cross-correlation vector.

[0104] Take the sample value error as an example, MSE, the calculation formula for MSE is as follows:

number

[0105] To simplify the calculation formula for MSE, we use E to represent the mean square error MSE.

number

[0106] The weights w of the first prediction block corresponding to each prediction template are calculated by minimizing the MSE. p The specific steps to derive and obtain are as follows:

[0107] (1) First, w p Find the partial derivative with respect to and set it to zero.

number

number

[0108] (2) Prediction templates refT0, refT1, ... refT P-1 After determining, the equation obtained in step (1) can be expanded into matrix form to give

number

[0109] (3) Since the autocorrelation matrix and cross-correlation vector in (2) are both known quantities, solving the linear equation in (2) gives the weights w0, ... w0 of the predicted samples corresponding to each candidate reference term. p-1 can be obtained by calculating

[0110] In step 403, one or more first prediction blocks of the current block are determined based on the prediction parameters of the current block.

[0111] In some embodiments, the encoder may determine a first prediction block for the current block based on an intra prediction mode indicated by the prediction parameters, and / or may determine a first prediction block for the current block based on motion parameters for the current block indicated by the prediction parameters, where the motion parameters include at least one parameter of a motion vector (MV) and a reference frame index. A "reference frame" in embodiments of the present invention may be understood as a reference image.

[0112] In an embodiment of the present invention, the prediction parameters of the current block are the same as the prediction parameters of the template of the current block. That is, the prediction mode for obtaining the prediction template is also used to obtain the first predicted block of the current block. In this way, a prediction template and a first predicted block are associated because they use the same prediction mode, and therefore the weight of the prediction template is the weight of the associated first predicted block.

[0113] In step 404, the one or more first prediction blocks are fused according to the weights of the one or more prediction templates to obtain a second prediction block of the current block.

[0114] In some embodiments, for intra prediction or unidirectional inter prediction, the encoder may determine a residual block of the current block based on a second prediction block of the current block, and generate a bitstream based on the residual block.

[0115] In some other embodiments, for bidirectional inter prediction, assuming that the second predicted block is a predicted block of the current block obtained based on a forward reference frame and the third predicted block is a predicted block of the current block obtained based on a backward reference frame, the encoder needs to further perform weighted fusion on the second predicted block and the third predicted block to obtain a fourth predicted block, and then the encoder determines a residual block of the current block based on the fourth predicted block of the current block, and generates a bitstream based on the residual block, where the method of determining the third predicted block is the same as the method of determining the second predicted block, and only the reference frame during inter prediction is different.

[0116] An embodiment of the present invention further provides a decoding method applied to a decoder. FIG. 12 is a schematic diagram of the implementation flow of the decoding method according to an embodiment of the present invention. As shown in FIG. 12, the method includes steps 1201 to 1205 as follows:

[0117] In step 1201, a template for a current block and one or more prediction templates for the template for the current block are determined.

[0118] In step 1202, weights of the one or more prediction templates are determined based on the template of the current block and the one or more prediction templates.

[0119] In step 1203, one or more first prediction blocks of the current block are determined based on the prediction parameters of the current block.

[0120] In step 1204, the one or more first prediction blocks are fused according to the weights of the one or more prediction templates to obtain a second prediction block of the current block.

[0121] In step 1205, a reconstructed value of the current block is determined based on the second predicted block.

[0122] In some embodiments, a decoder decodes a bitstream, determines a residual block corresponding to the current block, and determines a reconstructed value of the current block based on the residual block and the second predictive block.

[0123] In some embodiments, the decoder may determine a first prediction block for the current block based on the intra-prediction mode indicated by the prediction parameters.

[0124] In some embodiments, the decoder may determine a first prediction block for the current block based on the motion parameters of the current block indicated by the prediction parameters.

[0125] In some embodiments, the motion parameters include at least one of the following parameters: a motion vector, a reference frame index.

[0126] In some embodiments, the prediction parameters of the current block are the same as the prediction parameters of the template of the current block.

[0127] In some embodiments, the decoder may determine a prediction template for the template of the current block by the following method, which includes: performing HoG calculations on samples of the template of the current block to obtain gradient directions and gradient magnitudes of corresponding samples; determining an angle mode based on the gradient directions and gradient magnitudes; and predicting the template of the current block based on the angle mode to obtain the prediction template.

[0128] In some embodiments, determining the angular mode based on the gradient direction and gradient magnitude includes converting the gradient direction to a predefined candidate angular mode, and determining the angular mode from the converted candidate angular mode based on the gradient magnitude.

[0129] In some embodiments, determining the angular modes based on the gradient strengths includes obtaining the N candidate angular modes based on N largest gradient strengths, and / or obtaining one or more of the angular modes based on candidate angular modes corresponding to gradient strengths greater than or equal to a first threshold, and / or obtaining one or more of the angular modes based on candidate angular modes corresponding to second gradient strengths whose difference value from at least one first gradient strength is less than or equal to a second threshold.

[0130] The decoder may obtain the value of N by analyzing the bitstream, or may determine the N angle modes using the same method as the encoder.

[0131] In some embodiments, the magnitude of the at least one first gradient strength is adjacent to the magnitude of the second gradient strength.

[0132] In some embodiments, the decoder may determine a prediction template for the template of the current block by the following method, which includes predicting the template of the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template, and obtaining the prediction template based on a sample value error between the candidate prediction template and the template of the current block.

[0133] In some embodiments, obtaining the prediction templates based on sample value errors between the candidate prediction templates and the template for the current block includes obtaining the N prediction templates based on candidate prediction templates corresponding to the N smallest sample value errors, or obtaining one or more prediction templates based on candidate prediction templates corresponding to sample value errors less than or equal to a third threshold.

[0134] In some embodiments, the sample error type is at least one of SATD, SAD, MAD, MAE, NCC, MSE, and SSE.

[0135] In some embodiments, the candidate prediction modes in the mode list are: As mentioned above the angle modes obtained using step (a), the prediction modes in the MPM list, the prediction modes in the MPM list after the planar mode has been deleted, the angle modes obtained by extending the angle modes in the MPM list whose angles are greater than or equal to a fourth threshold, and the angle modes obtained by extending the angle modes in the MPM list whose angles are less than or equal to a fifth threshold.

[0136] In some embodiments, in the MPM list, a DC mode is after or before an angular mode constructed based on neighboring blocks of the current block.

[0137] In some embodiments, in the mode list, the N angular modes are after or before an angular mode constructed based on neighboring blocks of the current block.

[0138] In some embodiments, the step of predicting a template for the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template includes predicting a template for the current block based on sample values ​​of a reference region of the template for the current block and the candidate prediction mode to obtain a candidate prediction template, wherein the reference region includes a non-adjacent region and / or an adjacent region of the template for the current block.

[0139] In some embodiments, the reference region includes the top left region, the top region, the top right region, the left region and / or the bottom left region of the template of the current block.

[0140] In some embodiments, the step of predicting a template for the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template includes: combining a candidate prediction mode in the mode list with a reference row in a reference row list of the template for the current block to obtain a combination list; and predicting a template for the current block based on the candidate prediction mode and the reference row indicated by the combination in the combination list to obtain a candidate prediction template.

[0141] In some embodiments, the Reference Row Indices included in the Reference Row List are {1,3,5,7,12}, {1,4,5,7,10,12}, or {1,2,3,7,10,11}.

[0142] In some embodiments, a decoder may determine a prediction template for a template of the current block by the following method, the method including: determining at least one candidate BV; and determining a prediction template for a template of the current block based on the candidate BV, the current block, and a current image.

[0143] In some embodiments, the encoder may derive at least one candidate BV based on a Merge candidate list, an AMVP candidate list, and / or a preset value. Further, in some embodiments, the encoder may obtain the first N candidate BVs in the Merge candidate list or the AMVP candidate list. Illustratively, the preset value is equal to 0.

[0144] In some embodiments, the decoder may determine a prediction template for the template of the current block by the following method, which includes: determining at least one candidate motion vector; and performing motion compensation on the template of the current block based on the candidate motion vector and a reference frame of the current block to obtain the prediction template.

[0145] In an embodiment of the present invention, the decoder may parse the bitstream to obtain the at least one candidate MV, or may determine the at least one candidate MV through the same operation as the encoder side.

[0146] In some embodiments, the reference frame of the current block is a forward reference frame or a backward reference frame of the current block.

[0147] In some embodiments, determining the at least one candidate MV includes obtaining the at least one candidate MV based on a Merge candidate list or an AMVP candidate list.

[0148] In some embodiments, obtaining at least one candidate MV based on the Merge candidate list or the AMVP candidate list includes obtaining the first N candidate MVs in the Merge candidate list or the AMVP candidate list.

[0149] In some embodiments, the candidate MVs in the Merge candidate list or AMVP candidate list are MVs subdivided by template matching, DMVR and / or Multipass DMVR, and / or the candidate MVs in the Merge candidate list or AMVP candidate list are candidate MVs sorted according to a predetermined order relative to those in the original candidate list.

[0150] Of course, the candidate MVs in the Merge candidate list or the AMVP candidate list may be unfragmented MVs.

[0151] In some embodiments, the one or more prediction templates further include a first offset term, which is a template generated based on a sample value bit depth of the template of the current block, and the one or more first prediction blocks further include a second offset term, which is a block generated based on a sample value bit depth of the current block.

[0152] In some embodiments, the one or more prediction templates further include a third offset term, which is a prediction template obtained by predicting a template of the current block based on a specific prediction mode or a specific MV, and the one or more first prediction blocks further include a fourth offset term, which is a prediction block obtained by predicting the current block based on the specific prediction mode or the specific MV.

[0153] In some embodiments, the particular prediction mode includes a planar mode and / or a DC mode.

[0154] In some embodiments, the method further includes, in response to determining that the particular prediction mode is different from the intra-prediction mode indicated by the prediction parameters, predicting a template for the current block based on the particular prediction mode.

[0155] In some embodiments, the method further includes, in response to determining that the particular MV differs from the MV of the current block indicated by the prediction parameters, predicting a template of the current block based on the particular MV.

[0156] In some embodiments, the one or more prediction templates further include a first nonlinear term, which is a prediction template obtained by performing a transformation on a candidate prediction template having the smallest sample value error, and the one or more first prediction blocks further include a second nonlinear term, which is a prediction block obtained by performing the transformation on a first prediction block corresponding to the candidate prediction template having the smallest sample value error.

[0157] In some embodiments, the transformation comprises a squaring operation.

[0158] In some embodiments, the one or more prediction templates further include a prediction template obtained by predicting a template of the current block based on an intermediate prediction mode before each fusion operation in the Chroma Fusion, OBMC, MHP, and / or SGPM methods.

[0159] In some embodiments, if the one or more prediction templates are less than N, the prediction templates are replenished to N by adding at least one of the first offset term, the third offset term, the first nonlinear term, and a first preset value, and correspondingly, the first prediction blocks are replenished to N by adding at least one of the second offset term, the fourth offset term, the second nonlinear term, and a second preset value.

[0160] Illustratively, the first preset value is equal to zero.

[0161] In some embodiments, determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates includes determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates so as to minimize a sample value error between the template of the current block and a target prediction value of the template of the current block, wherein the target prediction value is equal to a weighted sum of sample values ​​of the one or more prediction templates.

[0162] In some embodiments, determining weights for the one or more prediction templates based on the template for the current block and the one or more prediction templates comprises: determining an autocorrelation matrix for the prediction template based on sample values ​​of the prediction template; determining a cross-correlation vector between the prediction template and the template for the current block based on the sample values ​​of the prediction template and the sample values ​​of the template for the current block; and determining weights for the one or more prediction templates based on the autocorrelation matrix and the cross-correlation vector.

[0163] In some embodiments, the sample error type is one of MSE, SATD, SAD, MAD, MAE, NCC, and SSE.

[0164] It should be noted that technical details not disclosed in the embodiment of the decoding method can be understood by referring to the description of the embodiment of the encoding method described above.

[0165] (1) DIMD is an intra-prediction technique that is performed on both the encoder and decoder sides. The technique reconstructs adjacent four rows and four columns. sample is used as a template, and a window of size 3x3 is used around the current block to find the sampleThe HoG calculation is performed on the horizontal and vertical Sobel filters. sample Based on the horizontal gradient iDy and vertical gradient iDx of sample The angular mode to which the vector belongs and the corresponding intensity value ((abs(iDx)+abs(iDy))) are derived and accumulated. sample After the calculation is completed, the intra prediction modes IPM (Intra Prediction Mode) corresponding to the two highest histogram accumulated intensity values ​​ampl1 and ampl2 are selected for the current coding block and denoted as mode1 and mode2. The predicted values ​​obtained in these two prediction modes and the predicted value obtained in the planar mode are weighted and combined to obtain a final intra predicted value. The weights of the predicted values ​​of the two intra prediction modes derived from the histogram are determined by the corresponding histogram intensities ampl1 and ampl2, and the weight of the planar predicted value is a fixed weight.

[0166] The flowchart of DIMD prediction is shown in Figure 13.

[0167] The weights w1 and w2 of mode1 and mode2 in DIMD are calculated as follows, and the weight of the planar mode is w3.

number

[0168] The predicted value of DIMD is calculated as follows:

number

[0169] (2) Template-based intra-mode derivation and fusion (TIMD) prediction process TIMD is an intra-prediction technique, which relies on neighboring reconstruction sample is a template, and sample the template reference sample Then, traverse each prediction mode in the most probable mode MPM list of length N to obtain prediction values ​​for N templates, and then sample The sum of absolute transform differences (SATD) between the predicted value and its reconstructed value is calculated, and two modes mode1 and mode2 with the smallest template cost are selected, whose corresponding costs are costMode1 and costMode2, respectively. If the condition costMode2<2*costMode1 is met, the prediction values ​​obtained in mode1 and mode2 for the current coding block are weighted and combined to obtain a final prediction value. The weights are calculated based on the template matching costs costMode1 and costMode2. If the condition costMode2<2*costMode1 is not met, only mode1 is used to predict the current block.

[0170] The flowchart of TIMD prediction is shown in Figure 14.

[0171] The weights w1 and w2 of mode1 and mode2 in TIMD are calculated as follows:

number

[0172] The predicted value of TIMD is calculated as follows:

number

[0173] (3) Template-based multiple reference line intra prediction process TMRL creates a multi-reference row candidate list based on the multi-reference row MRL. The reference row candidate list is {1,3,5,7,12}, and its length is 5. The numbers in the list represent the index of the reference row. The first adjacent row / column of the current block are also included. sample is the template for the current block. At the same time, TMRL expands the most probable mode MPM list, so that the length of the expanded MPM list is 10. Here, planar mode is deleted, DC mode (if not already included) is added after the modes of the five adjacent PUs and the DIMD mode, and angle modes with angle differences of ±1 to ±4 are added. A TMRL combination list is then created, with a total of 5 × 10 = 50 combinations. Next, the template prediction values ​​for these 50 combinations are calculated for the template for the current block, and the SAD between these 50 prediction values ​​and the reconstructed value of the template for the current block is calculated. The combination with the smallest SAD for the first 20 is selected as the final combination list for TMRL. TMRL prediction is performed for the current block using these 20 combinations, and the encoder selects the optimal combination and transmits it to the decoder.

[0174] The TMRL prediction flowchart is shown in Figure 15.

[0175] (4) Inter-Weighted Prediction In the inter prediction process, motion compensation can be divided into unidirectional motion compensation and bidirectional motion compensation according to the slice type. The current block determines the best matching block from the reference image in the forward reference list List0 or the backward reference list List1 based on the best motion vector information, and then determines the predicted block for the current block based on the matching block. When unidirectional prediction (i.e., List0 prediction or List1 prediction) is performed, the best matching block is determined from the List0 reference image or the List1 reference image, and weighting is performed on the matching block to obtain the inter predicted block for the current block. When bidirectional prediction is performed, the best matching block is determined from the List0 reference image and the List1 reference image, respectively, and weighted prediction is performed on the List0 best matching block and the List1 best matching block to obtain the inter predicted block for the current block.

[0176] Existing inter-weighted prediction techniques mainly include default weighted prediction algorithm (Default WP), explicit weighted prediction algorithm (Explicit WP) and CU-level weighted bidirectional weighted prediction algorithm (BCW). A detailed introduction is given below.

[0177] The default weighted prediction can be handled in three cases depending on the use of the reference list.

[0178] In case 1, only the reference list List0 is used, and the prediction sample pbSamples[x][y] is calculated as follows:

number

[0179] In case 2, only the reference list List1 is used, and the prediction sample pbSamples[x][y] is calculated as follows:

number

[0180] In case 3, both the reference list List0 and the reference list List1 are used, and the predicted sample pbSamples[x][y] is calculated as follows:

number

[0181] Where, predSamplesL0[x][y] and predSamplesL1[x][y] respectively represent the List0 predicted value and List1 predicted value of the current CU, and x and y respectively represent the List0 predicted value and List1 predicted value of the current CU. sample The shift1, shift2, offset1, and offset2 do not need to be transmitted in the bitstream, but can be determined based on the bit depth bitDepth of the input sequence.

number

[0182] For explicit weighted prediction, in the motion compensation process, a predetermined weight value and offset value are used to calculate the following: A weighted correction needs to be made to predSamplesL0[x][y] and predSamplesL1[x][y].

[0183] Explicit weighted prediction can be handled in three cases depending on the use of the reference list.

[0184] In case 1, only the reference list List0 is used, and the prediction sample pbSamples[x][y] is calculated as follows:

number

[0185] In case 2, only the reference list List1 is used, and the prediction sample pbSamples[x][y] is calculated as follows:

number

[0186] In case 3, both reference lists List1 and List2 are used, and the predicted sample pbSamples[x][y] is calculated as follows:

number

[0187] Wherein, w0 and w1 are weight values, o0 and o1 are corresponding offset amounts, and the values ​​of the above four parameters are determined by decoding the bitstream.

[0188] The CU-level weighted bidirectional weighted prediction (BCW) process is as follows:

[0189] For a bidirectionally predicted CU, the VVC encoder or decoder can further use BCW to determine its weighting coefficients for weighted prediction. Five candidate weighting coefficients are preset in BCW, and the weighting coefficients of BCW are determined according to the index numbers of the weighting coefficients. The data processing process is as follows:

[0190] If bcwIdx, i.e., BCW weighting coefficient index, is 0 or ciip_flag is 1, the prediction sample pbSamples[x][y] is calculated as follows:

number

[0191] If bcwIdx is not 0 and ciip_flag is 0, the forward prediction block weight w1 is bcwWLut[bcwIdx], where bcwWLut[k]={4, 5, 3, 10, -2}, and the forward prediction block weight w0 is 8-w1 correspondingly. sample pbSamples[x][y] is calculated as follows:

number

[0192] Inter unidirectional weighted prediction is as shown in FIG. 16, bidirectional weighted prediction is as shown in FIG. 17, and the flow of inter weighted prediction is as shown in FIG.

[0193] (1) Conventional DIMD techniques use adjacent templates for reconstruction. sample HoG calculation is performed using sample The angle mode and corresponding intensity values ​​to which the histogram belongs are obtained, and the intensity values ​​are accumulated. The corresponding modes with the top two accumulated intensity values ​​in the histogram are selected as prediction modes. Finally, the predicted values ​​of these two modes and the planar mode are weighted to obtain the final predicted value of the current block. Among them, the weights of the top two modes are calculated based on the accumulated intensity values, and the weight of the planar mode is a fixed weight. This leads to the following problems: if the planar mode is used for weighting with a fixed weight, the adaptability of the weighted fusion is reduced, and if only the modes corresponding to the top two accumulated intensity values ​​are selected as prediction modes, the reference information is limited.

[0194] (2) Conventional TIMD technology uses the current block template and template reference sampleThe MPM list is traversed using the criterion to predict a template, the SATD between the predicted template and the reconstruction value of the template is calculated, and two corresponding modes with relatively small SATD are selected as prediction modes, and the predicted values ​​obtained in these two prediction modes are weighted. The weights of these two modes are calculated based on the SATD. This leads to the following problem: if weighting is performed using only the two modes with relatively small SATD, the reference information is limited.

[0195] (3) The conventional TMRL technology constructs an intra prediction mode list and a reference row candidate list, combines them to obtain a TMRL candidate list, and predicts a template according to the combination of the template of the current block and the template in the TMRL candidate list. sample and template reconstruction sample The encoder calculates the SAD between the first 20 combinations, selects the best combination, and transmits it to the decoder. This causes the following problems: It is necessary to use extra bits to transmit the best combination, which increases the transmission burden. If the prediction results of the combinations selected by SAD are similar, transmitting only the best combination will limit the reference information.

[0196] (4) Conventional inter-weighted prediction techniques include unidirectional prediction and bidirectional prediction. In the case of unidirectional prediction, motion compensation is performed using only the optimal mv to obtain a predicted block. In the case of bidirectional prediction, motion compensation is performed using only the optimal mv in either direction to obtain a predicted block, and then weighting is performed using a fixed weight. This results in the following problems: For unidirectional and bidirectional prediction, if motion compensation is performed using only the optimal mv, reference information is limited; for bidirectional prediction, if fixed weights are used to weight the predicted block, the adaptability of the prediction is not high.

[0197] All of the above prior art references are limited, which significantly reduces the applicability and accuracy of predictions.

[0198] Based on this, an exemplary application of the embodiment of the present invention in one practical application scenario will be described below.

[0199] In this main aspect, we propose minMSE-based fusion, a weighted fusion technique based on minimizing MSE, for all of the above conventional techniques. Below, we improve each of the four conventional techniques, DIMD, TIMD, TMRL, and inter-weighted prediction, based on minMSE-based fusion.

[0200] 3.1. Main Aspects of minMSE-based Decoder Intra Prediction Fusion (minMSE-based DIPF) The main technical aspects of minMSE-based DIPF are as follows:

[0201] The minMSE-based DIPF technique of this main aspect mainly improves steps 1901, 1903, and 1904 in the DIMD prediction flowchart. The minMSE-based DIPF prediction flowchart is shown in Figure 19, where the neighboring template in Figure 19 is an example of the template of the current block described above, the histogram intensity value is the gradient intensity described above, the intra-prediction mode described in step 1903 is the angle mode described above, and the encoder / decoder predicts the template of the current block based on the angle mode to obtain the predicted template. The predicted value of the current block described in step 1904 is an example of the second predicted block of the current block described above.

[0202] The specific implementation flow of step 1904 is shown in FIG. 20, where the predicted block in FIG. 20 is the first predicted block mentioned above.

[0203] 3.1.1 Adjacent Templates and Adjacent References to Templates sample Get The acquired neighboring template has a height of L2 and a width of L1. In DIMD, L1 = L2 = 3. In practice, L1 and L2 may be adaptively determined based on the block size, or may be transmitted in the form of a bitstream. sample are the top left, top, top right, left, and bottom left regions of the adjacent template. If available, they are acquired. If not available, they are not acquired. sample is a reference that is not adjacent to this sample For example, the second line from the top of the template sample The adjacent template to be obtained and the adjacent reference of the template may be sample A schematic diagram of this is shown in Figure 5. The shape of the template is various, and it may be L-shaped, of which the L-shaped template may include an upper left, upper and left template, or an upper left, upper, upper right, left and lower left template, or only an upper template or only a left template, or only an upper template and a left template.

[0204] 3.1.2, all adjacent templates sample Perform HoG calculation on the As shown in Figure 7, in the template domain, T = 3, and the value of T can be adaptively selected based on the block size or transmitted via the bitstream. DIMD is the center of the template for the 3x3 horizontal and vertical sober filters shown in the right diagram of Figure 7. sample Calculate the horizontal and vertical gradients Gx and Gy, respectively, and then calculate the corresponding gradients Gx and Gy by atan(Gy / Gx). sample The angle can be calculated and converted to one of the 65 angle modes IPM in VVC. The sum of the absolute values ​​of Gx and Gy is then calculated as the cumulative intensity value for that angle.

[0205] All the centers of the template's regions sampleBy repeating the above process for , the intra-angle mode histogram can be obtained (see Figure 8).

[0206] 3.1.3. Select the intra prediction mode corresponding to the N highest histogram intensity values ​​from the histogram. From the intra angle mode histogram, intra prediction modes with the top N relatively large intensity values ​​(i.e., selected candidate angle modes) are selected. Here, the intra prediction modes with the top N relatively large intensity values ​​may be directly obtained, or a threshold may be set and the corresponding intra prediction modes may be selected as one of the N candidate prediction modes only if the intensity value is greater than or equal to the threshold, or an intensity value difference threshold may be set and the intra prediction modes corresponding to the intensity value may be selected as one of the N candidate prediction modes only if the difference between adjacent intensity values ​​is less than or equal to the intensity value difference threshold. The intensity value is the gradient magnitude described above.

[0207] 3.1.4. Based on N intra prediction modes, minMSE-based fusion weighted prediction is performed to obtain the predicted value. 3.1.4.1 Obtaining N prediction templates and N prediction blocks N intra prediction modes and neighboring sample , and obtain N predicted blocks predBlock of the current block. sample The current template is predicted based on , to obtain a predicted template refT.

[0208] 3.1.4.2 Calculate weights based on minimizing MSE using N prediction templates and the template of the current block (i.e., the template of the current block). In the embodiment of the minMSE-based DIPF technique of this main aspect, the prediction templates refT obtained in N prediction modes and the template refpredT to be predicted based on minMSE are sampleDerive the weights of weighted fusion by minimizing the MSE according to the value.

[0209] To make the process of deriving weights more flexible, this main mode adds an offset term when deriving the weights of weighted fusion. The offset value Bias can be any one constant within the range [0, (1 << bitDepth) - 1], and this main mode sets Bias to 1 << (bitDepth - 1). sample Since BiasTerm is a constant, in the actual calculation process, it needs to be extended to a matrix and represented as BiasTerm. Specifically,

[0210] for each template refT among the N prediction templates, when m = 0, 1,..., M - 1, where M is the number in the template, BiasTerm sample = Bias, BiasTerm m = Bias, for each prediction block predBlock corresponding to the N prediction templates refT, when x = 0, 1,..., nTbW - 1 and y = 0,..., nTbH - 1, BiasTerm x,y = Bias.

[0211] After adding the offset term, this main mode needs to derive N + 1 weighted weights. For the sake of easy explanation, here the variable P (where P = N + 1) is used to record the number of weighted weights, and the prediction template sample / offset term sample is collectively referred to as the candidate reference sample refT N and refT N+1 so that all reference quantities participating in the operation can be uniformly represented by refT p Similarly, the prediction block corresponding to the prediction template and the offset term are collectively referred to as the candidate prediction sample refBlock p where p = 0, 1,..., P - 1. The specific implementation is as follows. D

[0212] Specifically, in the process of minimizing MSE, the autocorrelation matrix of the first P candidate reference samples refT and the cross-correlation vector between the first P candidate reference samples refT and the adjacent template samples curT of the current coding block are input, and the weight of the prediction block corresponding to each candidate reference term is output.

[0213] The formula for calculating MSE is:

number

number

[0214] The weights w of the prediction blocks corresponding to each candidate reference sample are calculated by minimizing the MSE. p The specific steps to derive are as follows:

[0215] (1) First, w p Find the partial derivative with respect to and set it to zero.

number

number

[0216] (2) Candidate reference sample regions refT0, refT1, ... refT p-1 After determining, the equation obtained in step (1) can be expanded into matrix form to give

number

[0217] (3) Since the autocorrelation matrix and cross-correlation vector in (2) are both known quantities, solving the linear equation in (2) gives the weights W0, ...W of the predicted samples corresponding to each candidate reference term. p-1 can be calculated.

[0218] 3.1.4.3 Weighted prediction using N prediction blocks and weights Based on each candidate prediction block (i.e., P first prediction blocks) and their corresponding weighted fusion weights, a final prediction block (i.e., second prediction block) is calculated. Specifically, the value of each candidate prediction block is multiplied by its corresponding weight and accumulated to obtain the current prediction block (i.e., weighted prediction). The calculation formula is as follows:

[0219] For x=0...nTbW-1, y=0...nTbH-1, the predicted value is calculated as follows:

number

[0220] Each predicted value predSample x,y Once spatially stored, it becomes the output prediction block of the minMSE-based DIPF.

[0221] 3.2. Main Aspects of minMSE and Template-based Intra Prediction Fusion (minMSE and Template-based TIPF) The specific technical aspects of TIPF are as follows:

[0222] This main aspect of the TIPF technology is mainly improving steps 1403 and 1404 in the flowchart of TIMD prediction. The flowchart of TIPF prediction is shown in Figure 21, and the specific implementation of step 2104 is shown in Figure 22. Here, all the templates in step 2103 are sample The predicted value of is the sample value of the template prediction template, sample The reconstructed values ​​are the sample values ​​of the template, and the N types of modes are the N types of intra prediction modes.

[0223] 3.2.1 Adjacent Templates and Adjacent References to Templates sample Get The adjacent template to be obtained has a height of L1 and a width of L2. In TIMD, L1=L2=4, but the values ​​of L1 and L2 may be adaptively selected based on the block size and transmitted via the bitstream. sample are the top left, top, top right, left, and bottom left regions of the adjacent template. If available, they are acquired. If not available, they are not acquired. sample is a reference that is not adjacent to this sample For example, the second line from the top of the template sample The adjacent template to be obtained and the adjacent reference of the template may be sample A schematic diagram of this is shown in Figure 5. The shape of the template is various, and it may be L-shaped, of which the L-shaped template may include an upper left, upper and left template, or an upper left, upper, upper right, left and lower left template, or only an upper template or only a left template, or only an upper template and a left template.

[0224] 3.2.2, Traversing the MPM list and sample Predict and get a prediction template VVC constructs a six-candidate MPM list by two adjacent CUs (left adjacent block and top adjacent block, denoted by A and B in FIG. 23).

[0225] The rules for constructing an MPM list are as follows:

[0226] If the left and top are at the same angle,

number

[0227] If both left and top are non-angled,

number

[0228] If one of left and top is an angle and the other is a non-angle, then max is the larger of the two angles,

number

[0229] If both left and top are angles and are different from each other, then max is the larger angle of them; If the absolute value of the difference between the left and top is 1,

number

number

number

number

[0230] It is worth noting that in addition to the MPM list, some modifications and extensions to the MPM list may be made, e.g. The mode derived by DIMD may be placed before or after the angular mode derived by the neighboring CU, or the Planar mode may be removed and the DC mode may be placed after the angular mode derived by the neighboring CU, and some angular prediction modes may be added, for example, by expanding for relatively large and relatively small angles in the list, for example, by setting Max to +4 and Min to -4.

[0231] 3.2.3, all in the template sample and the predicted value of sample Calculate the sum of SATD between the reconstructed values ​​of and select the N modes with the smallest SATD. The SATD calculation formula is:

number

[0232] Here, the threshold value can be set to, for example, Ther, and only prediction modes whose template matching costs are smaller than or equal to Ther are determined as candidate prediction modes.

[0233] 3.2.4. Perform minMSE-based fusion weighted prediction based on N intra prediction modes to obtain the predicted value of the current block. 3.2.4.1. Obtain N prediction templates and N prediction blocks N intra prediction modes and neighboring sample Use it to make predictions to obtain the predicted blocks predBlock of N current blocks, and based on the N intra prediction modes and adjacent sample of the template, predict the current template to obtain the predicted template refT.

[0234] In this embodiment of the main mode TIPF technology, the predicted template refT obtained in N prediction modes and the template refpredT to be predicted based on minMSE sample values are used to minimize the MSE to derive the weighted fusion weights.

[0235] To make the weight derivation process more flexible, this main mode adds an offset term when deriving the weighted fusion weights. The offset value Bias can be any one constant within the image sample range [0, (1<<bitDepth)-1], and this main mode sets Bias to 1<<(bitDepth-1).

[0236] Since BiasTerm is a constant, in the actual calculation process, it needs to be extended to a matrix and represented as BiasTerm. Specifically, For each template refT among the N predicted templates, when m = 0, 1... M-1 and M is the number in the template sample number, BiasTerm m = Bias, For each predicted block predBlock corresponding to the N predicted templates refT, when x = 0, 1... nTbW-1 and y = 0... nTbH-1, BiasTerm x,y = Bias.

[0237] After adding the offset term, this main aspect needs to derive N+1 weights. For ease of explanation, we use a variable P (where P=N+1) to record the number of weights, and we also use the predicted template sample / offset term sample as the candidate reference sample refT. N and refT N+1 , and all reference quantities participating in the operation are refT p Similarly, the prediction block and offset term corresponding to the prediction template can be expressed as the candidate prediction sample refBlock p where p=0, 1..., P-1. The specific implementation is as follows:

[0238] Specifically, in the process of minimizing MSE, the autocorrelation matrix of the first P candidate reference samples refT and the cross-correlation vector between the first P candidate reference samples refT and the adjacent template samples curT of the current coding block are input, and the weight of the prediction block corresponding to each candidate reference term is output.

[0239] The formula for calculating MSE is: of the same position in each candidate reference template sample , i.e., for m=0, 1..., M-1,

number

number

[0240] The weights w of the prediction blocks corresponding to each candidate reference sample are calculated by minimizing the MSE. p The specific steps to derive and obtain are as follows:

[0241] (1) First, w p Find the partial derivative with respect to and set it to 0,

number

number

[0242] (2) Candidate reference sample regions refT0, refT1, ... refT P-1 After determining, the equation obtained in step (1) can be expanded into matrix form to give

number

[0243] (3) Since the autocorrelation matrix and cross-correlation vector in (2) are both known quantities, solving the linear equation in (2) gives the weights w0, ... w0 of the predicted samples corresponding to each candidate reference term. P-1 can be obtained by calculating

[0244] 3.1.4.3 Weighted prediction using N prediction blocks and weights The final predicted block is calculated based on each candidate predicted block and its corresponding weighted fusion weight. Specifically, the value of each candidate predicted block is multiplied by its corresponding weight and accumulated to obtain the current predicted block (i.e., weighted prediction). The calculation formula is as follows:

[0245] For x=0...nTbW-1, y=0...nTbH-1, the predicted value is calculated as follows:

number

[0246] Each predicted value predSample x,y Once spatially stored, it becomes the output prediction block of the TIPF.

[0247] 3.3. Main Aspects of minMSE and Template-based Multiple Reference Line Intra Prediction Fusion (minMSE-based TMRLF) The main technical aspects of TMRLF are as follows:

[0248] The TMRLF technology of this main aspect mainly improves steps 1504 and 1505 in the TMRL prediction flowchart. The TMRLF prediction flowchart is shown in Figure 24, where the intra prediction mode list is an example of the mode list described above, the multi-reference candidate list is the current block template reference row list described above, and the TMRL combination list is an example of the combination list described above. A specific implementation of step 2405 is shown in Figure 22.

[0249] 3.3.1, adjacent first row / column sample as a template and multi-reference row sample to build a multi-reference candidate list The template of the current block and the plurality of reference rows are shown in FIG. 10B. In TMRL, the template of the current block is the first row and the first column adjacent to the current block. sample In addition, the template of the current block is the first M rows and first M columns adjacent to the current block. sample The shape of the template may be various, and may be L-shaped, of which the L-shaped template may include an upper left, upper, and left template, or an upper left, upper, upper right, left, and lower left template, or may use only the upper template or only the left template, or may include only the upper template and the left template.

[0250] 3.3.2 Expanding the MPM List to Build an Intra-Prediction Mode Candidate List The MPM list is extended in length by TMRL to 10 and differs from the existing MPM list in that planar mode is excluded from the proposed intra-prediction mode candidate list, DC mode (if not already included) is added after the modes of the five adjacent PUs and DIMD mode, and angular modes with angle differences of ±1 to ±4 are added.

[0251] In addition, the expanded MPM list may be further expanded or modified to an intra-prediction mode candidate list of length M, for example: The mode derived by the DIMD may be placed before or after the angular mode derived by the adjacent CU, or the planar mode may be retained and the DC mode may be placed after the angular mode derived by the adjacent CU.

[0252] 3.3.3. Building a TMRL combination list For the current block, TMRL uses five extended reference rows {1, 3, 5, 7, 12}, and the extended MPM list contains a total of 10 prediction modes. Therefore, a TMRL combination list can be constructed, and the combination list contains a total of 5 × 10 = 50 combinations.

[0253] Additionally, the TMRLF may extend or modify the candidate reference row list to a candidate reference row list of length N, which may be {1, 4, 5, 7, 10, 12} or {1, 2, 3, 7, 10, 11} when N=6, in which case the combination list includes a total of N×M combinations, where M is the length of the intra-prediction mode candidate list.

[0254] 3.3.4. Traverse the TMRL combination list, calculate the SAD between the predicted value and the reconstructed value of the template of the current block, and select the top N combinations with the smallest SAD to form the final TMRL combination list. The formula for calculating SAD is as follows:

number

[0255] Here, the threshold value can be set to, for example, Ther, and only prediction modes whose template matching costs are smaller than or equal to Ther are set as candidate prediction modes.

[0256] 3.3.5. Perform minMSE-based fusion weighted prediction based on N combinations to obtain the predicted value of the current block. 3.3.5.1. Obtain N prediction templates and N prediction blocks N intra prediction modes and neighboring sample Prediction is performed using sample The current template is predicted based on , to obtain a predicted template refT.

[0257] 3.3.5.2. Calculating weights based on minMSE-based fusion using N prediction templates and the template of the current block In the embodiment of the TIPF technique of this main aspect, the prediction template refT obtained in N prediction modes and the template refpredT to be predicted based on minMSE are sample The weighted fusion weights are derived by minimizing the MSE using the values.

[0258] To make the weight derivation process more flexible, this main embodiment adds an offset term when deriving the weighted fusion weight. The offset value Bias can be any constant within the image sample range [0, (1 << bitDepth) - 1], and this main embodiment sets Bias to 1 << (bitDepth - 1).

[0259] Since the BiasTerm is a constant, it needs to be extended to a matrix and represented as BiasTerm in the actual calculation process. Specifically, For each template refT among the N prediction templates, when m = 0, 1,..., M - 1, and M is the number in the template sample is the case, BiasTerm m = Bias, and For each prediction block predBlock corresponding to the N prediction templates refT, when x = 0, 1,..., nTbW - 1 and y = 0,..., nTbH - 1, BiasTerm x,y = Bias.

[0260] After adding the offset term, this main embodiment needs to derive N + 1 weighted weights. For ease of explanation, here the variable P is used to record the number of weighted weights, where P = N + 1, and the prediction template sample / offset term sample is collectively referred to as the candidate reference sample refT N and refT N+1 and all reference quantities participating in the operation can be uniformly represented by refT p Similarly, the prediction block corresponding to the prediction template and the offset term are collectively referred to as the candidate prediction sample refBlock p and p = 0, 1,..., P - 1. The specific implementation is as follows.

[0261] Specifically, in the process of minimizing MSE, the autocorrelation matrix of the first P candidate reference samples refT and the cross-correlation vector between the first P candidate reference samples refT and the adjacent template samples curT of the current coding block are input, and the weight of the prediction block corresponding to each candidate reference term is output.

[0262] The formula for calculating MSE is: of the same position in each candidate reference template sample , i.e., for m=0, 1..., M-1,

number

number

[0263] The weights w of the prediction blocks corresponding to each candidate reference sample are calculated by minimizing the MSE. p The specific steps to derive and obtain are as follows:

[0264] (1) First, w p Find the partial derivative with respect to and set it to 0,

number

number

[0265] (2) Candidate reference sample regions refT0, refT1, ... refT P-1 After determining, the equation obtained in step (1) can be expanded into matrix form to give

number

[0266] (3) Since the autocorrelation matrix and cross-correlation vector in (2) are both known quantities, solving the linear equation in (2) gives the weights w0, ... w0 of the predicted samples corresponding to each candidate reference term. P-1 can be obtained by calculating

[0267] 3.3.5.3 Weighted prediction using N prediction blocks and weights The final predicted block is calculated based on each candidate predicted block and its corresponding weighted fusion weight. Specifically, the value of each candidate predicted block is multiplied by its corresponding weight and accumulated to obtain the current predicted block (i.e., weighted prediction). The calculation formula is as follows:

[0268] For x=0...nTbW-1, y=0...nTbH-1, the predicted value is calculated using the following formula:

number

[0269] Each predicted value predSample x,y When spatially stored, it becomes the output prediction block of the TMRLF.

[0270] 3.4. Main Aspects of minMSE and Template-based Inter Weighted Prediction Fusion (minMSE and Template-based IWPF) The main technical aspects of IWPF are as follows:

[0271] The IWPF technique of this main aspect mainly improves steps 1801 to 1803 in the flowchart of inter-weighted prediction. The IWPF prediction flowchart is shown in Figure 25, where the neighboring template of the reference block shown in Figure 25 is an example of the prediction template mentioned above.

[0272] 3.4.1 Obtain N candidate MVs In the inter prediction process, an MV candidate list including a Merge candidate list and an AMVP candidate list is constructed, and in this main aspect, the first N candidate MVs are obtained. Note that the candidate MVs here may be MVs before or after being subdivided by template matching TM, or MVs before or after being subdivided by DMVR or Multipass DMVR.

[0273] 3.4.2 Obtaining N reference blocks based on N candidate MVs Motion compensation is performed based on the N candidate MVs to obtain N reference blocks from the reference frame, which are denoted as refBlock.

[0274] 3.4.3 Obtain the neighboring templates of the current block and N reference blocks A neighboring template curT of the current block and neighboring templates refT of N reference blocks are obtained, and the template sizes are TemplateSizeW and TemplateSizeH. The template may have various shapes, such as an L-shape, and the L-shaped template may include an upper left, upper, and left template, or an upper left, upper, upper right, left, and lower left template, or only an upper template or only a left template, or only an upper template and a left template, etc. A schematic diagram is shown in Figure 26.

[0275] 3.4.4 Calculate weights based on minMSE using N reference block templates and the current block template In this embodiment of the IWPF technique, the templates refT of N reference blocks and the template refpredT to be predicted based on minMSE are sample The weights for weighted fusion are derived by minimizing the MSE using the values.

[0276] To make the weight derivation process more flexible, this main mode adds an offset term when deriving the weights of weighted fusion. The offset value Bias can be any constant within the range of the image sample range [0, (1 << bitDepth) - 1], and this main mode sets Bias to 1 << (bitDepth - 1).

[0277] Since BiasTerm is a constant, it needs to be extended to a matrix and represented as BiasTerm in the actual calculation process. Specifically, For each template refT among the N reference templates, when m = 0, 1,..., M - 1, and M is the number in the template sample count, BiasTerm m = Bias, For each reference block refBlock corresponding to the N reference templates refT, when x = 0, 1,..., nTbW - 1 and y = 0,..., nTbH - 1, BiasTerm x,y = Bias.

[0278] After adding the offset term, this main mode needs to derive N + 1 weighted weights. For the sake of easy explanation, here the variable P (where P = N + 1) is used to record the number of weighted weights, and the reference template sample / offset term sample is collectively referred to as the candidate reference sample refT N and refT N+1 and, thus, all the reference quantities participating in the operation can be uniformly represented by refT P Similarly, the reference block corresponding to the reference template and the offset term are collectively referred to as the candidate reference sample refBlock p and, among them, p = 0, 1,..., P - 1. The specific implementation is as follows.

[0279] Specifically, in the process of minimizing MSE, the autocorrelation matrix of the first P candidate reference samples refT and the cross-correlation vector between the first P candidate reference samples refT and the adjacent template samples curT of the current coding block are input, and the weight of the prediction block corresponding to each candidate reference term is output.

[0280] The formula for calculating MSE is: of the same position in each candidate reference template sample , i.e., for m=0, 1..., M-1,

number

number

[0281] The weights w of the prediction blocks corresponding to each candidate reference sample are calculated by minimizing the MSE. p The specific steps to derive and obtain are as follows:

[0282] (1) First, w p Find the partial derivative with respect to and set it to 0,

number

number

[0283] (2) Candidate reference sample regions refT0, refT1, ... refT P-1 After determining, the equation obtained in step (1) can be expanded into matrix form to give

number

[0284] (3) Since the autocorrelation matrix and cross-correlation vector in (2) are both known quantities, solving the linear equation in (2) gives the weights w0, ...W of the reference samples corresponding to each candidate reference term. P-1 can be obtained by calculating

[0285] 3.4.5 Perform weighted prediction based on N reference blocks and weights to obtain predicted values The final predicted block is calculated based on each candidate reference block and its corresponding weighted fusion weight. Specifically, the value of each candidate reference block is multiplied by its corresponding weight and accumulated to obtain the current predicted block (i.e., weighted prediction). The calculation formula is as follows:

[0286] For x=0...nTbW-1, y=0...nTbH-1, the predicted value is calculated as follows:

number

[0287] Each predicted value predSample x,y When spatially stored, it becomes the output prediction block of the IWPF.

[0288] This main aspect proposes DIPF, TIPF, TMRLF, and IWPF techniques based on minimizing MSE, respectively, for the conventional DIMD, TIMD, TMRL, and inter-weighted prediction techniques. DIPF, TIPF, and TMRLF can improve the accuracy of intra prediction, and IWPF can improve the accuracy of inter prediction.

[0289] For DIPF, adjacent templates sampleA histogram is obtained by performing HoG calculations using the formula: where different histogram bars correspond to different intra-prediction modes. Next, the top N intra-prediction modes with the highest histogram bar values ​​are selected to predict the template and the current block, resulting in N prediction templates and N predicted blocks. Furthermore, weights for different prediction modes are obtained based on minimizing MSE using the N prediction templates and the current template, and weighted prediction is performed using the N predicted blocks and weights to obtain a final predicted block. This method fully considers the modes corresponding to different histogram bars, rather than simply considering the modes corresponding to the top two highest histogram bars. By adaptively assigning weights to predicted blocks based on different prediction template information, the importance of different predicted block information to the prediction of the current block is fully taken into account. When tested under the All Intra condition with ECM7.0 at 48-frame intervals, this method achieved BD-rate changes of -0.xx%, -0.xx%, and -0.xx% for the Y, Cb, and Cr components, respectively, with a 48-frame interval.

[0290] For TIPF, adjacent templates sample and template references sample traverses the intra prediction modes in the MPM list by obtaining N prediction templates and sampleThe method traverses the intra-prediction modes in the MPM list using the weighting coefficients to obtain N predicted blocks. Then, using the N prediction templates and the current template, weights for different prediction modes are obtained based on minimizing MSE, and weighted prediction is performed using the N predicted blocks and weighting coefficients to obtain the final predicted block. This method adaptively assigns weights to the predicted blocks based on different prediction template information, fully considering the different importance of information from different predicted blocks to the prediction of the current block. When tested with ECM7.0 under the All Intra condition at 48-frame intervals, this method achieved a BD-rate change (i.e., average bitrate change at the same psnr) of -0.xx%, -0.xx%, and -0.xx% for Y, Cb, and Cr, respectively.

[0291] For TMRLF, a reference row candidate list and an intra-prediction mode candidate list are constructed, and then these are combined to obtain a TMRL combination list. sample The TMRL combination list is traversed using

[0000] to obtain N prediction templates with small SADs, and the current block is predicted using N prediction modes in these N combinations to obtain N predicted blocks. Next, weights for different prediction modes are obtained using the N prediction templates and the current template based on minimizing MSE, and weighted prediction is performed using the N predicted blocks and weights to obtain the final predicted block. This method adaptively assigns weights to the predicted blocks based on different prediction template information, fully considering the different importance of information from different predicted blocks to the prediction of the current block. When tested with ECM7.0 under the All Intra condition at 48-frame intervals, this method achieved a BD-rate change (i.e., average bitrate change at equivalent psnr) of -0.xx%, -0.xx%, and -0.xx% for Y, Cb, and Cr, respectively.

[0292] For IWPF, N candidate motion vectors are obtained and motion compensated to obtain N candidate reference blocks. Neighboring templates of the current block and neighboring templates of the N candidate reference blocks are obtained. Weighted weights for the N reference blocks are obtained based on minimizing MSE using the N reference block templates and the current template. Weighted prediction is then performed using the N reference blocks and weights to obtain a final predicted block. This method fully considers reference block information in different motion information and adaptively assigns weights to reference blocks based on different reference block template information, fully considering the different importance of different reference block information to the prediction of the current block. When tested under Random Access (RA) conditions in ECM7.0, this method achieved a BD-rate change (i.e., average bitrate change at the same psnr) of -0.xx%, -0.xx%, and -0.xx% for Y, Cb, and Cr, respectively.

[0293] The innovation of the DIPF, TIPF, TMRLF and IWPF techniques based on MSE minimization in this main aspect is that they all utilize different reference block template information to adaptively assign weights to reference blocks based on MSE minimization.

[0294] (1) DIPF fully utilizes the information of predicted blocks corresponding to different prediction templates in different intra-prediction modes, and calculates the weights of predicted blocks corresponding to different prediction templates by minimizing the MSE based on the prediction template and the current template. (2) TIPF fully utilizes the information of predicted blocks corresponding to different prediction templates in different intra-prediction modes, and calculates the weights of predicted blocks corresponding to different prediction templates by minimizing the MSE based on the prediction template and the current template. (3) TMRLF fully utilizes the information of predicted blocks corresponding to different prediction templates in different intra-prediction modes and different reference rows, and calculates the weights of predicted blocks corresponding to different prediction templates by minimizing the MSE based on the prediction template and the current template. (4) IWPF fully utilizes the reference block information corresponding to different reference templates in different MV information, and calculates the reference block weights corresponding to different reference block templates by minimizing the MSE based on the reference block template and the current block template.

[0295] Expansion Plan 1 Of the above four techniques, in the weight calculation part, that is, when calculating the weight using the minMSE method, it is possible to not add an offset term, or to add other modes, such as planar or DC mode, and use the predicted value obtained as the offset term (mode overlap check is required), or to add a nonlinear term, for example, to select the template that minimizes the template matching cost. sample It may be the square of .

[0296] For TIPF, the SATD selection policy in TIPF may be changed, and the first N prediction modes may be selected using, for example, an SAD function, an MSE function, an SSE function, or the like.

[0297] For the SAD selection policy in TMRLF, other functions may be used to select the first N prediction modes, such as a SATD function, an MSE function, an SSE function, etc.

[0298] Expansion Plan 3 In the case of bidirectional weighted prediction, as opposed to inter prediction, the predicted block in each direction may be a predicted block obtained based on minMSE weighting.

[0299] Expansion Plan 4 The min-MSE-based fusion method can be applied to other techniques that require weighted fusion, improving its adaptability. Other conventional techniques that include weighted fusion include CIIP, Chroma Fusion, OBMC, SGPM, GPM, and MHP.

[0300] For example, in the CIIP technique, the weights of the intra-mode predicted block and the inter-mode predicted block of the current block are fixed. sample is used as a template, the template is predicted according to the current intra prediction mode to obtain an intra prediction template, and the neighboring template of the inter reference block is obtained according to the MV information. Based on the intra prediction template, the neighboring template of the inter reference block, and the template of the current block, the weights of the intra prediction block and the inter prediction block are calculated according to min-MSE.

[0301] For example, with regard to Inter-MHP technology, the neighboring sample is used as a template, and predicted values ​​of the current template are obtained by different inter prediction modes, which are called prediction templates. Based on the prediction template and the current template, weights of predicted blocks obtained by different inter prediction modes are calculated, and weighting is performed on the predicted blocks.

[0302] Based on the above-mentioned embodiment, the encoding device of the embodiment of the present invention is applied to an encoder. FIG. 27 is a structural schematic diagram of the encoding device according to the embodiment of the present invention. As shown in FIG. 27, the encoding device 27: a first determination module 271 configured to determine a template of a current block and one or more prediction templates of the template of the current block; a second determination module 272 configured to determine weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; a third determination module 273 configured to determine one or more first prediction blocks of the current block based on prediction parameters of the current block; and a first fusion module 274 configured to fuse the one or more first prediction blocks based on weights of the one or more prediction templates to obtain a second prediction block of the current block.

[0303] In some embodiments, the third determination module 273 is configured to determine a first prediction block for the current block based on an intra prediction mode indicated by the prediction parameters.

[0304] In some embodiments, the third determination module 273 is configured to determine a first prediction block of the current block based on the motion parameters or BV of the current block indicated by the prediction parameters.

[0305] In some embodiments, the motion parameters include at least one of the following parameters: MV, reference image index.

[0306] In some embodiments, the prediction parameters of the current block are the same as the prediction parameters of the template of the current block.

[0307] In some embodiments, the first determination module 271 is configured to perform HoG calculation on samples of the template of the current block, obtain gradient directions and gradient magnitudes of corresponding samples, determine an angle mode based on the gradient directions and gradient magnitudes, and predict the template of the current block based on the angle mode to obtain the predicted template.

[0308] In some embodiments, the first determination module 271 is configured to convert the gradient directions into predefined candidate angular modes and determine the angular modes from the converted candidate angular modes based on the gradient magnitudes.

[0309] In some embodiments, the first determination module 271 is configured to obtain N candidate angle modes based on the candidate angle modes corresponding to the N largest gradient strengths, and / or to obtain one or more of the angle modes based on candidate angle modes corresponding to gradient strengths greater than or equal to a first threshold, and / or to obtain one or more of the angle modes based on candidate angle modes corresponding to second gradient strengths whose difference value from at least one first gradient strength is less than or equal to a second threshold.

[0310] In some embodiments, the magnitude of the at least one first gradient strength is adjacent to the magnitude of the second gradient strength.

[0311] In some embodiments, the first determination module 271 is configured to predict a template of the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template, and to obtain the prediction template based on a sample value error between the candidate prediction template and the template of the current block.

[0312] In some embodiments, the first determination module 271 is configured to obtain N prediction templates based on candidate prediction templates corresponding to the N smallest sample value errors, or to obtain one or more of the prediction templates based on candidate prediction templates corresponding to sample value errors less than or equal to a third threshold.

[0313] In some embodiments, the sample error type is at least one of SATD, SAD, MAD, MAE, NCC, MSE, and SSE.

[0314] In some embodiments, the candidate prediction modes in the mode list are: the angular modes obtained in the steps described above; A prediction mode in the MPM list, The prediction modes in the MPM list after the Planar mode has been removed, and An angle mode obtained by expanding an angle mode in the MPM list whose angle is greater than or equal to a fourth threshold; The MPM list includes at least one of an angle mode obtained by expanding an angle mode that is smaller than or equal to a fifth threshold.

[0315] In some embodiments, in the MPM list, a DC mode is after or before an angular mode constructed based on neighboring blocks of the current block.

[0316] In some embodiments, in the mode list, the N angular modes are after or before an angular mode constructed based on neighboring blocks of the current block.

[0317] In some embodiments, the first determination module 271 is configured to predict the template of the current block based on the sample values ​​of the reference region of the template of the current block and the candidate prediction mode to obtain a candidate prediction template, where the reference region includes a non-adjacent region and / or an adjacent region of the template of the current block.

[0318] In some embodiments, the reference region includes the top left region, the top region, the top right region, the left region, and / or the bottom left region of the template of the current block.

[0319] In some embodiments, the first determination module 271 is configured to combine a candidate prediction mode in the mode list and a reference row in the reference row list of the template of the current block to obtain a combination list, and predict the template of the current block based on the candidate prediction mode and the reference row indicated by the combination in the combination list to obtain a candidate prediction template.

[0320] In some embodiments, the reference row indices included in the reference row list are {1,3,5,7,12}, {1,4,5,7,10,12}, or {1,2,3,7,10,11}.

[0321] In some embodiments, the first determination module 271 is configured to determine at least one candidate MV, and perform motion compensation on a template of the current block based on the candidate MV and a reference image of the current block to obtain the prediction template.

[0322] In some embodiments, the reference image of the current block is a forward reference image or a backward reference image of the current block.

[0323] In some embodiments, the first determination module 271 is configured to obtain at least one candidate MV based on a Merge candidate list or an AMVP candidate list.

[0324] In some embodiments, the first determination module 271 is configured to obtain the first N candidate MVs in the Merge candidate list or the AMVP candidate list.

[0325] In some embodiments, the candidate MVs in the Merge candidate list or AMVP candidate list are MVs subdivided by template matching, DMVR, and / or Multipass DMVR, and / or candidate MVs sorted according to a predetermined order relative to those in the original candidate list.

[0326] In some embodiments, the first determination module 271 is configured to determine at least one candidate BV, and determine a prediction template of a template of the current block based on the candidate BV, the current block, and the current image.

[0327] In some embodiments, the first determination module 271 is configured to obtain at least one candidate BV based on a Merge candidate list, an AMVP candidate list, and / or a preset value.

[0328] In some embodiments, the first determination module 271 is configured to obtain the first N candidate BVs in the Merge candidate list or the AMVP candidate list.

[0329] In some embodiments, the preset value is equal to zero.

[0330] In some embodiments, the one or more prediction templates further include a first offset term, wherein the first offset term is a template generated based on a sample value bit depth of a template of the current block; The one or more first prediction blocks further include a second offset term, where the second offset term is a block generated based on a sample value bit depth of the current block.

[0331] In some embodiments, the one or more prediction templates further include a third offset term, which is a prediction template obtained by predicting a template of the current block based on a specific prediction mode or a specific MV, and the one or more first prediction blocks further include a fourth offset term, which is a prediction block obtained by predicting the current block based on the specific prediction mode or the specific MV.

[0332] In some embodiments, the particular prediction mode includes a planar mode and / or a DC mode.

[0333] In some embodiments, the first determination module 271 is further configured to predict a template for the current block based on the particular prediction mode in response to determining that the particular prediction mode is different from the intra prediction mode indicated by the prediction parameters.

[0334] In some embodiments, the first determination module 271 is further configured to predict a template for the current block based on the specific MV in response to determining that the specific MV is different from the MV of the current block indicated by the prediction parameters.

[0335] In some embodiments, the one or more prediction templates further include a first nonlinear term, which is a prediction template obtained by performing a transformation on a candidate prediction template having the smallest sample value error, and the one or more first prediction blocks further include a second nonlinear term, which is a prediction block obtained by performing the transformation on a first prediction block corresponding to the candidate prediction template having the smallest sample value error.

[0336] In some embodiments, if the one or more prediction templates are less than N, the prediction templates are replenished to N by adding at least one of the first offset term, the third offset term, the first nonlinear term, and a first preset value, and correspondingly, the first prediction blocks are replenished to N by adding at least one of the second offset term, the fourth offset term, the second nonlinear term, and a second preset value.

[0337] In some embodiments, the first preset value is equal to zero.

[0338] In some embodiments, the transformation comprises a squaring operation.

[0339] In some embodiments, the one or more prediction templates further include a prediction template obtained by predicting a template of the current block based on an intermediate prediction mode before each fusion operation in the Chroma Fusion, OBMC, MHP, and / or SGPM methods.

[0340] In some embodiments, the second determination module 272 is configured to determine weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates, so as to minimize a sample value error between the template of the current block and a target prediction value of the template of the current block, wherein the target prediction value is equal to a weighted sum of sample values ​​of the one or more prediction templates.

[0341] In some embodiments, the second determination module 272 is configured to determine an autocorrelation matrix of the prediction template based on sample values ​​of the prediction template, determine a cross-correlation vector between the prediction template and the template of the current block based on sample values ​​of the prediction template and sample values ​​of the template of the current block, and determine weights of the one or more prediction templates based on the autocorrelation matrix and the cross-correlation vector.

[0342] In some embodiments, the sample error type is one of MSE, SATD, SAD, MAD, MAE, NCC, and SSE.

[0343] In some embodiments, the first fusion module 274 is further configured to determine a residual block of the current block based on the second prediction block of the current block, and generate a bitstream based on the residual block.

[0344] Based on the above-mentioned embodiment, an embodiment of the present invention provides a decoding device applied to a decoder. FIG. 28 is a structural diagram of a decoding device according to an embodiment of the present invention. As shown in FIG. 28, the decoding device 28 includes: a fourth determination module 281 configured to determine a template of a current block and one or more prediction templates of the template of the current block; a fifth determination module 282 configured to determine weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; a sixth determining module 283 configured to determine one or more first prediction blocks of the current block based on prediction parameters of the current block; a second fusion module 284 configured to fuse the one or more first prediction blocks based on weights of the one or more prediction templates to obtain a second prediction block of the current block; and a seventh determination module 285 configured to determine a reconstructed value of the current block based on the second predicted block.

[0345] In some embodiments, the seventh determination module 285 is configured to decode the bitstream, determine a residual block corresponding to the current block, and determine a reconstructed value of the current block based on the residual block and the second predictive block.

[0346] In some embodiments, the sixth determination module 283 is configured to determine a first prediction block for the current block based on an intra prediction mode indicated by the prediction parameters.

[0347] In some embodiments, the sixth determination module 283 is configured to determine a first prediction block of the current block based on the motion parameters of the current block indicated by the prediction parameters.

[0348] In some embodiments, the motion parameters include at least one of the following parameters: a motion vector, a reference image index.

[0349] In some embodiments, the prediction parameters of the current block are the same as the prediction parameters of the template of the current block.

[0350] In some embodiments, the fourth determination module 281 is configured to perform HoG calculation on samples of the template of the current block to obtain gradient directions and gradient magnitudes of corresponding samples, determine an angle mode based on the gradient directions and gradient magnitudes, and predict the template of the current block based on the angle mode to obtain the predicted template.

[0351] In some embodiments, the fourth determination module 281 is configured to convert the gradient directions into predefined candidate angular modes and determine the angular modes from the converted candidate angular modes based on the gradient magnitudes.

[0352] In some embodiments, the fourth determination module 281 is configured to obtain the N angle modes based on the candidate angle modes corresponding to the N largest gradient strengths, and / or to obtain one or more of the angle modes based on the candidate angle modes corresponding to gradient strengths greater than or equal to a first threshold, and / or to obtain one or more of the angle modes based on the candidate angle modes corresponding to second gradient strengths whose difference value from at least one first gradient strength is less than or equal to a second threshold.

[0353] In some embodiments, the magnitude of the at least one first gradient strength is adjacent to the magnitude of the second gradient strength.

[0354] In some embodiments, the fourth determination module 281 is configured to predict a template of the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template, and to obtain the prediction template based on a sample value error between the candidate prediction template and the template of the current block.

[0355] In some embodiments, the fourth determination module 281 is configured to obtain N prediction templates based on candidate prediction templates corresponding to the N smallest sample value errors, or to obtain one or more of the prediction templates based on candidate prediction templates corresponding to sample value errors less than or equal to a third threshold.

[0356] In some embodiments, the sample error type is at least one of SATD, SAD, MAD, MAE, NCC, MSE, and SSE.

[0357] In some embodiments, the candidate prediction modes in the mode list are: The angular modes obtained by the steps described above; A prediction mode in the MPM list, The prediction modes in the MPM list after the Planar mode has been removed, and An angle mode obtained by expanding an angle mode in the MPM list whose angle is greater than or equal to a fourth threshold; and angular modes obtained by expanding angular modes in the MPM list that are smaller than or equal to a fifth threshold.

[0358] In some embodiments, in the MPM list, a DC mode is after or before an angular mode constructed based on neighboring blocks of the current block.

[0359] In some embodiments, in the mode list, the N angular modes are after or before an angular mode constructed based on neighboring blocks of the current block.

[0360] In some embodiments, the fourth determination module 281 is configured to predict the template of the current block based on the sample values ​​of the reference region of the template of the current block and the candidate prediction mode to obtain a candidate prediction template, where the reference region includes a non-adjacent region and / or an adjacent region of the template of the current block.

[0361] In some embodiments, the reference region includes the top left region, the top region, the top right region, the left region, and / or the bottom left region of the template of the current block.

[0362] In some embodiments, the fourth determination module 281 is configured to combine a candidate prediction mode in the mode list and a reference row in the reference row list of the template of the current block to obtain a combination list, and predict the template of the current block based on the candidate prediction mode and the reference row indicated by the combination in the combination list to obtain a candidate prediction template.

[0363] In some embodiments, the reference row indices included in the reference row list are {1,3,5,7,12}, {1,4,5,7,10,12}, or {1,2,3,7,10,11}.

[0364] In some embodiments, the fourth determination module 281 is configured to determine at least one candidate MV, and perform motion compensation on a template of the current block based on the candidate MV and a reference image of the current block to obtain the prediction template.

[0365] In some embodiments, the reference image of the current block is a forward reference image or a backward reference image of the current block.

[0366] In some embodiments, the fourth determination module 281 is configured to obtain at least one candidate MV based on the Merge candidate list or the AMVP candidate list.

[0367] In some embodiments, the fourth determination module 281 is configured to obtain the first N candidate MVs in the Merge candidate list or the AMVP candidate list.

[0368] In some embodiments, the candidate MVs in the Merge candidate list or AMVP candidate list are MVs subdivided by template matching, DMVR, and / or Multipass DMVR, and / or the candidate MVs in the Merge candidate list or AMVP candidate list are candidate MVs sorted according to a predetermined order relative to those in the original candidate list.

[0369] In some embodiments, the fourth determination module 281 is configured to determine at least one candidate BV, and determine a prediction template of a template for the current block based on the candidate BV, the current block, and the current image.

[0370] In some embodiments, the fourth determination module 281 is configured to obtain at least one candidate BV based on a Merge candidate list, an AMVP candidate list, and / or a preset value.

[0371] In some embodiments, the fourth determination module 281 is configured to obtain the first N candidate BVs in the Merge candidate list or the AMVP candidate list.

[0372] In some embodiments, the preset value is equal to zero.

[0373] In some embodiments, the one or more prediction templates further include a first offset term, which is a template generated based on a sample value bit depth of the template of the current block, and the one or more first prediction blocks further include a second offset term, which is a block generated based on a sample value bit depth of the current block.

[0374] In some embodiments, the one or more prediction templates further include a third offset term, which is a prediction template obtained by predicting a template of the current block based on a specific prediction mode or a specific MV, and the one or more first prediction blocks further include a fourth offset term, which is a prediction block obtained by predicting the current block based on the specific prediction mode or the specific MV.

[0375] In some embodiments, the particular prediction mode includes a planar mode and / or a DC mode.

[0376] In some embodiments, the fourth determination module 281 is further configured to predict a template of the current block based on the particular prediction mode in response to determining that the particular prediction mode is different from the intra prediction mode indicated by the prediction parameters.

[0377] In some embodiments, the fourth determination module 281 is further configured to predict a template for the current block based on the specific MV in response to determining that the specific MV is different from the MV of the current block indicated by the prediction parameters.

[0378] In some embodiments, the one or more prediction templates further include a first nonlinear term, which is a prediction template obtained by performing a transformation on a candidate prediction template having the smallest sample value error, and the one or more first prediction blocks further include a second nonlinear term, which is a prediction block obtained by performing the transformation on a first prediction block corresponding to the candidate prediction template having the smallest sample value error.

[0379] In some embodiments, if the one or more prediction templates are less than N, the prediction templates are replenished to N by adding at least one of the first offset term, the third offset term, the first nonlinear term, and a first preset value, and correspondingly, the first prediction blocks are replenished to N by adding at least one of the second offset term, the fourth offset term, the second nonlinear term, and a second preset value.

[0380] In some embodiments, the first preset value is equal to zero.

[0381] In some embodiments, the transformation comprises a squaring operation.

[0382] In some embodiments, the one or more prediction templates further include a prediction template obtained by predicting a template of the current block based on an intermediate prediction mode before each fusion operation in the Chroma Fusion, OBMC, MHP, and / or SGPM methods.

[0383] In some embodiments, the fifth determination module 282 is configured to determine weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates, so as to minimize a sample value error between the template of the current block and a target prediction value of the template of the current block, wherein the target prediction value is equal to a weighted sum of sample values ​​of the one or more prediction templates.

[0384] In some embodiments, the fifth determination module 282 is configured to determine an autocorrelation matrix of the prediction template based on sample values ​​of the prediction template, determine a cross-correlation vector between the prediction template and the template of the current block based on sample values ​​of the prediction template and sample values ​​of the template of the current block, and determine weights of the one or more prediction templates based on the autocorrelation matrix and the cross-correlation vector.

[0385] In some embodiments, the sample error type is one of MSE, SATD, SAD, MAD, MAE, NCC, and SSE.

[0386] The above description of the embodiment of the encoding / decoding device is similar to the description of the embodiment of the encoding / decoding method described above, and has similar beneficial effects as the embodiment of the encoding / decoding method. For technical details not disclosed in the embodiment of the device of the present invention, please refer to the description of the embodiment of the encoding / decoding method of the present invention for understanding.

[0387] The division of the device into modules described in the embodiments of the present invention is merely an example and represents a division of logical functions, and other division modes may be adopted in actual implementation. Furthermore, each functional unit in each embodiment of the present invention may be integrated into a single processing unit, may exist physically separately, or two or more units may be integrated into a single unit. The integrated units may be realized in the form of hardware, may be implemented in the form of a software functional unit, or may be implemented in a form that combines software and hardware.

[0388] It should be noted that in the embodiments of the present invention, the above-described methods may be implemented in the form of software functional modules and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, essential parts of the technical solutions of the embodiments of the present invention or parts contributing to the relevant technology may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing an electronic device to execute all or part of the methods described in each embodiment of the present invention. The storage medium may include various media capable of storing program code, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk. Thus, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0389] An embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed, the encoder-side method or the decoder-side method is realized.

[0390] An embodiment of the present invention provides an encoder, and as shown in FIG. 29, the encoder 29 includes a first communication interface 291, a first memory 292, and a first processor 293, and each component is coupled together via a first bus system 294. It can be understood that the first bus system 294 is used to realize connection communication between these components. In addition to a data bus, the first bus system 294 also includes a power bus, a control bus, and a status signal bus. However, for clarity of explanation, all of the various buses in FIG. 29 are referred to as the first bus system 294. Among them, The first communication interface 291 is used to receive and transmit signals in the process of transmitting and receiving information to and from other external network elements; The first memory 292 is used to store a computer program executable by the first processor 293; The first processor 293, when running the computer program, is used to perform the decoding method described in the embodiment of the present invention.

[0391] It is understood that the first memory 292 in the embodiment of the present invention may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external high-speed cache. By way of example and not limitation, many types of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct Rambus random access memory (DRRAM). The first memory 292 of the systems and methods described herein includes, but is not limited to, these and any other suitable types of memory.

[0392] The first processor 293 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above-described method can be completed by a hardware integrated logic circuit in the first processor 293 or by instructions in software form. The first processor 293 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Each method, step, and logic block diagram disclosed in the embodiments of the present invention can be realized or executed. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor. The steps of the methods disclosed in the embodiments of the present invention may be directly executed by a hardware decoding processor or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium known to those skilled in the art, such as a random memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 292, and the first processor 293 reads the information in the first memory 292 and completes the steps of the above-mentioned method in combination with its hardware.

[0393] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. In a hardware implementation, the processing unit can be implemented as one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processing (DSPs), Digital Signal Processing Devices (DSP Devices, DSPDs), Programmable Logic Devices (PLDs), Field-Programmable Gate Arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, or other electronic units for performing the functions described herein, or a combination thereof. In a software implementation, the techniques described herein can be implemented by modules (e.g., processes, functions, etc.) that perform the functions described herein. The software code is stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

[0394] Preferably, in another embodiment, the first processor 293 is further configured to execute the computer program to perform any of the encoder-side method embodiments described above.

[0395] An embodiment of the present invention provides a decoder, and as shown in FIG. 30, the decoder 300 includes a second communication interface 3001, a second memory 3002, and a second processor 3003, and each component is coupled via a second bus system 3004. It can be understood that the second bus system 3004 is used to realize connection communication between these components. The second bus system 3004 includes a power bus, a control bus, and a status signal bus in addition to a data bus. However, for clarity of explanation, all of the various buses are referred to as the second bus system 3004 in FIG. 300. Here, The second communication interface 3001 is used to receive and transmit signals in the process of transmitting and receiving information to and from other external network elements; The second memory 3002 is used to store a computer program executable by the second processor 3003; The second processor 3003, when running the computer program, is used to perform the encoding method described in the embodiment of the present invention.

[0396] It will be understood that the second memory 3002 has similar hardware functions to the first memory 3002, and the second processor 3003 has similar hardware functions to the first processor 293, and will not be described in detail again here.

[0397] An embodiment of the present invention provides an electronic device, the electronic device including a processor and a computer-readable storage medium, the processor being adapted to execute a computer program, the computer-readable storage medium storing the computer program, and when the computer program is executed by the processor, the encoding method and / or decoding method according to the embodiment of the present invention are realized. The electronic device may be any type of device with video encoding and / or video decoding capabilities, for example, a mobile phone, a tablet computer, a laptop, a personal computer, a television, a projector, or a monitoring device.

[0398] It should be noted here that the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium, and device embodiments of the present invention, please refer to the description of the method embodiments of the present invention.

[0399] References throughout the specification to "one embodiment," "one embodiment," or "some embodiments" should be understood to mean that a particular feature, structure, or characteristic associated with an embodiment is included in at least one embodiment of the present invention. Therefore, the appearances of "in one embodiment," "in one embodiment," or "in some embodiments" throughout the specification do not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In various embodiments of the present invention, the magnitude of the numbers of the above-described steps does not imply a sequential order of execution. The execution order of each step is determined by its function and inherent logic and does not limit the implementation of the embodiments of the present invention. The numbers of the above-described embodiments of the present invention are for illustrative purposes only and do not represent the superiority or inferiority of the embodiments. The preceding description of each embodiment tends to emphasize the differences between the embodiments; similarities or homologies may be mutually referenced and will not be repeated in this document for the sake of brevity.

[0400] The term "and / or" in this document merely describes the association relationship between related objects and indicates that three relationships can exist, for example, object A and / or object B indicates that three situations can exist: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0401] It should be noted that, as used herein, the terms "comprise," "comprises," or any other variation thereof, are intended to be non-exclusive inclusive, and thus a process, method, article, or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed or that are inherent to the process, method, article, or apparatus. Unless further limited, an element qualified by "comprises one or more" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises that element.

[0402] It should be understood that in some embodiments of the present invention, the disclosed devices and methods may be realized in other ways. The above-described embodiments are merely examples. For example, the division of the modules is merely a division of logical functions. In actual implementation, other division methods are possible. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not implemented. Furthermore, the coupling, direct coupling, or communication connection between each element shown or described may be an indirect coupling or communication connection via some interfaces, devices, or modules, and may be electrical, mechanical, or other types.

[0403] The modules described above as separate components may or may not be physically separated, and the parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed across multiple network units, and some or all of the modules may be selected according to actual needs to achieve the objectives of the aspects of this embodiment.

[0404] Furthermore, the core functional modules in each embodiment of the present invention may all be integrated into one processing unit, each module may exist as a single unit, or two or more modules may be integrated into one unit, and the above-mentioned integrated modules may be realized in the form of hardware or in the form of a hardware and software functional unit.

[0405] Those skilled in the art can understand that the realization of all or part of the steps of the above-mentioned method embodiments can be completed by hardware associated with program instructions, and the above-mentioned program can be stored in a computer-readable storage medium, which, when executed, performs the steps comprising the above-mentioned method embodiments, and the above-mentioned storage medium includes various media that can store program code, such as a mobile storage device, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0406] Alternatively, the above-mentioned integrated units of the present invention can be realized in the form of software functional modules and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essential part of the technical solutions of the embodiments of the present invention or the part contributing to the related technology can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing an electronic device to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a mobile storage device, a ROM, a magnetic disk, or an optical disk.

[0407] The methods disclosed in the several method embodiments of the present invention can be arbitrarily combined, if they do not conflict, to obtain new method embodiments.

[0408] Features disclosed in several product embodiments of the present invention may be combined in any manner that does not conflict to obtain new product embodiments.

[0409] Features disclosed in several method or apparatus embodiments according to the present invention can be combined in any manner that does not conflict to arrive at new method or apparatus embodiments.

[0410] The above description is merely an embodiment of the present invention, and the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that are easily conceivable by those skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be in accordance with the scope of protection of the claims.

Claims

1. A coding method applied to an encoder, comprising: determining a template for a current block and one or more prediction templates for the template for the current block; determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; determining one or more first prediction blocks of the current block based on prediction parameters of the current block; fusing the one or more first prediction blocks based on weights of the one or more prediction templates to obtain a second prediction block of the current block; Encoding method.

2. determining a first prediction block for the current block based on the intra prediction mode indicated by the prediction parameters; The method of claim 1.

3. determining a first prediction block of the current block based on the motion parameters or BV of the current block indicated by the prediction parameters; 3. The method according to claim 1 or 2.

4. The motion parameters include at least one of parameters MV and reference image index. The method of claim 3.

5. The prediction parameters of the current block are the same as the prediction parameters of the template of the current block.

5. The method according to any one of claims 1 to 4.

6. determining a prediction template for a template of the current block, performing HoG calculations on the samples of the template of the current block to obtain gradient directions and gradient magnitudes of the corresponding samples; determining an angular mode based on the gradient direction and gradient magnitude; predicting a template for the current block based on the angle mode to obtain the predicted template; The method of claim 1.

7. The step of determining the angle mode based on the gradient direction and the gradient magnitude includes: converting the gradient directions into predefined candidate angular modes; determining the angular mode from the transformed candidate angular modes based on the gradient magnitude. The method of claim 6.

8. The step of determining the angular mode based on the gradient strength includes: obtaining N angular modes based on candidate angular modes corresponding to the N largest gradient strengths; and / or obtaining one or more of the angular modes based on candidate angular modes corresponding to gradient strengths greater than or equal to a first threshold; and / or obtaining the one or more angular modes based on candidate angular modes corresponding to second gradient strengths whose difference values ​​from the at least one first gradient strength are less than or equal to a second threshold value; The method of claim 7.

9. the magnitude of the at least one first gradient strength is adjacent to the magnitude of the second gradient strength; The method of claim 8.

10. determining a prediction template for a template of the current block, predicting a template for the current block based on candidate prediction modes in a mode list to obtain candidate prediction templates; and obtaining the prediction template based on a sample value error between a candidate prediction template and a template of the current block. The method of claim 1.

11. The step of obtaining the prediction template based on the sample value error between the candidate prediction template and the template of the current block includes: obtaining N prediction templates based on candidate prediction templates corresponding to the N smallest sample value errors, or obtaining one or more prediction templates based on candidate prediction templates corresponding to sample value errors less than or equal to a third threshold. The method of claim 10.

12. The sample value error type is at least one of SATD, SAD, MAD, MAE, NCC, MSE, and SSE.

12. The method according to claim 10 or 11.

13. The candidate prediction modes in the mode list are: the angular modes obtained by the steps of any one of claims 6 to 9; a prediction mode in the MPM list; the prediction modes in the MPM list after the planar mode has been removed; and An angle mode obtained by expanding an angle mode in the MPM list whose angle is greater than or equal to a fourth threshold value; and at least one of an angle mode obtained by expanding an angle mode in the MPM list in which the angle mode is smaller than or equal to a fifth threshold value, The method of claim 10.

14. In the MPM list, a DC mode is located after or before an angular mode constructed based on neighboring blocks of the current block. The method of claim 13.

15. In the mode list, the N angular modes are located after or before an angular mode constructed based on neighboring blocks of the current block. The method of claim 13.

16. The step of predicting a template of the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template includes: predicting a template of the current block based on sample values ​​of a reference region of the template of the current block and the candidate prediction mode to obtain a candidate prediction template; the reference region includes a non-adjacent region and / or an adjacent region of the template of the current block; 16. The method according to any one of claims 10 to 15.

17. the reference region includes an upper left region, an upper region, an upper right region, a left region, and / or a lower left region of the template of the current block; 17. The method of claim 16.

18. The step of predicting a template of the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template includes: combining candidate prediction modes in the mode list and reference rows in a reference row list of the template of the current block to obtain a combination list; predicting a template for the current block based on a candidate prediction mode and a reference row indicated by a combination in the combination list to obtain a candidate prediction template; 16. The method according to any one of claims 10 to 15.

19. The reference row indexes included in the reference row list are {1, 3, 5, 7, 12}, {1, 4, 5, 7, 10, 12}, or {1, 2, 3, 7, 10, 11}.

20. The method of claim 18.

20. determining a prediction template for a template of the current block, determining at least one candidate MV; performing motion compensation on a template of the current block based on the candidate MV and a reference image of the current block to obtain the prediction template; 20. The method of any one of claims 1 to 19.

21. The reference image of the current block is a forward reference image or a backward reference image of the current block.

21. The method of claim 20.

22. The step of determining at least one candidate MV as described above includes: obtaining at least one candidate MV based on the Merge candidate list or the AMVP candidate list; 21. The method of claim 20.

23. The above-mentioned method of obtaining at least one candidate MV based on the Merge candidate list or the AMVP candidate list includes: obtaining the first N candidate MVs in the Merge candidate list or AMVP candidate list; 23. The method of claim 22.

24. The candidate MVs in the Merge candidate list or AMVP candidate list are MVs subdivided by template matching, DMVR, and / or Multipass DMVR; and / or The candidate MVs in the Merge candidate list or AMVP candidate list are candidate MVs sorted according to a predetermined order relative to those in the original candidate list.

24. The method of claim 23.

25. determining a prediction template for a template of the current block, determining at least one candidate BV; determining a prediction template for a template of the current block based on the candidate BV, the current block, and a current image; 25. The method of any one of claims 1 to 24.

26. and further comprising obtaining at least one candidate BV based on a Merge candidate list, an AMVP candidate list, and / or a preset value.

26. The method of claim 25.

27. Obtain the first N candidate BVs in the Merge candidate list or AMVP candidate list; 27. The method of claim 26.

28. The preset value is equal to 0.

27. The method of claim 26.

29. The one or more prediction templates further include a first offset term, the first offset term being a template generated based on a sample value bit depth of a template of the current block; The one or more first prediction blocks further include a second offset term, and the second offset term is a block generated based on a sample value bit depth of the current block.

29. The method of any one of claims 1 to 28.

30. the one or more prediction templates further include a third offset term, the third offset term being a prediction template obtained by predicting a template of the current block based on a specific prediction mode or a specific motion vector; The one or more first prediction blocks further include a fourth offset term, and the fourth offset term is a prediction block obtained by predicting the current block based on the specific prediction mode or the specific motion vector.

30. The method of any one of claims 1 to 29.

31. The specific prediction mode includes a planar mode and / or a DC mode.

31. The method of claim 30.

32. and, in response to determining that the particular prediction mode is different from an intra-prediction mode indicated by the prediction parameters, predicting a template for the current block based on the particular prediction mode.

31. The method of claim 30.

33. and in response to determining that the particular motion vector is different from a motion vector of the current block indicated by the prediction parameters, predicting a template of the current block based on the particular motion vector.

31. The method of claim 30.

34. the one or more prediction templates further include a first nonlinear term, the first nonlinear term being a prediction template obtained by performing a transformation on a candidate prediction template having the smallest sample value error; the one or more first prediction blocks further include a second nonlinear term, the second nonlinear term being a prediction block obtained by performing the transformation on the first prediction block corresponding to the candidate prediction template having the smallest sample value error; 29. The method of any one of claims 10 to 28.

35. if the one or more prediction templates are less than N, supplementing the prediction templates to N by adding at least one of the first offset term, the third offset term, the first nonlinear term, and a first preset value; correspondingly, supplementing the first prediction block to N by adding at least one of the second offset term, the fourth offset term, the second nonlinear term, and a second preset value; 35. The method of claim 29, 30, or 34.

36. the first preset value is equal to 0; 36. The method of claim 35.

37. The transformation includes a squaring operation.

30. The method of any one of claims 10 to 29.

38. The one or more prediction templates further include a prediction template obtained by predicting a template of the current block based on an intermediate prediction mode before a fusion operation in a Chroma Fusion, OBMC, MHP, and / or SGPM method.

38. The method of any one of claims 1 to 37.

39. determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates so that a sample value error between the template of the current block and a target prediction value of the template of the current block is minimized; the target prediction value is equal to a weighted sum of sample values ​​of the one or more prediction templates; 39. The method of any one of claims 1 to 38.

40. determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; determining an autocorrelation matrix of the prediction template based on sample values ​​of the prediction template; determining a cross-correlation vector between the prediction template and the template for the current block based on sample values ​​of the prediction template and sample values ​​of the template for the current block; determining weights for the one or more prediction templates based on the autocorrelation matrix and the cross-correlation vector.

40. The method of claim 39.

41. The sample error type is one of MSE, SATD, SAD, MAD, MAE, NCC, and SSE.

41. The method of claim 39 or 40.

42. determining a residual block of the current block based on a second predicted block of the current block; generating a bitstream based on the residual block. The method of claim 1.

43. A decoding method applied to a decoder, comprising: determining a template for a current block and one or more prediction templates for the template for the current block; determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; determining one or more first prediction blocks of the current block based on prediction parameters of the current block; fusing the one or more first prediction blocks based on weights of the one or more prediction templates to obtain a second prediction block of the current block; determining a reconstructed value of the current block based on the second predicted block. Decryption method.

44. determining a reconstructed value of the current block based on the second predicted block, decoding a bitstream to determine a residual block corresponding to the current block; determining a reconstructed value of the current block based on the residual block and the second predicted block; 44. The method of claim 43.

45. determining a first prediction block for the current block based on the intra prediction mode indicated by the prediction parameters; 44. The method of claim 43.

46. determining a first prediction block of the current block based on the motion parameters of the current block indicated by the prediction parameters; 44. The method of claim 43.

47. The motion parameters include at least one of a motion vector and a reference image index.

47. The method of claim 46.

48. The prediction parameters of the current block are the same as the prediction parameters of the template of the current block.

48. The method of any one of claims 43 to 47.

49. determining a prediction template for a template of the current block, performing HoG calculations on the samples of the template of the current block to obtain gradient directions and gradient magnitudes of the corresponding samples; determining an angular mode based on the gradient direction and gradient magnitude; predicting a template for the current block based on the angle mode to obtain the predicted template; 44. The method of claim 43.

50. The step of determining the angle mode based on the gradient direction and the gradient magnitude includes: converting the gradient directions into predefined candidate angular modes; determining the angular mode from the transformed candidate angular modes based on the gradient magnitude.

50. The method of claim 49.

51. The step of determining the angular mode based on the gradient strength includes: obtaining N angular modes based on candidate angular modes corresponding to the N largest gradient strengths; and / or obtaining one or more of the angular modes based on candidate angular modes corresponding to gradient strengths greater than or equal to a first threshold; and / or obtaining the one or more angular modes based on candidate angular modes corresponding to second gradient strengths whose difference values ​​from the at least one first gradient strength are less than or equal to a second threshold value; 51. The method of claim 50.

52. the magnitude of the at least one first gradient strength is adjacent to the magnitude of the second gradient strength; 52. The method of claim 51.

53. determining a prediction template for a template of the current block, predicting a template for the current block based on candidate prediction modes in a mode list to obtain candidate prediction templates; and obtaining the prediction template based on a sample value error between a candidate prediction template and a template of the current block.

44. The method of claim 43.

54. The step of obtaining the prediction template based on the sample value error between the candidate prediction template and the template of the current block includes: obtaining N prediction templates based on candidate prediction templates corresponding to the N smallest sample value errors, or obtaining one or more prediction templates based on candidate prediction templates corresponding to sample value errors less than or equal to a third threshold.

54. The method of claim 53.

55. The sample value error type is at least one of SATD, SAD, MAD, MAE, NCC, MSE, and SSE.

55. The method of claim 53 or 54.

56. The candidate prediction modes in the mode list are: the angular modes obtained by the steps of any one of claims 49 to 52; and a prediction mode in the MPM list; the prediction modes in the MPM list after the planar mode has been removed; and An angle mode obtained by expanding an angle mode in the MPM list whose angle is greater than or equal to a fourth threshold value; and at least one of an angle mode obtained by expanding an angle mode in the MPM list in which the angle mode is smaller than or equal to a fifth threshold value, 48. The method of claim 47.

57. In the MPM list, a DC mode is located after or before an angular mode constructed based on neighboring blocks of the current block.

57. The method of claim 56.

58. In the mode list, the N angular modes are located after or before an angular mode constructed based on neighboring blocks of the current block.

57. The method of claim 56.

59. The step of predicting a template of the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template includes: predicting a template of the current block based on sample values ​​of a reference region of the template of the current block and the candidate prediction mode to obtain a candidate prediction template; the reference region includes a non-adjacent region and / or an adjacent region of the template of the current block; 59. The method of any one of claims 53 to 58.

60. the reference region includes an upper left region, an upper region, an upper right region, a left region, and / or a lower left region of the template of the current block; 60. The method of claim 59.

61. The step of predicting a template of the current block based on a candidate prediction mode in a mode list to obtain a candidate prediction template includes: combining candidate prediction modes in the mode list and reference rows in a reference row list of the template of the current block to obtain a combination list; predicting a template for the current block based on a candidate prediction mode and a reference row indicated by a combination in the combination list to obtain a candidate prediction template; 59. The method of any one of claims 53 to 58.

62. The reference row indexes included in the reference row list are {1, 3, 5, 7, 12}, {1, 4, 5, 7, 10, 12}, or {1, 2, 3, 7, 10, 11}.

62. The method of claim 61.

63. determining a prediction template for a template of the current block, determining at least one candidate MV; performing motion compensation on a template of the current block based on the candidate MV and a reference image of the current block to obtain the prediction template; 63. The method of any one of claims 43 to 62.

64. The reference image of the current block is a forward reference image or a backward reference image of the current block.

64. The method of claim 63.

65. The step of determining at least one candidate MV as described above includes: obtaining at least one candidate MV based on the Merge candidate list or the AMVP candidate list; 64. The method of claim 63.

66. The above-mentioned method of obtaining at least one candidate MV based on the Merge candidate list or the AMVP candidate list includes: obtaining the first N candidate MVs in the Merge candidate list or AMVP candidate list; 66. The method of claim 65.

67. The candidate MVs in the Merge candidate list or the AMVP candidate list are MVs subdivided by template matching, DMVR, and / or Multipass DMVR; and / or The candidate MVs in the Merge candidate list or AMVP candidate list are candidate MVs sorted according to a predetermined order relative to those in the original candidate list.

67. The method of claim 66.

68. determining a prediction template for a template of the current block, determining at least one candidate BV; determining a prediction template for a template of the current block based on the candidate BV, the current block, and a current image; 68. The method of any one of claims 43 to 67.

69. and further comprising obtaining at least one candidate BV based on a Merge candidate list, an AMVP candidate list, and / or a preset value.

69. The method of claim 68.

70. Obtain the first N candidate BVs in the Merge candidate list or AMVP candidate list; 69. The method of claim 68.

71. The preset value is equal to 0.

70. The method of claim 69.

72. The one or more prediction templates further include a first offset term, the first offset term being a template generated based on a sample value bit depth of a template of the current block; The one or more first prediction blocks further include a second offset term, and the second offset term is a block generated based on a sample value bit depth of the current block.

68. The method of any one of claims 43 to 67.

73. the one or more prediction templates further include a third offset term, the third offset term being a prediction template obtained by predicting a template of the current block based on a specific prediction mode or a specific motion vector; The one or more first prediction blocks further include a fourth offset term, and the fourth offset term is a prediction block obtained by predicting the current block based on the specific prediction mode or the specific motion vector.

68. The method of any one of claims 43 to 67.

74. The specific prediction mode includes a planar mode and / or a DC mode.

74. The method of claim 73.

75. and, in response to determining that the particular prediction mode is different from an intra-prediction mode indicated by the prediction parameters, predicting a template for the current block based on the particular prediction mode.

74. The method of claim 73.

76. and in response to determining that the particular motion vector is different from a motion vector of the current block indicated by the prediction parameters, predicting a template of the current block based on the particular motion vector.

74. The method of claim 73.

77. the one or more prediction templates further include a first nonlinear term, the first nonlinear term being a prediction template obtained by performing a transformation on a candidate prediction template having the smallest sample value error; the one or more first prediction blocks further include a second nonlinear term, the second nonlinear term being a prediction block obtained by performing the transformation on the first prediction block corresponding to the candidate prediction template having the smallest sample value error; 72. The method of any one of claims 53 to 71.

78. if the one or more prediction templates are less than N, supplementing the prediction templates to N by adding at least one of the first offset term, the third offset term, the first nonlinear term, and a first preset value; correspondingly, supplementing the first prediction block to N by adding at least one of the second offset term, the fourth offset term, the second nonlinear term, and a second preset value; 78. The method of claim 72, 73, or 77.

79. the first preset value is equal to 0; 79. The method of claim 78.

80. The transformation includes a squaring operation.

78. The method of claim 77.

81. The one or more prediction templates further include a prediction template obtained by predicting a template of the current block based on an intermediate prediction mode before a fusion operation in a Chroma Fusion, OBMC, MHP, and / or SGPM method.

81. The method of any one of claims 49 to 80.

82. determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates so that a sample value error between the template of the current block and a target prediction value of the template of the current block is minimized; the target prediction value is equal to a weighted sum of sample values ​​of the one or more prediction templates; 82. The method of any one of claims 43 to 81.

83. determining weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; determining an autocorrelation matrix of the prediction template based on sample values ​​of the prediction template; determining a cross-correlation vector between the prediction template and the template for the current block based on sample values ​​of the prediction template and sample values ​​of the template for the current block; determining weights for the one or more prediction templates based on the autocorrelation matrix and the cross-correlation vector.

83. The method of claim 82.

84. The sample error type is one of MSE, SATD, SAD, MAD, MAE, NCC, and SSE.

84. The method of claim 82 or 83.

85. A coding device applied to an encoder, a first determination module configured to determine a template for a current block and one or more prediction templates for the template for the current block; a second determination module configured to determine weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; a third determination module configured to determine one or more first prediction blocks of the current block based on prediction parameters of the current block; a first fusion module configured to fuse the one or more first prediction blocks based on weights of the one or more prediction templates to obtain a second prediction block of the current block. Encoding device.

86. a first memory and a first processor; the first memory is used to store a computer program operable on the first processor; The first processor is used to carry out the method of any one of claims 1 to 42 when running the computer program. Encoder.

87. A decoding device applied to a decoder, a fourth determination module configured to determine a template of a current block and one or more prediction templates of the template of the current block; a fifth determining module configured to determine weights of the one or more prediction templates based on the template of the current block and the one or more prediction templates; a sixth determining module configured to determine one or more first prediction blocks of the current block based on prediction parameters of the current block; a second fusion module configured to fuse the one or more first prediction blocks based on weights of the one or more prediction templates to obtain a second prediction block of the current block; a seventh determination module configured to determine a reconstructed value of the current block based on the second predicted block; Decryption device.

88. a second memory and a second processor; the second memory is used to store a computer program operable on the second processor; The second processor is used to execute the method of any one of claims 43 to 84 when it runs the computer program. decoder.

89. A bitstream comprising: A bitstream generated by a residual block determined based on a second predicted block of the current block, The second predicted block is obtained by a method according to any one of claims 1 to 41. Bitstream.

90. a processor adapted to execute a computer program; a computer-readable storage medium; The computer-readable storage medium stores a computer program that, when executed by the processor, implements the method of any one of claims 1 to 42, or that, when executed by the processor, implements the method of any one of claims 43 to 84. electronic equipment.

91. A computer program is stored The computer program, when executed, implements the method of any one of claims 1 to 43, or when executed, implements the method of any one of claims 43 to 84. A computer-readable storage medium.

Images