Prediction method, apparatus, device, system, and storage medium

By determining the weight derivation mode of current blocks based on initial prediction modes, the method reduces coding costs and maintains encoding efficiency by eliminating the need to transmit mode information in the bitstream.

JP2024545531AActive Publication Date: 2024-12-09GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024539671
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-12-31
Filing Date
2022-04-12
Publication Date
2024-12-09
Estimated Expiration
2042-04-12

AI Technical Summary

Technical Problem

Existing video encoding technologies face increased coding costs due to the need to transmit mode information for predicting current blocks, which occupies more encoding resources and reduces efficiency.

Method used

Determine the weight derivation mode of the current block based on K initial prediction modes, allowing the decoding side to derive the mode without transmitting it in the bitstream, thereby reducing coding costs.

Benefits of technology

This approach reduces coding costs by eliminating the need to transmit mode information, enhancing encoding efficiency and maintaining consistency between encoding and decoding sides.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024545531000001_ABST
    Figure 2024545531000001_ABST
Patent Text Reader

Abstract

In this application, a prediction method and apparatus, a device, a system and a storage medium are provided. In this application, the weight derivation mode of the current block is determined according to K initial prediction modes, so that the weight derivation mode determination method is enriched. In addition, the encoding side of this application does not need to transmit the index of the weight derivation mode in the bitstream, and the decoding side can determine the weight derivation mode of the current block according to K initial prediction modes, thereby saving codewords and reducing coding cost.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present application relates to the technical field of video encoding and decoding, and in particular to a prediction method and apparatus, a device, a system, and a storage medium. [Background technology]

[0002] Digital video technology can be incorporated into various video devices, such as digital televisions, smartphones, computers, e-readers, or video players. With the development of video technology, the amount of video data is increasing. To facilitate the transmission of video data, video devices utilize video compression technology so that the video data can be transmitted or stored more efficiently.

[0003] Since video has temporal or spatial redundancy, prediction can eliminate or reduce the redundancy in the video to improve compression efficiency. In some cases, a current block is predicted using multiple prediction modes to obtain a predicted value of the current block. In these cases, in order to maintain consistency between the encoding side and the decoding side, the encoding side needs to write mode information required to predict the current block into the bitstream. This occupies more encoding resources and increases coding costs. Summary of the Invention

[0004] The embodiments of the present application provide a prediction method and apparatus, a device, a system, and a storage medium, which determine a weight derivation mode of a current block based on K initial prediction modes, thereby reducing coding cost.

[0005] In a first aspect, the present application provides a prediction method applied to a decoder, which may include the following: Decode a bitstream to determine K initial prediction modes, where K is a positive integer greater than 1; determine a weight derivation mode of a current block according to the K initial prediction modes; and determine a predicted value of the current block according to the weight derivation mode of the current block.

[0006] In a second aspect, the present application provides a prediction method, which may include: determining K initial prediction modes, where K is a positive integer greater than 1; determining a weight derivation mode of a current block according to the K initial prediction modes; and determining a predicted value of a current block according to the weight derivation mode of the current block.

[0007] In a third aspect, the present application provides a prediction device, configured to perform the method of the first aspect or any of its embodiments, in particular comprising functional units configured to perform the method of the first aspect or any of its embodiments.

[0008] In a fourth aspect, the present application provides a prediction device, configured to perform the method of the second aspect or any of its embodiments, in particular comprising functional units configured to perform the method of the second aspect or any of its embodiments.

[0009] In a fifth aspect, there is provided a video decoder comprising a processor and a memory configured to store a computer program, the processor being configured to perform the method of the first aspect or any embodiment thereof by calling and executing the computer program stored in the memory.

[0010] In a sixth aspect, there is provided a video encoder comprising a processor and a memory configured to store a computer program, the processor being configured to perform the method of the second aspect or any embodiment thereof by calling and executing the computer program stored in the memory.

[0011] In a seventh aspect, there is provided a video encoding / decoding system, comprising a video encoder and a video decoder, the video decoder being configured to perform the method of the first aspect or any embodiment thereof, the video encoder being configured to perform the method of the second aspect or any embodiment thereof.

[0012] In an eighth aspect, there is provided a chip configured to perform the method of the first or second aspect, or each embodiment of the first or second aspect. Specifically, the chip includes a processor configured to call up and execute a computer program from a memory, thereby causing a device to which the chip is attached to perform the method of the first or second aspect, or each embodiment of the first or second aspect.

[0013] In a ninth aspect, there is provided a computer readable storage medium configured to store a computer program, the computer program causing a computer to perform the method of the first or second aspect or an embodiment of the first or second aspect.

[0014] In a tenth aspect, there is provided a computer program product, the computer program product comprising computer program instructions configured to cause a computer to carry out a method according to the first or second aspect above, or a respective embodiment of the first or second aspect.

[0015] In an eleventh aspect, there is provided a computer program which, when executed on a computer, is configured to cause the computer to carry out the method of the first or second aspect or a respective embodiment of the first or second aspect.

[0016] In a twelfth aspect, there is provided a bitstream, the bitstream being generated according to the method of the second aspect.

[0017] According to the above technical solution, the weight derivation mode of current block is determined according to K initial prediction modes, so that the weight derivation mode determination method is more enriched.In addition, the encoding side of the present application does not need to transmit the index of weight derivation mode in bitstream, and the decoding side can determine the weight derivation mode of current block according to K initial prediction modes, thereby saving codewords and reducing coding cost. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a block diagram illustrating a video encoding / decoding system according to an embodiment of the present application. [Diagram 2] FIG. 2 is a block diagram illustrating a video encoder according to an embodiment of the present application. [Diagram 3] FIG. 3 is a block diagram illustrating a video decoder according to an embodiment of the present application. [Figure 4] FIG. 4 is a schematic diagram showing the weight assignment. [Diagram 5]FIG. 5 is a schematic diagram showing the weight assignment. [Figure 6A] FIG. 6A is a schematic diagram illustrating inter prediction. [Figure 6B] FIG. 6B is a schematic diagram illustrating weighted inter prediction. [Figure 7A] FIG. 7A is a schematic diagram illustrating intra prediction. [Figure 7B] FIG. 7B is a schematic diagram illustrating intra prediction. [Figure 8A] FIG. 8A is a schematic diagram illustrating intra prediction. [Figure 8B] FIG. 8B is a schematic diagram illustrating intra prediction. [Figure 8C] FIG. 8C is a schematic diagram illustrating intra prediction. [Figure 8D] FIG. 8D is a schematic diagram illustrating intra prediction. [Figure 8E] FIG. 8E is a schematic diagram illustrating intra prediction. [Figure 8F] FIG. 8F is a schematic diagram illustrating intra prediction. [Figure 8G] FIG. 8G is a schematic diagram illustrating intra prediction. [Figure 8H] FIG. 8H is a schematic diagram illustrating intra prediction. [Figure 8I] FIG. 8I is a schematic diagram illustrating intra prediction. [Figure 9] FIG. 9 is a schematic diagram showing intra prediction modes. [Figure 10] FIG. 10 is a schematic diagram showing intra prediction modes. [Figure 11] FIG. 11 is a schematic diagram showing intra prediction modes. [Figure 12] FIG. 12 is a schematic diagram illustrating weighted intra prediction. [Figure 13] FIG. 13 is a schematic diagram illustrating template matching. [Figure 14] FIG. 14 is a flow chart illustrating a prediction method according to an embodiment of the present application. [Figure 15]FIG. 15 is a schematic diagram of a case where a current block is predicted using two prediction modes. [Figure 16] FIG. 16 is a schematic diagram showing template division. [Figure 17A] FIG. 17A is a schematic diagram showing another template division. [Figure 17B] FIG. 17B is a schematic diagram showing another template division. [Figure 18] FIG. 18 is a schematic diagram showing the size of the template. [Figure 19] FIG. 19 is a flow chart illustrating a prediction method according to an embodiment of the present application. [Figure 20] FIG. 20 is a block diagram illustrating a prediction device according to an embodiment of the present application. [Figure 21] FIG. 21 is a block diagram illustrating a prediction device according to an embodiment of the present application. [Figure 22] FIG. 22 is a block diagram illustrating an electronic device according to an embodiment of the present application. [Figure 23] FIG. 23 is a block diagram illustrating a video encoding / decoding system according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The present application is applicable to the fields of image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, real-time video encoding and decoding, etc. For example, the technical solution of the present application can be combined with an audio video coding standard (AVS), and examples of AVS include the H.264 / audio video coding (AVC) standard, the H.265 / high efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. Alternatively, the technical solution of the present application may be combined with other proprietary or industry standards, including ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), including scalable video coding (SVC) and multi-view video coding (MVC) extensions. Note that the technology of the present application is not limited to any particular encoding / decoding standard or technology.

[0020] For ease of understanding, a video encoding and decoding system according to an embodiment of the present application will first be introduced with reference to FIG.

[0021] FIG. 1 is a block diagram showing a video encoding / decoding system according to an embodiment of the present application. FIG. 1 is merely an example, and the video encoding / decoding system according to the embodiment of the present application includes, but is not limited to, that shown in FIG. 1. As shown in FIG. 1, the video encoding / decoding system 100 includes an encoding device 110 and a decoding device 120. The encoding device is configured to encode (may be understood as compressing) video data to generate a bitstream, and transmit the bitstream to a decoding device. The decoding device is configured to decode the bitstream generated by the encoding device to obtain decoded video data.

[0022] The encoding device 110 according to an embodiment of the present application may be understood as a device having a video encoding function, and the decoding device 120 may be understood as a device having a video decoding function, that is, the encoding device 110 and the decoding device 120 according to an embodiment of the present application may include a broader range of devices, such as smartphones, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set top boxes (STBs), televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, etc.

[0023] In some embodiments, encoding device 110 may transmit encoded video data (e.g., a bitstream) to decoding device 120 over channel 130. Channel 130 may include one or more media and / or devices that may transmit the encoded video data from encoding device 110 to decoding device 120.

[0024] In one example, the channel 130 includes one or more communication media that enable the encoding device 110 to transmit encoded video data directly to the decoding device 120 in real time. In this example, the encoding device 110 can modulate the encoded video data according to a communication standard and transmit the modulated video data to the decoding device 120. The communication medium includes a wireless communication medium, such as a radio frequency spectrum. Alternatively, the communication medium can also include a wired communication medium, such as one or more physical transmission lines.

[0025] In another example, the channel 130 includes a storage medium. The storage medium can store the video data encoded by the encoding device 110. The storage medium can include various locally accessible data storage media, such as an optical disk, a digital versatile disc (DVD), a flash memory, etc. In this example, the decoding device 120 can obtain the encoded video data from the storage medium.

[0026] In another example, channel 130 may include a storage server, which may store the video data encoded by encoding device 110. In this example, decoding device 120 may download the encoded video data stored in the storage server from the storage server. Alternatively, the storage server may store the encoded video data and transmit the encoded video data to decoding device 120, examples of which include a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0027] In some embodiments, encoding device 110 includes a video encoder 112 and an output interface 113. Output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0028] In some embodiments, the encoding device 110 may further include a video source 111 in addition to a video encoder 112 and an output interface 113 .

[0029] Video source 111 may include at least one of a video acquisition device (e.g., a video camera), a video archive, a video input interface, and a computer graphics system, the video input interface configured to receive video data from a video content provider, and the computer graphics system configured to generate the video data.

[0030] The video encoder 112 generates a bitstream by encoding video data from the video source 111. The video data may include one or more pictures or a sequence of pictures. The bitstream includes encoding information for the pictures or sequence of pictures. The encoding information may include encoding image data and related data. The related data may include sequence parameter sets (abbreviated as SPS), picture parameter sets (abbreviated as PPS), and other syntax structures. An SPS may include parameters that apply to one or more sequences, and a PPS may include parameters that apply to one or more pictures. A syntax structure is a collection of zero or more syntax elements arranged in a specified order in the bitstream.

[0031] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data may also be stored in a storage medium or a storage server to be subsequently read by the decoding device 120.

[0032] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122 .

[0033] In some embodiments, the decoding device 120 may further include a display device 123 in addition to the input interface 121 and the video decoder 122 .

[0034] The input interface 121 includes a receiver and / or a modem and is capable of receiving encoded video data via a channel 130.

[0035] The video decoder 122 is configured to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to a display device 123 .

[0036] The display device 123 displays the decoded video data and may be built into the decoding device 120 or may be external to the decoding device 120. The display device 123 may include a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0037] In addition, FIG. 1 is only an example, and the technical solution of the embodiment of the present application is not limited to FIG. 1. For example, the technology of the present application can also be applied to one-sided video encoding or one-sided video decoding.

[0038] The following describes a video encoding framework according to an embodiment of the present application.

[0039] 2 is a block diagram illustrating a video encoder according to an embodiment of the present application. The video encoder 200 can be configured to perform lossy or lossless compression on images. The lossless compression can be visually lossless or mathematically lossless compression.

[0040] The video encoder 200 is applicable to image data in luma-chroma (YCbCr, YUV) format. For example, the ratio of YUV can be 4:2:0, 4:2:2, or 4:4:4, where Y indicates luma, Cb (U) indicates blue chroma, Cr (V) indicates red chroma, and U, V indicate chroma to describe color and saturation. For example, in color format, 4:2:0 indicates that every four pixels have four luma components and two chroma components (YYYYCbCr), 4:2:2 indicates that every four pixels have four luma components and four chroma components (YYYYCbCrCbCr), and 4:4:4 indicates full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0041] For example, the video encoder 200 reads video data, and for each image in the video data, divides each image into several coding tree units (CTUs). In some examples, a CTU may be called a "tree block", a "largest coding unit (LCU)", or a "coding tree block (CTB)". Each CTU may be associated with a pixel block having the same size as the CTU in the image, and each pixel may correspond to one luminance (or luma) sample and two chrominance (or chroma) samples. Thus, each CTU may be associated with one luminance sample block and two chrominance sample blocks. The size of the CTU may be, for example, 128×128, 64×64, 32×32, etc. The CTU may be subsequently divided into several coding units (CUs) to be encoded. The CU may be a rectangular block or a square block. The CU can be further divided into a prediction unit (PU) and a transform unit (TU), so that encoding, prediction and transformation are separated and processing can be flexible. In one example, the CTU is divided into CUs in a quadtree manner, and the CU is divided into TUs and PUs in a quadtree manner.

[0042] The video encoder and the video decoder can support various PU sizes. Assuming that the size of a particular CU is 2N×2N, the video encoder and the video decoder can support a PU size of 2N×2N or N×N for intra prediction, and can support a symmetric PU having a size of 2N×2N, 2N×N, N×2N, N×N or similar for inter prediction. The video encoder and the video decoder can also support asymmetric PUs having a size of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0043] 2, the video encoder 200 may include a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filtering unit 260, a decoding image buffer 270, and an entropy encoding unit 280. However, the video encoder 200 may include more, fewer, or different functional units.

[0044] Optionally, in this application, the current block may be referred to as the current CU or the current PU, etc. The prediction block may be referred to as the prediction image block or the image prediction block. The reconstructed image block may be referred to as the reconstruction block or the image reconstruction block.

[0045] In some embodiments, the prediction unit 210 includes an inter prediction unit 211 and an intra estimation unit 212. Since there is a strong correlation between adjacent samples in a video picture, the video encoding and decoding technology uses the method of intra prediction to eliminate the spatial redundancy between adjacent samples. Since there is a strong similarity between adjacent pictures in a video, the video encoding and decoding technology uses the method of inter prediction to eliminate the temporal redundancy between adjacent pictures, thereby improving the coding efficiency.

[0046] The inter prediction unit 211 can be used for inter prediction. Inter prediction can include motion estimation and motion compensation, where image information of different images can be referenced, and temporal redundancy is eliminated by finding a reference block from a reference image using the motion information and generating a prediction block based on the reference block. The image used for inter prediction can be a P frame and / or a B frame. A P frame refers to a forward predicted image, and a B frame refers to a bidirectional predicted image. In inter prediction, a reference block is found from a reference image using the motion information and a prediction block is generated based on the reference block. The motion information includes a reference image list including reference images, a reference image index, and a motion vector. The motion vector can be an integer-sample motion vector or a fractional-sample motion vector. If the motion vector is a fractional-sample motion vector, an interpolation filter needs to be applied to the reference image to generate the required fractional sample block. The integer sample block or fractional sample block in the reference image found based on the motion vector is called a reference block. There are techniques for using the reference block as a prediction block as it is, and techniques for generating a prediction block by processing based on the reference block. Processing based on the reference block to generate a prediction block can also be understood as using the reference block as a prediction block, and then processing based on the prediction block to generate a new prediction block.

[0047] The intra prediction unit 212 predicts sample information in a current image block by only referring to information of the same image to eliminate spatial redundancy. The image used for intra prediction may be an I-frame.

[0048] Intra prediction includes various prediction modes, and taking the international digital video coding standard H series as an example, the H.264 / AVC standard includes 8 angular prediction modes and 1 non-angular prediction mode, while H.265 / HEVC is extended to 33 angular prediction modes and 2 non-angular prediction modes. The intra prediction modes used in HEVC include planar mode, DC (Direct Current), and 33 angular modes, for a total of 35 prediction modes. The intra modes used in VVC include planar, DC, and 65 angular modes, for a total of 67 prediction modes.

[0049] Furthermore, with an increase in the number of angle modes, intra prediction becomes more accurate and better meets the needs of the development of high-definition and ultra-high-definition digital videos.

[0050] Residual unit 220 may generate a residual block of a CU based on the sample block of the CU and the prediction block of the PU of the CU. For example, residual unit 220 may generate a residual block of a CU such that each sample in the residual block is equal to the difference between a sample in the sample block of the CU and a corresponding sample in the prediction block of the PU of the CU.

[0051] The transform / quantization unit 230 may quantize the transform coefficients. The transform / quantization unit 230 may quantize the transform coefficients associated with the TUs of a CU based on a quantization parameter (QP) value associated with the CU. The video encoder 200 may adjust the degree of quantization applied to the transform coefficients associated with a CU by adjusting the QP value associated with the CU.

[0052] The inverse transform / quantization unit 240 may reconstruct a residual block from the quantized transform coefficients by applying inverse quantization and inverse transform, respectively, to the quantized transform coefficients.

[0053] Reconstruction unit 250 may generate a reconstructed image block associated with a TU by adding samples in the reconstructed residual block to corresponding samples in one or more prediction blocks generated by prediction unit 210. In this manner, by reconstructing the sample blocks of each TU of a CU, video encoder 200 may reconstruct the sample blocks of the CU.

[0054] The loop filtering unit 260 processes the inverse transformed and dequantized samples to compensate for distortion information and provide a better reference for encoding subsequent samples. For example, the loop filtering unit 260 may perform a deblocking filtering process to reduce blocking artifacts of sample blocks associated with a CU.

[0055] In some embodiments, the loop filtering unit 260 includes a deblocking filtering unit and a sample adaptive offset (SAO) / adaptive loop filtering (ALF) unit, where the deblocking filtering unit is used for deblocking and the SAO / ALF unit is used to remove ringing effects.

[0056] The decoding image buffer 270 can store the reconstructed sample block. The inter prediction unit 211 can perform inter prediction on the PU of another image using the reference image including the reconstructed sample block. Also, the intra estimation unit 212 can perform intra prediction on the other PU in the same image as the CU using the reconstructed sample block in the decoding image buffer 270.

[0057] The entropy encoding unit 280 may receive the quantized transform coefficients from the transform / quantization unit 230. The entropy encoding unit 280 may generate entropy coded data by performing one or more entropy coding operations on the quantized transform coefficients.

[0058] FIG. 3 is a block diagram illustrating a video decoder according to an embodiment of the present application.

[0059] 3, the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization and transform unit 330, a reconstruction unit 340, a loop filtering unit 350, and a decoded image buffer 360. It should be noted that the video decoder 300 may include more, fewer, or different functional units.

[0060] The video decoder 300 may receive a bitstream. The entropy decoding unit 310 may extract syntax elements from the bitstream by parsing the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 may parse entropy coded syntax elements in the bitstream. The prediction unit 320, the inverse quantization and transformation unit 330, the reconstruction unit 340, and the loop filtering unit 350 may decode the video data based on the syntax elements extracted from the bitstream, i.e., generate decoded video data.

[0061] In some embodiments, the prediction unit 320 includes an inter prediction unit 321 and an intra estimation unit 322 .

[0062] The intra estimation unit 322 may generate a prediction block of the PU by performing intra prediction. The intra estimation unit 322 may use an intra prediction mode to generate a prediction block of the PU based on a sample block of a spatially neighboring PU. The intra estimation unit 322 may further determine the intra prediction mode of the PU based on one or more syntax elements parsed from the bitstream.

[0063] The inter prediction unit 321 may construct a first reference image list (list 0) and a second reference image list (list 1) based on syntax elements parsed from the bitstream. In addition, if the PU is encoded with inter prediction, the entropy decoding unit 310 may analyze motion information of the PU. The inter prediction unit 321 may determine one or more reference blocks of the PU based on the motion information of the PU. The inter prediction unit 321 may generate a prediction block of the PU based on the one or more reference blocks of the PU.

[0064] The inverse quantization and transform unit 330 may inverse quantize (i.e., dequantize) the transform coefficients associated with the TU. The inverse quantization and transform unit 330 may determine the degree of quantization using a QP value associated with the CU of the TU.

[0065] After inverse quantizing the transform coefficients, the inverse quantization and transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients to generate a residual block associated with the TU.

[0066] The reconstruction unit 340 reconstructs a sample block of a CU using a residual block associated with a TU of the CU and a prediction block of a PU of the CU. For example, the reconstruction unit 340 can reconstruct the sample block of a CU by adding a sample in the residual block to a corresponding sample in the prediction block to obtain a reconstructed image block.

[0067] The loop filtering unit 350 may perform a deblocking filtering process to reduce block artifacts in the sample blocks associated with the CU.

[0068] The video decoder 300 may store the reconstructed image of the CU in the decoding image buffer 360. The video decoder 300 may use the reconstructed image in the decoding image buffer 360 as a reference image for subsequent prediction, or may transmit the reconstructed image to a display device for display.

[0069] The basic flow of video encoding and decoding is as follows: On the encoding side, an image (frame) is divided into blocks, and for a current block, the prediction unit 210 performs intra prediction or inter prediction to generate a prediction block of the current block. The residual unit 220 can calculate a residual block based on the prediction block and the original block of the current block, i.e., the difference between the prediction block and the original block of the current block, and the residual block is also called residual information. The residual block is transformed and quantized by the transform / quantization unit 230, so that information that is not sensitive to the human eye can be removed and visual redundancy can be eliminated. Alternatively, the residual block before being transformed and quantized by the transform / quantization unit 230 can be called a time-domain residual block, and the time-domain residual block after being transformed and quantized by the transform / quantization unit 230 can be called a frequency residual block or a frequency-domain residual block. The entropy encoding unit 280 may receive the quantized transformation coefficients output by the transform / quantization unit 230 and output a bitstream by entropy coding the quantized transformation coefficients. For example, the entropy encoding unit 280 may remove character redundancies based on a target context model and probability information of the binary bitstream.

[0070] On the decoding side, the entropy decoding unit 310 may obtain prediction information, a quantization coefficient matrix, etc., of a current block by analyzing a bitstream, and the prediction unit 320 generates a prediction block of the current block by performing intra prediction or inter prediction on the current block based on the prediction information. The inverse quantization and transformation unit 330 uses the quantization coefficient matrix obtained from the bitstream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed block forms a reconstructed image. The loop filtering unit 350 loop filters the reconstructed image based on an image or block to obtain a decoded image. On the encoding side, a similar process to that on the decoding side is also required to obtain a decoded image. The decoded image may be called a reconstructed image, and the reconstructed image may be a reference image for inter prediction of a subsequent image.

[0071] In addition, the block division information determined by the encoding side, and mode information or parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc. are carried in the bitstream as necessary. The decoding side analyzes the bitstream and determines the same block division information, mode information or parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc. as the encoding side by analyzing the bitstream from existing information. This ensures that the decoding image obtained by the encoding side is the same as the decoding image obtained by the decoding side.

[0072] The above is a basic flow of video encoding and decoding in a block-based mixed coding framework. As technology develops, some modules or steps of the framework or flow may be optimized. This application applies to, but is not limited to, the basic flow of video encoding and decoding in a block-based mixed coding framework.

[0073] In some embodiments, the current block may be a current coding unit (CU) or a current prediction unit (PU), etc. Due to the need for parallel processing, an image may be divided into slices, etc., and slices in the same image may be processed in parallel, i.e., there is no data dependency between them. A "frame" is a commonly used term, and one frame may generally be understood to be one image. Also, a frame in this application may be replaced with an image, a slice, etc.

[0074] The video encoding and decoding standard currently under development, Versatile Video Coding (VVC), has an inter-prediction mode called geometric partitioning mode (GPM). The video encoding and decoding standard currently under development, Audio Video Coding (AVS), has an inter-prediction mode called angular weighted prediction (AWP). These two modes have different names and implementations, but they share a common principle.

[0075] It should be noted that in conventional unidirectional prediction, only one reference block having the same size as the current block is used, whereas in conventional bidirectional prediction, two reference blocks having the same size as the current block are used, and the value of each sample in the predicted block is the average value of the values ​​of samples at the corresponding positions of the two reference blocks, that is, all samples in each reference block occupy 50%. In bidirectional weighted prediction, the proportions of the two reference blocks may be different, for example, all samples in the first reference block occupy 75%, and all samples in the second reference block occupy 25%. However, all samples in the same reference block have the same proportion. Some other optimization methods, such as decoder side motion vector refinement (DMVR) technology, bidirectional optical flow (BIO), etc., cause some changes in the reference samples or predicted samples. Also, in GPM or AWP, two reference blocks having the same size as the size of the current block are used, but in some sample positions, the sample value in the corresponding position of the first reference block is used 100%, in some other sample positions, the sample value in the corresponding position of the second reference block is used 100%, and in the transition region (intersection region), the sample values ​​in the corresponding positions of these two reference blocks are used at a certain rate. How these weights are specifically assigned is determined by the prediction mode of GPM or AWP. Alternatively, in GPM or AWP, two reference blocks having a size different from the size of the current block are used, that is, it is considered that a necessary part of each reference block is used as a reference block. That is, a part with a weight other than 0 is used as a reference block, and a part with a weight of 0 is removed.

[0076] Illustratively, FIG. 4 is a schematic diagram showing weight allocation. As shown in FIG. 4, a schematic diagram of weight allocation of multiple division modes of GPM in a 64×64 current block according to an embodiment of the present application is shown, and the GPM has 64 division modes. FIG. 5 is a schematic diagram showing weight allocation. As shown in FIG. 5, a schematic diagram of weight allocation of multiple division modes of AWP in a 64×64 current block according to an embodiment of the present application is shown, and the AWP has 56 division modes. In both FIG. 4 and FIG. 5, in various division modes, the black area indicates that the weight value of the corresponding position of the first reference block is 0%, the white area indicates that the weight value of the corresponding position of the first reference block is 100%, and the gray area indicates that the weight value of the corresponding position of the first reference block is any weight value greater than 0% and less than 100% with different color shades. The weight value of the corresponding position of the second reference block is a value obtained by subtracting the weight value of the corresponding position of the first reference block from 100%.

[0077] The method of deriving weights in GPM and AWP is different. In GPM, angles and offsets are determined based on different modes, and then weight matrices for different modes are calculated. In AWP, a one-dimensional weight line is generated first, and then the whole matrix is ​​tiled with the one-dimensional weight line using a method similar to intra-angle prediction.

[0078] In early encoding and decoding technologies, there is only a rectangular partitioning method, regardless of the partitioning of CU, PU, ​​or transform unit (TU). In GPM and AWP, the effect of non-rectangular partitioning of prediction is realized when no partitioning is performed. In GPM and AWP, a mask of the weights of two reference blocks, i.e., a weight map, is used. From this mask, the weights of the two reference blocks for generating a prediction block are determined. Or, it can be simply understood as follows: A part of the position of the prediction block comes from the first reference block, a part of the position of the prediction block comes from the second reference block, and a transition area is obtained by weighting the corresponding positions of the two reference blocks, so that the transition can be made smoother. In GPM and AWP, since the current block is not divided into two CUs or PUs by a partition line, the entire current block is subjected to transformation, quantization, inverse transformation, inverse quantization, etc. of the residual after prediction.

[0079] In GPM, a weight matrix is ​​used to simulate a geometric partition, or more precisely, a prediction partition. To implement GPM, in addition to a weight matrix, two predictors are required, and each predictor is determined by one unidirectional motion information. The two unidirectional motion information are from a motion information candidate list, for example, from a merge motion information candidate list (mergeCandList). GPM uses two indexes in the bitstream to determine the two unidirectional motion information from mergeCandList.

[0080] In inter prediction, motion information is used to represent "motion." Basic motion information includes reference frame (also called reference picture) information and motion vector (MV) information. In general bidirectional prediction, two reference blocks are used to predict a current block. As the two reference blocks, one forward reference block and one backward reference block may be used. Alternatively, both of the two reference blocks may be forward reference blocks or backward reference blocks. Forward means that a time corresponding to a reference picture is before a current picture, and backward means that a time corresponding to a reference picture is after a current picture. In other words, forward means that a position of a reference picture in a video is located before a current picture, and backward means that a position of a reference picture in a video is located after a current picture. In other words, forward means that a picture order count (POC) of a reference picture is smaller than a POC of a current picture, and backward means that a POC of a reference picture is larger than a POC of a current picture. To be able to use bidirectional prediction, it is naturally necessary to find two reference blocks. Therefore, two sets of reference image information and motion vector information are required. Each set is considered as one unidirectional motion information, and the two sets are combined to form one bidirectional motion information. In concrete implementation, the unidirectional motion information and the bidirectional motion information can use the same data structure, and in the bidirectional motion information, both of the two sets of reference image information and motion vector information are valid, and in the unidirectional motion information, one of the two sets of reference image information and motion vector information is invalid.

[0081] In some embodiments, two reference picture lists are supported, denoted as RPL0 and RPL1, where RPL is an abbreviation of Reference Picture List. In some embodiments, P slices can only use RPL0, and B slices can use RPL0 and RPL1. For one slice, each reference picture list has several reference pictures, and the encoder and decoder use a reference picture index to find a particular reference picture. In some embodiments, the motion information is represented by a reference picture index and a motion vector. For example, for the bidirectional motion information, a reference picture index refIdxL0 corresponding to RPL0, a motion vector mvL0 corresponding to RPL0, and a reference picture index refIdxL1 corresponding to RPL1, a motion vector mvL0 corresponding to RPL1 are used. The reference picture index corresponding to RPL0 and the reference picture index corresponding to RPL1 can be understood as the reference picture information. In some embodiments, two flag bits indicate whether the motion information corresponding to RPL0 is used, and whether the motion information corresponding to RPL0 is used, respectively, and the two flag bits are denoted as predFlagL0 and predFlagL1. Also, predFlagL0 and predFlagL1 can be understood to indicate whether the unidirectional motion information is "valid or not". Although the data structure of motion information is not specified, the reference image index, motion vector, and "valid or not" flag bit corresponding to each reference image list all represent motion information. In the text of some standards, the motion information does not appear, and the motion vector is used. The reference image index and the flag indicating whether the corresponding motion information is used can be considered as an attachment of the motion vector. In this application, for convenience of explanation, "motion information" is used, but it should be understood that it is also possible to explain using "motion vector".

[0082] The motion information used for the current block may be stored. Based on the adjacent positional relationship, the motion information of a previously encoded / decoded block (e.g., an adjacent block) may be used for a subsequent block to be encoded / decoded in the current image. Such encoded / decoded motion information is called spatial motion information because it uses spatial correlation. The motion information used for each block of the current image may be stored. Based on the reference relationship, the motion information of a previously encoded / decoded image may be used for a subsequent image to be encoded / decoded. Such encoded / decoded motion information is called temporal motion information because it uses temporal correlation. In the method of storing the motion information used for each block of the current image, a fixed size matrix such as a 4×4 matrix is ​​used as the minimum unit, and one set of motion information is stored independently in each minimum unit. In this way, each time a block is encoded / decoded, the minimum units corresponding to the position of the block may store the motion information of the block. In this way, when spatial motion information or temporal motion information is used, the motion information corresponding to the position can be directly found based on the position. For example, when conventional unidirectional prediction is used for a 16x16 block, all 4x4 minimum units corresponding to the block store the motion information of this unidirectional prediction.When GPM or AWP is used for a block, all minimum units corresponding to the block determine the motion information to be stored in each minimum unit according to GPM or AWP mode, the first motion information, the second motion information, and the position of each minimum unit.As a method, when all 4x4 samples corresponding to a minimum unit are derived from the first motion information, this minimum unit stores the first motion information.When all 4x4 samples corresponding to a minimum unit are derived from the second motion information, this minimum unit stores the second motion information.If a 4x4 sample corresponding to one minimum unit comes from both the first motion information and the second motion information, AWP selects and stores one of them, while GPM combines and stores the two motion information as bidirectional motion information if the two motion information point to different reference image lists, otherwise it stores only the second motion information.

[0083] Optionally, the mergeCandList is constructed based on spatial motion information, temporal motion information, history-based motion information, and some other motion information. Exemplarily, in mergeCandList, spatial motion information is derived using positions 1 to 5 in FIG. 6A, and temporal motion information is derived using positions 6 or 7 in FIG. 6A. In the history-based motion information, every time a block is encoded or decoded, the motion information of the block is added to a first-in first-out (FIFO) list, and during the addition, some checks are required, such as whether the motion information overlaps with existing motion information in the list. In this way, when encoding or decoding a current block, the motion information in the history-based list can be referenced.

[0084] In some embodiments, the syntax description of the GPM is as shown in Table 1.

[0085] [Table 1]

[0086] As shown in Table 1, in merge mode, CIIP (combined inter-intra prediction) or GPM can be used for the current block if regular_merge_flag is not 1. If CIIP is not used for the current block, GPM is used for the current block, that is, as shown in the syntax "if( !ciip_flag[x0][y0] )" in Table 1.

[0087] As shown in Table 1 above, GPM requires three pieces of information to be transmitted in the bitstream: merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1. x0, y0 are used to determine the coordinates (x0, y0) of the upper left corner luminance sample of the current block relative to the upper left corner luminance sample of the image. merge_gpm_partition_idx is used to determine the partition shape of GPM, which is "simulated partition" as shown above. merge_gpm_partition_idx is the index of the weight derivation mode or weight derivation mode described in this embodiment. merge_gpm_idx0 is the index value of the first motion information in the candidate list, and merge_gpm_idx1 is the index value of the second motion information in the candidate list. merge_gpm_idx1 needs to be transmitted only if the length of the candidate list (MaxNumGpmMergeCand)>2. Otherwise, merge_gpm_idx1 can be determined directly.

[0088] In some embodiments, the GPM decoding process includes the following steps.

[0089] Input information for the decoding process includes the coordinates (xCb, yCb) of the luminance position of the top left corner of the current block relative to the luminance position of the top left corner of the image, the width cbWidth of the luminance component of the current block, the height cbHeight of the luminance component of the current block, luma motion vectors mvA and mvB with 1 / 16 fractional sample accuracy, chroma motion vectors mvCA and mvCB, reference image indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB.

[0090] Exemplarily, the motion information may be represented by a combination of a motion vector, a reference image index, and a prediction list flag. In some embodiments, two reference image lists are supported, and each reference image list may have multiple reference images. In unidirectional prediction, only one reference block in one reference image in one reference image list is used as a reference, and in bidirectional prediction, one reference block in one reference image in one reference image list and one reference block in one reference image in another reference image list are used as references. In GPM, two unidirectional predictions are used. A in the above mvA and mvB, mvCA and mvCB, refIdxA and refIdxB, predListFlagA and predListFlagB can be understood as a first prediction mode, and B can be understood as a second prediction mode. Optionally, X represents A or B, predListFlagX indicates whether the first or second reference image list is used for X, refIdxX indicates the reference image index in the reference image list used for X, mvX indicates the luma motion vector used for X, and mvCX indicates the chroma motion vector used for X. A combination of motion vectors, reference image indices, and prediction list flags may represent motion information as described in this application.

[0091] The output information of the decoding process includes a (cbWidth)X(cbHeight) matrix of luma prediction samples predSamplesL, optionally a (cbWidth / SubWidthC)X(cbHeight / SubHeightC) matrix of Cb chroma prediction samples, and optionally a (cbWidth / SubWidthC)X(cbHeight / SubHeightC) matrix of Cr chroma prediction samples.

[0092] For illustrative purposes, the luminance component is taken as an example, and the processing of the chroma component is similar to that of the luminance component.

[0093] Assume that the sizes of predSamplesLAL and predSamplesLBL are both (cbWidth)Χ(cbHeight), and are matrices of prediction samples obtained based on two prediction modes. predSamplesL is derived based on the following method. Determine predSamplesLAL based on luma motion vector mvA, chroma motion vector mvCA, reference image index refIdxA, and prediction list flag predListFlagA, and determine predSamplesLBL based on luma motion vector mvB, chroma motion vector mvCB, reference image index refIdxB, and prediction list flag predListFlagB. That is, prediction is performed based on each of the motion information of the two prediction modes, and the detailed process is not detailed. Usually, GPM is a merge mode, and the two prediction modes of GPM can both be considered to be merge modes.

[0094] Based on merge_gpm_partition_idx[xCb][yCb], use Table 2 to determine the GPM “split” angle index variable angleIdx and distance index variable distanceIdx.

[0095] [Table 2]

[0096] Since GPM can be used for any of the three components (e.g., Y, Cb, Cr), in some standard texts, the process of generating a GPM predicted sample matrix for one component is included in a sub-process called the weighted sample prediction process for GPM. This sub-process is called for each of the three components, but the parameters that are called are different, and here we will use only the luminance component as an example. The prediction matrix predSamplesL[xL][yL] (xL = 0..cbWidth - 1, yL = 0..cbHeight - 1) for the current luminance block is derived from the weighted sample prediction process for GPM. nCbW is set to cbWidth, nCbH is set to cbHeight, and the predicted sample matrices predSamplesLAL and predSamplesLBL, angleIdx, and distanceIdx generated in the two prediction modes are input.

[0097] In some embodiments, the weighted sample prediction and derivation process of the GPM includes the following steps.

[0098] The inputs of this process include the width of the current block nCbW, the height of the current block nCbH, two (nCbW)Χ(nCbH) predicted sample matrices predSamplesLA and predSamplesLB, a GPM “split” angle index variable angleIdx, a GPM distance index variable distanceIdx, and a component index variable cIdx. Take luminance as an example in this example. cIdx being 0 indicates the luminance component.

[0099] The output of this process contains the predicted sample values ​​of the GPM’s (nCbW)Χ(nCbH) matrix, pbSamples.

[0100] Exemplarily, the variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor are derived based on the following method. nW = ( cIdx = = 0 ) ? nCbW : nCbW * SubWidthC, nH = ( cIdx = = 0 ) ? nCbH : nCbH * SubHeightC, shift1 = Max( 5, 17 - BitDepth ), where BitDepth is the bit depth for encoding and decoding. offset1 = 1 << ( shift1 - 1 ), where "<<" means left shift. displacementX = angleIdx, displacementY = (angleIdx + 8) % 32, partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1, shiftHor = ( angleIdx % 16 = = 8 | | ( angleIdx % 16 != 0 && nH >= nW ) ) ? 0 : 1.

[0101] The variables offsetX and offsetY are derived based on the following method. If the value of shiftHor is 0, offsetX = ( -nW ) >> 1 offsetY = ( ( -nH ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nH ) >> 3 : -( ( distanceIdx * nH ) >> 3 ) ).

[0102] If the value of shiftHor is 1, offsetX = ( ( -nW ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nW ) >> 3 : -( ( distanceIdx * nW ) >> 3 ) offsetY = ( - nH ) >> 1.

[0103] The variables xL and yL are derived based on the following method. xL = ( cIdx = = 0 ) ? x : x * SubWidthC yL = ( cIdx = = 0 ) ? y : y * SubHeightC

[0104] The variable wValue indicating the weight of the prediction sample at the current position is derived based on the following method: wValue is the weight of the prediction sample value predSamplesLA[x][y] in the prediction matrix of the first prediction mode at (x,y), and (8 - wValue) is the weight of the prediction sample value predSamplesLB[x][y] in the prediction matrix of the first prediction mode at (x,y).

[0105] The distance matrix disLut is determined based on Table 3.

[0106] [Table 3]

[0107] weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ], weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx, wValue = Clip3( 0, 8, ( weightIdxL + 4 ) >> 3 ).

[0108] The predicted sample values ​​pbSamples[ x ][ y ] are derived based on the following method. pbSamples[ x ][ y ] = Clip3( 0, ( 1 << BitDepth ) - 1, ( predSamplesLA[ x ][ y ] * wValue + predSamplesLB[ x ][ y ] * ( 8 - wValue ) + offset1 ) >> shift1 ).

[0109] In addition, one weight value is derived for each position of the current block, and one GPM predicted value pbSamples[x][y] is calculated. In this method, the weight wValue does not need to be written in the form of a matrix, but if the wValue of each position is stored in a matrix, the matrix can be understood as a weight matrix. The principle of calculating the weight for each sample and weighting it to calculate the GPM predicted value is the same as the principle of calculating all weights and then weighting them uniformly to calculate the GPM predicted sample matrix. The term weight matrix is ​​used in many explanations of this application for the purpose of making the expression easier to understand, and the diagram drawn with the weight matrix is ​​more intuitive, but in fact it can also be explained with the weight of each position. For example, the weight matrix derivation mode can also be expressed as a weight derivation mode.

[0110] In some embodiments, as shown in FIG. 6B, the GPM decoding process can be expressed as follows: Analyze the bitstream to determine whether the GPM technique is used for the current block, and if the GPM technique is used for the current block, determine a weight derivation mode (or "split" mode, or weight matrix derivation mode), a first motion information, and a second motion information; Determine a first prediction block based on the first motion information, determine a second prediction block based on the second motion information, determine a weight matrix based on the weight matrix derivation mode, and determine a prediction block of the current block based on the first prediction block, the second prediction block, and the weight matrix.

[0111] It should be noted that in the embodiment of the present application, GPM or AWP belongs to a type of prediction technology. A flag indicating whether GPM or AWP is used needs to be transmitted in the bitstream. The flag can indicate whether GPM or AWP is used for the current block. If GPM or AWP is used, the encoder needs to transmit the specific mode to be used (i.e., any of 64 partition modes of GPM or any of 56 partition modes of AWP) and two unidirectional motion information index values ​​in the bitstream. That is, for the current block, the decoder can obtain information on whether GPM or AWP is used by decoding the bitstream. If it is determined that GPM or AWP is used, the decoder can analyze the prediction mode parameter of GPM or AWP and two motion information index values. For example, if the current block is divided into two partitions, a first index value corresponding to the first partition and a second index value corresponding to the second partition can be analyzed.

[0112] Specifically, in the GPM mode, when the GPM is used, the prediction mode parameters in the GPM (e.g., a specific partition mode of the GPM) are transmitted in the bitstream. Typically, the GPM includes 64 partition modes. In the AWP mode, when the AWP is used, the prediction mode parameters in the AWP (e.g., a specific partition mode of the AWP) are transmitted in the bitstream. Typically, the AWP includes 56 partition modes.

[0113] In inter prediction modes, for example, GPM and AWP, two unidirectional motion information are needed to search two reference blocks. In the current implementation, the encoder side uses related information of the encoded / decoded part before the current block to build one unidirectional motion information candidate list, selects unidirectional motion information from the unidirectional motion information candidate list, and writes the indexes of these two unidirectional motion information in the unidirectional motion information candidate list into the bitstream. The decoder side adopts the same method as the encoder side, that is, uses related information of the decoded part before the current block to build one unidirectional motion information candidate list. This unidirectional motion information candidate list is the same as the candidate list built by the encoder side. In this way, the indexes of the two unidirectional motion information are analyzed from the bitstream, and then the two unidirectional motion information (i.e., the two unidirectional motion information that need to be used for the current block) are searched from the unidirectional motion information candidate list.

[0114] In other words, the unidirectional motion information described in this application can include motion vector information (i.e., the value of (x, y)) and corresponding reference image information (i.e., the reference image list and the reference image index in the reference image list). In one representation manner, the reference image index in two reference image lists is recorded. The reference image index corresponding to one of the two reference image lists is valid, for example, 0, 1, 2, etc., and the reference image index corresponding to the other is invalid, for example, -1. The reference image list with the valid reference image index is the reference image list used for the motion information of the current block. According to the reference image index, the corresponding reference image can be found from the reference image list. Each reference image list has a corresponding motion vector, and the motion vector corresponding to the valid reference image list is valid, and the motion vector corresponding to the invalid reference image list is invalid. The decoder can find the required reference image from the reference image information in the unidirectional motion information, and according to the position and motion vector (i.e., the value of (x, y)) of the current block, can find the reference block from the reference image, and can further determine the inter-prediction value of the current block.

[0115] In the intra prediction method, the current block is predicted using the reconstructed samples that have been encoded and decoded around the current block as reference samples. FIG. 7A is a schematic diagram showing intra prediction. As shown in FIG. 7A, the size of the current block is 4×4, and the samples in the left column and the top row of the current block are reference samples for the current block, and the intra prediction uses these reference samples to predict the current block. All of these reference samples may be available, that is, they may all have been encoded and decoded. Alternatively, some of these reference samples may be unavailable. For example, the current block is located at the leftmost part of the entire image, and the reference sample to the left of the current block is unavailable. Or, when encoding and decoding the current block, the lower left sample of the current block has not yet been encoded and decoded, so the lower left reference sample is also unavailable. When the reference sample is unavailable, it may be filled in using the available reference sample, or using some value or some method, or it may not be filled in.

[0116] FIG. 7B is a schematic diagram showing intra prediction. As shown in FIG. 7B, in the multiple reference line (MRL) intra prediction method, more reference samples can be used to improve encoding and decoding efficiency. For example, four reference rows / columns are used as reference samples for the current block.

[0117] Furthermore, there are multiple prediction modes in intra prediction, and Figs. 8A to 8I are schematic diagrams showing intra prediction. As shown in Figs. 8A to 8I, H.264 can include nine modes when performing intra prediction on a 4x4 block. In mode 0 shown in Fig. 8A, a sample above the current block is copied to the current block along the vertical direction as a predicted value. In mode 1 shown in Fig. 8B, a reference sample to the left of the current block is copied to the current block along the horizontal direction as a predicted value. In mode 2 (DC) shown in Fig. 8C, the average value of eight samples A to D and I to L is used as a predicted value for all samples. In modes 3 to 8 shown in Figs. 8D to 8I, the reference sample is copied to the corresponding position of the current block along a certain angle. Since some positions of the current block cannot exactly correspond to the reference sample, it is necessary to use a weighted average value of the reference sample or an interpolated factional sample of the reference sample.

[0118] There are also other modes such as Planar mode. As technology develops and blocks are enlarged, the number of angle prediction modes is also increasing. FIG. 9 is a schematic diagram showing intra prediction modes. As shown in FIG. 9, for example, the intra prediction modes used in HEVC include Planar mode, DC mode, and 33 types of angle modes, totaling 35 types of prediction modes. FIG. 10 is a schematic diagram showing intra prediction modes. As shown in FIG. 10, the intra modes used in VVC include Planar mode, DC mode, and 65 types of angle modes, totaling 67 types of prediction modes. FIG. 11 is a schematic diagram showing intra prediction modes. As shown in FIG. 11, the intra modes used in AVS3 include DC mode, Plane mode, Bilinear mode, and 63 types of angle modes, totaling 66 types of prediction modes.

[0119] There are also techniques for improving prediction, such as improving fractional sample interpolation of reference samples and filtering predicted samples. For example, the multiple intra prediction filter (MIPF) in AVS3 uses different filters to generate predicted values ​​for blocks with different sizes. For samples at different positions in the same block, one filter is used to generate predicted values ​​for samples close to the reference sample, and another filter is used to generate predicted values ​​for samples far from the reference sample. Techniques for filtering predicted samples, such as the intra prediction filter (IPF) in AVS3, can filter predicted values ​​using a reference sample.

[0120] In intra prediction, encoding and decoding efficiency can be improved by using an intra mode coding technique using a Most Probable Modes (MPM) list. A mode list is constructed using intra prediction modes of encoded and decoded surrounding blocks, intra prediction modes derived based on intra prediction modes of encoded and decoded surrounding blocks (e.g., adjacent modes), and intra prediction modes with general or high usage probability (e.g., DC, Planar, Bilinear modes, etc.). Since textures have spatial continuity, referring to intra prediction modes of encoded and decoded surrounding blocks utilizes spatial relevance. MPM can be used to predict intra prediction modes. That is, it is considered that the probability that MPM is used for the current block is higher than the probability that MPM is not used for the current block. Therefore, fewer codewords are used for MPM during binarization, which saves overhead and improves encoding and decoding efficiency.

[0121] In GPM, two inter prediction blocks are combined using a weight matrix. In fact, the use of weight matrices can be extended to combine any two prediction blocks, such as two inter prediction blocks, two intra prediction blocks, one inter prediction block and one intra prediction block, etc. Furthermore, in screen content coding (SCC), intra block copy (IBC) or palette prediction blocks can be one or two prediction blocks.

[0122] In this application, intra prediction, inter prediction, IBC prediction, and palette prediction are referred to as different prediction methods. For ease of explanation, they are collectively referred to as prediction modes. The prediction mode can be understood as information on which an encoder / decoder can generate a prediction block of a current block. For example, in intra prediction, the prediction mode can be an intra prediction mode such as DC, Planar, and various intra angle prediction modes. Of course, some other auxiliary information can be superimposed, such as an optimization method of an intra reference sample, an optimization method after generating an initial prediction block (such as filtering). For example, in inter prediction, the prediction mode can be a skip mode, a merge mode, or a merge with motion vector difference (MMVD) mode, or an advanced motion vector predition (AMVP), and may be unidirectional prediction, bidirectional prediction, or multi hypothesis prediction. When unidirectional prediction is used for the inter prediction mode, one piece of motion information, which is unidirectional motion information, needs to be determined, and a prediction block can be determined based on the motion information. When bidirectional prediction is used for the inter prediction mode, one bidirectional motion information or two unidirectional motion information needs to be determined, and a prediction block can be determined based on the motion information. When multi-hypothesis prediction is used for the inter prediction mode, multiple unidirectional motion information needs to be determined, and a prediction block can be determined based on the motion information. Skip mode, merge mode, and normal inter mode can support unidirectional prediction, bidirectional prediction, or multi-hypothesis prediction. If one prediction mode is an inter prediction mode, motion information can be determined, and a prediction block can be determined based on the motion information.Template matching methods can be used based on skip mode, merge mode, MMVD mode, and normal inter mode, and such prediction modes can still be referred to as skip mode, merge mode, MMVD mode, and normal inter mode, or can also be referred to as skip mode using template matching, merge mode using template matching, MMVD mode using template matching, and normal inter mode using template matching.

[0123] In skip mode and merge mode, it is not necessary to transmit motion vector difference MVD in the bitstream, and in skip mode, it is also not necessary to transmit residual in the bitstream. MMVD can be considered as a special merge mode, and it represents some specific MVD by some flag bits. These specific MVD have only some possible pre-defined values. One example is MMVD mode in VVC. In MMVD mode, mmvd_direction_idx represents the direction of MVD, and the possible values ​​of mmvd_direction_idx are 0, 1, 2, and 3. 0 means that the horizontal component of MMVD is positive and the vertical direction is 0, 1 means that the horizontal component of MMVD is negative and the vertical direction is 0, 2 means that the horizontal component of MMVD is 0 and the vertical direction is positive, and 3 means that the horizontal component of MMVD is 0 and the vertical direction is negative. mmvd_distance_idx represents the absolute value of the above positive or negative value. Possible values ​​of mmvd_distance_idx are 0-7, which represent 1, 2, 4, 8, 16, 32, 64, 128 respectively if ph_mmvd_fullpel_only_flag == 0, and 4, 8, 16, 32, 64, 128, 256, 512 respectively if ph_mmvd_fullpel_only_flag == 1. A normal inter mode MVD can in theory represent any possible MVD within its valid range.

[0124] In this way, the information that needs to be determined for the GPM can be expressed as one weight derivation mode and two prediction modes. The weight derivation mode is used to determine a weight matrix or weights, and the two prediction modes are used to determine one prediction block or predicted value, respectively. The weight derivation mode is sometimes called a partition mode, but since it is a simulated partition, it is more familiar to call it the weight derivation mode in this application.

[0125] Optionally, the two prediction modes can be from the same or different prediction schemes, including but not limited to intra prediction, inter prediction, IBC, and palette.

[0126] A specific example is as follows: GPM is used for the current block, and this example is used for an inter-coded block, and merge mode in intra prediction and inter prediction can be used. As shown in Table 4, the addition of one syntax element intra_mode_idx indicates which prediction mode is an intra prediction mode. For example, intra_mode_idx is 0 indicates that both prediction modes are inter prediction modes, i.e., mode0IsInter is 1 and mode0IsInter is 1. Intra_mode_idx is 1 indicates that the first prediction mode is an intra prediction mode and the second prediction mode is an inter prediction mode, i.e., mode0IsInter is 0 and mode0IsInter is 1. Intra_mode_idx is 2 indicates that the first prediction mode is an inter prediction mode and the second prediction mode is an intra prediction mode, i.e., mode0IsInter is 1 and mode0IsInter is 0. When intra_mode_idx is 3, this indicates that both of the two prediction modes are intra prediction modes, that is, mode0IsInter is 0 and mode0IsInter is 0.

[0127] [Table 4]

[0128] In some embodiments, as shown in Figure 12, the GPM decoding process can be expressed as follows: Analyze the bitstream to determine whether the GPM technique is used for the current block, and if the GPM technique is used for the current block, determine a weight derivation mode (or "split" mode, or weight matrix derivation mode), a first intra prediction mode, and a second intra prediction mode; Determine a first prediction block based on the first intra prediction mode, determine a second prediction block based on the second intra prediction mode, determine a weight matrix based on the weight matrix derivation mode, and determine a prediction block of the current block based on the first prediction block, the second prediction block, and the weight matrix.

[0129] The template matching method is first used for inter prediction. In template matching, some areas around the current block are used as templates, using the correlation between adjacent samples. Before encoding and decoding the current block, the blocks to the left and above it have already been encoded and decoded according to the coding order. Of course, the implementation of existing hardware decoders does not necessarily ensure that the blocks to the left and above it have already been decoded before decoding the current block. Of course, what is referred to here is the inter block. For example, in HEVC, when generating a prediction block for an inter-coded block, the prediction process of the inter block can be performed in parallel without requiring the surrounding reconstructed samples. However, for an intra-coded block, the reconstructed samples to the left and above must be used as reference samples. In theory, the samples to the left and above are available, i.e., they can be realized by making corresponding adjustments to the hardware design. In contrast, the samples to the right and below are not available in the coding order in existing standards such as VVC.

[0130] As shown in FIG. 13, the left and upper rectangular regions of the current block are set as templates, and the height of the left template is generally the same as the height of the current block, and the width of the upper template is generally the same as the width of the current block. The template may have a different height or width than the current block. The best matching position of the template is found in the reference image to determine the motion information or motion vector of the current block. This process can be described as follows: In a certain reference image, a search is performed within a certain range from a starting position. Search rules such as a search range and a search step size can be set in advance. Each time a position is moved to, the degree of matching between the template corresponding to that position and the templates around the current block is calculated. The so-called degree of matching can be measured by several distortion costs such as sum of absolute difference (SAD) and sum of absolute transformed difference (SATD). Generally, the transformation used for SATD is Hadamard transformation, mean-square error (MSE), etc. The smaller the value of SAD, SATD, MSE, etc., the higher the degree of matching. The cost is calculated using the predicted block of the template corresponding to the position and the reconstruction block of the template around the current block. In addition to searching for integer sample positions, a search for fractional sample positions can also be performed. The motion information of the current block can be determined based on the position with the highest degree of matching found. Using the relationship between adjacent samples, the motion information suitable for the template can be the motion information suitable for the current block. Of course, the template matching method is not necessarily effective for all blocks, so several methods can be used to determine whether the template matching method is used for the current block. For example, a control switch is used for the current block to indicate whether the template matching method is used.One name for such a template matching method is decoder side motion vector derivation (DMVD). Both the encoder and the decoder can use a template to perform a search to derive motion information, or find better motion information based on the original motion information. There is no need to transmit a specific motion vector or a motion vector difference, and both the encoder and the decoder perform the search according to the same rules, ensuring consistency between encoding and decoding. The template matching method can improve compression performance, but it also requires the decoder to perform a "search", which brings about a certain degree of complexity for the decoder.

[0131] Although the method of applying template matching to inter prediction has been described above, the template matching method can also be used for intra prediction. For example, the intra prediction mode is determined using a template. Similarly, for the current block, a certain range of regions on the upper and left sides of the current block can be used as a template, for example, the left rectangular region and the upper rectangular region still shown in FIG. 13. When encoding and decoding the current block, the reconstructed samples in the template can be used. This process can be described as follows. A set of candidate intra prediction modes is determined for the current block, and the candidate intra prediction modes constitute a subset of all available intra prediction modes. Of course, the candidate intra prediction modes may be a universal set of all available intra prediction modes. It can be determined based on a trade-off between performance and complexity. The set of candidate intra prediction modes can be determined according to MPM or some rules (e.g., equal interval selection). Calculate the cost of each candidate intra prediction mode in the template, such as SAD, SATD, MSE, etc. The mode is used to perform prediction with the template to obtain a predicted block, and the predicted block and the reconstruction block of the template are used to calculate the cost. The mode with a low cost may be matched with the template better, and the intra-prediction mode that performs well with the template may be the intra-prediction mode that performs well with the current block by using the similarity between adjacent samples. Select one or more modes with a low cost. Of course, the above two steps may be repeated, for example, after selecting one or more modes with a low cost, determine again a set of candidate intra-prediction modes, calculate again the cost for the newly determined set of candidate intra-prediction modes, and select one or more modes with a low cost. It can also be understood as coarse selection and fine selection.The final selected intra prediction mode is determined as the intra prediction mode of the current block, or the final selected intra prediction modes are determined as candidates for the intra prediction mode of the current block. Of course, the set of candidate intra prediction modes can also be sorted by template matching method alone, for example, sorting the MPM list, that is, for each mode in the MPM list, obtain a prediction block against the template to determine the cost, and sort the modes in ascending order of cost. In general, the earlier a mode is located in the MPM list, the less overhead in the bitstream. This can improve compression efficiency.

[0132] The template matching method can be used to determine the two prediction modes of the GPM. When the template matching method is used in the GPM, for a current block, one control switch can be used to control whether template matching is used for the two prediction modes of the current block, or two control switches can be used to control whether template matching is used for each of the two prediction modes, respectively.

[0133] Another aspect is how to use template matching. For example, when GPM is used in merge mode, for example, GPM in VVC uses merge_gpm_idxX to determine one piece of motion information from mergeCandList, where X is 0 or 1. For the Xth motion information, one method is to optimize using a template matching method based on the motion information. That is, when one piece of motion information is determined from mergeCandList based on merge_gpm_idxX and template matching is used for the motion information, optimization is performed based on the motion information using a template matching method. Another method is to determine one piece of motion information by directly searching based on default motion information, rather than determining one piece of motion information from mergeCandList using merge_gpm_idxX.

[0134] If the Xth prediction mode is an intra prediction mode and the template matching method is used for the Xth prediction mode of the current block, the intra prediction mode can be determined using the template matching method without needing to indicate the index of the intra prediction mode in the bitstream, or the candidate set or MPM list is determined using the template matching method, and the index of the intra prediction mode is indicated in the bitstream.

[0135] In the GPM, after determining the weight derivation mode, the area occupied by each prediction mode can be determined. The so-called occupied area can be understood as an area where the weight corresponding to the prediction mode is the maximum value, or an area where the weight corresponding to the prediction mode is equal to or greater than a certain threshold. The reason why the GPM can improve compression performance is that the two parts "split" based on the GPM are different. Therefore, when determining the prediction mode of the GPM using a template matching method, the template can also be split. In the existing technology, the template can be divided into three types: left, upper, and all (left + upper). The split of the template is related to the weight derivation mode. For example, as shown in Table 5, the split of the template in the existing technology is related to the "split" angle or the "split" angle index angleIdx.

[0136] [Table 5] JPEG2024545531000007.jpg54156

[0137] For example, the left template is marked as TM_A, the top template as TM_L, and the total (left + top) template as TM_AL. The relationship between the templates and the "split" angle index is shown in Table 5, where some angle indexes such as 1, 6, and 7 are not used in the current GPM and therefore do not have a corresponding template and are indicated by / .

[0138] Hereinafter, with reference to FIG. 14, a video decoding method according to an embodiment of the present application will be described by taking the decoding side as an example.

[0139] Figure 14 is a flowchart illustrating a prediction method according to an embodiment of the present application. The embodiment of the present application is applied to the video decoder shown in Figure 1 and Figure 3. As shown in Figure 14, the method of the embodiment of the present application includes the following contents:

[0140] S101: Decode a bitstream to determine K initial prediction modes.

[0141] As can be seen from the above, in order to further improve prediction accuracy, a current block can be predicted using multiple prediction modes. In this case, in order to maintain prediction consistency between the encoding side and the decoding side, in some embodiments, the encoding side needs to indicate the weight derivation mode in the bitstream, for example, by writing the index of the weight derivation mode into the bitstream. Taking VVC as an example, the 64 weight derivation modes of GPM are encoded with equal probability. That is, during binarization and de-binarization, a fixed length of 6 bits is used for 64 weight derivation modes, which increases the coding cost.

[0142] In the embodiment of the present application, in order to reduce coding cost, the decoding side derives weight derivation mode according to prediction mode. Specifically, the decoding side determines K initial prediction modes by decoding bitstream, then determines the weight derivation mode of current block according to these K initial prediction modes, and further determines the predicted value of current block according to the weight derivation mode of current block.

[0143] The K initial prediction modes of the current block include the following several examples:

[0144] Example 1: All of the K initial prediction modes are intra prediction modes.

[0145] Example 2: All of the K initial prediction modes are inter prediction modes.

[0146] Example 3: Among the K initial prediction modes, at least one is an intra prediction mode and at least one is an inter prediction mode.

[0147] Example 4: Among the above K initial prediction modes, at least one is an intra prediction mode, and at least one is a non-inter prediction mode and a non-intra prediction mode, such as an intra block copy (IBC) prediction mode or a palette prediction mode.

[0148] Example 5: Among the K initial prediction modes, at least one is an inter prediction mode, and at least one is a non-inter prediction mode and a non-intra prediction mode, such as an IBC prediction mode or a palette prediction mode.

[0149] Example 6: None of the K initial prediction modes are intra prediction modes or inter prediction modes. For example, one initial prediction mode is an IBC prediction mode and one initial prediction mode is a palette prediction mode.

[0150] It should be noted that the embodiment of the present application does not limit the specific types of the K initial prediction modes.

[0151] 15 is a schematic diagram of a case where a current block is predicted using two prediction modes. As shown in FIG. 15, during prediction of the current block, a first prediction value is determined using a first prediction mode, and a second prediction value is determined using a second prediction mode, and then the first prediction value and the second prediction value are weighted using a weight to obtain a prediction value of the current block.

[0152] In the above S101, the manner in which the decoding side decodes the bitstream and determines the K initial prediction modes includes, but is not limited to, the following:

[0153] Method 1: The bitstream includes indexes of K initial prediction modes. Therefore, the decoding side can obtain the indexes of these K initial prediction modes by decoding the bitstream, and determine the K initial prediction modes based on the indexes. For example, assuming that the K initial prediction modes are intra prediction modes, the encoding side writes the indexes of these K initial intra prediction modes into the bitstream. Assuming that K=2 and the two initial prediction modes are angular prediction mode 31 and angular prediction mode 35 among the 65 angular prediction modes of VVC shown in FIG. 10, the encoding side writes index 31 and index 35 into the bitstream. The decoding side decodes the bitstream to obtain index 31 and index 35, and further obtains the intra prediction mode corresponding to index 31 and the intra prediction mode corresponding to index 35 from the 65 angular prediction modes of VVC shown in FIG. 10 as two initial prediction modes.

[0154] Method 2: The above S101 includes steps S101-A1 to S101-A3. S101-A1: determine a list of selected prediction modes, the list of selected prediction modes including at least two selected prediction modes; S101-A2: Decode the bitstream to obtain a prediction mode index. S101-A3: Determine K initial prediction modes from the list of selected prediction modes based on a prediction mode index.

[0155] In some embodiments, if the K initial prediction modes are all intra prediction modes, the selected prediction modes included in the list of selected prediction modes are all intra prediction modes, for example, including at least one of DC mode, planar mode, angular mode, etc. Optionally, the list of selected prediction modes may also include intra prediction modes in an MPM list.

[0156] In some embodiments, if all of the K initial prediction modes are inter prediction modes, all of the selected prediction modes included in the list of selected prediction modes are inter prediction modes, for example including at least one of skip mode, merge mode, normal inter prediction mode, etc.

[0157] In some embodiments, if the K initial prediction modes include both inter-prediction modes and intra-prediction modes, the list of selected prediction modes includes at least one intra-prediction mode and at least one inter-prediction mode.

[0158] Optionally, the list of selected prediction modes may also include modes such as IBC mode, palette mode, and the like.

[0159] Optionally, the list of selected prediction modes may include unidirectional prediction, bidirectional prediction, multi-hypothesis prediction, and the like.

[0160] The present application does not limit the types and number of the selected prediction modes included in the above list of selected prediction modes.

[0161] The embodiment of the present application does not limit the manner of determining the list of selected prediction modes.

[0162] In one example, the selected prediction modes included in the list of selected prediction modes are preset modes.

[0163] In one example, the list of selected prediction modes is an MPM list.

[0164] In one example, the list of selected prediction modes is a set of prediction modes determined according to some rules, such as equal interval selection.

[0165] In one example, both the encoding side and the decoding side default that the selected prediction mode in the list of selected prediction modes is determined by template matching. For example, the decoding side predicts the template of the current block using a prediction mode to obtain a predicted value of the template, and determines the cost of the prediction mode according to the predicted value of the template and the reconstructed value of the template, for example, SAD cost, SATD cost, or MSE cost. Thereby, the decoding side can sort the prediction modes according to the cost of each prediction mode to obtain the list of selected prediction modes, for example, determine a number of prediction modes with the smallest cost as selected prediction modes to configure the list of selected prediction modes.

[0166] In an embodiment of the present application, the prediction mode index included in the bitstream may be an index of at least one initial prediction mode among the K initial prediction modes. For example, the prediction mode index is an index of the K initial prediction modes, or is an index of the first initial prediction mode among the K initial prediction modes. A decoding side decodes the bitstream to obtain the prediction mode index, and determines the K initial prediction modes from the determined selected prediction mode list according to the prediction mode index.

[0167] Exemplarily, when the prediction mode index included in the bitstream is the index of K initial prediction modes, the decoding side determines the K selected prediction modes corresponding to the indexes of the K initial prediction modes in the list of selected prediction modes as the K initial prediction modes.

[0168] For example, when a prediction mode index included in a bitstream is the index of the first initial prediction mode among K initial prediction modes, the decoding side determines the selected prediction mode corresponding to the index in the list of selected prediction modes as the first initial prediction mode among the K initial prediction modes, and determines (K-1) selected prediction modes corresponding to the (K-1) indexes following the index in the list of selected prediction modes as the remaining (K-1) initial prediction modes among the K initial prediction modes.

[0169] In an embodiment of the present application, the decoding side determines K initial prediction modes based on the above steps, determines a weight derivation mode of the current block based on these K initial prediction modes, and further predicts the current block based on the weight derivation mode of the current block to determine a predicted value of the current block.

[0170] In some embodiments, before determining the K initial prediction modes, the decoding side needs to first determine whether to perform weighted prediction on the current block using K different prediction modes. If the decoding side determines that the current block is to perform weighted prediction using K different prediction modes, it performs S101 to determine the K initial prediction modes.

[0171] In one possible embodiment, the decoding side can determine whether to perform weighted prediction using two different prediction modes for the current block by determining a prediction mode parameter of the current block.

[0172] Optionally, in an embodiment of the present application, the prediction mode parameter may indicate whether GPM mode or AWP mode can be used for the current block, i.e., whether prediction can be performed for the current block using K different prediction modes.

[0173] It should be noted that in the embodiment of the present application, the prediction mode parameter may be understood as a flag bit indicating whether the GPM mode or the AWP mode flag bit is used. Specifically, the encoding side may use one variable as the prediction mode parameter, and thereby the setting of the prediction mode parameter is realized by setting the value of the variable. Exemplarily, in the present application, when the GPM mode or the AWP mode is used for the current block, the encoding side may set the value of the prediction mode parameter to one indicating that the GPM mode or the AWP mode is used for the current block, and specifically, the encoding side may set the value of the variable to 1. Exemplarily, in the present application, when the GPM mode or the AWP mode is not used for the current block, the encoding side may set the value of the prediction mode parameter to one indicating that the GPM mode or the AWP mode is not used for the current block, and specifically, the encoding side may set the value of the variable to 0. Furthermore, in the embodiment of the present application, after completing the setting of the prediction mode parameter, the encoding side may write the prediction mode parameter into a bitstream and transmit it to the decoding side. Thereby, the decoding side may obtain the prediction mode parameter after analyzing the bitstream.

[0174] Based on the above, the decoding side decodes the bitstream to obtain a prediction mode parameter, and then determines whether to use the GPM mode or the AWP mode for the current block according to the prediction mode parameter. If the current block uses the GPM mode or the AWP mode, that is, if prediction is performed using K different prediction modes, determine K initial prediction modes.

[0175] It should be noted that in the embodiment of the present application, GPM mode or AWP mode is a kind of prediction method. Specifically, K different prediction modes are determined for the current block, then K predicted values ​​are determined according to these K different prediction modes, and then weights can be determined, and the K predicted values ​​are combined according to the weights to finally obtain the predicted value of the current block.

[0176] In some embodiments, when using the GPM mode or the AWP mode, the size of the current block may be restricted. In the prediction method according to the embodiment of the present application, it is necessary to generate K predicted values ​​using K different prediction modes, and weight them based on the weights to obtain a predicted value of the current block. In order to reduce complexity, and considering the trade-off between compression performance and complexity, in the embodiment of the present application, it is possible to restrict the use of the GPM mode or the AWP mode for a block having a certain size. Therefore, in the present application, the decoding side can first determine a size parameter of the current block, and then determine whether to use the GPM mode or the AWP mode for the current block according to the size parameter.

[0177] It should be noted that in the embodiment of the present application, the size parameters of the current block may include the height and width of the current block. Thus, the decoding side may use the height and width of the current block to restrict the use of the GPM mode or the AWP mode. Exemplarily, in the present application, if the width is greater than a first threshold and the height is greater than a second threshold, it is determined that the current block is to use the GPM mode or the AWP mode. As can be seen above, one possible restriction is to use the GPM mode or the AWP mode only if the width of the block is greater than (or equal to or greater than) the first threshold and the height of the block is greater than (or equal to or greater than) the second threshold. The values ​​of the first and second thresholds may be 8, 16, 32, etc., and the first threshold may be equal to the second threshold. Exemplarily, in the present application, if the width is less than a third threshold and the height is greater than a fourth threshold, it is determined that the current block is to use the GPM mode or the AWP mode. As seen above, one possible restriction is to only utilize AWP mode if the width of the block is less than (or less than or equal to) the third threshold and the height of the block is greater than (or greater than or equal to) the fourth threshold. The values ​​of the third and fourth thresholds may be 8, 16, 32, etc., and the third threshold may be equal to the fourth threshold.

[0178] In some embodiments, in the embodiment of the present application, the size of the block that can use the GPM mode or the AWP mode can be restricted by the restriction on the sample parameter. Exemplarily, in the present application, the decoding side can first determine the sample parameter of the current block, and then determine whether the current block can use the GPM mode or the AWP mode based on the sample parameter and the fifth threshold. As can be seen above, one possible restriction is to use the GPM mode or the AWP mode only if the number of samples in the block is greater than (or equal to or greater than) the fifth threshold. The value of the fifth threshold may be 8, 16, 32, etc. That is, in the present application, the current block can use the GPM mode or the AWP mode only under the condition that the size parameter of the current block meets the size requirement.

[0179] In some embodiments, there may be an image-level flag in the present application to determine whether the technical solution of the present application is applied to the current image to be decoded. For example, the present application may be configured to apply to intraframes (e.g., I frames) and not to interframes (e.g., B frames, P frames). Or, the present application may be configured to apply to interframes and not to intraframes. Or, the present application may be configured to apply to some interframes and not to other parts of interframes. Since intraframe prediction can also be used for interframes, the present application may also be applied to interframes.

[0180] In some embodiments, there may also be flags below the image level and above the CU level (e.g., tile, slice, patch, LCU, etc.) to determine whether the technical proposal of the present application is applied to this area.

[0181] S102: Determine a weight derivation mode of a current block according to the K initial prediction modes.

[0182] In this application, the weight derivation mode is used to determine the weight used for the current block. Specifically, the weight derivation mode can be a mode for deriving weights. For a block having a certain length and width, one weight matrix can be derived for each weight derivation mode. For blocks having the same size, the weight matrices derived by different weight derivation modes are different.

[0183] For example, in this application, there are 56 weight derivation modes in AWP and 64 weight derivation modes in GPM.

[0184] In the embodiment of the present application, the decoding side determines the weight derivation mode of the current block according to the determined K initial prediction modes. In some cases, the encoding side does not need to write the information related to the weight derivation mode of the current block into the bitstream, thereby saving codewords and reducing coding cost.

[0185] The embodiment of the present application does not limit the manner in which the decoding side determines the weight derivation mode of the current block based on the K initial prediction modes.

[0186] In some embodiments, the decoding side determines the weight derivation mode of the current block according to the template of the current block by K initial prediction modes. Taking VVC as an example, GPM has 64 weight derivation modes, and the decoding side estimates which weight derivation mode is more likely to be selected and which weight derivation mode is less likely to be selected by analyzing the template. Taking Inter GPM as an example, a typical scene is a scene where two objects move toward each other, and GPM can be used for the edges of the two objects. Assuming that both the current block and the template contain the edges of the objects, the edges in the template can be used to infer the edges of the current block. As an example, two pieces of motion information are given, and two predicted values ​​of the template are obtained based on the two pieces of motion information, which are respectively denoted as the first template predicted value and the second template predicted value. A template weight is derived based on a certain weight derivation mode, and a predicted value of the template corresponding to the weight derivation mode is determined based on the first template predicted value, the second template predicted value and the template weight. Since the reconstruction value of the template is available, the cost of the weight derivation mode in the template can be obtained based on the predicted value of the template corresponding to the weight derivation mode and the reconstruction value of the template. Thus, the probability that the weight derivation mode is selected can be estimated based on the cost of each weight derivation mode in the template. One possible embodiment is to sort the weight derivation modes based on the cost of each weight derivation mode in the template. The mode with the smallest cost is located at the front, and the mode with the largest cost is located at the back. It is considered that the sorted weight derivation modes are roughly sorted from the most probable to the least probable, so that they can be processed using the method of variable length coding. One possible embodiment is as follows: A list of weight derivation modes is constructed, and the encoding side writes the index in the list of the weight derivation mode of the current block selected for the current block into the bitstream.The decoder analyzes the index in the bitstream to determine the weight derivation mode of the current block from the list.

[0187] It can be seen from the above that, in the embodiment of the present application, in the process of determining the weight derivation mode of the current block based on the K initial prediction modes, it is necessary to determine the template weight based on the weight derivation mode.

[0188] The process of determining template weights based on the weight derivation mode according to an embodiment of the present application is described below.

[0189] Template matching uses the correlation between adjacent samples to select some areas around the current block as templates. Before encoding and decoding the current block, the blocks to its left and above have already been decoded according to the coding order. Inter prediction determines the motion information or motion vector of the current block by finding the best matching position of the template from the reference image. Intra prediction uses the template to determine the intra prediction mode of the current block.

[0190] The present application does not limit the specific shape of the template of the current block.

[0191] In some embodiments, the template includes at least one of the above decoded region, the left decoded region, and the top-left decoded region of the current block.

[0192] Optionally, the width of the upper decoded area is the same as the width of the current block, the height of the left decoded area is the same as the height of the current block, the width of the upper left decoded area is the same as the width of the left decoded area, and the height of the upper left decoded area is the same as the height of the upper decoded area.

[0193] Currently, as shown in Table 5, for example, K=2, the K prediction modes include a first prediction mode and a second prediction mode, and the template corresponding to the first prediction mode and the template corresponding to the second prediction mode are the upper decoded region of the current block, or the left decoded region of the current block, or the left decoded region and the upper decoded region of the current block, respectively. For example, as shown in FIG. 16, taking the weight derivation mode with a GPM index of 2 as an example, the white area of ​​the weight matrix of the current block is the weight corresponding to the predicted value of the first prediction mode, and the black area is the weight corresponding to the predicted value of the second prediction mode. As shown in FIG. 16, the first template corresponding to the first prediction mode is the upper decoded region of the current block, and the second template corresponding to the second prediction mode is the left decoded region of the current block, but the template close to the second prediction mode includes a part of the upper decoded region in addition to the left region. Therefore, in the existing technology, the division of the template is not fine enough, and as a result, the weight derivation mode of the current block cannot be accurately derived based on the template.

[0194] In order to solve the technical problem, the present application can realize finer division of the template according to the weight derivation mode. For example, as shown in FIG. 17A, the present application extends a division line of a weight matrix corresponding to a weight derivation mode of a current block toward the template region to divide the template. For example, a template on one side of the division line is denoted as a first template, and a template on the other side of the division line is denoted as a second template. The first template corresponds to a first prediction mode, the second template corresponds to a second prediction mode, and the weight of the first template is determined based on the weight corresponding to the first prediction mode, and the weight of the second template is determined based on the weight corresponding to the second prediction mode. For example, the weight of the first template is the same as the weight corresponding to the first prediction mode, and the weight of the second template is the same as the weight corresponding to the second prediction mode.

[0195] In some embodiments, the first and second templates divided according to the above method may not be rectangular, for example, as shown in Figure 17A, the first and second templates have slanted sides, but the calculation of the cost for the irregular templates is relatively complicated.

[0196] To reduce the complexity of template matching, in some embodiments, the weight of the template can be determined by extending the weight matrix directly toward the template region, for example, toward the left or toward the top, to cover the template. For example, as shown in FIG. 17B, a rectangular region in the upper left region of the current block can be selectively added to the template, and the template and the current block together form a rectangle. Of course, it is also possible to use only the left and upper parts as the template. As shown in FIG. 17B, taking the upper left region as an example, the inverted L-shaped regions on the left, upper left, and upper sides are the template region, the rectangular region on the lower right is the current block, and the weight matrix extended to the upper left is the weight matrix of the template. In this way, instead of actually dividing the template, the weight is used to simulate the division of the template. It can also be said that the weight derivation mode is used to simulate the division of the template, or that the weight derivation mode is used to determine the template weight, and the cost of template matching is determined based on the template weight. Since different block shapes may result in different template divisions, in the present application, it is not necessary to set various rules for blocks having various shapes, and it is not necessary to set, for example, the correspondence between prediction modes and templates shown in Table 5. In addition, regarding the accuracy of "division," the simulated division in the present application is finer and more accurate than the method that distinguishes only the left region and the upper region.

[0197] In some embodiments, the left and upper regions of the current block shown in FIG. 17B are taken as the template for the current block, and the upper left matrix region is not considered.

[0198] During template matching, it is possible to use only one component (Y, Cb, Cr or R, G, B). For example, in the YUV format, Y is dominant, so Y can be used, but of course all components can be used. For ease of explanation, this application takes one component as an example, for example, template matching is performed on the Y component, and other components can refer to the Y component.

[0199] In an embodiment of the present application, the process of deriving the template weight based on the weight derivation mode and the process of deriving the weight of the predicted value can be combined, for example, the template weight and the weight of the predicted value are derived simultaneously. The weight of the predicted value can be understood as a weight corresponding to the predicted value. For example, a first predicted value is obtained using a first prediction mode, a second predicted value is obtained using a second prediction mode, a first weight of the first predicted value is determined based on the weight derivation mode, a second weight of the second predicted value is determined based on the weight derivation mode, and the sum of the product of the first predicted value and the first weight and the product of the second predicted value and the second weight is determined as the predicted value of the current block.

[0200] In this application, the first weight and the second weight are referred to as predictor weights to distinguish them from the template weights.

[0201] In some embodiments, determining the template weights based on the weight derivation mode includes the following steps. Step A: Determine angle indexing and distance indexing based on the weight derivation mode. Step B: Determine the template weights based on the angle index, the distance index, and the size of the template.

[0202] In the present application, the template weights can be derived in the same manner as the weights of the predictors are derived. For example, the angle indexing and the distance indexing can be determined based on the weight derivation mode first. The angle indexing can be understood as the angle indexing of the boundary of each weight derived by the weight derivation mode. For example, the angle indexing and the distance indexing corresponding to the weight derivation mode can be determined according to Table 2 above. For example, when the weight derivation mode is 27, the corresponding angle indexing is 12, and the corresponding distance indexing is 3. Then, the template weights are determined based on the angle index, the distance index, and the size of the template.

[0203] The manner of deriving template weights based on the weight derivation mode in the above step B includes, but is not limited to, the following:

[0204] Method 1: Determine the template weight directly according to the angle index, the distance index and the size of the template. In this case, the above step B includes the following steps B11 to B13. Step B11: Determine a first parameter of the sample in the template according to the angle index, the distance index and the size of the template. Step B12: Determine weights of the samples in the template according to the first parameters of the samples in the template. Step B13: Determine the template weight based on the weights of the samples in the template.

[0205] In this embodiment, the weights of the samples in the template are determined based on the angle index, the distance index, the size of the template, and the size of the current block, and a weight matrix consisting of the weights of each sample in the template is determined as the template weight.

[0206] The first parameter in this application is used to determine the weight, and in some embodiments, the first parameter is also referred to as a weight index.

[0207] In one possible embodiment, the offset and the first parameter can be determined as follows: The inputs of the template weight derivation process include the width nCbW of the current block, the height nCbH of the current block, the width nTmW of the left template, the height nTmH of the top template, the "split" angle index variable angleId of the GPM, the distance index variable distanceIdx of the GPM, and the component index variable cIdx, as shown in Fig. 18. For illustrative purposes, the present application takes the luminance component as an example, and cIdx being 0 indicates the luminance component.

[0208] The variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor are derived based on the following method. nW = ( cIdx = = 0 ) ? nCbW : nCbW * SubWidthC nH = ( cIdx = = 0 ) ? nCbH : nCbH * SubHeightC shift1 = Max( 5, 17 - BitDepth ), where BitDepth is the bit depth for encoding and decoding. offset1 = 1 << ( shift1 - 1 ) displacementX = angleIdx displacementY = (angleIdx + 8) % 32 partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1 shiftHor = ( angleIdx % 16 = = 8 | | ( angleIdx % 16 != 0 && nH >= nW ) ) ? 0 : 1

[0209] The offsets offsetX and offsetY are derived based on the following method. - If the value of shiftHor is 0, offsetX = ( -nW ) >> 1 offsetY = ( ( -nH ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nH ) >> 3 : -( ( distanceIdx * nH ) >> 3 ) ) - Otherwise (i.e., shiftHor has the value 1) offsetX = ( ( -nW ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nW ) >> 3 : -( ( distanceIdx * nW ) >> 3 ) offsetY = ( - nH ) >> 1

[0210] The template weight matrix wTemplateValue[x][y] (x = -nTmW..nCbW - 1, y = -nTmH..nCbH - 1, except when both x and y are 0 or greater) is derived based on the following method (note that in this example, the coordinates of the upper left corner of the current block are set to (0,0)). - The variables xL and yL are derived based on the following method: xL = ( cIdx = = 0 ) ? x : x * SubWidthC yL = ( cIdx = = 0 ) ? y : y * SubHeightC

[0211] The disLut is determined according to Table 3 above.

[0212] The first parameter weightIdx is derived based on the following method. weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ]

[0213] After determining the first parameter weightIdx according to the above method, the weight of the sample (x, y) in the template is determined according to weightIdx.

[0214] In this application, the manner of determining the weight of the sample in the template based on the first parameter of the sample in the template in the above step B12 includes but is not limited to the following:

[0215] Method 1: determining a second parameter of the sample in the template based on a first parameter of the sample in the template, and determining a weight of the sample in the template based on the second parameter of the sample in the template.

[0216] The second parameter is also used to determine the weight, which in some embodiments is also referred to as a weight index on the first component, which may be a luma component, a chroma component, etc.

[0217] For example, the weights of the samples in the template are determined based on the following formula: weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx wTemplateValue[x][y] = Clip3( 0, 8, ( weightIdxL + 4 ) >> 3 )

[0218] wTemplateValue[x][y] is the weight of sample (x,y) in the template. weightIdxL is the second parameter of sample (x,y) in the template, also called the weight index in the first component (e.g., luminance component). wTemplateValue[x][y] is the weight of sample (x,y) in the template. partFlip is an intermediate variable determined based on the angle index angleIdx. For example, as described above, partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1. That is, the value of partFlip is 1 or 0. If the value of partFlip is 0, weightIdxL is 32 - weightIdx. If the value of partFlip is 1, weightIdxL is 32 + weightIdx. Here, 32 is merely an example, and the present application is not limited thereto.

[0219] Manner 2: Determine the weight of the sample in the template according to a first parameter of the sample in the template, a first preset value, and a second preset value.

[0220] In order to reduce the computational complexity of the template weight, in method 2, the weight of the sample in the template is limited to a first preset value or a second preset value, that is, the weight of the sample in the template is set to either the first preset value or the second preset value, thereby reducing the computational complexity of the template weight.

[0221] The present application does not limit the specific values ​​of the first preset value and the second preset value.

[0222] Optionally, the first preset value is one.

[0223] Optionally, the second preset value is zero.

[0224] As an example, the weights of the samples in the template can be determined based on the following formula: wTemplateValue[x][y] = (partFlip ? weightIdx: - weightIdx) > 0 ? 1 : 0

[0225] wTemplateValue[x][y] is the weight of the sample (x, y) in the template, and the 1 in the above “1:0” is the first preset value, and the 0 is the second preset value.

[0226] In the above method 1, the weight of each sample in the template is determined based on the weight derivation mode, and the weight matrix consisting of the weights of each sample in the template is taken as the template weight.

[0227] Method 2: Determine the weight of the current block and the template weight according to the weight derivation mode. That is, in this method 2, the merged region consisting of the current block and the template is taken as the whole, and the weight of the samples in the merged region is derived according to the weight derivation mode. Based on this, the above step B includes the following steps B21 and B22. Step B21: Determine the weights of the samples in the merged region consisting of the current block and the template according to the angle index, the distance index, the size of the template, and the size of the current block. Step B22: Determine the template weight based on the size of the template and the weight of the samples in the merged region.

[0228] In the second method, the current block and the template are taken as a whole, and the weight of the samples in the merged region formed by the current block and the template is determined according to the angle index, the distance index, the size of the template, and the size of the current block, and then the weight corresponding to the template in the weight of the samples in the merged region is determined as the template weight according to the size of the template. For example, as shown in FIG. 18, the weight corresponding to the L-shaped template region in the merged region is determined as the template weight.

[0229] In the method 2, in the weight determination process, not only the template weight is determined, but also the prediction weight is determined. That is, after removing the template weight from the weight corresponding to the merged region, what remains is the prediction weight. Therefore, the subsequent prediction process can be performed based on the prediction weight without the need to determine the prediction weight again. Therefore, the prediction step can be reduced and the prediction efficiency can be improved.

[0230] The present application does not limit the specific embodiment of determining the weights of samples in the merged region consisting of the current block and the template based on the angle index, the distance index, the size of the template, and the size of the current block.

[0231] In some embodiments, determining the weights of the samples in the merged region in the above step B21 includes the following steps B211 to B212. Step B211: Determine a first parameter of the samples in the merged region according to the angle index, the distance index, and the size of the merged region. Step B212: Determine weights of the samples in the merged region according to the first parameters of the samples in the merged region.

[0232] In this embodiment, the weights of the samples in the merged region are determined based on the angle index, the distance index, and the size of the merged region, and the weights of each sample in the merged region form a weight matrix.

[0233] In one possible embodiment, the offset and the first parameter can be determined as follows: The inputs of the merging region weight derivation process include the width nCbW of the current block, the height nCbH of the current block, the width nTmW of the left template, the height nTmH of the top template, the "split" angle index variable angleId of the GPM, the distance index variable distanceIdx of the GPM, and the component index variable cIdx, as shown in Fig. 18. For illustration, the present application takes the luminance component as an example, and cIdx being 0 indicates the luminance component.

[0234] The variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor are derived based on the following method. nW = ( cIdx = = 0 ) ? nCbW : nCbW * SubWidthC nH = ( cIdx = = 0 ) ? nCbH : nCbH * SubHeightC shift1 = Max( 5, 17 - BitDepth ), where BitDepth is the bit depth for encoding and decoding. offset1 = 1 << ( shift1 - 1 ) displacementX = angleIdx displacementY = (angleIdx + 8) % 32 partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1 shiftHor = ( angleIdx % 16 = = 8 | | ( angleIdx % 16 != 0 && nH >= nW ) ) ? 0 : 1

[0235] The offsets offsetX and offsetY are derived based on the following method. - If the value of shiftHor is 0, offsetX = ( -nW ) >> 1 offsetY = ( ( -nH ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nH ) >> 3 : -( ( distanceIdx * nH ) >> 3 ) ) - Otherwise (i.e., shiftHor has the value 1) offsetX = ( ( -nW ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nW ) >> 3 : -( ( distanceIdx * nW ) >> 3 ) offsetY = ( - nH ) >> 1

[0236] The weights of the samples in the merged region, wValueMatrix[x][y] (x = -nTmW..nCbW - 1, y = -nTmH..nCbH - 1), are derived based on the following method (note that in this example, the coordinates of the upper left corner of the current block are set to (0,0)). - The variables xL and yL are derived based on the following method: xL = ( cIdx = = 0 ) ? x : x * SubWidthC yL = ( cIdx = = 0 ) ? y : y * SubHeightC

[0237] The disLut is determined according to Table 3 above.

[0238] The first parameter weightIdx is derived based on the following method. weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ]

[0239] After determining the first parameter weightIdx according to the above method, the weight of the sample (x, y) in the merged region is determined according to weightIdx.

[0240] In this application, the manner of determining the weights of the samples in the merged region according to the first parameters of the samples in the merged region in the above step B212 includes, but is not limited to:

[0241] Method 1: determining a second parameter of the samples in the merged region according to the first parameter of the samples in the merged region, and determining weights of the samples in the merged region according to the second parameter of the samples in the merged region.

[0242] For example, the weights of the samples in the merged region are determined based on the following formula: weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ] weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx wValueMatrix [x][y] = Clip3( 0, 8, ( weightIdxL + 4 ) >> 3 )

[0243] wValueMatrix[x][y] is the weight of sample (x,y) in the merged region. weightIdxL is the second parameter of sample (x,y) in the merged region. wTemplateValue[x][y] is the weight of sample (x,y) in the merged region.

[0244] Manner 2: Determine the weights of the samples in the merged region according to a first parameter of the samples in the merged region, a first preset value, and a second preset value.

[0245] Illustratively, the weight of the samples in the merged region is a first preset value or a second preset value.

[0246] In order to reduce the computational complexity of the merging region weight, in the method 2, the weight of the sample in the merging region is limited to a first preset value or a second preset value, that is, the weight of the sample in the merging region is set to either the first preset value or the second preset value, thereby reducing the computational complexity of the merging region weight.

[0247] The present application does not limit the specific values ​​of the first preset value and the second preset value.

[0248] Optionally, the first preset value is one.

[0249] Optionally, the second preset value is zero.

[0250] As an example, the weights of the samples in the merged region can be determined based on the following formula: wTemplateValue[x][y] = (partFlip ? weightIdx: - weightIdx) > 0 ? 1 : 0

[0251] wValueMatrix[x][y] is the weight of sample (x, y) in the merged region, where 1 in the above “1:0” is the first preset value and 0 is the second preset value.

[0252] In the above method 2, the weight of each sample in the merged region consisting of the current block and the template is determined according to the weight derivation mode, and the weight corresponding to the template in the merged region is determined as the template weight according to the size of the template. In addition, in the method 2, the weight of the predicted value is determined without needing to perform the subsequent step of determining the weight of the predicted value, thereby reducing the prediction step and improving the prediction efficiency.

[0253] The above describes in detail the process of determining the template weight based on the weight derivation mode. The weight derivation mode may be any one of the weight derivation modes according to the embodiments of the present application, for example, the weight derivation mode of the current block or the i-th candidate prediction mode among N candidate prediction modes.

[0254] S103: Determine a predicted value of the current block according to a weight derivation mode of the current block.

[0255] In this application, the manner in which the decoding side determines the predicted value of the current block according to the weight derivation mode of the current block in S103 includes, but is not limited to, the following:

[0256] Method 1: The decoding side determines the predicted value of the current block according to the weight derivation mode of the current block and K initial prediction modes. Specifically, the decoding side determines the weight of the predicted value according to the weight derivation mode of the current block, then predicts the current block according to the K initial prediction modes to obtain K predicted values, and then uses the weight of the predicted value to perform a weighted average calculation on the K predicted values ​​to obtain the predicted value of the current block.

[0257] It should be noted that the weight of the prediction value derived based on the weight derivation mode of the current block can be understood as a weight corresponding to each sample in the current block, or a weight matrix corresponding to the current block. During determining the prediction value of the current block based on the weight of the prediction value, K prediction values ​​corresponding to each sample in the current block can be determined, and the prediction value corresponding to each sample can be determined based on the K prediction values ​​corresponding to each sample and the weight of the prediction value. The prediction value corresponding to each sample in the current block constitutes the prediction value of the current block. Optionally, determining the prediction value of the current block based on the weight of the prediction value can be performed based on a block, for example, determining K prediction values ​​of the current block, and weighting the K prediction values ​​of the current block according to the weight matrix of the current block to obtain the prediction value of the current block.

[0258] Method 2: The decoding side can improve the K initial prediction modes by using the weight derivation mode of the current block. In this case, the S103 step includes the following steps S103-A and S103-B. S103-A: Determine K adjusted prediction modes based on the weight derivation mode of the current block. S103-B: Determine a prediction value of the current block according to the K adjusted prediction modes and the weight derivation mode of the current block.

[0259] In an embodiment of the present application, the GPM can be considered to include (K+1) elements, i.e., one weight derivation mode and K prediction modes. One weight derivation mode and K prediction modes together generate one prediction block, which is used for the current block. Therefore, they are related to each other. For example, the current block contains the edges of two objects that are moving relatively, which is an ideal scenario for inter GPM. In theory, this "split" should occur at the edge of the object, but in practice, the "split" possibilities are limited and cannot cover all edges. An approximate "split" may be selected. There may be multiple approximate "splits", and which "split" is selected depends on which "split" in combination with the two prediction modes produces the best results. Similarly, which prediction mode is selected depends on which combination produces the best results. This is because even for parts that use the prediction mode, it is difficult to perfectly match this part with the current block in the case of natural video. The final selection may be the one with the highest coding efficiency. Another scenario in which GPM is often used is a scenario in which the current block contains a part with relative motion within an object. For example, in some cases where there is twisting or deformation due to arm swinging, the "splitting" becomes more ambiguous and may ultimately depend on which combination produces the best result. Another scenario is intra prediction, where some parts of a natural image have complex textures, some parts have gradual changes from one texture to another, and some parts cannot be simply represented in one direction. Intra GPM can provide more complex prediction blocks, and intra-coded blocks usually have larger residuals compared to inter-coded blocks with the same quantization. The choice of which prediction mode ultimately depends on which combination produces the best result.

[0260] As can be seen from the above, since the (K+1) elements affect each other, the prediction mode can be determined using the weight derivation mode, and the weight derivation mode can be determined using the prediction mode. In addition, some mode information can be refined. Refinement means that more related information is collected based on the initial mode information, and more methods such as calculation are used to obtain better mode information. Refine is generally an operation on the decoding side. Based on the template, the motion information can also be refined. That is, based on the initial motion information, a search is performed within a small range on the decoding side, and the motion information is optimized according to the degree of matching between the searched motion information and the template. The degree of matching between the motion information and the template can be understood as the degree of matching between the predicted value of the template based on the motion information and the reconstructed value of the template.

[0261] In GPM, the weight derivation mode can be determined based on the prediction mode first, and then the prediction mode can be improved based on the weight derivation mode. Or, the prediction mode can be determined based on the weight derivation mode first, and then the weight derivation mode can be improved based on the prediction mode. Of course, if the complexity allows, both can be performed alternately. The template can be the link to perform these methods, and the template weight can be the link to perform these methods. That is, the cost on the template is calculated based on the predicted value of the template in the prediction mode and the weight of the sub-template or template corresponding to the weight derivation mode. The cost on the template is used to derive or determine the prediction mode or weight derivation mode.

[0262] That is, in the embodiment of the present application, a weight derivation mode of a current block is determined based on K initial prediction modes, and then the weight derivation mode of the current block is used to improve the K initial prediction modes to obtain K adjusted prediction modes, and then a predicted value of the current block is determined based on the weight derivation mode of the current block and the K adjusted prediction modes.

[0263] Hereinafter, according to a specific embodiment, the process of determining a weight derivation mode of a current block based on the K initial prediction modes in S102, and the process of determining K adjusted prediction modes based on the weight derivation mode of a current block in S103-A will be described in detail.

[0264] In some embodiments, the above S102 includes the following steps S102-A to S102-C. S102-A: Determine N candidate weight derivation modes, where N is a positive integer greater than 1. S102-B: For an i-th candidate weight derivation mode among the N candidate weight derivation modes, determine a first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and the K initial prediction modes, where i is a positive integer from 1 to N. S102-C: Determine a weight derivation mode for the current block based on a first cost corresponding to each of the N candidate weight derivation modes.

[0265] In this embodiment, the decoding side first determines N candidate weight derivation modes, which constitute a list of weight derivation modes, and the decoding side calculates a first cost corresponding to each of the N candidate weight derivation modes, and then determines a weight derivation mode of the current block from the N candidate weight derivation modes based on the first cost.

[0266] For example, among the N candidate weight derivation modes, the candidate weight derivation mode with the smallest first cost is determined as the weight derivation mode of the current block.

[0267] As another example, the N candidate weight derivation modes are sorted based on a first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes. The bitstream is decoded to obtain a weight derivation mode index. A candidate weight derivation mode corresponding to the weight derivation mode index in the list of candidate weight derivation modes is determined as a weight derivation mode for the current block.

[0268] Optionally, the first cost may be a cost such as SAD, SATD or SEE.

[0269] The process of the decoding side determining the first cost corresponding to each of the N candidate weight derivation modes is the same. For ease of explanation, the process of determining the first cost corresponding to the i-th candidate weight derivation mode among the N candidate weight derivation modes is taken as an example.

[0270] According to the i-th candidate weight derivation mode and the K initial prediction modes in the above S102-B, the manner of determining the first cost corresponding to the i-th candidate weight derivation mode includes, but is not limited to, the following:

[0271] Scheme 1: Predict a template of a current block by using the i-th candidate weight derivation mode and K initial prediction modes to obtain a first predicted value of the template, and determine a first cost corresponding to the i-th weight derivation mode according to the first predicted value of the template and the reconstructed value of the template.

[0272] Specifically, a template weight is determined based on the i-th candidate weight derivation mode, and the template is predicted using K initial prediction modes to obtain K predicted values ​​of the template. The template weight is used to weight the K predicted values ​​corresponding to the template to obtain a predicted value of the template. For convenience of explanation, the predicted value of the template is represented as a first predicted value of the template. Then, a prediction cost for the template corresponding to the i-th candidate weight derivation mode is determined based on the first predicted value of the template and the reconstructed value of the template, and the predicted cost is represented as a first cost corresponding to the i-th weight derivation mode.

[0273] It should be noted that the specific process of determining the template weight based on the i-th candidate weight derivation mode can be referred to in the above specific content of determining the template weight based on the weight derivation mode, so it will not be described in detail here.

[0274] The above method can determine the first cost corresponding to each of the N candidate weight derivation modes, and then determine the candidate weight derivation mode with the smallest first cost among the N candidate weight derivation modes as the weight derivation mode of the current block. Or, sort the N candidate weight derivation modes according to the first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes. In the list of candidate weight derivation modes, determine the candidate weight derivation mode corresponding to the weight derivation mode index as the weight derivation mode of the current block.

[0275] According to the above Scheme 1, the scheme of determining K adjusted prediction modes according to the weight derivation mode of the current block in S103-A includes the following steps:

[0276] S103-A1: Improve K initial prediction modes based on a weight derivation mode of a current block to obtain K adjusted prediction modes.

[0277] Examples of S103-A1 include, but are not limited to, the following examples:

[0278] Example 1: determining at least one prediction mode for a weight derivation mode of a current block, the at least one prediction mode may be a prediction mode whose prediction angle is parallel or nearly parallel to a boundary surface of the weight derivation mode of the current block, or a prediction mode whose prediction angle is perpendicular or nearly perpendicular to a boundary surface of the weight derivation mode of the current block. At least one initial prediction mode of the K initial prediction modes is replaced with a prediction mode of the at least one prediction mode of the weight derivation mode of the current block, forming a new combination of prediction modes. Each of the new combinations of prediction modes includes K prediction modes. For each of the new combinations of prediction modes, a template is predicted using the K prediction modes included in the new combination of prediction modes and the weight derivation mode of the current block to obtain a predicted value of the template. A cost corresponding to the new combination of prediction modes is determined based on the predicted value of the template and the reconstructed value of the template, thereby obtaining a cost corresponding to each of the new combinations of prediction modes formed. Similarly, a current block is predicted using the weight derivation mode of the current block and the K initial prediction modes to determine a cost corresponding to the K initial prediction modes. In this way, the decoding side can obtain K adjusted prediction modes according to the cost corresponding to the new combination of prediction modes and the cost corresponding to the K initial prediction modes. For example, if the cost corresponding to the new combination 1 of prediction modes is the smallest, the K prediction modes included in the new combination 1 of prediction modes are determined as the K adjusted prediction modes. The new combination 1 of prediction modes may include at least one initial prediction mode, or may not include the initial prediction mode.

[0279] Example 2: Through the following steps S103-A11 to S103-A14, K initial prediction modes are improved to obtain K adjusted prediction modes.

[0280] S103-A11: Determine a first template weight based on a weight derivation mode of the current block.

[0281] For a specific process of determining the first template weight according to the weight derivation mode of the current block, reference may be made to the specific description of the above-mentioned embodiment of the template weight derivation, and the detailed description will not be given here, in which the weight derivation mode in the above-mentioned embodiment of the template weight derivation is the weight derivation mode of the current block, and the corresponding template weight is the above-mentioned first template weight.

[0282] S103-A12: For a j-th initial prediction mode among the K initial prediction modes, determine M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer greater than 1, and j is a positive integer from 1 to K.

[0283] In addition, there is no order in which S103-A12 and S103-A11 are executed, that is, S103-A12 may be executed after S103-A11, may be executed before S103-A11, or may be executed simultaneously with S103-A11, and the embodiment of the present application is not limited thereto.

[0284] In this example, the decoding side improves each of the K initial prediction modes using the weight derivation mode of the current block to obtain an adjusted prediction mode corresponding to each of the K initial prediction modes, i.e., obtains K adjusted prediction modes.

[0285] The process of determining the adjusted prediction mode corresponding to each of the K initial prediction modes is the same. For ease of description, the embodiment of the present application takes as an example determining the adjusted prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes.

[0286] In an embodiment of the present application, for a j-th initial prediction mode, the decoding side first determines M first candidate prediction modes corresponding to the j-th initial prediction mode.

[0287] The manner in which the decoding side determines the M first candidate prediction modes corresponding to the j-th initial prediction mode includes the following two cases.

[0288] Case 1: When the jth initial prediction mode is an intra prediction mode, M intra prediction modes similar to the jth initial prediction mode are determined as M first candidate prediction modes. For example, when the jth initial prediction mode is an angular prediction mode, M angular prediction modes similar (or similar) to the angle of the jth initial prediction mode are determined as M first candidate prediction modes corresponding to the jth initial prediction mode.

[0289] Case 2: When the j-th initial prediction mode is an inter prediction mode, a search is performed based on the j-th initial prediction mode to obtain M pieces of first motion information. Based on these M pieces of first motion information, M pieces of first candidate prediction modes corresponding to the j-th prediction mode are determined. For example, for the j-th initial prediction mode, a motion vector is (xInit, yInit), and a search range such as a rectangular area is set from xInit-sR to xInit+sR in the horizontal direction and from yInit-sR to yInit+sR in the vertical direction, where sR may be 2, 4, 8, etc. Each motion vector in the rectangular area is combined with other information of the j-th initial prediction mode, such as a reference image index and a prediction list flag, to determine the motion information, and further determine the prediction mode. As a result, M pieces of first candidate prediction modes corresponding to the j-th initial prediction mode can be obtained.

[0290] In some embodiments, one of the ways of performing a search based on the j-th initial prediction mode to obtain M pieces of first motion information is as follows: A search is performed within a preset search range based on a motion vector corresponding to the j-th initial prediction mode to obtain M offsets. Each offset includes an offset in a first direction (e.g., x direction) and an offset in a second direction (e.g., y direction). Further, M pieces of first motion information are obtained based on these M offsets. For example, for each of the M offsets, the motion vector corresponding to the j-th initial prediction mode is offset based on the offset to obtain the first motion information.

[0291] S103-A13: Determine second costs corresponding to predictions to the template by each of the M first candidate prediction modes based on the first template weights.

[0292] According to the method of the above case 1 or case 2, after determining M first candidate prediction modes corresponding to the j-th initial prediction mode, determine costs corresponding to the M first candidate prediction modes, and then determine an adjusted prediction mode corresponding to the j-th initial prediction mode based on the costs. For convenience of description, the cost corresponding to the first candidate prediction mode is denoted as the second cost.

[0293] Specifically, for each of the M first candidate prediction modes, predict a template of a current block using the first candidate prediction mode to obtain a predicted value, and then determine a second cost for the template corresponding to the first candidate prediction mode according to the first template weight and the predicted value.

[0294] Hereinafter, an example will be described in which the second cost for a template corresponding to one of the M first candidate prediction modes is calculated.

[0295] For convenience of explanation, the i-th sample in the template is taken as an example. The i-th sample can be understood as any sample in the template. That is, the process of determining the second cost of each sample in the template is the same, and it is sufficient to refer to the i-th sample. Specifically, the template is predicted using a first candidate prediction mode to obtain a predicted value of the template. The predicted value corresponding to the i-th sample in the predicted value of the template is written as the i-th predicted value, and the reconstructed value corresponding to the i-th sample in the reconstructed value of the template is written as the i-th reconstructed value. The second cost of the i-th sample corresponding to the first candidate prediction mode is determined based on the i-th predicted value, the i-th reconstructed value, and the weight of the i-th sample in the first template weight. Based on the above method, the second cost of each sample or multiple samples in the template corresponding to the first candidate prediction mode is determined. Then, the second cost for the template corresponding to the first candidate prediction mode is determined based on the second cost of each sample or multiple samples in the template. For example, the sum of the second costs of the samples in the template corresponding to the first candidate prediction mode is determined as the second cost for the template corresponding to the first candidate prediction mode.

[0296] Illustratively, taking the SAD cost as an example, the second cost of the i-th sample (x, y) in the template corresponding to the first candidate prediction mode can be determined based on the following Equation 1:

number

[0297] Exemplarily, a second cost for the template, corresponding to the first candidate prediction mode, may be determined based on the following Equation 2:

number

[0298] abs(predTemplateSamplesCandA[x][y]-recTemplateSamples[x][y]) is the absolute value of the difference between the predicted value predTemplateSamplesCandA and the reconstructed value recTemplateSamples of sample (x,y) in the template. The absolute value of the difference is called the second cost corresponding to sample (x,y). This value is multiplied by the weight value wTemplateValue[x][y] of the sample in the first template weight to obtain tempValueA[x][y]. tempValueA[x][y] can be considered as the second cost of this sample (x,y) corresponding to the first candidate prediction mode. The total second cost of the template, costCandA, corresponding to the first candidate prediction mode is the sum of the second costs of each sample in the template.

[0299] However, the above is taken as an example of determining the second cost for the template corresponding to the first candidate prediction mode according to SAD.Optionally, the second cost for the template corresponding to the first candidate prediction mode can also be determined according to cost calculation methods such as SATD, MSE, etc.

[0300] Through the above method, the second costs corresponding to each of the M first candidate prediction modes can be determined, and then the following S103-A14 are performed.

[0301] S103-A14: Determine an adjusted prediction mode corresponding to the j-th initial prediction mode based on the second costs corresponding to each of the M first candidate prediction modes.

[0302] For example, among the M first candidate prediction modes, the first candidate prediction mode having the smallest second cost is determined as the adjusted prediction mode corresponding to the j-th initial prediction mode.

[0303] In the above embodiment, an example is described of determining an adjusted prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes. However, by referring to the above description, it is possible to determine an adjusted prediction mode corresponding to each of the K initial prediction modes and obtain K adjusted prediction modes.

[0304] In the method 1, the decoding side first determines a weight derivation mode of the current block based on the K initial prediction modes, then improves the K initial prediction modes based on the weight derivation mode of the current block, for example, by improving according to the method of the above example 1 or example 2, to obtain K adjusted prediction modes, and then predicts the current block using the weight derivation mode of the current block and the K adjusted prediction modes to obtain a predicted value of the current block.

[0305] In an embodiment of the present application, in addition to using the above method 1 to determine the weight derivation mode of the current block and K adjusted prediction modes, the decoding side can also determine the weight derivation mode of the current block and K adjusted prediction modes by the following method 2.

[0306] Method 2: Improving prediction mode and determining weight derivation mode of current block are combined and performed simultaneously. In this case, determining a first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and K initial prediction modes in the above S102-B includes the following steps S102-B11 to S102-B12.

[0307] S102-B11: Improve K initial prediction modes based on the i-th candidate weight derivation mode to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode.

[0308] In the second method, for an i-th candidate weight derivation mode among N candidate prediction modes, first, improve K initial prediction modes using the i-th candidate weight derivation mode to obtain K improved prediction modes corresponding to the i-th candidate weight derivation mode. For convenience of explanation, the K improved prediction modes corresponding to the i-th candidate weight derivation mode are referred to as K first improved prediction modes. Then, a first cost corresponding to the i-th candidate weight derivation mode is determined based on the i-th candidate weight derivation mode and the K first improved prediction modes corresponding to the i-th candidate weight derivation mode. Thus, a more accurate weight derivation mode of the current block and K adjusted prediction modes can be obtained.

[0309] Examples of S102-B11 include, but are not limited to, the following examples:

[0310] Example 1: determining at least one prediction mode for an i-th candidate weight derivation mode, the at least one prediction mode may be a prediction mode with a prediction angle parallel or nearly parallel to a boundary surface of the i-th candidate weight derivation mode, or a prediction mode with a prediction angle perpendicular or nearly perpendicular to a boundary surface of the i-th candidate weight derivation mode. At least one initial prediction mode among the K initial prediction modes is replaced with a prediction mode among the at least one prediction mode for the i-th candidate weight derivation mode, forming a new combination of prediction modes. Each of the new combinations of prediction modes includes K prediction modes. For each of the new combinations of prediction modes, a template is predicted using the K prediction modes included in the new combination of prediction modes and the i-th candidate weight derivation mode to obtain a predicted value of the template. A cost corresponding to the new combination of prediction modes is determined based on the predicted value of the template and the reconstructed value of the template, thereby obtaining a cost corresponding to each of the new combinations of prediction modes formed. Similarly, a current block is predicted using the i-th candidate weight derivation mode and the K initial prediction modes to determine a cost corresponding to the K initial prediction modes. In this way, the decoding side can obtain K first improved prediction modes based on the cost corresponding to the new combination of prediction modes and the costs corresponding to the K initial prediction modes. For example, if the cost corresponding to the new combination 1 of prediction modes is the smallest, the K prediction modes included in the new combination 1 of prediction modes are determined as the K first improved prediction modes. The new combination 1 of prediction modes may include at least one initial prediction mode, or may not include any initial prediction mode.

[0311] Example 2: Through the following steps S102-B11-1 to S102-B11-4, K initial prediction modes are improved to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode.

[0312] S102-B11-1: Determine a second template weight based on the i-th candidate weight derivation mode.

[0313] The specific process of determining the second template weight according to the i-th candidate weight derivation mode can refer to the specific description of the above template weight derivation embodiment, and will not be detailed here. In this case, the weight derivation mode is the i-th candidate weight derivation mode, and the corresponding template weight is the second template weight.

[0314] S102-B11-2: For a j-th initial prediction mode among the K initial prediction modes, determine M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer and j is a positive integer from 1 to K.

[0315] It should be noted that there is no order in which S102-B11-2 and S102-B11-1 are executed. That is, S102-B11-2 may be executed after S102-B11-1, before S102-B11-1, or simultaneously with S102-B11-1, and the embodiment of the present application is not limited thereto.

[0316] In this example, the decoding side improves each of the K initial prediction modes using the i-th candidate weight derivation mode to obtain first improved prediction modes corresponding to each of the K initial prediction modes in the i-th candidate weight derivation mode, i.e., obtain K first improved prediction modes.

[0317] The process of determining the first improved prediction mode corresponding to each of the K initial prediction modes is the same. For ease of explanation, the embodiment of the present application takes as an example determining the first improved prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes.

[0318] In an embodiment of the present application, for a j-th initial prediction mode, the decoding side first determines M first candidate prediction modes corresponding to the j-th initial prediction mode.

[0319] The manner in which the decoding side determines the M first candidate prediction modes corresponding to the j-th initial prediction mode includes the following two cases.

[0320] Case 1: if the j-th initial prediction mode is an intra prediction mode, determine M intra prediction modes similar to the j-th initial prediction mode as M first candidate prediction modes.

[0321] Case 2: if the jth initial prediction mode is an inter prediction mode, perform a search based on the jth initial prediction mode to obtain M pieces of first motion information, and determine M first candidate prediction modes corresponding to the jth prediction mode based on the M pieces of first motion information.

[0322] In some embodiments, one of the ways of performing a search based on the j-th initial prediction mode to obtain M pieces of first motion information is as follows: A search is performed within a preset search range based on a motion vector corresponding to the j-th initial prediction mode to obtain M offsets. Each offset includes an offset in a first direction (e.g., x direction) and an offset in a second direction (e.g., y direction). Further, M pieces of first motion information are obtained based on these M offsets. For example, for each of the M offsets, the motion vector corresponding to the j-th initial prediction mode is offset based on the offset to obtain the first motion information.

[0323] S102-B11-3: Determine a third cost corresponding to prediction to the template by each of the M first candidate prediction modes based on the second template weight.

[0324] According to the method of the above case 1 or case 2, after determining M first candidate prediction modes corresponding to the j-th initial prediction mode, determine costs corresponding to the M first candidate prediction modes, and then determine a first improved prediction mode corresponding to the j-th initial prediction mode based on the costs. For convenience of explanation, the cost corresponding to the first candidate prediction mode in this case is denoted as a third cost.

[0325] Specifically, for each of the M first candidate prediction modes, predict a template of a current block using the first candidate prediction mode to obtain a predicted value, and then determine a third cost for the template corresponding to the first candidate prediction mode according to the second template weight and the predicted value.

[0326] Hereinafter, an example will be described in which the third cost for a template corresponding to one of the M first candidate prediction modes is calculated.

[0327] For convenience of explanation, the i-th sample in the template is taken as an example. The i-th sample can be understood as any sample in the template. That is, the process of determining the third cost of each sample in the template is the same, and it is sufficient to refer to the i-th sample. Specifically, the template is predicted using a first candidate prediction mode to obtain a predicted value of the template. The predicted value corresponding to the i-th sample in the predicted value of the template is written as the i-th predicted value, and the reconstructed value corresponding to the i-th sample in the reconstructed value of the template is written as the i-th reconstructed value. The third cost of the i-th sample corresponding to the first candidate prediction mode is determined based on the i-th predicted value, the i-th reconstructed value, and the weight of the i-th sample in the second template weight. Based on the above method, the third cost of each sample or multiple samples in the template corresponding to the first candidate prediction mode is determined. Then, the third cost for the template corresponding to the first candidate prediction mode is determined based on the third cost of each sample or multiple samples in the template. For example, the sum of the third costs of the samples in the template corresponding to the first candidate prediction mode is determined as the third cost for the template corresponding to the first candidate prediction mode.

[0328] For example, a third cost for the template corresponding to the first candidate prediction mode can be determined according to the above Equation 1 and Equation 2. Specifically, wTemplateValue[x][y] in the above Equation 1 is replaced with the corresponding weight value of sample (x, y) in the second template weight.

[0329] Through the above method, the third cost corresponding to each of the M first candidate prediction modes can be determined, and then the following S102-B11-4 is performed.

[0330] S102-B11-4: Determine an improved prediction mode corresponding to the j-th initial prediction mode based on the third costs corresponding to each of the M first candidate prediction modes.

[0331] For example, among the M first candidate prediction modes, the first candidate prediction mode having the smallest third cost is determined as the first improved prediction mode corresponding to the j-th initial prediction mode.

[0332] In the above embodiment, an example is described in which a first improved prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes is determined. However, by referring to the above description, it is possible to determine a first improved prediction mode corresponding to each of the K initial prediction modes and obtain K first improved prediction modes in the i-th candidate weight derivation mode.

[0333] S102-B12: Determine a first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and the K first refinement prediction modes.

[0334] Exemplarily, a template of a current block is predicted using the i-th candidate weight derivation mode and K first improved prediction modes to obtain a second predicted value of the template. A first cost corresponding to the i-th weight derivation mode is determined based on the second predicted value of the template and the reconstructed value of the template. For example, a second template weight is determined based on the i-th candidate weight derivation mode, and a template is predicted using K first improved prediction modes to obtain K predicted values. The second template weight is used to weight the K predicted values ​​to obtain a second predicted value of the template. A first cost corresponding to the i-th weight derivation mode is determined based on the second predicted value of the template and the reconstructed value of the template.

[0335] The above is taken as an example of determining the first cost corresponding to the i-th candidate weight derivation mode. For each of the N candidate weight derivation modes, the first cost corresponding to each of the N candidate weight derivation modes can be determined by using the method of determining the first cost corresponding to the i-th candidate weight derivation mode, and then the weight derivation mode of the current block is determined according to the first costs corresponding to each of the N candidate weight derivation modes.

[0336] For example, among the N candidate weight derivation modes, the candidate weight derivation mode with the smallest first cost is determined as the weight derivation mode of the current block.

[0337] As another example, the N candidate weight derivation modes are sorted based on a first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes. The bitstream is decoded to obtain a weight derivation mode index. A candidate weight derivation mode corresponding to the weight derivation mode index in the list of candidate weight derivation modes is determined as a weight derivation mode for the current block.

[0338] In the manner 2, K first improved prediction modes corresponding to the N candidate weight derivation modes are determined. Based on this, determining K adjusted prediction modes based on the weight derivation mode of the current block in the above S103-A includes the following steps:

[0339] S103-A2: Determine the K first refinement prediction modes corresponding to the weight derivation mode of the current block as the K adjusted prediction modes.

[0340] In the method 2, the prediction mode improvement and the determination of the weight derivation mode of the current block are combined, that is, while determining the first cost corresponding to the N candidate weight derivation modes, K first improved prediction modes corresponding to each candidate weight derivation mode are determined. Therefore, after determining the weight derivation mode of the current block based on the first cost, the K first improved prediction modes corresponding to the weight derivation mode of the current block can be determined as K adjusted prediction modes. Then, the current block can be predicted using the weight derivation mode of the current block and these K adjusted prediction modes to obtain a predicted value of the current block.

[0341] In the second method, for each of the N candidate weight derivation modes, K first improved prediction modes corresponding to each candidate weight derivation mode are determined. Then, a template is predicted according to each candidate weight derivation mode and the corresponding K first improved prediction modes, and a first cost is determined. Based on the first cost, the weight derivation mode of the current block can be more accurately determined from the N candidate weight derivation modes.

[0342] In an embodiment of the present application, in addition to using the above-mentioned method 1 or method 2 to determine the weight derivation mode of the current block and K adjusted prediction modes, the decoding side can also determine the weight derivation mode of the current block and K adjusted prediction modes by the following method 3.

[0343] Method 3: In order to reduce the computational complexity, during the initial prediction mode improvement, instead of using each candidate weight derivation mode to improve the initial prediction mode, use simple P sub-templates, for example, use the full template, left template, and upper template of the current block to improve the initial prediction mode.Therefore, the computational complexity of improving the initial prediction mode is reduced.In this case, determining the first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and K initial prediction modes in S102-B includes the following steps S102-B21 to S102-B23.

[0344] S102-B21: For a j-th initial prediction mode among the K initial prediction modes, determine a second refinement prediction mode for the P sub-templates corresponding to the j-th initial prediction mode, where P is a positive integer and j is a positive integer from 1 to K.

[0345] In this method 3, the decoding side improves K initial prediction modes using P sub-templates to obtain second improved prediction modes for P sub-templates corresponding to each of the K initial prediction modes. For each of the N candidate weight derivation modes, obtain K second improved prediction modes corresponding to the candidate weight derivation modes from the second improved prediction modes for P sub-templates corresponding to each of the K initial prediction modes based on the correspondence between the candidate weight derivation modes and the P sub-templates. Next, predict the template of the current block based on the candidate weight derivation modes and the corresponding K second improved prediction modes to determine the first cost corresponding to the candidate weight derivation modes. Finally, determine the weight derivation mode of the current block based on the first cost corresponding to each of the N candidate weight derivation modes. In method 3, the candidate prediction modes are not used alone to improve the K initial prediction modes, and the complexity of improving the initial prediction modes is reduced.

[0346] Exemplarily, the P sub-templates include at least one of a left template TM_L, an upper template TM_A, and a full template TM_AL of the current block, where the full template TM_AL includes a left template TM_L and an upper template TM_A.

[0347] In the embodiment of the present application, the process of the decoding side determining the second improved prediction mode for P sub-templates corresponding to each of the K initial prediction modes is the same. For the sake of simplicity, the embodiment of the present application takes as an example the determination of the second improved prediction mode for P sub-templates corresponding to the j-th initial prediction mode of the K initial prediction modes.

[0348] In one example, assuming that the P sub-templates include a left template TM_L, an upper template TM_A and a full template TM_AL of a current block, determining a second improved prediction mode for the P sub-templates corresponding to the jth initial prediction mode includes: determining a second improved prediction mode for the left template TM_L corresponding to the jth initial prediction mode, determining a second improved prediction mode for the upper template TM_A corresponding to the jth initial prediction mode, and determining a second improved prediction mode for the full template TM_AL corresponding to the jth initial prediction mode. That is, in this step, the decoding side determines a second improved prediction mode for each of the P sub-templates corresponding to the jth initial prediction mode.

[0349] The embodiments of the present application do not limit the specific format of determining the second refinement prediction mode for the P sub-templates corresponding to the j-th initial prediction mode.

[0350] In some embodiments, S102-B21 includes the following steps S102-B21-1 to S102-B21-3.

[0351] S102-B21-1: Determine M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer greater than 1.

[0352] The manner in which the decoding side determines the M first candidate prediction modes corresponding to the j-th initial prediction mode includes the following two cases.

[0353] Case 1: if the j-th initial prediction mode is an intra prediction mode, determine M intra prediction modes similar to the j-th initial prediction mode as M first candidate prediction modes.

[0354] Case 2: if the jth initial prediction mode is an inter prediction mode, perform a search based on the jth initial prediction mode to obtain M pieces of first motion information, and determine M first candidate prediction modes corresponding to the jth prediction mode based on the M pieces of first motion information.

[0355] In some embodiments, performing a search based on a j-th initial prediction mode to obtain M pieces of first motion information includes performing a search based on an i-th initial prediction mode to obtain M offsets, and obtaining the M pieces of first motion information based on the M offsets.

[0356] S102-B21-2: For the p-th sub-template of the P sub-templates, determine a fourth cost corresponding to prediction to the p-th sub-template by each of the M first candidate prediction modes, where p is a positive integer from 1 to P.

[0357] The decoding side determines M first candidate prediction modes corresponding to the j-th initial prediction mode according to the above method of Case 1 or Case 2, and then determines costs for each of the P sub-templates corresponding to the M first candidate prediction modes, and then determines a second improved prediction mode for each of the P sub-templates corresponding to the j-th initial prediction mode according to the costs. For convenience of explanation, the cost corresponding to the first candidate prediction mode in this case is denoted as a fourth cost.

[0358] Specifically, take the p-th sub-template of the P sub-templates as an example, for any first candidate prediction mode of the M first candidate prediction modes, predict the p-th sub-template using the first candidate prediction mode to obtain a predicted value of the p-th sub-template. Then, determine a fourth cost for the p-th sub-template corresponding to the first candidate prediction mode based on the predicted value of the p-th sub-template and the reconstructed value of the p-th sub-template. With reference to this method, a fourth cost for each of the P sub-templates corresponding to each of the M first candidate prediction modes can be determined.

[0359] As an example, assume that the j-th initial prediction mode corresponds to three first candidate prediction modes, denoted as a first candidate prediction mode 1, a first candidate prediction mode 2, and a first candidate prediction mode 3, and assume that the P sub-templates include a left template TM_L, an upper template TM_A, and an entire template TM_AL. First, for the left template TM_L, the first candidate prediction mode 1, the first candidate prediction mode 2, and the first candidate prediction mode 3 are used to predict the left template TM_L, respectively, to obtain a prediction value 1, a prediction value 2, and a prediction value 3, respectively. Then, based on the prediction value 1 and the reconstruction value of the left template TM_L, a fourth cost for the left template TM_L corresponding to the first candidate prediction mode 1 is determined. Based on the prediction value 2 and the reconstruction value of the left template TM_L, a fourth cost for the left template TM_L corresponding to the first candidate prediction mode 2 is determined. A fourth cost for the left template TM_L, corresponding to the first candidate prediction mode 3, is determined based on the prediction value 3 and the reconstruction value of the left template TM_L.

[0360] Similarly, the upper template TM_A is predicted using the first candidate prediction mode 1, the first candidate prediction mode 2, and the first candidate prediction mode 3, respectively, to obtain a prediction value 4, a prediction value 5, and a prediction value 6, respectively. Next, based on the prediction value 4 and the reconstruction value of the upper template TM_A, a fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 1 is determined. Based on the prediction value 5 and the reconstruction value of the upper template TM_A, a fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 2 is determined. Based on the prediction value 6 and the reconstruction value of the upper template TM_A, a fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 3 is determined.

[0361] Similarly, the first candidate prediction mode 1, the first candidate prediction mode 2, and the first candidate prediction mode 3 are used to predict the entire template TM_AL to obtain a prediction value 7, a prediction value 8, and a prediction value 9, respectively. Next, based on the prediction value 7 and the reconstruction value of the entire template TM_AL, a fourth cost for the entire template TM_AL corresponding to the first candidate prediction mode 1 is determined. Based on the prediction value 8 and the reconstruction value of the entire template TM_AL, a fourth cost for the entire template TM_AL corresponding to the first candidate prediction mode 2 is determined. Based on the prediction value 9 and the reconstruction value of the entire template TM_AL, a fourth cost for the entire template TM_AL corresponding to the first candidate prediction mode 3 is determined.

[0362] According to the above method, the fourth cost for each of the P sub-templates corresponding to each of the M first candidate prediction modes corresponding to the j-th initial prediction mode is determined. Next, execute S102-B21-3.

[0363] S102-B21-3: Determine a second refinement prediction mode for the p-th sub-template corresponding to the j-th initial prediction mode based on the fourth cost for the p-th sub-template corresponding to each of the M first candidate prediction modes.

[0364] For example, among the M first candidate prediction modes, the first candidate prediction mode having the smallest fourth cost for the pth sub-template is determined as the second improved prediction mode for the pth sub-template corresponding to the jth initial prediction mode.

[0365] Continuing with the above example, assume that the jth initial prediction mode corresponds to three first candidate prediction modes, denoted as the first candidate prediction mode 1, the first candidate prediction mode 2, and the first candidate prediction mode 3, and assume that the P sub-templates include a left template TM_L, an upper template TM_A, and an entire template TM_AL. Assuming that for the left template TM_L, the fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 1 is smaller than the fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 2, and the fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 2 is smaller than the fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 3, then the first candidate prediction mode 1 can be determined as the second improved prediction mode for the left template TM_L corresponding to the jth initial prediction mode.

[0366] Similarly, assuming that for the upper template TM_A, the fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 2 is less than the fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 1, and the fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 1 is less than the fourth cost for the upper template TM_A corresponding to the first candidate prediction mode 3, then the first candidate prediction mode 2 can be determined as the second improved prediction mode for the upper template TM_A corresponding to the jth initial prediction mode.

[0367] Similarly, assuming that for all templates TM_AL, the fourth cost for all templates TM_AL corresponding to the first candidate prediction mode 2 is less than the fourth cost for all templates TM_AL corresponding to the first candidate prediction mode 1, and the fourth cost for all templates TM_AL corresponding to the first candidate prediction mode 1 is less than the fourth cost for all templates TM_AL corresponding to the first candidate prediction mode 3, then the first candidate prediction mode 2 can be determined as the second improved prediction mode for all templates TM_AL corresponding to the jth initial prediction mode.

[0368] The above method allows determining a second refinement prediction mode for each of the P sub-templates, corresponding to each of the K initial prediction modes.

[0369] For example, assume that K=2, that is, the two initial prediction modes are respectively denoted as the first prediction mode and the second prediction mode, and assume that the P sub-templates are the left template TM_L, the top template TM_A, and the full template TM_AL. According to the above method, the second improved prediction modes for each of the left template TM_L, the top template TM_A, and the full template corresponding to each of the first prediction mode and the second prediction mode are determined, as shown in Table 6.

[0370] [Table 6]

[0371] S102-B22: Based on the i-th candidate weight derivation mode, determine K second improvement prediction modes corresponding to the i-th candidate weight derivation mode from the second improvement prediction modes for the P sub-templates corresponding to each of the K initial prediction modes.

[0372] Specifically, for an i-th candidate weight derivation mode among N candidate weight derivation modes, K second improved prediction modes corresponding to the i-th candidate weight derivation mode are determined from second improved prediction modes for P sub-templates corresponding to each of the K initial prediction modes based on the sub-template corresponding to the i-th candidate weight derivation mode.

[0373] For example, take K=2 as an example, the sub-templates corresponding to the i-th candidate weight derivation mode are the upper template TM_A and the full template TM_AL. With reference to Table 6 above, it can be determined that the second improved prediction mode for the upper template TM_A corresponding to the first prediction mode is the second improved prediction mode 12, and the second improved prediction mode for the full template TM_AL corresponding to the second prediction mode is the second improved prediction mode 23. Thus, it can be determined that the two second improved prediction modes corresponding to the i-th candidate weight derivation mode are the second improved prediction mode 12 and the second improved prediction mode 23, respectively.

[0374] In one possible embodiment, the decoding side determines a division angle corresponding to the i-th candidate weight derivation mode, and based on the correspondence between the division angle and the template corresponding to the prediction mode (specifically referring to Table 5 above), determines K second improvement prediction modes corresponding to the i-th candidate weight derivation mode from the second improvement prediction modes for P sub-templates corresponding to each of the K initial prediction modes.

[0375] Exemplarily, taking K=2 as an example, the angle index corresponding to the i-th candidate weight derivation mode is determined based on the above Table 2. For example, when the i-th candidate weight derivation mode is 27, the corresponding angle index is 12. In the correspondence relationship between the division angle and the template corresponding to the prediction mode shown in the above Table 5, the template corresponding to the first prediction mode and the template corresponding to the second prediction mode at the angle index are queried, and two second improved prediction modes corresponding to the i-th candidate weight derivation mode are determined from the above Table 6 based on the template corresponding to the first prediction mode and the template corresponding to the second prediction mode. For example, it is queried from Table 5 that the template corresponding to the first prediction mode at the angle index is the upper template TM_A, and the template corresponding to the second prediction mode is the full template TM_AL. Then, based on Table 6 above, second improved prediction mode 12 corresponding to the upper template TM_A in the second improved prediction mode corresponding to the first prediction mode, and second improved prediction mode 23 corresponding to all templates TM_AL in the second improved prediction mode corresponding to the second prediction mode are determined as two second improved prediction modes corresponding to the i-th candidate weight derivation mode.

[0376] According to the above method, K second improved prediction modes corresponding to the i-th candidate weight derivation mode among the N candidate weight derivation modes are determined, and then the following steps S102-B23 are performed.

[0377] S102-B23: Determine a first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and the K second refinement prediction modes corresponding to the i-th candidate weight derivation mode.

[0378] Specifically, the template of the current block is predicted using the i-th candidate weight derivation mode and the K second improved prediction modes to obtain a third predicted value of the template, and a first cost corresponding to the i-th weight derivation mode is determined based on the third predicted value of the template and the reconstructed value of the template. For example, a template weight is determined based on the i-th candidate weight derivation mode, and the template of the current block is predicted using the K second improved prediction modes respectively to obtain K predicted values. The determined template weight is weighted to obtain a predicted value of the template corresponding to the i-th candidate weight derivation mode. Furthermore, a first cost corresponding to the i-th candidate weight derivation mode is determined based on the predicted value of the template and the reconstructed value of the template.

[0379] The above is taken as an example of determining the first cost corresponding to the i-th candidate weight derivation mode. For each of the N candidate weight derivation modes, the first cost corresponding to each of the N candidate weight derivation modes can be determined by using the method of determining the first cost corresponding to the i-th weight derivation mode, and then the weight derivation mode of the current block is determined according to the first costs corresponding to each of the N candidate weight derivation modes.

[0380] For example, among the N candidate weight derivation modes, the candidate weight derivation mode with the smallest first cost is determined as the weight derivation mode of the current block.

[0381] As another example, the N candidate weight derivation modes are sorted based on a first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes. The bitstream is decoded to obtain a weight derivation mode index. A candidate weight derivation mode corresponding to the weight derivation mode index in the list of candidate weight derivation modes is determined as a weight derivation mode for the current block.

[0382] In this method 3, K initial prediction modes are improved using P sub-templates to obtain second improved prediction modes for each of the P sub-templates corresponding to each of the K initial prediction modes. In this way, when determining the first costs corresponding to each of the N candidate weight derivation modes in the subsequent process, instead of improving the K initial prediction modes according to different candidate weight derivation modes, the K second improved prediction modes corresponding to each candidate weight derivation mode are collectively queried from the second improved prediction modes for each of the P sub-templates corresponding to each of the determined K initial prediction modes. This reduces the complexity of improving the initial prediction mode, increases the speed of calculating the first costs corresponding to each candidate weight derivation mode, and can quickly determine the weight derivation mode of the current block from the N candidate weight derivation modes.

[0383] In the manner 3, K second improved prediction modes corresponding to the N candidate weight derivation modes are determined based on the P sub-templates. Based on this, the manner of determining K adjusted prediction modes based on the weight derivation mode of the current block in the above S103-A includes, but is not limited to:

[0384] Manner 1: Determine the K second refinement prediction modes corresponding to the weight derivation mode of the current block as the K adjusted prediction modes.

[0385] In the above-mentioned method 3, while determining the first costs corresponding to each of the N candidate weight derivation modes, K second improved prediction modes corresponding to each candidate weight derivation mode are determined. In this case, after determining the weight derivation mode of the current block based on the first costs, the K second improved prediction modes corresponding to the weight derivation mode of the current block can be directly determined as the K adjusted prediction modes.

[0386] Method 2: To further improve the accuracy of the K adjusted prediction modes, improve the K second improved prediction modes corresponding to the weight derivation mode of the current block to obtain the K adjusted prediction modes. In this case, the above S103-A includes the following steps:

[0387] S103-A3: Improve the K second refinement prediction modes based on the weight derivation mode of the current block to obtain K adjusted prediction modes.

[0388] Examples of S103-A3 implementations include, but are not limited to, the following examples:

[0389] Example 1: determining at least one prediction mode for a weight derivation mode of a current block, the at least one prediction mode may be a prediction mode with a prediction angle parallel or nearly parallel to a boundary surface of the weight derivation mode of the current block, or a prediction mode with a prediction angle perpendicular or nearly perpendicular to a boundary surface of the weight derivation mode of the current block. At least one second improved prediction mode of the K second improved prediction modes is replaced with a prediction mode of the at least one prediction mode of the weight derivation mode of the current block, forming a new combination of a plurality of prediction modes. Each new combination of prediction modes includes K prediction modes. For each new combination of prediction modes, a template is predicted using the K prediction modes included in the new combination of prediction modes and the weight derivation mode of the current block to obtain a predicted value of the template. A cost corresponding to the new combination of prediction modes is determined based on the predicted value of the template and the reconstruction value of the template, thereby obtaining a cost corresponding to each of the new combinations of the formed plurality of prediction modes. Similarly, a current block is predicted using the weight derivation mode of the current block and the K second improved prediction modes to determine a cost corresponding to the K second improved prediction modes. In this way, the decoding side can obtain K adjusted prediction modes according to the cost corresponding to the new combination of prediction modes and the costs corresponding to the K second improved prediction modes. For example, if the cost corresponding to the new combination 1 of prediction modes is the smallest, the K prediction modes included in the new combination 1 of prediction modes are determined as the K adjusted prediction modes. The new combination 1 of prediction modes may include at least one initial prediction mode, or may not include the initial prediction mode.

[0390] Example 2: Through the following steps S103-A31 to S103-A34, K second prediction modes are improved to obtain K adjusted prediction modes.

[0391] S103-A31: Determine a first template weight based on a weight derivation mode of the current block.

[0392] S103-A32: For a k-th second improved prediction mode among the K second improved prediction modes, determine P second candidate prediction modes corresponding to the k-th second improved prediction mode, where P is a positive integer greater than 1, and k is a positive integer from 1 to K.

[0393] In addition, there is no order in which S103-A32 and S103-A31 are executed, that is, S103-A32 may be executed after S103-A31, may be executed before S103-A31, or may be executed simultaneously with S103-A31, and the embodiment of the present application is not limited thereto.

[0394] In this example, the decoding side improves each of the K second improved prediction modes using the weight derivation mode of the current block to obtain adjusted prediction modes corresponding to each of the K second improved prediction modes, i.e., obtains K adjusted prediction modes.

[0395] The process of determining the adjusted prediction mode corresponding to each of the K second improved prediction modes is the same. For ease of description, the embodiment of the present application takes as an example determining the adjusted prediction mode corresponding to the k-th second improved prediction mode among the K second improved prediction modes.

[0396] In an embodiment of the present application, for a k-th second refinement prediction mode, the decoding side first determines P second candidate prediction modes corresponding to the k-th second refinement prediction mode.

[0397] The manner in which the decoding side determines the P second candidate prediction modes corresponding to the k-th second refinement prediction mode includes the following two cases.

[0398] Case 1: When the kth second improved prediction mode is an intra prediction mode, P intra prediction modes similar to the kth second improved prediction mode are determined as P second candidate prediction modes. For example, when the kth second improved prediction mode is an angle prediction mode, P angle prediction modes similar (or similar) to the angle of the kth second improved prediction mode are determined as P second candidate prediction modes corresponding to the kth second improved prediction mode.

[0399] Case 2: When the kth second improved prediction mode is an inter prediction mode, a search is performed based on the kth second improved prediction mode to obtain P pieces of second motion information. Based on these P pieces of second motion information, P pieces of second candidate prediction modes corresponding to the kth second improved prediction mode are obtained. For example, for the kth second improved prediction mode, the motion vector is (xInit, yInit), and a search range such as a rectangular area is set from xInit-sR to xInit+sR in the horizontal direction and from yInit-sR to yInit+sR in the vertical direction, where sR may be 2, 4, 8, etc. Each motion vector in the rectangular area is combined with other information of the kth second improved prediction mode, such as a reference image index and a prediction list flag, to determine the motion information, and further determine the prediction mode. Thereby, P pieces of second candidate prediction modes corresponding to the kth second improved prediction mode can be obtained.

[0400] In some embodiments, one of the ways of performing a search based on the k-th second improved prediction mode to obtain P pieces of second motion information is as follows: A search is performed within a preset search range based on a motion vector corresponding to the k-th second improved prediction mode to obtain P pieces of offsets. Each offset includes an offset in a first direction (e.g., x direction) and an offset in a second direction (e.g., y direction). Furthermore, P pieces of second motion information are obtained based on these M offsets. For example, for each of the P offsets, the motion vector corresponding to the k-th second improved prediction mode is offset based on the offset to obtain second motion information.

[0401] S103-A33: Determine a fifth cost corresponding to prediction to the template by each of the P second candidate prediction modes based on the first template weight.

[0402] According to the method of the above case 1 or case 2, after determining P second candidate prediction modes corresponding to the k-th second improved prediction mode, determine costs corresponding to the P second candidate prediction modes, and then determine an adjusted prediction mode corresponding to the k-th second improved prediction mode based on the costs. For convenience of explanation, the cost corresponding to the second candidate prediction mode is denoted as a fifth cost.

[0403] Specifically, for each of the P second candidate prediction modes, the template of the current block is predicted using the second candidate prediction mode to obtain a predicted value. Then, a fifth cost for the template corresponding to the first candidate prediction mode is determined based on the first template weight and the predicted value. Taking one sample in the template as an example, the second candidate prediction mode is used to predict the sample to obtain a predicted value of the sample. A template weight corresponding to the sample is determined from the first template weight, and the product of the predicted value of the sample and the template weight is determined as the predicted value of the sample in the second candidate prediction mode. With reference to this method, a predicted value of each sample in the template in the second candidate prediction mode can be obtained. These predicted values ​​form a predicted value of the template corresponding to the second candidate prediction mode. A fifth cost corresponding to the second candidate prediction mode is determined based on the predicted value of the template and the reconstructed value of the template.

[0404] By using the above method, the fifth costs corresponding to each of the P second candidate prediction modes are determined, and then the following steps S103-A14 are performed.

[0405] S103-A34: Determine an adjusted prediction mode corresponding to the k-th second refinement prediction mode based on the fifth costs corresponding to each of the P second candidate prediction modes.

[0406] For example, among the P second candidate prediction modes, the second candidate prediction mode having the smallest fifth cost is determined as the adjusted prediction mode corresponding to the k-th second improved prediction mode.

[0407] In the above embodiment, an example is described of determining an adjusted prediction mode corresponding to the kth second improved prediction mode among the K second improved prediction modes. However, by referring to the above description, it is possible to determine an adjusted prediction mode corresponding to each of the K second improved prediction modes and obtain K adjusted prediction modes.

[0408] In the second method, the decoding side first improves K initial prediction modes using P sub-templates to obtain K second improved prediction modes corresponding to the weight derivation mode of the current block, and then improves the K second improved prediction modes corresponding to the weight derivation mode of the current block using the weight derivation mode of the current block to obtain K accurate adjusted prediction modes. Thus, when predicting the current block using these K adjusted prediction modes and the weight derivation mode of the current block, the prediction accuracy can be further improved.

[0409] In an embodiment of the present application, the above method can determine the weight derivation mode of the current block and K adjusted prediction modes, and then use the weight derivation mode of the current block and the K adjusted prediction modes to predict the current block and obtain a predicted value of the current block.

[0410] In this application, the manner of determining the predicted value of the current block according to the weight derivation mode of the current block and the K adjusted prediction modes includes, but is not limited to, the following several manners:

[0411] Manner 1: in determining the first template weight according to the weight derivation mode of the current block, the first template weight is determined according to the weight derivation mode of the current block, but the weight of the prediction value corresponding to the sample in the current block is not determined. In this case, the above S103-B includes the following steps: S103-B11: Determine K predicted values ​​based on the K adjusted prediction modes. S103-B12: Determine the weight of the prediction value according to the weight derivation mode of the current block. S103-B13: Determine a prediction value of the current block according to the K prediction values ​​and the weights of the prediction values.

[0412] It should be noted that in this application, the weight derivation mode of the current block is used to determine the weight of the predicted value used for the current block. Specifically, the weight derivation mode of the current block may be a mode for deriving the weight of the predicted value. For a block having a certain length and width, one predicted value weight matrix may be derived for each weight derivation mode. For blocks having the same size, the predicted value weight matrices derived by different weight derivation modes are different.

[0413] As can be understood, in an embodiment of the present application, when the decoding side determines a prediction value of a current block based on the K adjusted prediction modes and the weights of the prediction values, the decoding side first determines K prediction values ​​based on the K adjusted prediction modes, and then performs a weighted average calculation on the K prediction values ​​using the weights of the prediction values ​​to finally obtain a prediction value of the current block.

[0414] In this method 1, determining the first template weight based on the derivation mode of the weight of the current block and determining the weight of the predicted value based on the derivation mode of the weight of the current block are two independent processes that do not interfere with each other.

[0415] In some embodiments, the present application may also use the following Scheme 2 to determine the predicted value of the current block.

[0416] Manner 2: during determining the first template weight, determine the weight of the sample in the merged region composed of the template region and the current block according to the weight derivation mode of the current block, in this case, the above S103-B includes the following steps: S103-B21: Determine K predicted values ​​based on the K adjusted prediction modes. S103-B22: Determine weights of the K predictors based on the weights of the samples in the merged region. S103-B23: Determine a predicted value of the current block according to the K predicted values ​​and weights of the K predicted values.

[0417] In the method 2, during weight derivation, the weight of the sample in the merged region is derived according to the weight derivation mode of the current block. The merged region includes the current block and the template region of the current block. Then, the weight corresponding to the current block in the merged region is determined as the weight of the predicted value, and the weight corresponding to the template region in the merged region is determined as the first template weight. That is, in the method 2, the template region and the current block are regarded as one whole region, so that the first template weight and the weight of the predicted value can be derived in one step, the number of steps of weight derivation can be reduced, and the prediction effect can be improved.

[0418] In some embodiments, the prediction process is performed on a sample-by-sample basis, and the weights are corresponding weights for the samples. In this case, when predicting the current block, a certain sample A in the current block is predicted using each of the K adjusted prediction modes to obtain K predicted values ​​of sample A in the K adjusted prediction modes. The weights of the K predicted values ​​of sample A are determined according to the weight derivation mode of the current block, and the K predicted values ​​are weighted to obtain a final predicted value of sample A. The predicted values ​​of each sample in the current block can be obtained by performing the above steps for each sample in the current block. The predicted values ​​of each sample in the current block constitute the predicted value of the current block. For example, K=2 is used to predict a certain sample A in the current block to obtain a first predicted value of the sample A, and a second prediction mode is used to predict the sample A to obtain a second predicted value of the sample A, and the first predicted value and the second predicted value are weighted according to the predicted value weight corresponding to sample A to obtain a predicted value of sample A.

[0419] For example, when K=2, if the first prediction mode and the second prediction mode are both intra prediction modes, a first predicted value is obtained by performing prediction using the first intra prediction mode, a second predicted value is obtained by performing prediction using the second intra prediction mode, and the first predicted value and the second predicted value are weighted based on the weight of the prediction value to obtain a predicted value of the current block. For example, a first intra prediction mode is used to predict sample A to obtain a first predicted value of sample A, a second intra prediction mode is used to predict sample A to obtain a second predicted value of sample A, and the first predicted value and the second predicted value are weighted based on the weight of the prediction value corresponding to sample A to obtain a predicted value of sample A.

[0420] In some embodiments, when the jth adjusted prediction mode among the K adjusted prediction modes is an inter prediction mode, determining a prediction value of the current block based on the K adjusted prediction modes and the weight derivation mode of the current block in S103-B above includes the following steps: S103-B31: Determine motion information based on the j-th adjusted prediction mode. S103-B32: Determine the j-th predicted value based on the motion information. S103-B33: Determine (K-1) predicted values ​​based on prediction modes other than the j-th adjusted prediction mode among the K adjusted prediction modes. S103-B34: Determine the weights of the K predictors according to the weight derivation mode of the current block. S103-B35: Determine a prediction value of the current block according to the j-th prediction value, the (K-1) prediction values, and the weight of the prediction values.

[0421] Taking K=2 as an example, when the first prediction mode is an intra prediction mode and the second prediction mode is an inter prediction mode, prediction is performed using the intra prediction mode to obtain a first predicted value, prediction is performed using the inter prediction mode to obtain a second predicted value, and the first predicted value and the second predicted value are weighted based on the weight of the predicted value to obtain a predicted value of the current block. In this example, each sample in the current block is predicted using the intra prediction mode to obtain a predicted value of each sample in the current block. The predicted value of each sample in the current block constitutes the first predicted value of the current block. Motion information is determined using the inter prediction mode, an optimal matching block of the current block is determined based on the motion information, and the optimal matching block is determined as the second predicted value of the current block. A weighting operation is performed on the first predicted value and the second predicted value of the current block for each sample using the weight of the predicted value of each sample in the current block to obtain a predicted value of the current block. For example, for sample A in the current block, a first predicted value corresponding to sample A in the first predicted value of the current block and a second predicted value corresponding to sample A in the second predicted value of the current block are weighted based on the weight of the predicted value of sample A to obtain a predicted value of sample A.

[0422] In some embodiments, when K is greater than 2, the weights of the predictors corresponding to any two of the K adjusted prediction modes may be determined based on the weight derivation mode of the current block. The weights of the predictors corresponding to the other of the K adjusted prediction modes may be preset values. For example, when K=3, the weights of the predictors corresponding to the first prediction mode and the weights of the predictors corresponding to the second prediction mode are derived based on the weight derivation mode of the current block, and the weights of the predictors corresponding to the third prediction mode are preset values. In some embodiments, when the total weights of the predictors corresponding to the K adjusted prediction modes are fixed, e.g., 8, the weights of the predictors corresponding to each of the K adjusted prediction modes may be determined based on the ratio of the preset weights. Assuming that the weights of the predictors corresponding to the third prediction mode account for 1 / 4 of the total weights of the entire predictors, the weights of the predictors corresponding to the third prediction mode may be determined to be 2, and the remaining 3 / 4 of the total weights of the predictors are assigned to the first prediction mode and the second prediction mode. For example, if the weight of the prediction value corresponding to the first prediction mode derived based on the weight derivation mode of the current block is 3, the weight of the prediction value corresponding to the first prediction mode is determined to be (3 / 4)*3, and the weight of the prediction value corresponding to the second prediction mode is determined to be (3 / 4)*5.

[0423] The decoding side determines the predicted value of the current block according to the above method. The decoding side also obtains the quantized coefficient of the current block by decoding the bitstream, inversely quantizes the quantized coefficient to obtain the transform coefficient, and inversely transforms the transform coefficient to obtain the residual value of the current block. The predicted value and the residual value of the current block are added to obtain the reconstructed value of the current block.

[0424] In the prediction method according to the embodiment of the present application, the decoding side decodes the bitstream, determines K initial prediction modes, determines the weight derivation mode of the current block according to the K initial prediction modes, and determines the predicted value of the current block according to the weight derivation mode of the current block. That is, in the present application, the weight derivation mode of the current block is determined according to the K initial prediction modes, thereby enriching the weight derivation mode determination method. In addition, even if the encoding side in the present application does not transmit the index of the weight derivation mode in the bitstream, the decoding side can determine the weight derivation mode of the current block according to the K initial prediction modes. Thus, it is possible to save codewords and reduce coding costs.

[0425] The prediction method of the present application has been introduced above by taking the decoding side as an example, and will now be described by taking the encoding side as an example.

[0426] Figure 19 is a flowchart illustrating a prediction method according to an embodiment of the present application. The embodiment of the present application is applied to the video encoder shown in Figure 1 and Figure 2. As shown in Figure 19, the method of the embodiment of the present application includes the following contents:

[0427] S201: K initial prediction modes are determined.

[0428] It should be noted that in the embodiment of the present application, a video image may be divided into a plurality of image blocks, and a current block is each current image block to be encoded, and may be referred to as a coding block (CB). Each coding block may include a first color component, a second color component, and a third color component. Specifically, in the present application, if a first prediction is performed and the first color component is a luma component, i.e., the color component to be predicted is a luma component, the coding block to be predicted may be referred to as a luma block. Or, if a second prediction is performed and the second color component is a chroma component, i.e., the color component to be predicted is a chroma component, the coding block to be predicted may be referred to as a chroma block.

[0429] As can be seen from the above, in order to further improve prediction accuracy, a current block can be predicted using multiple prediction modes. In this case, in order to maintain prediction consistency between the encoding side and the decoding side, in some embodiments, the encoding side needs to indicate the weight derivation mode in the bitstream, for example, by writing the index of the weight derivation mode into the bitstream. Taking VVC as an example, the 64 weight derivation modes of GPM are encoded with equal probability. That is, during binarization and de-binarization, a fixed length of 6 bits is used for 64 weight derivation modes, which increases the coding cost.

[0430] In the embodiment of the present application, in order to reduce coding cost, the decoding side derives weight derivation mode according to prediction mode. Specifically, the decoding side determines K initial prediction modes by decoding bitstream, then determines the weight derivation mode of current block according to these K initial prediction modes, and further determines the predicted value of current block according to the weight derivation mode of current block. Correspondingly, the encoding side also uses the same prediction method as the decoding side, that is, determines K initial prediction modes, determines the weight derivation mode of current block according to these K initial prediction modes, and further determines the predicted value of current block according to the weight derivation mode of current block.

[0431] In some embodiments, for ease of explanation, the weight derivation mode of the current block is noted as the weight derivation mode of the current block.

[0432] The K initial prediction modes of the current block include the following several examples:

[0433] Example 1: All of the above K initial prediction modes are intra prediction modes.

[0434] Example 2: All of the K initial prediction modes are inter prediction modes.

[0435] Example 3: Among the K initial prediction modes, at least one is an intra prediction mode and at least one is an inter prediction mode.

[0436] Example 4: Among the above K initial prediction modes, at least one is an intra prediction mode, and at least one is a non-inter prediction mode and a non-intra prediction mode, such as an intra block copy (IBC) prediction mode or a palette prediction mode.

[0437] Example 5: Among the K initial prediction modes, at least one is an inter prediction mode, and at least one is a non-inter prediction mode and a non-intra prediction mode, such as an IBC prediction mode or a palette prediction mode.

[0438] Example 6: None of the K initial prediction modes are intra prediction modes or inter prediction modes. For example, one initial prediction mode is an IBC prediction mode and one initial prediction mode is a palette prediction mode.

[0439] It should be noted that the embodiment of the present application does not limit the specific types of the K initial prediction modes.

[0440] The manner of determining the K initial prediction modes in S201 above includes, but is not limited to, the following:

[0441] Method 1: The K initial prediction modes are preset K prediction modes.

[0442] Method 2: The above S201 includes steps S201-A1 to S201-A2. S201-A1: determine a list of selected prediction modes, where the list of selected prediction modes includes at least two selected prediction modes; S201-A3: Determine K initial prediction modes based on the list of selected prediction modes.

[0443] In some embodiments, if the K initial prediction modes are all intra prediction modes, the selected prediction modes included in the list of selected prediction modes are all intra prediction modes, for example, including at least one of DC mode, planar mode, angular mode, etc. Optionally, the list of selected prediction modes may also include intra prediction modes in an MPM list.

[0444] In some embodiments, if all of the K initial prediction modes are inter prediction modes, all of the selected prediction modes included in the list of selected prediction modes are inter prediction modes, for example including at least one of skip mode, merge mode, normal inter prediction mode, etc.

[0445] In some embodiments, if the K initial prediction modes include both inter-prediction modes and intra-prediction modes, the list of selected prediction modes includes at least one intra-prediction mode and at least one inter-prediction mode.

[0446] Optionally, the list of selected prediction modes may also include modes such as IBC mode, palette mode, and the like.

[0447] Optionally, the list of selected prediction modes may include unidirectional prediction, bidirectional prediction, multi-hypothesis prediction, and the like.

[0448] The present application does not limit the types and number of the selected prediction modes included in the above list of selected prediction modes.

[0449] The embodiment of the present application does not limit the manner of determining the list of selected prediction modes.

[0450] In one example, the selected prediction modes included in the list of selected prediction modes are preset modes.

[0451] In one example, the list of selected prediction modes is an MPM list.

[0452] In one example, the list of selected prediction modes is a set of prediction modes determined according to some rules, such as equal interval selection.

[0453] In one example, both the encoding side and the decoding side default that the selected prediction modes in the list of selected prediction modes are determined by template matching. For example, the encoding side predicts the template of the current block using a prediction mode to obtain a predicted value of the template, and determines the cost of the prediction mode according to the predicted value of the template and the reconstructed value of the template, for example, SAD cost, SATD cost, or MSE cost. Thereby, the encoding side can sort the prediction modes according to the cost of each prediction mode to obtain the list of selected prediction modes, for example, determine a number of prediction modes with the smallest cost as selected prediction modes to configure the list of selected prediction modes.

[0454] In the embodiment of the present application, after determining the list of selected prediction modes according to the above method, K selected prediction modes are selected from the list of selected prediction modes as the K initial prediction modes.

[0455] Exemplarily, the encoding side randomly selects K selected prediction modes from the list of selected prediction modes as the K initial prediction modes.

[0456] Exemplarily, the encoding side selects K selected prediction modes with the smallest costs from the list of selected prediction modes as the K initial prediction modes.

[0457] After determining the K initial prediction modes, the encoding side writes a prediction mode index into the bitstream. The prediction mode index may be an index of at least one of the K initial prediction modes. For example, the prediction mode index may be an index of the K initial prediction modes, or may be an index of the first initial prediction mode of the K initial prediction modes. In this way, the decoding side decodes the bitstream to obtain the prediction mode index, and determines the K initial prediction modes from the determined selected prediction mode list based on the prediction mode index.

[0458] In an embodiment of the present application, the encoding side determines K initial prediction modes based on the above steps, determines a weight derivation mode of the current block based on the K initial prediction modes, and further predicts the current block based on the weight derivation mode of the current block to determine a predicted value of the current block.

[0459] In some embodiments, before determining the K initial prediction modes, the encoding side needs to first determine whether to perform weighted prediction on the current block using K different prediction modes. If the encoding side determines that the current block is to perform weighted prediction using K different prediction modes, it performs S201 to determine the K initial prediction modes.

[0460] In one possible embodiment, the encoding side can determine whether to perform weighted prediction using two different prediction modes for the current block by determining a prediction mode parameter of the current block.

[0461] Optionally, in an embodiment of the present application, the prediction mode parameter may indicate whether GPM mode or AWP mode can be used for the current block, i.e., whether prediction can be performed for the current block using K different prediction modes.

[0462] It should be noted that in the embodiment of the present application, the prediction mode parameter may be understood as a flag bit indicating whether the GPM mode or the AWP mode flag bit is used. Specifically, the encoding side may use one variable as the prediction mode parameter, and thereby the setting of the prediction mode parameter is realized by setting the value of the variable. Exemplarily, in the present application, when the GPM mode or the AWP mode is used for the current block, the encoding side may set the value of the prediction mode parameter to one indicating that the GPM mode or the AWP mode is used for the current block, and specifically, the encoding side may set the value of the variable to 1. Exemplarily, in the present application, when the GPM mode or the AWP mode is not used for the current block, the encoding side may set the value of the prediction mode parameter to one indicating that the GPM mode or the AWP mode is not used for the current block, and specifically, the encoding side may set the value of the variable to 0. Furthermore, in the embodiment of the present application, after completing the setting of the prediction mode parameter, the encoding side may write the prediction mode parameter into a bitstream and transmit it to the decoding side. Thereby, the decoding side may obtain the prediction mode parameter after analyzing the bitstream.

[0463] Based on the above, the encoding side determines whether to use the GPM mode or the AWP mode for the current block according to the prediction mode parameter. If the GPM mode or the AWP mode is used for the current block, that is, if prediction is performed using K different prediction modes, the encoding side determines K initial prediction modes.

[0464] It should be noted that in the embodiment of the present application, GPM mode or AWP mode is a kind of prediction method. Specifically, K different prediction modes are determined for the current block, then K predicted values ​​are determined according to these K different prediction modes, and then weights can be determined, and the K predicted values ​​are combined according to the weights to finally obtain the predicted value of the current block.

[0465] In some embodiments, when using the GPM mode or the AWP mode, the size of the current block may be restricted. In the prediction method according to the embodiment of the present application, it is necessary to generate K predicted values ​​using K different prediction modes, and weight them based on the weights to obtain a predicted value of the current block. In order to reduce complexity, and considering the trade-off between compression performance and complexity, in the embodiment of the present application, it is possible to restrict the use of the GPM mode or the AWP mode for a block having a certain size. Therefore, in the present application, the encoding side can first determine a size parameter of the current block, and then determine whether to use the GPM mode or the AWP mode for the current block according to the size parameter.

[0466] It should be noted that in the embodiment of the present application, the size parameters of the current block may include the height and width of the current block. Thus, the encoding side may use the height and width of the current block to restrict the use of the GPM mode or the AWP mode. Exemplarily, the present application determines that the current block is to use the GPM mode or the AWP mode if the width is greater than a first threshold and the height is greater than a second threshold. As can be seen above, one possible restriction is to use the GPM mode or the AWP mode only if the width of the block is greater than (or equal to or greater than) the first threshold and the height of the block is greater than (or equal to or greater than) the second threshold. The values ​​of the first and second thresholds may be 8, 16, 32, etc., and the first threshold may be equal to the second threshold. Exemplarily, the present application determines that the current block is to use the GPM mode or the AWP mode if the width is less than a third threshold and the height is greater than a fourth threshold. As seen above, one possible restriction is to only utilize AWP mode if the width of the block is less than (or less than or equal to) the third threshold and the height of the block is greater than (or greater than or equal to) the fourth threshold. The values ​​of the third and fourth thresholds may be 8, 16, 32, etc., and the third threshold may be equal to the fourth threshold.

[0467] In some embodiments, in the embodiment of the present application, the size of the block that can use the GPM mode or the AWP mode can be limited by the restriction on the sample parameter. Exemplarily, in the present application, the encoding side can first determine the sample parameter of the current block, and then determine whether the GPM mode or the AWP mode can be used for the current block based on the sample parameter and the fifth threshold. As can be seen above, one possible restriction is to use the GPM mode or the AWP mode only if the number of samples in the block is greater than (or equal to or greater than) the fifth threshold. The value of the fifth threshold may be 8, 16, 32, etc. That is, in the present application, the GPM mode or the AWP mode can be used for the current block only under the condition that the size parameter of the current block meets the size requirement.

[0468] In some embodiments, there may be an image-level flag in the present application to determine whether the technical solution of the present application is applied to the current image to be decoded. For example, the present application may be configured to apply to intraframes (e.g., I frames) and not to interframes (e.g., B frames, P frames). Or, the present application may be configured to apply to interframes and not to intraframes. Or, the present application may be configured to apply to some interframes and not to other parts of interframes. Since intraframe prediction can also be used for interframes, the present application may also be applied to interframes.

[0469] In some embodiments, there may also be flags below the image level and above the CU level (e.g., tile, slice, patch, LCU, etc.) to determine whether the technical proposal of the present application is applied to this area.

[0470] S202: Determine a weight derivation mode of a current block according to the K initial prediction modes.

[0471] In this application, the weight derivation mode is used to determine the weight used for the current block. Specifically, the weight derivation mode can be a mode for deriving weights. For a block having a certain length and width, one weight matrix can be derived for each weight derivation mode. For blocks having the same size, the weight matrices derived by different weight derivation modes are different.

[0472] For example, in this application, there are 56 weight derivation modes in AWP and 64 weight derivation modes in GPM.

[0473] For ease of explanation, in the embodiment of the present application, the finally determined weight derivation mode for prediction of the current block is referred to as the weight derivation mode of the current block.

[0474] In the embodiment of the present application, the encoding side determines the weight derivation mode of the current block according to the determined K initial prediction modes. In some cases, the encoding side does not need to write the information related to the weight derivation mode of the current block into the bitstream, thereby saving codewords and reducing coding cost.

[0475] The embodiment of the present application does not limit the manner in which the encoding side determines the weight derivation mode of the current block based on the K initial prediction modes.

[0476] In some embodiments, the encoding side determines the weight derivation mode of the current block according to the template of the current block by K initial prediction modes. Taking VVC as an example, GPM has 64 weight derivation modes, and the encoding side estimates which weight derivation mode is more likely to be selected and which weight derivation mode is less likely to be selected by analyzing the template. Taking Inter GPM as an example, a typical scene is a scene where two objects move toward each other, and GPM can be used for the edges of the two objects. Assuming that both the current block and the template contain the edges of the objects, the edges in the template can be used to infer the edges of the current block. As an example, two pieces of motion information are given, and two predicted values ​​of the template are obtained based on the two pieces of motion information, which are respectively denoted as the first template predicted value and the second template predicted value. A template weight is derived based on a certain weight derivation mode, and a predicted value of the template corresponding to the weight derivation mode is determined based on the first template predicted value, the second template predicted value, and the template weight. Since the reconstruction value of the template is available, the cost of the weight derivation mode in the template can be obtained based on the predicted value of the template corresponding to the weight derivation mode and the reconstruction value of the template. Thus, the probability that the weight derivation mode is selected can be estimated based on the cost of each weight derivation mode in the template. One possible embodiment is to sort the weight derivation modes based on the cost of each weight derivation mode in the template. The mode with the smallest cost is located at the front, and the mode with the largest cost is located at the back. It is considered that the sorted weight derivation modes are roughly sorted from the most probable to the least probable, so that they can be processed using the method of variable length coding. One possible embodiment is as follows: A list of weight derivation modes is constructed, and the encoding side writes the index in the list of the weight derivation mode of the current block selected for the current block into the bitstream.The decoder analyzes the index in the bitstream to determine the weight derivation mode of the current block from the list.

[0477] It can be seen from the above that, in the embodiment of the present application, in the process of determining the weight derivation mode of the current block based on the K initial prediction modes, it is necessary to determine the template weight based on the weight derivation mode.

[0478] The process of determining template weights based on the weight derivation mode according to an embodiment of the present application is described below.

[0479] Template matching uses the correlation between adjacent samples to select some areas around the current block as templates. Before encoding and decoding the current block, the blocks to its left and above have already been decoded according to the coding order. Inter prediction determines the motion information or motion vector of the current block by finding the best matching position of the template from the reference image. Intra prediction uses the template to determine the intra prediction mode of the current block.

[0480] The present application does not limit the specific shape of the template of the current block.

[0481] In some embodiments, the template includes at least one of the above decoded region, the left decoded region, and the top-left decoded region of the current block.

[0482] Optionally, the width of the upper decoded area is the same as the width of the current block, the height of the left decoded area is the same as the height of the current block, the width of the upper left decoded area is the same as the width of the left decoded area, and the height of the upper left decoded area is the same as the height of the upper decoded area.

[0483] Currently, as shown in Table 5, for example, K=2, the K prediction modes include a first prediction mode and a second prediction mode, and the template corresponding to the first prediction mode and the template corresponding to the second prediction mode are the upper decoded region of the current block, or the left decoded region of the current block, or the left decoded region and the upper decoded region of the current block, respectively. For example, as shown in FIG. 16, taking the weight derivation mode with a GPM index of 2 as an example, the white area of ​​the weight matrix of the current block is the weight corresponding to the predicted value of the first prediction mode, and the black area is the weight corresponding to the predicted value of the second prediction mode. As shown in FIG. 16, the first template corresponding to the first prediction mode is the upper decoded region of the current block, and the second template corresponding to the second prediction mode is the left decoded region of the current block, but the template close to the second prediction mode includes a part of the upper decoded region in addition to the left region. Therefore, in the existing technology, the division of the template is not fine enough, and as a result, the weight derivation mode of the current block cannot be accurately derived based on the template.

[0484] In order to solve the technical problem, the present application can realize finer division of the template according to the weight derivation mode. For example, as shown in FIG. 17A, the present application extends a division line of a weight matrix corresponding to a weight derivation mode of a current block toward the template region to divide the template. For example, a template on one side of the division line is denoted as a first template, and a template on the other side of the division line is denoted as a second template. The first template corresponds to a first prediction mode, the second template corresponds to a second prediction mode, and the weight of the first template is determined based on the weight corresponding to the first prediction mode, and the weight of the second template is determined based on the weight corresponding to the second prediction mode. For example, the weight of the first template is the same as the weight corresponding to the first prediction mode, and the weight of the second template is the same as the weight corresponding to the second prediction mode.

[0485] In some embodiments, the first and second templates divided according to the above method may not be rectangular, for example, as shown in Figure 17A, the first and second templates have slanted sides, but the calculation of the cost for the irregular templates is relatively complicated.

[0486] To reduce the complexity of template matching, in some embodiments, the weight of the template can be determined by extending the weight matrix directly toward the template region, for example, toward the left or toward the top, to cover the template. For example, as shown in FIG. 17B, a small rectangular region in the upper left region of the current block can be selectively added to the template, and the template and the current block together form a rectangle. Of course, it is also possible to use only the left and upper parts as the template. As shown in FIG. 17B, taking the upper left region as an example, the inverted L-shaped regions on the left, upper left, and upper sides are the template region, the rectangular region on the lower right is the current block, and the weight matrix extended to the upper left is the weight matrix of the template. In this way, the template is not actually divided, but the weight is used to simulate the division of the template. It can also be said that the weight derivation mode is used to simulate the division of the template, or that the weight derivation mode is used to determine the template weight, and the cost of template matching is determined based on the template weight. Since different block shapes may result in different template divisions, in the present application, it is not necessary to set various rules for blocks having various shapes, and it is not necessary to set, for example, the correspondence between prediction modes and templates shown in Table 5. In addition, regarding the accuracy of "division," the simulated division in the present application is finer and more accurate than the method that distinguishes only the left region and the upper region.

[0487] In some embodiments, the left and upper regions of the current block shown in FIG. 17B are taken as the template for the current block, and the upper left matrix region is not considered.

[0488] During template matching, it is possible to use only one component (Y, Cb, Cr or R, G, B). For example, in the YUV format, Y is dominant, so Y can be used, but of course all components can be used. For ease of explanation, this application takes one component as an example, for example, template matching is performed on the Y component, and other components can refer to the Y component.

[0489] In an embodiment of the present application, the process of deriving the template weight based on the weight derivation mode and the process of deriving the weight of the predicted value can be combined, for example, the template weight and the weight of the predicted value are derived simultaneously. The weight of the predicted value can be understood as a weight corresponding to the predicted value. For example, a first predicted value is obtained using a first prediction mode, a second predicted value is obtained using a second prediction mode, a first weight of the first predicted value is determined based on the weight derivation mode, a second weight of the second predicted value is determined based on the weight derivation mode, and the sum of the product of the first predicted value and the first weight and the product of the second predicted value and the second weight is determined as the predicted value of the current block.

[0490] In this application, the first weight and the second weight are referred to as predictor weights to distinguish them from the template weights.

[0491] In some embodiments, determining the template weights based on the weight derivation mode includes the following steps. Step A: Determine angle indexing and distance indexing based on the weight derivation mode. Step B: Determine the template weights based on the angle index, the distance index, and the size of the template.

[0492] In the present application, the template weights can be derived in the same manner as the weights of the predictors are derived. For example, the angle indexing and the distance indexing can be determined based on the weight derivation mode first. The angle indexing can be understood as the angle indexing of the boundary of each weight derived by the weight derivation mode. For example, the angle indexing and the distance indexing corresponding to the weight derivation mode can be determined according to Table 2 above. For example, when the weight derivation mode is 27, the corresponding angle indexing is 12, and the corresponding distance indexing is 3. Then, the template weights are determined based on the angle index, the distance index, and the size of the template.

[0493] The manner of deriving template weights based on the weight derivation mode in the above step B includes, but is not limited to, the following:

[0494] Method 1: Determine the template weight directly according to the angle index, the distance index and the size of the template. In this case, the above step B includes the following steps B11 to B13. Step B11: Determine a first parameter of the sample in the template according to the angle index, the distance index and the size of the template. Step B12: Determine weights of the samples in the template according to the first parameters of the samples in the template. Step B13: Determine the template weight based on the weights of the samples in the template.

[0495] In this embodiment, the weights of the samples in the template are determined based on the angle index, the distance index, the size of the template, and the size of the current block, and a weight matrix consisting of the weights of each sample in the template is determined as the template weight.

[0496] The first parameter in this application is used to determine the weight, and in some embodiments, the first parameter is also referred to as a weight index.

[0497] In one possible embodiment, the offset and the first parameter can be determined in the manner of the above embodiment, and the specific content can be referred to the description of the above embodiment, and will not be described in detail here.

[0498] After determining the first parameter weightIdx according to the above method, the weight of the sample (x, y) in the template is determined according to weightIdx.

[0499] In this application, the manner of determining the weight of the sample in the template based on the first parameter of the sample in the template in the above step B12 includes but is not limited to the following:

[0500] Method 1: determining a second parameter of the sample in the template based on a first parameter of the sample in the template, and determining a weight of the sample in the template based on the second parameter of the sample in the template.

[0501] The second parameter is also used to determine the weight, which in some embodiments is also referred to as a weight index on the first component, which may be a luma component, a chroma component, etc.

[0502] Manner 2: Determine the weight of the sample in the template according to a first parameter of the sample in the template, a first preset value, and a second preset value.

[0503] In order to reduce the computational complexity of the template weight, in method 2, the weight of the sample in the template is limited to a first preset value or a second preset value, that is, the weight of the sample in the template is set to either the first preset value or the second preset value, thereby reducing the computational complexity of the template weight.

[0504] The present application does not limit the specific values ​​of the first preset value and the second preset value.

[0505] Optionally, the first preset value is one.

[0506] Optionally, the second preset value is zero.

[0507] In the above method 1, the weight of each sample in the template is determined based on the weight derivation mode, and the weight matrix consisting of the weights of each sample in the template is taken as the template weight.

[0508] Method 2: Determine the weight of the current block and the template weight according to the weight derivation mode. That is, in this method 2, the merged region consisting of the current block and the template is taken as the whole, and the weight of the samples in the merged region is derived according to the weight derivation mode. Based on this, the above step B includes the following steps B21 and B22. Step B21: Determine the weights of the samples in the merged region consisting of the current block and the template according to the angle index, the distance index, the size of the template, and the size of the current block. Step B22: Determine the template weight based on the size of the template and the weight of the samples in the merged region.

[0509] In the second method, the current block and the template are taken as a whole, and the weight of the samples in the merged region formed by the current block and the template is determined according to the angle index, the distance index, the size of the template, and the size of the current block, and then the weight corresponding to the template in the weight of the samples in the merged region is determined as the template weight according to the size of the template. For example, as shown in FIG. 18, the weight corresponding to the L-shaped template region in the merged region is determined as the template weight.

[0510] In the method 2, in the weight determination process, not only the template weight is determined, but also the prediction weight is determined. That is, after removing the template weight from the weight corresponding to the merged region, what remains is the prediction weight. Therefore, the subsequent prediction process can be performed based on the prediction weight without the need to determine the prediction weight again. Therefore, the prediction step can be reduced and the prediction efficiency can be improved.

[0511] The present application does not limit the specific embodiment of determining the weights of the samples in the merged region consisting of the current block and the template based on the angle index, the distance index, the size of the template, and the size of the current block.

[0512] In some embodiments, determining the weights of the samples in the merged region in the above step B21 includes the following steps B211 to B212. Step B211: Determine a first parameter of the samples in the merged region according to the angle index, the distance index, and the size of the merged region. Step B212: Determine weights of the samples in the merged region according to first parameters of the samples in the merged region.

[0513] In this embodiment, the weights of the samples in the merged region are determined based on the angle index, the distance index, and the size of the merged region, and the weights of each sample in the merged region form a weight matrix.

[0514] For specific details, please refer to the description of the above embodiment and will not be described in detail here.

[0515] In this application, the manner of determining the weights of the samples in the merged region according to the first parameters of the samples in the merged region in the above step B212 includes, but is not limited to:

[0516] Method 1: determining a second parameter of the samples in the merged region according to the first parameter of the samples in the merged region, and determining weights of the samples in the merged region according to the second parameter of the samples in the merged region.

[0517] Manner 2: Determine the weights of the samples in the merged region according to a first parameter of the samples in the merged region, a first preset value, and a second preset value.

[0518] Illustratively, the weight of the samples in the merged region is a first preset value or a second preset value.

[0519] In order to reduce the computational complexity of the merging region weight, in the method 2, the weight of the sample in the merging region is limited to a first preset value or a second preset value, that is, the weight of the sample in the merging region is set to either the first preset value or the second preset value, thereby reducing the computational complexity of the merging region weight.

[0520] The present application does not limit the specific values ​​of the first preset value and the second preset value.

[0521] Optionally, the first preset value is one.

[0522] Optionally, the second preset value is zero.

[0523] In the above method 2, the weight of each sample in the merged region consisting of the current block and the template is determined according to the weight derivation mode, and the weight corresponding to the template in the merged region is determined as the template weight according to the size of the template. In addition, in the method 2, the weight of the predicted value is determined without needing to perform the subsequent step of determining the weight of the predicted value, thereby reducing the prediction step and improving the prediction efficiency.

[0524] The above describes in detail the process of determining the template weight based on the weight derivation mode. The weight derivation mode may be any one of the weight derivation modes according to the embodiments of the present application, for example, the weight derivation mode of the current block or the i-th candidate prediction mode among N candidate prediction modes.

[0525] S203: Determine a predicted value of the current block according to a weight derivation mode of the current block.

[0526] In this application, the manner in which the encoding side determines the predicted value of the current block according to the weight derivation mode of the current block in S203 includes, but is not limited to, the following:

[0527] Method 1: The encoding side determines the predicted value of the current block according to the weight derivation mode of the current block and the K initial prediction modes. Specifically, the encoding side determines the weight of the predicted value according to the weight derivation mode of the current block, then predicts the current block according to the K initial prediction modes to obtain K predicted values, and then uses the weight of the predicted value to perform a weighted average calculation on the K predicted values ​​to obtain the predicted value of the current block.

[0528] It should be noted that the weight of the prediction value derived based on the weight derivation mode of the current block can be understood as a weight corresponding to each sample in the current block, or a weight matrix corresponding to the current block. During determining the prediction value of the current block based on the weight of the prediction value, K prediction values ​​corresponding to each sample in the current block can be determined, and the prediction value corresponding to each sample can be determined based on the K prediction values ​​corresponding to each sample and the weight of the prediction value. The prediction value corresponding to each sample in the current block constitutes the prediction value of the current block. Optionally, determining the prediction value of the current block based on the weight of the prediction value can be performed based on a block, for example, determining K prediction values ​​of the current block, and weighting the K prediction values ​​of the current block according to the weight matrix of the current block to obtain the prediction value of the current block.

[0529] Method 2: The encoding side can improve the K initial prediction modes by using the weight derivation mode of the current block. In this case, the S203 step includes the following steps S203-A and S203-B. S203-A: Determine K adjusted prediction modes based on the weight derivation mode of the current block. S203-B: Determine a prediction value of the current block according to the K adjusted prediction modes and the weight derivation mode of the current block.

[0530] In an embodiment of the present application, the GPM can be considered to include (K+1) elements, i.e., one weight derivation mode and K prediction modes. One weight derivation mode and K prediction modes together generate one prediction block, which is used for the current block. Therefore, they are related to each other. For example, the current block contains the edges of two objects that are moving relatively, which is an ideal scenario for inter GPM. In theory, this "split" should occur at the edge of the object, but in practice, the "split" possibilities are limited and cannot cover all edges. An approximate "split" may be selected. There may be multiple approximate "splits", and which "split" is selected depends on which "split" in combination with the two prediction modes produces the best results. Similarly, which prediction mode is selected depends on which combination produces the best results. This is because even for parts that use the prediction mode, it is difficult to perfectly match this part with the current block in the case of natural video. The final selection may be the one with the highest coding efficiency. Another scenario in which GPM is often used is a scenario in which the current block contains a part with relative motion within an object. For example, in some cases where there is twisting or deformation due to arm swinging, the "splitting" becomes more ambiguous and may ultimately depend on which combination produces the best result. Another scenario is intra prediction, where some parts of a natural image have complex textures, some parts have gradual changes from one texture to another, and some parts cannot be simply represented in one direction. Intra GPM can provide more complex prediction blocks, and intra-coded blocks usually have larger residuals compared to inter-coded blocks with the same quantization. The choice of which prediction mode ultimately depends on which combination produces the best result.

[0531] As can be seen from the above, since the (K+1) elements affect each other, the prediction mode can be determined using the weight derivation mode, and the weight derivation mode can be determined using the prediction mode. In addition, some mode information can be refined. Refinement means that more related information is collected based on the initial mode information, and more methods such as calculation are used to obtain better mode information. Refine is generally an encoding side operation. Based on the template, the motion information can also be refined. That is, based on the initial motion information, a search is performed within a small range on the encoding side, and the motion information is optimized according to the degree of matching between the searched motion information and the template. The degree of matching between the motion information and the template can be understood as the degree of matching between the predicted value of the template based on the motion information and the reconstructed value of the template.

[0532] In GPM, the weight derivation mode can be determined based on the prediction mode first, and then the prediction mode can be improved based on the weight derivation mode. Or, the prediction mode can be determined based on the weight derivation mode first, and then the weight derivation mode can be improved based on the prediction mode. Of course, if the complexity allows, both can be performed alternately. The template can be the link to perform these methods, and the template weight can be the link to perform these methods. That is, the cost on the template is calculated based on the predicted value of the template in the prediction mode and the weight of the sub-template or template corresponding to the weight derivation mode. The cost on the template is used to derive or determine the prediction mode or weight derivation mode.

[0533] That is, in the embodiment of the present application, a weight derivation mode of a current block is determined based on K initial prediction modes, and then the weight derivation mode of the current block is used to improve the K initial prediction modes to obtain K adjusted prediction modes, and then a predicted value of the current block is determined based on the weight derivation mode of the current block and the K adjusted prediction modes.

[0534] For a specific process of determining a predicted value of a current block based on the above K adjusted prediction modes and a weight derivation mode of the current block, please refer to the following specific description of predicting a current block based on a weight derivation mode and K prediction modes to determine a predicted value of the current block, and will not be described in detail here.

[0535] Hereinafter, according to a specific embodiment, the process of determining a weight derivation mode of a current block based on the K initial prediction modes in S202, and the process of determining K adjusted prediction modes based on the weight derivation mode of a current block in S203-A will be described in detail.

[0536] In some embodiments, the above S202 includes the following steps S202-A to S202-C. S202-A: Determine N candidate weight derivation modes, where N is a positive integer greater than 1. S202-B: For an i-th candidate weight derivation mode among the N candidate weight derivation modes, determine a first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and the K initial prediction modes, where i is a positive integer from 1 to N. S202-C: Determine a weight derivation mode for the current block based on a first cost corresponding to each of the N candidate weight derivation modes.

[0537] In this embodiment, the encoding side first determines N candidate weight derivation modes, which constitute a list of weight derivation modes, and then calculates a first cost corresponding to each of the N candidate weight derivation modes, and determines a weight derivation mode for the current block from the N candidate weight derivation modes based on the first cost.

[0538] For example, among the N candidate weight derivation modes, the candidate weight derivation mode with the smallest first cost is determined as the weight derivation mode of the current block.

[0539] As another example, the N candidate weight derivation modes are sorted according to the first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes. The weight derivation mode of the current block is determined according to the list of candidate weight derivation modes. For example, any one of the list of candidate weight derivation modes is determined as the weight derivation mode of the current block. In this embodiment, the encoding side writes a weight derivation mode index corresponding to the weight derivation mode of the current block into the bitstream. Thereby, the decoding side determines the weight derivation mode of the current block from the list of determined candidate weight derivation modes according to the weight derivation mode index.

[0540] Optionally, the first cost may be a cost such as SAD, SATD or SEE.

[0541] The process in which the encoding side determines the first cost corresponding to each of the N candidate weight derivation modes is the same. For ease of explanation, the process of determining the first cost corresponding to the i-th candidate weight derivation mode among the N candidate weight derivation modes is taken as an example.

[0542] According to the i-th candidate weight derivation mode and the K initial prediction modes in the above S202-B, the manner of determining the first cost corresponding to the i-th candidate weight derivation mode includes, but is not limited to, the following:

[0543] Scheme 1: Predict a template of a current block by using the i-th candidate weight derivation mode and K initial prediction modes to obtain a first predicted value of the template, and determine a first cost corresponding to the i-th weight derivation mode according to the first predicted value of the template and the reconstructed value of the template.

[0544] Specifically, a template weight is determined based on the i-th candidate weight derivation mode, and the template is predicted using K initial prediction modes to obtain K predicted values ​​of the template. The template weight is used to weight the K predicted values ​​corresponding to the template to obtain a predicted value of the template. For convenience of explanation, the predicted value of the template is represented as a first predicted value of the template. Then, a prediction cost for the template corresponding to the i-th candidate weight derivation mode is determined based on the first predicted value of the template and the reconstructed value of the template, and the predicted cost is represented as a first cost corresponding to the i-th weight derivation mode.

[0545] It should be noted that the specific process of determining the template weight based on the i-th candidate weight derivation mode can be referred to the following specific content of determining the template weight based on the weight derivation mode, so it will not be described in detail here.

[0546] The above method can determine the first cost corresponding to each of the N candidate weight derivation modes, and then determine the candidate weight derivation mode with the smallest first cost among the N candidate weight derivation modes as the weight derivation mode of the current block. Or, sort the N candidate weight derivation modes according to the first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes. In the list of candidate weight derivation modes, determine the candidate weight derivation mode corresponding to the weight derivation mode index as the weight derivation mode of the current block.

[0547] According to the above Scheme 1, the scheme of determining K adjusted prediction modes according to the weight derivation mode of the current block in S203-A includes the following steps:

[0548] S203-A1: Improve K initial prediction modes based on a weight derivation mode of a current block to obtain K adjusted prediction modes.

[0549] Examples of S203-A1 include, but are not limited to, some of the following examples:

[0550] Example 1: determining at least one prediction mode for a weight derivation mode of a current block, the at least one prediction mode may be a prediction mode whose prediction angle is parallel or nearly parallel to a boundary surface of the weight derivation mode of the current block, or a prediction mode whose prediction angle is perpendicular or nearly perpendicular to a boundary surface of the weight derivation mode of the current block. At least one initial prediction mode of the K initial prediction modes is replaced with a prediction mode of the at least one prediction mode of the weight derivation mode of the current block, forming a new combination of prediction modes. Each of the new combinations of prediction modes includes K prediction modes. For each of the new combinations of prediction modes, a template is predicted using the K prediction modes included in the new combination of prediction modes and the weight derivation mode of the current block to obtain a predicted value of the template. A cost corresponding to the new combination of prediction modes is determined based on the predicted value of the template and the reconstructed value of the template, thereby obtaining a cost corresponding to each of the new combinations of prediction modes formed. Similarly, a current block is predicted using the weight derivation mode of the current block and the K initial prediction modes to determine a cost corresponding to the K initial prediction modes. In this way, the encoding side can obtain K adjusted prediction modes based on the cost corresponding to the new combination of prediction modes and the costs corresponding to the K initial prediction modes. For example, if the cost corresponding to the new combination 1 of prediction modes is the smallest, the K prediction modes included in the new combination 1 of prediction modes are determined as the K adjusted prediction modes. The new combination 1 of prediction modes may include at least one initial prediction mode, or may not include any initial prediction modes.

[0551] Example 2: Through the following steps S203-A11 to S203-A14, K initial prediction modes are improved to obtain K adjusted prediction modes.

[0552] S203-A11: Determine a first template weight based on the weight derivation mode of the current block.

[0553] For a specific process of determining the first template weight based on the weight derivation mode of the current block, reference may be made to the specific description of the embodiment of the template weight derivation below, which will not be described in detail here.

[0554] S203-A12: For a j-th initial prediction mode among the K initial prediction modes, determine M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer greater than 1, and j is a positive integer from 1 to K.

[0555] In addition, there is no order in which S203-A12 and S203-A11 are executed. That is, S203-A12 may be executed after S203-A11, may be executed before S203-A11, or may be executed simultaneously with S203-A11, and the embodiment of the present application is not limited thereto.

[0556] In this example, the encoding side improves each of the K initial prediction modes using the weight derivation mode of the current block to obtain an adjusted prediction mode corresponding to each of the K initial prediction modes, i.e., obtains K adjusted prediction modes.

[0557] The process of determining the adjusted prediction mode corresponding to each of the K initial prediction modes is the same. For ease of description, the embodiment of the present application takes as an example determining the adjusted prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes.

[0558] In an embodiment of the present application, for a j-th initial prediction mode, the encoding side first determines M first candidate prediction modes corresponding to the j-th initial prediction mode.

[0559] The manner in which the encoding side determines the M first candidate prediction modes corresponding to the j-th initial prediction mode includes the following two cases.

[0560] Case 1: When the jth initial prediction mode is an intra prediction mode, M intra prediction modes similar to the jth initial prediction mode are determined as M first candidate prediction modes. For example, when the jth initial prediction mode is an angular prediction mode, M angular prediction modes similar (or similar) to the angle of the jth initial prediction mode are determined as M first candidate prediction modes corresponding to the jth initial prediction mode.

[0561] Case 2: When the j-th initial prediction mode is an inter prediction mode, a search is performed based on the j-th initial prediction mode to obtain M pieces of first motion information. Based on these M pieces of first motion information, M pieces of first candidate prediction modes corresponding to the j-th prediction mode are determined. For example, for the j-th initial prediction mode, a motion vector is (xInit, yInit), and a search range such as a rectangular area is set from xInit-sR to xInit+sR in the horizontal direction and from yInit-sR to yInit+sR in the vertical direction, where sR may be 2, 4, 8, etc. Each motion vector in the rectangular area is combined with other information of the j-th initial prediction mode, such as a reference image index and a prediction list flag, to determine the motion information, and further determine the prediction mode. As a result, M pieces of first candidate prediction modes corresponding to the j-th initial prediction mode can be obtained.

[0562] In some embodiments, performing a search based on the jth initial prediction mode to obtain M pieces of first motion information includes performing a search based on the jth initial prediction mode to obtain M offsets, and obtaining the M pieces of first motion information based on the M offsets.

[0563] S203-A13: Determine second costs corresponding to predictions to the template by each of the M first candidate prediction modes based on the first template weights.

[0564] According to the method of the above case 1 or case 2, after determining M first candidate prediction modes corresponding to the j-th initial prediction mode, determine costs corresponding to the M first candidate prediction modes, and then determine an adjusted prediction mode corresponding to the j-th initial prediction mode based on the costs. For convenience of description, the cost corresponding to the first candidate prediction mode is denoted as the second cost.

[0565] Specifically, for each of the M first candidate prediction modes, predict a template of a current block using the first candidate prediction mode to obtain a predicted value, and then determine a second cost for the template corresponding to the first candidate prediction mode according to the first template weight and the predicted value.

[0566] Hereinafter, an example will be described in which the second cost for a template corresponding to one of the M first candidate prediction modes is calculated.

[0567] For convenience of explanation, the i-th sample in the template is taken as an example. The i-th sample can be understood as any sample in the template. That is, the process of determining the second cost of each sample in the template is the same, and it is sufficient to refer to the i-th sample. Specifically, the template is predicted using a first candidate prediction mode to obtain a predicted value of the template. The predicted value corresponding to the i-th sample in the predicted value of the template is written as the i-th predicted value, and the reconstructed value corresponding to the i-th sample in the reconstructed value of the template is written as the i-th reconstructed value. The second cost of the i-th sample corresponding to the first candidate prediction mode is determined based on the i-th predicted value, the i-th reconstructed value, and the weight of the i-th sample in the first template weight. Based on the above method, the second cost of each sample or multiple samples in the template corresponding to the first candidate prediction mode is determined. Then, the second cost for the template corresponding to the first candidate prediction mode is determined based on the second cost of each sample or multiple samples in the template. For example, the sum of the second costs of the samples in the template corresponding to the first candidate prediction mode is determined as the second cost for the template corresponding to the first candidate prediction mode.

[0568] Illustratively, taking the SAD cost as an example, the second cost of the i-th sample (x, y) in the template corresponding to the first candidate prediction mode can be determined based on Equation 1 above.

[0569] However, the above is taken as an example of determining the second cost for the template corresponding to the first candidate prediction mode according to SAD.Optionally, the second cost for the template corresponding to the first candidate prediction mode can also be determined according to cost calculation methods such as SATD, MSE, etc.

[0570] Through the above method, the second costs corresponding to each of the M first candidate prediction modes can be determined, and then the following S203-A14 are performed.

[0571] S203-A14: Determine an adjusted prediction mode corresponding to the j-th initial prediction mode based on the second costs corresponding to each of the M first candidate prediction modes.

[0572] For example, among the M first candidate prediction modes, the first candidate prediction mode having the smallest second cost is determined as the adjusted prediction mode corresponding to the j-th initial prediction mode.

[0573] In the above embodiment, an example is described of determining an adjusted prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes. However, by referring to the above description, it is possible to determine an adjusted prediction mode corresponding to each of the K initial prediction modes and obtain K adjusted prediction modes.

[0574] In the method 1, the decoding side first determines a weight derivation mode of the current block based on the K initial prediction modes, then improves the K initial prediction modes based on the weight derivation mode of the current block, for example, by improving according to the method of the above example 1 or example 2, to obtain K adjusted prediction modes, and then predicts the current block using the weight derivation mode of the current block and the K adjusted prediction modes to obtain a predicted value of the current block.

[0575] In an embodiment of the present application, in addition to using the above method 1 to determine the weight derivation mode and K adjusted prediction modes of the current block, the encoding side can also determine the weight derivation mode and K adjusted prediction modes of the current block by the following method 2.

[0576] Method 2: Improving prediction mode and determining weight derivation mode of current block are combined and performed simultaneously. In this case, determining a first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and K initial prediction modes in the above S202-B includes the following steps S202-B11 to S202-B12.

[0577] S202-B11: Improve K initial prediction modes based on the i-th candidate weight derivation mode to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode.

[0578] In the second method, for an i-th candidate weight derivation mode among N candidate prediction modes, first, improve K initial prediction modes using the i-th candidate weight derivation mode to obtain K improved prediction modes corresponding to the i-th candidate weight derivation mode. For convenience of explanation, the K improved prediction modes corresponding to the i-th candidate weight derivation mode are referred to as K first improved prediction modes. Then, a first cost corresponding to the i-th candidate weight derivation mode is determined based on the i-th candidate weight derivation mode and the K first improved prediction modes corresponding to the i-th candidate weight derivation mode. Thus, a more accurate weight derivation mode of the current block and K adjusted prediction modes can be obtained.

[0579] Examples of S202-B11 include, but are not limited to, the following examples:

[0580] Example 1: determining at least one prediction mode for an i-th candidate weight derivation mode, the at least one prediction mode may be a prediction mode with a prediction angle parallel or nearly parallel to a boundary surface of the i-th candidate weight derivation mode, or a prediction mode with a prediction angle perpendicular or nearly perpendicular to a boundary surface of the i-th candidate weight derivation mode. At least one initial prediction mode among the K initial prediction modes is replaced with a prediction mode among the at least one prediction mode for the i-th candidate weight derivation mode, forming a new combination of prediction modes. Each of the new combinations of prediction modes includes K prediction modes. For each of the new combinations of prediction modes, a template is predicted using the K prediction modes included in the new combination of prediction modes and the i-th candidate weight derivation mode to obtain a predicted value of the template. A cost corresponding to the new combination of prediction modes is determined based on the predicted value of the template and the reconstructed value of the template, thereby obtaining a cost corresponding to each of the new combinations of prediction modes formed. Similarly, a current block is predicted using the i-th candidate weight derivation mode and the K initial prediction modes to determine a cost corresponding to the K initial prediction modes. In this way, the encoding side can obtain K first improved prediction modes based on the cost corresponding to the new combination of prediction modes and the costs corresponding to the K initial prediction modes. For example, if the cost corresponding to the new combination 1 of prediction modes is the smallest, the K prediction modes included in the new combination 1 of prediction modes are determined as the K first improved prediction modes. The new combination 1 of prediction modes may include at least one initial prediction mode, or may not include any initial prediction modes.

[0581] Example 2: Through the following steps S202-B11-1 to S202-B11-4, K initial prediction modes are improved to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode.

[0582] S202-B11-1: Determine a second template weight based on the i-th candidate weight derivation mode.

[0583] The specific process of determining the second template weight based on the i-th candidate weight derivation mode may refer to the specific description of the embodiment of template weight derivation below, and will not be described in detail here.

[0584] S202-B11-2: For a j-th initial prediction mode among the K initial prediction modes, determine M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer and j is a positive integer from 1 to K.

[0585] It should be noted that there is no order in which S202-B11-2 and S202-B11-1 are executed. That is, S202-B11-2 may be executed after S202-B11-1, before S202-B11-1, or simultaneously with S202-B11-1, and the embodiment of the present application is not limited thereto.

[0586] In this example, the encoding side improves each of the K initial prediction modes using the i-th candidate weight derivation mode to obtain first improved prediction modes corresponding to each of the K initial prediction modes in the i-th candidate weight derivation mode, i.e., obtain K first improved prediction modes.

[0587] The process of determining the first improved prediction mode corresponding to each of the K initial prediction modes is the same. For ease of explanation, the embodiment of the present application takes as an example determining the first improved prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes.

[0588] In an embodiment of the present application, for a j-th initial prediction mode, the encoding side first determines M first candidate prediction modes corresponding to the j-th initial prediction mode.

[0589] The manner in which the encoding side determines the M first candidate prediction modes corresponding to the j-th initial prediction mode includes the following two cases.

[0590] Case 1: if the j-th initial prediction mode is an intra prediction mode, determine M intra prediction modes similar to the j-th initial prediction mode as M first candidate prediction modes.

[0591] Case 2: if the jth initial prediction mode is an inter prediction mode, perform a search based on the jth initial prediction mode to obtain M pieces of first motion information, and determine M first candidate prediction modes corresponding to the jth prediction mode based on the M pieces of first motion information.

[0592] In some embodiments, performing a search based on the jth initial prediction mode to obtain M pieces of first motion information includes performing a search based on the jth initial prediction mode to obtain M offsets, and obtaining the M pieces of first motion information based on the M offsets.

[0593] S202-B11-3: Determine a third cost corresponding to prediction to the template by each of the M first candidate prediction modes based on the second template weight.

[0594] According to the method of the above case 1 or case 2, after determining M first candidate prediction modes corresponding to the j-th initial prediction mode, determine costs corresponding to the M first candidate prediction modes, and then determine a first improved prediction mode corresponding to the j-th initial prediction mode based on the costs. For convenience of explanation, the cost corresponding to the first candidate prediction mode in this case is denoted as a third cost.

[0595] Specifically, for each of the M first candidate prediction modes, predict a template of a current block using the first candidate prediction mode to obtain a predicted value, and then determine a third cost for the template corresponding to the first candidate prediction mode according to the second template weight and the predicted value.

[0596] Hereinafter, an example will be described in which the third cost for a template corresponding to one of the M first candidate prediction modes is calculated.

[0597] For convenience of explanation, the i-th sample in the template is taken as an example. The i-th sample can be understood as any sample in the template. That is, the process of determining the third cost of each sample in the template is the same, and it is sufficient to refer to the i-th sample. Specifically, the template is predicted using a first candidate prediction mode to obtain a predicted value of the template. The predicted value corresponding to the i-th sample in the predicted value of the template is written as the i-th predicted value, and the reconstructed value corresponding to the i-th sample in the reconstructed value of the template is written as the i-th reconstructed value. The third cost of the i-th sample corresponding to the first candidate prediction mode is determined based on the i-th predicted value, the i-th reconstructed value, and the weight of the i-th sample in the second template weight. Based on the above method, the third cost of each sample or multiple samples in the template corresponding to the first candidate prediction mode is determined. Then, the third cost for the template corresponding to the first candidate prediction mode is determined based on the third cost of each sample or multiple samples in the template. For example, the sum of the third costs of the samples in the template corresponding to the first candidate prediction mode is determined as the third cost for the template corresponding to the first candidate prediction mode.

[0598] For example, a third cost for the template corresponding to the first candidate prediction mode can be determined according to the above Equation 1 and Equation 2. Specifically, wTemplateValue[x][y] in the above Equation 1 is replaced with the corresponding weight value of sample (x, y) in the second template weight.

[0599] Through the above method, the third cost corresponding to each of the M first candidate prediction modes can be determined, and then the following S202-B11-4 is performed.

[0600] S202-B11-4: Determine an improved prediction mode corresponding to the j-th initial prediction mode based on the third costs corresponding to each of the M first candidate prediction modes.

[0601] For example, among the M first candidate prediction modes, the first candidate prediction mode having the smallest third cost is determined as the first improved prediction mode corresponding to the j-th initial prediction mode.

[0602] In the above embodiment, an example is described in which a first improved prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes is determined. However, by referring to the above description, it is possible to determine a first improved prediction mode corresponding to each of the K initial prediction modes and obtain K first improved prediction modes in the i-th candidate weight derivation mode.

[0603] S202-B12: Determine a first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and the K first refinement prediction modes.

[0604] Exemplarily, a template of a current block is predicted using the i-th candidate weight derivation mode and K first improved prediction modes to obtain a second predicted value of the template. A first cost corresponding to the i-th weight derivation mode is determined based on the second predicted value of the template and the reconstructed value of the template. For example, a second template weight is determined based on the i-th candidate weight derivation mode, and a template is predicted using K first improved prediction modes to obtain K predicted values. The second template weight is used to weight the K predicted values ​​to obtain a second predicted value of the template. A first cost corresponding to the i-th weight derivation mode is determined based on the second predicted value of the template and the reconstructed value of the template.

[0605] The above is taken as an example of determining the first cost corresponding to the i-th candidate weight derivation mode. For each of the N candidate weight derivation modes, the first cost corresponding to each of the N candidate weight derivation modes can be determined by using the method of determining the first cost corresponding to the i-th candidate weight derivation mode, and then the weight derivation mode of the current block is determined according to the first costs corresponding to each of the N candidate weight derivation modes.

[0606] For example, among the N candidate weight derivation modes, the candidate weight derivation mode with the smallest first cost is determined as the weight derivation mode of the current block.

[0607] As another example, the N candidate weight derivation modes are sorted based on a first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes, and a weight derivation mode for the current block is determined based on a weight derivation mode index in the list of candidate weight derivation modes.

[0608] In the manner 2, K first improved prediction modes corresponding to the N candidate weight derivation modes are determined. Based on this, determining K adjusted prediction modes based on the weight derivation mode of the current block in the above S203-A includes the following steps:

[0609] S203-A2: Determine the K first refinement prediction modes corresponding to the weight derivation mode of the current block as the K adjusted prediction modes.

[0610] In the method 2, the prediction mode improvement and the determination of the weight derivation mode of the current block are combined, that is, while determining the first cost corresponding to the N candidate weight derivation modes, K first improved prediction modes corresponding to each candidate weight derivation mode are determined. Therefore, after determining the weight derivation mode of the current block based on the first cost, the K first improved prediction modes corresponding to the weight derivation mode of the current block can be determined as K adjusted prediction modes. Then, the current block can be predicted using the weight derivation mode of the current block and these K adjusted prediction modes to obtain a predicted value of the current block.

[0611] In the second method, for each of the N candidate weight derivation modes, K first improved prediction modes corresponding to each candidate weight derivation mode are determined. Then, a template is predicted according to each candidate weight derivation mode and the corresponding K first improved prediction modes, and a first cost is determined. Based on the first cost, the weight derivation mode of the current block can be more accurately determined from the N candidate weight derivation modes.

[0612] In an embodiment of the present application, in addition to using the above-mentioned method 1 or method 2 to determine the weight derivation mode of the current block and K adjusted prediction modes, the encoding side can also determine the weight derivation mode of the current block and K adjusted prediction modes by the following method 3.

[0613] Method 3: In order to reduce the computational complexity, during the initial prediction mode improvement, instead of using each candidate weight derivation mode to improve the initial prediction mode, use simple P sub-templates, for example, use the full template, left template, and upper template of the current block to improve the initial prediction mode.Therefore, the computational complexity of improving the initial prediction mode is reduced.In this case, determining the first cost corresponding to the i-th candidate weight derivation mode according to the i-th candidate weight derivation mode and K initial prediction modes in S202-B includes the following steps S202-B21 to S202-B23.

[0614] S202-B21: For a j-th initial prediction mode among the K initial prediction modes, determine a second refinement prediction mode for the P sub-templates corresponding to the j-th initial prediction mode, where P is a positive integer and j is a positive integer from 1 to K.

[0615] In this method 3, the encoding side improves K initial prediction modes using P sub-templates to obtain second improved prediction modes for P sub-templates corresponding to each of the K initial prediction modes. For each of the N candidate weight derivation modes, based on the correspondence between the candidate weight derivation modes and the P sub-templates, obtain K second improved prediction modes corresponding to the candidate weight derivation modes from the second improved prediction modes for P sub-templates corresponding to each of the K initial prediction modes. Next, based on the candidate weight derivation modes and the corresponding K second improved prediction modes, predict the template of the current block to determine the first cost corresponding to the candidate weight derivation mode. Finally, based on the first cost corresponding to each of the N candidate weight derivation modes, determine the weight derivation mode of the current block. In method 3, the candidate prediction modes are not used alone to improve the K initial prediction modes, and the complexity of improving the initial prediction modes is reduced.

[0616] Exemplarily, the P sub-templates include at least one of a left template TM_L, an upper template TM_A, and a full template TM_AL of the current block, where the full template TM_AL includes a left template TM_L and an upper template TM_A.

[0617] In the embodiment of the present application, the process in which the encoding side determines the second improved prediction mode for the P sub-templates corresponding to each of the K initial prediction modes is the same. For ease of explanation, the embodiment of the present application takes as an example the determination of the second improved prediction mode for the P sub-templates corresponding to the j-th initial prediction mode of the K initial prediction modes.

[0618] In one example, assuming that the P sub-templates include a left template TM_L, an upper template TM_A, and a full template TM_AL of a current block, determining a second refinement prediction mode for the P sub-templates corresponding to the jth initial prediction mode includes: determining a second refinement prediction mode for the left template TM_L corresponding to the jth initial prediction mode, determining a second refinement prediction mode for the upper template TM_A corresponding to the jth initial prediction mode, and determining a second refinement prediction mode for the full template TM_AL corresponding to the jth initial prediction mode. That is, in this step, the encoding side determines a second refinement prediction mode for each of the P sub-templates corresponding to the jth initial prediction mode.

[0619] The embodiments of the present application do not limit the specific format of determining the second refinement prediction mode for the P sub-templates corresponding to the j-th initial prediction mode.

[0620] In some embodiments, S202-B21 includes the following steps S202-B21-1 to S202-B21-3.

[0621] S202-B21-1: Determine M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer greater than 1.

[0622] The manner in which the encoding side determines the M first candidate prediction modes corresponding to the j-th initial prediction mode includes the following two cases.

[0623] Case 1: if the j-th initial prediction mode is an intra prediction mode, determine M intra prediction modes similar to the j-th initial prediction mode as M first candidate prediction modes.

[0624] Case 2: if the jth initial prediction mode is an inter prediction mode, perform a search based on the jth initial prediction mode to obtain M pieces of first motion information, and determine M first candidate prediction modes corresponding to the jth prediction mode based on the M pieces of first motion information.

[0625] In some embodiments, performing a search based on the jth initial prediction mode to obtain M pieces of first motion information includes performing a search based on the ith initial prediction mode to obtain M offsets, and obtaining the M pieces of first motion information based on the M offsets.

[0626] S202-B21-2: For the p-th sub-template of the P sub-templates, determine a fourth cost corresponding to prediction to the p-th sub-template by each of the M first candidate prediction modes, where p is a positive integer from 1 to P.

[0627] The encoding side determines M first candidate prediction modes corresponding to the j-th initial prediction mode according to the above method of Case 1 or Case 2, and then determines costs for each of the P sub-templates corresponding to the M first candidate prediction modes, and then determines a second improved prediction mode for each of the P sub-templates corresponding to the j-th initial prediction mode according to the costs. For convenience of explanation, the cost corresponding to the first candidate prediction mode in this case is denoted as a fourth cost.

[0628] Specifically, take the p-th sub-template of the P sub-templates as an example, for any first candidate prediction mode of the M first candidate prediction modes, predict the p-th sub-template using the first candidate prediction mode to obtain a predicted value of the p-th sub-template. Then,...

Claims

1. A prediction method applied in a decoder, comprising: Decoding the bitstream to determine K initial prediction modes, where K is a positive integer greater than 1; determining a weight derivation mode of a current block according to the K initial prediction modes; determining a predicted value of the current block according to a weight derivation mode of the current block; Including, A prediction method comprising:

2. determining a predicted value of the current block according to a weight derivation mode of the current block, determining K adjusted prediction modes based on the weight derivation mode of the current block; determining a prediction value of the current block based on the K adjusted prediction modes and a weight derivation mode of the current block; Including, 2. The method of claim 1 .

3. determining a weight derivation mode of a current block based on the K initial prediction modes, determining N candidate weight derivation modes, where N is a positive integer greater than 1; For an i-th candidate weight derivation mode among the N candidate weight derivation modes, determine a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K initial prediction modes, where i is a positive integer from 1 to N; determining a weight derivation mode for the current block based on a first cost corresponding to each of the N candidate weight derivation modes; Including, 3. The method of claim 2 .

4. determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K initial prediction modes, predicting a template of the current block using the i-th candidate weight derivation mode and the K initial prediction modes to obtain a first predicted value of the template; determining a first cost corresponding to the i-th weight derivation mode based on a first predicted value of the template and a reconstructed value of the template; Including, 4. The method according to claim 3 .

5. determining K adjusted prediction modes based on a weight derivation mode of the current block, improving the K initial prediction modes based on a weight derivation mode of the current block to obtain the K adjusted prediction modes.

5. The method of claim 4.

6. Improving the K initial prediction modes based on a weight derivation mode of the current block to obtain the K adjusted prediction modes, determining a first template weight based on a weight derivation mode of the current block; For a j-th initial prediction mode among the K initial prediction modes, determining M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer greater than 1 and j is a positive integer from 1 to K; determining a second cost corresponding to a prediction to the template by each of the M first candidate prediction modes based on the first template weights; determining an adjusted prediction mode corresponding to the jth initial prediction mode based on second costs corresponding to each of the M first candidate prediction modes; Including, 6. The method of claim 5 .

7. Determining an adjusted prediction mode corresponding to the jth initial prediction mode based on second costs corresponding to each of the M first candidate prediction modes includes: determining a first candidate prediction mode among the M first candidate prediction modes, the first candidate prediction mode having the smallest second cost, as an adjusted prediction mode corresponding to the jth initial prediction mode; 7. The method of claim 6.

8. determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K initial prediction modes, improving the K initial prediction modes based on the i-th candidate weight derivation mode to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode; determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K first refinement prediction modes; Including, 4. The method according to claim 3 .

9. Improving the K initial prediction modes based on the i-th candidate weight derivation mode to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode, determining a second template weight based on the i-th candidate weight derivation mode; determining, for a j-th initial prediction mode among the K initial prediction modes, M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer and j is a positive integer from 1 to K; determining a third cost corresponding to a prediction to the template by each of the M first candidate prediction modes based on the second template weights; determining a first refinement prediction mode corresponding to the j-th initial prediction mode based on a third cost corresponding to each of the M first candidate prediction modes; Including, 9. The method of claim 8.

10. determining a first refinement prediction mode corresponding to the j-th initial prediction mode based on a third cost corresponding to each of the M first candidate prediction modes, determining a first candidate prediction mode having a smallest third cost among the M first candidate prediction modes as a first improved prediction mode corresponding to the j-th initial prediction mode; 10. The method of claim 9.

11. Determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K first refinement prediction modes includes: predicting a template of the current block using the i-th candidate weight derivation mode and the K first refinement prediction modes to obtain a second predicted value of the template; determining a first cost corresponding to the i-th weight derivation mode based on a second predicted value of the template and a reconstructed value of the template; Including, 9. The method of claim 8.

12. determining K adjusted prediction modes based on a weight derivation mode of the current block, determining K first refinement prediction modes corresponding to a weight derivation mode of the current block as the K adjusted prediction modes; 9. The method of claim 8.

13. determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K initial prediction modes, determining, for a j-th initial prediction mode among the K initial prediction modes, a second refinement prediction mode for P sub-templates corresponding to the j-th initial prediction mode, where P is a positive integer and j is a positive integer from 1 to K; determining K second refinement prediction modes corresponding to the i-th candidate weight derivation mode from second refinement prediction modes for the P sub-templates corresponding to each of the K initial prediction modes based on the i-th candidate weight derivation mode; determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K second refinement prediction modes; Including, 4. The method according to claim 3 .

14. determining K second refinement prediction modes corresponding to the i-th candidate weight derivation mode from second refinement prediction modes for the P sub-templates corresponding to each of the K initial prediction modes based on the i-th candidate weight derivation mode, determining a division angle corresponding to the i-th candidate weight derivation mode; determining K second improved prediction modes corresponding to the i-th candidate weight derivation mode from second improved prediction modes for the P sub-templates corresponding to each of the K initial prediction modes based on a correspondence relationship between the division angle and a template corresponding to a prediction mode; Including, 14. The method of claim 13.

15. The P sub-templates include at least one of a left template, an upper template, and a full template of the current block.

14. The method of claim 13.

16. Determining a second refinement prediction mode for the P sub-templates corresponding to the jth initial prediction mode includes: determining M first candidate prediction modes corresponding to the jth initial prediction mode, where M is a positive integer greater than 1; determining, for a p-th sub-template among the P sub-templates, a fourth cost corresponding to prediction to the p-th sub-template by each of the M first candidate prediction modes, where p is a positive integer from 1 to P; determining a second refinement prediction mode for the p-th sub-template corresponding to the j-th initial prediction mode based on a fourth cost for the p-th sub-template corresponding to each of the M first candidate prediction modes; Including, 16. The method of claim 15 .

17. determining a second refinement prediction mode for the p-th sub-template corresponding to the j-th initial prediction mode based on a fourth cost for the p-th sub-template corresponding to each of the M first candidate prediction modes, comprising: determining a first candidate prediction mode among the M first candidate prediction modes, the first candidate prediction mode having a smallest fourth cost for the p-th sub-template, as a second improved prediction mode for the p-th sub-template corresponding to the j-th initial prediction mode; 17. The method of claim 16.

18. Determining a fourth cost corresponding to prediction onto the p-th sub-template by each of the M first candidate prediction modes includes: predicting the p-th sub-template using a first candidate prediction mode among the M first candidate prediction modes to obtain a predicted value of the p-th sub-template; determining a fourth cost for the pth sub-template corresponding to the first candidate prediction mode based on the predicted value of the pth sub-template and the reconstructed value of the pth sub-template; Including, 17. The method of claim 16.

19. Determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K second refinement prediction modes includes: predicting a template of the current block using the i-th candidate weight derivation mode and the K second refinement prediction modes to obtain a third predicted value of the template; determining a first cost corresponding to the i-th weight derivation mode based on a third predicted value of the template and the reconstructed value of the template; Including, 14. The method of claim 13.

20. determining K adjusted prediction modes based on a weight derivation mode of the current block, determining the K adjusted prediction modes based on K second refinement prediction modes corresponding to a weight derivation mode of the current block; 14. The method of claim 13.

21. determining the K adjusted prediction modes based on K second refinement prediction modes corresponding to a weight derivation mode of the current block; determining K second refinement prediction modes corresponding to a weight derivation mode of the current block as the K adjusted prediction modes; 21. The method of claim 20.

22. determining the K adjusted prediction modes based on K second refinement prediction modes corresponding to a weight derivation mode of the current block; improving the K second refinement prediction modes based on a weight derivation mode of the current block to obtain the K adjusted prediction modes.

21. The method of claim 20.

23. Improving the K second improved prediction modes based on a weight derivation mode of the current block to obtain the K adjusted prediction modes, determining a first template weight based on a weight derivation mode of the current block; For a k-th second improved prediction mode among the K second improved prediction modes, determining P second candidate prediction modes corresponding to the k-th second improved prediction mode, where P is a positive integer greater than 1 and k is a positive integer from 1 to K; determining a fifth cost corresponding to a prediction to the template by each of the P second candidate prediction modes based on the first template weights; determining an adjusted prediction mode corresponding to the kth second refinement prediction mode based on a fifth cost corresponding to each of the P second candidate prediction modes; Including, 23. The method of claim 22.

24. determining an adjusted prediction mode corresponding to the kth second refinement prediction mode based on a fifth cost corresponding to each of the P second candidate prediction modes, determining a second candidate prediction mode having the smallest fifth cost among the P second candidate prediction modes as an adjusted prediction mode corresponding to the kth second improved prediction mode; 24. The method of claim 23.

25. When the j-th initial prediction mode is an intra prediction mode, determining M first candidate prediction modes corresponding to the j-th initial prediction mode includes: determining M intra prediction modes similar to the jth initial prediction mode as the M first candidate prediction modes; 17. The method according to claim 6, 10 or 16.

26. When the j-th initial prediction mode is an inter prediction mode, determining M first candidate prediction modes corresponding to the j-th initial prediction mode includes: performing a search based on the j-th initial prediction mode to obtain M pieces of first motion information; obtaining the M first candidate prediction modes based on the M first motion information; Including, 17. The method according to claim 6, 10 or 16.

27. Obtaining M pieces of first motion information by performing a search based on the j-th initial prediction mode, performing a search based on the i-th initial prediction mode to obtain M offsets; obtaining the M pieces of first motion information based on the M offsets; Including, 27. The method of claim 26.

28. When the k-th second refinement prediction mode is an intra prediction mode, determining P second candidate prediction modes corresponding to the k-th second refinement prediction mode includes: determining P intra prediction modes similar to the k-th second refinement prediction mode as the P second candidate prediction modes; Including, 24. The method of claim 23.

29. When the k-th second refinement prediction mode is an inter prediction mode, determining P second candidate prediction modes corresponding to the k-th second refinement prediction mode includes: performing a search based on the k-th second refinement prediction mode to obtain P pieces of second motion information; obtaining the P second candidate prediction modes based on the P second motion information; Including, 24. The method of claim 23.

30. performing a search based on the k-th second refinement prediction mode to obtain P pieces of second motion information; performing a search based on the kth second refinement prediction mode to obtain P offsets; obtaining P pieces of second motion information based on the P offsets; Including, 30. The method of claim 29.

31. determining a weight derivation mode for the current block based on costs corresponding to each of the N candidate weight derivation modes, determining a candidate weight derivation mode having the smallest first cost among the N candidate weight derivation modes as a weight derivation mode for the current block; 25. The method according to any one of claims 3 to 24.

32. determining a weight derivation mode for the current block based on a first cost corresponding to each of the N candidate weight derivation modes, sorting the N candidate weight derivation modes based on a first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes; decoding the bitstream to obtain a weight derivation mode index; determining a candidate weight derivation mode in the list of candidate weight derivation modes corresponding to the weight derivation mode index as a weight derivation mode for the current block; Including, 25. The method according to any one of claims 3 to 24.

33. Decoding the bitstream to determine K initial prediction modes includes: determining a list of selected prediction modes, the list of selected prediction modes including at least two selected prediction modes; decoding the bitstream to obtain a prediction mode index; determining the K initial prediction modes from the list of selected prediction modes based on the prediction mode index; Including, 25. The method according to any one of claims 1 to 24.

34. The method comprises: determining angle indexing and distance indexing based on the weight derivation mode; determining a template weight based on the angle index, the distance index, and a size of the template; When the weight derivation mode is a weight derivation mode of the current block, the template weight is a first template weight, and when the weight derivation mode is a weight derivation mode of an i-th candidate, the template weight is a second template weight.

24. The method of claim 6, 9 or 23.

35. determining the template weight based on the angle index, the distance index, and a size of the template; determining first parameters of samples in the template based on the angle index, the distance index, and a size of the template, the first parameters being used to determine weights; determining weights for samples in the template based on first parameters for the samples in the template; determining the template weights based on weights of samples in the template; Including, 35. The method of claim 34.

36. determining a predicted value of the current block based on the K adjusted prediction modes and a weight derivation mode of the current block, determining K predicted values ​​based on the K adjusted prediction modes; determining a weight of a prediction value according to a weight derivation mode of the current block; determining a prediction value of the current block based on the K prediction modes and the weight of the prediction value; Including, 36. The method of claim 35.

37. determining the template weight based on the angle index, the distance index, and a size of the template; determining weights of samples in a merged region consisting of the current block and the template based on the angle index, the distance index, a size of the template, and a size of the current block; determining the template weight based on a size of the template and weights of samples within the merged region; Including, 35. The method of claim 34.

38. determining a predicted value of the current block based on the K adjusted prediction modes and a weight derivation mode of the current block, determining K predicted values ​​based on the K adjusted prediction modes; determining a weight for a prediction value based on weights of samples within the merged region; determining a prediction value for the current block based on the K prediction values ​​and weights of the prediction values; Including, 38. The method of claim 37.

39. determining weights of the K predictors based on weights of samples in the merged region, determining weights of the K predictors based on weights corresponding to samples in the current block in the merged region; 39. The method of claim 38.

40. the template of the current block includes at least one of an upper decoded region, a left decoded region, and a top-left decoded region of the current block; 25. The method according to any one of claims 4 to 24.

41. 1. A method of prediction comprising: determining K initial prediction modes, where K is a positive integer greater than 1; determining a weight derivation mode of a current block according to the K initial prediction modes; determining a predicted value of the current block according to a weight derivation mode of the current block; Including, A prediction method comprising:

42. determining a predicted value of the current block according to a weight derivation mode of the current block, determining K adjusted prediction modes based on the weight derivation mode of the current block; determining a prediction value of the current block based on the K adjusted prediction modes and a weight derivation mode of the current block; Including, 42. The method of claim 41 .

43. determining a weight derivation mode for the current block based on the K initial prediction modes, determining N candidate weight derivation modes, where N is a positive integer greater than 1; For an i-th candidate weight derivation mode among the N candidate weight derivation modes, determine a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K initial prediction modes, where i is a positive integer from 1 to N; determining a weight derivation mode for the current block based on a first cost corresponding to each of the N candidate weight derivation modes; Including, 43. The method of claim 42.

44. determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K initial prediction modes, predicting a template of the current block using the i-th candidate weight derivation mode and the K initial prediction modes to obtain a first predicted value of the template; determining a first cost corresponding to the i-th weight derivation mode based on a first predicted value of the template and a reconstructed value of the template; Including, 44. The method of claim 43.

45. determining K adjusted prediction modes based on a weight derivation mode of the current block, improving the K initial prediction modes based on a weight derivation mode of the current block to obtain the K adjusted prediction modes.

45. The method of claim 44.

46. Improving the K initial prediction modes based on a weight derivation mode of the current block to obtain the K adjusted prediction modes, determining a first template weight based on a weight derivation mode of the current block; For a j-th initial prediction mode among the K initial prediction modes, determining M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer greater than 1 and j is a positive integer from 1 to K; determining a second cost corresponding to a prediction to the template by each of the M first candidate prediction modes based on the first template weights; determining an adjusted prediction mode corresponding to the jth initial prediction mode based on second costs corresponding to each of the M first candidate prediction modes; Including, 46. ​​The method of claim 45.

47. Determining an adjusted prediction mode corresponding to the jth initial prediction mode based on second costs corresponding to each of the M first candidate prediction modes includes: determining a first candidate prediction mode among the M first candidate prediction modes, the first candidate prediction mode having the smallest second cost, as an adjusted prediction mode corresponding to the jth initial prediction mode; 47. The method of claim 46.

48. determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K initial prediction modes, improving the K initial prediction modes based on the i-th candidate weight derivation mode to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode; determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K first refinement prediction modes; Including, 44. The method of claim 43.

49. Improving the K initial prediction modes based on the i-th candidate weight derivation mode to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode, determining a second template weight based on the i-th candidate weight derivation mode; determining, for a j-th initial prediction mode among the K initial prediction modes, M first candidate prediction modes corresponding to the j-th initial prediction mode, where M is a positive integer and j is a positive integer from 1 to K; determining a third cost corresponding to a prediction to the template by each of the M first candidate prediction modes based on the second template weights; determining a first refinement prediction mode corresponding to the j-th initial prediction mode based on a third cost corresponding to each of the M first candidate prediction modes; Including, 49. The method of claim 48.

50. determining a first refinement prediction mode corresponding to the j-th initial prediction mode based on a third cost corresponding to each of the M first candidate prediction modes, determining a first candidate prediction mode having a smallest third cost among the M first candidate prediction modes as a first improved prediction mode corresponding to the j-th initial prediction mode; 50. The method of claim 49.

51. Determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K first refinement prediction modes includes: predicting a template of the current block using the i-th candidate weight derivation mode and the K first refinement prediction modes to obtain a second predicted value of the template; determining a first cost corresponding to the i-th weight derivation mode based on a second predicted value of the template and a reconstructed value of the template; Including, 48. The method of claim 47.

52. determining K adjusted prediction modes based on a weight derivation mode of the current block, determining K first refinement prediction modes corresponding to a weight derivation mode of the current block as the K adjusted prediction modes; 49. The method of claim 48.

53. determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K initial prediction modes, determining, for a j-th initial prediction mode among the K initial prediction modes, a second refinement prediction mode for P sub-templates corresponding to the j-th initial prediction mode, where P is a positive integer and j is a positive integer from 1 to K; determining K second refinement prediction modes corresponding to the i-th candidate weight derivation mode from second refinement prediction modes for the P sub-templates corresponding to each of the K initial prediction modes based on the i-th candidate weight derivation mode; determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K second refinement prediction modes; Including, 44. The method of claim 43.

54. determining K second refinement prediction modes corresponding to the i-th candidate weight derivation mode from second refinement prediction modes for the P sub-templates corresponding to each of the K initial prediction modes based on the i-th candidate weight derivation mode, determining a division angle corresponding to the i-th candidate weight derivation mode; determining K second improved prediction modes corresponding to the i-th candidate weight derivation mode from second improved prediction modes for the P sub-templates corresponding to each of the K initial prediction modes based on a correspondence relationship between the division angle and a template corresponding to a prediction mode; Including, 54. The method of claim 53.

55. The P sub-templates include at least one of a left template, an upper template, and a full template of the current block.

54. The method of claim 53.

56. Determining a second refinement prediction mode for the P sub-templates corresponding to the jth initial prediction mode includes: determining M first candidate prediction modes corresponding to the jth initial prediction mode, where M is a positive integer greater than 1; determining, for a p-th sub-template among the P sub-templates, a fourth cost corresponding to prediction to the p-th sub-template by each of the M first candidate prediction modes, where p is a positive integer from 1 to P; determining a second refinement prediction mode for the p-th sub-template corresponding to the j-th initial prediction mode based on a fourth cost for the p-th sub-template corresponding to each of the M first candidate prediction modes; Including, 56. The method of claim 55.

57. determining a second refinement prediction mode for the p-th sub-template corresponding to the j-th initial prediction mode based on a fourth cost for the p-th sub-template corresponding to each of the M first candidate prediction modes, comprising: determining a first candidate prediction mode among the M first candidate prediction modes, the first candidate prediction mode having a smallest fourth cost for the p-th sub-template, as a second improved prediction mode for the p-th sub-template corresponding to the j-th initial prediction mode; 57. The method of claim 56.

58. Determining a fourth cost corresponding to prediction onto the p-th sub-template by each of the M first candidate prediction modes includes: predicting the p-th sub-template using a first candidate prediction mode among the M first candidate prediction modes to obtain a predicted value of the p-th sub-template; determining a fourth cost for the pth sub-template corresponding to the first candidate prediction mode based on the predicted value of the pth sub-template and the reconstructed value of the pth sub-template; Including, 57. The method of claim 56.

59. Determining a first cost corresponding to the i-th candidate weight derivation mode based on the i-th candidate weight derivation mode and the K second refinement prediction modes includes: predicting a template of the current block using the i-th candidate weight derivation mode and the K second refinement prediction modes to obtain a third predicted value of the template; determining a first cost corresponding to the i-th weight derivation mode based on a third predicted value of the template and the reconstructed value of the template; Including, 54. The method of claim 53.

60. determining K adjusted prediction modes based on a weight derivation mode of the current block, determining the K adjusted prediction modes based on K second refinement prediction modes corresponding to a weight derivation mode of the current block; 54. The method of claim 53.

61. determining the K adjusted prediction modes based on K second refinement prediction modes corresponding to a weight derivation mode of the current block; determining K second refinement prediction modes corresponding to a weight derivation mode of the current block as the K adjusted prediction modes; 61. The method of claim 60.

62. determining the K adjusted prediction modes based on K second refinement prediction modes corresponding to a weight derivation mode of the current block; improving the K second refinement prediction modes based on a weight derivation mode of the current block to obtain the K adjusted prediction modes.

61. The method of claim 60.

63. Improving the K second improved prediction modes based on a weight derivation mode of the current block to obtain the K adjusted prediction modes, determining a first template weight based on a weight derivation mode of the current block; For a k-th second improved prediction mode among the K second improved prediction modes, determining P second candidate prediction modes corresponding to the k-th second improved prediction mode, where P is a positive integer greater than 1 and k is a positive integer from 1 to K; determining a fifth cost corresponding to a prediction to the template by each of the P second candidate prediction modes based on the first template weights; determining an adjusted prediction mode corresponding to the kth second refinement prediction mode based on a fifth cost corresponding to each of the P second candidate prediction modes; Including, 63. The method of claim 62.

64. determining an adjusted prediction mode corresponding to the kth second refinement prediction mode based on a fifth cost corresponding to each of the P second candidate prediction modes, determining a second candidate prediction mode having the smallest fifth cost among the P second candidate prediction modes as an adjusted prediction mode corresponding to the kth second improved prediction mode; 64. The method of claim 63.

65. When the j-th initial prediction mode is an intra prediction mode, determining M first candidate prediction modes corresponding to the j-th initial prediction mode includes: determining M intra prediction modes similar to the jth initial prediction mode as the M first candidate prediction modes; 57. The method of claim 46, 50 or 56.

66. When the j-th initial prediction mode is an inter prediction mode, determining M first candidate prediction modes corresponding to the j-th initial prediction mode includes: performing a search based on the j-th initial prediction mode to obtain M pieces of first motion information; obtaining the M first candidate prediction modes based on the M first motion information; Including, 57. The method of claim 46, 50 or 56.

67. Obtaining M pieces of first motion information by performing a search based on the j-th initial prediction mode, performing a search based on the i-th initial prediction mode to obtain M offsets; obtaining the M pieces of first motion information based on the M offsets; Including, 67. The method of claim 66.

68. When the k-th second refinement prediction mode is an intra prediction mode, determining P second candidate prediction modes corresponding to the k-th second refinement prediction mode includes: determining P intra prediction modes similar to the k-th second refinement prediction mode as the P second candidate prediction modes; Including, 64. The method of claim 63.

69. When the k-th second refinement prediction mode is an inter prediction mode, determining P second candidate prediction modes corresponding to the k-th second refinement prediction mode includes: performing a search based on the k-th second refinement prediction mode to obtain P pieces of second motion information; obtaining the P second candidate prediction modes based on the P second motion information; Including, 64. The method of claim 63.

70. performing a search based on the k-th second refinement prediction mode to obtain P pieces of second motion information; performing a search based on the kth second refinement prediction mode to obtain P offsets; obtaining P pieces of second motion information based on the P offsets; Including, 70. The method of claim 69.

71. determining a weight derivation mode for the current block based on costs corresponding to each of the N candidate weight derivation modes, determining a candidate weight derivation mode having the smallest first cost among the N candidate weight derivation modes as a weight derivation mode for the current block; 65. The method according to any one of claims 43 to 64.

72. determining a weight derivation mode for the current block based on a first cost corresponding to each of the N candidate weight derivation modes, sorting the N candidate weight derivation modes based on a first cost corresponding to each of the N candidate weight derivation modes to obtain a list of candidate weight derivation modes; determining a weight derivation mode for the current block based on the list of candidate weight derivation modes; Including, 65. The method according to any one of claims 43 to 64.

73. The method comprises: writing a weight derivation mode index corresponding to a weight derivation mode of the current block in the list of candidate weight derivation modes into a bitstream.

72. The method of claim 71 .

74. The method comprises: writing a prediction mode index to the bitstream; The prediction mode index is used to determine the K initial prediction modes.

42. The method of claim 41 .

75. The method comprises: determining angle indexing and distance indexing based on the weight derivation mode; determining a template weight based on the angle index, the distance index, and a size of the template; When the weight derivation mode is a weight derivation mode of the current block, the template weight is a first template weight, and when the weight derivation mode is a weight derivation mode of an i-th candidate, the template weight is a second template weight.

64. The method of claim 46, 49 or 63.

76. determining the template weight based on the angle index, the distance index, and a size of the template; determining first parameters of samples in the template based on the angle index, the distance index, and a size of the template, the first parameters being used to determine weights; determining weights for samples in the template based on first parameters for the samples in the template; determining the template weights based on weights of samples in the template; Including, 76. The method of claim 75.

77. determining a predicted value of the current block based on the K adjusted prediction modes and a weight derivation mode of the current block, determining K predicted values ​​based on the K adjusted prediction modes; determining a weight of a prediction value according to a weight derivation mode of the current block; determining a prediction value of the current block based on the K prediction modes and the weight of the prediction value; Including, 77. The method of claim 76.

78. determining the template weight based on the angle index, the distance index, and a size of the template; determining weights of samples in a merged region consisting of the current block and the template based on the angle index, the distance index, a size of the template, and a size of the current block; determining the template weight based on a size of the template and weights of samples within the merged region; Including, 76. The method of claim 75.

79. determining a predicted value of the current block based on the K adjusted prediction modes and a weight derivation mode of the current block, determining K predicted values ​​based on the K adjusted prediction modes; determining a weight for a prediction value based on weights of samples within the merged region; determining a prediction value for the current block based on the K prediction values ​​and weights of the prediction values; Including, 79. The method of claim 78.

80. determining weights of the K predictors based on weights of samples in the merged region, determining weights of the K predictors based on weights corresponding to samples in the current block in the merged region; 80. The method of claim 79.

81. the template includes at least one of an upper decoded region, a left decoded region, and a top-left decoded region of the current block. The method according to any one of claims 41 to 64.

82. A prediction device, comprising: A decoding unit, a determination unit, and a prediction unit, The decoding unit is configured to decode the bitstream to determine K initial prediction modes, where K is a positive integer greater than 1; The determining unit is configured to determine a weight derivation mode of a current block according to the K initial prediction modes; the prediction unit is configured to determine a prediction value of the current block based on a weight derivation mode of the current block; A prediction device comprising:

83. A prediction device, comprising: A first determination unit, a second determination unit, and a prediction unit, The first determining unit is configured to determine K initial prediction modes, where K is a positive integer greater than 1; The second determining unit is configured to determine a weight derivation mode of a current block based on the K initial prediction modes; the prediction unit is configured to determine a prediction value of the current block based on a weight derivation mode of the current block; A prediction device comprising:

84. An electronic device comprising a processor and a memory, the memory is configured to store a computer program; The processor is configured to implement the method according to any one of claims 1 to 40 or the method according to any one of claims 41 to 81 by calling and executing a computer program stored in the memory.

1. An electronic device comprising:

85. 1. A video encoding and decoding system, comprising: A video encoder and a video decoder, The video decoder is configured to implement a method according to any one of claims 1 to 40, The video encoder is configured to implement a method according to any one of claims 41 to 81. A video encoding / decoding system comprising:

86. 1. A computer-readable storage medium, comprising: the computer-readable storage medium is configured to store a computer program; The computer program causes a computer to carry out the method according to any one of claims 1 to 40 or the method according to any one of claims 41 to 81, A computer-readable storage medium comprising:

87. Produced according to the method of any one of claims 41 to 81, A bitstream comprising:

Citation Information

Patent Citations

  • Prediction block generation device, image coding device, image decoding device, and program

    JP2021027464A

  • Method, apparatus, and computer program for video coding

    JP2022521699A

  • Method and apparatus for video coding

    WO2020176865A1