VIDEO CODEC METHOD, DEVICE, APPARATUS, SYSTEM, AND STORAGE MEDIUM

Abstract

The present application provides a video codec method, apparatus, device, system, and storage medium, in which the present application sets a weight derivation mode and K prediction modes as one combination, and in this way, in the code stream, the weight derivation mode and K prediction modes used for the current block are indicated in the form of a combination, and at this time, the decoding side obtains a first index for indicating a first combination consisting of one weight derivation mode and K prediction modes by decoding the code stream, and determines a first weight derivation mode and K first prediction modes included in the first combination based on the first index, and predicts the current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block, and in this application, the first weight derivation mode and K first prediction modes corresponding to the current block are indicated in the form of a combination, which makes the syntax simpler, saves codewords, and does not require judgment of various situations, and further improves coding efficiency.

Description

[Technical field]

[0001] The present application relates to the technical field of video codec, and in particular to a video codec method, apparatus, device, system, and storage medium. [Background technology]

[0002] Digital video technology can be incorporated into various video devices, such as digital televisions, smartphones, computers, electronic readers, or video players, etc. With the development of video technology, the amount of data contained in video data is large, and in order to facilitate the transmission of video data, video devices implement video compression techniques to transmit or store video data more efficiently.

[0003] Since video has temporal or spatial redundancy, prediction can eliminate or reduce the redundancy in the video and improve compression efficiency. Conventionally, in order to improve prediction effect, a current block can be predicted using multiple prediction modes. However, when predicting a current block using multiple prediction modes, a lot of information needs to be transmitted in the code stream, which further increases the coding cost. Summary of the Invention [Problem to be solved by the invention]

[0004] The present embodiments provide a video codec method, device, apparatus, system, and storage medium that can reduce encoding costs. [Means for solving the problem]

[0005] According to a first aspect, the present application provides a video decoding method for use in a decoder, comprising: Decoding the codestream to obtain a first index, the first index being used to indicate a first combination of one weight derivation mode and K prediction modes, where K is a positive integer greater than 1; determining a first weight derivation mode and K first prediction modes included in the first combination based on the first index; predicting a current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block.

[0006] According to a second aspect, an embodiment of the present invention is a video encoding method, comprising: determining a first combination including a first weight derivation mode and K first prediction modes; predicting a current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block; and obtaining a codestream including a first index for indicating the first combination based on the predicted value of the current block.

[0007] According to a third aspect, the present application provides a video decoding device for performing the method of the first aspect or each implementation thereof, specifically the device includes functional units for performing the method of the first aspect or each implementation thereof.

[0008] According to a fourth aspect, the present application provides a video encoding device for performing the method of the second aspect or each implementation thereof, specifically the device comprising functional units for performing the method of the second aspect or each implementation thereof.

[0009] According to a fifth aspect, there is provided a video decoder including a processor and a memory, the memory adapted to store a computer program, and the processor adapted to retrieve and execute the computer program stored in the memory in order to perform the method of the first aspect or each implementation thereof.

[0010] According to a sixth aspect, there is provided a video encoder including a processor and a memory, the memory adapted to store a computer program, and the processor adapted to retrieve and execute the computer program stored in the memory in order to perform the method of the second aspect or each implementation thereof.

[0011] According to a seventh aspect, there is provided a video codec system including a video encoder and a video decoder, the video decoder being adapted to perform the method of the first aspect or any of its implementations, and the video encoder being adapted to perform the method of the second aspect or any of its implementations.

[0012] According to an eighth aspect, there is provided a chip for implementing the method according to any one of the first to second aspects or each of the implementations thereof. Specifically, the chip includes a processor for calling and executing a computer program from a memory in order to execute the method according to any one of the first to second aspects or each of the implementations thereof in a device to which the chip is attached.

[0013] According to a ninth aspect, there is provided a computer-readable storage medium used to store a computer program, the computer program causing a computer to execute a method according to any one of the first to second aspects or each of their realization forms.

[0014] According to a tenth aspect, there is provided a computer program product including computer program instructions that cause a computer to execute a method according to any one of the first to second aspects or each of their realization forms.

[0015] According to an eleventh aspect, there is provided a computer program which, when executed on a computer, causes the computer to execute a method according to any one of the first to second aspects or each of their realization modes.

[0016] According to a twelfth aspect, there is provided a codestream generated based on the method of the second aspect, optionally wherein the codestream includes a first index, the first index being used to indicate a first combination of one weight derivation mode and K prediction modes, where K is a positive integer greater than 1. Effect of the Invention

[0017] Based on the above technical solution, the present application provides a combination of a weight derivation mode and K prediction modes, and indicates the weight derivation mode and K prediction modes used for a current block in the code stream in the form of a combination; at this time, the decoding side obtains a first index for indicating a first combination consisting of one weight derivation mode and K prediction modes by decoding the code stream, and determines the first weight derivation mode and K first prediction modes included in the first combination based on the first index, and predicts the current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block; in the present application, the first weight derivation mode and K first prediction modes corresponding to the current block are indicated in the form of a combination, which makes the syntax simpler, saves codewords, and does not require judgment of various situations, and further improves coding efficiency. [Brief description of the drawings]

[0018] [Figure 1] 1 is a schematic block diagram of a video codec system according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a schematic block diagram of a video encoder according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic block diagram of a video decoder according to an embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram of weight assignment. [Diagram 5] FIG. 2 is a schematic diagram of weight assignment. [Figure 6A] FIG. 1 is a schematic diagram of inter prediction. [Figure 6B] FIG. 1 is a schematic diagram of weighted inter prediction. [Figure 7A] FIG. 2 is a schematic diagram of intra prediction. [Figure 7B] FIG. 1 is a schematic diagram of intra prediction. [Figure 8A-8I] FIG. 2 is a schematic diagram of intra prediction. [Figure 9] FIG. 2 is a schematic diagram of intra-prediction modes. [Figure 10] FIG. 2 is a schematic diagram of intra-prediction modes. [Figure 11] FIG. 2 is a schematic diagram of intra-prediction modes. [Figure 12] FIG. 1 is a schematic diagram of an MIP. [Figure 13] FIG. 1 is a schematic diagram of weighted prediction. [Figure 14] FIG. 1 is a schematic diagram of template matching. [Figure 15] 1 is a schematic diagram illustrating a flowchart of a video decoding method according to an embodiment of the present application; [Figure 16A] FIG. 2 is a schematic diagram of weight assignment. [Figure 16B] FIG. 13 is a schematic diagram of another weight assignment. [Figure 17] FIG. 2 is a schematic diagram of the derivation of template weights; [Figure 18] FIG. 2 is a schematic diagram of a template. [Figure 19] 1 is a schematic diagram illustrating a flowchart of a video encoding method according to an embodiment of the present application; [Figure 20] 1 is a schematic block diagram of a video decoding device according to an embodiment of the present application; [Figure 21] 1 is a schematic block diagram of a video encoding device according to an embodiment of the present application; [Figure 22] FIG. 1 is a schematic block diagram of an electronic device according to an embodiment of the present invention. [Figure 23] 1 is a schematic block diagram of a video codec system according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The present application can be used in the image codec field, the video codec field, the hardware video codec field, the dedicated circuit video codec field, the real-time video codec field, etc. For example, aspects of the present application can be combined with audio video coding standards (abbreviated as AVS), such as the H.264 / audio video coding (abbreviated as AVC) standard, the H.265 / high efficiency video coding (abbreviated as HEVC) standard, and the H.266 / versatile video coding (abbreviated as VVC) standard. Alternatively, aspects of the present application may operate in combination with other proprietary or industry standards, including ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also referred to as ISO / IEC MPEG-4 AVC), including Scalable Video Coding (SVC) and Multiview Video Coding (MVC) extensions. It should be understood that the present technology is not limited to any particular codec standard or technology.

[0020] For ease of understanding, a video codec system according to an embodiment of the present invention will be first introduced with reference to FIG.

[0021] FIG. 1 is a schematic block diagram of a video codec system according to an embodiment of the present application. It should be noted that FIG. 1 is merely an example, and the video codec system according to the embodiment of the present application includes, but is not limited to, that shown in FIG. 1. As shown in FIG. 1, the video codec system 100 includes an encoding device 110 and a decoding device 120. The encoding device is used to encode (may be understood as compressing) video data to generate a code stream, and transmit the code stream to the decoding device. The decoding device decodes the code stream encoded and generated by the encoding device to obtain decoded video data.

[0022] The encoding device 110 of the present embodiment may be understood as a device having a video encoding function, and the decoding device 120 may be understood as a device having a video decoding function, i.e., the encoding device 110 and the decoding device 120 of the present embodiment include a broader range of devices, such as smartphones, desktop personal computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, etc.

[0023] In some embodiments, encoding device 110 may transmit encoded video data (e.g., a codestream) to decoding device 120 over channel 130. Channel 130 may include one or more media and / or devices that may transmit the encoded video data from encoding device 110 to decoding device 120.

[0024] In one example, channel 130 includes one or more communication media through which encoding device 110 can directly transmit encoded video data in real time to decoding device 120. In this example, encoding device 110 can modulate the encoded video data according to a communication standard and transmit the modulated video data to decoding device 120. The communication medium can include a wireless communication medium, such as a radio frequency spectrum, and optionally, the communication medium can also include a wired communication medium, such as one or more physical transmission lines.

[0025] In another example, channel 130 includes a storage medium capable of storing the video data encoded by encoding device 110. The storage medium may include a variety of locally accessible data storage media, such as optical disks, DVDs, flash memory, etc. In this example, decoding device 120 may obtain the encoded video data from the storage medium.

[0026] In another example, channel 130 may include a storage server that can store video data encoded by encoding device 110. In this example, decoding device 120 can download the stored encoded video data from the storage server. Optionally, the storage server may store the encoded video data and transmit the encoded video data to decoding device 120, such as a web server (e.g., for a website), a File Transfer Protocol (FTP) server, etc.

[0027] In some embodiments, encoding device 110 includes a video encoder 112 and an output interface 113. Output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0028] In some embodiments, encoding device 110 may further include a video source 111 in addition to video encoder 112 and output interface 113 .

[0029] The video source 111 may include at least one of a video collection device (e.g., a video camera), a video archive, a video input interface used to receive video data from a video content provider, and a computer graphics system used to generate the video data.

[0030] The video encoder 112 encodes video data from the video source 111 to generate a codestream. The video data may include one or more pictures or a sequence of pictures. The codestream contains coding information for a picture or a sequence of pictures in the form of a bitstream. The coding information may include coded image data and associated data. The associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures. An SPS may contain parameters for one or more sequences. A PPS may contain parameters for one or more pictures. A syntax structure is a set of zero or more syntax elements arranged in a specified order in the codestream.

[0031] The video encoder 112 directly transmits the encoded video data to the decoding device 120 via the output interface 113. The encoded video data may be further stored in a storage medium or a storage server for subsequent reading by the decoding device 120.

[0032] In some embodiments, decoding device 120 includes an input interface 121 and a video decoder 122 .

[0033] In some embodiments, decoding device 120 may further include a display device 123 in addition to input interface 121 and video decoder 122 .

[0034] The input interface 121 includes a receiver and / or a modem and is capable of receiving encoded video data via a channel 130.

[0035] The video decoder 122 is used for decoding the encoded video data to obtain decoded video data, and transmitting the decoded video data to a display device 123 .

[0036] Display device 123 displays the decoded video data and may be integrated into decoding device 120 or may be external to decoding device 120. Display device 123 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0037] It should be noted that FIG. 1 is merely an example, and the technical solutions of the embodiments of the present application are not limited to FIG. 1, for example, the technology of the present application can also be used for one-way video encoding or one-way video decoding.

[0038] In the following, a video encoding framework according to an embodiment of the present invention is introduced.

[0039] 2 is a schematic block diagram of a video encoder according to an embodiment of the present application. It should be understood that the video encoder 200 can be used to perform lossy compression of an image, or lossless compression of an image. The lossless compression can be visually lossless compression or mathematically lossless compression.

[0040] The video encoder 200 can be used for image data in luminance chroma (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2 or 4:4:4, where Y represents Luma, Cb (U) represents Blue chroma, Cr (V) represents Red chroma, and U and V represent Chroma, which are used to describe Hue and Saturation. For example, in color format, 4:2:0 represents four luma components and two chroma components (YYYYCbCr) for every four pixels, 4:2:2 represents four luma components and four chroma components (YYYYCbCrCbCr) for every four pixels, and 4:4:4 represents full pixel representation (YYYYCbCrCbCrCbCrCbCr).

[0041] For example, the video encoder 200 reads video data, and for each frame image in the video data, divides the image of one frame into a number of coding tree units (CTUs), and in some examples, the CTU may be called a "tree block", a "largest coding unit" (LCU) or a "coding tree block" (CTB). Each CTU may be associated with a pixel block of the same size in the image. Each pixel may correspond to one luminance (or luma) sample and two chrominance (or chroma) samples. Thus, each CTU may be associated with one luminance sample block and two chroma sample blocks. The size of one CTU may be, for example, 128×128, 64×64, 32×32, etc. One CTU may be further divided into a number of coding units (CUs) for encoding, and the CU may be a rectangular block or a square block. The CU may be further partitioned into a prediction unit (PU) and a transform unit (TU), which allows for more flexibility in the process of encoding, prediction, and transform separation. In one example, the CTU is partitioned into CUs according to a quad tree, and the CU is partitioned into TUs and PUs according to the quad tree.

[0042] The video encoder and the video decoder can support various PU sizes. Given a particular CU size of 2N×2N, the video encoder and the video decoder can support a PU size of 2N×2N or N×N for intra prediction, and can support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N or similar sizes for inter prediction. The video encoder and the video decoder can further support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0043] 2, the video encoder 200 may include a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filtering unit 260, a decoded image cache 270, and an entropy coding unit 280. Illustratively, the video encoder 200 may include more, less, or different functional components.

[0044] Alternatively, in this application, the current block may be referred to as a current coding unit (CU) or a current prediction unit (PU), etc. The prediction block may be referred to as a predicted image block or an image prediction block, and the reconstructed image block may be referred to as a reconstruction block or an image reconstruction image block.

[0045] In some embodiments, the prediction unit 210 includes an inter prediction unit 211 and an intra estimation unit 212. Because there is a strong correlation between adjacent pixels in a frame of a video, the video codec technology uses the method of intra prediction to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent frames in a video, the video codec technology uses the method of inter prediction to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0046] The inter prediction unit 211 can be used for inter prediction, which may include motion estimation and motion compensation, and can refer to image information of different frames, and the inter prediction uses motion information to find a reference block from a reference frame and generates a prediction block based on the reference block to eliminate time redundancy. The frame used for inter prediction may be a P frame and / or a B frame, where P frame means a forward prediction frame and B frame means a bidirectional prediction frame. The inter prediction uses motion information to find a reference block from a reference frame and generates a prediction block based on the reference block. The motion information includes a reference frame list where the reference frame is located, a reference frame index, and a motion vector. The motion vector may be full pixel or sub-pixel, and when the motion vector is sub-pixel, it is necessary to use interpolation filtering in the reference frame to create a required sub-pixel block, and here the full pixel or sub-pixel block in the reference frame found based on the motion vector is called a reference block. There are some techniques that directly use the reference block as a prediction block, and there are some techniques that reprocess based on the reference block to generate a prediction block. Re-processing based on a reference block to generate a predictive block may be understood as using the reference block as a predictive block and processing the predictive block to generate a new predictive block.

[0047] The intra prediction unit 212 only refers to information of the image of the same frame to predict pixel information in the current image block, so as to eliminate spatial redundancy. The frame used for intra prediction may be an I-frame.

[0048] There are various prediction modes for intra prediction, and taking the international digital video coding standard H series as an example, the H.264 / AVC standard has 8 types of angular prediction modes and 1 type of non-angular prediction mode, while H.265 / HEVC is extended to 33 types of angular prediction modes and 2 types of non-angular prediction modes. The intra prediction modes used in HEVC include planar mode, DC, and 33 types of angular modes, for a total of 35 types of prediction modes. The intra modes used in VVC include planar, DC, and 65 types of angular modes, for a total of 67 types of prediction modes.

[0049] It should be noted that as the angle modes increase, the intra prediction becomes more accurate and meets the demands of the development of high definition and ultra high definition digital videos.

[0050] The residual unit 220 may generate a residual block of a CU based on the pixel block of the CU and the prediction block of the PU of the CU. For example, the residual unit 220 generates the residual block of a CU such that each sample in the residual block has a value equal to the difference between a sample in the pixel block of the CU and a corresponding sample in the prediction block of the PU of the CU.

[0051] The transform / quantization unit 230 may quantize the transform coefficients. The transform / quantization unit 230 may quantize the transform coefficients associated with the TUs of a CU based on a quantization parameter (QP) value associated with the CU. The video encoder 200 may adjust the degree of quantization used for the transform coefficients associated with a CU by adjusting the QP value associated with the CU.

[0052] Each of the inverse transform / quantization units 240 may use inverse quantization and inverse transformation on the quantized transform coefficients to reconstruct a residual block from the quantized transform coefficients.

[0053] Reconstruction unit 250 may add the samples of the reconstructed residual block to corresponding samples of one or more predictive blocks generated by prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of a CU in this manner, video encoder 200 may reconstruct pixel blocks of the CU.

[0054] The loop filtering unit 260 is used to process the pixels after inverse transformation and inverse quantization, compensate for the distortion information, and provide a better reference for subsequent coding pixels, and can perform a deblocking filtering operation, for example, to reduce the blocking effect of pixel blocks associated with a CU.

[0055] In some embodiments, the loop filtering unit 260 includes a deblocking filtering unit for removing blocking effects and a sample point adaptive compensation / adaptive loop filtering (SAO / ALF) unit for removing ringing effects.

[0056] The decoded image cache 270 can store the reconstructed pixel blocks. The inter prediction unit 211 can perform inter prediction on PUs of other images using a reference image including the reconstructed pixel blocks. And, the intra estimation unit 212 can perform intra prediction on other PUs in the same image as the CU using the reconstructed pixel blocks in the decoded image cache 270.

[0057] The entropy encoding unit 280 may receive the quantized transform coefficients from the transform / quantization unit 230. The entropy encoding unit 280 may perform one or more entropy coding operations on the quantized transform coefficients to generate entropy coded data.

[0058] FIG. 3 is a schematic block diagram of a video decoder according to an embodiment of the present invention.

[0059] 3, the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transform unit 330, a reconstruction unit 340, a loop filtering unit 350, and a decoded image cache 360. It should be noted that the video decoder 300 may include more, less, or different functional components.

[0060] The video decoder 300 may receive a codestream. The entropy decoding unit 310 may parse the codestream to extract syntax elements from the codestream. As part of parsing the codestream, the entropy decoding unit 310 may parse the entropy encoded syntax elements in the codestream. The prediction unit 320, the inverse quantization / transform unit 330, the reconstruction unit 340, and the loop filtering unit 350 may decode the video data based on the syntax elements extracted from the codestream, i.e., generate decoded video data.

[0061] In some embodiments, the prediction unit 320 includes an intra estimation unit 322 and an inter prediction unit 321 .

[0062] The intra estimation unit 322 may perform intra prediction to generate a predictive block of the PU. The intra estimation unit 322 may use an intra prediction mode to generate a predictive block of the PU based on pixel blocks of spatially neighboring PUs. The intra estimation unit 322 may further determine an intra prediction mode of the PU based on one or more syntax elements parsed from the codestream.

[0063] The inter prediction unit 321 may construct a first reference image list (list 0) and a second reference image list (list 1) based on syntax elements parsed from the codestream. Note that, when inter prediction coding is used for the PU, the entropy decoding unit 310 may analyze the motion information of the PU. The inter prediction unit 321 may determine one or more reference blocks of the PU based on the motion information of the PU. The inter prediction unit 321 may generate a prediction block of the PU based on the one or more reference blocks of the PU.

[0064] The inverse quantization / transform unit 330 may inverse quantize (i.e., dequantize) the transform coefficients associated with the TU. The inverse quantization / transform unit 330 may determine the degree of quantization using a QP value associated with the TU of the CU.

[0065] After inverse quantizing the transform coefficients, the inverse quantization / transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients to generate a residual block associated with the TU.

[0066] The reconstruction unit 340 reconstructs a pixel block of the CU using a residual block associated with a TU of the CU and a prediction block of a PU of the CU. For example, the reconstruction unit 340 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

[0067] The loop filtering unit 350 may perform a deblocking filtering operation to reduce the blocking effect of pixel blocks associated with a CU.

[0068] The video decoder 300 can store the reconstructed image of the CU in the decoded image cache 360. The video decoder 300 can use the reconstructed image in the decoded image cache 360 ​​as a reference image for subsequent prediction, or can transmit the reconstructed image to a display device for display.

[0069] The basic flow of the video codec is as follows. On the encoding side, an image of one frame is divided into blocks, and for a current block, the prediction unit 210 generates a prediction block of the current block using intra prediction or inter prediction. The residual unit 220 can calculate a residual block, i.e., a difference between the prediction block and the original block of the current block, based on the prediction block and the original block of the current block, and the residual block may be called residual information. The residual block can eliminate visual redundancy by removing information that is not sensitive to human eyes through processes such as transformation and quantization by the transform / quantization unit 230. Alternatively, the residual block before being transformed and quantized by the transform / quantization unit 230 may be called a time domain residual block, and the time domain residual block after being transformed and quantized by the transform / quantization unit 230 may be called a frequency residual block or a frequency domain residual block. The entropy coding unit 280 may receive the quantized transform coefficients output by the transform / quantization unit 230, perform entropy coding on the quantized transform coefficients, and output a code stream. For example, the entropy coding unit 280 may eliminate character redundancy based on a target context model and probability information of the binary code stream.

[0070] On the decoding side, the entropy decoding unit 310 can analyze the code stream to obtain prediction information, a quantization coefficient matrix, etc., of the current block, and the prediction unit 320 uses intra prediction or inter prediction for the current block based on the prediction information to generate a prediction block for the current block. The inverse quantization / transform unit 330 uses the quantization coefficient matrix obtained from the code stream to perform inverse quantization and inverse transformation on the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed block constitutes a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on an image or block to obtain a decoded image. On the encoding side, it is also necessary to perform operations similar to those on the decoding side to obtain a decoded image. The decoded image may be called a reconstructed image, and the reconstructed image may be a reference frame of inter prediction for a subsequent frame.

[0071] It should be noted that the block partition information determined by the encoding side, and mode information or parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc. are included in the code stream if necessary. The decoding side analyzes the code stream and determines the same block partition information, mode information or parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc. as the encoding side based on existing information, thereby ensuring that the decoded image obtained by the encoding side is similar to the decoded image obtained by the decoding side.

[0072] The above is the basic flow of a video codec device in a block-based hybrid coding framework, and as technology develops, some modules or steps of the framework or flow may be optimized, and the present application applies to, but is not limited to, the basic flow of a video codec device in the block-based hybrid coding framework.

[0073] In some embodiments, the current block may be a current coding unit (CU) or a current prediction unit (PU), etc. Due to the need for parallel processing, an image may be divided into slices, etc., and parallel processing may be performed on slices in the same image, that is, there is no data dependency between them. The term "frame" is a general term, and one frame may generally be understood to be one image. In the application, the frame may be replaced with an image, a slice, etc.

[0074] The Versatile Video Coding (VVC) video codec standard currently being developed has an inter prediction mode called Geometric partitioning mode (GPM). The Audio Video coding Standard (AVS) video codec standard currently being developed has an inter prediction mode called Angular Weighted prediction (AWP). Although the two modes have different names and are implemented differently, they share the same principle.

[0075] It should be noted that the traditional unidirectional prediction only finds one reference block with the same size as the current block, while the traditional bidirectional prediction uses two reference blocks with the same size as the current block, and the pixel value of each point of the prediction block is the average value of the corresponding positions of the two reference blocks, that is, all points of each reference block account for 50%. In bidirectional weighted prediction, the proportions of the two reference blocks may be different, for example, all points in the first reference block account for 75%, and all points in the second reference block account for 25%. However, the proportions of all points in the same reference block are the same. Some other optimization methods, such as the Decoder side Motion Vector Refinement (DMVR) technology, Bi-directional Optical Flow (BIO), etc., may cause slight changes in the reference pixels or prediction pixels, but are not related to the above principle. BIO may be abbreviated as BDOF. GPM or AWP uses two reference blocks of the same size as the current block, but some pixel positions use 100% of the pixel value of the corresponding position of the first reference block, some pixel positions use 100% of the pixel value of the corresponding position of the second reference block, and the boundary area (also called blending area) uses the pixel values ​​of the corresponding positions of the two reference blocks at a certain ratio. The weights of the boundary area are also gradually blended. Specifically, how these weights are assigned is determined by the mode of GPM or AWP. The weight of each pixel position is determined based on the mode of GPM or AWP. It goes without saying that in the mode of GPM or AWP, such as when the block size is small, it may not be possible to ensure that some pixel positions use 100% of the pixel value of the corresponding position of the first reference block, and some pixel positions use 100% of the pixel value of the corresponding position of the second reference block. GPM or AWP uses two reference blocks of different sizes from the current block, that is, it may be considered that each uses a necessary part as a reference block.That is, the part with a weight other than 0 is used as the reference block, and the part with a weight of 0 is removed. This is a matter of implementation, and is not the point of this application.

[0076] Illustratively, FIG. 4 is a schematic diagram of weight allocation, and as shown in FIG. 4, a schematic diagram of weight allocation of multiple partition modes in a 64×64 current block of a GPM according to an embodiment of the present application is shown, and the GPM has 64 types of partition modes. FIG. 5 is a schematic diagram of weight allocation, and as shown in FIG. 5, a schematic diagram of weight allocation of multiple partition modes in a 64×64 current block of an AWP according to an embodiment of the present application is shown, and the AWP has 56 types of partition modes. Of course, in either FIG. 4 or FIG. 5, in each type of partition mode, the black area represents that the weight value of the corresponding position of the first reference block is 0%, the white area represents that the weight value of the corresponding position of the first reference block is 100%, and the gray area represents that the weight value of the corresponding position of the first reference block is any one of 0% and 100% or less depending on the shade of the color, and the weight value of the corresponding position of the second reference block is 100% minus the weight value of the corresponding position of the first reference block.

[0077] GPM and AWP have different weight derivation methods. GPM determines the angle and offset amount based on each type of mode, and then calculates the weight matrix for each mode. AWP first creates a one-dimensional weight line, and then tile it over the entire matrix using a method similar to intra-angle prediction.

[0078] It should be understood that the early codec technology only has a rectangular partitioning scheme, regardless of the partitioning of CU, PU, ​​and transform unit (TU). GPM and AWP achieve the effect of non-rectangular partitioning of prediction without partitioning. GPM and AWP use a mask of the weights of two reference blocks, i.e., the weight diagram. This mask determines the weight when generating a prediction block based on two reference blocks, or it may be simply understood that some positions of the prediction block come from the first reference block and some positions come from the second reference block, and the blending area is obtained by weighting the corresponding positions of the two reference blocks, thereby making the blending smoother. GPM and AWP do not partition the current block into two CUs or PUs according to the partition line, so that the entire current block is treated as one block even in the transformation, quantization, inverse transformation, inverse quantization, etc. of the residual after prediction.

[0079] The GPM uses a weight matrix to simulate a geometric partition, more precisely, a prediction partition. In order to implement the GPM, in addition to the weight matrix, two prediction values ​​are required, each determined by one unidirectional motion information. The two unidirectional motion information are from one motion information candidate list, for example, a merge motion information candidate list (mergeCandList). The GPM uses two indexes in the codestream to determine the two unidirectional motion information from the mergeCandList.

[0080] Inter prediction expresses "motion" with motion information. Basic motion information includes information on a reference frame (or called a reference picture) and information on a motion vector (MV). Conventional bidirectional prediction predicts a current block using two reference blocks. The two reference blocks may be one forward reference block and one backward reference block. Alternatively, both may be forward or backward. The so-called forward direction means that the time corresponding to the reference frame is before the current frame, and the backward direction means that the time corresponding to the reference frame is after the current frame. Alternatively, the forward direction means that the position of the reference frame in the video is before the current frame, and the backward direction means that the position of the reference frame in the video is after the current frame. Alternatively, the forward direction means that the picture order count (POC) of the reference frame is smaller than the POC of the current frame, and the backward direction means that the POC of the reference frame is larger than the POC of the current frame. To use bidirectional prediction, two reference blocks need to be found, and two sets of reference frame information and motion vector information are required. Each of those sets may be understood as one unidirectional motion information, and the two sets are combined to form one bidirectional motion information. In concrete implementation, the unidirectional motion information and the bidirectional motion information may use a similar data structure, but the bidirectional motion information has two sets of reference frame information and motion vector information that are both valid, while the unidirectional motion information has one set of reference frame information and motion vector information that are invalid.

[0081] In some embodiments, two reference frame lists, RPL0 and RPL1, are supported, where RPL is an abbreviation of Reference Picture List. In some embodiments, P slice can only use RPL0, and B slice can use RPL0 and RPL1. For one slice, each reference frame list has multiple reference frames, and the codec device finds one reference frame by a reference frame index. In some embodiments, the motion information is represented by a reference frame index and a motion vector. For example, for the bidirectional motion information, a reference frame index refIdxL0 corresponding to reference frame list 0, a motion vector mvL0 corresponding to reference frame list 0, a reference frame index refIdxL1 corresponding to reference frame list 1, and a motion vector mvL1 corresponding to reference frame list 1 are used. Here, the reference frame index corresponding to reference frame list 0 and the reference frame index corresponding to reference frame list 1 may be understood as information of the reference frame. In some embodiments, whether to use the motion information corresponding to the reference frame list 0 and whether to use the motion information corresponding to the reference frame list 1 are represented by two flag bits, which are denoted as predFlagL0 and predFlagL1, respectively. predFlagL0 and predFlagL1 may be understood to represent whether the one-way motion information is "valid or not". Although no particular reference is made to a data structure called motion information, the motion information is represented by a reference frame index, a motion vector, and a flag bit indicating "valid or not" corresponding to each reference frame list. Although some standard texts use motion vectors instead of motion information, the reference frame index and the flag indicating whether to use the corresponding motion information may be considered to be appended to the motion vector. In this application, for convenience of explanation, "motion information" is used, but it should be understood that "motion vector" may also be used for explanation.

[0082] The motion information used for the current block may be stored. A block coded later than the current frame may use the motion information of a previously coded block, for example, an adjacent block, based on the adjacent positional relationship. This utilizes the spatial domain relationship, so this coded motion information is called spatial domain motion information. The motion information used for each block of the current frame may be stored. A subsequently coded frame may use the motion information of a previously coded frame based on the reference relationship. This utilizes the temporal domain relationship, so this coded frame motion information is called temporal domain motion information. In general, a fixed size matrix, for example, a 4×4 matrix, is used as the smallest unit for independently storing a set of motion information, and motion information used for each block of the current frame is stored. In this way, each time a block is coded, the motion information of this block can be stored in the smallest unit corresponding to its position. In this way, by using the spatial domain motion information or the temporal domain motion information, the motion information corresponding to the position can be found directly based on its position. For example, if a certain 16×16 block uses conventional unidirectional prediction, all 4×4 smallest units corresponding to this block store the unidirectional prediction motion information. When a block uses GPM or AWP, all minimum units corresponding to this block determine the motion information to be stored in each minimum unit based on the mode of GPM or AWP, the first motion information, the second motion information, and the position of each minimum unit. As one method, if all 4×4 pixels corresponding to a minimum unit are from the first motion information, the first motion information is stored in this minimum unit, if all 4×4 pixels corresponding to a minimum unit are from the second motion information, the second motion information is stored in this minimum unit, and if 4×4 pixels corresponding to a minimum unit are from both the first motion information and the second motion information, the AWP selects and stores one of the motion information.In the GPM, if two pieces of motion information point to different reference frame lists, they are combined and stored as bidirectional motion information; otherwise, only the second piece of motion information is stored.

[0083] Alternatively, the mergeCandList is constructed based on motion information in the spatial domain, motion information in the temporal domain, history-based motion information, and some other motion information. Exemplarily, mergeCandList estimates motion information in the spatial domain using positions 1 to 5 in FIG. 6A, and estimates motion information in the temporal domain using positions 6 or 7 in FIG. 6A. History-based motion information may require adding motion information of a block to a first-in-first-out list every time a block is coded, and checking whether the motion information overlaps with existing motion information in the list during the addition process. In this way, the motion information in the history-based list can be referenced when coding a current block.

[0084] In some embodiments, a syntax description for the GPM is shown in Table 1.

[0085] Table 1 JPEG2025513075000043.jpg224170As shown in Table 1, in merge mode, the current block may use CIIP or GPM if regular_merge_flag is not 1. If the current block does not use CIIP, it uses GPM, that is, as shown in the syntax "if(!ciip_flag[x0][y0])" in Table 1.

[0086] As can be seen from Table 1 above, the GPM needs to transmit three pieces of information in the codestream, namely, merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1. x0, y0 are used to determine the coordinates (x0, y0) of the upper left luminance pixel of the current block relative to the upper left luminance pixel of the image. merge_gpm_partition_idx determines the shape of the partitioning by the GPM, and as explained above, the GPM "simulates partitioning", and merge_gpm_partition_idx is the weight matrix derivation mode or weight matrix derivation mode index, or weight derivation mode or weight derivation mode index, as referred to in this specification. merge_gpm_idx0 is the first merge candidate index, and the first merge candidate index is used to determine the first motion information based on the mergeCandList, and is also called the first merge candidate. merge_gpm_idx1 is the second merge candidate index, which is used to determine the second motion information based on mergeCandList, and is also called the second merge candidate. If MaxNumGpmMergeCand>2, merge_gpm_idx1 needs to be decoded only if the length of the candidate list is greater than 2, otherwise it can be determined directly.

[0087] In some embodiments, the decoding process of the GPM includes the following steps.

[0088] The information input during the decoding process includes the coordinates (xCb, yCb) of the luminance position of the upper left corner of the current block relative to the upper left corner of the image, the width cbWidth of the luminance component of the current block, the height cbHeight of the luminance component of the current block, luminance motion vectors mvA and mvB with 1 / 16 pixel accuracy, chroma motion vectors mvCA and mvCB, reference frame indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB.

[0089] Exemplarily, the motion information may be represented by a combination of a motion vector, a reference frame index, and a prediction list flag. VVC supports two reference frame lists, and each reference frame list may include multiple reference frames. Unidirectional prediction uses only one reference block of one reference frame in one reference frame list as a reference, and bidirectional prediction uses each reference block of each reference frame in the two reference frame lists as a reference. GPM in VVC uses two unidirectional predictions. A in the above mvA and mvB, mvCA and mvCB, refIdxA and refIdxB, predListFlagA and predListFlagB may be understood as a first prediction mode, and B may be understood as a second prediction mode. Let A or B be represented by X, then predListFlagX represents that X uses the first or second reference frame list, refIdxX represents the reference frame index in the reference frame list used for X, mvX represents the luma motion vector used for X, and mvCX represents the chroma motion vector used for X. Again, in VVC, the combination of motion vector, reference frame index, and prediction list flag can be considered to represent motion information as described herein.

[0090] The information output during the decoding process includes a luminance prediction sample matrix predSamplesL of (cbWidth) x (cbHeight), and, if necessary, a prediction sample matrix of the Cb chrominance component of (cbWidth / SubWidthC) x (cbHeight / SubHeightC), and, if necessary, a prediction sample matrix of the Cr chrominance component of (cbWidth / SubWidthC) x (cbHeight / SubHeightC).

[0091] For illustrative purposes, the following description takes the luminance component as an example, and the processing of the chroma components is similar to that of the luminance component.

[0092] The size of predSamplesLAL and predSamplesLBL is (cbWidth)×(cbHeight), which is a prediction sample matrix created based on two prediction modes. predSamplesL is derived in the following manner. predSamplesLAL and predSamplesLBL are determined based on luma motion vectors mvA and mvB, chroma motion vectors mvCA and mvCB, reference frame indexes refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB, respectively. That is, prediction is performed based on motion information of two prediction modes, and the detailed process will not be described further. In general, GPM is a merge mode, and it is considered that both of the two prediction modes of GPM are merge modes.

[0093] Based on merge_gpm_partition_idx[xCb][yCb], the GPM partition angle index variable angleIdx and distance index variable distanceIdx are determined using Table 2.

[0094] Table 2 - Correspondence between angleIdx, distanceIdx and merge_gpm_partition_idx JPEG2025513075000044.jpg130170It should be noted that all three components (e.g., Y, Cb, Cr) can use GPM, so some standard texts put the process of generating the predicted sample matrix of one component into one sub-flow, namely the weighted sample prediction process for GPM, and all three components call this flow, but only the calling parameters are different, and here only the luminance component is taken as an example. The predicted matrix predSamplesL[xL][yL] (xL = 0..cbWidth - 1, yL = 0..cbHeight - 1) of the current luminance block is derived by the weighted prediction process of GPM. nCbW is set as cbWidth, nCbH is set as cbHeight, and prediction sample matrices predSamplesLAL and predSamplesLBL created based on two prediction modes, as well as angleIdx and distanceIdx are input.

[0095] In some embodiments, the weighted prediction derivation process of the GPM includes the following steps.

[0096] The inputs of this process include the width nCbW of the current block, the height nCbH of the current block, two (nCbW)×(nCbH) predicted sample matrices predSamplesLA and predSamplesLB, a GPM segment angle index variable angleIdx, a GPM distance index variable distanceIdx, and a component index variable cIdx. Since this example takes luminance as an example, the cIdx is 0, which represents the luminance component.

[0097] The output of this process is a (nCbW) × (nCbH) GPM prediction sample matrix pbSamples.

[0098] Exemplarily, the variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor are derived in the following manner.

[0099] nW =(cIdx = = 0)? nCbW : nCbW * SubWidthC, nH =(cIdx = = 0)? nCbH : nCbH * SubHeightC, shift1 = Max(5, 17 - BitDepth), where BitDepth is the bit depth of the codec. offset1 = 1 << (shift1 - 1), "<<" means to move left, displacementX = angleIdx, displacementY =(angleIdx + 8)% 32, partFlip =(angleIdx >= 13 && angleIdx <= 27)? 0 : 1, shiftHor =(angleIdx % 16 = = 8 | | (angleIdx % 16 != 0 && nH >= nW))? 0 : 1.

[0100] The variables offsetX and offsetY are derived in the following manner.

[0101] If the value of shiftHor is 0, offsetX = (-nW) >> 1, offsetY =((-nH) >> 1)+(angleIdx < 16 ?(distanceIdx * nH) >> 3 : -((distanceIdx * nH) >> 3)).

[0102] If the value of shiftHor is 1, offsetX =((-nW) >> 1)+(angleIdx < 16 ?(distanceIdx * nW) >> 3 : -((distanceIdx * nW) >> 3), offsetY = (- nH) >> 1.

[0103] The variables xL and yL are derived in the following manner.

[0104] xL =(cIdx = = 0)? x : x * SubWidthC, yL =(cIdx = = 0)? y : y * SubHeightC, The variable wValue, which represents the weight of the prediction sample at the current position, is derived in the following manner, where wValue is the weight of the prediction value predSamplesLA[x][y] of the prediction matrix of the first prediction mode at point (x,y), and (8 - wValue) is the weight of the prediction value predSamplesLB[x][y] of the prediction matrix of the first prediction mode at point (x,y).

[0105] The distance matrix disLut is determined according to Table 3.

[0106] Table 3 JPEG2025513075000045.jpg51162weightIdx =(((xL + offsetX) << 1)+ 1)* disLut[ displacementX ] +(((yL + offsetY) << 1)+ 1)* disLut[ displacementY ], weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx, wValue = Clip3(0, 8, (weightIdxL + 4) >> 3), The predicted sample values ​​pbSamples[x][y] are derived as follows:

[0107] pbSamples[ x ][ y ] = Clip3(0, (1 << BitDepth)- 1, (predSamplesLA[ x ][ y ] * wValue + predSamplesLB[ x ][ y ] *(8 - wValue)+ offset1) >> shift1).

[0108] It should be noted that one weight value is derived for each position of the current block, and one predicted value pbSamples[x][y] of the GPM is calculated. In this method, the weight wValue does not need to be written in the form of a matrix, but as can be understood, if the wValue of each position is stored in one matrix, it will become one weight matrix. The principle is the same between calculating and weighting the weight for each point to obtain a predicted value of the GPM and calculating all the weights and weighting them collectively to obtain a predicted sample matrix of the GPM. In this application, the term weight matrix is ​​used in many explanations, but this is for ease of understanding. It is more intuitive to draw a weight matrix, but in reality it may be described by the weight of each position. For example, the weight matrix derivation mode is also called a weight derivation mode.

[0109] In some embodiments, as shown in FIG. 6B, the decoding flow of GPM may be described as follows: Analyze the code stream, determine whether the current block uses GPM technology, and if the current block uses GPM technology, determine a weight derivation mode (or partition mode or weight matrix derivation mode), and first and second motion information. Respectively determine a first prediction block according to the first motion information, determine a second prediction block according to the second motion information, determine a weight matrix according to the weight matrix derivation mode, and determine a prediction block of the current block according to the first prediction block, the second prediction block, and the weight matrix.

[0110] The intra prediction method predicts a current block using the coded reconstructed pixels around the current block as reference pixels. FIG. 7A is a schematic diagram of intra prediction. As shown in FIG. 7A, the size of the current block is 4×4, and the pixels in one row to the left and one column above the current block are reference pixels of the current block, and intra prediction predicts the current block using these reference pixels. All of these reference pixels may be obtained, i.e., all have been coded. Some may not be obtained. For example, if the current block is the leftmost block of the entire frame, the reference pixel to the left of the current block cannot be obtained. Or, when coding the current block, the lower left part of the current block is not coded, so the lower left reference pixel cannot be obtained. When the reference pixel is not obtained, the obtained reference pixel, value, or method may be used to fill in, or may not be used to fill in.

[0111] FIG. 7B is a schematic diagram of intra prediction. As shown in FIG. 7B, a multiple reference line (MRL) intra prediction method can improve codec efficiency by using more reference pixels, for example, using four reference rows / columns as reference pixels for the current block.

[0112] Furthermore, intra prediction has various prediction modes, and Figures 8A-8I are schematic diagrams of intra prediction, and as shown in Figures 8A-8I, in H.264, intra prediction for a 4x4 block may mainly include nine types of modes. In mode 0 shown in Figure 8A, the pixel above the current block is copied vertically to the current block as a predicted value, in mode 1 shown in Figure 8B, the left reference pixel is copied horizontally to the current block as a predicted value, in mode 2 DC shown in Figure 8C, the average value of eight points A to D and I to L is used as the predicted value of all points, and in each of modes 3 to 8 shown in Figures 8D-8I, the reference pixel is copied to the corresponding position of the current block at a certain angle, but since some positions of the current block cannot exactly correspond to the reference pixel, it may be necessary to use the weighted average value of the reference pixel or the subpixel of the interpolated reference pixel.

[0113] There are other models such as Plane and Planar, but with the development of technology and the expansion of blocks, the number of angle prediction models is also increasing. FIG. 9 is a schematic diagram of intra prediction modes. As shown in FIG. 9, the intra prediction modes used in HEVC include Planar, DC, and 33 types of angle modes, totaling 35 types of prediction modes. FIG. 10 is a schematic diagram of intra prediction modes. As shown in FIG. 10, the intra modes used in VVC include Planar, DC, and 65 types of angle modes, totaling 67 types of prediction modes. FIG. 11 is a schematic diagram of intra prediction modes. As shown in FIG. 11, VS3 uses DC, Plane, Bilinear, PCM, and 62 types of angle modes, totaling 66 types of prediction modes.

[0114] There are also techniques to improve prediction, for example, improving sub-pixel interpolation of reference pixels or filtering predicted pixels. For example, multiple combination intra prediction filtering (MIPF) in AVS3 uses different filters to generate predicted values ​​for different block sizes. For pixels at different positions in the same block, one filter is used to generate predicted values ​​for pixels close to the reference pixels and another filter is used to generate predicted values ​​for pixels far from the reference pixels. A technique for filtering predicted pixels is, for example, intra prediction filtering (IPF) in AVS3, which can filter predicted values ​​with reference pixels.

[0115] In intra prediction, codec efficiency can be improved by using intra mode encoding technology of Most Probable Modes List (MPM). A mode list is configured using intra prediction modes of coded surrounding blocks, intra prediction modes derived based on intra prediction modes of coded surrounding blocks, such as adjacent modes, and some commonly used or highly used intra prediction modes, such as DC, Planar, and Bilinear modes. Since textures have a certain spatial continuity, referring to intra prediction modes of coded surrounding blocks is using spatial association. MPM can be used as a prediction of intra prediction modes. That is, it is considered that the probability that the current block uses MPM is higher than the probability that it does not use MPM. Therefore, fewer codewords are used for MPM during binarization, thereby saving overhead and improving codec efficiency.

[0116] In some embodiments, intra prediction may be performed using matrix-based intra prediction (MIP), also called matrix weighted intra prediction. As shown in FIG. 12, to predict a block with width W and height H, MIP needs to input H reconstructed pixels in one column to the left of the current block and W reconstructed pixels in one row above the current block. MIP generates a predicted block according to three steps: averaging of reference pixels, matrix multiplication, and interpolation. Matrix multiplication is the core of MIP. MIP can be considered as a process of inputting pixels (reference pixels) in the manner of matrix multiplication to generate a predicted block. MIP provides various matrices, and differences in prediction methods are reflected in different matrices, and the same input pixel will obtain different results if different matrices are used. The processes of averaging and interpolation of reference pixels are a design that trades off performance and complexity. For larger blocks, the averaging of reference pixels can achieve an effect close to downsampling, and the input can be adapted to a smaller matrix, while the interpolation can achieve an upsampling effect. In this way, it is not necessary to provide MIP matrices for blocks of various sizes, but only matrices of one or more specific sizes. With increasing demands for compression performance and improved hardware capabilities, more complex MIPs may appear in the next generation of standards.

[0117] MIPs are somewhat similar to planar, but are clearly more complex and flexible than planar.

[0118] GPM combines two inter prediction blocks with a weight matrix, and can be extended to any combination of two prediction blocks, such as two inter prediction blocks, two intra prediction blocks, one inter prediction block and one intra prediction block. In addition, in the coding of screen content, the prediction block of IBC (intra block copy) or palette can be used as one or two prediction blocks.

[0119] In this application, intra, inter, IBC, and palette are called different prediction methods. For convenience of explanation, the term prediction mode is used here. It may be understood that the codec device can generate information of one prediction block for a current block based on the prediction mode. For example, in intra prediction, the prediction mode may be a certain intra prediction mode, such as DC, Planar, various intra angle prediction modes, etc., and of course, one or several auxiliary information, such as an optimization method of intra reference pixels, an optimization method after generating a preliminary prediction block (e.g., filtering), etc., may be added. For example, in inter prediction, the prediction mode may be a skip mode, a merge mode, or an MMVD (merge with motion vector difference) mode, or an AMVP (advanced motion vector prediction), a unidirectional prediction, a bidirectional prediction, or a multiple hypothesis prediction. When the inter prediction mode uses one-way prediction, the other prediction mode needs to be able to determine one motion information, and can determine a prediction block based on the one motion information. When the inter prediction mode uses bidirectional prediction, the other prediction mode needs to be able to determine two motion information, and can determine a prediction block based on the two motion information.

[0120] The information that the GPM needs to determine may be described as one weight derivation mode and two prediction modes. The weight derivation mode is used to determine a weight matrix or weights, and each of the two prediction modes determines one prediction block or predicted value. The weight derivation mode is also called a partition mode. However, since it simulates partitioning, it is referred to as the weight derivation mode in this application.

[0121] The two prediction modes may be from the same or different prediction schemes, including but not limited to intra prediction, inter prediction, IBC, and palette.

[0122] Alternatively, the two prediction modes may be from the same or different prediction schemes, including but not limited to intra prediction, inter prediction, IBC, and palette.

[0123] A specific example is as follows: The current block uses GPM. This example is used for an inter-coded block, and merge modes in intra prediction and inter prediction are available. As shown in Table 4, a syntax element intra_mode_idx indicating which prediction mode is an intra prediction mode is added, and for example, when intra_mode_idx is 0, it indicates that both prediction modes are inter prediction modes, i.e., mode0IsInter is 1 and mode0IsInter is 1. When Intra_mode_idx is 1, it indicates that the first prediction mode is an intra prediction mode and the second prediction mode is an inter prediction mode, i.e., mode0IsInter is 0 and mode0IsInter is 1. When intra_mode_idx is 2, it indicates that the first prediction mode is an inter prediction mode and the second prediction mode is an intra prediction mode, i.e., mode0IsInter is 1 and mode0IsInter is 0. When intra_mode_idx is 3, this indicates that both of the two prediction modes are intra prediction modes, that is, mode0IsInter is 0 and mode0IsInter is 0.

[0124] Table 4 JPEG2025513075000046.jpg81162 In some embodiments, as shown in FIG. 13, the decoding flow of GPM may be described as follows: Analyze the code stream, determine whether the current block uses GPM technology, and if the current block uses GPM technology, determine a weight derivation mode (or partition mode or weight matrix derivation mode), and a first prediction mode and a second prediction mode. Determine a first prediction block according to the first prediction mode, determine a second prediction block according to the second prediction mode, determine a weight matrix according to the weight matrix derivation mode, and determine a prediction block of the current block according to the first prediction block, the second prediction block, and the weight matrix.

[0125] The template matching method is first used for inter prediction, and uses the relationship between adjacent pixels to use a part of the surrounding area of ​​the current block as a template. When the current block is coded, its left and upper sides are already coded according to the coding order. Of course, when realizing with a conventional hardware decoder, it is not necessarily possible to ensure that its left and upper sides are already coded when the decoding of the current block is started. Of course, what is described here is an inter block, and when generating a predicted block in an inter coded block in HEVC, for example, the surrounding reconstructed pixels are not required, so the prediction process of the inter block can be performed in parallel. However, an intra coded block necessarily requires the reconstructed pixels on the left and upper sides as reference pixels. Theoretically, the left and upper sides can be obtained, that is, the hardware design can be adjusted accordingly. Meanwhile, the right and lower sides cannot be obtained in the coding order of conventional standards, such as VVC.

[0126] As shown in FIG. 14, the rectangular regions on the left and top of the current block are used as templates, and the height of the left template portion is usually the same as the height of the current block, and the width of the top template portion is usually the same as the width of the current block, but may be different. The best matching position of the template is found in the reference frame, and the motion information or motion vector of the current block is determined thereby. This process may be roughly described as searching within a certain range from a starting position in a reference frame. Search rules such as the search range and search step size can be set in advance. Every time a position is moved to, the matching degree between the template corresponding to the position and the template surrounding the current block is calculated, and the so-called matching degree can be evaluated by some distortion cost, for example, SAD (sum of absolute difference) and SATD (sum of absolute transformed difference). The transformations usually used for SATD are Hadamard transformation, MSE (mean-square error), etc., and the smaller the value of SAD, SATD, MSE, etc., the higher the matching degree. The cost is calculated using the predicted block of the template corresponding to the position and the reconstructed block of the template surrounding the current block. Besides searching the full pixel position, sub-pixel position can be searched, and the motion information of the current block is determined based on the position with the highest degree of matching. Depending on the relationship between adjacent pixels, the motion information suitable for the template may also be suitable for the current block. Of course, the template matching method is not necessarily applied to all blocks, so a method such as using a control switch in the current block to indicate whether to use template matching or not can be used to determine whether the current block uses the template matching method. One of the names of this template matching method is DMVD (decoder side motion vector derivation).Both the encoder and the decoder can use the template to search, thereby derive motion information, or find better motion information based on the original motion information. It does not need to transmit a specific motion vector or a differential motion vector, and both the encoder and the decoder perform search according to the same rule, thereby ensuring the consistency of encoding and decoding. The template matching method can improve the compression performance, but it also needs to "search" in the decoder, which brings some complexity to the decoder.

[0127] The above is the use of the template matching method for intra-blocking, and the template matching method can also be used for intra-blocking, for example, the intra-prediction mode is determined using a template. Similarly, for the current block, a certain range of regions on the upper and left sides of the current block, for example, the rectangular region on the left side and the rectangular region on the upper side as shown in the above figure, may be used as a template. When the current block is coded, the reconstructed pixels in the template are obtained. This process may be roughly described as determining a set of candidate intra-prediction modes for the current block, the candidate intra-prediction modes constituting a subset of all available intra-prediction modes. Of course, the candidate intra-prediction modes may be the entire set of all available intra-prediction modes. This may be determined based on a trade-off between performance and complexity. The set of candidate intra-prediction modes may be determined based on MPM or some rules, for example, equal interval screening. The cost of each candidate intra-prediction mode in the template, for example, SAD, SATD, MSE, etc., is calculated. The template is predicted using the mode to generate a predicted block, and the cost is calculated for the predicted block and the reconstructed block of the template. A mode with a low cost may match the template better, and may utilize the similarity between neighboring pixels, and an intra-prediction mode that performs well in the template may also perform well in the current block. Select one or more modes with a low cost. Of course, the above two steps may be repeated, for example, after selecting one or more modes with a low cost, re-determine a set of candidate intra-prediction modes, recalculate the cost for the newly determined set of candidate intra-prediction modes, and select one or more modes with a low cost. This may be understood as rough selection and fine selection. Determine the one intra-prediction mode that is finally selected as the intra-prediction mode of the current block, or make the multiple intra-prediction modes that are finally selected candidates for the intra-prediction mode of the current block.Of course, the candidate intra-prediction mode set may be ordered only by the method of template matching, for example, ordering the MPM list, that is, for each mode in the MPM list, a prediction block is created in a template and a cost is determined, and the modes are ordered in ascending order of cost. Usually, the modes in the front of the MPM list have a smaller overhead in the codestream, thus achieving the purpose of improving compression efficiency.

[0128] The template matching method can be used to determine two prediction modes of the GPM. When the template matching method is used in the GPM, for a current block, one control switch can control whether two prediction modes of the current block use template matching, and two control switches can respectively control whether each of the two prediction modes uses template matching.

[0129] Meanwhile, how to use template matching is as follows. For example, when GPM is used in merge mode, GPM in VVC, for example, uses merge_gpm_idxX to determine one piece of motion information from mergeCandList, where capital X is 0 or 1. For the Xth motion information, one method is to optimize the motion information by the template matching method. That is, one piece of motion information is determined from mergeCandList based on merge_gpm_idxX, and when template matching is used for the motion information, the motion information is optimized by the template matching method. Another method is to directly search based on one default motion information and determine one piece of motion information without using merge_gpm_idxX to determine one piece of motion information from mergeCandList.

[0130] If the Xth prediction mode is an intra prediction mode and the Xth prediction mode of the current block uses a template matching method, it is not necessary to indicate the index of the intra prediction mode in the code stream, and one intra prediction mode can be determined using the template matching method, or it is necessary to determine one candidate set or MPM list using the template matching method and indicate the index of the intra prediction mode in the code stream.

[0131] As can be seen from the above, GPM has three elements: one weight matrix and two prediction modes. The advantage of GPM is that it can realize more autonomous combinations by the weight matrix. On the other hand, GPM needs to determine more information, so the overhead in the code stream is larger. Taking GPM in VVC as an example, GPM in VVC is used for merge mode. In the code stream, merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1 are used to determine the weight matrix, the first prediction mode, and the second prediction mode, respectively. Each of the weight matrix and the two prediction modes has multiple possible selections, for example, the weight matrix in VVC has 64 possible selections. Each of merge_gpm_idx0 and merge_gpm_idx1 has a maximum of six possible selections in VVC, and of course, VVC specifies that merge_gpm_idx0 and merge_gpm_idx1 do not overlap. Thus, GPM has 65 x 6 x 5 possible choices. If MMVD is used to optimize two motion information (prediction modes), it can provide several more possible choices for each prediction mode. This number is quite large.

[0132] If the GPM uses two intra prediction modes, each of which can use 67 normal intra prediction modes in VVC, and the two intra prediction modes are different, there are 64 x 67 x 66 possible choices. Of course, to save overhead, each prediction mode can be restricted to only use a subset of all normal intra prediction modes, but there are still many possible choices.

[0133] Currently, each of the one weight derivation mode and two prediction modes of GPM needs to transmit the necessary information in the code stream according to the respective rules. For example, one weight derivation mode has its own syntax element or elements, the first prediction mode has its own syntax element or elements, and the second prediction mode has its own syntax element or elements. Of course, in some cases, the standard may restrict that the second prediction mode cannot be the same as the first prediction mode, or some optimization means can be used for two prediction modes simultaneously (which may be understood as using for the current block), but the three elements are relatively independent in writing and parsing syntax elements, i.e., it requires many codewords to indicate the weight derivation mode and the prediction mode, and the coding overhead is large.

[0134] In order to solve the above technical problems, the present application combines a weight derivation mode and K prediction modes into one combination, and thus indicates the weight derivation mode and K prediction modes used for the current block in the code stream in a combined form, thereby further saving codewords and improving encoding efficiency.

[0135] Below, the video decoding method according to the embodiment of the present application will be introduced with reference to FIG. 15 by taking the decoding side as an example.

[0136] Figure 15 is a schematic diagram illustrating a flow chart of a video decoding method according to an embodiment of the present application, which is used in the video decoder shown in Figures 1 and 3. As shown in Figure 15, the method of the present embodiment includes the following steps:

[0137] S101, the code stream is decoded to obtain a first index.

[0138] The first index is used to indicate a first combination of one weight derivation mode and K prediction modes, where K is a positive integer greater than 1.

[0139] As can be seen from the above, one weight derivation mode and K prediction modes jointly generate one prediction block, which operates on the current block, i.e., determines weights according to the weight derivation mode, predicts the current block according to the K prediction modes, obtains K predicted values, and performs weighting processing on the K predicted values ​​according to the weights to obtain a predicted value of the current block. It can be seen from this that the one weight derivation mode and the K prediction modes jointly operate on the current block as one combination, and there is a relationship between them.

[0140] For example, the current block includes edges of K objects that move relatively, which is one suitable scenario for inter GPM. Theoretically, this division should occur at the edge of the object, but in reality, there are limited types of divisions, and it is not possible to cover all edges, so a close division may be selected, but such a close division may not be of only one type, and which type is selected depends on which type of division and the K prediction modes produce the best result. Similarly, which type of prediction mode is selected depends on which type of combination produces the best result, because even for a part that uses this prediction mode, it is difficult for natural video to perfectly match the part with the current block, and the final selection may be the one with the highest coding efficiency. Another place where GPM is often used is when the current block includes parts of objects that move relatively, such as where twisting and deformation occurs due to arm swinging, and such divisions are more blurred and may ultimately depend on which type of combination produces the best result. Yet another scenario is intra prediction, knowing that some textures in natural images are complex, some have gradients from one texture to another, and some may not be easily represented in one direction, intra GPM can provide more complex prediction blocks, and in general, intra-coded blocks will have a larger residual versus inter-coded blocks that are identically quantized, and the choice of which type of prediction mode to choose may ultimately depend on which type of combination results are optimal.

[0141] Based on the above description, one weight derivation mode and K prediction modes can act on the current block as one combination, and in order to save codewords and reduce encoding costs, in this embodiment, the weight derivation mode and K prediction modes corresponding to the current block are combined into one combination, i.e., a first combination, and a first index is used to indicate the first combination. Compared to indicating each of the weight derivation mode and the K prediction modes, this embodiment uses fewer codewords and further reduces encoding costs.

[0142] Specifically, the encoding side determines a first combination of one weight derivation mode and K prediction modes corresponding to the current block, and for convenience of explanation, the weight derivation mode included in the first combination is set as the first weight derivation mode, and the K prediction modes included in the first combination are set as the K first prediction modes. The encoding side predicts the current block using the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block, for example, derives a weight using the first weight derivation mode, predicts the current block using each of the K prediction modes to obtain K predicted values, and performs a weighting operation on the K predicted values ​​based on the derived weights to obtain a predicted value of the current block.

[0143] In order to maintain consistency between the encoding side and the decoding side, the encoding side indicates the first combination to the decoding side through a first index, i.e., writes the first index into the code stream. The decoding side obtains the first index by decoding the code stream, and further determines a first combination corresponding to a current block based on the first index, predicts the current block using the first weight derivation mode and the K first prediction modes included in the first combination, and obtains a predicted value of the current block.

[0144] In the embodiment of the present application, the specific format of the syntax element of the first index is not limited.

[0145] In one possible implementation, if the current block is predicted using the GPM technique, gpm_cand_idx represents the first index.

[0146] Since the first index is used to indicate the first combination, in some embodiments the first index is also referred to as a first combination index or an index of the first combination.

[0147] In one example, the syntax for adding the first index to the codestream is shown in Table 5.

[0148] Table 5 JPEG2025513075000047.jpg33163gpm_cand_idx is the first index.

[0149] In some embodiments, before decoding the first index, the decoding side first determines whether the current block uses K different prediction modes to perform weighted prediction processing. If the decoding side determines that the current block uses K different prediction modes to perform weighted prediction processing, it executes the above S101 to decode the code stream and obtain the first index. If the decoding side determines that the current block does not use K different prediction modes to perform weighted prediction processing, it skips the above step S101.

[0150] In one possible implementation, the decoding side can determine a prediction mode parameter of the current block to determine whether the current block undergoes weighted prediction processing using K different prediction modes.

[0151] Optionally, in the implementation of the present application, the prediction mode parameter can indicate whether the current block can use GPM mode or AWP mode, i.e., whether the current block can be predicted using K different prediction modes.

[0152] As can be understood, in the present embodiment, the prediction mode parameter may be understood as representing whether to use the GPM mode or the AWP mode flag bit. Specifically, the encoder may use one variable as the prediction mode parameter, and thereby realize the setting of the prediction mode parameter by setting the value of the variable. Exemplarily, in the present application, if the current block uses the GPM mode or the AWP mode, the encoder may set the value of the prediction mode parameter to indicate that the current block uses the GPM mode or the AWP mode, and specifically, the encoder may set the value of the variable to 1. Exemplarily, in the present application, if the current block does not use the GPM mode or the AWP mode, the encoder may set the value of the prediction mode parameter to indicate that the current block does not use the GPM mode or the AWP mode, and specifically, the encoder may set the value of the variable to 0. Furthermore, in the present embodiment, after completing the setting of the prediction mode parameter, the encoder may write the prediction mode parameter into the code stream and transmit it to the decoder, so that the decoder can obtain the prediction mode parameter after parsing the code stream.

[0153] Based on this, the decoding side decodes the code stream to obtain a prediction mode parameter, and further determines whether the current block uses GPM mode or AWP mode based on the prediction mode parameter, and if the current block uses GPM mode or AWP mode, i.e., predicts using K different prediction modes, determines the weight derivation mode of the current block.

[0154] In some embodiments, as shown in Table 5, the present embodiment may further perform a condition restriction on whether the current block uses the GPM mode or the AWP mode, i.e., if it is determined that the current block satisfies the preset condition, it may determine that the current block performs weighted prediction using K prediction modes, and the decoding side further decodes the code stream and obtains a first index.

[0155] Exemplarily, when using the GPM mode or the AWP mode, the size of the current block may be limited.

[0156] As can be understood, the prediction method according to the embodiment of the present application needs to generate K predicted values ​​using K different prediction modes, and then weight them based on the weights to obtain a predicted value of the current block, so in order to reduce complexity and consider the trade-off between compression performance and complexity, the embodiment of the present application can restrict the use of the GPM mode or AWP mode for a block of a certain size. Therefore, in the present application, the decoder can first determine the size parameter of the current block, and then determine whether the current block uses the GPM mode or the AWP mode based on the size parameter.

[0157] In the present embodiment, the size parameters of the current block may include the height and width of the current block, so that the decoder can determine whether the current block uses GPM mode or AWP mode based on the height and width of the current block.

[0158] Exemplarily, the present application determines that the current block uses the GPM mode or the AWP mode if the width is greater than the first threshold and the height is greater than the second threshold. As can be seen, one possible restriction is to use the GPM mode or the AWP mode only if the width of the block is greater than (or equal to) the first threshold and the height of the block is greater than (or equal to) the second threshold. The values ​​of the first and second thresholds may be 8, 16, 32, etc., and the first threshold may be equal to the second threshold.

[0159] Exemplarily, the present application determines that the current block uses the GPM mode or the AWP mode if the width is smaller than the third threshold and the height is larger than the fourth threshold. As can be seen, one possible restriction is to use the GPM mode or the AWP mode only if the width of the block is smaller than (or less than) the third threshold and the height of the block is larger than (or more than) the fourth threshold. The values ​​of the third and fourth thresholds may be 8, 16, 32, etc., and the third threshold may be equal to the fourth threshold.

[0160] Furthermore, in the present embodiment, the size of blocks in which the GPM mode or the AWP mode can be used can be restricted by further restricting pixel parameters.

[0161] For example, in this application, the decoder may first determine the pixel parameters of the current block, and then further determine whether the current block can use the GPM mode or the AWP mode based on the pixel parameters and the fifth threshold. As can be seen, one possible restriction is to use the GPM mode or the AWP mode only if the number of pixels of the block is greater than (or equal to or greater than) the fifth threshold. The value of the fifth threshold may be 8, 16, 32, etc.

[0162] That is, in this application, the current block can use the GPM mode or the AWP mode only under the condition that the size parameter of the current block meets the size requirement.

[0163] For example, in this application, there may be a frame-level flag that determines whether the frame to be decoded currently uses this application. For example, it may be set so that intra frames (e.g., I frames) use this application and inter frames (e.g., B frames, P frames) do not use this application. Or it may be set so that intra frames do not use this application and inter frames use this application. Or it may be set so that some inter frames use this application and some inter frames do not use this application. Since inter frames may use intra prediction, it is possible that inter frames use this application.

[0164] In some embodiments, there may also be a flag below the frame level and above the CU level (eg, tile, slice, patch, LCU, etc.) that determines whether to use the present application in a region.

[0165] S102, determining a first weight derivation mode and K first prediction modes included in a first combination based on a first index.

[0166] The decoding side decodes the codestream to obtain a first index, and then determines a first weight derivation mode and K first prediction modes included in a first combination based on the first index.

[0167] In this application, the weight derivation mode is used to determine the weight used for the current block. Specifically, the weight derivation mode may be a mode for deriving weights. For a block of a given length and width, each kind of weight derivation mode can derive one weight matrix, and for blocks of the same size, the weight matrices derived by different weight derivation modes are different.

[0168] Illustratively, in this application, the AWP has 56 types of weight derivation modes, and the GPM has 64 types of weight derivation modes.

[0169] The K different first prediction modes included in the first combination include the following examples.

[0170] In example 1, the K different first prediction modes are all intra prediction modes, for example, the current block is an intra coding block and is not applicable to coding of screen content.

[0171] In example 2, the K different first prediction modes are all inter prediction modes, for example, the current block is an inter coded block.

[0172] Example 3: among the K different first prediction modes, at least one is an intra prediction mode, and at least one is an inter prediction mode.

[0173] Example 4: Among the above K different first prediction modes, at least one is an intra prediction mode, and at least one is a prediction mode that is neither inter nor intra, such as an intra block copy IBC prediction mode or a palette prediction mode.

[0174] Example 5: Among the K different first prediction modes, at least one is an inter prediction mode, and at least one is a prediction mode that is neither inter nor intra, such as an IBC prediction mode or a palette prediction mode.

[0175] Example 6: None of the K different first prediction modes are intra prediction modes or inter prediction modes. For example, one is an IBC prediction mode and another is a palette prediction mode.

[0176] It should be clarified that the present embodiment does not limit the specific types of the K different first prediction modes included in the first combination.

[0177] In this embodiment, the decoding side determines the first weight derivation mode and K first prediction modes included in the first combination based on the first index, including but not limited to the following methods.

[0178] In the first method, the encoding side and the decoding side determine a first combination corresponding to a first index from a plurality of preset candidate combinations. For example, the plurality of preset candidate combinations include any number of candidate combinations, such as 2, 3, 4, 5, 6, 7, 8, 9, etc., including 8 candidate combinations, each candidate combination includes one weight derivation mode and K prediction modes, and the 8 candidate combinations have different identifiers (or indexes). Thus, after obtaining the first index, the decoding side queries the candidate combination corresponding to the first index in the plurality of candidate combinations, determines the candidate combination corresponding to the first index as the first combination, and further determines the weight derivation mode included in the candidate combination as the first weight derivation mode, and determines the K prediction modes included in the candidate combination as the K first prediction modes.

[0179] In the method 2, the decoding side and the encoding side each determine one list, and since the list includes a plurality of candidate combinations, the list is also called a candidate combination list, and a first combination corresponding to a first index is determined from the candidate combination list. At this time, the above S102 is as follows: A step S102-A of determining a candidate combination list; and Step S102-B of determining a weight derivation mode and the K prediction modes included in a candidate combination corresponding to a first index in the candidate combination list as a first weight derivation mode and the K first prediction modes.

[0180] The candidate combination list includes a plurality of candidate combinations, and any one of the plurality of candidate combinations includes one weight derivation mode and K prediction modes.

[0181] Exemplarily, a list of candidate combinations is shown in Table 6.

[0182] Table 6 JPEG2025513075000048.jpg46155 As shown in Table 6, the candidate combination list includes a plurality of candidate combinations, and any two of the plurality of candidate combinations are not completely similar, that is, the weight derivation mode and at least one of the K prediction modes included in any two candidate combinations are different. For example, the weight derivation mode in the candidate combination 1 is different from the weight derivation mode in the candidate combination 2, or the weight derivation mode in the candidate combination 1 is the same as the weight derivation mode in the candidate combination 2, and at least one of the K prediction modes is different, or the weight derivation mode in the candidate combination 1 is different from the weight derivation mode in the candidate combination 2, and at least one of the K prediction modes is different.

[0183] For example, the order of the candidate combination in the candidate combination list in Table 6 above is used as an index, and optionally, the index of the candidate combination in the candidate combination list may be implemented in other ways, and the embodiment of the present application is not limited thereto.

[0184] In the method 2, the decoding side decodes the codestream, obtains a first index, and determines the candidate combination list shown in Table 6 above, queries the candidate combination list based on the first index, and obtains a first weight derivation mode and K prediction modes included in the first combination indicated by the first index.

[0185] For example, the first index is index 1, and in the candidate combination list shown in Table 6, the candidate combination corresponding to index 1 is candidate combination 2, that is, the first combination indicated by the first index is candidate combination 2. In this way, the decoding side determines the weight derivation mode and K prediction modes included in candidate combination 1 as the first weight derivation mode and K first prediction modes included in the first combination, and predicts the current block using the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block.

[0186] In this method 2, the encoding side and the decoding side can respectively determine the same candidate combination list, for example, the encoding side and the decoding side each determine one list including N candidate combinations, and each candidate combination includes one weight derivation mode and K prediction modes. In the code stream, the encoding side only needs to write one final selected candidate combination, for example the first combination, and the decoding side analyzes the first combination finally selected by the encoding side, specifically, the decoding side decodes the code stream, obtains a first index, and determines the first combination from the candidate combination list determined by the decoding side according to the first index.

[0187] In the present embodiment, there is no restriction on the specific manner in which the decoding side determines the candidate combination list.

[0188] In some embodiments, the candidate combination list is a pre-existing one, and the decoding side can decode the codestream to obtain a first index, and then obtain or read the candidate combination list based on the first index, and further query the candidate combination corresponding to the first index in the candidate combination list based on the first index.

[0189] In some embodiments, the candidate combination list is transmitted by the encoding side to the decoding side, for example by the encoding side before encoding the current block.

[0190] In some embodiments, the candidate combination list is uploaded to the cloud by the encoding side, and the decoding side reads the candidate combination list from the cloud after decoding the codestream and obtaining the first index.

[0191] In some embodiments, the candidate combination list is constructed by a decoding side, for example, the decoding side decodes a codestream to obtain a first index, and then constructs the candidate combination list in response to the first index.

[0192] In the embodiment of the present application, the decoding side does not limit the manner of constructing the candidate combination list, for example, by using information related to the current block to analyze the generation probability of each combination of different weight derivation modes and different prediction modes, and constructs the candidate combination list according to the generation probability of each combination.

[0193] Optionally, the information related to the current block includes mode information of surrounding blocks of the current block, reconstructed pixels of the current block, and so on.

[0194] In some embodiments, the decryption side: A candidate combination list is constructed in step S102-A1 of constructing a candidate combination list based on the template of the current block.

[0195] Template matching utilizes the correlation between neighboring pixels and uses a part of the surrounding area of ​​the current block as a template. When the current block is coded, its left and upper sides are already coded according to the coding order. During inter prediction, the best matching position of the template is found in the reference frame, and the motion information or motion vector of the current block is determined accordingly. During intra prediction, the template is used to determine the intra prediction mode of the current block.

[0196] The template of the current block is composed of a reconstructed region around the current block and is associated with the current block, and based on this, the embodiment of the present invention constructs a candidate combination list based on the template of the current block.

[0197] This application does not limit the specific shape of the template of the current block.

[0198] In some embodiments, the template includes at least one of the above coded region, the left coded region, and the top-left coded region of the current block.

[0199] Optionally, the width of the upper coded area is the same as the width of the current block, the height of the left coded area is the same as the height of the current block, the width of the upper left coded area is the same as the width of the left coded area, and the height of the upper left coded area is the same as the height of the upper coded area.

[0200] For example, for each combination, predict a template of the current block using the combination, obtain a predicted value of the template under each combination, and build a candidate combination list based on the predicted value of the template under each combination.

[0201] For each combination, predict the template of the current block using the combination, obtain a predicted value of the template for each combination, specifically, derive a weight of the template using a weight derivation mode included in the combination, predict the template using each of the K prediction modes included in the combination to obtain K predicted values ​​of the template, and perform a weighting process on the K predicted values ​​of the template based on the derived weight of the template to obtain a predicted value of the template for the combination. Finally, construct a candidate combination list based on the predicted value of the template for each combination.

[0202] It should be noted that the weight derived according to the weight derivation mode can be understood as a weight corresponding to each pixel point in the derived template, or as a weight matrix corresponding to the derived template. When determining the predicted value of the template according to the weight, K predicted values ​​corresponding to each pixel point in the template can be determined, and the predicted value corresponding to each pixel point can be determined according to the K predicted values ​​corresponding to each pixel point and the weight, and the predicted value corresponding to each pixel point in the template constitutes the predicted value of the template. Optionally, when determining the predicted value of the template according to the weight, it can be further performed according to a block, for example, determining the predicted value of the template, and weighting the K predicted values ​​of the template according to the weight matrix of the template to obtain the predicted value of the template.

[0203] In some embodiments, S102-A1 is a step S102-A11 of determining R second combinations, where any one of the R second combinations includes one weight derivation mode and K prediction modes, and the weight derivation mode and the K prediction modes included in any two of the R second combinations are not completely similar, and R is a positive integer greater than 1; A step S102-A12 of predicting a template using the weight derivation mode and the K prediction modes in the second combination for any one of the R second combinations to obtain a template prediction value corresponding to the second combination; and constructing a candidate combination list based on template prediction values ​​corresponding to each of the R second combinations.

[0204] In this embodiment, when the decoding side constructs a candidate combination list, it first determines R second combinations, and the specific value of the R second combinations is not limited in this application, for example, 8, 16, 32, etc., and each of the R second combinations includes one weight derivation mode and K prediction modes, and the weight derivation mode and the K prediction modes included in any two combinations of the R second combinations are not completely similar. Then, for each of the R second combinations, it predicts the template of the current block using the weight derivation mode and the K prediction modes included in the second combination, and obtains a predicted value of the template in the second combination. Finally, it constructs a candidate combination list based on the template predicted value corresponding to each of the R second combinations.

[0205] In the above S102-A13, constructing a candidate combination list based on the template prediction value corresponding to each second combination among the R second combinations includes, but is not limited to, the following manners.

[0206] Method 1: construct a candidate combination list based on the magnitude of the template prediction value corresponding to each second combination among the R second combinations.

[0207] For example, the R second combinations are ordered based on the magnitude of the template prediction value corresponding to each of the R second combinations, and the ordered R second combinations are determined as a candidate combination list, where the candidate combination list includes the R candidate combinations.

[0208] Further, for example, the R second combinations are ordered based on the magnitude of the template prediction value corresponding to each of the R second combinations, and N second combinations are selected from the ordered R second combinations to construct a candidate combination list, where the candidate combination list includes the N candidate combinations.

[0209] Method 2, S102-A13 above, a step S102-A13-1 of determining a cost corresponding to the second combination based on the template prediction value and the template reconstruction value corresponding to the second combination; and constructing a candidate combination list based on a cost corresponding to each of the R second combinations.

[0210] Since the template of the current block is a reconstructed region, the decoding side can obtain the reconstruction value of the template, and thus, for each second combination among the R second combinations, the prediction distortion cost corresponding to the second combination can be determined based on the predicted value of the template and the reconstruction value of the template in the second combination. The manner of determining the cost corresponding to the second combination includes, but is not limited to, SAD, SATD, SEE, etc. Then, a candidate combination list is constructed based on the cost corresponding to each second combination among the R second combinations.

[0211] The manner of constructing the candidate combination list based on the cost corresponding to each of the R second combinations in S102-A13-2 includes, but is not limited to, the following examples.

[0212] Example 1: Ranking the R second combinations based on the cost corresponding to each of the R second combinations, and determining the ordered R second combinations as a candidate combination list.

[0213] The candidate combination list generated in the first example includes R candidate combinations.

[0214] Optionally, the R candidate combinations in the candidate combination list are ordered in ascending order of cost, ie, the costs corresponding to the R candidate combinations in the candidate combination list increase in the ordered order.

[0215] Ordering the R second combinations based on the cost corresponding to each second combination among the R second combinations orders the R second combinations in order of decreasing cost.

[0216] Example 2: N second combinations are selected from the R second combinations based on the costs corresponding to the second combinations, and the list of the N second combinations is determined as a candidate combination list.

[0217] Optionally, the N second combinations are N second combinations with the smallest cost among the R second combinations, for example, based on the cost corresponding to each second combination among the R second combinations, the N second combinations with the smallest cost are selected from the R second combinations to form a candidate combination list, where the candidate combination list includes the N candidate combinations.

[0218] Optionally, the N candidate combinations in the candidate combination list are ordered in ascending order of cost, ie, the costs corresponding to the N candidate combinations in the candidate combination list increase in the ordered order.

[0219] The process of determining the R second combinations in S102-A11 will be described below.

[0220] In some embodiments, the R second combinations are preset. Thus, a template of the current block is predicted using each of the R second combinations preset, a template prediction value corresponding to each of the second combinations is obtained, a cost corresponding to each of the second combinations is determined according to the template prediction value and the template reconstruction value corresponding to each of the second combinations, and the R second combinations are ordered according to the cost corresponding to each of the second combinations, and the ordered R second combinations are set as a candidate combination list, or N second combinations with the smallest cost are selected from the ordered R second combinations to form the candidate combination list.

[0221] In some embodiments, S102-A11 is a step S102-A11-1 of determining P weight derivation modes and Q prediction modes, where P is a positive integer and Q is a positive integer equal to or greater than K; and a step S102-A11-2 of constructing R second combinations based on the P weight derivation modes and the Q prediction modes, where any one of the R second combinations includes one weight derivation mode of the P weight derivation modes and K prediction modes of the Q prediction modes, where P are all positive integers and Q is a positive integer greater than or equal to K.

[0222] In this embodiment, the decoding side first determines P weight derivation modes and Q prediction modes, and then constructs R second combinations based on the determined P weight derivation modes and Q prediction modes.

[0223] To explain by way of example, let us take the second combination as including one weight derivation mode and two prediction modes, where the P weight derivation modes are weight derivation mode 1 and weight derivation mode 2, and the Q prediction modes are prediction mode 1, prediction mode 2, and prediction mode 3. The two weight derivation modes and three prediction modes can be combined into 2×3×2 second combinations, totaling 12.

[0224] In the present embodiment, there is no specific limit to the number of the P weight derivation modes and the Q prediction modes.

[0225] In one possible implementation, the current block is an intra-coded block, where there are 64 possible weight derivation modes of the GPM, and there are 67 possible intra prediction modes of the GPM, which can be found in the VVC standard. However, this is not a limitation that there are only 64 possible weights of the GPM, or what kind of 64 types there are, and it should be noted that the selection of 64 types of GPM of VVC is also a trade-off between improved prediction effect and increased overhead in the code stream. In the present application, since the weight derivation mode is no longer coded using one fixed logic, theoretically, the present application can use more diverse weights and more flexibly. Similarly, it is not a limitation that there are only 67 intra prediction modes of the GPM, or what kind of 67 types there are. In theory, all possible intra prediction modes can be used for the GPM. For example, the intra angle prediction mode can be made more fine-grained, and more intra angle prediction modes can be generated, and the GPM can also use more intra angle prediction modes. For example, regarding the MIP (matrix-based intra prediction) mode of VVC, this application may use it, but considering that MIP can select multiple sub-modes, here, for easy understanding, MIP is not added to this embodiment.In addition, there are also some wide-angle modes, which can also be used in this application, and this embodiment will not describe them further.

[0226] Assuming that K=2, the K prediction modes include a first prediction mode and a second prediction mode, and there are 67 types of available prediction modes (i.e., Q=67), there are 67 possible first prediction modes, and since the second prediction mode is different from the first prediction mode, there are 66 types of second prediction modes, and there are 64 types of weight derivation modes (i.e., P=64), in the present application, one second combination is formed with any two different prediction modes and any one weight derivation mode, and there are a total of 64*67*66 types of possible second combinations.

[0227] In this implementation, the P weight derivation modes are all possible weight derivation modes, for example 64 weight derivation modes in GPM, and the Q prediction modes are all possible prediction modes, for example 67 intra prediction modes in GPM, and an enumeration method is used to obtain all possible second combinations, for example 64*67*66 possible second combinations, and each second combination among the 64*67*66 possible second combinations is used to predict the template of the current block, and the distortion cost of each second combination is calculated, and a candidate combination list corresponding to the current block is obtained based on the distortion cost of each second combination.

[0228] In some embodiments, in order to reduce the amount of data and improve the speed of building the candidate combination list, instead of trying every type of prediction mode, some prediction modes may be selected and tried.

[0229] At this time, the implementation form of determining the Q prediction modes in the above S102-A11-1 includes, but is not limited to, the following methods.

[0230] In the first method, the Q prediction modes are preset prediction modes.

[0231] Method 2: determine at least one of a candidate prediction mode list for a current block, an alternative prediction mode list corresponding to each of the K prediction modes, and a prediction mode corresponding to a weight derivation mode, and determine Q prediction modes based on at least one of the candidate prediction mode list, the alternative prediction mode list corresponding to each of the K prediction modes, and a prediction mode corresponding to the weight derivation mode.

[0232] The candidate prediction mode list includes a plurality of candidate prediction modes, and the alternative prediction mode list corresponding to any one of the K prediction modes includes at least one alternative prediction mode.

[0233] For example, Q prediction modes are determined based on the candidate prediction mode list of the current block.

[0234] Also, for example, Q prediction modes are determined based on the alternative prediction mode lists corresponding to the K prediction modes, respectively.

[0235] Also, for example, the Q prediction modes are determined based on the prediction mode corresponding to the weight derivation mode.

[0236] Also, for example, the Q prediction modes are determined based on the candidate prediction mode list of the current block and the alternative prediction mode lists corresponding to the K prediction modes, respectively.

[0237] Also, for example, the Q prediction modes are determined based on the candidate prediction mode list of the current block and the prediction mode corresponding to the weight derivation mode.

[0238] Also, for example, the Q prediction modes are determined based on the alternative prediction mode lists corresponding to the K prediction modes and the prediction mode corresponding to the weight derivation mode.

[0239] Also, for example, the Q prediction modes are determined based on the candidate prediction mode list, the alternative prediction mode lists corresponding to each of the K prediction modes, and the prediction mode corresponding to the weight derivation mode.

[0240] Determining the alternative prediction mode list corresponding to each of the K prediction modes may be understood as determining one alternative prediction mode list for each prediction mode among the K prediction modes, and when subsequently constructing a second combination, a certain prediction mode in the second combination is selected from the alternative prediction mode list corresponding to the prediction mode. For example, K=2, the K prediction modes include a first prediction mode and a second prediction mode, and the decoding side constructs one alternative prediction mode list 1 for the first prediction mode and one alternative prediction mode list 2 for the second prediction mode. In this way, when subsequently constructing a different second combination, one alternative prediction mode is selected from the alternative prediction mode list 1 as the first prediction mode and one alternative prediction mode is selected from the alternative prediction mode list 2 as the second prediction mode, and thus one weight derivation mode and the currently selected first prediction mode and second prediction mode constitute one second combination.

[0241] In the embodiment of the present application, there is no limitation on the method of determining the alternative prediction mode list corresponding to each prediction mode among the K prediction modes.

[0242] In one possible implementation form, for any one of the K prediction modes, a candidate prediction mode list corresponding to the prediction mode and at least one of the prediction modes corresponding to the weight derivation mode are determined, and an alternative prediction mode list corresponding to the prediction mode is determined based on the candidate prediction mode list and at least one of the prediction modes corresponding to the weight derivation mode.

[0243] In this embodiment, the process of determining a candidate prediction mode list corresponding to a prediction mode among the K prediction modes is almost similar to the process of determining a candidate prediction mode list corresponding to the current block, and specifically, the following can be referred to.

[0244] In some embodiments, the candidate prediction mode list includes one or more inter prediction modes, such as at least one of skip, merge, general inter prediction mode, unidirectional prediction, bidirectional prediction, multiple hypothesis prediction, etc.

[0245] In some embodiments, the candidate prediction mode list includes one or more intra-prediction modes, such as at least one of a direct current (DC) mode, a planar mode, an angle mode, etc. Optionally, the candidate prediction mode list includes at least one intra-prediction mode in an MPM list.

[0246] In some embodiments, the candidate prediction mode list may further include modes such as IBC, palette, and the like.

[0247] In this application, there is no limitation on the types and number of prediction modes included in the candidate prediction mode list.

[0248] In some embodiments, the candidate prediction mode list is determined in at least one of the following ways:

[0249] Method 1: The candidate prediction mode list includes preset modes.

[0250] Method 2: The candidate prediction mode list includes the modes in the MPM list.

[0251] In some embodiments, the candidate intra-prediction mode list may be an MPM list of a current block, for example, in VVC, the current block may obtain an MPM list of length 6. In addition, in some subsequent technological advances, there is a scheme called secondary MPM that can derive an MPM list of length 22, that is, the sum of the length of the first MPM list and the length of the second MPM list is 22. That is, the embodiments of the present application may use MPM to screen intra-prediction modes.

[0252] In some embodiments, if the determined candidate prediction mode list does not include the preset mode, the preset mode is added to the candidate prediction mode list.

[0253] Optionally, the pre-set mode includes at least one of a DC, a horizontal mode, and a vertical mode.

[0254] Optionally, a pre-set mode may be added under certain conditions, in one example when the number of prediction modes among the candidate prediction modes is less than or equal to a certain threshold, which may be 3, 4, 5, 6, etc.

[0255] Method 3: The candidate prediction mode list includes a set of candidate prediction modes that are determined based on some rules, such as equal interval screening.

[0256] Method 4: determining a candidate prediction mode list based on a prediction mode used for at least one block adjacent to the current block.

[0257] For example, a prediction mode used in one or more blocks adjacent to the current block is added to a candidate prediction mode list to obtain a candidate prediction mode list for the current block, or a candidate prediction mode list corresponding to a kth prediction mode among the K prediction modes, where the kth prediction mode is any one of the K prediction modes.

[0258] Also, for example, a prediction mode corresponding to the k-th prediction mode among prediction modes used for one or more blocks adjacent to the current block is added to the candidate prediction mode list corresponding to the k-th prediction mode. For example, if K=2, the k-th prediction mode is the first prediction mode, and two blocks adjacent to the current block also perform weighted prediction using two prediction modes, the two prediction modes used for the first adjacent block are prediction mode 1 and prediction mode 3, and the two prediction modes used for the second adjacent block are prediction mode 4 and prediction mode 5, respectively, and thus prediction mode 1 and prediction mode 4 can be added to the candidate prediction mode list corresponding to the first prediction mode.

[0259] It should be noted that the above methods 1 to 4 may be used alone or in any combination as methods for determining the candidate prediction mode list.

[0260] In some embodiments, the candidate prediction mode list includes at least one of a candidate intra-prediction mode list and a candidate inter-prediction mode list, where the candidate intra-prediction mode list includes at least one candidate intra-prediction mode and the candidate inter-prediction mode list includes at least one candidate inter-prediction mode.

[0261] Based on the above method, a candidate prediction mode list for the current block is obtained, and then Q prediction modes are determined based on the candidate prediction mode list, for example, all or some of the candidate prediction modes included in the candidate prediction mode list are determined as all or some of the Q prediction modes.

[0262] The process of determining a prediction mode corresponding to a weight derivation mode will be described below.

[0263] In the present embodiment, the prediction mode corresponding to the weight derivation mode does not mean to indicate a specific one, and may be, for example, a prediction mode corresponding to one preset weight derivation mode, or a prediction mode corresponding to a plurality of preset weight derivation modes. In some embodiments, the prediction mode corresponding to the weight derivation mode may be understood as a prediction mode list corresponding to the weight derivation mode, and the prediction mode list includes at least one prediction mode.

[0264] In some embodiments, the prediction mode corresponding to the weight derivation mode includes a prediction mode corresponding to at least one weight derivation mode of the P weight derivation modes, in which determining the prediction mode corresponding to the weight derivation mode includes: for a p-th weight derivation mode of the P weight derivation modes, determining a prediction mode corresponding to the p-th weight derivation mode, and determining the prediction mode corresponding to the weight derivation mode based on the prediction mode corresponding to the at least one weight derivation mode of the P weight derivation modes, where p is a positive integer.

[0265] In this embodiment, the process of determining a prediction mode corresponding to each weight derivation mode among the P weight derivation modes is almost the same, and for convenience of explanation, the following description will be given using the p-th weight derivation mode among the P weight derivation modes as an example.

[0266] Determining the prediction mode corresponding to the p-th weight derivation mode includes the following two methods.

[0267] Method 1: When at least one of the prediction modes corresponding to the p-th weight derivation mode is an intra prediction mode, an angle index is determined based on the p-th weight derivation mode, and the intra prediction mode corresponding to the angle index is determined as at least one of the prediction modes corresponding to the p-th weight derivation mode.

[0268] The angle index is used to indicate the boundary angle index of the weight.

[0269] In some embodiments, the field angleIdx represents an angle index.

[0270] The above Table 2 shows the correspondence between merge_gpm_partition_idx and angleIdx. By referring to the above Table 2, the angle index can be derived based on the p-th weight derivation mode.

[0271] In the present application, the angle index and the intra prediction mode have a corresponding relationship, that is, different angle indexes correspond to different intra prediction modes.

[0272] For example, the angle index and the intra prediction mode have the corresponding relationship shown in Table 7.

[0273] Table 7 JPEG2025513075000049.jpg41139 In this method 1, taking K=2 as an example, when the first prediction mode or the second prediction mode is an intra prediction mode, the angle index is determined based on the p-th weight derivation mode, for example, the angle index corresponding to the p-th weight derivation mode is derived based on the above Table 2. Then, in the above Table 7, the intra prediction mode corresponding to the angle index is determined, for example, the angle index is 2, the corresponding intra prediction mode is 42, and the intra prediction mode 42 is determined as the first prediction mode or the second prediction mode.

[0274] Method 2: When at least one of the prediction modes corresponding to the p-th weight derivation mode is an intra prediction mode, an intra prediction mode corresponding to the p-th weight derivation mode is determined, and at least one of the intra prediction modes corresponding to the p-th weight derivation mode is determined as at least one of the prediction modes corresponding to the p-th weight derivation mode.

[0275] The intra prediction mode corresponding to the p-th weight derivation mode includes at least one of an intra prediction mode in which the prediction direction is parallel to the boundary line of the weights, an intra prediction mode in which the prediction direction is perpendicular to the boundary line, and a planar mode.

[0276] It should be noted that the intra prediction modes with a prediction direction parallel to the weight boundary include one or more intra prediction modes with a prediction direction parallel or nearly parallel to the weight boundary, and the intra prediction modes with a prediction direction perpendicular to the weight boundary include one or more intra prediction modes with a prediction direction perpendicular or nearly perpendicular to the weight boundary.

[0277] In the method 2, taking K=2 as an example, when the first prediction mode and / or the second prediction mode are intra prediction modes, the first prediction mode and / or the second prediction mode are determined from the intra prediction mode corresponding to the weight derivation mode. For example, the first prediction mode and / or the second prediction mode may be an intra prediction mode whose prediction direction is the same straight line as or approximately the same straight line as the weight partition line (also called a boundary line). Or, the first prediction mode and / or the second prediction mode may be an intra prediction mode whose prediction direction is perpendicular or approximately perpendicular to the weight boundary line. For example, when the weight boundary line is horizontal, such as the modes with GPM indexes of 18, 19, 50, and 51 in FIG. 4, the first prediction mode and / or the second prediction mode is the horizontal mode 18 or the vertical mode 50.

[0278] The decoding side determines a prediction mode corresponding to at least one of the P weight derivation modes based on the above steps, and further determines a prediction mode corresponding to the weight derivation mode based on the prediction mode corresponding to the at least one of the P weight derivation modes. For example, all or a part of the prediction modes corresponding to the at least one of the P weight derivation modes are set as the prediction modes corresponding to the weight derivation modes.

[0279] Furthermore, there may be overlapping prediction modes among the prediction modes corresponding to the P weight derivation modes, and the overlapping prediction modes are removed and another different prediction mode is determined as the prediction mode corresponding to the weight derivation mode.

[0280] In the embodiment of the present application, in order to reduce the number of the R second combinations, the prediction modes are screened, specifically, the Q prediction modes are determined based on the above method.

[0281] In some embodiments, in order to reduce the complexity on the decoding side, the number of the Q prediction modes is limited, for example, Q is less than or equal to a first preset threshold. In the present application, the specific value of the first preset threshold is not limited, and can be determined according to actual needs, for example, the first preset threshold is 6, that is, 6 prediction modes are selected to construct R second combinations, thereby controlling the number of second combinations.

[0282] In some embodiments, the magnitude of Q may be understood to be related to the size and / or shape of the current block, where the shape of the current block is determined by the aspect ratio of the current block.

[0283] During prediction, for small blocks, the difference in the impact of similar prediction modes on the prediction result is not large, while for large blocks, the difference in the impact of similar prediction modes on the prediction result is more obvious. Based on this, in the embodiment of the present application, different Q values ​​are set for blocks of different sizes, i.e., a large Q value is set for a large block, and a small Q value is set for a small block.

[0284] At this time, when determining the Q value corresponding to the current block, it is set based on the size of the current block, for example, when the size of the current block is larger than the first numerical value, Q is equal to or larger than the second preset threshold, and for example, when the size of the current block is equal to or smaller than the first numerical value, Q is smaller than the third preset threshold. In the embodiment of the present application, the specific magnitudes of the first numerical value, the second prediction threshold, and the third preset threshold are not limited, and the third preset threshold is smaller than the second preset threshold.

[0285] The process of determining the P weight derivation modes in S102-A11-1 will be described below.

[0286] As can be seen from the above, GPM includes 64 types of weight derivation modes, and AWP includes 56 types of weight derivation modes.

[0287] In some embodiments, the P weight derivation modes are the 64 weight derivation modes included in the GPM or the 56 weight derivation modes included in the AWP.

[0288] In some embodiments, P weight derivation modes are selected from M preset weight derivation modes, where M is a positive integer equal to or greater than P.

[0289] The M pre-set weight derivation modes may be 64 types of weight derivation modes included in the GPM, or may be 56 types of weight derivation modes included in the AWP, or may be a portion of the 64 types of weight derivation modes included in the GPM, or may be a portion of the 56 types of weight derivation modes included in the AWP.

[0290] In this embodiment, in order to further reduce the number of R second combinations, M preset weight derivation modes can be screened, and P weight derivation modes can be selected from the M preset weight derivation modes to construct R second combinations.

[0291] In some embodiments, weight derivation modes corresponding to a preset division angle and / or a preset offset amount are removed from the M weight derivation modes to obtain P weight derivation modes. The same division angle in the weight derivation modes may correspond to multiple offset amounts, such as weight derivation modes 10, 11, 12 and 13 shown in FIG. 16A, which have the same division angle but different offset amounts, and thus some of the weight derivation modes corresponding to the preset offset amount can be removed and / or some of the weight derivation modes corresponding to the preset division angle can be removed. In this way, the total number of possible second combinations can be reduced. And the difference between each possible second combination can be more clearly seen.

[0292] In some embodiments, the screening conditions corresponding to different blocks may be different, and thus when determining the P weight derivation modes corresponding to the current block, first determine the screening conditions corresponding to the current block, and then select the P weight derivation modes from the M weight derivation modes based on the screening conditions corresponding to the current block.

[0293] In some embodiments, the screening conditions corresponding to the current block include screening conditions corresponding to the size of the current block and / or screening conditions corresponding to the shape of the current block. During prediction, for small blocks, the difference in the impact of similar weight derivation modes on the prediction result is not large, while for large blocks, the difference in the impact of similar weight derivation modes on the prediction result is more obvious. Based on this, in the present embodiment, different P values ​​are set for blocks of different sizes, i.e., a large P value is set for large blocks, and a small P value is set for small blocks.

[0294] In some embodiments, the screening condition includes an array, the array including M elements, the M elements corresponding one-to-one to the M weight derivation modes, and an element corresponding to each weight derivation mode is used to indicate whether the weight derivation mode is usable or not, which may be understood as whether the weight derivation mode is usable for attempting to construct a subsequent second combination by one of the P weight derivation modes.

[0295] The above array may be a one-dimensional array or a two-dimensional array.

[0296] For example, taking GPM as an example, there are 64 weight derivation modes, and an array containing 64 elements is set, and the value of each element indicates whether the corresponding weight derivation mode is available or not. Taking one-dimensional numerical values ​​as an example, a specific example is as follows, and an array of g_sgpm_splitDir is set. g_sgpm_splitDir

[64] = { 1,1,1,0,1,0,1,0, 1,0,1,0,1,0,1,0, 1,0,1,1,1,0,1,0, 1,0,1,0,1,0,1,0, 0,0,0,0,1,1,0,1, 0,0,1,0,0,1,0,0, 1,0,1,1,0,1,0,0, 1,0,0,1,0,0,1,0 }, When the value of g_sgpm_splitDir[x] is 1, it indicates that the weight derivation mode for index x is available, and when the value is 0, it indicates that the weight derivation mode for index x is not available.

[0297] In some embodiments, when the screening conditions corresponding to the current block include screening conditions corresponding to the size of the current block and screening conditions corresponding to the shape of the current block, and for the same weight derivation mode, the screening conditions corresponding to the size of the current block and the screening conditions corresponding to the shape of the current block both indicate that the weight derivation mode is usable, the weight derivation mode is determined to be one of the P weight derivation modes, and when at least one of the screening conditions corresponding to the size of the current block and the screening conditions corresponding to the shape of the current block indicates that the weight derivation mode is not usable, it is determined that the weight derivation mode does not constitute one of the P weight derivation modes.

[0298] In some embodiments, multiple arrays may be used to implement screening conditions corresponding to different block sizes and screening conditions corresponding to different block shapes, respectively.

[0299] In some embodiments, screening conditions corresponding to different block sizes and screening conditions corresponding to different block shapes may be implemented using a two-dimensional array, i.e., the two-dimensional array includes screening conditions corresponding to block sizes and also includes screening conditions corresponding to block shapes.

[0300] As an example, the screening conditions corresponding to a block of size A and shape B are shown below, and the screening conditions are represented by a two-dimensional array.

[0301] g_sgpm_splitDir

[64] = { (1,1),(1,1),(1,1),(1,0),(1,0),(0,0),(1,0),(1,1), (1,1),(0,0),(1,1),(1,0),(1,0),(0,0),(1,0),(1,1), (0,1),(0,0),(1,1),(0,0),(1,0),(0,0),(1,0),(0,0), (1,1),(0,0),(0,1),(1,0),(1,0),(1,0),(1,0),(0,0), (0,0),(0,0),(1,1),(0,0),(1,1),(1,1),(1,0),(0,1), (0,0),(0,0),(1,1),(0,0),(1,0),(0,0),(1,0),(0,0), (1,0),(0,0),(1,1),(1,0),(1,0),(1,0),(0,0),(0,0), (1,1),(0,0),(1,1),(0,0),(0,0),(1,0),(1,1),(0,0) }, When all the values ​​of g_sgpm_splitDir[x] are 1, it indicates that the weight derivation mode with index x is available, and when one of the values ​​of g_sgpm_splitDir[x] is 0, it indicates that the weight derivation mode with index x is unavailable. For example, g_sgpm_splitDir[4]=(1,0) indicates that the weight derivation mode 4 is available for blocks with a block size of A but is unavailable for blocks with a shape of B, so when the block size is A and the shape is B, the weight derivation mode is unavailable.

[0302] After the decoding side determines the Q prediction modes and the P weight derivation modes based on the above steps, the decoding side configures R different second combinations based on the Q prediction modes and the P weight derivation modes.

[0303] In some embodiments, in order to further improve the construction speed of the candidate combination list, the decoding side re-screens the determined Q prediction modes and P weight derivation modes, and at this time, constructing R second combinations based on the P weight derivation modes and Q prediction modes determined in S102-A11-2 includes the following steps S102-A11-21 to S102-A11-23.

[0304] S102-A11-21, select S weight derivation modes from P weight derivation modes, where S is a positive integer less than or equal to P;

[0305] Specifically, weight derivation modes with low occurrence probability are removed from the P weight derivation modes determined above, and S weight derivation modes after screening are obtained.

[0306] The selection of S weight derivation modes from P weight derivation modes in the above S102-A11-21 includes, but is not limited to, the following manners.

[0307] Method 1: For an i-th weight derivation mode among P weight derivation modes, weights in a template of K second prediction modes are determined based on the i-th weight derivation mode, the K second prediction modes being any K prediction modes among Q prediction modes, where i is a positive integer from 1 to P, and if a weight in the template of any one of the K prediction modes is smaller than a first preset value, the i-th weight derivation mode is removed from the P weight derivation modes to obtain S weight derivation modes.

[0308] In the method 1, when the template weight derived by a certain weight derivation mode reduces the influence of the template of a certain prediction mode or eliminates the influence of the template, the weight derivation mode is not used. For example, the weight derivation mode 52 in FIG. 4 (square block) reduces the weight of the template of the second prediction mode, and the weight derivation mode 54, for example, reduces the weight of the template of the second prediction mode to 0, that is, the weight derivation mode 54 has no influence of the template of the second prediction mode, and the predicted value of the template is all determined by the first prediction mode. In this case, the second prediction mode has no influence, and such a weight derivation mode needs to be removed from the P weight derivation modes.

[0309] It should be noted that the same weight derivation mode of blocks with different shapes may have different effects of the two prediction modes. The current block may be square or rectangular, so the length may be longer than the width, or the width may be longer than the length, and the ratio may be 1:2, 1:4, etc. Figures 16A and 16B show the weight derivation modes in 32x64 and 64x32 blocks of GPM, and it can be seen that the intersection points between the boundary line and the block edge of the same weight derivation mode in different shapes are different. This is because the block shape changes, but the angle of the boundary line does not change according to the change in the block shape. As shown in Figure 16A, for a weight derivation mode with index 52, in a 32x64 block, when the boundary of the weight derivation mode with index 52 is extended into the template area of ​​the current block, the weight in the template of the second prediction mode is no longer smaller than the preset value due to the intersection with the template area, that is, it indicates that the second prediction mode has an effect on the template. However, as shown in FIG. 16B, for a 64x32 block with a weight derivation mode of index 52, extending the boundary of the weight derivation mode of index 52 into the template region of the current block will not intersect with the template region, resulting in a weight of 0 in the template for the second prediction mode, i.e., indicating no effect in the template for the second prediction mode.

[0310] In the above method 1, from the P weight derivation modes, weight derivation modes in which the weight in the template of any one of the K prediction modes is smaller than a first preset value are removed, and S weight derivation modes are obtained.

[0311] In the present embodiment, the first preset value is not limited to a specific value, and may be, for example, a small value equal to or greater than 0.

[0312] Method 2: For an i-th weight derivation mode among P weight derivation modes, a cost when predicting a template using the i-th weight derivation mode is determined, where i is a positive integer from 1 to P, and S weight derivation modes are selected from the P weight derivation modes based on the cost corresponding to the i-th weight derivation mode.

[0313] In the method 2, S weight derivation modes are selected from the P weight derivation modes by calculating a cost corresponding to each weight derivation mode among the P weight derivation modes.

[0314] In the present embodiment, the weight derivation mode and K prediction modes are treated as one combination, and the cost is calculated. Thus, in order to facilitate the calculation, the costs of P weight derivation modes are calculated based on the given K prediction modes, that is, each of the P weight derivation modes is combined with the set K prediction modes to obtain P combinations, the costs corresponding to each combination of the P combinations are calculated, and the costs of the P weight derivation modes are further obtained.

[0315] For example, the K given prediction modes are prediction mode 1 and prediction mode 2, and for the i-th weight derivation mode among the P weight derivation modes, the i-th weight derivation mode, prediction mode 1, and prediction mode 2 form one combination, which is called combination i. The combination i is used to predict a template of the current block, and a predicted value of the template in the combination i is obtained. Based on the predicted value of the template in the combination i and the reconstructed value of the template, a prediction distortion cost corresponding to the combination i is determined, and the prediction distortion cost corresponding to the combination i is determined as the cost corresponding to the i-th weight derivation mode. In this way, a cost corresponding to any one of the P weight derivation modes can be determined.

[0316] Based on the above method, a cost corresponding to the i-th weight derivation mode among the P weight derivation modes is determined, and then S weight derivation modes are selected from the P weight derivation modes based on the cost corresponding to the i-th weight derivation mode.

[0317] In the method 2, the decoding side selects S weight derivation modes from P weight derivation modes based on the cost corresponding to the i-th weight derivation mode, including the following method.

[0318] In the first method, when the cost corresponding to the i-th weight derivation mode is smaller than a second preset value, a weight derivation mode similar to the i-th weight derivation mode is selected from the P weight derivation modes, and S weight derivation modes are determined based on the i-th weight derivation mode and the similar weight derivation modes of the i-th weight derivation mode. The similar weight derivation mode of the i-th weight derivation mode may be understood as a weight derivation mode whose prediction result is similar to the prediction result of the i-th weight derivation mode, for example, the similar weight derivation mode of the i-th weight derivation mode includes a weight derivation mode whose partition angle is close to the partition angle of the i-th weight derivation mode and / or a weight derivation mode whose offset amount is close to the offset amount of the i-th weight derivation mode, and the offset amount of the weight derivation mode may be understood as an intercept of the boundary line of the weight derivation mode at the side of the current block.

[0319] For example, the above-mentioned segment angles being close may include the segment angles being the same and the offset amounts being different, for example, weight derivation mode 11 and weight derivation mode 13 in FIG. 4, and may further include the segment angles being the same and the offset amounts being similar, for example, weight derivation mode 11 and weight derivation mode 12 in FIG. 4, and may further include the segment angles being similar and the offset amounts being different, for example, weight derivation mode 9 and weight derivation mode 11 in FIG. 4, and may further include the segment angles being similar and the offset amounts being similar, for example, weight derivation mode 9 and weight derivation mode 12 in FIG. 4.

[0320] For example, the offset amount being close may include the offset amount being close and the segment angle being the same, for example, weight derivation mode 29 and weight derivation mode 30 in FIG. 4, and may further include the offset amount being the same or close and the segment angle being similar, for example, weight derivation mode 2 and weight derivation mode 38 in FIG. 4.

[0321] In some embodiments, a similar weight derivation mode of an i-th weight derivation mode may be understood as a weight derivation mode whose index is close to the index of the i-th weight derivation mode.

[0322] In the first method, it is shown that if the cost corresponding to the i-th weight derivation mode is smaller than a second preset value, the i-th weight derivation mode may be used to predict the current block, and in this case, the i-th weight derivation mode is selected from the P weight derivation modes to construct the subsequent R second combinations. In addition, since a weight derivation mode similar to the i-th weight derivation mode has similar characteristics to the i-th weight derivation mode, a weight derivation mode similar to the i-th weight derivation mode is selected from the P weight derivation modes to construct the subsequent R second combinations. Then, one weight derivation mode is selected from the other weight derivation modes of the P weight derivation modes as a new i-th weight derivation mode, and the above steps are repeated to obtain S weight derivation modes.

[0323] In the present embodiment, the magnitude of the second preset value is not limited, and is specifically determined according to actual needs.

[0324] In some embodiments, the decoding side may further select the S weight derivation modes from the P weight derivation modes based on the following second scheme.

[0325] In the second method, when the cost corresponding to the i-th weight derivation mode is greater than a third preset value, the i-th weight derivation mode and weight derivation modes similar to the i-th weight derivation mode are removed from the P weight derivation modes, and at least one weight derivation mode is obtained after the removal, and S weight derivation modes are determined based on the at least one weight derivation mode after the removal.

[0326] In the second method, when the cost corresponding to the i-th weight derivation mode is smaller than a third preset value, it is shown that the i-th weight derivation mode may not provide a good prediction effect when predicting the current block, and in this case, the i-th weight derivation mode is removed from the P weight derivation modes. In addition, since the weight derivation mode similar to the i-th weight derivation mode has similar characteristics to the i-th weight derivation mode, the weight derivation mode similar to the i-th weight derivation mode is also removed from the P weight derivation modes to obtain a weight derivation mode set after removal. Then, one weight derivation mode is selected as a new i-th weight derivation mode from the weight derivation mode set after removal, and the above steps are repeated to determine the weight derivation modes included in the weight derivation mode set finally obtained as the S weight derivation modes.

[0327] In the present embodiment, the magnitude of the third preset value is not limited, and is specifically determined according to actual demand, and the third preset value is greater than the second preset value.

[0328] Based on the above steps, the decoding side screens S weight derivation modes from P weight derivation modes, and then performs the following step S102-A11-22.

[0329] S102-A11-22 selects T prediction modes from the Q prediction modes, where T is a positive integer equal to or smaller than Q.

[0330] In the present embodiment, the method of selecting T prediction modes from Q prediction modes is not limited.

[0331] In some embodiments, T pre-defined prediction modes are selected from the Q prediction modes.

[0332] In some embodiments, for an i-th prediction mode among the Q prediction modes, the decoding side determines a cost when predicting a template using the i-th prediction mode, where i is a positive integer from 1 to Q, and selects T prediction modes from the Q prediction modes based on the cost corresponding to the i-th prediction mode.

[0333] In the present embodiment, the weight derivation mode and K prediction modes are treated as one combination, and the cost is calculated. Thus, in order to facilitate the calculation, the costs of Q weight derivation modes are calculated based on the given weight derivation mode and other K-1 prediction modes, that is, each of the Q prediction modes is combined with the set weight derivation mode and the K-1 prediction modes to obtain Q combinations, the costs corresponding to each combination among the Q combinations are calculated, and the costs of the Q prediction modes are further obtained.

[0334] For example, the given K-1 prediction modes are set as prediction mode 1, the given weight derivation mode is set as weight derivation mode 1, and for the i-th prediction mode among the Q prediction modes, the weight derivation mode 1, the i-th prediction mode, and the prediction mode 1 form one combination, which is set as combination i. The combination i is used to predict the template of the current block, and a predicted value of the template in the combination i is obtained. Based on the predicted value of the template in the combination i and the reconstructed value of the template, a prediction distortion cost corresponding to the combination i is determined, and the prediction distortion cost corresponding to the combination i is determined as the cost corresponding to the i-th prediction mode. In this way, the cost corresponding to any one of the Q prediction modes can be determined.

[0335] Based on the above method, a cost corresponding to the i-th prediction mode among the Q prediction modes is determined, and then T prediction modes are selected from the Q prediction modes based on the cost corresponding to the i-th prediction mode.

[0336] The decoding side selects T prediction modes from Q prediction modes based on the cost corresponding to the i-th prediction mode, including the following manner.

[0337] In the first method, when the cost corresponding to the i-th prediction mode is smaller than a fourth preset value, a prediction mode similar to the i-th prediction mode is selected from the Q prediction modes, and T prediction modes are determined based on the i-th prediction mode and the similar prediction mode of the i-th prediction mode. A prediction mode similar to the i-th prediction mode may be understood as a prediction mode whose prediction result is similar (or close) to the prediction result of the i-th prediction mode, for example, a prediction mode whose prediction direction (or angle) is close to the prediction direction (or angle) of the i-th prediction mode, or a prediction mode whose prediction mode index is close to the index of the i-th prediction mode, for example, a prediction mode whose index is 1, 2, etc. larger than the index of the i-th prediction mode, or a prediction mode whose index is 1, 2, etc. smaller than the index of the i-th prediction mode.

[0338] In the first method, when the cost corresponding to the i-th prediction mode is smaller than a fourth preset value, it is indicated that the i-th prediction mode may be used to predict the current block and provide a good prediction effect. In this case, the i-th prediction mode is selected from the Q prediction modes to construct the subsequent R second combinations. In addition, a prediction mode similar to the i-th prediction mode has similar characteristics to the i-th prediction mode, so a prediction mode similar to the i-th prediction mode is selected from the Q prediction modes to construct the subsequent R second combinations. Then, one prediction mode is selected from the other prediction modes of the Q prediction modes as a new i-th prediction mode, and the above steps are repeated to obtain another T prediction modes.

[0339] In the present embodiment, the magnitude of the fourth preset value is not limited, and is specifically determined according to actual needs.

[0340] In a second method, when a cost corresponding to the i-th prediction mode is greater than a fifth preset value, the i-th prediction mode and prediction modes similar to the i-th prediction mode are removed from the Q prediction modes, at least one prediction mode is obtained after the removal, and T prediction modes are determined based on the at least one prediction mode after the removal.

[0341] In the second method, when the cost corresponding to the i-th prediction mode is greater than a fifth preset value, it is possible that the i-th prediction mode is used to predict the current block, and in this case, the i-th prediction mode is removed from the Q prediction modes. In addition, since a prediction mode similar to the i-th prediction mode has similar characteristics to the i-th prediction mode, the similar prediction mode to the i-th prediction mode is also removed from the Q prediction modes to obtain a prediction mode set after removal. Then, one prediction mode is selected as a new i-th prediction mode from the prediction mode set after removal, and the above steps are repeated to finally obtain a prediction mode included in the prediction mode set, which is determined as the T prediction modes.

[0342] In the present embodiment, the magnitude of the fifth preset value is not limited, and is specifically determined according to actual needs, and the fifth preset value is greater than the fourth preset value.

[0343] Based on the above steps, select S weight derivation modes from the P weight derivation modes, select T prediction modes from the Q prediction modes, and then execute S102-A11-23 below.

[0344] S102-A11-23, constructing R second combinations based on the S weight derivation modes and the T prediction modes.

[0345] Specifically, one weight derivation mode is selected from the S weight derivation modes, and K prediction modes are selected from the T prediction modes, and this one weight derivation mode and the K prediction modes constitute one second combination, and the step is repeatedly performed to obtain R second combinations.

[0346] As can be seen from the above, one second combination includes one weight derivation mode and K prediction modes, and when screening prediction modes in this way, when the K elements in one combination are fixed, the possible selection of another element can be limited, for example, when K=2, when used with a combination of two elements, the possible selection of another element can be limited.

[0347] Hereinafter, the implementation process of the above S102-A11-2 will be described by taking the screening process of one prediction mode when the weight derivation mode and one prediction mode are fixed as an example.

[0348] In some embodiments, S102-A11-2 above includes: for an i-th weight derivation mode among the P weight derivation modes, determining a cost when predicting a template using the i-th weight derivation mode and a j-th prediction mode among the Q prediction modes; if the cost corresponding to the combination of the i-th weight derivation mode and the j-th prediction mode is greater than a sixth preset value, removing the j-th prediction mode and a similar prediction mode to the j-th prediction mode from the Q prediction modes to obtain at least one prediction mode after the removal; and constructing R second combinations based on the i-th weight derivation mode and the at least one prediction mode after the removal.

[0349] In this embodiment, when a weight derivation mode and one prediction mode are fixed, another prediction mode is screened. For example, when a certain intra prediction mode is set as the first prediction mode in a certain weight derivation mode, if a small cost cannot be obtained, the intra prediction mode similar to the certain intra prediction mode in the certain weight derivation mode is not tried to be set as the first prediction mode.

[0350] Specifically, for the i-th weight derivation mode among the P weight derivation modes, K=2, that is, the combination includes the i-th weight derivation mode, the first prediction mode and the second prediction mode. The second prediction mode is set as prediction mode 1, and the prediction mode 1 may be one of the Q prediction modes or may be other prediction modes than the Q prediction modes. Determine a possible selection of the first prediction mode from the Q prediction modes, specifically, the j-th prediction mode among the Q prediction modes is set as the first prediction mode, and in this case, when predicting a template with a combination j consisting of the i-th weight derivation mode, the j-th prediction mode and the prediction mode 1, determine a predicted value of the template, and determine a cost corresponding to the combination j based on the predicted value, and determine the cost corresponding to the combination j as the cost corresponding to the j-th prediction mode. Then, it is determined whether the cost corresponding to the jth prediction mode is greater than a sixth preset value. If the cost corresponding to the jth prediction mode is greater than the sixth preset value, it is indicated that the combination of the jth prediction mode, the ith weight derivation mode, and the prediction mode 1 cannot realize accurate prediction of the template. In this case, the jth prediction mode is removed from the Q prediction modes. Since a prediction mode similar to the jth prediction mode has similar characteristics to the jth prediction mode, the similar prediction mode of the jth prediction mode is removed from the Q prediction modes to obtain a prediction mode set after removal. One prediction mode is reselected from the prediction mode set after removal as a new jth prediction mode, and the above steps are repeated to obtain a final prediction mode set after removal corresponding to the ith weight derivation mode.

[0351] Based on the above steps, a final set of removed prediction modes corresponding to each weight derivation mode among the P weight derivation modes can be determined, and thus R second combinations are constructed based on the P weight derivation modes and their respective corresponding final set of removed prediction modes.

[0352] It should be noted that the above embodiment shows a method of screening prediction modes in the form of combinations, and optionally, any one of the weight derivation modes and prediction modes may be further screened in the form of combinations, and R second combinations may finally be constructed.

[0353] The decoding side determines R second combinations based on the above methods, and then, for any one of the R second combinations, uses the weight derivation mode in the second combination and the K prediction modes to predict a template and obtain a template prediction value corresponding to the second combination.

[0354] Hereinafter, a process of predicting a template of a current block using any one of the second combinations and obtaining a predicted value of the template will be described.

[0355] First, the weights of the templates are determined using the weight derivation mode included in the second combination.

[0356] Currently, as shown in FIG. 13 above, the template of the current block is the coded area above the current block, or the coded area to the left of the current block, or the coded area to the left and above the current block.

[0357] In some embodiments, the weight matrix may be directly extended to the template region, for example, extended to the upper left side, and the weight matrix may be overlaid on the template, thereby determining the weight of the template. For example, as shown in FIG. 17, a small rectangular region in the upper left region of the current block may be added to the template, and the template and the current block may be combined to form a rectangle. Of course, only the left and upper parts may be used as the template. As shown in FIG. 17, the upper left region is taken as an example, and the region of the inverted L-shaped block consisting of the left side, upper left side, and upper side is the template region, and the rectangular region on the lower right side is the current block, and the weight matrix extended to the upper left side becomes the weight matrix of the template.

[0358] In some embodiments, the process of deriving the template weight based on the weight derivation mode and the process of deriving the weight of the predictor are merged, for example, the template weight and the weight of the predictor are derived simultaneously. The weight of the predictor may be understood as a weight corresponding to a predictor, for example, a first predicted value is obtained using a first prediction mode, a second predicted value is obtained using a second prediction mode, a first weight of the first predicted value is determined based on the weight derivation mode, a second weight of the second predicted value is determined based on the weight derivation mode, and a sum value of the product of the first predicted value and the first weight and the product of the second predicted value and the second weight is determined as the predicted value of the current block.

[0359] In this application, the first and second weights are referred to as predictor weights to distinguish them from the template weights.

[0360] In some embodiments, determining the template weights based on the weight derivation mode includes: A step 1 of determining an angle index and a distance index based on a weight derivation mode; and step 2 determining a template weight based on the angle index, the distance index and the template magnitude.

[0361] In the present application, the template weight may be derived in a similar manner to the derivation of the weight of the predicted value. For example, first, an angle index and a distance index are determined based on a weight derivation mode, and the angle index may be understood as a boundary angle index of each weight derived by the weight derivation mode. Exemplarily, an angle index and a distance index corresponding to the weight derivation mode may be determined based on the above Table 2, for example, when the weight derivation mode is 27, the corresponding angle index is 12 and the distance index is 3. Then, the template weight is determined based on the angle index, the distance index and the size of the template.

[0362] Determining the template weight based on the angle index, distance index and template size in the above step 2 includes, but is not limited to, the following manners.

[0363] Method 1: Determine the template weight directly based on the angle index, distance index and template size. At this time, the above step 2 is: determining 21 first parameters of pixel points in the template based on the angle index, the distance index and the size of the template; determining 22 weights for pixel points in the template based on first parameters of the pixel points in the template; determining 23 template weights based on the weights of the pixel points in the template.

[0364] In this embodiment, weights of pixel points in the template are determined based on the angle index, the distance index, the size of the template, and the size of the current block, and a weight matrix consisting of the weights of each point in the template is determined as the template weight.

[0365] The first parameter herein is used to determine the weight, and in some embodiments, the first parameter is also referred to as a weight index.

[0366] In one possible implementation, the offset amount and the first parameter may be determined based on the following scheme.

[0367] As shown in FIG. 17, the inputs for the template weight derivation process include the width nCbW of the current block, the height nCbH of the current block, the width nTmW of the left template, the height nTmH of the upper template, the GPM “division” angle index variable angleId, the GPM distance index variable distanceIdx, and the component index variable cIdx. For example, in this application, since the luminance component is taken as an example, cIdx is 0, which represents the luminance component.

[0368] The variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor are derived in the following manner.

[0369] nW = ( cIdx = = 0 ) ? nCbW : nCbW * SubWidthC nH = ( cIdx = = 0 ) ? nCbH : nCbH * SubHeightC shift1 = Max( 5, 17 - BitDepth ), where BitDepth is the bit depth of the codec.

[0370] offset1 = 1 << ( shift1 - 1 ) displacementX = angleIdx displacementY = (angleIdx + 8) % 32 partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1 shiftHor = ( angleIdx % 16 = = 8 | | ( angleIdx % 16 != 0 && nH >= nW ) ) ? 0 : 1 The offset amounts offsetX and offsetY are derived as follows.

[0371] - If the value of shiftHor is 0, offsetX = ( -nW ) >> 1 offsetY = ( ( -nH ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nH ) >> 3 : -( ( distanceIdx * nH ) >> 3 ) ) - Otherwise (i.e., the value of shiftHor is 1), offsetX = ( ( -nW ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nW ) >> 3 : -( ( distanceIdx * nW ) >> 3 ) offsetY = ( - nH ) >> 1 Template weight matrix wTemplateValue[x][y] (x = -nTmW..nCbW - 1, y = -nTmH..nCbH - 1, except when both x and y are 0 or greater). However, in this example, the coordinates of the upper left corner of the current block are (0,0) and it is derived as follows:

[0372] The variables xL and yL are derived in the following way:

[0373] xL =( cIdx = = 0 ) ? x : x * SubWidthC yL =( cIdx = = 0 ) ? y : y * SubHeightC The disLut is determined according to Table 3 above.

[0374] The first parameter weightIdx is derived in the following manner.

[0375] weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ] After determining the first parameter weightIdx based on the above method, the weight of the point (x, y) in the template is determined based on weightIdx.

[0376] In the present application, determining the weight of the pixel point in the template based on the first parameter of the pixel point in the template in the above step 22 includes, but is not limited to, the following manners.

[0377] Method 1: determine a second parameter of the pixel point in the template based on a first parameter of the pixel point in the template, and determine a weight of the pixel point in the template based on the second parameter of the pixel point in the template.

[0378] A second parameter is also used to determine the weight. In some embodiments, the second parameter is also referred to as a weight index on the first component, which may be a luma component, a chroma component, etc.

[0379] For example, the weight of each pixel point in the template is determined by the following formula.

[0380] weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx wTemplateValue[x][y] = Clip3( 0, 8, ( weightIdxL + 4 ) >> 3 ) where wTemplateValue[x][y] is the weight of point (x,y) in the template, weightIdxL is the second parameter of point (x,y) in the template, also called the weight index in the first component (e.g., the luminance component), wTemplateValue[x][y] is the weight of point (x,y) in the template, and partFlip is an intermediate variable that is determined based on the angle index angleIdx, for example as described above, partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1, that is, the value of partFlip is 1 or 0, and when partFlip is 0, weightIdxL is 32 - weightIdx, and when partFlip is 1, weightIdxL is 32 + weightIdx, it should be noted that 32 here is just an example and the present application is not limited thereto.

[0381] Method 2: determining a weight of a pixel point in the template according to a first parameter of the pixel point in the template, a first threshold value and a second threshold value.

[0382] In order to reduce the calculation complexity of the template weight, in method 2, the weight of the pixel point in the template is limited to a first threshold or a second threshold, that is, the weight of the pixel point in the template is not the first threshold but the second threshold, which further reduces the calculation complexity of the template weight.

[0383] In the present application, the first threshold value and the second threshold value are not limited to specific values.

[0384] Optionally, the first threshold is one.

[0385] Optionally, the second threshold is zero.

[0386] In one example, the weights of pixel points in the template may be determined by the following formula:

[0387] wTemplateValue[x][y] =(partFlip ? weightIdx: - weightIdx) > 0 ? 1 : 0 wTemplateValue[x][y] is the weight of the point (x, y) in the template, and the 1 in the above "1:0" is the first threshold and the 0 is the second threshold.

[0388] In the above method 1, the weight of each point in the template is determined by the weight derivation mode, and a weight matrix consisting of the weights of each point in the template is defined as the template weight.

[0389] Method 2: Determine the weights of the current block and the template according to the weight derivation mode, that is, in method 2, the entire merged region consisting of the current block and the template is taken as one region, and the weights of the pixel points in the merged region are derived according to the weight derivation mode. Based on this, the above step 2 is as follows: Step 2-1: determining weights of pixel points in a merged region consisting of the current block and the template based on an angle index, a distance index, a size of the template, and a size of the current block; and step 2-2 determining template weights based on the size of the template and the weights of pixel points in the merge region.

[0390] In method 2, the current block and the template are treated as one region, and weights of pixel points in a merged region consisting of the current block and the template are determined based on the angle index, the distance index, the size of the template, and the size of the current block. Furthermore, based on the size of the template, a weight corresponding to the template in the merged region is determined as the template weight. For example, as shown in FIG. 17, a weight corresponding to an L-shaped template region in the merged region is determined as the template weight.

[0391] In the method 2, in the process of determining the weight, the template weight is determined and the prediction weight is also determined. That is, after removing the template weight from the weight corresponding to the merged region, the remaining weight is the prediction weight. Then, the subsequent prediction process can be performed based on the prediction weight, and there is no need to re-determine the prediction weight, which further reduces the prediction steps and improves the prediction efficiency.

[0392] This application does not limit the specific implementation of determining the weights of pixel points in the merged region consisting of the current block and the template based on the angle index, the distance index, the size of the template, and the size of the current block as described above.

[0393] In some embodiments, determining weights of pixel points in the merged region in step 2-1 above comprises: A step 2-11 of determining a first parameter of the pixel point in the merged region based on the angle index, the distance index and the size of the merged region; and determining weights of the pixel points in the merged region based on the first parameters of the pixel points in the merged region.

[0394] In this implementation, weights of pixel points in the merged region and a weight matrix consisting of weights of each point in the merged region are determined based on the angle index, the distance index, and the size of the merged region.

[0395] In one possible implementation, the offset amount and the first parameter may be determined based on the following scheme.

[0396] As shown in FIG. 17, the inputs to the process of deriving the merge region weight include the width nCbW of the current block, the height nCbH of the current block, the width nTmW of the left template, the height nTmH of the upper template, the GPM “division” angle index variable angleId, the GPM distance index variable distanceIdx, and the component index variable cIdx. For example, in this application, since the luminance component is taken as an example, cIdx is 0, which represents the luminance component.

[0397] The variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor are derived in the following manner.

[0398] nW = ( cIdx = = 0 ) ? nCbW : nCbW * SubWidthC nH = ( cIdx = = 0 ) ? nCbH : nCbH * SubHeightC shift1 = Max( 5, 17 - BitDepth ), where BitDepth is the bit depth of the codec.

[0399] offset1 = 1 << ( shift1 - 1 ) displacementX = angleIdx displacementY = (angleIdx + 8) % 32 partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1 shiftHor = ( angleIdx % 16 = = 8 | | ( angleIdx % 16 != 0 && nH >= nW ) ) ? 0 : 1 The offset amounts offsetX and offsetY are derived as follows.

[0400] - If the value of shiftHor is 0, offsetX = ( -nW ) >> 1 offsetY = ( ( -nH ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nH ) >> 3 : -( ( distanceIdx * nH ) >> 3 ) ) - Otherwise (i.e., the value of shiftHor is 1), offsetX = ( ( -nW ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nW ) >> 3 : -( ( distanceIdx * nW ) >> 3 ) offsetY = ( - nH ) >> 1 Weight matrix of pixel points in the merged region, wValueMatrix[x][y] (x = -nTmW..nCbW - 1, y = -nTmH..nCbH - 1). In this example, the coordinates of the upper left corner of the current block are (0,0) and are derived as follows:

[0401] The variables xL and yL are derived in the following way:

[0402] xL = ( cIdx = = 0 ) ? x : x * SubWidthC yL = ( cIdx = = 0 ) ? y : y * SubHeightC The disLut is determined according to Table 3.

[0403] The first parameter weightIdx is derived in the following manner.

[0404] weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ] After determining the first parameter weightIdx based on the above method, determine the weight of the point (x, y) in the merged region based on weightIdx.

[0405] In the present application, determining the weight of the pixel point in the merged region based on the first parameter of the pixel point in the merged region in the above step 2-12 includes, but is not limited to, the following manners.

[0406] Method 1: determine a second parameter of the pixel point in the merged region according to a first parameter of the pixel point in the merged region, and determine a weight of the pixel point in the merged region according to the second parameter of the pixel point in the merged region.

[0407] For example, the weights of the pixel points in the merged region are determined based on the following formula:

[0408] weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ] weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx wValueMatrix [x][y] = Clip3( 0, 8, ( weightIdxL + 4 ) >> 3 ) wValueMatrix[x][y] is the weight of the point (x,y) in the merged region, weightIdxL is the second parameter of the point (x,y) in the merged region, and wTemplateValue[x][y] is the weight of the point (x,y) in the merged region.

[0409] Method 2: determine weights of pixel points in the merged region according to a first parameter, a first threshold and a second threshold of the pixel points in the merged region.

[0410] Exemplarily, the weight of the pixel point in the merged region is a first threshold or a second threshold.

[0411] In order to reduce the calculation complexity of the merged region weight, the weight of the pixel point in the merged region in method 2 is limited to the first threshold or the second threshold, that is, the weight of the pixel point in the merged region is not the first threshold but the second threshold, which further reduces the calculation complexity of the merged region weight.

[0412] In the present application, the first threshold value and the second threshold value are not limited to specific values.

[0413] Optionally, the first threshold is one.

[0414] Optionally, the second threshold is zero.

[0415] In one example, the weight of a pixel point in the merged region may be determined by the following formula:

[0416] wTemplateValue[x][y] = (partFlip ? weightIdx: - weightIdx) > 0 ? 1 : 0 wValueMatrix[x][y] is the weight of the point (x, y) in the merged region, and the 1 in the above "1:0" is the first threshold and the 0 is the second threshold.

[0417] In the above-mentioned method 2, the weight of each point in the merged region consisting of the current block and the template is determined according to the weight derivation mode, and the weight corresponding to the template in the merged region is determined as the template weight according to the size of the template. In addition, in the above-mentioned method 2, the weight of the predicted value is determined at the same time, so that there is no need to perform a subsequent step of determining the weight of the predicted value, thereby reducing the prediction steps and improving the prediction efficiency.

[0418] Based on the above method, determine template weights corresponding to the weight derivation modes included in the second combination, then predict the template using the K prediction modes in the second combination to obtain K predicted values, and weight the K predicted values ​​based on the template weights to obtain a predicted value of the template.

[0419] The predictions of a template may be understood as a matrix of predictions of pixel points in the template.

[0420] In some embodiments, the predicted values ​​are also referred to as predicted samples.

[0421] A cost for the second combination is then determined based on the predicted and reconstructed values ​​of the template.

[0422] The above-mentioned method of determining the cost of the second combination includes, but is not limited to, the following methods.

[0423] Method 1: adopt a matrix method to determine the cost of the second combination. Specifically, a loss is determined according to the predicted value and reconstructed value of the template, and the loss is the first loss. The predicted value and reconstructed value of the template are matrices, so the first loss obtained is also a matrix, for example, the absolute value of the difference between the predicted value and reconstructed value of the template is determined as the first loss, and the first loss is determined as the cost of the second combination.

[0424] Method 2: A point-by-point calculation method is adopted to determine the cost of the second combination.

[0425] Specifically, for the i-th pixel point in the template, determine a predicted value of each prediction mode of the K prediction modes in the second combination at the i-th pixel point, determine a corresponding template weight in the template weight of the i-th pixel point, and obtain a predicted value of the i-th pixel point according to the template weight of the i-th pixel point and the K predicted values ​​of the i-th pixel point. Obtain a cost at the i-th pixel point of the second combination according to the predicted value and the reconstructed value of the i-th pixel point. Based on the method, a predicted distortion cost at each pixel point in the template of the second combination can be determined, and finally, the sum of the predicted distortion costs at each pixel point in the template of the second combination is determined as the cost of the second combination.

[0426] Based on the above method, the cost of each of the R second combinations can be determined.

[0427] Then, a candidate combination list is constructed based on the cost of each of the R second combinations.

[0428] For example, the R second combinations are ordered in ascending order of cost of the second combination, and the ordered R second combinations are determined as a candidate combination list.

[0429] Also, for example, based on the magnitude of the cost of the second combination, N second combinations having the smallest cost are selected from the R second combinations to form a candidate combination list.

[0430] Optionally, N is 8 or 16 or 32, etc.

[0431] Based on the above method, a candidate combination list is determined, and each candidate combination in the candidate combination list is ordered in ascending order of cost.

[0432] In this way, the decoding side queries the candidate combination corresponding to the first index in the candidate combination list shown in Table 6 based on the first index, and determines the candidate combination corresponding to the first index as the first combination, that is, determines the weight derivation mode included in the candidate combination as the first weight derivation mode, and determines the K prediction modes included in the candidate combination as the K first prediction modes.Then, the decoding side executes the following S103.

[0433] S103, predicting a current block according to a first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block.

[0434] Specifically, the weight of the prediction value is determined based on a first weight derivation mode, K prediction values ​​are determined based on K prediction modes, the K prediction values ​​are weighted based on the weight of the prediction value, and the weighted result is determined as the prediction value of the current block.

[0435] In this application, determining the weights corresponding to each of the K predicted values ​​based on the weight derivation mode includes, but is not limited to, the following manners.

[0436] Method 1: when determining the template weight, the template weight is determined according to the weight derivation mode, and the weight corresponding to the pixel point in the current block is determined. In this case, the above S103 is as follows: S103-A1 determining weights of the predicted values ​​based on a first weight derivation mode; S103-A2 determining K predicted values ​​based on the K first prediction modes; and determining a predicted value for the current block based on the K predicted values ​​and the weights of the predicted values.

[0437] The above S103-A1 and S103-A2 have no priority in the execution order, that is, S103-A1 may be executed before S103-A2, or after S103-A2, or may be executed in parallel with S103-A2.

[0438] It should be noted that in this application, the first weight derivation mode is used to determine the prediction weights used for the current block. For a block of a given length and width, each weight derivation mode can derive one prediction weight matrix, and for a block of the same size, the prediction weight matrices derived by different weight derivation modes may be different. Thus, the prediction weights determined by the decoding side according to the first weight derivation mode are one weight matrix, and the weight matrix includes prediction weights corresponding to each pixel point in the current block.

[0439] In some embodiments, when predicting the current block, a pixel point A in the current block is predicted using each prediction mode among the K first prediction modes, K prediction values ​​for pixel point A of the K first prediction modes are obtained, and a weight of the prediction value of pixel point A is determined based on the first weight derivation mode, for example, a weight matrix including weights corresponding to each pixel point in the current block is determined based on the first weight derivation mode, and thus the weight of the prediction value of pixel point A can be determined in the weight matrix. Then, the weight of the prediction value of pixel point A is used to weight the K prediction values ​​to obtain a prediction value of pixel point A. The above steps are performed for each pixel point in the current block to obtain a prediction value of each pixel point in the current block, and the prediction value of each pixel point in the current block constitutes a prediction value of the current block. Taking K=2 as an example, a first prediction mode is used to predict a pixel point A in a current block to obtain a first predicted value for the pixel point A, a second prediction mode is used to predict the pixel point A to obtain a second predicted value for the pixel point A, and the first predicted value and the second predicted value are weighted based on the weight of the predicted value corresponding to the pixel point A to obtain a predicted value for the pixel point A.

[0440] For example, in this application, the AWP of the AVS3 has 56 types of weight derivation modes, and the GPM of the VVC has 64 types of weight derivation modes.

[0441] As can be understood, in this embodiment of the present application, when the decoding side determines a prediction value based on K first prediction modes and weights of the prediction values, the decoding side can first determine K prediction values ​​based on the K first prediction modes, and then use the weights of the prediction values ​​to perform a weighted average calculation on the K prediction values, and finally obtain a prediction value of the current block.

[0442] In the method 1, it can be understood that the process of determining the template weights based on the first weight derivation mode and the process of determining the weights of the predicted values ​​are two independent processes and do not interfere with each other.

[0443] In some embodiments, the present application may further adopt the following method 2 to determine the weight of the predicted value.

[0444] Method 2: when determining the template weight, if the weight of a pixel point in a merged region consisting of the template region and the current block is determined according to a first weight derivation mode, S103: a step S103-B1 of determining K prediction values ​​based on the K first prediction modes; a step S103-B2 of determining weights of the prediction values ​​based on weights of pixel points in the merged region; and determining a prediction value for the current block based on the K prediction values ​​and the weights of the prediction values.

[0445] The above S103-B1 and S103-B2 have no priority in the execution order, that is, S103-B1 may be executed before S103-B2, or after S103-B2, or may be executed in parallel with S103-B2.

[0446] In this method 2, when deriving weight, derive weight of pixel point in merged region including current block and template region of current block according to the first weight derivation mode, and determine weight corresponding to current block in merged region as weight of predicted value, and determine weight corresponding to template region in merged region as template weight.That is, in this method 2, template region and current block are regarded as one whole region, and template weight and predicted value weight are derived, and further reduce weight derivation steps and improve prediction effect.

[0447] In some embodiments, the prediction process is performed on a pixel point basis, and the weights are also weights corresponding to pixel points. At this time, when predicting the current block, a pixel point A in the current block is predicted using each of the K first prediction modes, and K predicted values ​​for the pixel point A of the K first prediction modes are obtained, and the weight of the predicted value of the pixel point A is determined based on the first weight derivation mode, specifically, a weight matrix corresponding to the merged region is determined based on the first weight derivation mode, and the weight matrix includes weights corresponding to each pixel point in the current block, and thus the weight of the predicted value of the pixel point A can be determined from the weight matrix. Then, the weight of the predicted value of the pixel point A is used to weight the K predicted values ​​to obtain the predicted value of the pixel point A. The above steps are performed for each pixel point in the current block to obtain a predicted value of each pixel point in the current block, and the predicted value of each pixel point in the current block constitutes the predicted value of the current block. Taking K=2 as an example, a first prediction mode is used to predict a pixel point A in a current block to obtain a first predicted value for the pixel point A, a second prediction mode is used to predict the pixel point A to obtain a second predicted value for the pixel point A, and the first predicted value and the second predicted value are weighted based on the weight of the predicted value corresponding to the pixel point A to obtain a predicted value for the pixel point A.

[0448] In one example, taking K=2 as an example, when the first prediction mode and the second prediction mode are both intra prediction modes, a first intra prediction mode is adopted for prediction to obtain a first predicted value, a second intra prediction mode is adopted for prediction to obtain a second predicted value, and the first predicted value and the second predicted value are weighted according to the weight of the predicted value to obtain a predicted value of the current block. For example, a first intra prediction mode is adopted for predicting pixel point A to obtain a first predicted value of pixel point A, a second intra prediction mode is adopted for predicting pixel point A to obtain a second predicted value of pixel point A, and the first predicted value and the second predicted value are weighted according to the weight of the predicted value corresponding to pixel point A to obtain a predicted value of pixel point A.

[0449] In some embodiments, when the j-th prediction mode among the K first prediction modes is an inter prediction mode, determining a prediction value of the current block based on the K first prediction modes and a first weight derivation mode in S103 includes: a step S103-C1 of determining motion information based on the j-th prediction mode; a step S103-C2 of determining a j-th prediction based on the motion information; a step S103-C3 of determining K-1 predicted values ​​based on prediction modes other than the j-th prediction mode among the K first prediction modes; a weighting step S103-C4 of determining K predicted values ​​based on a first weight derivation mode; and determining a prediction value for the current block based on the j-th prediction value, the K-1 prediction values ​​and the weights of the prediction values ​​S103-C5.

[0450] Take K=2 as an example, when the first prediction mode is an intra prediction mode and the second prediction mode is an inter prediction mode, the intra prediction mode is used for prediction to obtain a first predicted value, the inter prediction mode is used for prediction to obtain a second predicted value, and the first and second predicted values ​​are weighted according to the weight of the predicted value to obtain a predicted value of the current block. In this example, the intra prediction mode is used to predict each point in the current block to obtain a predicted value of each point in the current block, and the predicted value of each point in the current block constitutes the first predicted value of the current block. The inter prediction mode is used to determine motion information, and the optimal matching block of the current block is determined according to the motion information, and the optimal matching block is determined as the second predicted value of the current block. For the weight of the predicted value of each pixel point in the current block, a weighting operation is performed on the first predicted value and the second predicted value of the current block one by one to obtain a predicted value of the current block. For example, for pixel point A in the current block, a first predicted value corresponding to pixel point A among the first predicted values ​​of the current block and a second predicted value corresponding to pixel point A among the second predicted values ​​of the current block are weighted based on the weight of the predicted value of pixel point A to obtain a predicted value of pixel point A.

[0451] In some embodiments, when K is greater than 2, weights of the prediction values ​​corresponding to two of the K first prediction modes may be determined based on the first weight derivation mode, and weights of the prediction values ​​corresponding to the other of the K first prediction modes may be preset values. For example, when K=3, the first weights of the prediction values ​​corresponding to the first and second prediction modes are derived based on the weight derivation mode, and the weight of the prediction value corresponding to the third prediction mode is a preset value. In some embodiments, when the weights of the total prediction values ​​corresponding to the K first prediction modes are constant, for example, 8, weights of the prediction values ​​corresponding to each of the K first prediction modes may be determined based on the ratio of the preset weights, and if the weight of the prediction value corresponding to the third prediction mode accounts for 1 / 4 of the total weights of the prediction values, the weight of the prediction value of the third prediction mode may be determined to be 2, and the other 3 / 4 of the total weights of the prediction values ​​may be assigned to the first and second prediction modes. For example, when deriving a weight 3 of a prediction value corresponding to a first prediction mode based on a first weight derivation mode, the weight of the prediction value corresponding to the first prediction mode is determined to be (3 / 4)*3, and the weight of the prediction value corresponding to the second prediction mode is determined to be (3 / 4)*5.

[0452] Based on the above method, a predicted value of the current block is determined, and at the same time, the code stream is decoded to obtain the quantized coefficients of the current block, and inverse quantization and inverse transformation are performed on the quantized coefficients of the current block to obtain the residual of the current block, and the predicted value of the current block and the residual are added to obtain a reconstructed value of the current block.

[0453] As can be seen from the above, the candidate combinations in the candidate combination list of the present embodiment are ordered in ascending order of cost, and the candidate combinations with the lowest cost are more likely to be selected. Based on this, in order to further reduce the coding cost, some of the top candidate combinations in the candidate combination list are coded using short codewords, and some of the bottom candidate combinations in the candidate combination list are coded using long codewords, thereby improving the overall coding efficiency.

[0454] In some embodiments, the first index is embedded in the codestream using a variable length coding scheme, for example Columbus coding.

[0455] In some embodiments, when the number of candidate combinations included in the candidate combination list is smaller than a fourth preset threshold, the first index is embedded in the code stream by a fixed-length coding method. That is, in the present embodiment, when the number of candidate combinations included in the candidate combination list is small, each candidate combination is coded by a uniform preset bit length, and the present embodiment does not limit the specific value of the fourth preset threshold, for example, the fourth preset threshold=16.

[0456] In some embodiments, a high level syntax may be used to control the switching of the combination encoding technique according to the embodiment of the present application, for example, at least one flag indicating whether the current block can use the combination encoding technique according to the embodiment of the present application, i.e., whether it can be decoded using the first combination.

[0457] In one possible implementation form, the at least one flag set may be a flag of different levels, and the corresponding level is used to indicate whether the combined encoding technology proposal according to the embodiment of the present application can be used or not.

[0458] Optionally, the at least one flag includes at least one of a sequence level flag, an image level flag, a slur level flag, a unit level flag, and a block level flag.

[0459] For example, the at least one flag includes a sequence-level flag for indicating whether the current sequence can use the combined coding technique according to the present invention.

[0460] Exemplarily, a syntax element sps_cgpm_enabled_flag is added to a parameter set. cgpm is considered to be an abbreviation of combined geometric partitioning mode. When the value of sps_cgpm_enabled_flag is 1, it indicates that the current sequence can use the technical solution of the present application, and when the value of sps_cgpm_enabled_flag is 0, it indicates that the current sequence cannot use the technical solution of the present application.

[0461] When the technical solution according to the embodiment of the present application is used for the inter mode, the inter mode already has sps_gpm_enbled_flag, so the technical solution may be controlled by sps_gpm_enbled_flag. The technical solution can also be used for the intra mode, so the intra mode may use sps_cgpm_enabled_flag or sps_gpm_enabled_flag together with the inter mode, and of course, for better flexibility, the intra mode and the inter mode may use different flags, for example, setting sps_sgpm_enabled_flag. sgpm is considered to be an abbreviation of spatial geometric partitioning mode. The sps_sgpm_enabled_flag controls whether the intra mode can use the technical solution, and the sps_cgpm_enabled_flag or sps_gpm_enabled_flag controls whether the inter mode can use the technical solution.

[0462] Also for example, the at least one flag includes an image-level flag for indicating whether the current image is capable of using the technical solution of the present application.

[0463] Alternatively, if the at least one flag includes a picture level flag, the picture level flag may be located in a picture header.

[0464] Also, for example, the at least one flag includes a slice-level flag for indicating whether the current slice is capable of using the technical solution of the present application.

[0465] Optionally, if the at least one flag includes a slice level flag, the slice level flag may be located in a slice header.

[0466] Also, for example, the at least one flag includes a unit-level flag for indicating whether the CTU is currently capable of using the technical solution of the present application.

[0467] Also, for example, the at least one flag includes a block (CU) level flag for indicating whether the current block can use the technical solution of the present application.

[0468] Taking intra as an example, the block level flag is cu_sgpm_flag. When the value of cu_sgpm_flag is 1, it indicates that the current block can use the technical solution of the present application, and when the value of cu_sgpm_flag is 0, it indicates that the current block cannot use the technical solution of the present application. Currently, GPM in VVC is used in inter merge mode and is obtained by estimation. Of course, it may be directly represented by CU level flag in the future, and is not limited here.

[0469] In this way, the decoding side first decodes the code stream, obtains the at least one flag, and judges whether the current block can use the technical solution of the present application according to the at least one flag; if it determines that the current block cannot use the technical solution of the present application according to the at least one flag, skip the method of the embodiment of the present application; if it determines that the current block can use the technical solution of the present application according to the at least one flag, execute the method of the embodiment of the present application, that is, decodes the code stream, and obtains the first index.

[0470] In the video decoding method according to the embodiment of the present application, a first index is obtained by decoding a code stream to indicate a first combination consisting of one weight derivation mode and K prediction modes, where K is a positive integer greater than 1, and a first weight derivation mode and K first prediction modes included in the first combination are determined based on the first index, and a current block is predicted based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block. In the present application, the weight derivation mode and the K prediction modes are treated as one combination, and thus the weight derivation mode and the K prediction modes used for the current block are indicated in the code stream in the form of a combination, which makes the syntax simpler, saves codewords, does not require judgment of various situations, and further improves coding efficiency.

[0471] The prediction method of the present invention has been introduced above by taking the decoding side as an example, and will be explained below by taking the encoding side as an example.

[0472] Figure 19 is a schematic diagram illustrating a flow chart of a video encoding method according to an embodiment of the present application, which is used in the video encoder shown in Figures 1 and 2. As shown in Figure 19, the method of the present embodiment includes the following steps:

[0473] S201, a first combination is determined.

[0474] The first combination includes a first weight derivation mode and K first prediction modes.

[0475] Based on the above description, one weight derivation mode and K prediction modes can act on the current block as one combination, and in order to save codewords and reduce encoding costs, in this embodiment, the weight derivation mode and K prediction modes corresponding to the current block are combined into one combination, i.e., a first combination, and a first index is used to indicate the first combination. Compared to indicating each of the weight derivation mode and the K prediction modes, this embodiment uses fewer codewords and further reduces encoding costs.

[0476] Specifically, the encoding side determines a first combination of one weight derivation mode and K prediction modes corresponding to the current block, and for convenience of explanation, the weight derivation mode included in the first combination is set as the first weight derivation mode, and the K prediction modes included in the first combination are set as the K first prediction modes. The encoding side predicts the current block using the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block, for example, derives a weight using the first weight derivation mode, predicts the current block using each of the K prediction modes to obtain K predicted values, and performs a weighting operation on the K predicted values ​​based on the derived weights to obtain a predicted value of the current block.

[0477] In some embodiments, before determining the first combination, the encoding side needs to first determine whether the current block performs weighted prediction processing using K different prediction modes. If the encoding side determines that the current block performs weighted prediction processing using K different prediction modes, it executes the above S201 to determine the first combination. If the encoding side determines that the current block does not perform weighted prediction processing using K different prediction modes, it skips the above step S201.

[0478] In one possible implementation, the encoding side can determine a prediction mode parameter for the current block to determine whether the current block undergoes weighted prediction processing using K different prediction modes.

[0479] Optionally, in the implementation of the present application, the prediction mode parameter can indicate whether the current block can use GPM mode or AWP mode, i.e., whether the current block can be predicted using K different prediction modes.

[0480] As can be understood, in the present embodiment, the prediction mode parameter may be understood as representing whether to use the GPM mode or the AWP mode flag bit. Specifically, the encoder may use one variable as the prediction mode parameter, and thereby realize the setting of the prediction mode parameter by setting the value of the variable. Exemplarily, in the present application, if the current block uses the GPM mode or the AWP mode, the encoder may set the value of the prediction mode parameter to indicate that the current block uses the GPM mode or the AWP mode, and specifically, the encoder may set the value of the variable to 1. Exemplarily, in the present application, if the current block does not use the GPM mode or the AWP mode, the encoder may set the value of the prediction mode parameter to indicate that the current block does not use the GPM mode or the AWP mode, and specifically, the encoder may set the value of the variable to 0. Furthermore, in the present embodiment, after completing the setting of the prediction mode parameter, the encoder may write the prediction mode parameter into the code stream and transmit it to the decoder, so that the decoder can obtain the prediction mode parameter after parsing the code stream.

[0481] In some embodiments, as shown in Table 5, the present embodiment may further perform a condition restriction on whether the current block uses the GPM mode or the AWP mode, i.e., if it is determined that the current block satisfies the preset condition, it may determine that the current block performs weighted prediction using K prediction modes, and further determine a first combination.

[0482] Exemplarily, when using the GPM mode or the AWP mode, the size of the current block may be limited.

[0483] As can be understood, the prediction method according to the embodiment of the present application requires K prediction values ​​to be generated using K different prediction modes, and then weighted based on the weights to obtain a prediction value of the current block, so in order to reduce complexity while considering the trade-off between compression performance and complexity, the embodiment of the present application may restrict the use of the GPM mode or AWP mode for a block of a certain size. Therefore, in the present application, the encoding side may first determine a size parameter of the current block, and then determine whether the current block uses the GPM mode or the AWP mode based on the size parameter.

[0484] In the present embodiment, the size parameters of the current block may include the height and width of the current block, so that the encoder can determine whether the current block uses GPM mode or AWP mode based on the height and width of the current block.

[0485] Exemplarily, the present application determines that the current block uses the GPM mode or the AWP mode if the width is greater than the first threshold and the height is greater than the second threshold. As can be seen, one possible restriction is to use the GPM mode or the AWP mode only if the width of the block is greater than (or equal to) the first threshold and the height of the block is greater than (or equal to) the second threshold. The values ​​of the first and second thresholds may be 8, 16, 32, etc., and the first threshold may be equal to the second threshold.

[0486] Exemplarily, the present application determines that the current block uses the GPM mode or the AWP mode if the width is smaller than the third threshold and the height is larger than the fourth threshold. As can be seen, one possible restriction is to use the GPM mode or the AWP mode only if the width of the block is smaller than (or less than) the third threshold and the height of the block is larger than (or more than) the fourth threshold. The values ​​of the third and fourth thresholds may be 8, 16, 32, etc., and the third threshold may be equal to the fourth threshold.

[0487] Furthermore, in the present embodiment, the size of blocks in which the GPM mode or the AWP mode can be used can be restricted by further restricting pixel parameters.

[0488] For example, in this application, the encoder may first determine the pixel parameters of the current block, and then further determine whether the current block can use the GPM mode or the AWP mode based on the pixel parameters and the fifth threshold. As can be seen, one possible restriction is to use the GPM mode or the AWP mode only if the number of pixels of the block is greater than (or equal to) the fifth threshold. The value of the fifth threshold may be 8, 16, 32, etc.

[0489] That is, in this application, the current block can use the GPM mode or the AWP mode only under the condition that the size parameter of the current block meets the size requirement.

[0490] For example, in this application, there may be a frame-level flag that determines whether the frame to be currently encoded uses this application. For example, it may be set so that intra frames (e.g., I frames) use this application and inter frames (e.g., B frames, P frames) do not use this application. Or it may be set so that intra frames do not use this application and inter frames use this application. Or it may be set so that some inter frames use this application and some inter frames do not use this application. Since inter frames may use intra prediction, it is possible that inter frames use this application.

[0491] In some embodiments, there may also be a flag below the frame level and above the CU level (eg, tile, slice, patch, LCU, etc.) that determines whether a region uses the technical proposal of the present application.

[0492] In this embodiment, the first combination includes a first weight derivation mode and K first prediction modes.

[0493] The first weight derivation mode is used to determine the weights used for the current block. Specifically, the first weight derivation mode may be a mode for deriving weights. For a block of a given length and width, each type of weight derivation mode can derive a weight matrix, and for blocks of the same size, the weight matrices derived by different weight derivation modes are different.

[0494] Illustratively, in this application, the AWP has 56 types of weight derivation modes, and the GPM has 64 types of weight derivation modes.

[0495] The K different first prediction modes included in the first combination include the following examples.

[0496] In example 1, the K different first prediction modes are all intra prediction modes, for example, the current block is an intra coding block and is not applicable to coding of screen content.

[0497] In example 2, the K different first prediction modes are all inter prediction modes, for example, the current block is an inter coded block.

[0498] Example 3: among the K different first prediction modes, at least one is an intra prediction mode, and at least one is an inter prediction mode.

[0499] Example 4: Among the above K different first prediction modes, at least one is an intra prediction mode, and at least one is a prediction mode that is neither inter nor intra, such as an intra block copy IBC prediction mode or a palette prediction mode.

[0500] Example 5: Among the K different first prediction modes, at least one is an inter prediction mode, and at least one is a prediction mode that is neither inter nor intra, such as an IBC prediction mode or a palette prediction mode.

[0501] Example 6: None of the K different first prediction modes are intra prediction modes or inter prediction modes. For example, one is an IBC prediction mode and another is a palette prediction mode.

[0502] It should be clarified that the present embodiment does not limit the specific types of the K different first prediction modes included in the first combination.

[0503] In the embodiment of the present application, the encoding side determines the first combination in the following manners, but is not limited to these.

[0504] Method 1, first combination is a combination that is set in advance.

[0505] In the second method, the encoding side determines a list, which includes multiple candidate combinations, and is also called a candidate combination list. The first combination is determined from the candidate combination list. At this time, the above S201 is as follows: A step S201-A of determining a candidate combination list; and step S201-B of determining a first combination from the candidate combination list.

[0506] The candidate combination list includes a plurality of candidate combinations, and any one of the plurality of candidate combinations includes one weight derivation mode and K prediction modes.

[0507] Exemplarily, the candidate combination list is shown in Table 6 above.

[0508] As shown in Table 6, the candidate combination list includes a plurality of candidate combinations, and any two of the plurality of candidate combinations are not completely similar, that is, the weight derivation mode and at least one of the K prediction modes included in any two candidate combinations are different. For example, the weight derivation mode in the candidate combination 1 is different from the weight derivation mode in the candidate combination 2, or the weight derivation mode in the candidate combination 1 is the same as the weight derivation mode in the candidate combination 2, and at least one of the K prediction modes is different, or the weight derivation mode in the candidate combination 1 is different from the weight derivation mode in the candidate combination 2, and at least one of the K prediction modes is different.

[0509] For example, the order of the candidate combination in the candidate combination list in Table 6 above is used as an index, and optionally, the index of the candidate combination in the candidate combination list may be implemented in other ways, and the embodiment of the present application is not limited thereto.

[0510] In the method 2, the encoding side determines one candidate combination in the candidate combination list as the first combination.

[0511] For example, the encoding side determines the first candidate combination in the candidate combination list as the first combination.

[0512] In this method 2, the encoding side and the decoding side can respectively determine the same candidate combination list, for example, the encoding side and the decoding side each determine one list including N candidate combinations, and each candidate combination includes one weight derivation mode and K prediction modes. In the code stream, the encoding side only needs to write one final selected candidate combination, for example the first combination, and the decoding side analyzes the first combination finally selected by the encoding side, specifically, the decoding side decodes the code stream, obtains a first index, and determines the first combination from the candidate combination list determined by the decoding side according to the first index.

[0513] In the present embodiment, there is no restriction on the specific manner in which the encoding side determines the candidate combination list.

[0514] In some embodiments, the candidate combination list is pre-existing, and the encoding side obtains the candidate combination list when it determines that the current block can perform weighted prediction using K prediction modes.

[0515] In some embodiments, the candidate combination list is constructed by the encoding side. In this case, in some embodiments, the encoding side can transmit the constructed candidate combination list to the decoding side. In some embodiments, the encoding side uploads the constructed candidate combination list to a cloud, and the decoding side reads the candidate combination list from the cloud after decoding the codestream and obtaining the first index.

[0516] The process by which the encoding side builds the candidate combination list will be described below.

[0517] In the present embodiment, there is no restriction on the method in which the encoding side constructs the candidate combination list.

[0518] In some embodiments, the encoding side: A candidate combination list is constructed in step S201-A1 for constructing a candidate combination list based on the template of the current block.

[0519] Template matching utilizes the correlation between neighboring pixels and uses a part of the surrounding area of ​​the current block as a template. When the current block is coded, its left and upper sides are already coded according to the coding order. During inter prediction, the best matching position of the template is found in the reference frame, and the motion information or motion vector of the current block is determined accordingly. During intra prediction, the template is used to determine the intra prediction mode of the current block.

[0520] The template of the current block is composed of a reconstructed region around the current block and is associated with the current block, and based on this, the embodiment of the present invention constructs a candidate combination list based on the template of the current block.

[0521] This application does not limit the specific shape of the template of the current block.

[0522] In some embodiments, the template includes at least one of the above coded region, the left coded region, and the top-left coded region of the current block.

[0523] Optionally, the width of the upper coded area is the same as the width of the current block, the height of the left coded area is the same as the height of the current block, the width of the upper left coded area is the same as the width of the left coded area, and the height of the upper left coded area is the same as the height of the upper coded area.

[0524] For example, for each combination, predict a template of the current block using the combination, obtain a predicted value of the template under each combination, and build a candidate combination list based on the predicted value of the template under each combination.

[0525] For each combination, predict the template of the current block using the combination, obtain a predicted value of the template for each combination, specifically, derive a weight of the template using a weight derivation mode included in the combination, predict the template using each of the K prediction modes included in the combination to obtain K predicted values ​​of the template, and perform a weighting process on the K predicted values ​​of the template based on the derived weight of the template to obtain a predicted value of the template for the combination. Finally, construct a candidate combination list based on the predicted value of the template for each combination.

[0526] It should be noted that the weight derived according to the weight derivation mode can be understood as a weight corresponding to each pixel point in the derived template, or as a weight matrix corresponding to the derived template. When determining the predicted value of the template according to the weight, K predicted values ​​corresponding to each pixel point in the template can be determined, and the predicted value corresponding to each pixel point can be determined according to the K predicted values ​​corresponding to each pixel point and the weight, and the predicted value corresponding to each pixel point in the template constitutes the predicted value of the template. Optionally, when determining the predicted value of the template according to the weight, it can be further performed according to a block, for example, determining the predicted value of the template, and weighting the K predicted values ​​of the template according to the weight matrix of the template to obtain the predicted value of the template.

[0527] In some embodiments, S201-A1 is a step S201-A11 of determining R second combinations, where any one of the R second combinations includes one weight derivation mode and K prediction modes, and the weight derivation mode and the K prediction modes included in any two of the R second combinations are not completely similar, and R is a positive integer greater than 1; A step S201-A12 of predicting a template for any one of the R second combinations using a weight derivation mode in the second combination and the K prediction modes to obtain a template prediction value corresponding to the second combination; and constructing a candidate combination list based on template prediction values ​​corresponding to each of the R second combinations.

[0528] In this embodiment, when the encoding side constructs a candidate combination list, it first determines R second combinations, and the specific value of the R second combinations is not limited in this application, for example, 8, 16, 32, etc., and each of the R second combinations includes one weight derivation mode and K prediction modes, and the weight derivation mode and the K prediction modes included in any two combinations of the R second combinations are not completely similar. Then, for each of the R second combinations, it predicts the template of the current block using the weight derivation mode and the K prediction modes included in the second combination, and obtains a prediction value of the template in the second combination. Finally, it constructs a candidate combination list based on the template prediction value corresponding to each of the R second combinations.

[0529] In the above S201-A13, constructing a candidate combination list based on the template prediction value corresponding to each second combination among the R second combinations includes, but is not limited to, the following manners.

[0530] Method 1: construct a candidate combination list based on the magnitude of the template prediction value corresponding to each second combination among the R second combinations.

[0531] For example, the R second combinations are ordered based on the magnitude of the template prediction value corresponding to each of the R second combinations, and the ordered R second combinations are determined as a candidate combination list, where the candidate combination list includes the R candidate combinations.

[0532] Further, for example, the R second combinations are ordered based on the magnitude of the template prediction value corresponding to each of the R second combinations, and N second combinations are selected from the ordered R second combinations to construct a candidate combination list, where the candidate combination list includes the N candidate combinations.

[0533] Method 2, S201-A13 above, a step S201-A13-1 of determining a cost corresponding to the second combination based on the template prediction value and the template reconstruction value corresponding to the second combination; and constructing a candidate combination list based on a cost corresponding to each of the R second combinations.

[0534] Since the template of the current block is a reconstructed region, the encoding side can obtain the reconstruction value of the template, and thus, for each second combination among the R second combinations, the encoding side can determine a prediction distortion cost corresponding to the second combination based on the predicted value of the template in the second combination and the reconstruction value of the template. The manner of determining the cost corresponding to the second combination includes, but is not limited to, SAD, SATD, SEE, etc. Then, a candidate combination list is constructed based on the cost corresponding to each second combination among the R second combinations.

[0535] The manner of constructing the candidate combination list based on the cost corresponding to each of the R second combinations in S201-A13-2 includes, but is not limited to, the following examples.

[0536] Example 1: Ranking the R second combinations based on the cost corresponding to each of the R second combinations, and determining the ordered R second combinations as a candidate combination list.

[0537] The candidate combination list generated in the first example includes R candidate combinations.

[0538] Optionally, the R candidate combinations in the candidate combination list are ordered in ascending order of cost, ie, the costs corresponding to the R candidate combinations in the candidate combination list increase in the ordered order.

[0539] Ordering the R second combinations based on the cost corresponding to each second combination among the R second combinations orders the R second combinations in order of decreasing cost.

[0540] Example 2: N second combinations are selected from the R second combinations based on the costs corresponding to the second combinations, and the list of the N second combinations is determined as a candidate combination list.

[0541] Optionally, the N second combinations are the N second combinations with the smallest cost among the R second combinations, for example, based on the cost corresponding to each second combination among the R second combinations, the N second combinations with the smallest cost are selected from the R second combinations to form a candidate combination list, where the candidate combination list includes the N candidate combinations.

[0542] Optionally, the N candidate combinations in the candidate combination list are ordered in ascending order of cost, ie, the costs corresponding to the N candidate combinations in the candidate combination list increase in the ordered order.

[0543] The process of determining the R second combinations in S201-A11 above will be described below.

[0544] In some embodiments, the R second combinations are preset. Thus, a template of the current block is predicted using each of the R second combinations preset, a template prediction value corresponding to each of the second combinations is obtained, a cost corresponding to each of the second combinations is determined according to the template prediction value and the template reconstruction value corresponding to each of the second combinations, and the R second combinations are ordered according to the cost corresponding to each of the second combinations, and the ordered R second combinations are set as a candidate combination list, or N second combinations with the smallest cost are selected from the ordered R second combinations to form the candidate combination list.

[0545] In some embodiments, S201-A11 is a step S201-A11-1 of determining P weight derivation modes and Q prediction modes, where P is a positive integer and Q is a positive integer equal to or greater than K; and a step S201-A11-2 of constructing R second combinations based on the P weight derivation modes and Q prediction modes, where any one of the R second combinations includes one weight derivation mode of the P weight derivation modes and K prediction modes of the Q prediction modes, where P are all positive integers and Q is a positive integer greater than or equal to K.

[0546] In this embodiment, the encoding side first determines P weight derivation modes and Q prediction modes, and then constructs R second combinations based on the determined P derived weight derivation modes and Q prediction modes.

[0547] In the present embodiment, there is no specific limit to the number of the P weight derivation modes and the Q prediction modes.

[0548] In one possible implementation, the current block is an intra-coded block, where there are 64 possible weight derivation modes of the GPM, and there are 67 possible intra prediction modes of the GPM, which can be found in the VVC standard. However, this is not a limitation that there are only 64 possible weights of the GPM, or what kind of 64 types there are, and it should be noted that the selection of 64 types of GPM of VVC is also a trade-off between improved prediction effect and increased overhead in the code stream. In the present application, since the weight derivation mode is no longer coded using one fixed logic, theoretically, the present application can use more diverse weights and more flexibly. Similarly, it is not a limitation that there are only 67 intra prediction modes of the GPM, or what kind of 67 types there are. In theory, all possible intra prediction modes can be used for the GPM. For example, the intra angle prediction mode can be made more fine-grained, and more intra angle prediction modes can be generated, and the GPM can also use more intra angle prediction modes. For example, regarding the MIP (matrix-based intra prediction) mode of VVC, this application may use it, but considering that MIP can select multiple sub-modes, here, for easy understanding, MIP is not added to this embodiment.In addition, there are also some wide-angle modes, which can also be used in this application, and this embodiment will not describe them further.

[0549] Assuming that K=2, the K prediction modes include a first prediction mode and a second prediction mode, and there are 67 types of available prediction modes (i.e., Q=67), there are 67 possible first prediction modes, and since the second prediction mode is different from the first prediction mode, there are 66 types of second prediction modes, and there are 64 types of weight derivation modes (i.e., P=64), in the present application, one second combination is formed with any two different prediction modes and any one weight derivation mode, and there are a total of 64*67*66 types of possible second combinations.

[0550] In this implementation, the P weight derivation modes are all possible weight derivation modes, for example 64 weight derivation modes in GPM, and the Q prediction modes are all possible prediction modes, for example 67 intra prediction modes in GPM, and an enumeration method is used to obtain all possible second combinations, for example 64*67*66 possible second combinations, and each second combination among the 64*67*66 possible second combinations is used to predict the template of the current block, and the distortion cost of each second combination is calculated, and a candidate combination list corresponding to the current block is obtained based on the distortion cost of each second combination.

[0551] In some embodiments, in order to reduce the amount of data and improve the speed of building the candidate combination list, instead of trying every type of prediction mode, some prediction modes may be selected and tried.

[0552] At this time, the implementation form of determining the Q prediction modes in the above S201-A11-1 includes, but is not limited to, the following methods.

[0553] In the first method, the Q prediction modes are preset prediction modes.

[0554] Method 2: determine at least one of a candidate prediction mode list for a current block, an alternative prediction mode list corresponding to each of the K prediction modes, and a prediction mode corresponding to a weight derivation mode, and determine Q prediction modes based on at least one of the candidate prediction mode list, the alternative prediction mode list corresponding to each of the K prediction modes, and a prediction mode corresponding to the weight derivation mode.

[0555] The candidate prediction mode list includes a plurality of candidate prediction modes, and the alternative prediction mode list corresponding to any one of the K prediction modes includes at least one alternative prediction mode.

[0556] Determining the alternative prediction mode list corresponding to each of the K prediction modes may be understood as determining one alternative prediction mode list for each prediction mode among the K prediction modes, and when subsequently constructing a second combination, a certain prediction mode in the second combination is selected from the alternative prediction mode list corresponding to the prediction mode. For example, K=2, the K prediction modes include a first prediction mode and a second prediction mode, and the encoding side constructs one alternative prediction mode list 1 for the first prediction mode and one alternative prediction mode list 2 for the second prediction mode. In this way, when subsequently constructing a different second combination, one alternative prediction mode is selected from the alternative prediction mode list 1 as the first prediction mode and one alternative prediction mode is selected from the alternative prediction mode list 2 as the second prediction mode, and thus one weight derivation mode and the currently selected first prediction mode and second prediction mode constitute one second combination.

[0557] In the embodiment of the present application, there is no limitation on the method of determining the alternative prediction mode list corresponding to each prediction mode among the K prediction modes.

[0558] In one possible implementation form, for any one of the K prediction modes, a candidate prediction mode list corresponding to the prediction mode and at least one of the prediction modes corresponding to the weight derivation mode are determined, and an alternative prediction mode list corresponding to the prediction mode is determined based on the candidate prediction mode list and at least one of the prediction modes corresponding to the weight derivation mode.

[0559] In this embodiment, the process of determining a candidate prediction mode list corresponding to a prediction mode among the K prediction modes is almost similar to the process of determining a candidate prediction mode list corresponding to the current block, and specifically, the following can be referred to.

[0560] In some embodiments, the candidate prediction mode list includes one or more inter prediction modes, such as at least one of skip, merge, general inter prediction mode, unidirectional prediction, bidirectional prediction, multiple hypothesis prediction, etc.

[0561] In some embodiments, the candidate prediction mode list includes one or more intra-prediction modes, such as at least one of a direct current (DC) mode, a planar mode, an angle mode, etc. Optionally, the candidate prediction mode list includes at least one intra-prediction mode in an MPM list.

[0562] In some embodiments, the candidate prediction mode list may further include modes such as IBC, palette, and the like.

[0563] In this application, there is no limitation on the types and number of prediction modes included in the candidate prediction mode list.

[0564] In some embodiments, the candidate prediction mode list is determined in at least one of the following ways:

[0565] Method 1: The candidate prediction mode list includes preset modes.

[0566] Method 2: The candidate prediction mode list includes the modes in the MPM list.

[0567] In some embodiments, the candidate intra-prediction mode list may be an MPM list of a current block, for example, in VVC, the current block may obtain an MPM list of length 6. In addition, in some subsequent technological advances, there is a scheme called secondary MPM that can derive an MPM list of length 22, that is, the sum of the length of the first MPM list and the length of the second MPM list is 22. That is, the embodiments of the present application may use MPM to screen intra-prediction modes.

[0568] In some embodiments, if the determined candidate prediction mode list does not include the preset mode, the preset mode is added to the candidate prediction mode list.

[0569] Optionally, the pre-set mode includes at least one of a DC, a horizontal mode, and a vertical mode.

[0570] Optionally, a pre-set mode may be added under certain conditions, in one example when the number of prediction modes among the candidate prediction modes is less than or equal to a certain threshold, which may be 3, 4, 5, 6, etc.

[0571] Method 3: The candidate prediction mode list includes a set of candidate prediction modes that are determined based on some rules, such as equal interval screening.

[0572] Method 4: determining a candidate prediction mode list based on a prediction mode used for at least one block adjacent to the current block.

[0573] For example, a prediction mode used in one or more blocks adjacent to the current block is added to a candidate prediction mode list to obtain a candidate prediction mode list for the current block, or a candidate prediction mode list corresponding to a kth prediction mode among the K prediction modes, where the kth prediction mode is any one of the K prediction modes.

[0574] Also, for example, a prediction mode corresponding to the k-th prediction mode among prediction modes used for one or more blocks adjacent to the current block is added to the candidate prediction mode list corresponding to the k-th prediction mode. For example, if K=2, the k-th prediction mode is the first prediction mode, and two blocks adjacent to the current block also perform weighted prediction using two prediction modes, the two prediction modes used for the first adjacent block are prediction mode 1 and prediction mode 3, and the two prediction modes used for the second adjacent block are prediction mode 4 and prediction mode 5, respectively, and thus prediction mode 1 and prediction mode 4 can be added to the candidate prediction mode list corresponding to the first prediction mode.

[0575] It should be noted that the above methods 1 to 4 may be used alone or in any combination as methods for determining the candidate prediction mode list.

[0576] In some embodiments, the candidate prediction mode list includes at least one of a candidate intra-prediction mode list and a candidate inter-prediction mode list, where the candidate intra-prediction mode list includes at least one candidate intra-prediction mode and the candidate inter-prediction mode list includes at least one candidate inter-prediction mode.

[0577] Based on the above method, a candidate prediction mode list for the current block is obtained, and then Q prediction modes are determined based on the candidate prediction mode list, for example, all or some of the candidate prediction modes included in the candidate prediction mode list are determined as all or some of the Q prediction modes.

[0578] The process of determining a prediction mode corresponding to a weight derivation mode will be described below.

[0579] In the present embodiment, the prediction mode corresponding to the weight derivation mode does not mean to indicate a specific one, and may be, for example, a prediction mode corresponding to one preset weight derivation mode, or a prediction mode corresponding to a plurality of preset weight derivation modes. In some embodiments, the prediction mode corresponding to the weight derivation mode may be understood as a prediction mode list corresponding to the weight derivation mode, and the prediction mode list includes at least one prediction mode.

[0580] In some embodiments, the prediction modes corresponding to the weight derivation modes include a prediction mode corresponding to at least one weight derivation mode of the P weight derivation modes, in which determining the prediction mode corresponding to the weight derivation modes includes: for a p-th weight derivation mode of the P weight derivation modes, determining a prediction mode corresponding to the p-th weight derivation mode, and determining the prediction mode corresponding to the weight derivation mode based on the prediction mode corresponding to the at least one weight derivation mode of the P weight derivation modes, where p is a positive integer.

[0581] In this embodiment, the process of determining a prediction mode corresponding to each weight derivation mode among the P weight derivation modes is almost the same, and for convenience of explanation, the following description will be given using the p-th weight derivation mode among the P weight derivation modes as an example.

[0582] Determining the prediction mode corresponding to the p-th weight derivation mode includes the following two methods.

[0583] Method 1: When at least one of the prediction modes corresponding to the p-th weight derivation mode is an intra prediction mode, an angle index is determined based on the p-th weight derivation mode, and the intra prediction mode corresponding to the angle index is determined as at least one of the prediction modes corresponding to the p-th weight derivation mode.

[0584] The angle index is used to indicate the boundary angle index of the weight.

[0585] In some embodiments, the field angleIdx represents an angle index.

[0586] The above Table 2 shows the correspondence between merge_gpm_partition_idx and angleIdx. By referring to the above Table 2, the angle index can be derived based on the p-th weight derivation mode.

[0587] In the present application, the angle index and the intra prediction mode have a corresponding relationship, that is, different angle indexes correspond to different intra prediction modes.

[0588] For example, the angle index and the intra prediction mode have the corresponding relationship shown in Table 7.

[0589] In the method 1, taking K=2 as an example, when the first prediction mode or the second prediction mode is an intra prediction mode, the angle index is determined based on the p-th weight derivation mode, for example, the angle index corresponding to the p-th weight derivation mode is derived based on the above Table 2. Then, in the above Table 7, the intra prediction mode corresponding to the angle index is determined, for example, the angle index is 2, the corresponding intra prediction mode is 42, and the intra prediction mode 42 is determined as the first prediction mode or the second prediction mode.

[0590] Method 2: When at least one of the prediction modes corresponding to the p-th weight derivation mode is an intra prediction mode, an intra prediction mode corresponding to the p-th weight derivation mode is determined, and at least one of the intra prediction modes corresponding to the p-th weight derivation mode is determined as at least one of the prediction modes corresponding to the p-th weight derivation mode.

[0591] The intra prediction mode corresponding to the p-th weight derivation mode includes at least one of an intra prediction mode in which the prediction direction is parallel to the boundary line of the weights, an intra prediction mode in which the prediction direction is perpendicular to the boundary line, and a planar mode.

[0592] It should be noted that the intra prediction modes with a prediction direction parallel to the weight boundary include one or more intra prediction modes with a prediction direction parallel or nearly parallel to the weight boundary, and the intra prediction modes with a prediction direction perpendicular to the weight boundary include one or more intra prediction modes with a prediction direction perpendicular or nearly perpendicular to the weight boundary.

[0593] In the method 2, taking K=2 as an example, when the first prediction mode and / or the second prediction mode are intra prediction modes, the first prediction mode and / or the second prediction mode are determined from the intra prediction mode corresponding to the weight derivation mode. For example, the first prediction mode and / or the second prediction mode may be an intra prediction mode that is the same straight line as the weight partition line (also called a boundary line) or approximately the same straight line as the weight partition line. Or, the first prediction mode and / or the second prediction mode may be an intra prediction mode whose prediction direction is perpendicular or approximately perpendicular to the weight boundary line. For example, when the weight boundary line is horizontal, such as the modes with GPM indexes of 18, 19, 50, and 51 in FIG. 4, the first prediction mode and / or the second prediction mode is the horizontal mode 18 or the vertical mode 50.

[0594] The encoding side determines a prediction mode corresponding to at least one of the P weight derivation modes based on the above steps, and further determines a prediction mode corresponding to the weight derivation mode based on the prediction mode corresponding to the at least one of the P weight derivation modes. For example, all or a part of the prediction modes corresponding to the at least one of the P weight derivation modes are set as the prediction modes corresponding to the weight derivation modes.

[0595] Furthermore, there may be overlapping prediction modes among the prediction modes corresponding to the P weight derivation modes, and the overlapping prediction modes are removed and another different prediction mode is determined as the prediction mode corresponding to the weight derivation mode.

[0596] In the embodiment of the present application, in order to reduce the number of the R second combinations, the prediction modes are screened, specifically, the Q prediction modes are determined based on the above method.

[0597] In some embodiments, in order to reduce the complexity of the encoding side, the number of the Q prediction modes is limited, for example, Q is less than or equal to a first preset threshold. In the present application, the specific value of the first preset threshold is not limited, and can be determined according to actual needs, for example, the first preset threshold is 6, that is, 6 prediction modes are selected to construct R second combinations, thereby controlling the number of second combinations.

[0598] In some embodiments, the magnitude of Q may be understood to be related to the size and / or shape of the current block, where the shape of the current block is determined by the aspect ratio of the current block.

[0599] During prediction, for small blocks, the difference in the impact of similar prediction modes on the prediction result is not large, while for large blocks, the difference in the impact of similar prediction modes on the prediction result is more obvious. Based on this, in the embodiment of the present application, different Q values ​​are set for blocks of different sizes, i.e., a large Q value is set for a large block, and a small Q value is set for a small block.

[0600] At this time, when determining the Q value corresponding to the current block, it is set based on the size of the current block, for example, when the size of the current block is larger than the first numerical value, Q is equal to or larger than the second preset threshold, and for example, when the size of the current block is equal to or smaller than the first numerical value, Q is smaller than the third preset threshold. In the embodiment of the present application, the specific magnitudes of the first numerical value, the second prediction threshold, and the third preset threshold are not limited, and the third preset threshold is smaller than the second preset threshold.

[0601] The process of determining the P weight derivation modes in S201-A11-1 will be described below.

[0602] As can be seen from the above, GPM includes 64 types of weight derivation modes, and AWP includes 56 types of weight derivation modes.

[0603] In some embodiments, the P weight derivation modes are the 64 weight derivation modes included in the GPM or the 56 weight derivation modes included in the AWP.

[0604] In some embodiments, P weight derivation modes are selected from M preset weight derivation modes, where M is a positive integer equal to or greater than P.

[0605] The M pre-set weight derivation modes may be 64 types of weight derivation modes included in the GPM, or may be 56 types of weight derivation modes included in the AWP, or may be a portion of the 64 types of weight derivation modes included in the GPM, or may be a portion of the 56 types of weight derivation modes included in the AWP.

[0606] In this embodiment, in order to further reduce the number of R second combinations, M preset weight derivation modes can be screened, and P weight derivation modes can be selected from the M preset weight derivation modes to construct R second combinations.

[0607] In some embodiments, weight derivation modes corresponding to a preset division angle and / or a preset offset amount are removed from the M weight derivation modes to obtain P weight derivation modes. The same division angle in the weight derivation modes may correspond to multiple offset amounts, such as weight derivation modes 10, 11, 12 and 13 shown in FIG. 16A, which have the same division angle but different offset amounts, and thus some of the weight derivation modes corresponding to the preset offset amount can be removed and / or some of the weight derivation modes corresponding to the preset division angle can be removed. In this way, the total number of possible second combinations can be reduced. And the difference between each possible second combination can be more clearly seen.

[0608] In some embodiments, the screening conditions corresponding to different blocks may be different, and thus when determining the P weight derivation modes corresponding to the current block, first determine the screening conditions corresponding to the current block, and then select the P weight derivation modes from the M weight derivation modes based on the screening conditions corresponding to the current block.

[0609] In some embodiments, the screening conditions corresponding to the current block include screening conditions corresponding to the size of the current block and / or screening conditions corresponding to the shape of the current block. During prediction, for small blocks, the difference in the impact of similar weight derivation modes on the prediction result is not large, while for large blocks, the difference in the impact of similar weight derivation modes on the prediction result is more obvious. Based on this, in the present embodiment, different P values ​​are set for blocks of different sizes, i.e., a large P value is set for large blocks, and a small P value is set for small blocks.

[0610] In some embodiments, the screening condition includes an array, the array including M elements, the M elements corresponding one-to-one to the M weight derivation modes, and an element corresponding to each weight derivation mode is used to indicate whether the weight derivation mode is usable or not, which may be understood as whether the weight derivation mode is usable for attempting to construct a subsequent second combination by one of the P weight derivation modes.

[0611] The array may be a one-dimensional or two-dimensional numerical value.

[0612] In some embodiments, when the screening conditions corresponding to the current block include screening conditions corresponding to the size of the current block and screening conditions corresponding to the shape of the current block, and for the same weight derivation mode, the screening conditions corresponding to the size of the current block and the screening conditions corresponding to the shape of the current block both indicate that the weight derivation mode is usable, the weight derivation mode is determined to be one of the P weight derivation modes, and when at least one of the screening conditions corresponding to the size of the current block and the screening conditions corresponding to the shape of the current block indicates that the weight derivation mode is not usable, it is determined that the weight derivation mode does not constitute one of the P weight derivation modes.

[0613] In some embodiments, multiple arrays may be used to implement screening conditions corresponding to different block sizes and screening conditions corresponding to different block shapes, respectively.

[0614] In some embodiments, screening conditions corresponding to different block sizes and screening conditions corresponding to different block shapes may be implemented using a two-dimensional array, i.e., the two-dimensional array includes screening conditions corresponding to block sizes and also includes screening conditions corresponding to block shapes.

[0615] After the encoding side determines Q prediction modes and P weight derivation modes based on the above steps, it configures R different second combinations based on the Q prediction modes and P weight derivation modes.

[0616] In some embodiments, in order to further improve the speed of constructing the candidate combination list, the encoding side rescreens the determined Q prediction modes and P weight derivation modes, and at this time, constructing R second combinations based on the P weight derivation modes and Q prediction modes determined in S201-A11-2 includes the following steps S201-A11-21 to S201-A11-23.

[0617] S201-A11-21, select S weight derivation modes from P weight derivation modes, where S is a positive integer less than or equal to P;

[0618] Specifically, weight derivation modes with low occurrence probability are removed from the P weight derivation modes determined above, and S weight derivation modes after screening are obtained.

[0619] The selection of S weight derivation modes from P weight derivation modes in the above S201-A11-21 includes, but is not limited to, the following manners.

[0620] Method 1: For an i-th weight derivation mode among P weight derivation modes, weights in a template of K second prediction modes are determined based on the i-th weight derivation mode, the K second prediction modes being any K prediction modes among Q prediction modes, where i is a positive integer from 1 to P, and if a weight in the template of any one of the K prediction modes is smaller than a first preset value, the i-th weight derivation mode is removed from the P weight derivation modes to obtain S weight derivation modes.

[0621] In the method 1, when the template weight derived by a certain weight derivation mode reduces the influence of the template of a certain prediction mode or eliminates the influence of the template, the weight derivation mode is not used. For example, the weight derivation mode 52 in FIG. 4 (square block) reduces the weight of the template of the second prediction mode, and the weight derivation mode 54, for example, reduces the weight of the template of the second prediction mode to 0, that is, the weight derivation mode 54 has no influence of the template of the second prediction mode, and the predicted value of the template is all determined by the first prediction mode. In this case, the second prediction mode has no influence, and such a weight derivation mode needs to be removed from the P weight derivation modes.

[0622] In the above method 1, from the P weight derivation modes, weight derivation modes in which the weight in the template of any one of the K prediction modes is smaller than a first preset value are removed, and S weight derivation modes are obtained.

[0623] In the present embodiment, the first preset value is not limited to a specific value, and may be, for example, a small value equal to or greater than 0.

[0624] Method 2: For an i-th weight derivation mode among P weight derivation modes, a cost when predicting a template using the i-th weight derivation mode is determined, where i is a positive integer from 1 to P, and S weight derivation modes are selected from the P weight derivation modes based on the cost corresponding to the i-th weight derivation mode.

[0625] In the method 2, S weight derivation modes are selected from the P weight derivation modes by calculating a cost corresponding to each weight derivation mode among the P weight derivation modes.

[0626] In the present embodiment, the weight derivation mode and K prediction modes are treated as one combination, and the cost is calculated. Thus, in order to facilitate the calculation, the costs of P weight derivation modes are calculated based on the given K prediction modes, that is, each of the P weight derivation modes is combined with the set K prediction modes to obtain P combinations, the costs corresponding to each combination of the P combinations are calculated, and the costs of the P weight derivation modes are further obtained.

[0627] Based on the above method, a cost corresponding to the i-th weight derivation mode among the P weight derivation modes is determined, and then S weight derivation modes are selected from the P weight derivation modes based on the cost corresponding to the i-th weight derivation mode.

[0628] In the method 2, the encoding side selects S weight derivation modes from P weight derivation modes based on the cost corresponding to the i-th weight derivation mode, including the following methods.

[0629] In the first method, when the cost corresponding to the i-th weight derivation mode is smaller than a second preset value, a weight derivation mode similar to the i-th weight derivation mode is selected from the P weight derivation modes, and S weight derivation modes are determined based on the i-th weight derivation mode and the similar weight derivation modes of the i-th weight derivation mode. The similar weight derivation modes of the i-th weight derivation mode may be understood as weight derivation modes whose prediction results are similar to those of the i-th weight derivation mode.

[0630] In the present embodiment, the magnitude of the second preset value is not limited, and is specifically determined according to actual needs.

[0631] In some embodiments, the encoding side may further select S weight derivation modes from the P weight derivation modes based on the following second scheme.

[0632] In the second method, when the cost corresponding to the i-th weight derivation mode is greater than a third preset value, the i-th weight derivation mode and weight derivation modes similar to the i-th weight derivation mode are removed from the P weight derivation modes, and at least one weight derivation mode is obtained after the removal, and S weight derivation modes are determined based on the at least one weight derivation mode after the removal.

[0633] In the present embodiment, the magnitude of the third preset value is not limited, and is specifically determined according to actual demand, and the third preset value is greater than the second preset value.

[0634] Based on the above steps, the encoding side screens S weight derivation modes from P weight derivation modes, and then performs the following steps S201-A11-22.

[0635] S201-A11-22 selects T prediction modes from the Q prediction modes, where T is a positive integer equal to or smaller than Q.

[0636] In the present embodiment, the method of selecting T prediction modes from Q prediction modes is not limited.

[0637] In some embodiments, T pre-defined prediction modes are selected from the Q prediction modes.

[0638] In some embodiments, for an i-th prediction mode among the Q prediction modes, the encoding side determines a cost when predicting a template using the i-th prediction mode, where i is a positive integer from 1 to Q, and selects T prediction modes from the Q prediction modes based on the cost corresponding to the i-th prediction mode.

[0639] In the present embodiment, the weight derivation mode and K prediction modes are treated as one combination, and the cost is calculated. Thus, in order to facilitate the calculation, the costs of Q weight derivation modes are calculated based on the given weight derivation mode and other K-1 prediction modes, that is, each of the Q prediction modes is combined with the set weight derivation mode and the K-1 prediction modes to obtain Q combinations, the costs corresponding to each combination among the Q combinations are calculated, and the costs of the Q prediction modes are further obtained.

[0640] For example, the given K-1 prediction modes are set as prediction mode 1, the given weight derivation mode is set as weight derivation mode 1, and for the i-th prediction mode among the Q prediction modes, the weight derivation mode 1, the i-th prediction mode, and the prediction mode 1 form one combination, which is set as combination i. The combination i is used to predict the template of the current block, and a predicted value of the template in the combination i is obtained. Based on the predicted value of the template in the combination i and the reconstructed value of the template, a prediction distortion cost corresponding to the combination i is determined, and the prediction distortion cost corresponding to the combination i is determined as the cost corresponding to the i-th prediction mode. In this way, the cost corresponding to any one of the Q prediction modes can be determined.

[0641] Based on the above method, a cost corresponding to the i-th prediction mode among the Q prediction modes is determined, and then T prediction modes are selected from the Q prediction modes based on the cost corresponding to the i-th prediction mode.

[0642] The encoding side selects T prediction modes from Q prediction modes based on the cost corresponding to the i-th prediction mode, including the following manner.

[0643] In the first method, when the cost corresponding to the i-th prediction mode is smaller than a fourth preset value, a prediction mode similar to the i-th prediction mode is selected from the Q prediction modes, and T prediction modes are determined based on the i-th prediction mode and the similar prediction mode of the i-th prediction mode. A prediction mode similar to the i-th prediction mode may be understood as a prediction mode whose prediction result is similar (or close) to the prediction result of the i-th prediction mode, for example, a prediction mode whose prediction direction (or angle) is close to the prediction direction (or angle) of the i-th prediction mode, or a prediction mode whose prediction mode index is close to the index of the i-th prediction mode, for example, a prediction mode whose index is 1, 2, etc. larger than the index of the i-th prediction mode, or a prediction mode whose index is 1, 2, etc. smaller than the index of the i-th prediction mode.

[0644] In the present embodiment, the magnitude of the fourth preset value is not limited, and is specifically determined according to actual needs.

[0645] In a second method, when a cost corresponding to the i-th prediction mode is greater than a fifth preset value, the i-th prediction mode and prediction modes similar to the i-th prediction mode are removed from the Q prediction modes, at least one prediction mode is obtained after the removal, and T prediction modes are determined based on the at least one prediction mode after the removal.

[0646] In the present embodiment, the magnitude of the fifth preset value is not limited, and is specifically determined according to actual needs, and the fifth preset value is greater than the fourth preset value.

[0647] Based on the above steps, select S weight derivation modes from the P weight derivation modes, select T prediction modes from the Q prediction modes, and then execute the following S201-A11-23.

[0648] S201-A11-23, constructing R second combinations based on the S weight derivation modes and the T prediction modes.

[0649] Specifically, one weight derivation mode is selected from the S weight derivation modes, and K prediction modes are selected from the T prediction modes, and this one weight derivation mode and the K prediction modes constitute one second combination, and the step is repeatedly performed to obtain R second combinations.

[0650] In some embodiments, S201-A11-2 above includes: for an i-th weight derivation mode among the P weight derivation modes, determining a cost when predicting a template using the i-th weight derivation mode and a j-th prediction mode among the Q prediction modes; if the cost corresponding to the combination of the i-th weight derivation mode and the j-th prediction mode is greater than a sixth preset value, removing the j-th prediction mode and a prediction mode similar to the j-th prediction mode from the Q prediction modes to obtain at least one prediction mode after the removal; and constructing R second combinations based on the i-th weight derivation mode and the at least one prediction mode after the removal.

[0651] In this embodiment, when a weight derivation mode and one prediction mode are fixed, another prediction mode is screened. For example, when a certain intra prediction mode is set as the first prediction mode in a certain weight derivation mode, if a small cost cannot be obtained, the intra prediction mode similar to the certain intra prediction mode in the certain weight derivation mode is not tried to be set as the first prediction mode.

[0652] Based on the above steps, a final set of removed prediction modes corresponding to each weight derivation mode among the P weight derivation modes can be determined, and thus R second combinations are constructed based on the P weight derivation modes and their respective corresponding final set of removed prediction modes.

[0653] It should be noted that the above embodiment shows a method of screening prediction modes in the form of combinations, and optionally, any one of the weight derivation modes and prediction modes may be further screened in the form of combinations, and R second combinations may finally be constructed.

[0654] The encoding side determines R second combinations based on the above methods, and then, for any one of the R second combinations, uses the weight derivation mode in the second combination and the K prediction modes to predict a template and obtain a template prediction value corresponding to the second combination.

[0655] Hereinafter, a process of predicting a template of a current block using any one of the second combinations and obtaining a predicted value of the template will be described.

[0656] First, the weights of the templates are determined using the weight derivation mode included in the second combination.

[0657] Currently, as shown in FIG. 13 above, the template of the current block is the coded area above the current block, or the coded area to the left of the current block, or the coded area to the left and above the current block.

[0658] In some embodiments, the weight matrix may be directly extended to the template region, for example, extended to the upper left side, and the weight matrix may be overlaid on the template, thereby determining the weight of the template. For example, as shown in FIG. 17, a small rectangular region in the upper left region of the current block may be added to the template, and the template and the current block may be combined to form a rectangle. Of course, only the left and upper parts may be used as the template. As shown in FIG. 17, the upper left region is taken as an example, and the region of the inverted L-shaped block consisting of the left side, upper left side, and upper side is the template region, and the rectangular region on the lower right side is the current block, and the weight matrix extended to the upper left side becomes the weight matrix of the template.

[0659] In some embodiments, the process of deriving the template weight based on the weight derivation mode and the process of deriving the weight of the predictor are merged, for example, the template weight and the weight of the predictor are derived simultaneously. The weight of the predictor may be understood as a weight corresponding to a predictor, for example, a first predicted value is obtained using a first prediction mode, a second predicted value is obtained using a second prediction mode, a first weight of the first predicted value is determined based on the weight derivation mode, a second weight of the second predicted value is determined based on the weight derivation mode, and a sum value of the product of the first predicted value and the first weight and the product of the second predicted value and the second weight is determined as the predicted value of the current block.

[0660] In this application, the first and second weights are referred to as predictor weights to distinguish them from the template weights.

[0661] In some embodiments, determining the template weights based on the weight derivation mode includes: Step 3 of determining an angle index and a distance index based on a weight derivation mode; and step 4 determining template weights based on the angle index, the distance index and the template magnitude.

[0662] In the present application, the template weight may be derived in a similar manner to the derivation of the weight of the predicted value. For example, first, an angle index and a distance index are determined based on a weight derivation mode, and the angle index may be understood as a boundary angle index of each weight derived by the weight derivation mode. Exemplarily, an angle index and a distance index corresponding to the weight derivation mode may be determined based on the above Table 2, for example, when the weight derivation mode is 27, the corresponding angle index is 12 and the distance index is 3. Then, the template weight is determined based on the angle index, the distance index and the size of the template.

[0663] Determining the template weight based on the angle index, distance index and template size in the above step 4 includes, but is not limited to, the following manners.

[0664] Method 1: Determine the template weight directly based on the angle index, distance index and template size. At this time, the above step 4 is: determining 41 a first parameter of a pixel point in the template based on the angle index, the distance index and the size of the template; determining 42 weights for pixel points in the template based on first parameters of the pixel points in the template; determining 43 template weights based on the weights of the pixel points in the template.

[0665] In this embodiment, weights of pixel points in the template are determined based on the angle index, the distance index, the size of the template, and the size of the current block, and a weight matrix consisting of the weights of each point in the template is determined as the template weight.

[0666] The first parameter herein is used to determine the weight, and in some embodiments, the first parameter is also referred to as a weight index.

[0667] In one possible implementation, the offset amount and the first parameter may be determined based on the following scheme.

[0668] As shown in FIG. 18, the inputs for the template weight derivation process include the width nCbW of the current block, the height nCbH of the current block, the width nTmW of the left template, the height nTmH of the upper template, the GPM “division” angle index variable angleId, the GPM distance index variable distanceIdx, and the component index variable cIdx. For example, in this application, since the luminance component is taken as an example, cIdx is 0, which represents the luminance component.

[0669] The variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor are derived in the following manner.

[0670] nW = ( cIdx = = 0 ) ? nCbW : nCbW * SubWidthC nH = ( cIdx = = 0 ) ? nCbH : nCbH * SubHeightC shift1 = Max( 5, 17 - BitDepth ), where BitDepth is the bit depth of the codec.

[0671] offset1 = 1 << ( shift1 - 1 ) displacementX = angleIdx displacementY = (angleIdx + 8) % 32 partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1 shiftHor = ( angleIdx % 16 = = 8 | | ( angleIdx % 16 != 0 && nH >= nW ) ) ? 0 : 1 The offset amounts offsetX and offsetY are derived as follows.

[0672] - If the value of shiftHor is 0, offsetX = ( -nW ) >> 1 offsetY = ( ( -nH ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nH ) >> 3 : -( ( distanceIdx * nH ) >> 3 ) ) - Otherwise (i.e., the value of shiftHor is 1), offsetX = ( ( -nW ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nW ) >> 3 : -( ( distanceIdx * nW ) >> 3 ) offsetY = ( - nH ) >> 1 Template weight matrix wTemplateValue[x][y] (x = -nTmW..nCbW - 1, y = -nTmH..nCbH - 1, except when both x and y are 0 or greater). However, in this example, the coordinates of the upper left corner of the current block are (0,0) and it is derived as follows:

[0673] The variables xL and yL are derived in the following way:

[0674] xL =( cIdx = = 0 ) ? x : x * SubWidthC yL =( cIdx = = 0 ) ? y : y * SubHeightC The disLut is determined according to Table 3 above.

[0675] The first parameter weightIdx is derived in the following manner.

[0676] weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ] After determining the first parameter weightIdx based on the above method, the weight of the point (x, y) in the template is determined based on weightIdx.

[0677] In the present application, determining the weight of the pixel point in the template based on the first parameter of the pixel point in the template in the above step 42 includes, but is not limited to, the following manners.

[0678] Method 1: determine a second parameter of the pixel point in the template based on a first parameter of the pixel point in the template, and determine a weight of the pixel point in the template based on the second parameter of the pixel point in the template.

[0679] A second parameter is also used to determine the weight. In some embodiments, the second parameter is also referred to as a weight index on the first component, which may be a luma component, a chroma component, etc.

[0680] For example, the weight of each pixel point in the template is determined by the following formula.

[0681] weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx wTemplateValue[x][y] = Clip3( 0, 8, ( weightIdxL + 4 ) >> 3 ) where wTemplateValue[x][y] is the weight of point (x,y) in the template, weightIdxL is the second parameter of point (x,y) in the template, also called the weight index in the first component (e.g., the luminance component), wTemplateValue[x][y] is the weight of point (x,y) in the template, and partFlip is an intermediate variable that is determined based on the angle index angleIdx, for example as described above, partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1, that is, the value of partFlip is 1 or 0, and when partFlip is 0, weightIdxL is 32 - weightIdx, and when partFlip is 1, weightIdxL is 32 + weightIdx, it should be noted that 32 here is just an example and the present application is not limited thereto.

[0682] Method 2: determining a weight of a pixel point in the template according to a first parameter of the pixel point in the template, a first threshold value and a second threshold value.

[0683] In order to reduce the calculation complexity of the template weight, in method 2, the weight of the pixel point in the template is limited to a first threshold or a second threshold, that is, the weight of the pixel point in the template is not the first threshold but the second threshold, which further reduces the calculation complexity of the template weight.

[0684] In the present application, the first threshold value and the second threshold value are not limited to specific values.

[0685] Optionally, the first threshold is one.

[0686] Optionally, the second threshold is zero.

[0687] In one example, the weights of pixel points in the template may be determined by the following formula:

[0688] wTemplateValue[x][y] =(partFlip ? weightIdx: - weightIdx) > 0 ? 1 : 0 wTemplateValue[x][y] is the weight of the point (x, y) in the template, and the 1 in the above "1:0" is the first threshold and the 0 is the second threshold.

[0689] In the above method 1, the weight of each point in the template is determined by the weight derivation mode, and a weight matrix consisting of the weights of each point in the template is defined as the template weight.

[0690] Method 2: Determine the weights of the current block and the template according to the weight derivation mode, that is, in method 2, the entire merged region consisting of the current block and the template is taken as one region, and the weights of the pixel points in the merged region are derived according to the weight derivation mode. Based on this, the above step 4 is: Step 4-1: determining weights of pixel points in a merged region consisting of the current block and the template based on an angle index, a distance index, a size of the template, and a size of the current block; and step 4-2 of determining template weights based on the size of the template and the weights of pixel points in the merge region.

[0691] In method 2, the current block and the template are treated as one region, and weights of pixel points in the merged region consisting of the current block and the template are determined based on the angle index, the distance index, the size of the template, and the size of the current block. Furthermore, based on the size of the template, the weight corresponding to the template in the merged region is determined as the template weight. For example, as shown in FIG. 18, the weight corresponding to an L-shaped template region in the merged region is determined as the template weight.

[0692] In the method 2, in the process of determining the weight, the template weight is determined and the prediction weight is also determined. That is, after removing the template weight from the weight corresponding to the merged region, the remaining weight is the prediction weight. Then, the subsequent prediction process can be performed based on the prediction weight, and there is no need to re-determine the prediction weight, which further reduces the prediction steps and improves the prediction efficiency.

[0693] This application does not limit the specific implementation of determining the weights of pixel points in the merged region consisting of the current block and the template based on the angle index, the distance index, the size of the template, and the size of the current block as described above.

[0694] In some embodiments, determining weights of pixel points in the merged region in step 4-1 above comprises: A step 4-11 of determining a first parameter of the pixel point in the merged region based on the angle index, the distance index and the size of the merged region; and determining weights of the pixel points in the merged region based on the first parameters of the pixel points in the merged region.

[0695] In this implementation, weights of pixel points in the merged region and a weight matrix consisting of weights of each point in the merged region are determined based on the angle index, the distance index, and the size of the merged region.

[0696] In one possible implementation, the offset amount and the first parameter may be determined based on the following scheme.

[0697] As shown in FIG. 18, the inputs to the process of deriving the merge region weight include the width nCbW of the current block, the height nCbH of the current block, the width nTmW of the left template, the height nTmH of the upper template, the GPM “division” angle index variable angleId, the GPM distance index variable distanceIdx, and the component index variable cIdx. For example, in this application, since the luminance component is taken as an example, cIdx is 0, which represents the luminance component.

[0698] The variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor are derived in the following manner.

[0699] nW = ( cIdx = = 0 ) ? nCbW : nCbW * SubWidthC nH = ( cIdx = = 0 ) ? nCbH : nCbH * SubHeightC shift1 = Max( 5, 17 - BitDepth ), where BitDepth is the bit depth of the codec.

[0700] offset1 = 1 << ( shift1 - 1 ) displacementX = angleIdx displacementY = (angleIdx + 8) % 32 partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1 shiftHor = ( angleIdx % 16 = = 8 | | ( angleIdx % 16 != 0 && nH >= nW ) ) ? 0 : 1 The offset amounts offsetX and offsetY are derived as follows.

[0701] - If the value of shiftHor is 0, offsetX = ( -nW ) >> 1 offsetY = ( ( -nH ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nH ) >> 3 : -( ( distanceIdx * nH ) >> 3 ) ) - Otherwise (i.e., the value of shiftHor is 1), offsetX = ( ( -nW ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nW ) >> 3 : -( ( distanceIdx * nW ) >> 3 ) offsetY = ( - nH ) >> 1 Weight matrix of pixel points in the merged region, wValueMatrix[x][y] (x = -nTmW..nCbW - 1, y = -nTmH..nCbH - 1). In this example, the coordinates of the upper left corner of the current block are (0,0) and are derived as follows:

[0702] The variables xL and yL are derived in the following way:

[0703] xL = ( cIdx = = 0 ) ? x : x * SubWidthC yL = ( cIdx = = 0 ) ? y : y * SubHeightC The disLut is determined according to Table 3.

[0704] The first parameter weightIdx is derived in the following manner.

[0705] weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ] After determining the first parameter weightIdx based on the above method, determine the weight of the point (x, y) in the merged region based on weightIdx.

[0706] In the present application, determining the weight of the pixel point in the merged region based on the first parameter of the pixel point in the merged region in the above step 4-12 includes, but is not limited to, the following manners.

[0707] Method 1: determine a second parameter of the pixel point in the merged region according to a first parameter of the pixel point in the merged region, and determine a weight of the pixel point in the merged region according to the second parameter of the pixel point in the merged region.

[0708] For example, the weights of the pixel points in the merged region are determined based on the following formula:

[0709] weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ] weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx wValueMatrix [x][y] = Clip3( 0, 8, ( weightIdxL + 4 ) >> 3 ) wValueMatrix[x][y] is the weight of the point (x,y) in the merged region, weightIdxL is the second parameter of the point (x,y) in the merged region, and wTemplateValue[x][y] is the weight of the point (x,y) in the merged region.

[0710] Method 2: determine weights of pixel points in the merged region according to a first parameter, a first threshold and a second threshold of the pixel points in the merged region.

[0711] Exemplarily, the weight of the pixel point in the merged region is a first threshold or a second threshold.

[0712] In order to reduce the calculation complexity of the merged region weight, the weight of the pixel point in the merged region in method 2 is limited to the first threshold or the second threshold, that is, the weight of the pixel point in the merged region is not the first threshold but the second threshold, which further reduces the calculation complexity of the merged region weight.

[0713] In the present application, the first threshold value and the second threshold value are not limited to specific values.

[0714] Optionally, the first threshold is one.

[0715] Optionally, the second threshold is zero.

[0716] In one example, the weight of a pixel point in the merged region may be determined by the following formula:

[0717] wTemplateValue[x][y] = (partFlip ? weightIdx: - weightIdx) > 0 ? 1 : 0 wValueMatrix[x][y] is the weight of the point (x, y) in the merged region, and the 1 in the above "1:0" is the first threshold and the 0 is the second threshold.

[0718] In the above-mentioned method 2, the weight of each point in the merged region consisting of the current block and the template is determined according to the weight derivation mode, and the weight corresponding to the template in the merged region is determined as the template weight according to the size of the template. In addition, in the above-mentioned method 2, the weight of the predicted value is determined at the same time, so that there is no need to perform a subsequent step of determining the weight of the predicted value, thereby reducing the prediction steps and improving the prediction efficiency.

[0719] Based on the above method, determine template weights corresponding to the weight derivation modes included in the second combination, then predict the template using the K prediction modes in the second combination to obtain K predicted values, and weight the K predicted values ​​based on the template weights to obtain a predicted value of the template.

[0720] The predictions of a template may be understood as a matrix of predictions of pixel points in the template.

[0721] In some embodiments, the predicted values ​​are also referred to as predicted samples.

[0722] A cost for the second combination is then determined based on the predicted and reconstructed values ​​of the template.

[0723] The above-mentioned method of determining the cost of the second combination includes, but is not limited to, the following methods.

[0724] Method 1: adopt a matrix method to determine the cost of the second combination. Specifically, a loss is determined according to the predicted value and reconstructed value of the template, and the loss is the first loss. The predicted value and reconstructed value of the template are matrices, so the first loss obtained is also a matrix, for example, the absolute value of the difference between the predicted value and reconstructed value of the template is determined as the first loss, and the first loss is determined as the cost of the second combination.

[0725] Method 2: A point-by-point calculation method is adopted to determine the cost of the second combination.

[0726] Specifically, for the i-th pixel point in the template, determine a predicted value of each prediction mode of the K prediction modes in the second combination at the i-th pixel point, determine a corresponding template weight in the template weight of the i-th pixel point, and obtain a predicted value of the i-th pixel point according to the template weight of the i-th pixel point and the K predicted values ​​of the i-th pixel point. Obtain a cost at the i-th pixel point of the second combination according to the predicted value and the reconstructed value of the i-th pixel point. Based on the method, a predicted distortion cost at each pixel point in the template of the second combination can be determined, and finally, the sum of the predicted distortion costs at each pixel point in the template of the second combination is determined as the cost of the second combination.

[0727] Based on the above method, the cost of each of the R second combinations can be determined.

[0728] Then, a candidate combination list is constructed based on the cost of each of the R second combinations.

[0729] For example, the R second combinations are ordered in ascending order of cost of the second combination, and the ordered R second combinations are determined as a candidate combination list.

[0730] Also, for example, based on the magnitude of the cost of the second combination, N second combinations having the smallest cost are selected from the R second combinations to form a candidate combination list.

[0731] Optionally, N is 8 or 16 or 32, etc.

[0732] Based on the above method, a candidate combination list is determined, and each candidate combination in the candidate combination list is ordered in ascending order of cost.

[0733] In this way, the encoding side determines the first combination in the candidate combination list shown in Table 6, for example, the candidate combination with the smallest cost in the candidate combination list is determined as the first combination, that is, the weight derivation mode included in the candidate combination is determined as the first weight derivation mode, and the K prediction modes included in the candidate combination are determined as the K first prediction modes. Then, the encoding side executes S202 below.

[0734] S202, predicting a current block according to a first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block.

[0735] For example, a weight of a prediction value is determined based on a first weight derivation mode, K prediction values ​​are determined based on K first prediction modes, and a prediction value of the current block is determined based on the K prediction values ​​and the weights of the prediction values.

[0736] In one example, the weights of the predictor values ​​derived by the first weight derivation mode are a weight matrix.

[0737] The implementation process of S202 is almost the same as S103 above, so please refer to the description of S103 above and will not be described further here.

[0738] S203, obtain a codestream including a first index for indicating a first combination according to a prediction value of a current block.

[0739] Specifically, after determining the predicted value of the current block based on the above steps, a residual of the current block is obtained based on the current block and the predicted value of the current block, the residual of the current block is transformed to obtain transform coefficients, the transform coefficients are quantized to obtain quantized coefficients, and the quantized coefficients are coded to obtain a code stream.

[0740] In this embodiment, in order to ensure the consistency between encoding and decoding, the encoding side indicates the first combination to the decoding side by a first index, i.e., writes the first index into the code stream. In this way, the decoding side obtains the first index by decoding the code stream, and further determines the first combination corresponding to the current block based on the first index, predicts the current block using the first weight derivation mode and K first prediction modes included in the first combination, and obtains a predicted value of the current block.

[0741] In the present embodiment, the specific format of the syntax element of the first index is not l...

Claims

1. 1. A video decoding method, comprising: Decoding the codestream to obtain a first index, the first index being used to indicate a first combination of a weight derivation mode and K prediction modes, where K is a positive integer greater than 1; determining a first weight derivation mode and K first prediction modes included in the first combination based on the first index; predicting a current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block.

2. Determining a first weight derivation mode and K first prediction modes included in the first combination based on the first index includes: determining a candidate combination list, the candidate combination list including a plurality of candidate combinations, and any one of the plurality of candidate combinations including one weight derivation mode and K prediction modes; determining a weight derivation mode and K prediction modes included in a candidate combination corresponding to the first index in the candidate combination list as the first weight derivation mode and the K first prediction modes.

3. The step of determining the candidate combination list includes:

3. The method of claim 2, further comprising constructing the candidate combination list.

4. The constructing of the candidate combination list includes:

4. The method of claim 3, comprising constructing the list of candidate combinations based on a template of the current block.

5. The constructing the candidate combination list based on the template of the current block includes: determining R second combinations, where any one of the R second combinations includes one weight derivation mode and K prediction modes, and the weight derivation mode and the K prediction modes included in any two of the R second combinations are not completely similar, and the R is a positive integer greater than 1; predicting the template using a weight derivation mode and K prediction modes in the second combination for any one of the R second combinations, and obtaining a template prediction value corresponding to the second combination; and constructing the candidate combination list based on a template prediction value corresponding to each second combination among the R second combinations.

6. constructing the candidate combination list based on a template prediction value corresponding to each second combination among the R second combinations, determining a cost corresponding to the second combination based on the template prediction value and the template reconstruction value corresponding to the second combination; and constructing the list of candidate combinations based on a cost corresponding to each second combination among the R second combinations.

7. constructing the candidate combination list based on a cost corresponding to each second combination among the R second combinations, ordering the R second combinations based on a cost corresponding to each second combination among the R second combinations; and determining the R second ordered combinations as the list of candidate combinations.

8. The ordering of the R second combinations based on a cost corresponding to each second combination among the R second combinations includes:

8. The method of claim 7, further comprising ordering the R second combinations in ascending order of cost.

9. constructing the candidate combination list based on a cost corresponding to each second combination among the R second combinations, selecting N second combinations from the R second combinations based on costs corresponding to the second combinations; and determining the list of N second combinations as the list of candidate combinations.

10. 10. The method of claim 9, wherein the N second combinations are the N second combinations with the smallest cost among the R second combinations.

11. Determining the R second combinations includes: determining P weight derivation modes and Q prediction modes, where P is a positive integer and Q is a positive integer equal to or greater than K; 11. The method according to claim 5, further comprising: constructing the R second combinations based on the P weight derivation modes and the Q prediction modes, wherein any one of the R second combinations includes a weight derivation mode of the P weight derivation modes and K prediction modes of the Q prediction modes.

12. Determining the Q prediction modes includes: determining at least one of a candidate prediction mode list of the current block, an alternative prediction mode list corresponding to each of the K prediction modes, and a prediction mode corresponding to a weight derivation mode, the candidate prediction mode list including a plurality of candidate prediction modes, and an alternative prediction mode list corresponding to any one of the K prediction modes including at least one alternative prediction mode; and determining the Q prediction modes based on at least one of the candidate prediction mode list, an alternative prediction mode list corresponding to each of the K prediction modes, and a prediction mode corresponding to the weight derivation mode.

13. Determining an alternative prediction mode list corresponding to each of the K prediction modes includes: determining, for any one prediction mode among the K prediction modes, at least one of a candidate prediction mode list corresponding to the prediction mode and a prediction mode corresponding to a weight derivation mode; and determining an alternative prediction mode list corresponding to the prediction mode based on at least one of the candidate prediction mode list and a prediction mode corresponding to the weight derivation mode.

14. Determining the candidate prediction mode list includes:

14. The method of claim 12 or 13, comprising determining the candidate prediction mode list based on a prediction mode used for at least one block neighboring the current block.

15. The method comprises: The method of claim 14 , further comprising: if the candidate prediction mode list does not include a preset mode, adding the preset mode to the candidate prediction mode list.

16. 16. The method of claim 15, wherein the pre-set modes include at least one of a DC, a horizontal mode, and a vertical mode.

17. 15. The method of claim 14, wherein the candidate prediction mode list includes at least one of a candidate intra-prediction mode list and a candidate inter-prediction mode list, the candidate intra-prediction mode list including at least one candidate intra-prediction mode and the candidate inter-prediction mode list including at least one candidate inter-prediction mode.

18. Determining a prediction mode corresponding to the weight derivation mode includes: For a p-th weight derivation mode among the P weight derivation modes, determining a prediction mode corresponding to the p-th weight derivation mode, where the p is a positive integer; and determining a prediction mode corresponding to at least one of the P weight derivation modes based on a prediction mode corresponding to the at least one weight derivation mode of the P weight derivation modes.

19. When at least one of the prediction modes corresponding to the p-th weight derivation mode is an intra prediction mode, determining the prediction mode corresponding to the p-th weight derivation mode includes: determining an angle index based on the pth weight derivation mode; and determining an intra-prediction mode corresponding to the angular index as at least one of the prediction modes corresponding to the p-th weight derivation mode.

20. When at least one of the prediction modes corresponding to the p-th weight derivation mode is an intra prediction mode, determining the prediction mode corresponding to the p-th weight derivation mode includes: determining an intra prediction mode corresponding to the p-th weight derivation mode; and determining at least one of the intra prediction modes corresponding to the p-th weight derivation mode as at least one of the prediction modes corresponding to the p-th weight derivation mode.

21. The method of claim 20, wherein the intra prediction mode corresponding to the p-th weight derivation mode includes at least one of an intra prediction mode in which a prediction angle is parallel to the boundary line corresponding to the weight derivation mode, an intra prediction mode in which a prediction angle is perpendicular to the boundary line corresponding to the weight derivation mode, and a planar mode.

22. The method of claim 11 , wherein Q is less than or equal to a first preset threshold.

23. The method of claim 11 , wherein the magnitude of Q is related to the size and / or shape of the current block.

24. The method of claim 23, wherein Q is greater than or equal to a second preset threshold if the size of the current block is greater than a first value.

25. The method of claim 23, wherein Q is less than a third preset threshold if the size of the current block is less than or equal to a first value.

26. The step of determining the P weight derivation modes includes the steps of:

12. The method of claim 11, comprising selecting the P weight derivation modes from M preset weight derivation modes, where M is a positive integer greater than or equal to P.

27. The selection of the P weight derivation modes from the M preset weight derivation modes includes:

27. The method according to claim 26, further comprising removing a weight derivation mode corresponding to a preset division angle and / or a preset offset amount from the M weight derivation modes to obtain the P weight derivation modes.

28. The selection of the P weight derivation modes from the M preset weight derivation modes includes: determining a screening condition corresponding to the current block; and selecting the P weight derivation modes from the M weight derivation modes based on the screening criteria.

29. The method of claim 28 , wherein the screening conditions corresponding to the current block include screening conditions corresponding to a size of the current block and / or screening conditions corresponding to a shape of the current block.

30. 30. The method of claim 29, wherein the screening condition includes an array including M elements in one-to-one correspondence with the M weight derivation modes, the element corresponding to the weight derivation mode being used to indicate whether the weight derivation mode is enabled or not.

31. constructing the R second combinations based on the P weight derivation modes and the Q prediction modes, selecting S weight derivation modes from the P weight derivation modes, where S is a positive integer less than or equal to P; selecting T prediction modes from the Q prediction modes, where T is a positive integer less than or equal to Q; and constructing the R second combinations based on the S weight derivation modes and the T prediction modes.

32. The selecting S weight derivation modes from the P weight derivation modes includes: determining weights of K second prediction modes in the template based on an i-th weight derivation mode among the P weight derivation modes, the K second prediction modes being any K prediction modes among the Q prediction modes, where the i is a positive integer from 1 to P; 32. The method of claim 31 , further comprising: removing the i-th weight derivation mode from the P weight derivation modes to obtain the S weight derivation modes when a weight in the template of any one of the K prediction modes is smaller than a first preset value.

33. The selecting S weight derivation modes from the P weight derivation modes includes: determining a cost of predicting the template using an i-th weight derivation mode for an i-th weight derivation mode among the P weight derivation modes, where i is a positive integer from 1 to P; and selecting the S weight derivation modes from the P weight derivation modes based on a cost corresponding to the i-th weight derivation mode.

34. selecting the S weight derivation modes from the P weight derivation modes based on a cost corresponding to the i-th weight derivation mode, selecting a weight derivation mode similar to the i-th weight derivation mode from the P weight derivation modes when a cost corresponding to the i-th weight derivation mode is smaller than a second preset value; and determining the S weight derivation modes based on the i-th weight derivation mode and similar weight derivation modes of the i-th weight derivation mode.

35. selecting the S weight derivation modes from the P weight derivation modes based on a cost corresponding to the i-th weight derivation mode, When a cost corresponding to the i-th weight derivation mode is greater than a third preset value, removing the i-th weight derivation mode and weight derivation modes similar to the i-th weight derivation mode from the P weight derivation modes to obtain at least one weight derivation mode after the removal; and determining the S weight derivation modes based on the removed at least one weight derivation mode.

36. The selecting T prediction modes from the Q prediction modes includes: determining a cost for predicting the template using an i-th prediction mode among the Q prediction modes, where i is a positive integer from 1 to Q; and selecting the T prediction modes from the Q prediction modes based on a cost corresponding to the i-th prediction mode.

37. The selecting of the T prediction modes from the Q prediction modes based on a cost corresponding to the i-th prediction mode includes: selecting a prediction mode similar to the i-th prediction mode from the Q prediction modes when a cost corresponding to the i-th prediction mode is smaller than a fourth preset value; and determining the T prediction modes based on the i-th prediction mode and similar prediction modes of the i-th prediction mode.

38. The selecting of the T prediction modes from the Q prediction modes based on a cost corresponding to the i-th prediction mode includes: When a cost corresponding to the i-th prediction mode is greater than a fifth preset value, removing the i-th prediction mode and a prediction mode similar to the i-th prediction mode from the Q prediction modes to obtain at least one prediction mode after the removal; and determining the T prediction modes based on the removed at least one prediction mode.

39. constructing the R second combinations based on the P weight derivation modes and the Q prediction modes, determining a cost for predicting the template using an i-th weight derivation mode among the P weight derivation modes and a j-th prediction mode among the Q prediction modes; When a cost corresponding to the combination of the i-th weight derivation mode and the j-th prediction mode is greater than a sixth preset value, removing the j-th prediction mode and a prediction mode similar to the j-th prediction mode from the Q prediction modes to obtain at least one prediction mode after the removal; and constructing the R second combinations based on the i-th weight derivation mode and the removed at least one prediction mode.

40. The step of predicting a current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block includes: determining weights for the predictors based on the first weight derivation mode; determining the K predicted values ​​based on the K first prediction modes; 11. The method according to claim 1, further comprising: weighting the K predictors based on weights of the K predictors to obtain a prediction of the current block.

41. 41. The method of claim 40, wherein the predictor weights are a weight matrix.

42. 11. The method of any one of claims 1 to 10, characterized in that the first index is embedded in the codestream in a variable length coding scheme.

43. The method according to any one of claims 2 to 10, characterized in that, when the number of candidate combinations included in the candidate combination list is smaller than a fourth preset threshold, the first index is incorporated into the code stream using a fixed-length coding method.

44. The method comprises: decoding the codestream and obtaining at least one flag to indicate decoding using a first combination; The step of decoding the codestream and obtaining the first index includes:

11. The method of claim 1, further comprising: decoding the codestream and obtaining the first index if the at least one flag indicates decoding using the first combination.

45. 45. The method of claim 44, wherein the at least one flag comprises at least one of a sequence level flag, a picture level flag, a slice level flag, a unit level flag, and a block level flag.

46. 1. A video encoding method, comprising: determining a first combination comprising a first weight derivation mode and K first prediction modes; predicting a current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block; and obtaining a codestream including a first index for indicating the first combination based on the predicted value of the current block.

47. The step of determining the first combination includes: determining a candidate combination list, the candidate combination list including a plurality of candidate combinations, and any one of the plurality of candidate combinations including one weight derivation mode and K prediction modes; and determining the first combination from the list of candidate combinations.

48. The step of determining the candidate combination list includes:

48. The method of claim 47, further comprising constructing the candidate combination list.

49. The constructing of the candidate combination list includes:

49. The method of claim 48, comprising building the list of candidate combinations based on a template of the current block.

50. The constructing the candidate combination list based on the template of the current block includes: determining R second combinations, where any one of the R second combinations includes one weight derivation mode and K prediction modes, and the weight derivation mode and the K prediction modes included in any two of the R second combinations are not completely similar, and the R is a positive integer greater than 1; predicting the template using a weight derivation mode and K prediction modes in the second combination for any one of the R second combinations, and obtaining a template prediction value corresponding to the second combination; and constructing the list of candidate combinations based on a template prediction value corresponding to each second combination among the R second combinations.

51. constructing the candidate combination list based on a template prediction value corresponding to each second combination among the R second combinations, determining a cost corresponding to the second combination based on the template prediction value and the template reconstruction value corresponding to the second combination; and constructing the list of candidate combinations based on a cost corresponding to each second combination among the R second combinations.

52. constructing the candidate combination list based on a cost corresponding to each second combination among the R second combinations, ordering the R second combinations based on a cost corresponding to each second combination among the R second combinations; and determining the R second ordered combinations as the list of candidate combinations.

53. The ordering of the R second combinations based on a cost corresponding to each second combination among the R second combinations includes:

53. The method of claim 52, comprising ordering the R second combinations in order of increasing cost.

54. constructing the candidate combination list based on a cost corresponding to each second combination among the R second combinations, selecting N second combinations from the R second combinations based on costs corresponding to the second combinations; and determining the list of N second combinations as the list of candidate combinations.

55. 55. The method of claim 54, wherein the N second combinations are the N second combinations among the R second combinations that have the smallest cost.

56. Determining the R second combinations includes: determining P weight derivation modes and Q prediction modes, where P is a positive integer and Q is a positive integer equal to or greater than K; 56. The method of claim 49, further comprising: constructing the R second combinations based on the P weight derivation modes and the Q prediction modes, wherein any one of the R second combinations includes a weight derivation mode of the P weight derivation modes and K prediction modes of the Q prediction modes.

57. Determining the Q prediction modes includes: determining at least one of a candidate prediction mode list of the current block, an alternative prediction mode list corresponding to each of the K prediction modes, and a prediction mode corresponding to a weight derivation mode, the candidate prediction mode list including a plurality of candidate prediction modes, and an alternative prediction mode list corresponding to any one of the K prediction modes including at least one alternative prediction mode; and determining the Q prediction modes based on at least one of the candidate prediction mode list, an alternative prediction mode list corresponding to each of the K prediction modes, and a prediction mode corresponding to the weight derivation mode.

58. Determining an alternative prediction mode list corresponding to each of the K prediction modes includes: determining, for any one prediction mode among the K prediction modes, at least one of a candidate prediction mode list corresponding to the prediction mode and a prediction mode corresponding to a weight derivation mode; and determining an alternative prediction mode list corresponding to the prediction mode based on at least one of the candidate prediction mode list and a prediction mode corresponding to the weight derivation mode.

59. Determining the candidate prediction mode list includes:

59. The method of claim 57 or 58, comprising determining the candidate prediction mode list based on a prediction mode used for at least one block neighboring the current block.

60. The method comprises:

60. The method of claim 59, further comprising adding a preset mode to the candidate prediction mode list if the candidate prediction mode list does not include the preset mode.

61. 61. The method of claim 60, wherein the pre-set modes include at least one of a DC, a horizontal mode, and a vertical mode.

62. 60. The method of claim 59, wherein the candidate prediction mode list includes at least one of a candidate intra-prediction mode list and a candidate inter-prediction mode list, the candidate intra-prediction mode list including at least one candidate intra-prediction mode and the candidate inter-prediction mode list including at least one candidate inter-prediction mode.

63. Determining a prediction mode corresponding to the weight derivation mode includes: For a p-th weight derivation mode among the P weight derivation modes, determining a prediction mode corresponding to the p-th weight derivation mode, where the p is a positive integer; 59. The method of claim 57 or 58, comprising: determining a prediction mode corresponding to at least one of the P weight derivation modes based on a prediction mode corresponding to the weight derivation mode.

64. When at least one of the prediction modes corresponding to the p-th weight derivation mode is an intra prediction mode, determining the prediction mode corresponding to the p-th weight derivation mode includes: determining an angle index based on the pth weight derivation mode; and determining an intra-prediction mode corresponding to the angular index as at least one of the prediction modes corresponding to the p-th weight derivation mode.

65. When at least one of the prediction modes corresponding to the p-th weight derivation mode is an intra prediction mode, determining the prediction mode corresponding to the p-th weight derivation mode includes: determining an intra prediction mode corresponding to the p-th weight derivation mode; and determining at least one of the intra prediction modes corresponding to the p-th weight derivation mode as at least one of the prediction modes corresponding to the p-th weight derivation mode.

66. 66. The method of claim 65, wherein the intra prediction mode corresponding to the p-th weight derivation mode includes at least one of an intra prediction mode whose prediction angle is parallel to the boundary line corresponding to the weight derivation mode, an intra prediction mode whose prediction angle is perpendicular to the boundary line corresponding to the weight derivation mode, and a planar mode.

67. 57. The method of claim 56, wherein Q is less than or equal to a first preset threshold.

68. 57. The method of claim 56, wherein the magnitude of Q is related to the size and / or shape of the current block.

69. 69. The method of claim 68, wherein Q is greater than or equal to a second preset threshold if the size of the current block is greater than a first value.

70. 69. The method of claim 68, wherein Q is less than a third preset threshold if the size of the current block is less than or equal to a first value.

71. The step of determining the P weight derivation modes includes the steps of:

57. The method of claim 56, comprising selecting the P weight derivation modes from M preset weight derivation modes, where M is a positive integer greater than or equal to P.

72. The selection of the P weight derivation modes from the M preset weight derivation modes includes:

72. The method of claim 71, further comprising removing a weight derivation mode corresponding to a preset division angle and / or a preset offset amount from the M weight derivation modes to obtain the P weight derivation modes.

73. The selection of the P weight derivation modes from the M preset weight derivation modes includes: determining a screening condition corresponding to the current block; and selecting the P weight derivation modes from the M weight derivation modes based on the screening criteria.

74. 74. The method of claim 73, wherein the screening conditions corresponding to the current block include screening conditions corresponding to a size of the current block and / or screening conditions corresponding to a shape of the current block.

75. 75. The method of claim 74, wherein the screening condition includes an array including M elements in one-to-one correspondence with the M weight derivation modes, the element corresponding to the weight derivation mode being used to indicate whether the weight derivation mode is enabled or not.

76. Constructing the R second combinations based on the P weight derivation modes and the Q prediction modes includes: selecting S weight derivation modes from the P weight derivation modes, where S is a positive integer less than or equal to P; selecting T prediction modes from the Q prediction modes, where T is a positive integer less than or equal to Q; and constructing the R second combinations based on the S weight derivation modes and the T prediction modes.

77. The selecting S weight derivation modes from the P weight derivation modes includes: determining weights of K second prediction modes in the template based on an i-th weight derivation mode among the P weight derivation modes, the K second prediction modes being any K prediction modes among the Q prediction modes, where the i is a positive integer from 1 to P; 77. The method of claim 76, further comprising: removing the i-th weight derivation mode from the P weight derivation modes to obtain the S weight derivation modes when a weight in the template of any one of the K prediction modes is smaller than a first preset value.

78. The selecting S weight derivation modes from the P weight derivation modes includes: determining a cost of predicting the template using an i-th weight derivation mode for an i-th weight derivation mode among the P weight derivation modes, where i is a positive integer from 1 to P; and selecting the S weight derivation modes from the P weight derivation modes based on a cost corresponding to the i-th weight derivation mode.

79. selecting the S weight derivation modes from the P weight derivation modes based on a cost corresponding to the i-th weight derivation mode, selecting a weight derivation mode similar to the i-th weight derivation mode from the P weight derivation modes when a cost corresponding to the i-th weight derivation mode is smaller than a second preset value; and determining the S weight derivation modes based on the i-th weight derivation mode and similar weight derivation modes of the i-th weight derivation mode.

80. selecting the S weight derivation modes from the P weight derivation modes based on a cost corresponding to the i-th weight derivation mode, When a cost corresponding to the i-th weight derivation mode is greater than a third preset value, removing the i-th weight derivation mode and weight derivation modes similar to the i-th weight derivation mode from the P weight derivation modes to obtain at least one weight derivation mode after the removal; and determining the S weight derivation modes based on the removed at least one weight derivation mode.

81. The selecting T prediction modes from the Q prediction modes includes: determining a cost for predicting the template using an i-th prediction mode among the Q prediction modes, where i is a positive integer from 1 to Q; and selecting the T prediction modes from the Q prediction modes based on a cost corresponding to the i-th prediction mode.

82. The selecting of the T prediction modes from the Q prediction modes based on a cost corresponding to the i-th prediction mode includes: selecting a prediction mode similar to the i-th prediction mode from the Q prediction modes when a cost corresponding to the i-th prediction mode is smaller than a fourth preset value; and determining the T prediction modes based on the i-th prediction mode and similar prediction modes of the i-th prediction mode.

83. The selecting of the T prediction modes from the Q prediction modes based on a cost corresponding to the i-th prediction mode includes: When a cost corresponding to the i-th prediction mode is greater than a fifth preset value, removing the i-th prediction mode and a prediction mode similar to the i-th prediction mode from the Q prediction modes to obtain at least one prediction mode after the removal; and determining the T prediction modes based on the removed at least one prediction mode.

84. Constructing the R second combinations based on the P weight derivation modes and the Q prediction modes includes: determining a cost for predicting the template using an i-th weight derivation mode among the P weight derivation modes and a j-th prediction mode among the Q prediction modes; When a cost corresponding to the combination of the i-th weight derivation mode and the j-th prediction mode is greater than a sixth preset value, removing the j-th prediction mode and a prediction mode similar to the j-th prediction mode from the Q prediction modes to obtain at least one prediction mode after the removal; and constructing the R second combinations based on the i-th weight derivation mode and the removed at least one prediction mode.

85. The step of predicting a current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block includes: determining weights for the predictors based on the first weight derivation mode; determining the K predicted values ​​based on the K first prediction modes; 56. The method of any one of claims 46 to 55, comprising weighting the K predictors based on the predictor weights to obtain a prediction of the current block.

86. 86. The method of claim 85, wherein the predictor weights are a weight matrix.

87. The method comprises:

56. The method of any one of claims 46 to 55, further comprising incorporating said first index into said codestream in a variable length coding manner.

88. The method comprises:

56. The method of claim 47, further comprising incorporating the first index into the codestream in a fixed-length coding manner if the number of candidate combinations included in the candidate combination list is less than a fourth preset threshold.

89. The method comprises: determining at least one flag to indicate whether to encode using the first combination; The step of determining the first combination includes:

56. The method of any one of claims 46 to 55, comprising determining the first combination if the at least one flag indicates to code using a first combination.

90. 90. The method of claim 89, wherein the at least one flag comprises at least one of a sequence level flag, a picture level flag, a slice level flag, a unit level flag, and a block level flag.

91. 1. A video decoding device, comprising: a decoding unit for decoding the code stream to obtain a first index, the first index being used to indicate a first combination of a weight derivation mode and K prediction modes, the K being a positive integer greater than 1; a determining unit for determining a first weight derivation mode and K first prediction modes included in the first combination based on the first index; a prediction unit for predicting a current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block.

92. 1. A video encoding device, comprising: a determining unit for determining a first combination including a first weight derivation mode and K first prediction modes; a prediction unit for predicting a current block based on the first weight derivation mode and the K first prediction modes to obtain a predicted value of the current block; and an encoding unit for obtaining a codestream including a first index for indicating the first combination based on the predicted value of the current block.

93. An electronic device comprising a processor and a memory, The memory is used to store a computer program; The electronic device, characterized in that the processor is used to call and execute a computer program stored in the memory in order to realize the method according to any one of claims 1 to 45 or 46 to 90.

94. 1. A video codec system, comprising a video encoder and a video decoder, The video decoder is adapted to implement a method according to any one of claims 1 to 45, A video codec system, characterized in that the video encoder is adapted to implement the method according to any one of claims 46 to 90.

95. A computer-readable storage medium used to store a computer program, A computer-readable storage medium, comprising a computer program causing a computer to execute the method according to any one of claims 1 to 45 or 46 to 90.

96. 1. A codestream comprising: a first index, the first index being used to indicate a first combination of one weight derivation mode and K prediction modes, where K is a positive integer greater than 1.

Images