Encoding method, decoding method, encoder, decoder, storage medium, and bitstream

Abstract

The present application provides an encoding method, a decoding method, an encoder, and a decoder. The decoding method comprises: determining a first prediction block of the current block on the basis of an affine parameter; determining a second prediction block of the current block; on the basis of the first prediction block and the second prediction block, using a geometric partition mode to determine a third prediction block of the current block; and determining a reconstruction block of the current block on the basis of the third prediction block.

Description

Encoding / decoding methods, codecs, storage media, and bitstreams Technical Field

[0001] This application relates to the field of video encoding and decoding, and more particularly to an encoding and decoding method, an encoder and decoder, a storage medium, and a bitstream. Background Technology

[0002] Video encoding and decoding technologies typically use affine models to represent motion forms that are more complex than translational motion. For example, during prediction, motion compensation can be performed based on affine models to achieve predictive coding of the current block. As the application of affine model-based predictive coding techniques becomes increasingly widespread, improving their predictive performance is a problem that needs to be addressed. Summary of the Invention

[0003] This application provides an encoding / decoding method, an encoding / decoding method, a storage medium, and a bitstream. The various aspects involved in this application are described below.

[0004] In a first aspect, a decoding method is provided, applied to a decoder, comprising: determining a first prediction block of a current block based on affine parameters; determining a second prediction block of the current block; determining a third prediction block of the current block using a geometric partitioning pattern based on the first prediction block and the second prediction block; and determining a reconstructed block of the current block based on the third prediction block.

[0005] Secondly, an encoding method is provided for an encoder, comprising: determining a first prediction block of a current block based on affine parameters; determining a second prediction block of the current block; determining a third prediction block of the current block using a geometric partitioning pattern based on the first prediction block and the second prediction block; and determining a residual block of the current block based on the third prediction block.

[0006] Thirdly, a decoder is provided, the decoder comprising: a first determining unit configured to determine a first prediction block of a current block based on affine parameters; a second determining unit configured to determine a second prediction block of the current block; a third determining unit configured to determine a third prediction block of the current block based on the first prediction block and the second prediction block, using a geometric partitioning mode; and a fourth determining unit configured to determine a reconstructed block of the current block based on the third prediction block.

[0007] Fourthly, a decoder is provided, the decoder comprising: a memory for storing a computer program; and a processor for executing the method as described in the first aspect when running the computer program.

[0008] Fifthly, an encoder is provided, the encoder comprising: a first determining unit configured to determine a first prediction block of a current block based on affine parameters; a second determining unit configured to determine a second prediction block of the current block; a third determining unit configured to determine a third prediction block of the current block using a geometric partitioning mode based on the first prediction block and the second prediction block; and a fourth determining unit configured to determine a residual block of the current block based on the third prediction block.

[0009] In a sixth aspect, an encoder is provided, the encoder comprising: a memory for storing a computer program; and a processor for executing the method as described in the second aspect when running the computer program.

[0010] In a seventh aspect, a non-volatile computer-readable storage medium is provided for storing a bit stream, the bit stream being generated by an encoding method using an encoder, or the bit stream being decoded by a decoding method using a decoder, wherein the decoding method is as described in the first aspect and the encoding method is as described in the second aspect.

[0011] Eighthly, a bitstream is provided, the bitstream comprising a bitstream generated by the method described in the second aspect.

[0012] A ninth aspect provides a computer-readable storage medium storing a computer program that, when executed, implements the method described in the first aspect or the method described in the second aspect.

[0013] In a tenth aspect, a computer program product is provided, comprising a computer program that, when executed, implements the method as described in the first or second aspect.

[0014] This application combines geometric partitioning patterns with affine models, enabling it to effectively handle scenarios with diverse motion types or moving objects, thereby improving prediction performance in such scenarios. Attached Figure Description

[0015] Figure 1 is a structural example diagram of a video encoder applicable to embodiments of this application.

[0016] Figure 2 is a structural example diagram of a video decoder applicable to embodiments of this application.

[0017] Figure 3 shows an example diagram of the control points of an affine model.

[0018] Figure 4 is an example diagram of the motion vectors of sub-blocks or sample points of the current block.

[0019] Figure 5 is an example diagram illustrating the principle of the merge mode.

[0020] Figure 6 is an example diagram showing the positional relationship between the current block and its surrounding adjacent and non-adjacent blocks.

[0021] Figure 7 is a schematic diagram of the division angle in the geometric division pattern.

[0022] Figure 8 is an example of the weighted prediction process in the geometric partitioning pattern.

[0023] Figure 9 shows an example of template matching technology.

[0024] Figure 10 is a flowchart illustrating the decoding method provided in an embodiment of this application.

[0025] Figure 11A is a schematic diagram of the division angle in a geometric division mode provided in an embodiment of this application.

[0026] Figure 11B is a schematic diagram of the division angle in a geometric division mode provided by another embodiment of this application.

[0027] Figure 12A is an example diagram of a design method for a predefined position provided in an embodiment of this application.

[0028] Figure 12B is an example diagram of a predefined location design method provided in another embodiment of this application.

[0029] Figure 13 is a flowchart illustrating the encoding method provided in an embodiment of this application.

[0030] Figure 14 is a schematic diagram of the structure of a decoder provided in one embodiment of this application.

[0031] Figure 15 is a schematic diagram of the structure of a decoder provided in another embodiment of this application.

[0032] Figure 16 is a schematic diagram of the encoder provided in one embodiment of this application.

[0033] Figure 17 is a schematic diagram of the encoder provided in another embodiment of this application. Detailed Implementation

[0034] Video encoding and decoding framework

[0035] Figure 1 is a schematic block diagram of a video encoder involved in an embodiment of this application.

[0036] It should be understood that the video encoder 100 can be used for lossy compression of images or lossless compression of images. The lossless compression can be visually lossless compression or mathematically lossless compression.

[0037] This video encoder 100 can be applied to image data in luminance / chrominance (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2, or 4:4:4, where Y represents luminance (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) used to describe color and saturation. For example, in color format, 4:2:0 means that there are 4 luminance components and 2 chrominance components (YYYYCbCr) per 4 pixels; 4:2:2 means that there are 4 luminance components and 4 chrominance components (YYYYCbCrCbCr) per 4 pixels; and 4:4:4 means full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0038] For example, the video encoder 100 reads video data and, for each image (or sub-image or frame) in the video data, divides an image into several coding tree units (CTUs). In some examples, a CTU may be called a "tree block," "Largest Coding Unit" (LCU), or "coding tree block" (CTB). Each CTU can be associated with a pixel block of equal size within the image. Each pixel can correspond to one luminance (or luma) sample and two chrominance (or chroma) samples. Therefore, each CTU can be associated with one luminance sample block and two chrominance sample blocks. A CTU can be a square block (or, the shape of the CTU is square). The size of a CTU is, for example, 256×256, 128×128, 64×64, 32×32, etc. A CTU can be further divided into several coding units (CUs) for encoding. CUs can be rectangular blocks or square blocks. The CU can be further divided into prediction units (PUs) and transform units (TUs), thus separating encoding, prediction, and transformation for more flexible processing. In one example, the CTU is divided into CUs using a quadtree structure, and the CUs are further divided into TUs and PUs using a quadtree structure. In some embodiments, the division of the prediction unit and the transform unit can be different.

[0039] It's important to note that video codec standards (such as VVC and HEVC) allow encoders to determine the size and partitioning of CUs and / or PUs based on the video content. For example, regions with simple textures or motion may tend to use larger blocks, while regions with complex textures or motion may tend to use smaller blocks. The deeper the block partitioning, the more complex and closer the blocks can be to the actual texture or motion, but correspondingly, the overhead for representing these partitions is also greater.

[0040] In some embodiments, as shown in FIG1, the video encoder 100 may include: a prediction unit 110, a residual unit 120, a transform / quantization unit 130, an inverse transform / quantization unit 140, a reconstruction unit 150, a loop filtering unit 160, a decoded image buffer 170, and an entropy coding unit 180. It should be noted that the video encoder 100 may include more, fewer, or different functional components.

[0041] In some embodiments, the prediction unit 110 includes an inter-frame prediction unit 111 and an intra-frame prediction unit 112. Because there is a strong correlation between adjacent pixels in an image of a video, intra-frame prediction is used in video encoding and decoding techniques to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent images in a video, inter-frame prediction is used in video encoding and decoding techniques to eliminate temporal redundancy between adjacent images, thereby improving coding efficiency.

[0042] Inter-frame prediction unit 111 can be used for inter-frame prediction, which can include motion estimation and motion compensation. Referring to image information from different images, inter-frame prediction uses motion information to find reference blocks in the reference images and generates prediction blocks based on these reference blocks to eliminate temporal redundancy. The motion information includes a list of reference images containing the reference image, the reference image index, and motion vectors. Motion vectors can be integer-pixel or fractional-pixel. If the motion vector is fractional-pixel, interpolation filtering needs to be used in the reference image to create the required fractional-pixel blocks. Here, the integer-pixel or fractional-pixel blocks in the reference image found based on the motion vectors are called reference blocks. Some techniques directly use the reference blocks as prediction blocks, while others process the reference blocks further to generate prediction blocks. Processing the reference blocks further to generate prediction blocks can also be understood as using the reference blocks as prediction blocks and then processing them to generate new prediction blocks.

[0043] Intra-frame prediction unit 112 refers only to information from the same image to predict pixel information within the current code image block, thereby eliminating spatial redundancy.

[0044] Intra-frame prediction has multiple prediction modes. Taking the international digital video coding standards H-series as an example, the H.264 / AVC standard has 8 angular prediction modes and 1 non-angular prediction mode, while H.265 / HEVC extends this to 33 angular prediction modes and 2 non-angular prediction modes. HEVC uses Planar, DC, and 33 angular modes for a total of 35 intra-frame prediction modes. VVC uses Planar, DC, and 65 angular modes for a total of 67 intra-frame prediction modes.

[0045] It should be noted that with the increase in angle modes, intra-frame prediction will be more accurate and better meet the needs of the development of high-definition and ultra-high-definition digital video.

[0046] The residual unit 120 can generate a residual block of the CU based on the pixel block of the CU and the prediction block of the PU of the CU. For example, the residual unit 120 can generate a residual block of the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel block of the CU and the corresponding sample in the prediction block of the PU of the CU.

[0047] Transform / quantization unit 130 can quantize transform coefficients. Transform / quantization unit 130 can quantize transform coefficients associated with the TU of the CU based on the quantization parameter (QP) value associated with the CU. Video encoder 100 can adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.

[0048] The inverse transform / quantization unit 140 can apply inverse quantization and inverse transform to the quantized transform coefficients to reconstruct the residual block from the quantized transform coefficients.

[0049] The reconstruction unit 150 can add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by the prediction unit 110 to produce a reconstructed image block associated with the TU. By reconstructing the sampled blocks of each TU of the CU in this way, the video encoder 100 can reconstruct the pixel blocks of the CU.

[0050] The loop filtering unit 160 is used to process the pixels after inverse transformation and inverse quantization to compensate for the distortion information and provide a better reference for subsequent encoded pixels. For example, it can perform deblocking filtering to reduce the block effect of pixel blocks associated with the CU.

[0051] In some embodiments, the loop filtering unit 160 includes a deblocking filtering unit, a sample adaptive compensation (SAO) unit, and an adaptive loop filtering (ALF) unit.

[0052] The decoded image buffer 170 can store reconstructed pixel blocks. The inter-frame prediction unit 111 can use a reference image containing the reconstructed pixel blocks to perform inter-frame prediction on PUs of other images. In addition, the intra-frame prediction unit 112 can use the reconstructed pixel blocks in the decoded image buffer 170 to perform intra-frame prediction on other PUs in the same image as the CU.

[0053] The entropy coding unit 180 can receive quantized transform coefficients from the transform / quantization unit 130. The entropy coding unit 180 can perform one or more entropy coding operations on the quantized transform coefficients to produce entropy-coded data.

[0054] Figure 2 is a schematic block diagram of a video decoder involved in an embodiment of this application.

[0055] As shown in Figure 2, the video decoder 200 includes: an entropy decoding unit 210, a prediction unit 220, an inverse quantization / transformation unit 230, a reconstruction unit 240, a loop filtering unit 250, and a decoded image buffer 260. It should be noted that the video decoder 200 may contain more, fewer, or different functional components.

[0056] Video decoder 200 can receive a bitstream. Entropy decoding unit 210 can parse the bitstream to extract syntax elements. As part of parsing the bitstream, entropy decoding unit 210 can parse the entropy-encoded syntax elements in the bitstream. Prediction unit 220, dequantization / transform unit 230, reconstruction unit 240, and loop filtering unit 250 can decode video data based on the syntax elements extracted from the bitstream, i.e., generate decoded video data.

[0057] In some embodiments, the prediction unit 220 includes an intra-frame prediction unit 222 and an inter-frame prediction unit 221.

[0058] Intra-prediction unit 222 can perform intra-prediction to generate prediction blocks for the PU. Intra-prediction unit 222 can use an intra-prediction mode to generate prediction blocks for the PU based on pixel blocks of spatially adjacent PUs. Intra-prediction unit 222 can also determine the intra-prediction mode of the PU based on one or more syntax elements parsed from the bitstream.

[0059] Inter-frame prediction unit 221 can construct a first reference image list (list 0) and a second reference image list (list 1) based on the syntax elements parsed from the bitstream. Furthermore, if the PU uses inter-frame prediction coding, entropy decoding unit 210 can parse the motion information of the PU. Inter-frame prediction unit 221 can determine one or more reference blocks of the PU based on the motion information of the PU. Inter-frame prediction unit 221 can generate prediction blocks for the PU based on one or more reference blocks of the PU.

[0060] The dequantization / transformation unit 230 reversibly quantizes (i.e., dequantizes) the transform coefficients associated with the TU. The dequantization / transformation unit 230 can use the QP value associated with the CU of the TU to determine the degree of quantization.

[0061] After the inverse quantization transform coefficients, the inverse quantization / transformation unit 230 can apply one or more inverse transforms to the inverse quantization transform coefficients to generate a residual block associated with the TU.

[0062] The reconstruction unit 240 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 240 can add the sample of the residual block to the corresponding sample of the prediction block to reconstruct the pixel block of the CU, thereby obtaining the reconstructed image block.

[0063] The loop filter unit 250 can perform deblocking filtering operations to reduce the block effect of pixel blocks associated with the CU.

[0064] The video decoder 200 can store the reconstructed image of the CU in the decoded image buffer 260. The video decoder 200 can use the reconstructed image in the decoded image buffer 260 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0065] The basic process of video encoding and decoding is as follows: At the encoding end, an image is divided into blocks. For the current block, prediction unit 110 uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. Residual unit 120 can calculate a residual block based on the prediction block and the original block of the current block, that is, the difference between the prediction block and the original block of the current block. This residual block can also be called residual information. This residual block is transformed and quantized by transform / quantization unit 130, which can remove information that is not sensitive to the human eye to eliminate visual redundancy. Optionally, the residual block before transformation and quantization by transform / quantization unit 130 can be called a temporal residual block, and the temporal residual block after transformation and quantization by transform / quantization unit 130 can be called a frequency residual block or a frequency domain residual block. Entropy coding unit 180 receives the quantized change coefficients output by transform / quantization unit 130, and can entropy code the quantized change coefficients to output a bitstream. For example, entropy coding unit 180 can eliminate character redundancy based on the target context model and the probability information of the binary bitstream.

[0066] At the decoding end, the entropy decoding unit 210 can parse the bitstream to obtain the prediction information and quantization coefficient matrix of the current block. The prediction unit 220 uses intra-frame prediction or inter-frame prediction to generate the prediction block of the current block based on the prediction information. The dequantization / transform unit 230 uses the quantization coefficient matrix obtained from the bitstream to perform dequantization and inverse transform on the quantization coefficient matrix to obtain the residual block. The reconstruction unit 240 adds the prediction block and the residual block to obtain the reconstructed block. The reconstructed blocks form the reconstructed image. The loop filtering unit 250 performs loop filtering on the reconstructed image based on the image or based on the blocks to obtain the decoded image. The encoding end also needs similar operations to the decoding end to obtain the decoded image. This decoded image can also be called the reconstructed image, which can be used as a reference image for inter-frame prediction of subsequent images.

[0067] It should be noted that the block partitioning information determined at the encoding end, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, are carried in the bitstream when necessary. The decoding end determines the same block partitioning information, prediction, transform, quantization, entropy coding, and loop filtering mode information or parameter information as the encoding end by parsing the bitstream and analyzing existing information, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end.

[0068] In some embodiments, the current block may be referred to as the current coding unit (CU), the current prediction unit (PU), or the current transform unit (TU), etc. The prediction block may also be referred to as the predicted image block or the image prediction block, and the reconstructed image block may also be referred to as the reconstruction block or the image reconstruction block.

[0069] In some embodiments, images can be divided into slices or similar components to facilitate parallel processing. Slices within the same image can be processed in parallel, meaning there is no data dependency between different slices within the same image.

[0070] It should be noted that the term "frame" as used in this application can generally be understood as a frame or an image, or in other words, the term "frame" as used in this application can also be replaced with an image or a slice, etc.

[0071] The above describes the basic flow of a video codec under a block-based hybrid coding framework. With the development of technology, some modules or steps of this framework or flow may be optimized. This application is applicable to the basic flow of a video codec under this block-based hybrid coding framework, but is not limited to this framework and flow.

[0072] The foregoing has described in detail the encoding / decoding framework provided in the embodiments of this application. These embodiments can be applied to prediction units within the encoding / decoding framework, such as intra-frame prediction units or inter-frame prediction units. The related technologies involved in the embodiments of this application will be described in detail below.

[0073] Inter-frame prediction

[0074] Inter-frame prediction utilizes temporal correlation to eliminate redundancy. To ensure that stuttering is imperceptible to the human eye, typical video frame rates may include 30, 50, 60, or even 120 frames per second. In such videos, the correlation between adjacent frames within the same scene is high. Inter-frame prediction technology uses this correlation to predict the content to be encoded, referencing the content of already encoded and decoded frames. Inter-frame prediction can significantly improve coding performance.

[0075] Translational prediction, as the most basic inter-frame prediction method, assumes that the content to be predicted is translated between the current image and the reference image. For example, if the content of the current block (such as CU or PU) is translated between the current image and the reference image, then this content can be found in the reference image using a motion vector (MV) and used as the predicted block for the current block. Translational motion accounts for a large proportion in video, such as stationary backgrounds, objects that are translated overall, and camera panning, all of which can be handled using translational prediction.

[0076] One-way prediction can be used in scenarios involving overall object translation or camera panning. However, some content in natural videos is not simply translated; for example, there may be subtle changes during the translation process, including changes in shape and color. In such scenarios, two-way prediction can be used. Two-way prediction finds two reference blocks in a reference image and performs a weighted average of these two reference blocks to obtain a predicted block that is as similar to the current block as possible. For example, in some scenarios, taking a weighted average of reference blocks from both before and after the current frame may result in a more similar prediction block than taking a single reference block. Two-way prediction can improve compression performance compared to one-way prediction.

[0077] The unidirectional and bidirectional predictions in the above translation predictions are both block-based predictions, such as predictions based on CU or PU. That is, the unidirectional and bidirectional predictions in the above translation predictions use a pixel matrix as the unit for prediction, and this pixel matrix can be a matrix block (such as a square or rectangle).

[0078] Sports Information

[0079] Inter-frame prediction uses motion information to represent "motion." Basic motion information includes information from a reference picture and motion vectors (MVs). In some embodiments, for a block (such as the current block) to use bidirectional prediction, it needs to find two reference blocks, requiring two sets of reference picture information and MV information. Each set of reference picture information and MV information can be understood as unidirectional motion information, and combining these two sets forms bidirectional motion information. It should be noted that a block can find not only one or two reference blocks, but also more reference blocks, using multiple reference blocks to generate the prediction block. This technique is generally called multi-hypothesis prediction. Correspondingly, the motion information is not limited to only two reference pictures and motion vectors; if multi-hypothesis prediction is used, there can be multiple reference pictures and motion vectors.

[0080] In practical implementation, unidirectional motion information and bidirectional motion information can use the same data structure. However, in bidirectional motion information, both sets of reference image information and motion vector information are valid, while in unidirectional motion information, one set of reference image information and MV information are invalid. In some embodiments, "valid" can also be described as "used," and "invalid" as "not used." In scenarios involving multiple hypotheses for prediction, the data structure of motion information can be extended accordingly. That is, a single set of motion information can contain multiple motion vectors and their corresponding reference image information. Or, a set of motion vectors can contain multiple motion vectors.

[0081] VVC can support two reference picture lists (RPLs), denoted as RPL0 and RPL1. For example, a B-image can support two RPLs. For the aforementioned bidirectional motion information, VVC can use the reference picture index (denoted as refIdxL0) and motion vector (denoted as mvL0) corresponding to RPL0, and the reference picture index (denoted as refIdxL1) and motion vector (denoted as mvL1) corresponding to RPL1. The reference picture indices for RPL0 and RPL1 can be understood as the information from the reference pictures mentioned above. As a possible implementation, VVC can use two flags to indicate whether to use the motion information corresponding to RPL0 and whether to use the motion information corresponding to RPL1, for example, denoted as predFlagL0 and predFlagL1 respectively. That is, VVC can use the predFlagL0 and predFlagL1 flags to indicate whether the two sets of unidirectional motion information are valid.

[0082] As can be seen from the above description, although VVC does not explicitly mention motion information as a data structure, it uses a reference image index, motion vector, and a flag indicating whether the motion information is valid (or whether the corresponding motion information is used) to represent motion information for each RPL. Alternatively, the standard VVC text does not use "motion information" but instead uses "motion vector." One interpretation is that the motion vector, reference image index, and flag indicating whether the motion information is valid can jointly describe the motion information. Another interpretation is that motion information can be described by motion vectors, with the reference image index and the flag indicating whether the motion information is valid serving as supplementary elements. It should be noted that for ease of description, this application still uses "motion information," but it should be understood that "motion vector" can also be used. In some embodiments, "motion information" can also be called or replaced by "motion parameters." It should also be noted that the "motion information" mentioned in this application can be replaced by information from motion vectors and reference images. For example, bidirectional motion information can also be described as a set of motion vectors and their corresponding reference image information. One set of motion vectors contains two motion vectors.

[0083] For a two-dimensional image, a motion vector can be represented by (x, y), that is, a horizontal component and a vertical component. Since videos are represented in pixels, and pixels are spaced apart, the motion of an object in adjacent images doesn't always correspond to an integer pixel distance. For example, in a video of a distant scene, the distance between two pixels might be 1 meter from a distant object, but that object might move 0.5 meters in two frames. This scenario cannot be well represented using integer pixel motion vectors. Therefore, motion vectors can be implemented at the sub-pixel level, such as 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, or 1 / 16 pixel precision, to represent motion with greater finer detail. In such cases, interpolation methods can be used to obtain the pixel values ​​at sub-pixel positions in the reference image.

[0084] It should be noted that motion information may also need to be transmitted in the bitstream. Generally, the finer the block division, the greater the overhead of motion information.

[0085] Motion vector difference (MVD)

[0086] One possible approach is to represent motion information by directly writing complete motion information into the bitstream. However, researchers have found that directly writing complete motion information into the bitstream results in significant overhead, especially in scenarios where the image is divided into multiple blocks and / or a block requires two or more reference images. Therefore, as another possible approach, motion information can be represented using motion vector predictor (MVP) and motion vector prediction (MVD). For example, motion vectors can be represented by summing MVP and MVD, i.e., MV = MVP + MVD. In this scenario, motion information can be represented by writing MVD into the bitstream. The decoder can parse the bitstream to obtain MVD, predict the MVP of the current block based on the MV of adjacent blocks, and then determine the MV of the current block based on MVP and MVD. It should be understood that the more accurate the MVP, the smaller the MVD, thus reducing the overhead in the bitstream.

[0087] In VVC, the parsing syntax of MVD is described in mvd_coding(), and the following text introduces the parsing syntax of MVD.

[0088] abs_mvd_greater0_flag[compIdx] indicates whether the absolute value of the current component of MVD is greater than 0.

[0089] abs_mvd_greater1_flag[compIdx] indicates whether the absolute value of the current component of MVD is greater than 1. If abs_mvd_greater1_flag[compIdx] does not appear in the bitstream, its value is inferred to be 0.

[0090] The value of abs_mvd_minus2[compIdx] plus 2 represents the absolute value of the current component of MVD.

[0091] If abs_mvd_minus2[compIdx] does not appear in the bitstream, its value is inferred to be -1.

[0092] `mvd_sign_flag[compIdx]` represents the sign of the current component in the MVD. If the value of `mvd_sign_flag[compIdx]` is 0, it means that the current component of the MVD is positive. Otherwise, it means that the current component of the MVD is negative.

[0093] Where compIdx equals 0 to represent the horizontal component and compIdx equals 1 to represent the vertical component.

[0094] The motion vector difference lMvd[compIdx] for compIdx of 0 and 1 is derived as follows: lMvd[compIdx]=abs_mvd_greater0_flag[compIdx]*(abs_mvd_minus2[compIdx]+2)* (1-2*mvd_sign_flag[compIdx]).

[0095] It should be noted that the examples of the above syntax may be modified in subsequent standards, but it is understandable that this method can theoretically represent any MVD within the allowed range.

[0096] Affine

[0097] The previous section introduced simple and commonly used translational motion (see the translation prediction section). However, in the real world, motion is not limited to translation; there are many other forms, such as shrinking, magnification, rotation, perspective motion (i.e., objects closer to the camera appear larger, and objects farther away appear smaller), and many other irregular forms of motion. These motions cannot be predicted between frames using translation prediction. Therefore, in inter-frame prediction scenarios, affine models (or affine motion models) can be used for inter-frame prediction because affines can be used to represent motions that are more complex than translation. The affine model will be introduced below with reference to Figures 3 and 4.

[0098] Figure 3 illustrates the control points in the affine model. The affine model shown in Figure 3(a) includes two control points, located at the top left and top right corners of the current block. The affine model shown in Figure 3(b) includes three control points, located at the top left, top right, and bottom left corners of the current block. As shown in Figure 3, the affine model can calculate the motion vector of each sub-block or pixel in the current block using a linear model based on the motion vector from two control points (i.e., four parameters, since a motion vector includes two parameters: a horizontal component and a vertical component) or three control points (i.e., six parameters). In other words, affines only require a few control points (such as two or three) to derive the motion vector for each sub-block or pixel in the current block. Compared to motion compensation based on the entire current block, affines can achieve more refined predictions; and compared to dividing the current block into smaller CUs, the overhead of affines is much smaller.

[0099] For an affine model with 2 control points (or 4 parameters), the motion vector at the current block position (x, y) is derived according to formula (1):

[0100] For an affine model with 3 control points (or 6 parameters), the motion vector at the current block position (x, y) is derived according to formula (2):

[0101] Where mv0 is used This represents the motion vector of the control point at the top-left corner of the current block. `mv1` is used... This indicates that `mv2` is the motion vector of the control point at the top right corner of the current block. MV represents the motion vector of the control point at the bottom left corner of the current block. W is the width of the current block. H is the height of the current block. The superscripts h and v represent the horizontal and vertical components of MV, respectively.

[0102] To simplify hardware implementation, the affine model used in VVC divides the current block into 4×4 sub-blocks, calculates a motion vector (MV) for each sub-block, and performs motion compensation. Figure 4 is an example of the affine model deriving motion vectors based on sub-blocks. It's understandable that with increased hardware processing power, the affine model can also perform pixel-based processing. The affine model can derive a motion vector for each pixel in the current block and perform motion compensation for that pixel based on that motion vector.

[0103] In VVC, the standard affine mode can be controlled via the parameter `inter_affine_flag`. For example, the decoder can parse the bitstream to determine whether the current block uses affine motion compensation. As one implementation, the decoder can parse the syntax element `inter_affine_flag` to determine whether the current CU uses affine motion compensation to generate predictions. A value of 1 for `inter_affine_flag[x0][y0]` indicates that the current block's predictions are generated using affine-based motion compensation. A value of 0 for `inter_affine_flag[x0][y0]` indicates that the current block does not use affine-based motion compensation to generate predictions. If `inter_affine_flag[x0][y0]` does not appear, its value is inferred to be 0. VVC supports 4-parameter and 6-parameter affine motion compensation. If the value of `inter_affine_flag` is 1, the decoder parses the syntax element `cu_affine_type_flag` to determine whether the current block uses a 4-parameter or 6-parameter affine model. If the current block uses a 4-parameter affine model, the decoder parses the difference between two motion vectors, denoted as `mvd0` and `mvd1`. If the current block uses a 6-parameter affine model, the decoder resolves three motion vector differences, denoted as mvd0, mvd1, and mvd2. Here, motion vector mv0 = mvp0 + mvd0, motion vector mv1 = mvp1 + mvd1 + mvd0, and motion vector mv2 = mvp2 + mvd2 + mvd0. For each motion vector addition, the horizontal and vertical components of that motion vector are actually added together.

[0104] For ease of explanation, the examples above all use a single prediction reference direction. That is, for affine prediction in one direction, where prediction only references RPL0 or RPL1, the above method is applied. For affine prediction in two directions, it can be understood that the decoder performs the above method separately for each prediction reference direction, i.e., generating MVP and MVD for each prediction reference direction, obtaining the MV of each control point, calculating the MV of each sub-block or point, and performing motion compensation.

[0105] Affine can also be used in merge mode. Affine merge uses the motion information of surrounding blocks to infer the motion information of the current block. One characteristic of merge mode is that it doesn't require encoding MVD. Of course, in VVC's merge mode, there's a technique called MMVD, which can be considered a special representation of MVD.

[0106] Merge mode

[0107] It's understandable that each inter-frame coded block requires motion information. To simplify the problem, we can assume that the CU partitioning equals the PU partitioning equals the TU partitioning, meaning each coding unit has a prediction unit and a transform unit of the same size and position. In reality, with more flexible CU partitioning, VVC tends to weaken the PU and TU partitioning compared to HEVC. Differences in any stage of prediction, transform, quantization, or entropy coding can lead to different CU partitioning. For example, if two regions have different motion information, the encoder might partition them into different CUs. Similarly, if two regions have the same or similar motion information but significantly different residual characteristics, the encoder might also partition them into different CUs. The partitioning method is determined by overall compression efficiency, not solely by any single factor. Therefore, the same object or regions with similar or identical motion can be partitioned into different CUs.

[0108] Figure 5 shows an example from HEVC. Figure 5(a) is the original image, showing a metal rod moving in the direction of the arrow, with minimal movement in the background area. Figure 5(b) shows the block division in HEVC. Figure 5(c) removes the boundaries of blocks with the same motion information from Figure (b). As can be seen from Figure 5, many adjacent blocks use the same motion information. In this case, encoding motion information separately for each block would be significantly wasteful.

[0109] As mentioned above, complete motion information for VVC includes the reference image index of RPL0, MV, and a flag indicating whether the motion information is valid; and the reference image index of RPL1, MV, and a flag indicating whether the motion information is valid. The basic principle of the merge mode is that the current block can inherit the motion information of adjacent blocks, including reference image information and motion vector information.

[0110] The merge mode can construct a merge candidate list. If the current block uses the merge mode, an index can be used to indicate which motion information to merge with, thus eliminating the need to encode the complete motion information. When constructing the merge candidate list, motion information from adjacent blocks in the spatial domain, motion information in the temporal domain, motion information from non-adjacent blocks in the spatial domain, motion information from non-adjacent blocks in the temporal domain, motion information based on history, and synthesized motion information can be added.

[0111] In this context, spatially adjacent blocks refer to blocks in the same image that are adjacent to the current block, while spatially non-adjacent blocks refer to blocks in the same image that are not adjacent to the current block. Motion information in the temporal domain and motion information of non-adjacent blocks in the temporal domain refer to motion information at a specified position on the collocated reference image. For ease of understanding, the adjacent and non-adjacent blocks of the current block are described below with reference to Figure 6.

[0112] As shown in Figure 6, the gray blocks in Figure 6 represent the current block. Positions 1, 2, 3, 4, and 5 represent the spatial positions of adjacent blocks used in the merging mode. Other positions marked with boxes (such as positions 7, 8, 9, and 11) represent the spatial positions of non-adjacent blocks used in the merging mode. Position 6 represents the position used for motion information in the temporal domain. If the corresponding position at the lower right corner of the current block is unavailable, the position corresponding to the center of the current block is used. Other positions marked with triangles (such as positions 10, 16, 16, and 18) represent the positions used for motion information of non-adjacent blocks in the temporal domain. In some embodiments, the encoder or decoder can derive temporal motion information based on the motion information at corresponding positions in the co-located reference image. The derivation method for temporal motion information will be described in detail later and will not be elaborated here.

[0113] Referring back to Figure 6, assume the current block's top-left corner is at (x, y), its width is W, and its height is H. Adjacent block 1 contains the coordinates (x+W, y-1). Adjacent block 2 contains the coordinates (x-1, y+H-1). Adjacent block 3 contains the coordinates (x+W, y-1). Adjacent block 4 contains the coordinates (x-1, y+H). Adjacent block 5 contains the coordinates (x-1, y-1). Non-adjacent blocks can also be positioned according to certain rules. In the following example, the positions of non-adjacent blocks can be calculated based on (x, y) and W, H. Assume the distance between each grid in the vertical direction in Figure 6 is H, and the distance between each grid in the horizontal direction is W. Non-adjacent block 7 contains the coordinates (xW-1, y+2*H-1). Non-adjacent block 8 contains the coordinates (x+2*W-1, yH-1). Non-adjacent block 9 is the block containing coordinates (xW-1, yH-1). For simplicity, the positions of other non-adjacent blocks are not calculated.

[0114] It should be noted that the block containing a certain coordinate mentioned above refers to the CU or PU containing that coordinate. In some embodiments, after determining the CU or PU containing a certain coordinate, the intra-prediction mode of that CU or PU can be found.

[0115] The aforementioned historical motion information is independent of location. The codec maintains a first-in, first-out (FIFO) list of motion information. Each time a block is encoded or decoded, the codec updates this list with the motion information of that block, ensuring that the updated list does not contain duplicate motion information already in it. Historical motion information is retrieved from this list.

[0116] Geometric partitioning mode (GPM)

[0117] HEVC supports a maximum CTU of 64×64 and can recursively perform quadtree partitioning. VVC supports a more flexible block partitioning method than HEVC, supporting a maximum CTU of 128×128 and supporting quadtree, ternary tree, and binary tree partitioning. Although block partitioning is becoming more flexible, regardless of whether it's CU, PU, ​​or TU, it can only be divided into rectangular blocks (note that VVC has weakened the partitioning of PU and TU). The boundaries of textures or motion in natural videos are diverse. For example, when encountering a slanted object boundary, if simply using rectangular blocks to approximate the object boundary, it is necessary to divide it into many small blocks, which will significantly increase the overhead. Therefore, compared to traditional partitioning methods, GPM can partition according to a certain angle, thus GPM can better handle textures and boundaries in natural videos.

[0118] GPM uses two prediction blocks of the same size as the current block. Some pixel locations within a GPM prediction block use 100% of the pixel values ​​from the corresponding locations in the first prediction block, and others use 100% from the corresponding locations in the second prediction block. In the boundary region (or transition region), the pixel values ​​from both prediction blocks are used proportionally. The weights in the boundary region also gradually transition. Of course, for scenarios such as screen content encoding, the transition region can be omitted. The specific allocation of these weights is determined by the GPM's "partitioning" mode. The weight of each pixel location is determined according to the GPM's "partitioning" mode. However, in some cases, such as when the block size is very small, certain GPM modes may not guarantee that some pixel locations will use 100% of the pixel values ​​from the first prediction block, and others will use 100% from the corresponding locations in the second prediction block. Alternatively, GPM can be considered to use two prediction blocks of different sizes than the current block, i.e., taking a portion of each block as needed. The portions with a weight of 0 are removed.

[0119] GPM can be understood as a prediction mode or prediction method because it ultimately produces a prediction block. Alternatively, GPM can be described as a "partitioning" mode that simulates the partitioning of the prediction block, similar to implementing PU partitioning, but without actually partitioning it. The first and second prediction blocks used in GPM can be generated by intra-frame prediction, inter-frame unidirectional prediction, or inter-frame bidirectional prediction.

[0120] Currently, GPM includes 64 partitioning methods, as shown in Figure 7. Referring to Figure 7, GPM supports 20 angles, with each angle corresponding to a maximum of 4 distances, resulting in 64 partitioning methods. In Figure 7, the black area represents the weight of the first predicted block at 0%, the white area represents the weight of the first predicted block at 100%, and the gray area, depending on its shade, represents a weight value greater than 0% and less than 100% for the first predicted block. The weight of the second predicted block is 100% minus the weight of the first predicted block.

[0121] The current block can be divided into two sub-partitions by locating a geometric line based on the selected angle and distance parameters. Each sub-partition can be individually compensated for unidirectional motion to obtain a unidirectional prediction value. Finally, the unidirectional prediction values ​​of the two sub-partitions are weighted and fused using a weight matrix to obtain the final GPM prediction value.

[0122] In GPM mode, the predicted value originates from two directions. In practice, the final predicted value can be obtained by constructing and combining two prediction blocks. These two prediction blocks are called part0 and part1, respectively. The two GPM partitions can obtain predicted values ​​using tools such as inter-frame prediction, intra-frame prediction, and intra-block copy (IBC). That is, GPM can combine two inter-frame prediction blocks, or one inter-frame prediction block and one intra-frame prediction block, as shown in Figure 8. When a partition uses inter-frame prediction to obtain the predicted value, motion compensation prediction can be performed using motion vectors to obtain the partition's predicted value. When a partition uses intra-frame prediction, the partition's predicted value can be derived using adjacent reconstructed pixels. When a partition uses IBC mode, the partition's predicted value can be obtained within the search area using IBC and block vectors. The weighted prediction of the two predicted values ​​can be calculated using a weight matrix, block size, and the coordinates of the current block's sample point relative to the top-left corner. For specific implementation methods, see the GPM decoding section in VVC, as described below.

[0123] GPM decoding in VVC

[0124] In VVC, the syntax for GPM is shown in the table below, which is derived from the merge data syntax in Section 7.3.11.7. In merge mode, if `regular_merge_flag` is not 1, the current block may use either combined inter and intra prediction (CIIP) or GPM. If the current block does not use CIIP, then it uses GPM; see the parentheses under `if(!ciip_flag[x0][y0])` in the table below.

[0125] As described in the syntax above, GPM transmits three types of information in the bitstream: `merge_gpm_partition_idx`, `merge_gpm_idx0`, and `merge_gpm_idx1`. `x0` and `y0` are used to determine the coordinates (x0, y0) of the brightness sample at the top left corner of the current block relative to the brightness sample at the top left corner of the image. `merge_gpm_partition_idx` is used to determine the partitioning method (or partitioning shape) of GPM. `merge_gpm_partition_idx` can be used to derive the weight matrix and perform partitioning based on the weight matrix; therefore, `merge_gpm_partition_idx` is sometimes also called the weight matrix derivation mode, the index of the weight matrix derivation mode, or the weight derivation mode index. `merge_gpm_idx0` is the first merge candidate index, used to determine the first motion information (or the first merge candidate) based on `mergeCandList`. `merge_gpm_idx1` is the second merge candidate index, used to determine the second motion information (or the second merge candidate) based on `mergeCandList`. It should be noted that `merge_gpm_idx1` only needs to be decoded when `MaxNumGpmMergeCand>2` (i.e., the candidate list length is greater than 2).

[0126] The following describes the GPM decoding process, specifically a portion of section 8.5.7 (Decoding process for geometric partitioning mode inter blocks) of the VVC standard text.

[0127] The inputs to the GPM decoding process include:

[0128] – The brightness position of the top-left corner of the current block relative to the top-left corner of the image (xCb, yCb).

[0129] – The width of the current block luminance component, cbWidth

[0130] – The height of the current block luminance component, cbHeight

[0131] – Luminance motion vectors mvA and mvB with 1 / 16 pixel precision

[0132] – Chromaticity motion vectors mvCA and mvCB

[0133] –Refer to frame indices refIdxA and refIdxB,

[0134] – Prediction list flags predListFlagA and predListFlagB.

[0135] VVC uses motion vectors, reference frame indices, and prediction list flags to represent motion information. VVC supports two reference frame lists, each of which may contain multiple reference frames. Unidirectional prediction uses only one reference block from one reference frame in one of the reference frame lists as a reference, while bidirectional prediction uses one reference block from each of the two reference frame lists. GPM in VVC uses two unidirectional predictions. In the above examples of mvA and mvB, mvCA and mvCB, refIdxA and refIdxB, predListFlagA and predListFlagB, A can be understood as the first prediction mode, and B as the second prediction mode. X can represent either A or B, predListFlagX indicates whether X uses the first or second reference frame list, refIdxX indicates the reference frame index in the reference frame list used by X, mvX indicates the luma motion vector used by X, and mvCX indicates the chroma motion vector used by X.

[0136] The output of the above process includes:

[0137] The brightness prediction sample matrix predSamplesL is calculated as –(cbWidth)*(cbHeight).

[0138] The predicted sample matrix of the Cb chromaticity component is given by –(cbWidth / SubWidthC)*(cbHeight / SubHeightC).

[0139] The predicted sample matrix of the Cr chromaticity component is –(cbWidth / SubWidthC)*(cbHeight / SubHeightC).

[0140] The following example illustrates the GPM decoding process using the luminance component as an example. The processing method for the chrominance component is similar to that for the luminance component, and will not be described again.

[0141] Let predSamplesLA and predSamplesLB be two prediction sample matrices calculated for a (cbWidth)*(cbHeight) block according to two prediction modes. predSamplesL can be derived as follows:

[0142] PredSamplesLA and predSamplesLB are determined based on the luma motion vectors mvA and mvB, the chroma motion vectors mvCA and mvCB, the reference frame indices refIdxA and refIdxB, and the prediction list flags predListFlagA and predListFlagB, respectively. In other words, predictions are performed based on the motion information of the two prediction modes. Detailed prediction processes can be found in relevant technical documents and will not be elaborated here. Since GPM in VVC is a merged mode, both prediction modes of GPM in VVC can be considered merged modes.

[0143] Based on merge_gpm_partition_idx[xCb][yCb], use Table 1 to determine the “partition” angle index angleIdx and distance index distanceIdx of GPM.

[0144] Table 1: Correspondence between angular index, distance index and merge_gpm_partition_idx

[0145] In the GPM weighted prediction calculation process, the weight of each sample point in the current block is calculated based on the distance of the sample point from the dividing line. Since all three components (such as Y, Cb, and Cr) of the current block can use GPM, the VVC standard text encapsulates the process of generating the GPM prediction sample matrix for one component into a sub-process, namely the weighted sample prediction process for geometric partitioning mode described in Section 8.5.7.2. All three components will call this process, but with different parameters. The following only illustrates the weighted prediction calculation process for the luminance component. The weighted prediction process of the prediction matrix predSamplesL[xL][yL] (where xL = 0..cbWidth–1, yL = 0..cbHeight–1) of the current luminance block is as follows.

[0146] The inputs to this process include:

[0147] - The width nCbW and height nCbH of the current block.

[0148] – Two (nCbW)*(nCbH) predicted sample matrices, predSamplesLA and predSamplesLB.

[0149] – The “partition” angle index angleIdx of GPM,

[0150] – Distance index of GPM, distanceIdx

[0151] – Component index cIdx (since this example only uses luminance, cIdx is 0, indicating the luminance component).

[0152] The output of this process is a predicted sample matrix pbSamples of (nCbW)*(nCbH).

[0153] The variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor can be derived as follows: nW = (cIdx == 0) ? nCbW : nCbW * SubWidthC; nH = (cIdx == 0) ? nCbH : nCbH * SubHeightC;

[0154] shift1 = Max(5, 17 - BitDepth), where BitDepth is the bit depth of the encoding and decoding;

[0155] offset1 = 1 << (shift1 - 1);

[0156] displacementX = angleIdx;

[0157] displacementY=(angleIdx+8)%32;

[0158] partFlip=(angleIdx>=13&&angleIdx<=27)? 0:1;

[0159] shiftHor=(angleIdx%16==8||(angleIdx%16!=0&&nH>=nW))? 0:1.

[0160] The purpose of shiftHor is to determine the displacement direction of different dividing lines at the same angle. When shiftHor is 0, the dividing line will be offset on the Y-axis; when shiftHor is 1, the dividing line will be offset on the X-axis.

[0161] Based on the current block's size and partitioning information, the offset values ​​offsetX and offsetY of the current block can be calculated. The variables offsetX and offsetY can be derived as follows:

[0162] – If the value of shiftHor is 0: offsetX = (-nW) >> 1; offsetY = ((-nH) >> 1) + (angleIdx < 16? (distanceIdx * nH) >> 3: - ((distanceIdx * nH) >> 3));

[0163] - Otherwise (i.e., shiftHor is 1): offsetX = ((-nW)>>1) + (angleIdx<16?(distanceIdx*nW)>>3:-((distanceIdx*nW)>>3); offsetY = (-nH)>>1.

[0164] Here, distanceIdx is the distance index. There are 4 distance indices under one dividing angle (represented by angleIdx). There are a total of 7 "divisions" between the dividing angle and its complementary dividing angle. In the horizontal or vertical direction, nW or nH can be divided into 8 equal intervals. Therefore, at the same angle, the unit horizontal offset or unit vertical offset between adjacent dividing lines is nW>>3 or nH>>3.

[0165] The predicted sample matrix pbSamples[x][y] (where x = 0..nCbW–1, y = 0..nCbH–1) can be derived as follows:

[0166] – Variables xL and yL are derived as follows: xL = (cIdx == 0) ? x: x * SubWidthC; yL = (cIdx == 0) ? y: y * SubHeightC.

[0167] – The variable wValue represents the weight of the predicted sample at the current position. wValue is the weight of the predicted value predSamplesLA[x][y] of point (x, y) in the first prediction mode, while (8-wValue) is the weight of the predicted value predSamplesLB[x][y] of point (x, y) in the second prediction mode.

[0168] wValue can be derived as follows, and the distance matrix disLut involved in this derivation can be determined based on Table 2.

[0169] Table 2 Definition of the distance matrix disLut weightIdx=(((xL+offsetX)<<1)+1)*disLut[displacementX]+ (((yL+offsetY)<<1)+1)*disLut[displacementY]; weightIdxL=partFlip? 32+weightIdx:32–weightIdx; wValue=Clip3(0,8,(weightIdxL+4)>>3);

[0170] – The values ​​of the predicted samples can be derived as follows: pbSamples[x][y] = Clip3(0, (1<<BitDepth)-1,(predSamplesLA[x][y]*wValue+predSamplesLB[ x][y]*(8-wValue)+offset1)> >shift1);

[0171] Here, offset1 can be used for rounding, and shift1 can be used to restore the weighted average prediction value to the same bit depth as the input video. offset1 and shift1 can be calculated using the following formulas: shift1 = Max(5, 17 - BitDepth) offset1 = 1 << (shift1 - 1)

[0172] It's important to note that the standard VVC documentation derives a weight value wValue for each sample location and then calculates a GPM prediction value pbSamples[x][y]. Therefore, in this weight calculation method, the weight wValue doesn't necessarily need to be written as a matrix. However, it's understandable that storing the wValue for each location in a matrix would create a weight matrix. Calculating the weight for each sample point separately and then weighting them together to obtain the GPM prediction value, or calculating all weights and then uniformly weighting them based on the weight matrix to obtain the GPM prediction sample matrix, both work on the same principle. The use of the weight matrix terminology in many descriptions in this application is for ease of understanding; it could also be described using the weight of each location.

[0173] In some embodiments, the GPM process can be described by steps 1-3.

[0174] In step 1, decode the bitstream and determine whether the current block uses GPM.

[0175] In step 2, if the current block uses GPM, determine the weight matrix derivation mode (“partition” mode), first motion information and second motion information.

[0176] In step 3, the first prediction block is determined based on the first motion information, the second prediction block is determined based on the second motion information, the weight matrix is ​​determined based on the weight matrix derivation mode, and the prediction block of the current block is determined based on the first prediction block, the second prediction block, and the weight matrix.

[0177] In some embodiments, the “weight matrix derivation mode” in step 2 can also be called the “weight derivation mode”. Correspondingly, step 3 can also be replaced by: determining the first predicted value based on the first motion information, determining the second predicted value based on the second motion information, determining the weight based on the weight derivation mode, and determining the predicted block of the current block based on the first predicted value, the second predicted value and the weight.

[0178] Template matching

[0179] Template matching can be used for inter-frame prediction. Template matching utilizes the correlation between adjacent pixels to use some regions surrounding the current block as templates. During the encoding and decoding of the current block, its left and top sides have already been encoded and decoded according to the encoding order. Therefore, theoretically, the left and top regions of the current block can be used as templates. It should be noted that currently, hardware decoder implementations cannot guarantee that the left and top sides of the current block have already been decoded when decoding begins. It should also be noted that inter-frame blocks mentioned here, such as those in HEVC, do not require surrounding reconstructed pixels when generating prediction blocks, thus the prediction process for inter-frame blocks can be performed in parallel. However, intra-frame coded blocks always require reconstructed pixels on the left and top sides as reference pixels. Theoretically, the left and top sides are available, meaning that hardware design adjustments are feasible. Relatively speaking, the right and bottom sides are not available under the current encoding order of standards such as VVC.

[0180] As shown in Figure 9, the rectangular regions on the left and top sides of the current block can be set as templates. The height of the template portion on the left is generally the same as the height of the current block, and the width of the template portion on the top side is generally the same as the width of the current block, although they can differ. In this scenario, the optimal matching position of the template can be found in the reference image to determine the motion information (or motion vector) of the current block. One implementation method for finding the optimal matching position of the template in the reference image is as follows: Starting from a starting position in a reference image, a search is performed within a certain range around it; each time the search moves (or moves) to a position, the matching degree between the template corresponding to that position and the templates surrounding the current block is calculated; the cost is calculated using the predicted block of the template corresponding to that position and the reconstructed block of the templates surrounding the current block.

[0181] In some embodiments, the search rules can be preset, such as the search range and search step size.

[0182] In some embodiments, the degree of matching described above can be measured by some distortion cost, such as the sum of absolute difference (SAD), the sum of absolute transformed difference (SATD), the mean-square error (MSE), etc.

[0183] In some embodiments, the transform used by SATD is the Hadamard transform.

[0184] It should be noted that the smaller the values ​​of SAD, SATD, MSE, etc., the higher the degree of matching.

[0185] In some embodiments, in addition to searching for integer pixel positions, a search for sub-pixel positions can also be performed, and the motion information of the current block is determined based on the position with the highest matching degree found. Utilizing the correlation between adjacent pixels, suitable motion information for the template may also be suitable motion information for the current block.

[0186] It's important to note that template matching may not be applicable to all blocks. Therefore, methods can be used to determine whether to use template matching for the current block, such as using a control switch within the current block to indicate whether template matching is used. This template matching method is called decoder-side motion vector derivation (DMVD). Both the encoder and decoder can use templates to search and derive motion information or find better motion information based on existing motion information. Furthermore, when determining motion information using template matching, the specific motion vectors or motion vector differences do not need to be transmitted in the bitstream; instead, both the encoder and decoder perform the same search rules to ensure consistency between encoding and decoding. Template matching can improve compression performance, but it requires a "search" at the decoding end, introducing some complexity.

[0187] Affine model-based motion compensation is typically used at the edges of moving objects, or at the edges of two different moving parts of the same object. As described above, an affine model derives the motion information of all sample points or sub-blocks within the current block from the motion information of several control points of that block. If the motion information of the control points is not entirely identical, theoretically, without considering trade-offs for accuracy, the motion information derived from the control point motion information for each sub-block or sample point may be different. However, in related techniques, the affine model is applied to an entire block; that is, all sub-blocks or sample points within a block follow the same affine model. Therefore, related techniques using affine models for prediction are actually based on the following assumption: all content within a block belongs to the same object and follows the same motion model. However, in some cases, there may be multiple moving objects within a block, and these objects may move in different ways, or different parts of an object within a block may move in different ways. For example, a block may contain two objects, one moving according to an affine model and the other moving according to a translation model. In such scenarios, the predictive performance of prediction patterns based on affine models provided by related technologies is not high.

[0188] Of course, one possible approach to the above scenario is to further divide the current block. That is, for regions rich in motion information, the current block can be divided into smaller blocks. However, so far, block division is still limited to rectangular blocks. While flexible block division can approximate the edges of moving objects, the edges of moving objects are highly variable, and rectangular blocks cannot perfectly fit all the edges of various moving objects. On the other hand, the more detailed the block division, the more division information needs to be transmitted in the bitstream, which is detrimental to improving compression efficiency.

[0189] To address the aforementioned issues, this application provides an encoding method comprising: determining a first prediction block of the current block based on affine parameters; determining a second prediction block of the current block; determining a third prediction block of the current block using a geometric partitioning pattern based on the first prediction block and the second prediction block; and determining a residual block of the current block based on the third prediction block.

[0190] This application also provides a decoding method, including: determining a first prediction block of the current block based on affine parameters; determining a second prediction block of the current block; determining a third prediction block of the current block using a geometric partitioning pattern based on the first prediction block and the second prediction block; and determining a reconstructed block of the current block based on the third prediction block.

[0191] As described above, this application combines a geometric partitioning model with an affine model. The geometric partitioning model does not require actual partitioning of blocks (it performs simulated partitioning) to effectively handle the boundaries of two different moving objects or regions. Therefore, combining the geometric partitioning model with an affine model allows for improved prediction performance of the boundaries of two different moving objects or regions through the simulated partitioning of the geometric partitioning model. Furthermore, the affine model can represent richer and more varied object motions, enabling the solution provided in this application to effectively handle scenarios with diverse motion types and moving objects, thus improving the prediction quality in such scenarios, reducing residuals, and minimizing distortion at the edges of moving objects.

[0192] The decoding method provided in the embodiments of this application will be described in detail below with reference to Figure 10.

[0193] Figure 10 is a schematic flowchart of the decoding method provided in an embodiment of this application. The method shown in Figure 10 can be applied to a decoder.

[0194] Referring to Figure 10, in step S1010, the first prediction block of the current block is determined based on the affine parameters. The current block mentioned here can be an encoding unit, decoding unit, prediction unit, or transform unit. The current block can include one or more samples (or sample points), and the samples in the current block can be luminance samples or chrominance samples.

[0195] The affine parameters mentioned in step S1010 refer to the parameters corresponding to the affine model. The affine model can be used to perform motion compensation on the current block, thereby determining the first predicted block of the current block. Therefore, in some implementations, step S1010 can also be expressed as: determining the first predicted block of the current block according to the affine model, or determining the first predicted block of the current block obtained by performing motion compensation using the affine model.

[0196] Affine parameters can include motion information of the control points of the current block. The control points of the current block can include one or more of the following: the control point at the top left corner, the control point at the bottom left corner, the control point at the top right corner, and the control point at the bottom right corner. Taking an example where the control points of the current block include the control points at the top left and top right corners, the affine parameters include four parameters (the corresponding affine model can be called a 4-parameter affine model), namely the x-component, y-component, x-component, and y-component of the control point at the top left and top right corners of the current block. Taking a block whose control points include the top-left, top-right, and bottom-left corners as an example, the affine parameters can include six parameters (the corresponding affine model can be called a 6-parameter affine model), namely the x-component of the top-left control point, the y-component of the top-right control point, the y-component of the bottom-left control point, and the y-component of the bottom-left control point. Taking a block whose control points include the bottom-left and bottom-right corners as an example, the affine parameters include four parameters, namely the x-component of the bottom-left control point, the y-component of the bottom-right control point, and the y-component of the bottom-right control point. Taking a block whose control points include the top-right and bottom-right corners as an example, the affine parameters include four parameters: the x-component of the top-right corner control point, the y-component of the top-right corner control point, the x-component of the bottom-right corner control point, and the y-component of the bottom-right corner control point. Taking a block whose control points include the bottom-left, top-right, and bottom-right corners as an example, the affine parameters include six parameters: the x-component of the bottom-left corner control point, the y-component of the bottom-left corner control point, the x-component of the top-right corner control point, the y-component of the top-right corner control point, the x-component of the bottom-right corner control point, and the y-component of the bottom-right corner control point.

[0197] This application does not specifically limit the method of "determining the first prediction block of the current block based on affine parameters". For example, the motion vector of each sample point in the current block can be determined based on the affine parameters, and then the first prediction block can be determined based on the motion vector of each sample point. As another example, the motion vector of each sub-block (the size of the sub-block can be, for example, 4×4) in the current block can be determined based on the affine parameters, and then the first prediction block can be determined based on the motion vector of each sub-block. The method of determining the motion vector of each sample point or sub-block can be referred to formula (1) or formula (2) above, and will not be described in detail here.

[0198] Referring again to Figure 10, in step S1020, the second prediction block of the current block is determined. This application embodiment does not specifically limit the method of determining the second prediction block. The second prediction block can be determined based on inter-frame prediction or intra-frame prediction. If the second prediction block is determined based on inter-frame prediction, it can be determined based on an affine model (in this case, the affine parameters corresponding to the second prediction block can be different from those corresponding to the first prediction block); or, it can be determined based on a translation model. For example, the second prediction block can be determined based on the motion information of the decoded blocks surrounding the current block. Alternatively, the motion information of the current block can be determined using template matching, and then the second prediction block can be determined (see the description in the "Template Matching" section above). Alternatively, the motion information of the decoded blocks surrounding the current block can be determined first, and then the motion information can be adjusted using template matching technology to obtain adjusted motion information (e.g., comparing the adjusted motion information with the motion information obtained based on template matching technology to find motion information that better matches the current block), and then the second prediction block can be determined based on the adjusted motion information. The determination method of the second prediction block will be illustrated in detail later with reference to the embodiments, and will not be described in detail here.

[0199] Referring again to Figure 10, in step S1030, based on the first and second prediction blocks, a geometric partitioning pattern (or a prediction pattern based on the geometric partitioning pattern) is used to determine the third prediction block for the current block. The geometric partitioning pattern simulates partitioning, essentially combining the contents of the first and second prediction blocks in a certain way, thereby improving prediction performance.

[0200] In some implementations, when using a geometric partitioning pattern for prediction, the third prediction block of the current block can be determined by weighted summation of the first and second prediction blocks based on weight parameters. For example, the bitstream can be decoded first to determine the first index. This first index can indicate the partitioning pattern (or weight-derived pattern) of the current block. The first index can be used to determine the weight parameters. Exemplarily, the first index can be used to determine a first distance parameter (e.g., distanceIdx) and / or a first angle parameter (e.g., angleIdx), which can be used to determine the weight parameters (e.g., wValue). As one possible implementation, the first distance parameter and / or the first angle parameter can be determined based on the first index, thereby determining the weight parameters. Alternatively, the first distance parameter and / or the first angle parameter can be determined in a different way, thereby determining the weight parameters. For example, the geometric partitioning pattern can be simplified to a certain extent (e.g., reducing the number of partitioning patterns (or weight-derived patterns) indicated by the first index), thereby reducing the overhead of the bitstream. Alternatively, different distance parameters and / or angle parameters can be defined to form a new geometric partitioning pattern. The implementation method will be described in detail later with reference to specific examples, and will not be described in detail here.

[0201] After determining the weight parameters, in some implementations, the first and second prediction blocks can be weighted and summed based on the weight parameters. For example, if the first prediction block is predSamplesLA, the second prediction block is predSamplesLB, the weight parameter is wValue, and the predicted value in the first prediction block is pbSamples, then the sample points in the first prediction block can be determined based on the following formula: pbSamples[x][y]=Clip3(0,(1<<BitDepth)-1,(predSamplesLA[x][y]*wValue+ predSamplesLB[x][y]*(8-wValue)+offset1)> >shift1);

[0202] The meaning of each parameter in the above formula can be found in the previous description, and will not be repeated here.

[0203] When determining the first prediction block, each sample point in the first prediction block can be determined using the above formula. In other words, after determining each sample point in the first prediction block using the above formula, the first prediction block is also determined accordingly.

[0204] Referring again to Figure 10, in step S1040, the reconstructed block of the current block is determined based on the third prediction block of the current block. For example, the bitstream can be decoded to determine the residual block of the current block; the reconstructed block of the current block is then determined based on the residual block and the third prediction block of the current block. Exemplarily, the reconstructed block of the current block can be determined by summing the residual block and the third prediction block of the current block.

[0205] As shown in Figure 10, this embodiment combines a geometric partitioning pattern with an affine model. The geometric partitioning pattern does not require actual block division to effectively handle the boundaries of two different moving objects or regions. Therefore, combining the geometric partitioning pattern with an affine model can, on the one hand, improve the prediction performance of the boundaries of two different moving objects or regions through the simulated partitioning of the geometric partitioning pattern; on the other hand, the affine model can represent richer and more varied object motions. This allows the solution provided in this embodiment to effectively handle scenarios with diverse motion types and moving objects, improving the prediction quality in such scenarios, reducing residuals, and minimizing distortion at the edges of moving objects.

[0206] As mentioned earlier, the first prediction block of the current block is obtained by motion compensation based on an affine model. The "affine" mentioned here can be an affine without motion vector difference (such as Affine Merge). The affine parameters in an affine without motion vector difference can be derived based on the motion information of the decoded block, without needing to carry the motion vector difference in the bitstream, thus reducing the bitstream overhead.

[0207] However, in some application scenarios, the motion information of the current block may not have appeared in adjacent blocks (e.g., a new moving object appears in the current block), or the motion information of the current block may differ significantly from the motion information of the decoded blocks. In such cases, if the motion information of the current block is derived solely based on the motion information of the decoded blocks, the prediction accuracy of the first prediction block will be reduced. Therefore, in some implementations, the "affine" mentioned in the embodiments of this application can refer to an affine with motion vector difference (such as Affine AMVP). Compared to an affine without motion vector difference, the introduction of motion vector difference can better describe the motion of new objects appearing in the current block, thereby improving the prediction performance of the first prediction block.

[0208] Taking an affine transformation with motion vector differences as an example, in practical implementation, the bitstream can be decoded first to determine one or more motion vector differences. Then, one or more motion information (referring to the motion information of the control points of the current block, which may include one or more of the control points at the top left, bottom left, top right, and bottom right corners of the current block) can be determined based on these one or more motion vector differences. For example, the motion information of the control points of the current block can be determined based on the one or more motion vector differences obtained from the decoded bitstream, and one or more motion vector predictors (MVPs). After obtaining the motion information of the control points of the current block, the affine parameters mentioned in step S1010 are obtained, and motion compensation can be performed based on these affine parameters to determine the first prediction block of the current block.

[0209] As one possible implementation, if the first prediction block is determined based on an affine model with motion vector differences, the number of one or more motion vector differences can be determined by a first parameter (e.g., represented by MotionModelIdc). The first parameter can be used to indicate the motion model corresponding to the first prediction block. In specific implementation, the bitstream can be decoded first to determine the first parameter. If the value of the first parameter is 0 (a value of 0 can represent a translation model), then the number of one or more motion vector differences is 1 (i.e., 1 motion vector difference needs to be parsed from the bitstream); if the value of the first parameter is 1 (a value of 1 can represent a 4-parameter affine model), then the number of one or more motion vector differences is 2 (i.e., 2 motion vector differences need to be parsed from the bitstream); if the value of the first parameter is 2 (a value of 1 can represent a 6-parameter affine model), then the number of one or more motion vector differences is 3 (i.e., 3 motion vector differences need to be parsed from the bitstream). In other words, if the first parameter indicates that the motion model corresponding to the first prediction block is a translation model, then the number of one or more motion vector differences is 1; if the first parameter indicates that the motion model corresponding to the first prediction block is a 4-parameter affine model, then the number of one or more motion vector differences is 2; if the first parameter indicates that the motion model corresponding to the first prediction block is a 6-parameter affine model, then the number of one or more motion vector differences is 3. For a syntactic description of how the number of one or more motion vector differences is determined by the first parameter, please refer to the introduction in the "MVD" section above, which will not be elaborated here.

[0210] Taking a single motion vector difference as an example, this single motion vector difference can include an x-component and a y-component. Taking a double motion vector difference as an example, each of the two motion vector differences can include an x-component and a y-component. Taking a triple motion vector difference as an example, each of the three motion vector differences can include an x-component and a y-component.

[0211] If the first prediction block is determined based on an affine model with motion vector difference, the geometric partitioning mode in step S1030 will differ significantly from the geometric partitioning mode provided by related technologies. In other words, in this implementation, the geometric partitioning mode mentioned in step S1030 is not equivalent to the geometric partitioning mode provided by related technologies. The geometric partitioning mode provided by related technologies is a sub-mode of the Merge mode, used to generate motion information for both the first and second prediction blocks from the Merge candidate list. The basic logic of the Merge mode is that the motion of the current block and one of its neighboring blocks (including neighboring blocks in the spatial domain and / or temporal domain) is the same, or extremely similar, thus the motion information of the neighboring blocks can be directly used as the motion information of the current block. However, the first prediction block in this implementation is determined based on an affine model with motion vector difference, which better addresses scenarios where the motion information in the current block has not appeared in neighboring blocks, or where the motion of the current block differs significantly from the motion of the decoded neighboring blocks. This is precisely why motion vector difference needs to be carried in the bitstream.

[0212] In some implementations, step S1010 in Figure 10 can be executed under certain conditions. For example, before executing step S1010, the bitstream can be decoded to determine whether the (current block) uses an affine model for motion compensation (or, whether an affine model is used to generate a prediction block). Further, in some implementations, whether the current block uses an affine model for motion compensation can be indicated by a syntax element carried in the bitstream. This syntax element can be, for example, a first identifier (e.g., represented by `inter_affine_flag`). Therefore, before executing step S1010, the bitstream can be decoded to determine the first identifier. The first identifier can be used to indicate whether the (current block) uses an affine model for motion compensation. Alternatively, the first identifier can be used to indicate whether the (current block) uses an affine model to generate a prediction block. The first identifier can include a first value and a second value. The first value can be 1, and the second value can be 0. Or, the first value can be 0, and the second value can be 1. The first value can be used to indicate that the current block uses an affine model for motion compensation. The second value can be used to indicate that the current block does not use an affine model for motion compensation. If the first identification information indicates that motion compensation is performed using an affine model, then step S1010 can be executed, that is, determining the first predicted block of the current block based on the affine parameters. Further, in some implementations, if the first identification information indicates that motion compensation is not performed using an affine model, then other methods can be used to determine the predicted block of the current block (e.g., motion compensation based on a translation model), instead of executing the method shown in Figure 10.

[0213] In some implementations, step S1020 in Figure 10 can be executed under certain conditions. For example, before executing step S1020, the bitstream can be decoded to determine whether the (current block) uses a geometric partitioning mode (or, whether a prediction block is generated using a prediction mode based on the geometric partitioning mode). Further, in some implementations, whether the current block uses a geometric partitioning mode can be indicated by a syntax element carried in the bitstream. This syntax element can be, for example, second identification information (e.g., represented by inter_gpm_flag). Therefore, before executing step S1020, the bitstream can be decoded to determine the second identification information, which can be used to indicate whether the (current block) uses a geometric partitioning mode. Alternatively, the second identification information can be used to indicate whether the (current block) uses a prediction block generated using a prediction mode based on the geometric partitioning mode. The second identification information can include a third value and a fourth value. The third value can be 1, and the fourth value can be 0. Or, the third value can be 0, and the fourth value can be 1. If the second identification information indicates that the current block uses a geometric partitioning mode, then step S1020 can be executed, i.e., determining the second prediction block of the current block. If the second identification information indicates that the current block does not use a geometric partitioning mode, the reconstructed block of the current block can be determined directly based on the first predicted block of the current block (e.g., by adding the first predicted block of the current block to the residual block of the current block to obtain the reconstructed block of the current block).

[0214] As mentioned in step S1030 above, when using the geometric partitioning mode, the first distance parameter and / or the first angle parameter can be determined based on the first index obtained from the bitstream, and then the weight parameter can be determined based on the first distance parameter and / or the first angle parameter. After obtaining the weight parameter, the first predicted block and the second predicted block of the current block can be weighted and summed based on the weight parameter. The first index indicates the partitioning mode (or weight derivation mode) of the current block. The first index can follow the implementation method provided by related technologies (see the introduction of Figure 7 above), that is, the value of the first index can be one of 64 preset values ​​(0 to 63), thereby indicating 64 partitioning modes (or weight derivation modes). Alternatively, the value of the first index can also be different from the implementation method provided by related technologies. The partitioning mode (or weight derivation mode) proposed in the embodiments of this application is illustrated in detail below.

[0215] This application proposes an improved geometric partitioning mode. Compared to related technologies, the geometric partitioning mode in this application supports a smaller number of partitioning modes (or weight-derived modes). For example, the number of partitioning methods (or weight-derived modes) used (or supported) by the geometric partitioning mode in this application is less than the number of partitioning methods used by the geometric partitioning mode based on the Merge mode. In other words, the number of candidate values ​​for the first index can be less than the number of partitioning methods used by the geometric partitioning mode based on the Merge mode. Taking the number of partitioning methods used by the geometric partitioning mode based on the Merge mode as an example, the number of partitioning methods used by the geometric partitioning mode in this application is less than 64. This is done because when the current block is predicted based on an affine model, it may be necessary to transmit one or more motion vector differences in the bitstream, and the transmission of one or more motion vector differences requires a large amount of bitstream data. Therefore, by reducing the number of partitioning modes (or weight-derived modes), the prediction information of the current block is not too much, thereby reducing the overhead required for transmitting prediction information in the bitstream. It should be noted that the improved geometric partitioning mode proposed in this application can be applied not only to affine scenarios but also to other scenarios. For example, in scenarios where geometric partitioning patterns are applied in VVC, these geometric partitioning patterns can all be the improved geometric partitioning patterns proposed in the embodiments of this application. The improved geometric partitioning patterns are described below.

[0216] In some implementations, the number of partitioning modes (or weight-derived modes) supported by the geometric partitioning mode in this embodiment can be less than or equal to 32. Correspondingly, the value of the first index (used to indicate the partitioning mode or weight-derived mode of the current block) can be selected from a first set of values, and the number of values ​​in this first set is less than or equal to 32; that is, the number of candidate values ​​for the first index is less than or equal to 32. This reduces the number of modes supported by the geometric partitioning mode by more than half, thereby reducing the overhead required to carry the first index in the bitstream.

[0217] In some implementations, the number of partitioning modes (or weight-derived modes) supported by the geometric partitioning mode in this embodiment can be less than or equal to 16. Correspondingly, the value of the first index (used to indicate the partitioning mode or weight-derived mode of the current block) can be selected from a first set of values, and the number of values ​​in this first set is less than or equal to 16; that is, the number of candidate values ​​for the first index is less than or equal to 16. In this way, the number of modes supported by the geometric partitioning mode can be reduced to less than a quarter of the original number, thereby reducing the overhead required to carry the first index in the bitstream.

[0218] For example, referring to Figure 11A, the geometric partitioning mode in this embodiment supports 16 partitioning modes (or weight-derived modes). These 16 partitioning modes (or weight-derived modes) can correspond to the partitioning modes (or weight-derived modes) with index values ​​of 0-1 and 18-31 in related technologies. The partitioning modes (or weight-derived modes) with index values ​​of 0-1 and 18-31 can basically represent the main partitioning modes among all 64 partitioning modes. It should be understood that if 16 partitioning modes (or weight-derived modes) are used, the index values ​​of these 16 partitioning modes (or weight-derived modes) can be numbered consecutively from 0 to 15. The labels in Figure 11A are only to show the correspondence between the partitioning modes (or weight-derived modes) supported by this embodiment and the partitioning modes (or weight-derived modes) in related technologies.

[0219] If 16 partitioning modes are used, the value of the first index can be one of the 16 preset candidate values. In this scenario, the first distance parameter and / or the first angle parameter can be determined based on the first index using Table 3.

[0220] Table 3

[0221] Understandably, in addition to reducing the number of partitioning modes (or weight derivation modes) supported by the geometric partitioning modes in related technologies, it is also possible to set partitioning modes (or weight derivation modes) that are different from the geometric partitioning modes in related technologies, such as setting different angle parameters (angleIdx), distance parameters (distanceIdx), and / or distance matrices (disLut).

[0222] As mentioned earlier, the first and second prediction blocks of the current block can be weighted and summed using a geometric partitioning pattern to determine the third prediction block. The sample points in the first prediction block can include first-class and second-class sample points. First-class sample points refer to sample points with a corresponding weight parameter value of 0, and second-class sample points refer to sample points with a corresponding weight parameter value of non-zero. Similarly, the sample points in the second prediction block can also include first-class and second-class sample points. In some implementations, assuming the number of first-class sample points in the first prediction block is a first quantity and the number of first-class sample points in the second prediction block is a second quantity, then the first quantity can be less than the second quantity. A first quantity less than a second quantity means that in the current block, the content using affine transformation is the main content of the current block. In the geometric partitioning pattern, this is equivalent to the part using affine transformation occupying a larger area in the current block, while other content is secondary and occupies a relatively smaller area. This configuration is chosen because, in this embodiment, prediction based on an affine model may require carrying motion vector differences in the bitstream. Carrying motion vector differences in the bitstream incurs significant overhead. Therefore, applying this embodiment to scenarios primarily involving affine transformations better balances bitstream overhead and prediction performance. Of course, this embodiment can also be applied when the number of first-class sample points in the first prediction block (i.e., the first number mentioned above) is greater than the number of first-class sample points in the second prediction block (i.e., the second number mentioned above).

[0223] In some implementations, reliable motion information is more easily obtained from the left, top, or upper left regions of the current block because they are close to the decoded regions surrounding the current block. Therefore, in some implementations, by setting an appropriate geometric partitioning pattern, the first prediction block can have a higher weight in the right, bottom, or lower right regions during weighting, while the second prediction block has a higher weight in the left, top, or upper left regions. This allows for greater utilization of reliable motion information from the decoded regions. In other words, when the motion information of the second prediction block is determined based on the motion information of its neighboring blocks, the second prediction block can obtain more reliable motion information, while more difficult-to-predict regions can be predicted using a relatively complex affine model with higher bitstream overhead, thus better balancing bitstream overhead and prediction performance. In a specific implementation, the first type of sample points (i.e., sample points with a weight parameter value of 0) in the first prediction block can be set in the left, top, or upper left regions of the first prediction block, while the first type of sample points in the second prediction block can be set in the right, bottom, or lower right regions of the second prediction block, thereby achieving the above objective. For example, referring to Figure 11B, by swapping the weights of the first prediction block and the second prediction block, the weight of the first prediction block in the lower right region and the weight of the second prediction block in the upper left region can be increased during weighting. It should be understood that if 16 partitioning modes (or weight derivation modes) are used, the index values ​​of these 16 partitioning modes (or weight derivation modes) can be consecutively numbered from 0 to 15. The labels in Figure 11B are only to show the correspondence between the partitioning modes (or weight derivation modes) supported by the embodiments of this application and the partitioning modes (or weight derivation modes) in related technologies.

[0224] In some implementations, reliable motion information is more easily obtained from the left, top, or top-left regions of the current block because they are close to the decoded regions surrounding the current block. Therefore, in some implementations, by setting an appropriate geometric partitioning pattern, the first prediction block can have a higher weight in the right, bottom, or bottom-right regions during weighting, while the second prediction block has a higher weight in the left, top, or top-left regions. This way, when the motion information of the second prediction block is determined based on the motion information of its neighboring blocks, the second prediction block can obtain more reliable motion information, while more difficult-to-predict regions can be predicted using a relatively complex affine model with higher bitstream overhead, thus better balancing bitstream overhead and prediction performance. In a specific implementation, the weight parameter of the sample point at the top-left corner of the first prediction block can be set to 0, and / or the weight parameter of the sample point at the bottom-right corner of the second prediction block can be set to 0. This ensures that the content corresponding to the first prediction block is mainly concentrated in the bottom-right region of the current block, and the content corresponding to the second prediction block is mainly concentrated in the top-left region of the current block.

[0225] To ensure that the first prediction block applies to the right, lower, or lower-right region, and the second prediction block applies to the left, upper, or upper-left region, there are several ways to achieve this. One possible implementation is to adjust the weighting process of the GPM corresponding to certain inter_gpm_partition_idx values. For example, when the value of inter_gpm_partition_idx is 2 to 11, this embodiment can use the first prediction block as predSamplesLA and the second prediction block as predSamplesLB, and weight them according to the weight wValue derived from the GPM to obtain the predicted value pbSamples: pbSamples[x][y] = Clip3(0,(1<<BitDepth)-1,(predSamplesLB[x][y]*wValue+predSamplesLA[x][y]*(8-wValue)+offset1)> >shift1). As another possible implementation, this embodiment can maintain the above weighting steps but adjust the value when calculating wValue. This can also be implemented in various ways, such as directly setting wValue = 8 - wValue when the value of inter_gpm_partition_idx is 2 to 11; or adjusting it during the derivation of wValue. Furthermore, this embodiment can also adjust angleIdx or partFlip to make the first prediction block act on the right, lower, or lower-right region, and the second prediction block act on the left, upper, or upper-left region. As an example, in this embodiment, partFlip is always 1.

[0226] In some implementations, when the first prediction block acts on the right, lower, or lower right region, and the second prediction block acts on the left, upper, or upper left region, the control point of the current block can include the control point at the lower right corner. This is because when the first prediction block acts on the right, lower, or lower right region, determining the motion vectors of all sub-blocks or sample points within the current block using the control point at the lower right corner helps improve the accuracy of the determined motion vectors, thereby improving prediction performance.

[0227] The prediction method of the second prediction block was briefly described in step S1020 above. Different prediction methods for the second prediction block may lead to different scenarios addressed by this application embodiment. For example, if the second prediction block is predicted based on an affine model (the affine parameters of which may differ from those used by the first prediction block), this application embodiment can handle scenarios where a block contains multiple objects, and these objects follow different affine models. As another example, if the second prediction block is predicted based on a translation model, this application embodiment can handle scenarios where a block contains multiple objects, and the motion of some objects follows a translation model while the motion of others follows an affine model.

[0228] The following example illustrates the motion pattern of the second prediction block. It should be noted that the implementation described below primarily addresses the following scenario (though its application is not limited to this scenario): a portion of the current block belongs to a moving object that follows an affine model (this portion could be content not present in adjacent blocks, or its motion significantly differs from that of decoded adjacent blocks; for better prediction of this portion, an affine model with motion vector differences is preferable); another portion of the current block contains content with the same or very similar motion as decoded blocks (such as decoded adjacent blocks surrounding the current block), or in other words, another portion of the current block can be predicted using the motion information of decoded blocks.

[0229] Since the content to be predicted by the second prediction block of the current block may have the same or similar motion information as the decoded block, motion information can be obtained from the decoded block to predict the current block. In other words, the second prediction block can be determined based on the motion information of the decoded block. The decoded block mentioned here can include one or more of the following: temporally adjacent blocks, spatially adjacent blocks, non-temporally adjacent blocks, and non-spatially adjacent blocks of the current block. As an example, the decoded block mentioned here is a spatially adjacent block of the current block, such as the left adjacent block, the top adjacent block, or the top-left adjacent block.

[0230] This application does not specifically limit the implementation method of "determining the second prediction block based on the motion information of the decoded block". One approach is to determine (or construct) a first candidate set (such as a first candidate list) based on the motion information of the decoded block; and then determine the second prediction block based on the motion information in the first candidate set. For example, at least one piece of motion information in the first candidate set can be indicated by a syntax element carried in the bitstream (this syntax element can be, for example, an index, hereinafter referred to as the second index), and then the second prediction block can be determined based on this at least one piece of motion information. Exemplarily, the second index can indicate a candidate (or a piece of motion information) in the first candidate set, or in other words, the second index can indicate the position (or order) of the motion information used to determine the second prediction block in the first candidate set.

[0231] For example, the table below provides a more specific implementation, where inter_gpm_mode_idx corresponds to the second index mentioned above.

[0232] In the above syntax elements, the content from the fifth to the second to last line is new content added in this embodiment of the application. Array indices x0 and y0 represent the coordinates (x0, y0) of the brightness sample at the top left corner of the current block relative to the brightness sample at the top left corner of the current image.

[0233] A value of 1 for inter_affine_flag[x0][y0] indicates that the current block uses motion compensation based on an affine model to generate predictions. A value of 0 for inter_affine_flag[x0][y0] indicates that the current block does not use motion compensation based on an affine model to generate predictions.

[0234] A value of 1 for `cu_affine_type_flag[x0][y0]` indicates that the current block uses a 6-parameter affine model to generate predictions. A value of 0 for `cu_affine_type_flag[x0][y0]` indicates that the current block uses a 4-parameter affine model to generate predictions. A value of 1 for `sps_6param_affine_enabled_flag` indicates that the current sequence is allowed to use a 6-parameter affine model to generate predictions. A value of 0 for `sps_6param_affine_enabled_flag` indicates that the current sequence is not allowed to use a 6-parameter affine model to generate predictions.

[0235] `inter_pred_idc[x0][y0]! = PRED_L1` indicates that the current block does not only reference image list 1 (RPL1). In other words, the current block may only reference RPL0 or both RPL0 and RPL1. Similarly, `inter_pred_idc[x0][y0]! = PRED_L0` indicates that the current block does not only reference image list 0 (RPL0). In other words, the current block may only reference RPL1 or both RPL0 and RPL1.

[0236] `ref_idx_l0[x0][y0]` and `ref_idx_l0[x0][y1]` represent the indices of reference image list 0 and reference image list 1, respectively, for the current block. `mvp_l0_flag[x0][y0]` represents the index of the motion vector prediction when referencing RPL0. `mvp_l1_flag[x0][y0]` represents the index of the motion vector prediction when referencing RPL1.

[0237] A value of 0 for `MotionModelIdc[x][y]` indicates the use of a translation model, a value of 1 indicates the use of a 4-parameter affine model, and a value of 2 indicates the use of a 6-parameter affine model. `mvd_coding` is used to parse the motion vector differences. For the same prediction direction, only one motion vector difference is parsed for translational motion, two for a 4-parameter affine motion model, and three for a 6-parameter affine motion model.

[0238] A value of 1 for inter_gpm_flag[x0][y0] indicates that the current block uses the geometric partitioning mode to generate predictions. A value of 0 for inter_gpm_flag[x0][y0] indicates that the current block does not use the geometric partitioning mode to generate predictions.

[0239] inter_gpm_partition_idx[x0][y0] indicates the partitioning mode used by the current block.

[0240] inter_gpm_mode_idx[x0][y0] indicates the index of the candidate used by the current block to generate the second prediction block in the first candidate set (or candidate list).

[0241] It is understandable that the positions of the syntax elements inter_gpm_flag, inter_gpm_partition_idx, and inter_gpm_mode_idx in the syntax element table are not limited to those shown in the table above. These syntax elements may also be located before inter_affine_flag, or they may appear before the syntax elements used to parse the motion vector difference.

[0242] Besides the implementation described above (i.e., constructing a first candidate set and then using a second index to indicate the motion information for determining the second prediction block), in other implementations, the second prediction block can be determined directly based on the motion information of the decoded block, without relying on the syntax elements transmitted in the bitstream, thereby reducing the overhead of the bitstream.

[0243] For example, a first candidate set can be determined based on the motion information of the decoded block (as an example, this first candidate set can reuse the merge candidate set of the current block; the advantage of doing so is that no new candidate set needs to be constructed, making it more hardware-friendly). Then, at least one piece of motion information is selected from the first candidate set based on a fixed position or order; next, a second prediction block is determined based on this at least one piece of motion information. For example, the second prediction block can be determined using the first piece of motion information in the first candidate set.

[0244] Alternatively, instead of constructing a candidate set, the motion information of the current block can be determined directly based on the motion information of the already encoded blocks, and then the second predicted block of the current block can be determined based on the motion information of the current block. As an example, motion information can be obtained from predefined locations around the current block, and then used as the motion information of the current block, thus determining the second predicted block of the current block. As another example, motion information can be obtained from predefined locations around the current block, and then the obtained motion information can be weighted and summed to obtain the motion information of the current block, thus determining the second predicted block of the current block based on the motion information of the current block.

[0245] For example, the table below provides a more specific implementation. Compared to the previous implementation, this implementation does not require carrying inter_gpm_mode_idx[x0][y0] in the bitstream.

[0246] The preceding text describes in detail how to determine the second prediction block based on the motion information of the encoded block, and points out that in some implementations, the motion information of the encoded block can be selected from the Merge candidate set of the current block. Besides this implementation, embodiments of this application can also set at least one predefined position in the decoded region surrounding the current block, and set a certain checking (or selection) order for the at least one predefined position. During the actual decoding process, the at least one predefined position can be checked according to a preset order to determine the motion information corresponding to the at least one predefined position (the motion information corresponding to the predefined position can refer to the motion information corresponding to the decoded block where the predefined position is located). After obtaining the motion information corresponding to at least one predefined position, the second prediction block can be determined based on the motion information corresponding to the at least one predefined position.

[0247] As mentioned in some implementations above, the weight parameters of sample points in the first and second prediction blocks can be set to give higher weights to the second prediction block in the left, upper, or upper-left regions (hereinafter referred to as the third region for ease of description), because reliable motion information is more easily obtained in these regions. Therefore, when setting at least one predefined position, K predefined positions (K is a positive integer greater than or equal to 1) can be set in the third region, and these K predefined positions can be sorted in a preset order as the first K positions (i.e., the first K positions).

[0248] In the implementation process, at least one predefined position can be checked in a preset order; then, a second candidate set (such as a second candidate list) is determined based on the motion information corresponding to the at least one predefined position, such that the motion information corresponding to the at least one predefined position is sorted in the second candidate set according to a preset order (that is, during the process of checking the motion information corresponding to the at least one predefined position, the motion information that is checked first is added to the beginning of the second candidate set). After obtaining the second candidate set, a second prediction block can be determined based on the motion information in the second candidate set. For example, the bitstream can be decoded to determine a third index. This third index is used to indicate at least one motion information in the second candidate set; then, the second prediction block is determined based on the at least one motion information. This implementation method is similar to the second index-based implementation method mentioned above, and can be referred to the previous text for details. Alternatively, at least one motion information can be determined from the second candidate set according to a fixed position or sorting, and the second prediction block can be determined based on the at least one motion information. This implementation method does not require transmitting index information in the bitstream, thereby reducing the bitstream overhead. As an example, the second prediction block is determined using the first motion information in the second candidate set.

[0249] Alternatively, in some implementations, instead of constructing the second candidate set mentioned above, the motion information corresponding to at least one predefined position is checked in a preset order. Once the first usable motion information is found, the second prediction block can be determined directly based on that first usable motion information. In other words, the motion information corresponding to at least one predefined position is checked in a preset order. When the first usable motion information is found, that motion information can be used as the motion information for the current block, and the second prediction block for the current block can be determined based on that motion information.

[0250] To facilitate understanding, the following example illustrates in more detail the design method of at least one predefined position and the method of determining the second prediction block based on the at least one predefined position, taking the example that the prediction content of the second prediction block mainly acts on the upper left region of the current block.

[0251] Figure 12A shows an example of a design method for having at least one predefined position. In the design method shown in Figure 12A, the top left corner includes a predefined position. During the decoding process, adjacent blocks containing coordinates (x-1, y-1) can be checked first, followed by adjacent blocks containing other positions.

[0252] Figure 12B provides another example of the design for at least one predefined position. In the design shown in Figure 12B, the upper left corner includes more predefined positions (i.e., positions 1 to 3 in Figure 12B, with corresponding coordinates of (x-1, y-1), (x, y-1), and (x-1, y)). This design primarily considers that the prediction content of the second prediction block mainly affects the upper left region of the current block. Therefore, detecting more blocks within the upper left corner helps to find more reliable motion information, thereby improving prediction performance. During decoding, the adjacent blocks at positions 1 to 3 can be checked first, followed by the adjacent blocks contained in other positions.

[0253] Based on at least one predefined location, a candidate list (corresponding to the second candidate set mentioned earlier) can be constructed. Since some predefined locations are close together, deduplication may be necessary when constructing the candidate list. Of course, if the available motion information is insufficient to fill the candidate list, default motion information, such as the motion vector (0,0), can be added, or new motion information can be generated based on existing motion information in the candidate list. Alternatively, instead of constructing a candidate list, the motion information corresponding to each predefined location can be checked sequentially until a usable motion information is found. After finding the usable motion information, it can be used as the motion information for the current block for motion compensation, thereby determining the second prediction block.

[0254] As mentioned in some previous implementations, it can be first determined whether to use an affine model for motion compensation, and if it is determined that an affine model is used, step S1010 is executed to determine the first prediction block; similarly, it can first determine whether to use a geometric partitioning mode, and if it is determined that a geometric partitioning mode is used, step S1020 is executed to determine the second prediction block. However, in the embodiments of this application, it is also possible to first determine whether to use a geometric partitioning mode, and then determine whether to use an affine model for motion compensation. That is to say, the embodiments of this application do not specifically limit the execution order of steps S1010 and S1020, and step S1020 can be executed first, followed by step S1010.

[0255] Because a geometric partitioning pattern is used, the pre-content provided by the first prediction block (the prediction block obtained based on the affine model) mentioned in the embodiments of this application may not occupy all the content of the current block. For example, in some partitioning patterns of the geometric partitioning pattern, the prediction content provided by the second prediction block can occupy the lower right part of the current block. In this case, how to select the control points of the current block when determining the first prediction block is a problem that needs to be considered. To address this problem, one possible implementation is to set the control points of the current block in accordance with related technologies, that is, to set the control points of the current block at the upper left and upper right corners of the current block, or to set the control points of the current block at the upper left, lower left, and upper right corners of the current block. This setting method is beneficial for compatibility with related technologies. If this implementation method is adopted, the influence of the geometric partitioning pattern does not need to be considered when determining the first prediction block. Alternatively, as another possible implementation method, new control points can be introduced for the current block, such as setting the control points at the lower right corner and / or the center point of the current block, so that the control points of the current block are more closely matched with the prediction content provided by the first prediction block.

[0256] It should be noted that the "motion information" mentioned in the preceding embodiments may include a reference image index and / or motion vectors. Where there is no conflict, the "motion information" mentioned in the preceding embodiments may also be replaced with "motion vectors".

[0257] The decoding method provided by the embodiments of this application has been described in detail above with reference to Figure 10. The encoding method provided by the embodiments of this application has been described in detail below with reference to Figure 13.

[0258] Figure 13 is a flowchart illustrating the encoding method provided in an embodiment of this application. The method shown in Figure 13 can be applied to an encoder.

[0259] Referring to Figure 13, in step S1310, the first prediction block of the current block is determined based on the affine parameters. The current block mentioned here can be a coding unit, a prediction unit, or a transform unit. The current block can include one or more samples (or sample points), and the samples in the current block can be luminance samples or chrominance samples.

[0260] The affine parameters mentioned in step S1310 refer to the parameters corresponding to the affine model. The affine model can be used to perform motion compensation on the current block, thereby determining the first predicted block of the current block. Therefore, in some implementations, step S1310 can also be expressed as: determining the first predicted block of the current block according to the affine model, or determining the first predicted block obtained by using the affine model to perform motion compensation on the current block.

[0261] Affine parameters can include motion information of the control points of the current block. The control points of the current block can include one or more of the following: the control point at the top left corner, the control point at the bottom left corner, the control point at the top right corner, and the control point at the bottom right corner. Taking an example where the control points of the current block include the control points at the top left and top right corners, the affine parameters include four parameters (the corresponding affine model can be called a 4-parameter affine model), namely the x-component, y-component, x-component, and y-component of the control point at the top left and top right corners of the current block. Taking a block whose control points include the top-left, top-right, and bottom-left corners as an example, the affine parameters can include six parameters (the corresponding affine model can be called a 6-parameter affine model), namely the x-component of the top-left control point, the y-component of the top-right control point, the y-component of the bottom-left control point, and the y-component of the bottom-left control point. Taking a block whose control points include the bottom-left and bottom-right corners as an example, the affine parameters include four parameters, namely the x-component of the bottom-left control point, the y-component of the bottom-right control point, and the y-component of the bottom-right control point. Taking a block whose control points include the top-right and bottom-right corners as an example, the affine parameters include four parameters: the x-component of the top-right corner control point, the y-component of the top-right corner control point, the x-component of the bottom-right corner control point, and the y-component of the bottom-right corner control point. Taking a block whose control points include the bottom-left, top-right, and bottom-right corners as an example, the affine parameters include six parameters: the x-component of the bottom-left corner control point, the y-component of the bottom-left corner control point, the x-component of the top-right corner control point, the y-component of the top-right corner control point, the x-component of the bottom-right corner control point, and the y-component of the bottom-right corner control point.

[0262] This application does not specifically limit the method of "determining the first prediction block of the current block based on affine parameters". For example, the motion vector of each sample point in the current block can be determined based on the affine parameters, and then the first prediction block can be determined based on the motion vector of each sample point. As another example, the motion vector of each sub-block (the size of the sub-block can be, for example, 4×4) in the current block can be determined based on the affine parameters, and then the first prediction block can be determined based on the motion vector of each sub-block. The method of determining the motion vector of each sample point or sub-block can be referred to formula (1) or formula (2) above, and will not be described in detail here.

[0263] Referring again to Figure 13, in step S1320, the second prediction block of the current block is determined. This application embodiment does not specifically limit the method of determining the second prediction block. The second prediction block can be determined based on inter-frame prediction or intra-frame prediction. If the second prediction block is determined based on inter-frame prediction, it can be determined based on an affine model (in this case, the affine parameters corresponding to the second prediction block can be different from those corresponding to the first prediction block); or, it can be determined based on a translation model. For example, the second prediction block can be determined based on the motion information of the coded blocks surrounding the current block. Alternatively, the motion information of the current block can be determined using template matching, and then the second prediction block can be determined (see the description in the "Template Matching" section above). Alternatively, the motion information of the coded blocks surrounding the current block can be determined first, and then the motion information can be adjusted using template matching technology to obtain adjusted motion information (e.g., comparing the adjusted motion information with the motion information obtained based on template matching technology to find motion information that better matches the current block), and then the second prediction block can be determined based on the adjusted motion information. The determination method of the second prediction block will be illustrated in detail later with reference to the embodiments, and will not be described in detail here.

[0264] Referring again to Figure 13, in step S1330, based on the first and second prediction blocks, a geometric partitioning pattern (or a prediction pattern based on the geometric partitioning pattern) is used to determine the third prediction block of the current block. The geometric partitioning pattern serves to simulate partitioning, essentially combining the contents of the first and second prediction blocks in a certain way.

[0265] In some implementations, when using a geometric partitioning pattern for prediction, a third prediction block can be determined by weighting the first and second prediction blocks according to weight parameters. For example, the weight parameters can be determined first based on a first index. Exemplarily, a first distance parameter (e.g., distanceIdx) and / or a first angle parameter (e.g., angleIdx) can be determined first based on the first index; then, the weight parameters can be determined based on the first distance parameter and / or the first angle parameter. The first index can indicate the partitioning pattern (or weight derivation pattern) of the current block. There is a predefined mapping relationship between the first index and the first distance parameter (e.g., distanceIdx) and / or the first angle parameter (e.g., angleIdx). The first distance parameter and / or the first angle parameter can be used to determine the weight parameters (e.g., wValue). As one possible implementation, the first distance parameter and / or the first angle parameter can be determined based on the first index, thereby determining the weight parameters. Alternatively, different methods can be used to determine the first distance parameter and / or the first angle parameter, thereby determining the weight parameters. For example, the geometric partitioning pattern can be simplified to a certain extent (e.g., reducing the number of partitioning patterns (or weight-derived patterns) indicated by the first index), thereby reducing the bitstream overhead. Alternatively, different distance parameters and / or angle parameters can be defined to form a new geometric partitioning pattern. This implementation method will be described in detail later with specific examples, and will not be elaborated here.

[0266] After determining the weight parameters, in some implementations, the first and second prediction blocks can be weighted and summed based on the weight parameters. For example, if the first prediction block is predSamplesLA, the second prediction block is predSamplesLB, the weight parameter is wValue, and the predicted value in the first prediction block is pbSamples, then the sample points in the first prediction block can be determined based on the following formula: pbSamples[x][y]=Clip3(0,(1<<BitDepth)-1,(predSamplesLA[x][y]*wValue+ predSamplesLB[x][y]*(8-wValue)+offset1)> >shift1);

[0267] The meaning of each parameter in the above formula can be found in the previous description, and will not be repeated here.

[0268] When determining the first prediction block, each sample point in the first prediction block can be determined using the above formula. In other words, after determining each sample point in the first prediction block using the above formula, the first prediction block is also determined accordingly.

[0269] In addition, after determining the weight parameters, the first index can be written into the bitstream for use by the decoder.

[0270] Referring again to Figure 13, in step S1340, the residual block of the current block is determined based on the third predicted block of the current block. For example, the residual block of the current block can be determined based on the original block and the third predicted block of the current block. Exemplarily, the residual block of the current block can be determined by subtracting the original block and the third predicted block of the current block. Then, the residual block of the current block can be written into the bitstream for use by the decoder.

[0271] As shown in Figure 13, this embodiment combines a geometric partitioning pattern with an affine model. The geometric partitioning pattern does not require actual block division to effectively handle the boundaries of two different moving objects or regions. Therefore, combining the geometric partitioning pattern with an affine model can, on the one hand, improve the prediction performance of the boundaries of two different moving objects or regions through the simulated partitioning of the geometric partitioning pattern; on the other hand, the affine model can represent richer and more varied object motions. This allows the solution provided in this embodiment to effectively handle scenarios with diverse motion types and moving objects, improving the prediction quality in such scenarios, reducing residuals, and minimizing distortion at the edges of moving objects.

[0272] As mentioned earlier, the first prediction block of the current block is obtained by motion compensation based on an affine model. The "affine" mentioned here can be an affine without motion vector difference (such as Affine Merge). The affine parameters in an affine without motion vector difference can be derived based on the motion information of the coded block, without needing to carry the motion vector difference in the bitstream, thus reducing the bitstream overhead.

[0273] However, in some application scenarios, the motion information of the current block may not have appeared in adjacent blocks (e.g., a new moving object appears in the current block), or the motion information of the current block may differ significantly from the motion information of the coded blocks. In such cases, if the motion information of the current block is derived solely based on the motion information of the coded blocks, the prediction accuracy of the first prediction block will be reduced. Therefore, in some implementations, the "affine" mentioned in the embodiments of this application can refer to an affine with motion vector difference (such as Affine AMVP). Compared to an affine without motion vector difference, the introduction of motion vector difference can better describe the motion of new objects appearing in the current block, thereby improving the prediction performance of the first prediction block.

[0274] Taking an affine transformation with motion vector differences as an example, in practical implementation, one or more motion information (referring to the motion information of the control points of the current block, which may include one or more of the control points at the top left, bottom left, top right, and bottom right corners of the current block) can be determined based on one or more motion vector differences. After obtaining the motion information of the control points of the current block, the affine parameters mentioned in step S1310 are obtained, and motion compensation can be performed based on these affine parameters to determine the first prediction block of the current block. Furthermore, these one or more motion vector differences can be written into the bitstream for use by the decoder.

[0275] As one possible implementation, if the first prediction block is determined based on an affine model with motion vector differences, the number of one or more motion vector differences can be determined by a first parameter (e.g., represented by MotionModelIdc). This first parameter indicates the motion model corresponding to the first prediction block. Specifically, the bitstream can be decoded first to determine the first parameter. If the first parameter is 0 (representing a translation model), the number of one or more motion vector differences is 1; if the first parameter is 1 (representing a 4-parameter affine model), the number of one or more motion vector differences is 2; if the first parameter is 2 (representing a 6-parameter affine model), the number of one or more motion vector differences is 3. In other words, if the first parameter indicates that the motion model corresponding to the first prediction block is a translation model, the number of one or more motion vector differences is 1; if the first parameter indicates that the motion model corresponding to the first prediction block is a 4-parameter affine model, the number of one or more motion vector differences is 2; if the first parameter indicates that the motion model corresponding to the first prediction block is a 6-parameter affine model, the number of one or more motion vector differences is 3. The syntax description of how the number of one or more motion vector differences is determined by the first parameter can be found in the "MVD" section above, and will not be elaborated here.

[0276] Taking a single motion vector difference as an example, this single motion vector difference can include an x-component and a y-component. Taking a double motion vector difference as an example, each of the two motion vector differences can include an x-component and a y-component. Taking a triple motion vector difference as an example, each of the three motion vector differences can include an x-component and a y-component.

[0277] If the first prediction block is determined based on an affine model with motion vector difference, the geometric partitioning mode in step S1330 will differ significantly from the geometric partitioning mode provided by related technologies. In other words, in this implementation, the geometric partitioning mode mentioned in step S1330 is not equivalent to the geometric partitioning mode provided by related technologies. The geometric partitioning mode provided by related technologies is a sub-mode of the Merge mode, used to generate motion information for both the first and second prediction blocks from the Merge candidate list. The basic logic of the Merge mode is that the motion of the current block and one of its neighboring blocks (including neighboring blocks in the spatial domain and / or temporal domain) is the same, or extremely similar, thus the motion information of the neighboring blocks can be directly used as the motion information of the current block. However, the first prediction block in this implementation is determined based on an affine model with motion vector difference, which better addresses scenarios where the motion information in the current block has not appeared in neighboring blocks, or where the motion of the current block differs significantly from the motion of the encoded neighboring blocks. This is precisely why it is necessary to carry motion vector difference in the bitstream.

[0278] In some implementations, step S1310 in Figure 13 can be executed under certain conditions. For example, before executing step S1310, it can be determined whether (the current block) uses an affine model for motion compensation (or, whether an affine model is used to generate a prediction block). Further, in some implementations, a syntax element can be written into the bitstream to indicate whether the current block uses an affine model for motion compensation. This syntax element can be, for example, a first identifier (e.g., represented by `inter_affine_flag`). This first identifier can be used to indicate whether (the current block) uses an affine model for motion compensation. Alternatively, the first identifier can be used to indicate whether (the current block) uses an affine model to generate a prediction block. The first identifier can include a first value and a second value. The first value can be 1, and the second value can be 0. Or, the first value can be 0, and the second value can be 1. The first value can be used to indicate that the current block uses an affine model for motion compensation. The second value can be used to indicate that the current block does not use an affine model for motion compensation.

[0279] In some implementations, step S1320 in Figure 13 can be executed under certain conditions. For example, before executing step S1320, it can be determined whether the (current block) uses a geometric partitioning mode (or, whether a prediction block is generated using a prediction mode based on the geometric partitioning mode). Further, in some implementations, a syntax element can be written into the bitstream to indicate whether the current block uses a geometric partitioning mode. This syntax element can be, for example, second identification information (e.g., represented by inter_gpm_flag). This second identification information can be used to indicate whether the (current block) uses a geometric partitioning mode. Alternatively, the second identification information can be used to indicate whether the (current block) uses a prediction block generated using a prediction mode based on the geometric partitioning mode. The second identification information can include a third value and a fourth value. The third value can be 1, and the fourth value can be 0. Or, the third value can be 0, and the fourth value can be 1.

[0280] As mentioned in step S1330 above, when using the geometric partitioning mode, the first and second predicted blocks of the current block can be weighted and summed according to the weight parameters. Furthermore, a first index can be written into the bitstream. This first index can be used to determine the first distance parameter and / or the first angle parameter, and then determine the weight parameters based on the first distance parameter and / or the first angle parameter. The value of the first index can be one of 64 preset values ​​(0-63), thereby indicating 64 partitioning modes (or weight derivation modes). Alternatively, the value of the first index can also differ from the implementation provided by related technologies. The following provides detailed examples of the partitioning modes (or weight derivation modes) proposed in the embodiments of this application.

[0281] This application proposes an improved geometric partitioning mode. Compared to related technologies, the geometric partitioning mode in this application supports a smaller number of partitioning modes (or weight-derived modes). For example, the number of partitioning methods (or weight-derived modes) used (or supported) by the geometric partitioning mode in this application is less than the number of partitioning methods used by the geometric partitioning mode based on the Merge mode. In other words, the number of candidate values ​​for the first index can be less than the number of partitioning methods used by the geometric partitioning mode based on the Merge mode. Taking the number of partitioning methods used by the geometric partitioning mode based on the Merge mode as an example, the number of partitioning methods used by the geometric partitioning mode in this application is less than 64. This is done because when the current block is predicted based on an affine model, it may be necessary to transmit one or more motion vector differences in the bitstream, and the transmission of one or more motion vector differences requires a large amount of bitstream data. Therefore, by reducing the number of partitioning modes (or weight-derived modes), the prediction information of the current block is not too much, thereby reducing the overhead required for transmitting prediction information in the bitstream. It should be noted that the improved geometric partitioning mode proposed in this application can be applied not only to affine scenarios but also to other scenarios. For example, in scenarios where geometric partitioning patterns are applied in VVC, these geometric partitioning patterns can all be the improved geometric partitioning patterns proposed in the embodiments of this application. The improved geometric partitioning patterns are described below.

[0282] In some implementations, the number of partitioning modes (or weight-derived modes) supported by the geometric partitioning mode in this embodiment can be less than or equal to 32. Correspondingly, the value of the first index (used to indicate the partitioning mode or weight-derived mode of the current block) can be selected from a first set of values, and the number of values ​​in this first set is less than or equal to 32; that is, the number of candidate values ​​for the first index is less than or equal to 32. This reduces the number of modes supported by the geometric partitioning mode by more than half, thereby reducing the overhead required to carry the first index in the bitstream.

[0283] In some implementations, the number of partitioning modes (or weight-derived modes) supported by the geometric partitioning mode in this embodiment can be less than or equal to 16. Correspondingly, the value of the first index (used to indicate the partitioning mode or weight-derived mode of the current block) can be selected from a first set of values, and the number of values ​​in this first set is less than or equal to 16; that is, the number of candidate values ​​for the first index is less than or equal to 16. In this way, the number of modes supported by the geometric partitioning mode can be reduced to less than a quarter of the original number, thereby reducing the overhead required to carry the first index in the bitstream.

[0284] For example, referring to Figure 11A, the geometric partitioning mode in this embodiment supports 16 partitioning modes (or weight-derived modes). These 16 partitioning modes (or weight-derived modes) can correspond to the partitioning modes (or weight-derived modes) with index values ​​of 0-1 and 18-31 in related technologies. The partitioning modes (or weight-derived modes) with index values ​​of 0-1 and 18-31 can basically represent the main partitioning modes among all 64 partitioning modes. It should be understood that if 16 partitioning modes (or weight-derived modes) are used, the index values ​​of these 16 partitioning modes (or weight-derived modes) can be numbered consecutively from 0 to 15. The labels in Figure 11A are only to show the correspondence between the partitioning modes (or weight-derived modes) supported by this embodiment and the partitioning modes (or weight-derived modes) in related technologies.

[0285] If 16 partitioning modes are used, the value of the first index can be one of the 16 preset candidate values. In this scenario, the first distance parameter and / or the first angle parameter can be determined based on the first index using Table 3.

[0286] Table 3

[0287] Understandably, in addition to reducing the number of partitioning modes (or weight derivation modes) supported by the geometric partitioning modes in related technologies, it is also possible to set partitioning modes (or weight derivation modes) that are different from the geometric partitioning modes in related technologies, such as setting different angle parameters (angleIdx), distance parameters (distanceIdx), and / or distance matrices (disLut).

[0288] As mentioned earlier, the first and second prediction blocks of the current block can be weighted and summed using a geometric partitioning pattern to determine the third prediction block. The sample points in the first prediction block can include first-class and second-class sample points. First-class sample points refer to sample points with a corresponding weight parameter value of 0, and second-class sample points refer to sample points with a corresponding weight parameter value of non-zero. Similarly, the sample points in the second prediction block can also include first-class and second-class sample points. In some implementations, assuming the number of first-class sample points in the first prediction block is a first quantity and the number of first-class sample points in the second prediction block is a second quantity, then the first quantity can be less than the second quantity. A first quantity less than a second quantity means that in the current block, the content using affine transformation is the main content of the current block. In the geometric partitioning pattern, this is equivalent to the part using affine transformation occupying a larger area in the current block, while other content is secondary and occupies a relatively smaller area. This configuration is chosen because, in this embodiment, prediction based on an affine model may require carrying motion vector differences in the bitstream. Carrying motion vector differences in the bitstream incurs significant overhead. Therefore, applying this embodiment to scenarios primarily involving affine transformations better balances bitstream overhead and prediction performance. Of course, this embodiment can also be applied when the number of first-class sample points in the first prediction block (i.e., the first number mentioned above) is greater than the number of first-class sample points in the second prediction block (i.e., the second number mentioned above).

[0289] In some implementations, reliable motion information is more easily obtained from the left, top, or top-left regions of the current block because they are close to the coded regions surrounding the current block. Therefore, in some implementations, by setting an appropriate geometric partitioning pattern, the weights of the right, bottom, or bottom-right regions of the first prediction block can be increased during weighting, while the weights of the left, top, or top-left regions of the second prediction block can be increased. This allows for greater utilization of reliable motion information from the decoded regions. In other words, when the motion information of the second prediction block is determined based on the motion information of its neighboring blocks, the second prediction block can obtain more reliable motion information, while more difficult-to-predict regions can be predicted using a relatively complex affine model with higher bitstream overhead, thus better balancing bitstream overhead and prediction performance. In a specific implementation, the first type of sample points (i.e., sample points with a weight parameter value of 0) in the first prediction block can be set in the left, top, or top-left region of the first prediction block, while the first type of sample points in the second prediction block can be set in the right, bottom, or bottom-right region of the second prediction block, thereby achieving the above objective. For example, referring to Figure 11B, by swapping the weights of the first prediction block and the second prediction block, the weight of the first prediction block in the lower right region and the weight of the second prediction block in the upper left region can be increased during weighting. It should be understood that if 16 partitioning modes (or weight derivation modes) are used, the index values ​​of these 16 partitioning modes (or weight derivation modes) can be consecutively numbered from 0 to 15. The labels in Figure 11B are only to show the correspondence between the partitioning modes (or weight derivation modes) supported by the embodiments of this application and the partitioning modes (or weight derivation modes) in related technologies.

[0290] In some implementations, reliable motion information is more easily obtained from the left, top, or top-left regions of the current block because they are close to the decoded regions surrounding the current block. Therefore, in some implementations, by setting an appropriate geometric partitioning pattern, the first prediction block can have a higher weight in the right, bottom, or bottom-right regions during weighting, while the second prediction block has a higher weight in the left, top, or top-left regions. This way, when the motion information of the second prediction block is determined based on the motion information of its neighboring blocks, the second prediction block can obtain more reliable motion information, while more difficult-to-predict regions can be predicted using a relatively complex affine model with higher bitstream overhead, thus better balancing bitstream overhead and prediction performance. In a specific implementation, the weight parameter of the sample point at the top-left corner of the first prediction block can be set to 0, and / or the weight parameter of the sample point at the bottom-right corner of the second prediction block can be set to 0. This ensures that the content corresponding to the first prediction block is mainly concentrated in the bottom-right region of the current block, and the content corresponding to the second prediction block is mainly concentrated in the top-left region of the current block.

[0291] To ensure that the first prediction block applies to the right, lower, or lower-right region, and the second prediction block applies to the left, upper, or upper-left region, there are several ways to achieve this. One possible implementation is to adjust the weighting process of the GPM corresponding to certain inter_gpm_partition_idx values. For example, when the value of inter_gpm_partition_idx is 2 to 11, this embodiment can use the first prediction block as predSamplesLA and the second prediction block as predSamplesLB, and weight them according to the weight wValue derived from the GPM to obtain the predicted value pbSamples: pbSamples[x][y] = Clip3(0,(1<<BitDepth)-1,(predSamplesLB[x][y]*wValue+predSamplesLA[x][y]*(8-wValue)+offset1)> >shift1). As another possible implementation, this embodiment can maintain the above weighting steps but adjust the value when calculating wValue. This can also be implemented in various ways, such as directly setting wValue = 8 - wValue when the value of inter_gpm_partition_idx is 2 to 11; or adjusting it during the derivation of wValue. Furthermore, this embodiment can also adjust angleIdx or partFlip to make the first prediction block act on the right, lower, or lower-right region, and the second prediction block act on the left, upper, or upper-left region. As an example, in this embodiment, partFlip is always 1.

[0292] In some implementations, when the first prediction block acts on the right, lower, or lower right region, and the second prediction block acts on the left, upper, or upper left region, the control point of the current block can include the control point at the lower right corner. This is because when the first prediction block acts on the right, lower, or lower right region, determining the motion vectors of all sub-blocks or sample points within the current block using the control point at the lower right corner helps improve the accuracy of the determined motion vectors, thereby improving prediction performance.

[0293] The prediction method of the second prediction block was briefly described in step S1320 above. Different prediction methods for the second prediction block may lead to different scenarios addressed by this application embodiment. For example, if the second prediction block is predicted based on an affine model (the affine parameters of which may differ from those used by the first prediction block), this application embodiment can handle scenarios where a block contains multiple objects, and these objects follow different affine models. As another example, if the second prediction block is predicted based on a translation model, this application embodiment can handle scenarios where a block contains multiple objects, and the motion of some objects follows a translation model while the motion of others follows an affine model.

[0294] The following example illustrates the motion pattern of the second prediction block. It should be noted that the implementation described below primarily addresses the following scenario (though its application is not limited to this scenario): a portion of the current block belongs to a moving object that follows an affine model (this portion could be content not present in adjacent blocks, or its motion significantly differs from that of encoded adjacent blocks; for better prediction of this portion, an affine model with motion vector differences is preferable); another portion of the current block contains content with the same or very similar motion as encoded blocks (such as encoded adjacent blocks surrounding the current block), or in other words, another portion of the current block can be predicted using motion information from decoded blocks.

[0295] Since the content to be predicted by the second prediction block of the current block may have the same or similar motion information as the encoded block, motion information can be obtained from the encoded block to predict the current block. In other words, the second prediction block can be determined based on the motion information of the encoded block. The encoded block mentioned here can include one or more of the following: temporally adjacent blocks, spatially adjacent blocks, non-temporally adjacent blocks, and non-spatially adjacent blocks of the current block. As an example, the encoded block mentioned here is a spatially adjacent block of the current block, such as the left adjacent block, the top adjacent block, or the top-left adjacent block.

[0296] This application does not specifically limit the implementation of "determining the second prediction block based on the motion information of the encoded block". One approach is to determine (or construct) a first candidate set (such as a first candidate list) based on the motion information of the encoded block; then, to determine the second prediction block based on the motion information in the first candidate set. For example, prediction blocks can be determined based on the motion information in the first candidate set, and then, based on the rate-distortion cost of each prediction block, at least one piece of motion information matching the current block can be determined, and the prediction block determined by the at least one piece of motion information can be used as the second prediction block. Then, the at least one piece of motion information in the first candidate set can be indicated by a syntax element carried in the bitstream (the syntax element can be, for example, an index, hereinafter referred to as the second index). Exemplarily, the second index can indicate a candidate (or a piece of motion information) in the first candidate set, or in other words, the second index can indicate the position (or order) of the motion information used to determine the second prediction block in the first candidate set.

[0297] For example, the table below provides a more specific implementation, where inter_gpm_mode_idx corresponds to the second index mentioned above.

[0298] In the above syntax elements, the content from the fifth to the second to last line is new content added in this embodiment of the application. Array indices x0 and y0 represent the coordinates (x0, y0) of the brightness sample at the top left corner of the current block relative to the brightness sample at the top left corner of the current image.

[0299] A value of 1 for inter_affine_flag[x0][y0] indicates that the current block uses motion compensation based on an affine model to generate predictions. A value of 0 for inter_affine_flag[x0][y0] indicates that the current block does not use motion compensation based on an affine model to generate predictions.

[0300] A value of 1 for `cu_affine_type_flag[x0][y0]` indicates that the current block uses a 6-parameter affine model to generate predictions. A value of 0 for `cu_affine_type_flag[x0][y0]` indicates that the current block uses a 4-parameter affine model to generate predictions. A value of 1 for `sps_6param_affine_enabled_flag` indicates that the current sequence is allowed to use a 6-parameter affine model to generate predictions. A value of 0 for `sps_6param_affine_enabled_flag` indicates that the current sequence is not allowed to use a 6-parameter affine model to generate predictions.

[0301] `inter_pred_idc[x0][y0]! = PRED_L1` indicates that the current block does not only reference image list 1 (RPL1). In other words, the current block may only reference RPL0 or both RPL0 and RPL1. Similarly, `inter_pred_idc[x0][y0]! = PRED_L0` indicates that the current block does not only reference image list 0 (RPL0). In other words, the current block may only reference RPL1 or both RPL0 and RPL1.

[0302] `ref_idx_l0[x0][y0]` and `ref_idx_l0[x0][y1]` represent the indices of reference image list 0 and reference image list 1, respectively, for the current block. `mvp_l0_flag[x0][y0]` represents the index of the motion vector prediction when referencing RPL0. `mvp_l1_flag[x0][y0]` represents the index of the motion vector prediction when referencing RPL1.

[0303] A value of 0 for `MotionModelIdc[x][y]` indicates the use of a translation model, a value of 1 indicates the use of a 4-parameter affine model, and a value of 2 indicates the use of a 6-parameter affine model. `mvd_coding` is used to parse the motion vector differences. For the same prediction direction, only one motion vector difference is parsed for translational motion, two for a 4-parameter affine motion model, and three for a 6-parameter affine motion model.

[0304] A value of 1 for inter_gpm_flag[x0][y0] indicates that the current block uses the geometric partitioning mode to generate predictions. A value of 0 for inter_gpm_flag[x0][y0] indicates that the current block does not use the geometric partitioning mode to generate predictions.

[0305] inter_gpm_partition_idx[x0][y0] indicates the partitioning mode used by the current block.

[0306] inter_gpm_mode_idx[x0][y0] indicates the index of the candidate used by the current block to generate the second prediction block in the first candidate set (or candidate list).

[0307] It is understandable that the positions of the syntax elements inter_gpm_flag, inter_gpm_partition_idx, and inter_gpm_mode_idx in the syntax element table are not limited to those shown in the table above. These syntax elements may also be located before inter_affine_flag, or they may appear before the syntax elements used to parse the motion vector difference.

[0308] Besides the implementation described above (i.e., constructing a first candidate set and then selecting appropriate motion information to determine the second prediction block based on rate-distortion cost), in other implementations, the second prediction block can also be determined directly based on predefined motion information without relying on the syntax elements transmitted in the bitstream, thereby reducing the overhead of the bitstream.

[0309] For example, a first candidate set can be determined based on the motion information of an already encoded block (as an example, this first candidate set can reuse the merged candidate set of the current block; the advantage of doing so is that no new candidate set needs to be constructed, making it more hardware-friendly). Then, at least one piece of motion information is selected from the first candidate set based on a fixed position or order; next, a second prediction block is determined based on this at least one piece of motion information. For example, the second prediction block can be determined using the first piece of motion information in the first candidate set.

[0310] Alternatively, instead of constructing a candidate set, the motion information of the current block can be determined directly based on the motion information of the already encoded blocks, and then the second predicted block of the current block can be determined based on the motion information of the current block. As an example, motion information can be obtained from predefined locations around the current block, and then used as the motion information of the current block, thus determining the second predicted block of the current block. As another example, motion information can be obtained from predefined locations around the current block, and then the obtained motion information can be weighted and summed to obtain the motion information of the current block, thus determining the second predicted block of the current block based on the motion information of the current block.

[0311] For example, the table below provides a more specific implementation. Compared to the previous implementation, this implementation does not require carrying inter_gpm_mode_idx[x0][y0] in the bitstream.

[0312] The preceding text described in detail how to determine the second prediction block based on the motion information of the encoded block, and pointed out that in some implementations, the motion information of the encoded block can be selected from the Merge candidate set of the current block. Besides this implementation, embodiments of this application can also set at least one predefined position in the encoded region surrounding the current block, and set a certain checking (or selection) order for the at least one predefined position. During the actual encoding process, the at least one predefined position can be checked according to a preset order to determine the motion information corresponding to the at least one predefined position (the so-called motion information corresponding to the predefined position can refer to the motion information corresponding to the encoded block where the predefined position is located). After obtaining the motion information corresponding to at least one predefined position, the second prediction block can be determined based on the motion information corresponding to the at least one predefined position.

[0313] As mentioned in some implementations above, the weight parameters of sample points in the first and second prediction blocks can be set to give higher weights to the second prediction block in the left, upper, or upper-left regions (hereinafter referred to as the third region for ease of description), because reliable motion information is more easily obtained in these regions. Therefore, when setting at least one predefined position, K predefined positions (K is a positive integer greater than or equal to 1) can be set in the third region, and these K predefined positions can be sorted in a preset order as the first K positions (i.e., the first K positions).

[0314] In the implementation process, at least one predefined position can be checked in a preset order; then, a second candidate set (such as a second candidate list) is determined based on the motion information corresponding to the at least one predefined position, such that the motion information corresponding to the at least one predefined position is sorted in the second candidate set according to a preset order (that is, during the process of checking the motion information corresponding to the at least one predefined position, the motion information that is checked first is added to the beginning of the second candidate set). After obtaining the second candidate set, a second prediction block can be determined based on the motion information in the second candidate set. For example, prediction can be performed based on the motion information in the second candidate set to determine the rate-distortion cost corresponding to the motion information in the second candidate set. Then, at least one piece of motion information can be selected from the second candidate set based on the rate-distortion cost corresponding to the motion information in the second candidate set, and the prediction block determined based on the at least one piece of motion information is used as the second prediction block. In addition, a third index can be written to the bitstream. This third index is used to indicate the at least one piece of motion information in the second candidate set for use by the decoder. Alternatively, at least one piece of motion information can be determined from the second candidate set based on a fixed position or sorting, and the second prediction block can be determined based on the at least one piece of motion information. This implementation method does not require transmitting index information in the bitstream, thereby reducing the overhead of the bitstream. As an example, the second prediction block is determined using the first motion information from the second candidate set.

[0315] Alternatively, in some implementations, instead of constructing the second candidate set mentioned above, the motion information corresponding to at least one predefined position is checked in a preset order. Once the first usable motion information is found, the second prediction block can be determined directly based on that first usable motion information. In other words, the motion information corresponding to at least one predefined position is checked in a preset order. When the first usable motion information is found, that motion information can be used as the motion information for the current block, and the second prediction block for the current block can be determined based on that motion information.

[0316] To facilitate understanding, the following example illustrates in more detail the design method of at least one predefined position and the method of determining the second prediction block based on the at least one predefined position, taking the example that the prediction content of the second prediction block mainly acts on the upper left region of the current block.

[0317] Figure 12A shows an example of a design approach for having at least one predefined location. In the design approach shown in Figure 12A, the top left corner includes a predefined location. During the encoding process, adjacent blocks containing coordinates (x-1, y-1) can be checked first, followed by adjacent blocks containing other locations.

[0318] Figure 12B provides another example of the design for at least one predefined position. In the design shown in Figure 12B, the upper left corner includes more predefined positions (i.e., positions 1 to 3 in Figure 12B, with corresponding coordinates of (x-1, y-1), (x, y-1), and (x-1, y)). This design primarily considers that the prediction content of the second prediction block mainly affects the upper left region of the current block. Therefore, detecting more blocks within the upper left corner helps to find more reliable motion information, thereby improving prediction performance. During the encoding process, the adjacent blocks at positions 1 to 3 can be checked first, followed by the adjacent blocks contained in other positions.

[0319] Based on at least one predefined location, a candidate list (corresponding to the second candidate set mentioned earlier) can be constructed. Since some predefined locations are close together, deduplication may be necessary when constructing the candidate list. Of course, if the available motion information is insufficient to fill the candidate list, default motion information, such as the motion vector (0,0), can be added, or new motion information can be generated based on existing motion information in the candidate list. Alternatively, a candidate list can be omitted, and the motion information corresponding to the predefined locations can be checked in the order of at least one predefined location until a usable motion information is found. After finding the usable motion information, it can be used as the motion information for the current block for motion compensation, thereby determining the second prediction block.

[0320] As mentioned in some previous implementations, it can be first determined whether to use an affine model for motion compensation, and if it is determined that an affine model is used, step S1310 is executed to determine the first prediction block; similarly, it can first determine whether to use a geometric partitioning mode, and if it is determined that a geometric partitioning mode is used, step S1320 is executed to determine the second prediction block. However, in the embodiments of this application, it is also possible to first determine whether to use a geometric partitioning mode, and then determine whether to use an affine model for motion compensation. That is to say, the embodiments of this application do not specifically limit the execution order of steps S1310 and S1320, and step S1320 can be executed first, followed by step S1310.

[0321] Because a geometric partitioning pattern is used, the pre-content provided by the first prediction block (the prediction block obtained based on the affine model) mentioned in the embodiments of this application may not occupy all the content of the current block. For example, in some partitioning patterns of the geometric partitioning pattern, the prediction content provided by the second prediction block can occupy the lower right part of the current block. In this case, how to select the control points of the current block when determining the first prediction block is a problem that needs to be considered. To address this problem, one possible implementation is to set the control points of the current block in accordance with related technologies, that is, to set the control points of the current block at the upper left and upper right corners of the current block, or to set the control points of the current block at the upper left, lower left, and upper right corners of the current block. This setting method is beneficial for compatibility with related technologies. If this implementation method is adopted, the influence of the geometric partitioning pattern does not need to be considered when determining the first prediction block. Alternatively, as another possible implementation method, new control points can be introduced for the current block, such as setting the control points at the lower right corner and / or the center point of the current block, so that the control points of the current block are more closely matched with the prediction content provided by the first prediction block.

[0322] It should be noted that the "motion information" mentioned in the preceding embodiments may include a reference image index and / or motion vectors. Where there is no conflict, the "motion information" mentioned in the preceding embodiments may also be replaced with "motion vectors".

[0323] The embodiments of this application are described in more detail below with specific examples. It should be noted that the examples below are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of this application to the specific numerical values ​​or scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or variations based on the given examples, and such modifications or variations also fall within the scope of the embodiments of this application.

[0324] This example includes the following steps:

[0325] Step a: Decode the bitstream to determine whether the current block uses an affine model to generate a prediction block. If the current block uses an affine model to generate a prediction block, decode at least one motion vector difference, determine a motion vector based on the motion vector difference, and determine the first prediction block based on the motion vector.

[0326] Step b: Decode the bitstream and determine whether the current block uses the geometric partitioning mode to generate a prediction block. If the current block uses the geometric partitioning mode, determine the second prediction block.

[0327] Step c: Determine the third prediction block of the current block based on the first prediction block, the second prediction block, and the geometric partitioning pattern.

[0328] In the above steps, the order of determining whether the current block uses an affine model and determining whether the current block uses a geometric partitioning pattern can be reversed.

[0329] This scheme has three elements: affine modeling, motion vector difference (GPM), and geometrical motion modeling (GPM). The GPM discussed here is not the same as the GPM mode in VVC. In VVC, GPM is a sub-mode of the merge mode, where the motion information for generating the first and second predicted values ​​comes from the merge candidate list. The basic logic of the merge mode is that the motion of the current block and one of its neighboring blocks, including temporal neighboring blocks, is the same or extremely similar, thus the motion information of the neighboring block can be directly used as the motion information of the current block. However, the scenario applied in this application is that the motion information in the current block has not appeared in neighboring blocks, or that the motion of the current block is significantly different from the motion of the already encoded and decoded neighboring blocks. This is the reason for needing motion vector difference. On the other hand, not all content in the current block conforms to the same motion model, such as an affine model or a translation model; rather, some parts conform to one motion model, and others conform to another. This is the reason for using GPM. The GPM described here is a method that combines two predicted values ​​using geometrical partitioning.

[0330] Similar to the current weighting process of GPM, this scheme uses the first prediction block as predSamplesLA and the second prediction block as predSamplesLB, and weights them according to the weights wValue derived from GPM to obtain the third prediction block pbSamples: pbSamples[x][y]=Clip3(0,(1<<BitDepth)-1,(predSamplesLA[x][y]*wValue+ predSamplesLB[x][y]*(8-wValue)+offset1)> >shift1).

[0331] In some embodiments, the affine control points still use one or more of the control points at the top left, top right, and bottom left corners of the current block. Although in this scheme, the first affine prediction block does not actually occupy all the content within the current block—for example, sometimes the first affine prediction block occupies the bottom right corner of the current block, such as in partitioning mode 31 in Figure 7—using the control points at the top left, top right, and bottom left corners of the current block is beneficial for unifying the processing. Under this premise, it can be understood that the influence of GPM is not considered in the operations of determining the motion vectors of control points using MVD, determining the motion vectors of sub-blocks or sample points within the current block using the affine model, and in the process of generating the first predicted value using the motion vectors.

[0332] The decoder parses a syntax element to determine whether the current block uses GPM mode. GPM supports multiple partitioning methods, as shown in Figure 7. If the current block determines to use GPM mode, the decoder parses another syntax element to select which partitioning mode to use. Because GPM partitioning is actually implemented using weights, and each partitioning mode can derive a different weight matrix, the partitioning modes of GPM are also referred to here as GPM weight-derived modes.

[0333] The remaining question is how to determine the second prediction block. As mentioned above, the affine motion information hasn't appeared in adjacent blocks, or its motion differs significantly from that of already encoded / decoded adjacent blocks. The other part, the second prediction block, should have the same or very similar motion in already encoded / decoded adjacent blocks. Therefore, in terms of the information to be written into the bitstream, the affine part carries multiple MVDs, providing the main new information and thus bearing the majority of the load. The information guiding the second prediction block can be simplified, ensuring that the total amount of prediction information for the entire block is not excessive. This scheme improves prediction quality, reducing residuals and distortion at the edges of moving objects. However, for a single block, it increases the overhead of prediction mode indication in the bitstream.

[0334] For the information guiding the second prediction block, we consider obtaining motion information from surrounding already encoded and decoded blocks, similar to the idea behind merge. One approach is to construct a candidate list by obtaining motion information from surrounding already encoded and decoded blocks, using a syntax element to indicate which candidate in the candidate list the information guiding the second prediction value is selected from, or in other words, the index of the selected motion information in the candidate list. The second approach requires no indication and directly derives the unique motion information based on the motion information of surrounding encoded blocks.

[0335] An example of using the syntax of Method 1 is as follows:

[0336] In the above syntax elements, the content from the fifth to the second to last line is new content added in this embodiment of the application. Array indices x0 and y0 represent the coordinates (x0, y0) of the brightness sample at the top left corner of the current block relative to the brightness sample at the top left corner of the current image.

[0337] A value of 1 for inter_affine_flag[x0][y0] indicates that the current block uses motion compensation based on an affine model to generate predictions. A value of 0 for inter_affine_flag[x0][y0] indicates that the current block does not use motion compensation based on an affine model to generate predictions.

[0338] A value of 1 for `cu_affine_type_flag[x0][y0]` indicates that the current block uses a 6-parameter affine model to generate predictions. A value of 0 for `cu_affine_type_flag[x0][y0]` indicates that the current block uses a 4-parameter affine model to generate predictions. A value of 1 for `sps_6param_affine_enabled_flag` indicates that the current sequence is allowed to use a 6-parameter affine model to generate predictions. A value of 0 for `sps_6param_affine_enabled_flag` indicates that the current sequence is not allowed to use a 6-parameter affine model to generate predictions.

[0339] `inter_pred_idc[x0][y0]! = PRED_L1` indicates that the current block does not only reference image list 1 (RPL1). In other words, the current block may only reference RPL0 or both RPL0 and RPL1. Similarly, `inter_pred_idc[x0][y0]! = PRED_L0` indicates that the current block does not only reference image list 0 (RPL0). In other words, the current block may only reference RPL1 or both RPL0 and RPL1.

[0340] `ref_idx_l0[x0][y0]` and `ref_idx_l0[x0][y1]` represent the indices of reference image list 0 and reference image list 1, respectively, for the current block. `mvp_l0_flag[x0][y0]` represents the index of the motion vector prediction when referencing RPL0. `mvp_l1_flag[x0][y0]` represents the index of the motion vector prediction when referencing RPL1.

[0341] A value of 0 for `MotionModelIdc[x][y]` indicates the use of a translation model, a value of 1 indicates the use of a 4-parameter affine model, and a value of 2 indicates the use of a 6-parameter affine model. `mvd_coding` is used to parse the motion vector differences. For the same prediction direction, only one motion vector difference is parsed for translational motion, two for a 4-parameter affine motion model, and three for a 6-parameter affine motion model.

[0342] A value of 1 for inter_gpm_flag[x0][y0] indicates that the current block uses the geometric partitioning mode to generate predictions. A value of 0 for inter_gpm_flag[x0][y0] indicates that the current block does not use the geometric partitioning mode to generate predictions.

[0343] inter_gpm_partition_idx[x0][y0] indicates the partitioning mode used by the current block.

[0344] inter_gpm_mode_idx[x0][y0] indicates the index of the candidate used by the current block to generate the second prediction block in the first candidate set (or candidate list).

[0345] Compared to the syntax of Method 1, Method 2 does not have the parsing of inter_gpm_mode_idx[x0][y0].

[0346] It is understandable that the positions of the syntax elements inter_gpm_flag, inter_gpm_partition_idx, and inter_gpm_mode_idx in the syntax element table are not limited to those shown in the table above. These syntax elements may also be located before inter_affine_flag, or they may appear before the syntax elements used to parse the motion vector difference.

[0347] The weight derivation mode in this scheme can be the same as the weight derivation mode of GPM in merge. Alternatively, the weight derivation mode in this scheme can be different from the weight derivation mode of GPM in merge. As mentioned above, the MVD portion of the affine mapping requires a significant amount of bitstream; therefore, this portion occupied by the MVD-enabled affine mapping can be considered the main content of the current block, reflected in the GPM partition, where this portion can occupy a larger area. The other portion, as secondary content, can occupy a smaller area in the GPM partition. Furthermore, since the left and top sides of the current block have already been encoded and decoded during encoding and decoding, it's understandable that the upper left part of the current block is more likely to obtain reliable motion vector prediction, while the lower right part is less likely to obtain reliable motion vector prediction. Therefore, the MVD-enabled affine mapping portion can be placed in the lower right part (including the right side region, lower side region, or lower right region), while the other portion can be placed in the upper left part (including the left side region, upper side region, or upper left region).

[0348] For example, this scheme only allows modes with indices 0-1 and 18-31 in the GPM of the VVC merge, a total of 16 modes. This means the number of modes is one-quarter of the GPM in the VVC merge, saving the overhead of encoding weight-derived modes. In some embodiments, the two weights of certain modes can be swapped so that the parts corresponding to the first prediction block are in the lower right and the parts corresponding to the second prediction block are in the upper left. As shown in Figure 11B, it can be understood that in this scheme, the mode indices of the GPM weight-derived modes are consecutive from 0 to 15. The labels in the figure are only to show the correspondence between the weight-derived modes of this scheme and the GPM modes in the VVC.

[0349] Specifically, the correspondence between the first index and the first angle parameter and the first distance parameter is shown in Table 3.

[0350] To ensure that the first prediction block applies to the lower right portion and the second prediction block applies to the upper left portion, there are several ways to achieve this. One approach is to adjust the GPM weighting process for certain first indices (e.g., represented by inter_gpm_partition_idx). For example, when the value of inter_gpm_partition_idx is between 2 and 11, this scheme uses the first prediction block as predSamplesLA and the second prediction block as predSamplesLB, and weights them according to the weights wValue derived from GPM to obtain the predicted value pbSamples: pbSamples[x][y]=Clip3(0,(1<<BitDepth)-1,(predSamplesLB[x][y]*wValue+predSamplesLA[x][y]*(8-wValue)+offset1)> >shift1). Alternatively, the weighting steps above can remain unchanged, but adjustments can be made when calculating the value of wValue. This can also be implemented in several ways, such as directly setting wValue = 8 - wValue when the value of inter_gpm_partition_idx is 2 to 11. Alternatively, adjustments can be made during the derivation of wValue, so angleIdx or partFlip can also be adjusted here. An example is that in this embodiment, partFlip is always 1.

[0351] Of course, you can also set different weight export modes than the merged GPM, such as setting different angleIdx, distanceIdx, and disLut, etc.

[0352] The method for determining the second prediction block is described below.

[0353] Method 1:

[0354] For Method 1, the merged candidate list is directly used as the candidate list. For Method 2, the first candidate in the merged candidate list is directly selected as the motion information for generating the second prediction block. The advantage of this approach is that no additional logic is needed to specifically construct the candidate list for this scheme, making it more hardware-friendly.

[0355] Method 2:

[0356] Considering the simplified GPM partitioning model described above, the second prediction block only applies to the upper left part of the current block. Therefore, the motion information of the adjacent blocks on the upper left side of the current block is more reliable than the motion information of the adjacent blocks on the lower left and upper right sides. Thus, the motion information of the adjacent blocks on the upper left side can be used first.

[0357] For example, first check the block containing coordinates (x-1, y-1), then check the other adjacent blocks. One possible order is shown in Figure 12A. Additionally, considering that the top left corner should be the most relevant region, more blocks can be checked within the top left corner range, as shown in Figure 12B, which checks the blocks containing positions 2 (x, y-1) and 3 (x-1, y).

[0358] For Method 1, prioritize checking the top-left corner. Then check other positions. Deduplication may be necessary when building the list. If the available motion information is insufficient to fill the candidate list, default motion information, such as the motion vector (0, 0), or generated motion information can be added. For Method 2, check in the above order until a usable motion information is found.

[0359] Understandably, once motion information or motion vector prediction is determined, template matching techniques can be used to adjust that motion vector.

[0360] The method embodiments of this application have been described in detail above with reference to Figures 1 to 13. The apparatus embodiments of this application will be described in detail below with reference to Figures 14 to 17. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be referred to the preceding method embodiments.

[0361] Figure 14 is a schematic diagram of the decoder structure provided in one embodiment of this application. The decoder 1400 in Figure 14 includes a first determining unit 1410, a second determining unit 1420, a third determining unit 1430, and a fourth determining unit 1440. The first determining unit 1410 is configured to determine a first prediction block of the current block based on affine parameters. The second determining unit 1420 is configured to determine a second prediction block of the current block. The third determining unit 1430 is configured to determine a third prediction block of the current block using a geometric partitioning pattern based on the first and second prediction blocks. The fourth determining unit 1440 is configured to determine a reconstructed block of the current block based on the third prediction block.

[0362] In some implementations, the decoder 1400 further includes: a decoding unit configured to decode a bitstream and determine one or more motion vector differences; and a fifth determining unit configured to determine motion information of the control points of the current block based on the one or more motion vector differences, wherein the affine parameters include the motion information of the control points of the current block.

[0363] In some implementations, the fifth determining unit is further configured to: determine the motion information of the control point of the current block based on the one or more motion vector differences and one or more motion vector predictions.

[0364] In some implementations, the control point of the current block includes one or more of the control points at the top left corner, bottom left corner, top right corner, and bottom right corner of the current block.

[0365] In some implementations, the control points of the current block include control points at the top left and top right corners of the current block; or the control points of the current block include control points at the top left, bottom left, and top right corners of the current block; or the control points of the current block include control points at the bottom left and bottom right corners of the current block; or the control points of the current block include control points at the top right and bottom right corners of the current block; or the control points of the current block include control points at the bottom left, top right, and bottom right corners of the current block.

[0366] In some implementations, the decoder 1400 further includes: a decoding unit configured to decode the bitstream and determine a first parameter; wherein, if the value of the first parameter is 0, the number of the one or more motion vector differences is 1; if the value of the first parameter is 1, the number of the one or more motion vector differences is 2; if the value of the first parameter is 2, the number of the one or more motion vector differences is 3.

[0367] In some implementations, the decoder 1400 further includes: a decoding unit configured to decode a bitstream, determine first identification information, the first identification information being used to indicate whether an affine model is used for motion compensation; and decode the bitstream to determine second identification information, the second identification information being used to indicate whether a geometric partitioning mode is used.

[0368] In some implementations, the first determining unit 1410 is further configured to: determine the first prediction block based on the affine parameters if the first identification information indicates that motion compensation is performed using an affine model; the second determining unit 1420 is further configured to: determine the second prediction block if the second identification information indicates that a geometric partitioning mode is used.

[0369] In some implementations, the decoder 1400 further includes: a decoding unit configured to decode a bitstream and determine a first index, the first index being used to determine weight parameters; the third determining unit is further configured to: perform weighted prediction based on the weight parameters, the first prediction block, and the second prediction block to determine the third prediction block.

[0370] In some implementations, the first index is used to determine a first distance parameter and / or a first angle parameter, which in turn is used to determine a weight parameter.

[0371] In some implementations, the number of candidate values ​​for the first index is less than the number of partitioning methods used in the geometric partitioning mode based on the merging pattern.

[0372] In some implementations, the number of candidate values ​​for the first index is less than or equal to 32; or, the number of candidate values ​​for the first index is less than or equal to 16.

[0373] In some implementations, the number of sample points with a weight parameter value of 0 in the first prediction block is less than the number of sample points with a weight parameter value of 0 in the second prediction block.

[0374] In some implementations, sample points with a weight parameter value of 0 in the first prediction block are located in a first region of the first prediction block, the first region including the left region, the upper region, or the upper left region of the first prediction block; and / or, sample points with a weight parameter value of 0 in the second prediction block are located in a second region of the second prediction block, the second region including the right region, the lower region, or the lower right region of the second prediction block.

[0375] In some implementations, the weight parameter value of the sample point at the top left corner of the first prediction block is 0; and / or the weight parameter value of the sample point at the bottom right corner of the second prediction block is 0.

[0376] In some implementations, the second determining unit is further configured to determine the second predictive block based on the motion information of the decoded block.

[0377] In some implementations, the second determining unit is further configured to: determine a first candidate set based on the motion information of the decoded block; and determine the second prediction block based on the motion information in the first candidate set.

[0378] In some implementations, the decoder 1400 further includes: a decoding unit configured to decode a bitstream, determining a second index, the second index being used to indicate at least one motion information in the first candidate set, the second prediction block being determined based on the at least one motion information.

[0379] In some implementations, the second prediction block is determined based on at least one motion information in the first candidate set, and the at least one motion information includes the first motion information in the first candidate set.

[0380] In some implementations, the first candidate set is the merge candidate set of the current block.

[0381] In some implementations, the decoder 1400 further includes: a sixth determining unit, configured to check at least one predefined position in the decoded region surrounding the current block according to a preset order, and determine motion information corresponding to the at least one predefined position; wherein the motion information of the decoded block includes the motion information corresponding to the at least one predefined position.

[0382] In some implementations, the decoded region includes a third region, which is the left side region, the top side region, or the upper left region of the current block. The at least one predefined position includes K positions in the third region, and the K positions are the first K positions in the preset order, where K is a positive integer greater than or equal to 1.

[0383] In some implementations, the second determining unit is further configured to: add motion information corresponding to the at least one predefined position to the second prediction block based on the second candidate set according to the preset order; and determine the second prediction block based on the motion information in the second candidate set.

[0384] In some implementations, the decoder 1400 further includes: a decoding unit configured to decode a bitstream and determine a third index, the third index being used to indicate at least one motion information in the second candidate set, the second prediction block being determined based on the at least one motion information.

[0385] In some implementations, the second prediction block is determined based on at least one motion information, wherein the at least one motion information is the first motion information determined based on the preset order.

[0386] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0387] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0388] Therefore, embodiments of this application provide a computer-readable storage medium for use with a decoder, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, implements the decoding method described in the first embodiment.

[0389] Based on the composition of the decoder 1400 and the computer-readable storage medium described above, refer to Figure 15, which shows a schematic diagram of the specific hardware structure of the decoder 1500 provided in this embodiment of the application. As shown in Figure 15, the decoder 1500 may include: a communication interface 1510, a memory 1520, and a processor 1530; the various components are coupled together through a bus system 1540. It is understood that the bus system 1540 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 1540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 1540 in Figure 15.

[0390] The communication interface 1510 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0391] Memory 1520 is used to store computer programs;

[0392] Processor 1530, when running the computer program, performs the following:

[0393] Determine the first predicted block of the current block based on the affine parameters;

[0394] Determine the second prediction block of the current block;

[0395] Based on the first prediction block and the second prediction block, a geometric partitioning pattern is used to determine the third prediction block of the current block;

[0396] The reconstruction block of the current block is determined based on the third prediction block.

[0397] It is understood that the memory 1520 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1520 of the systems and methods described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0398] The processor 1530 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1530 or by instructions in software form. The processor 1530 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1520. Processor 1530 reads the information in memory 1520 and completes the steps of the above method in conjunction with its hardware.

[0399] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0400] Alternatively, as another embodiment, the processor 1530 is also configured to execute the decoding method described in the foregoing embodiments when running the computer program.

[0401] Figure 16 is a schematic diagram of an encoder provided in one embodiment of this application. The encoder 1600 of Figure 16 includes a first determining unit 1610, a second determining unit 1620, a third determining unit 1630, and a fourth determining unit 1640. The first determining unit 1610 is configured to determine a first prediction block of the current block based on affine parameters. The second determining unit 1620 is configured to determine a second prediction block of the current block. The third determining unit 1630 is configured to determine a third prediction block of the current block using a geometric partitioning pattern based on the first and second prediction blocks. The fourth determining unit 1640 is configured to determine a residual block of the current block based on the third prediction block.

[0402] In some implementations, the encoder 1600 further includes an encoding unit configured to write one or more motion vector differences into a bitstream, the one or more motion vector differences being used to determine motion information of the control points of the current block, the affine parameters including the motion information of the control points of the current block.

[0403] In some implementations, the one or more motion information is determined based on the one or more motion vector differences and one or more motion vector predictions.

[0404] In some implementations, the control point of the current block includes one or more of the control points at the top left corner, bottom left corner, top right corner, and bottom right corner of the current block.

[0405] In some implementations, the control points of the current block include control points at the top left and top right corners of the current block; or the control points of the current block include control points at the top left, bottom left, and top right corners of the current block; or the control points of the current block include control points at the bottom left and bottom right corners of the current block; or the control points of the current block include control points at the top right and bottom right corners of the current block; or the control points of the current block include control points at the bottom left, top right, and bottom right corners of the current block.

[0406] In some implementations, the encoder 1600 further includes an encoding unit configured to write a first parameter into the bitstream; wherein, if the value of the first parameter is 0, the number of the one or more motion vector differences is 1; if the value of the first parameter is 1, the number of the one or more motion vector differences is 2; if the value of the first parameter is 2, the number of the one or more motion vector differences is 3.

[0407] In some implementations, the encoder 1600 further includes: an encoding unit configured to write first identification information into the bitstream, the first identification information being used to indicate whether an affine model is used for motion compensation; and to write second identification information into the bitstream, the second identification information being used to indicate whether a geometric partitioning mode is used.

[0408] In some implementations, the third determining unit is further configured to: perform weighted prediction based on the weight parameters, the first prediction block, and the second prediction block to determine the third prediction block; the encoder further includes: an encoding unit configured to write a first index into the bitstream, the first index being used to determine the weight parameters.

[0409] In some implementations, the first index is used to determine a first distance parameter and / or a first angle parameter, which in turn is used to determine the weight parameter.

[0410] In some implementations, the number of candidate values ​​for the first index is less than the number of partitioning methods used in the geometric partitioning mode based on the merging pattern.

[0411] In some implementations, the number of candidate values ​​for the first index is less than or equal to 32; or, the number of candidate values ​​for the first index is less than or equal to 16.

[0412] In some implementations, the number of sample points with a weight parameter value of 0 in the first prediction block is less than the number of sample points with a weight parameter value of 0 in the second prediction block.

[0413] In some implementations, sample points with a weight parameter value of 0 in the first prediction block are located in a first region of the first prediction block, the first region including the left region, the upper region, or the upper left region of the first prediction block; and / or, sample points with a weight parameter value of 0 in the second prediction block are located in a second region of the second prediction block, the second region including the right region, the lower region, or the lower right region of the second prediction block.

[0414] In some implementations, the weight parameter value of the sample point at the top left corner of the first prediction block is 0; and / or the weight parameter value of the sample point at the bottom right corner of the second prediction block is 0.

[0415] In some implementations, the second determining unit is further configured to determine the second prediction block based on the motion information of the encoded block.

[0416] In some implementations, the second determining unit is further configured to: determine a first candidate set based on the motion information of the encoded block; and determine the second prediction block based on the motion information in the first candidate set.

[0417] In some implementations, the encoder 1600 further includes an encoding unit configured to write a second index into the bitstream, the second index indicating at least one motion information in the first candidate set, the second prediction block being determined based on the at least one motion information.

[0418] In some implementations, the second prediction block is determined based on at least one motion information in the first candidate set, and the at least one motion information includes the first motion information in the first candidate set.

[0419] In some implementations, the first candidate set is the merge candidate set of the current block.

[0420] In some implementations, the encoder 1800 further includes an encoding unit configured to check at least one predefined position in the encoded region surrounding the current block according to a preset order, and determine motion information corresponding to the at least one predefined position; wherein the motion information of the encoded block includes the motion information corresponding to the at least one predefined position.

[0421] In some implementations, the decoded region includes a third region, which is the left side region, the upper side region, or the upper left region of the current block. The at least one predefined position includes K positions in the third region, and the K positions are the first K positions in the preset order, where K is a positive integer greater than or equal to 1.

[0422] In some implementations, the second determining unit is further configured to: add motion information corresponding to the at least one predefined position to a second candidate set according to the preset order; and determine the second prediction block according to the motion information in the second candidate set.

[0423] In some implementations, the encoder 1600 further includes an encoding unit configured to write a third index into the bitstream, the third index indicating at least one motion information in the second candidate set, the second prediction block being determined based on the at least one motion information.

[0424] In some implementations, the second prediction block is determined based on at least one motion information, wherein the at least one motion information is the first motion information determined based on the preset order.

[0425] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0426] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0427] Therefore, embodiments of this application provide a computer-readable storage medium applied to an encoder, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the decoding method described in any of the foregoing embodiments.

[0428] Based on the composition of the encoder 1600 described above and the computer-readable storage medium, refer to Figure 17, which shows a schematic diagram of the specific hardware structure of the encoder 1700 provided in this embodiment of the application. As shown in Figure 17, the encoder 1700 may include: a communication interface 1710, a memory 1720, and a processor 1730; the various components are coupled together through a bus system 1740. It is understood that the bus system 1740 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 1740 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 1740 in Figure 17.

[0429] The communication interface 1710 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0430] Memory 1720 is used to store computer programs;

[0431] Processor 1730, when running the computer program, performs the following:

[0432] Determine the first predicted block of the current block based on the affine parameters;

[0433] Determine the second prediction block of the current block;

[0434] Based on the first prediction block and the second prediction block, a geometric partitioning pattern is used to determine the third prediction block of the current block;

[0435] The residual block of the current block is determined based on the third prediction block.

[0436] It is understood that the memory 1720 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1720 of the systems and methods described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0437] The processor 1730 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1730 or by instructions in software form. The processor 1730 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1720. Processor 1730 reads the information in memory 1720 and completes the steps of the above method in conjunction with its hardware.

[0438] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0439] Alternatively, as another embodiment, the processor 1730 is also configured to execute the encoding method in the foregoing embodiments when running the computer program.

[0440] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0441] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0442] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0443] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0444] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0445] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A decoding method, applied to a decoder, comprising: Determine the first predicted block of the current block based on the affine parameters; Determine the second prediction block of the current block; Based on the first prediction block and the second prediction block, a geometric partitioning pattern is used to determine the third prediction block of the current block; The reconstruction block of the current block is determined based on the third prediction block.

2. The method according to claim 1, further comprising: Decode the bitstream to determine one or more motion vector differences; The motion information of the control point of the current block is determined based on the one or more motion vector differences, and the affine parameters include the motion information of the control point of the current block.

3. The method according to claim 2, wherein, The step of determining the motion information of the control point of the current block based on the one or more motion vector differences includes: Based on the one or more motion vector differences and one or more motion vector predictions, the motion information of the control points of the current block is determined.

4. The method according to claim 2 or 3, wherein, The control points of the current block include one or more of the control points at the top left corner, bottom left corner, top right corner, and bottom right corner of the current block.

5. The method according to claim 4, wherein: The control points of the current block include the control point at the top left corner and the control point at the top right corner of the current block; or The control points of the current block include the control point at the top left corner, the control point at the bottom left corner, and the control point at the top right corner of the current block; or The control points of the current block include the control point at the lower left corner and the control point at the lower right corner of the current block; or The control points of the current block include the control point at the upper right corner and the control point at the lower right corner of the current block; or The control points of the current block include the control point at the lower left corner, the control point at the upper right corner, and the control point at the lower right corner.

6. The method according to any one of claims 2 to 5, wherein, The method further includes: Decode the bitstream and determine the first parameter; Wherein, if the value of the first parameter is 0, then the number of the one or more motion vector differences is 1; If the value of the first parameter is 1, then the number of the one or more motion vector differences is 2; If the value of the first parameter is 2, then the number of the one or more motion vector differences is 3.

7. The method according to any one of claims 1 to 6, wherein, The method further includes: Decode the bitstream and determine the first identification information, which is used to indicate whether to use an affine model for motion compensation. Decode the bitstream and determine the second identification information, which is used to indicate whether a geometric partitioning mode is used.

8. The method according to claim 7, wherein: The step of determining the first predicted block of the current block based on affine parameters includes: If the first identification information indicates that motion compensation is performed using an affine model, then the first prediction block is determined based on the affine parameters; The second prediction block for determining the current block includes: If the second identification information indicates the use of a geometric partitioning pattern, then the second prediction block is determined.

9. The method according to any one of claims 1 to 8, wherein, The method further includes: Decode the bitstream and determine the first index, which is used to determine the weight parameters; The step of determining the third prediction block of the current block using a geometric partitioning pattern based on the first prediction block and the second prediction block includes: The third prediction block is determined by performing a weighted prediction based on the weight parameters, the first prediction block, and the second prediction block.

10. The method according to claim 9, wherein, The first index is used to determine the first distance parameter and / or the first angle parameter, and the first distance parameter and / or the first angle parameter is used to determine the weight parameter.

11. The method according to claim 9 or 10, wherein, The number of candidate values ​​for the first index is less than the number of partitioning methods used by the geometric partitioning mode based on the merging pattern.

12. The method according to any one of claims 9-11, wherein, The number of candidate values ​​for the first index is less than or equal to 32; or, the number of candidate values ​​for the first index is less than or equal to 16.

13. The method according to any one of claims 9 to 12, wherein, The number of sample points with a weight parameter value of 0 in the first prediction block is less than the number of sample points with a weight parameter value of 0 in the second prediction block.

14. The method according to any one of claims 9 to 13, wherein: Sample points with a weight parameter value of 0 in the first prediction block are located in a first region within the first prediction block, the first region including the left region, the upper region, or the upper-left region of the first prediction block; and / or, The sample points with a weight parameter value of 0 in the second prediction block are located in the second region of the second prediction block. The second region includes the right region, the lower region, or the lower right region of the second prediction block.

15. The method according to any one of claims 9 to 13, wherein: The weight parameter value of the sample point at the top left corner of the first prediction block is 0; and / or The weight parameter value of the sample point at the lower right corner of the second prediction block is 0.

16. The method according to any one of claims 1 to 15, wherein, The second prediction block for determining the current block includes: The second prediction block is determined based on the motion information of the decoded block.

17. The method according to claim 16, wherein, Determining the second prediction block based on the motion information of the decoded block includes: A first candidate set is determined based on the motion information of the decoded blocks; The second prediction block is determined based on the motion information in the first candidate set.

18. The method according to claim 17, wherein, The method further includes: The bitstream is decoded, and a second index is determined. The second index is used to indicate at least one motion information in the first candidate set. The second prediction block is determined based on the at least one motion information.

19. The method of claim 17, wherein, The second prediction block is determined based on at least one motion information in the first candidate set, and the at least one motion information includes the first motion information in the first candidate set.

20. The method according to any one of claims 17 to 19, wherein, The first candidate set is the merging candidate set of the current block.

21. The method according to claim 16, wherein, The method further includes: According to a preset order, at least one predefined position in the decoded region surrounding the current block is checked to determine the motion information corresponding to the at least one predefined position; The motion information of the decoded block includes motion information corresponding to at least one predefined position.

22. The method according to claim 21, wherein, The decoded region includes a third region, which is the left side region, the upper side region, or the upper left region of the current block. The at least one predefined position includes K positions in the third region, and the K positions are the first K positions in the preset order, where K is a positive integer greater than or equal to 1.

23. The method according to claim 21 or 22, wherein, Determining the second prediction block based on the motion information of the decoded block includes: According to the preset order, the motion information corresponding to the at least one predefined position is added to the second candidate set; The second prediction block is determined based on the motion information in the second candidate set.

24. The method according to claim 23, wherein, The method further includes: The bitstream is decoded, and a third index is determined, which indicates at least one motion information in the second candidate set. The second prediction block is determined based on the at least one motion information.

25. The method according to claim 21 or 22, wherein, The second prediction block is determined based on at least one motion information, wherein the at least one motion information is the first motion information determined based on the preset order.

26. An encoding method, applied to an encoder, comprising: Determine the first predicted block of the current block based on the affine parameters; Determine the second prediction block of the current block; Based on the first prediction block and the second prediction block, a geometric partitioning pattern is used to determine the third prediction block of the current block; The residual block of the current block is determined based on the third prediction block.

27. The method according to claim 26, wherein, The method further includes: One or more motion vector differences are written into the bitstream. The one or more motion vector differences are used to determine the motion information of the control points of the current block. The affine parameters include the motion information of the control points of the current block.

28. The method according to claim 27, wherein, The one or more motion information is determined based on the one or more motion vector differences and one or more motion vector predictions.

29. The method according to claim 27 or 28, wherein, The control points of the current block include one or more of the control points at the top left corner, bottom left corner, top right corner, and bottom right corner of the current block.

30. The method according to claim 29, wherein: The control points of the current block include the control point at the top left corner and the control point at the top right corner of the current block; or The control points of the current block include the control point at the top left corner, the control point at the bottom left corner, and the control point at the top right corner of the current block; or The control points of the current block include the control point at the lower left corner and the control point at the lower right corner of the current block; or The control points of the current block include the control point at the upper right corner and the control point at the lower right corner of the current block; or The control points of the current block include the control point at the lower left corner, the control point at the upper right corner, and the control point at the lower right corner.

31. The method according to any one of claims 27 to 30, wherein, The method further includes: Write the first parameter into the bitstream; Wherein, if the value of the first parameter is 0, then the number of the one or more motion vector differences is 1; If the value of the first parameter is 1, then the number of the one or more motion vector differences is 2; If the value of the first parameter is 2, then the number of the one or more motion vector differences is 3.

32. The method according to any one of claims 26 to 31, wherein, The method further includes: Write the first identification information into the bitstream. The first identification information is used to indicate whether to use an affine model for motion compensation. The second identification information is written into the bitstream, and the second identification information is used to indicate whether the geometric partitioning mode is used.

33. The method according to any one of claims 26 to 32, wherein, The step of determining the third prediction block of the current block using a geometric partitioning pattern based on the first prediction block and the second prediction block includes: The third prediction block is determined by performing a weighted prediction based on the weight parameters, the first prediction block, and the second prediction block. The method further includes: The first index is written into the bitstream, and the first index is used to determine the weight parameter.

34. The method according to claim 33, wherein, The first index is used to determine the first distance parameter and / or the first angle parameter, and the first distance parameter and / or the first angle parameter is used to determine the weight parameter.

35. The method according to claim 33 or 34, wherein, The number of candidate values ​​for the first index is less than the number of partitioning methods used by the geometric partitioning mode based on the merging pattern.

36. The method according to any one of claims 33-35, wherein, The number of candidate values ​​for the first index is less than or equal to 32; or, the number of candidate values ​​for the first index is less than or equal to 16.

37. The method according to any one of claims 33 to 36, wherein, The number of sample points with a weight parameter value of 0 in the first prediction block is less than the number of sample points with a weight parameter value of 0 in the second prediction block.

38. The method according to any one of claims 33 to 37, wherein: Sample points with a weight parameter value of 0 in the first prediction block are located in a first region within the first prediction block, the first region including the left region, the upper region, or the upper-left region of the first prediction block; and / or, The sample points with a weight parameter value of 0 in the second prediction block are located in the second region of the second prediction block. The second region includes the right region, the lower region, or the lower right region of the second prediction block.

39. The method according to any one of claims 33 to 37, wherein: The weight parameter value of the sample point at the top left corner of the first prediction block is 0; and / or The weight parameter value of the sample point at the lower right corner of the second prediction block is 0.

40. The method according to any one of claims 26 to 39, wherein, The second prediction block for determining the current block includes: The second prediction block is determined based on the motion information of the coded block.

41. The method according to claim 40, wherein, Determining the second prediction block based on the motion information of the coded block includes: A first candidate set is determined based on the motion information of the encoded blocks; The second prediction block is determined based on the motion information in the first candidate set.

42. The method according to claim 41, wherein, The method further includes: A second index is written into the bitstream, the second index being used to indicate at least one motion information in the first candidate set, and the second prediction block is determined based on the at least one motion information.

43. The method according to claim 41, wherein, The second prediction block is determined based on at least one motion information in the first candidate set, and the at least one motion information includes the first motion information in the first candidate set.

44. The method according to any one of claims 41 to 43, wherein, The first candidate set is the merging candidate set of the current block.

45. The method according to claim 40, wherein, The method further includes: According to a preset order, at least one predefined position in the encoded region surrounding the current block is checked to determine the motion information corresponding to the at least one predefined position; The motion information of the encoded block includes motion information corresponding to at least one predefined position.

46. ​​The method according to claim 45, wherein, The decoded region includes a third region, which is the left side region, the upper side region, or the upper left region of the current block. The at least one predefined position includes K positions in the third region, and the K positions are the first K positions in the preset order, where K is a positive integer greater than or equal to 1.

47. The method according to claim 45 or 46, wherein, Determining the second prediction block based on the motion information of the decoded block includes: According to the preset order, the motion information corresponding to the at least one predefined position is added to the second candidate set; The second prediction block is determined based on the motion information in the second candidate set.

48. The method according to claim 47, wherein, The method further includes: A third index is written into the bitstream, the third index being used to indicate at least one motion information in the second candidate set, and the second prediction block is determined based on the at least one motion information.

49. The method according to claim 45 or 46, wherein, The second prediction block is determined based on at least one motion information, wherein the at least one motion information is the first motion information determined based on the preset order.

50. A decoder, the decoder comprising: The first determining unit is configured to determine the first prediction block of the current block based on affine parameters; The second determining unit is configured to determine the second prediction block of the current block; The third determining unit is configured to determine the third predicting block of the current block using a geometric partitioning mode based on the first predicting block and the second predicting block; The fourth determining unit is configured to determine the reconstruction block of the current block based on the third prediction block.

51. A decoder, the decoder comprising: Memory, used to store computer programs; A processor, configured to perform the method as described in any one of claims 1 to 25 when running the computer program.

52. An encoder, the encoder comprising: The first determining unit is configured to determine the first prediction block of the current block based on affine parameters; The second determining unit is configured to determine the second prediction block of the current block; The third determining unit is configured to determine the third predicting block of the current block using a geometric partitioning mode based on the first predicting block and the second predicting block; The fourth determining unit is configured to determine the residual block of the current block based on the third prediction block.

53. An encoder, the encoder comprising: Memory, used to store computer programs; A processor, configured to perform the method as described in any one of claims 26 to 49 when running the computer program.

54. A non-volatile computer-readable storage medium for storing a bitstream, said bitstream being generated by an encoding method using an encoder, or said bitstream being decoded by a decoding method using a decoder, wherein, The decoding method is the method as described in any one of claims 1 to 25, and the encoding method is the method as described in any one of claims 26 to 49.

55. A bitstream comprising a bitstream generated by the method of any one of claims 26 to 49.

56. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 25, or the method as described in any one of claims 26 to 49.

Images