Decoding method and apparatus, coding method and apparatus, device, storage medium and program product

Abstract

The present application belongs to the technical field of coding and decoding. Disclosed are a decoding method and apparatus, a coding method and apparatus, a device, a storage medium and a program product, which can reduce the waste of transmission resources in code streams, thereby improving the coding efficiency. The decoding method in the embodiments of this application comprises: parsing a code stream, and determining a prediction mode of a current block; determining a first candidate, wherein the first candidate is determined from a hybrid candidate list, and the hybrid candidate list comprises at least one of a first-type candidate and a second-type candidate; and predicting the current block on the basis of the prediction mode and the first candidate, so as to obtain a prediction block of the current block. The decoding method in the embodiments of the present application can be applied to video coding and decoding.

Description

Decoding methods, encoding methods, devices, equipment, storage media, and program products Technical Field

[0001] This application belongs to the field of encoding and decoding technology, specifically relating to a decoding method, encoding method, apparatus, device, storage medium, and program product. Background Technology

[0002] Versatile Video Coding (VVC) is a next-generation video compression technology that offers higher compression efficiency and better image quality. When decoding a current block using VVC, prediction is typically performed based on a prediction mode, such as intra-frame prediction or inter-frame prediction. For inter-frame prediction, a merge mode can be used to predict the current block.

[0003] Currently, when predicting the current block using merge mode, the decoder can construct a merge candidate list of the corresponding type based on the indications of the syntax elements in the bitstream.

[0004] However, when constructing the corresponding type of merge candidate list based on the syntax elements in the bitstream, there may be a situation where the merge candidate list cannot be filled with candidates. Thus, when constructing merge candidate lists of different types, there may be a situation where the merge candidate list of each type cannot be filled with candidates. Therefore, it may be meaningless to indicate the construction of the corresponding type of merge candidate list through this syntax element, which wastes transmission resources and is not conducive to improving encoding efficiency. Summary of the Invention

[0005] This application provides a decoding method, encoding method, apparatus, device, storage medium, and program product that can reduce the waste of transmission resources in the bitstream, thereby improving encoding efficiency.

[0006] In a first aspect, a decoding method is provided, comprising: parsing a bitstream to determine a prediction mode for a current block; determining a first candidate, which is determined from a mixed candidate list, the mixed candidate list including at least one of a first type of candidate and a second type of candidate; and predicting the current block based on the prediction mode and the first candidate to obtain a predicted block for the current block.

[0007] This application provides a decoding method that determines the prediction mode of the current block by parsing the bitstream; then, a first candidate is determined from a mixed candidate list, which may include at least one of a first type of candidate and a second type of candidate; finally, the current block is predicted based on the prediction mode and the first candidate to obtain the predicted block of the current block. Thus, since a mixed candidate list can be constructed after parsing and determining the prediction mode, and this mixed candidate list not only has the ability to include candidates of the first type but also the ability to include candidates of the second type, rather than only having the ability to include one type of candidate, it is not necessary to construct a candidate list of a certain type based on the indication of syntax elements in the bitstream. Therefore, the construction of the mixed candidate list does not need to be indicated by transmitting corresponding syntax elements in the bitstream, thereby reducing the syntax elements transmitted in the bitstream, saving transmission resources, and improving encoding efficiency.

[0008] Secondly, an encoding method is provided, comprising: determining a prediction mode for a current block; determining a first candidate, the first candidate being determined from a mixed candidate list, the mixed candidate list including at least one of a first type of candidate and a second type of candidate; and predicting the current block based on the prediction mode and the first candidate to obtain a prediction block for the current block.

[0009] This application provides an encoding method that first determines the prediction mode of the current block; then, a first candidate is determined from a mixed candidate list, which may include at least one of a first type of candidate and a second type of candidate; finally, based on the prediction mode and the first candidate, the current block is predicted to obtain the predicted block of the current block. Thus, since the mixed candidate list not only has the ability to include candidates of the first type but also the ability to include candidates of the second type, and not just one type, it is not necessary to write syntax elements in the bitstream to instruct the decoder to construct candidates of the corresponding type. This reduces the number of syntax elements transmitted in the bitstream, saves transmission resources, and improves encoding efficiency.

[0010] Thirdly, a decoding apparatus is provided, comprising a parsing module and a prediction module. The parsing module is used to parse the bitstream and determine the prediction mode of the current block. The prediction module is used to determine a first candidate, which is determined from a mixed candidate list, the mixed candidate list including at least one of a first type of candidate and a second type of candidate; and, based on the prediction mode and the first candidate, to predict the current block to obtain a predicted block for the current block.

[0011] Fourthly, an encoding apparatus is provided, comprising a prediction module. The prediction module is configured to: determine a prediction mode for a current block; determine a first candidate determined from a mixed candidate list including at least one of a first type of candidate and a second type of candidate; and predict the current block based on the prediction mode and the first candidate to obtain a predicted block for the current block.

[0012] Fifthly, a decoder is provided, comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the decoding method as described in the first aspect above and any possible implementation thereof.

[0013] In a sixth aspect, an encoder is provided, the encoder including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the encoding method as described in the second aspect above and any possible implementation thereof.

[0014] In a seventh aspect, a readable storage medium is provided, on which a bitstream is stored, the bitstream being encoded based on the encoding method described in the first aspect above and any possible implementation thereof.

[0015] Eighthly, a readable storage medium is provided on which a program or instructions are stored, which, when executed by a processor, implement the steps of the decoding method as described in the first aspect and any possible implementation thereof, or implement the steps of the encoding method as described in the second aspect and any possible implementation thereof.

[0016] Ninthly, embodiments of this application provide a chip including a processor and a communication interface coupled to the processor. The processor is used to run programs or instructions to implement the steps of the decoding method as described in the first aspect and any possible implementation thereof, or to implement the steps of the encoding method as described in the second aspect and any possible implementation thereof.

[0017] Ninthly, a computer program product is provided, which is stored in a storage medium and is executed by at least one processor to implement the steps of the decoding method as described in the first aspect and any possible implementation thereof, or to implement the steps of the encoding method as described in the second aspect and any possible implementation thereof.

[0018] The effects of the third to ninth aspects mentioned above can be found in the beneficial effects of any possible implementation of the first or second aspect and each of the other aspects. To avoid repetition, they will not be elaborated here. Attached Figure Description

[0019] Figure 1 is a schematic diagram of the architecture of an encoding / decoding system provided in an embodiment of this application;

[0020] Figure 2 is a schematic block diagram of an encoder provided in an embodiment of this application;

[0021] Figure 3 is a schematic block diagram of a decoder provided in an embodiment of this application;

[0022] Figure 4 is a schematic diagram of a screen group structure provided in an embodiment of this application;

[0023] Figure 5 is a schematic diagram of an image segmentation mode provided in an embodiment of this application;

[0024] Figure 6 is a weighting diagram of 64 modes of GPM on a square block provided in an embodiment of this application;

[0025] Figure 7 is a schematic diagram of a GPM prediction process provided in an embodiment of this application;

[0026] Figure 8 is a flowchart illustrating a decoding method provided in an embodiment of this application;

[0027] Figure 9 is a schematic diagram of a time-domain motion vector prediction provided in an embodiment of this application;

[0028] Figure 10 is a schematic diagram of the current block and surrounding blocks provided in an embodiment of this application;

[0029] Figure 11 is a schematic diagram of an affine derivation of motion vectors provided in an embodiment of this application;

[0030] Figure 12 is a schematic diagram of another affine derivation of motion vectors provided in an embodiment of this application;

[0031] Figure 13 is a schematic diagram of an affine model for determining the current block according to an embodiment of this application;

[0032] Figure 14 is a schematic diagram of another affine model for determining the current block provided in an embodiment of this application;

[0033] Figure 15 is a schematic diagram of motion vector prediction provided in an embodiment of this application;

[0034] Figure 16 is a flowchart illustrating an encoding method provided in an embodiment of this application;

[0035] Figure 17 is a schematic diagram of a decoding device provided in an embodiment of this application;

[0036] Figure 18 is a schematic diagram of another decoding device provided in an embodiment of this application;

[0037] Figure 19 is a schematic diagram of an encoding device provided in an embodiment of this application;

[0038] Figure 20 is a schematic diagram of another encoding device provided in an embodiment of this application;

[0039] Figure 21 is a schematic diagram of another encoding device provided in an embodiment of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0041] In the description of the embodiments of this application, terms such as "first" and "second" are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, a first index can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, "A or B" covers three scenarios: Scenario 1: includes A but does not include B; Scenario 2: includes B but does not include A; Scenario 3: includes both A and B. The character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0042] In the description of the embodiments of this application, "at least one," "at least one item," "at least one of," etc., refer to any one, any two, or a combination of two or more of the included objects. For example, at least one of a, b, and c can be expressed as: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more, and its meaning is similar to that of "at least one."

[0043] In the description of the embodiments of this application, "multiple" means two or more. For example, multiple candidates means two or more candidates. "At least two" has a similar meaning to "multiple," and in some embodiments, the two can be used interchangeably.

[0044] In the description of embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0045] The encoding and decoding system used in the embodiments of this application will be described below with reference to Figure 1.

[0046] Referring to Figure 1, Figure 1 shows a schematic diagram of the architecture of an encoding / decoding system 10 used in an embodiment of this application. As shown in Figure 1, the encoding / decoding system 10 may include a source device 11 and a destination device 12. The source device 11 is used to encode images; therefore, the source device 11 may be referred to as an image encoding apparatus (device) or a video encoding apparatus (device). The destination device 12 is used to decode the encoded image data generated by the source device 11; therefore, the destination device 12 may be referred to as an image decoding apparatus (device) or a video decoding apparatus (device).

[0047] The source device 11 and the destination device 12 can take various forms, and this application embodiment does not specifically limit them. For example, the source device 11 and the destination device 12 can be a desktop computer, a mobile computing device, a laptop (e.g., a laptop computer), a tablet computer, a set-top box, a handset such as a so-called "smartphone", a television, a camera, a display device, a digital media player, a video game console, a vehicle computer, or other similar devices.

[0048] Optionally, the source device 11 and the destination device 12 shown in FIG1 can be two separate devices. Alternatively, the source device 11 and the destination device 12 can also be a single device, that is, the source device 11 or its corresponding functions and the destination device 12 or its corresponding functions can be integrated into the same device.

[0049] Optionally, the source device 11 and the destination device 12 can communicate with each other. For example, the destination device 12 can receive encoded image data from the source device 11. In one example, the source device 11 and the destination device 12 may include one or more communication devices that can be used to transmit the encoded image data from the source device 11 to the destination device 12. The one or more communication devices may include routers, switches, base stations, or any other possible devices that facilitate communication from the source device 11 to the destination device 12, which can be determined according to actual usage requirements, and this application embodiment does not limit this.

[0050] As shown in Figure 1, the source device 11 may include an encoder 112. Optionally, the source device 11 may also include an image preprocessor 111 and a communication interface 113. The image preprocessor 111 can be used to perform preprocessing on the received image to be encoded. For example, the preprocessing performed by the image preprocessor 111 may include any possible processing such as trimming, color format conversion (e.g., from RGB to YUV format), color correction, or noise reduction. The encoder 112 can be used to receive the image preprocessed by the image preprocessor 111, process the preprocessed image using a correlation prediction mode, and output encoded image data, such as a bitstream. In some embodiments, the encoder 112 can be used to perform the encoding process described in the various embodiments below. The communication interface 113 can be used to transmit the encoded image data output by the encoder 112 to the destination device 12 or any other device (such as a storage device) for storage or direct reconstruction. Other devices can be any devices used for decoding or storage. Of course, in actual implementation, the communication interface 113 can also encapsulate the encoded image data output by the encoder 112 into a suitable format before transmission.

[0051] Optionally, the image preprocessor 111, encoder 112, and communication interface 113 may be hardware components in the source device 11, software programs in the source device 11, or a combination of hardware components and software programs in the source device 11. The specific details can be determined according to actual usage requirements, and this application embodiment does not limit this.

[0052] The destination device 12 may include a decoder 122. Optionally, the destination device 12 may also include a communication interface 121 and an image post-processor 123. The communication interface 121 may be used to receive encoded image data from the source device 11 or any other source device, such as a storage device. The communication interface 121 may also decapsulate the data transmitted by the communication interface 113 to obtain encoded image data. The decoder 122 is used to receive the encoded image data and output decoded image data (also called reconstructed image data or reconstructed image data). In some embodiments, the decoder 122 may be used to perform the decoding process described in the various embodiments below. The image post-processor 123 may be used to perform post-processing on the decoded image data to obtain post-processed image data. The post-processing performed by the image post-processor 123 may include color format conversion (e.g., from YUV format to RGB format), color correction, retouching, or resampling, and any possible processing. The image post-processor 123 may also be used to transmit the post-processed image data to a display device for display.

[0053] Optionally, the aforementioned communication interface 121, decoder 122, and image post-processor 123 may be hardware components in the target device 12, software programs in the target device 12, or a combination of hardware components and software programs in the target device 12. The specific details can be determined according to actual usage requirements, and this application embodiment does not limit this.

[0054] To better understand the encoder 112 and decoder 122 shown in Figure 1, the structure of the encoder 112 and decoder 122 shown in Figure 1 will be briefly introduced below.

[0055] Referring to Figure 2, Figure 2 shows a possible structural schematic block diagram of an encoder 112 used to implement the encoding method of the embodiments of this application.

[0056] As shown in Figure 2, the encoder 112 may include a prediction processing unit 201, a residual calculation unit 202, a residual transformation or residual quantization unit 203, an entropy coding unit 204, an inverse transformation or inverse quantization unit (also called an inverse residual transformation or inverse quantization unit) 205, a reconstruction unit 206, and an in-loop filter unit 207. Optionally, the encoder 112 may also include a buffer (not shown in Figure 2) and a decoded image buffer 208. The buffer can be used to buffer the reconstructed image block (or rebuilt image block) output by the reconstruction unit 206; the decoded image buffer 208 can be used to buffer the filtered image block output by the in-loop filter unit 207.

[0057] In one example, the input to encoder 112 is an image block (i.e., a block to be encoded or an encoding unit) of the image to be encoded. In another example, the input to encoder 112 is the image to be encoded, which can be an image or a frame from the video to be encoded. Encoder 112 may include a segmentation unit (not shown in Figure 2) that can be used to segment the image to be encoded into multiple image blocks. Encoder 112 can complete the encoding of the image to be encoded or the video to be encoded by encoding multiple image blocks one by one. For example, encoder 112 can perform the encoding process for each image block separately to complete the encoding of the image to be encoded.

[0058] In one example, when encoding video using encoder 112, encoder 112 can use the segmentation units in encoder 20 to divide each image, sub-image, or frame in the video into largest coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 256x256, 128x128, 64x64, etc.). Each LCU or CTU can be further divided into rectangular coding units (CUs) according to rules. Coding units can also be divided into prediction units (PUs), transform units (TUs), etc.

[0059] In some embodiments, the residual transformation or residual quantization unit 203 can transform the residual from the spatial domain to the frequency domain to obtain residual coefficients, and then discretize (or quantize) the residual coefficients to obtain quantization transformation coefficients (quant.transf.coeff), thereby reducing the amount of data by reducing the precision of the coefficients.

[0060] In some embodiments, the prediction processing unit 201 may include an intra-prediction unit 2011 and an inter-prediction unit 2012. Since there is a strong correlation between adjacent pixels in a frame of a video, intra-prediction can eliminate spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent frames in a video, inter-prediction can eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0061] In one example, the intra-prediction unit 2011 refers to predicting the current sample value by referencing one or more encoded sample values ​​within the current frame (or image) to eliminate spatial redundancy. The frame used for intra-prediction can be an I-frame (keyframe).

[0062] In one example, the inter-frame prediction unit 2012 can be used to perform inter-frame prediction, which can include motion estimation and motion compensation. By analyzing the motion information between adjacent frames, the content of the current frame can be predicted, thereby reducing the amount of data that needs to be encoded.

[0063] In some embodiments, inter-frame prediction in video encoding and decoding utilizes temporal correlation to eliminate redundancy. To ensure that the human eye cannot perceive any stuttering, typical video frame rates can be 30, 50, 60, or even 120 frames per second. In such videos, the correlation between adjacent frames within the same scene is high, and inter-frame prediction techniques use this correlation to predict the content to be encoded by referencing the content of already encoded and decoded frames.

[0064] In some embodiments, a common inter-frame prediction method is translational prediction. Translational prediction assumes that the content to be predicted is translated between the current image and the reference image. For example, if the content of the current block (CU or PU) is translated between the current image and the reference image, then this content can be found in the reference image using a motion vector and used as the predicted block for the current block. Translational motion video or images account for a large proportion of encoding and decoding. Stationary backgrounds, objects that are translated overall, and camera panning can all be handled using translational prediction.

[0065] Because some content in videos or images is not simply translated—for example, there are subtle changes during translation, including changes in shape and color—bidirectional prediction can be used. This employs a sub-mode within the merge mode, namely geometric partitioning mode (GPM). Two reference blocks are found in the reference image, and a weighted average of these two reference blocks is taken to obtain a predicted block that is as similar as possible to the current block. For example, in some scenes, taking a weighted average of one reference block from before and one from after the current frame may result in a more similar prediction block than taking a single reference block. Based on this, bidirectional prediction further improves compression performance compared to unidirectional prediction.

[0066] The unidirectional and bidirectional prediction in the above-mentioned translation prediction are both block-based, such as coding units or prediction units. That is, prediction is performed using a pixel matrix as the unit. The most basic block is a rectangular block, such as a square or rectangle. Video codec standards such as HEVC and VVC allow encoders to determine the size and partitioning of coding units and prediction units based on the video content. Regions with simple textures or motion tend to use larger blocks, while regions with complex textures or motion tend to use smaller blocks. The deeper the block partitioning, the more complex and closer the blocks are to the actual texture or motion, but correspondingly, the overhead for representing these partitions is greater. Motion information may also need to be transmitted in the bitstream. And generally, the finer the block partitioning, the greater the overhead of motion information.

[0067] Specifically, in the embodiments of this application, encoder 112 can be used to implement the encoding method described in the embodiments below.

[0068] In one example, an encoding process implemented by encoder 112 may include the following steps:

[0069] Step 11: Encoder 112 divides the image into blocks, determines the prediction mode through prediction processing unit 201, and predicts the current block based on the prediction mode and the reconstructed block of the encoded image block to obtain the prediction block of the current block.

[0070] The reconstructed block of the encoded image block is obtained by the inverse transform or inverse quantization unit 205 and the reconstruction unit 206 sequentially processing the quantized residual coefficient block of the encoded image block.

[0071] Step 12: The residual calculation unit 202 subtracts the predicted block from the original block of the current block to obtain the residual block.

[0072] Step 13: The residual transformation or residual quantization unit 203 transforms and quantizes the residual block to obtain the quantization conversion coefficients (also known as the quantization coefficient matrix).

[0073] Step 14: Entropy coding unit 204 entropy codes the quantization conversion coefficients and encodes relevant syntax elements (e.g., prediction mode, coding mode) to obtain the bitstream.

[0074] In one example, the encoder can also reconstruct the current block using the inverse transform or inverse quantization unit 205, the reconstruction unit 206, and the loop filter unit 207 to obtain a reconstructed block, and store the reconstructed block in a buffer. This reconstructed block can be used as a reference block for inter-frame prediction of subsequent blocks.

[0075] It should be noted that the current block can be divided into prediction units during prediction and transformation units during transformation. The division of prediction units and transformation units can be different.

[0076] Referring to Figure 3, a possible structural schematic block diagram of the decoder 122 used to implement the decoding method of the embodiments of this application is shown.

[0077] Decoder 122 can be used to receive, for example, image data encoded by encoder 112 (i.e., an encoded bitstream, for example, an encoded bitstream including image blocks and associated syntax elements) to obtain decoded image blocks.

[0078] As shown in Figure 3, the decoder 122 may include a bitstream parsing unit 301, an inverse quantization unit 302, a residual inverse transform unit 303, a prediction processing unit 304, a reconstruction unit 305, and a loop filter unit 306. In some instances, the decoder 122 may perform a decoding process that is generally the inverse of the encoding process described in the encoder 112 shown in Figure 2. Optionally, the decoder 122 may also include a buffer and a filtered image buffer. The buffer can be used to buffer the reconstructed image blocks output by the reconstruction unit 305, and the filtered image buffer can be used to buffer the filtered image blocks output by the filter unit 306.

[0079] The bitstream parsing unit 301 can be used to decode the encoded bitstream to obtain quantized residual coefficients (or quantized residual values) and / or decoding parameters (e.g., decoding parameters may include any one or more of inter-frame prediction parameters, intra-frame prediction parameters, filter parameters, and / or other syntax elements performed on the encoding side). The bitstream parsing unit 301 can also be used to forward the aforementioned decoding parameters to the prediction processing unit 304, so that the prediction processing unit 304 can perform a prediction process based on the decoding parameters.

[0080] The dequantization unit 302 is used to dequantize (i.e., reverse quantization) the quantized residual coefficients output by the code stream parsing unit 301.

[0081] The inverse residual transformation unit 303 is used to perform an inverse transformation on the aforementioned dequantized residual coefficients (e.g., inverse DCT, inverse integer transformation, or a conceptually similar inverse transformation process) to obtain the reconstructed residual values. The block obtained after the inverse transformation is the residual block of the reconstructed block to be decoded in the sample domain.

[0082] The function of reconstruction unit 305 (e.g., a summer) can be the same as that of reconstruction unit 206 of encoder 112.

[0083] The prediction processing unit 304 is used to receive or acquire encoded image data (e.g., the encoded bitstream of the current image block) and reconstructed image data. The prediction processing unit 304 can also receive or acquire relevant parameters of the prediction mode and / or information about the selected prediction mode (i.e., the decoding parameters mentioned above) from, for example, the bitstream parsing unit 302, and make a prediction of the current image block based on the relevant data and decoding parameters in the reconstructed image data to obtain the prediction block of the current image block.

[0084] In some embodiments, the prediction processing unit 304 may include an intra prediction unit 3041 and an inter prediction unit 3042.

[0085] It should be noted that for other descriptions of the intra-frame prediction unit 3041 and the inter-frame prediction unit 3042, please refer to the relevant descriptions of the intra-frame prediction unit 2011 and the inter-frame prediction unit 2012 in the above embodiments. To avoid repetition, they will not be repeated here.

[0086] The reconstruction unit 305 can be used to add the reconstructed residual block to the prediction block to obtain the reconstructed block of the image to be decoded in the sample domain, for example, by adding the residual value in the reconstructed residual block to the predicted value in the prediction block.

[0087] The filter unit 306 can be used to filter the reconstructed block to obtain a filtered block, which is the decoded image block.

[0088] Specifically, in the embodiments of this application, the decoder 122 can be used to implement the decoding method described in the embodiments below.

[0089] It is understood that in the encoder 112 and decoder 122 provided in the embodiments of this application, the processing result of a certain stage may be further processed before being output to the next stage. For example, after the prediction, transformation or filtering stages, the processing result of the corresponding stage may be further processed by Clip or shift operations.

[0090] In one example, a decoding process implemented by decoder 122 may include the following steps:

[0091] Step 21: The code stream parsing unit 301 parses the prediction mode and the residual coding mode.

[0092] Step 22: The code stream parsing unit 301 parses the quantization-related values ​​(such as near values ​​or QP values) based on the prediction mode and residual coding mode.

[0093] Step 23: The inverse quantization unit 302 analyzes the prediction mode and quantization correlation values ​​to obtain the quantization coefficient matrix.

[0094] Step 24: The prediction processing unit 304 uses intra-frame prediction or inter-frame prediction on the current block based on the prediction mode to obtain the prediction block of the current block.

[0095] Step 25: The inverse residual transformation unit 303 performs an inverse transformation on the residual coefficients to obtain the residual block of the current block.

[0096] Step 26: Reconstruction unit 305 adds the predicted block and the residual block to obtain the reconstruction block.

[0097] Step 27: The filter unit 306 filters the reconstructed block to obtain the filtered block, which is the image block obtained after decoding.

[0098] It should be noted that the block partitioning information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, can be carried in the bitstream when necessary. The decoder determines the same block partitioning information, prediction, transform, quantization, entropy coding, and loop filtering mode information or parameter information as the encoder by parsing the bitstream and analyzing existing information, thereby ensuring that the decoded image obtained by the encoder and the decoded image obtained by the decoder are the same.

[0099] The above describes the basic flow of the encoder and decoder under the block-based hybrid coding framework. With the development of technology, some modules or steps of this framework or process may be optimized. This application is applicable to the basic flow of the encoder and decoder under the block-based hybrid coding framework, but is not limited to this framework and process.

[0100] It should be noted that the aforementioned current block can be the current coding unit (CU), the current prediction unit (PU), or the current coding tree unit (CTU), etc. Due to the need for parallel processing, images can be divided into slices, etc. Slices within the same image can be processed in parallel, meaning there is no data dependency between them. Furthermore, a frame can be understood as an image; in the embodiments of this application, a frame can be replaced with an image or a slice, etc.

[0101] In related technologies, within a video sequence, a picture order count (POC) can be used as the identifier for each image in the sequence, thus assigning a unique POC to each image. A P-image in the video sequence is an image that can be predicted using reference images preceding the current image based on the POC. If the current image is a P-image, it has a Reference Picture List (RPL), denoted as RPL0. RPL0 contains all reference images preceding the current image based on the POC. A B-image in the video sequence is an image that can be predicted using both reference images preceding and following the current image based on the POC. If the current image is a B-image, it has two Reference Picture Lists, denoted as RPL0 and RPL1. One configuration is that RPL0 contains all reference images preceding the current image based on the POC, and RPL1 contains all reference images following the current image based on the POC. For a given block, prediction can refer to only a reference block in RPL0 (forward prediction), only a reference block in RPL1 (backward prediction), or both reference blocks in RPL0 and RPL1 (bidirectional prediction). A simple method to refer to both reference blocks is to average the pixels at corresponding positions in both blocks to obtain the predicted block for the current block. Alternatively, another configuration does not restrict RPL0 to contain only reference images with the POC preceding the current image, nor does RPL1 contain only reference images with the POC following the current image. That is, RPL0 can include both reference images with the POC preceding and following the current image, and RPL1 can include both reference images with the POC preceding and following the current image. Therefore, for a given block, it is also possible to simultaneously refer to reference images with the POC preceding or following the current image. This type of B-image can be called a generalized B-image.

[0102] In some embodiments, taking the POC order and playback order as the same as an example, two encoding / decoding orders can be configured when encoding and decoding images. One is the encoding / decoding order configured with Random Access (RA), and the other is the encoding / decoding order configured with Low Delay (LD).

[0103] In one example, the encoding / decoding order in the RA configuration differs from the POC order. This allows B-images to reference information before and after the current image, significantly improving encoding performance. For example, a group of pictures (GOP) structure in RA is shown in Figure 4. The arrows in Figure 4 indicate reference relationships. I-images do not require reference images. After decoding the I-image with POC 0, the P-image with POC 4 is decoded, referencing the I-image with POC 0. Then, the B-image with POC 2 is decoded, referencing the I-image with POC 0 and the P-image with POC 4, and so on.

[0104] In one example, the encoding / decoding order of the LD configuration is the same as the POC order, and the current image can only refer to information preceding the current image. LD configurations can include two types: LDP (Low Delay P) and LDB (Low Delay B). The structure of LDP is IPPP…, meaning that an I image is encoded or decoded first, and subsequent encoded or decoded images are all P images. The structure of LDB is IBBB…, the difference from LDP is that each inter-frame image is a B image; that is, an I image is encoded or decoded first, and subsequent encoded or decoded images are all B images. In the case of LDB configuration, two reference image lists are used. The current block can simultaneously reference the reference block of an image in RPL0 and the reference block of an image in RPL1, specifically the reference block of an image preceding the current image in RPL0 and the reference block of an image preceding the current image in RPL1.

[0105] Thus, since bidirectional forecasting can reference backward information and can reduce forecasting errors through techniques such as weighted averaging, the compression efficiency of the RA configuration is higher than that of the LD configuration, and the compression efficiency of the LDB configuration is higher than that of the LDP configuration.

[0106] In some embodiments, a list of reference images for the current image may include multiple reference images, such as two, three, or four. When encoding a given current image, the number of reference images in RPL0 and RPL1 can be determined based on a certain configuration or algorithm. However, the same reference image may appear simultaneously in both RPL0 and RPL1; that is, when the encoder encodes the current block, it can simultaneously reference two reference blocks of the same reference image, or when the decoder decodes the current block, it can simultaneously reference two reference blocks of the same reference image.

[0107] In some embodiments, the encoder or decoder can use index values ​​from a list of reference images to correspond to reference images. If a list of reference images has a length of 4, then the index has four values: 0, 1, 2, and 3. For example, if the current frame's RPL0 includes four reference images with POCs of 5, 4, 3, and 0, then index 0 of RPL0 is the reference image for POC 5, index 1 of RPL0 is the reference image for POC 4, index 2 of RPL0 is the reference image for POC 3, and index 3 of RPL0 is the reference image for POC 0.

[0108] In some embodiments, inter-frame prediction can use motion information to represent "motion." Motion information can include information about a reference picture and information about motion vectors. To enable bidirectional prediction for a block, two reference blocks can be found, requiring two sets of reference picture information and motion vector information. Each set can be understood as a unidirectional motion information set, and combining these two sets forms a bidirectional motion information set. In specific implementations, unidirectional and bidirectional motion information can use the same data structure; however, in bidirectional motion information, both sets of reference picture information and motion vector information are valid, while in unidirectional motion information, one set of reference picture information and motion vector information is invalid. It can be understood that "validity" can also be referred to as "use," and "invalidity" can also be referred to as "non-use."

[0109] In some embodiments, when the codec performs inter-frame prediction, two reference image lists, denoted as RPL0 and RPL1, can be used. For the aforementioned bidirectional motion information, the reference image index refIdxL0 and motion vector mvL0 corresponding to RPL0, and the reference image index refIdxL1 and motion vector mvL0 corresponding to RPL1 can be used. The reference image indices for RPL0 and RPL1 can be understood as the information of the aforementioned reference images. Furthermore, the flag predFlagL0 can be used to indicate whether the motion vector corresponding to RPL0 is used, and the flag predFlagL1 can be used to indicate whether the motion vector corresponding to RPL1 is used. It can be understood that the flags predFlagL0 and predFlagL1 can be used to indicate whether the aforementioned unidirectional motion information is "valid". That is, combining the reference image index, motion vector, and "valid" flag for each reference image list can be used to represent motion information. Furthermore, it is understood that the description of motion information can be replaced by information from motion vectors and reference images, and the information from the reference image and the flag indicating whether to use the corresponding motion information can be considered as subordinate to the motion vectors. Moreover, bidirectional motion information can also be described as a set of motion vectors and their corresponding reference image information, where the set of motion vectors can contain two motion vectors.

[0110] In some embodiments, motion information may be used for ease of description, but it should be understood that motion vectors can also be used. Furthermore, the aforementioned motion information can also be referred to as motion parameters. Both have the same meaning.

[0111] For a two-dimensional image, motion vectors can be represented by (x, y), meaning they can be represented by horizontal and vertical components. Since images are represented as samples, and there are distances between samples, the motion of an object in adjacent images doesn't always correspond to a whole sample distance. For example, in a distant view video, the distance between two samples might be 1 meter on a distant object, but the object might move 0.5 meters in the time interval between two frames. In this scenario, whole sample motion vectors cannot accurately represent the object's motion. Therefore, motion vectors can be represented more precisely by sub-sample levels, such as 1 / 2, 1 / 4, 1 / 8, and 1 / 16 sample precision. Furthermore, interpolation can be used to obtain sample values ​​at sub-sample locations in the reference image.

[0112] In one example, motion information can be represented by complete motion information, or it can be obtained through motion vector prediction (MVP) and motion vector difference (MVD), i.e., MV = MVP + MVD. The more accurate the MVP, the smaller the MVD, thus reducing the overhead in the bitstream.

[0113] Understandably, during inter-frame prediction, each predicted block requires a motion vector. To simplify the problem, we can assume that the CU partitioning equals the PU partitioning equals the TU partitioning, meaning each coding unit has a prediction unit and a transform unit of the same size and position. In reality, with more flexible CU partitioning, the concepts of PU and TU are no longer used; instead, CUs are used exclusively. Differences in any stage of prediction, transform, quantization, or entropy coding can lead to different CU partitioning. For example, if two regions have different motion vectors, the encoder might partition them into different CUs; ​​similarly, if two regions have the same or similar motion vectors but significantly different residual characteristics, the encoder might also partition them into different CUs. The partitioning method is determined by overall compression efficiency, not solely by any single factor. Therefore, the same object or regions with similar or identical motion may be partitioned into different CUs. In such cases, encoding the motion vector separately for each block would be significantly wasteful.

[0114] For example, Figure 5 shows an example of video coding. Image a is the original image, showing a metal pole moving in the direction of the arrow, with minimal motion in the background area. Image b shows the block division, and image c removes the boundaries of blocks with the same motion information as in image b. As can be seen from Figure 5, many adjacent blocks use the same motion information. In this case, encoding the motion vector separately for each block would be significantly wasteful.

[0115] Therefore, a merge mode can be used, allowing the current block to inherit the motion vectors of its neighboring blocks. In merge mode, a list of merge candidates can be constructed. If the current block uses merge mode, an index can be used to indicate which motion vector the current block should merge with, thus eliminating the need to encode the complete motion vector.

[0116] In some embodiments, the boundaries of textures or motion in natural videos or images are diverse. For example, when encountering a slanted object boundary, simply using rectangular blocks to approximate the boundary would result in many small blocks, which would significantly increase overhead. GPM can better handle textures and boundaries in videos or images.

[0117] GPM uses two prediction blocks of the same size as the current block to predict its position. Within each GPM prediction block, some sample locations can use 100% of the sample values ​​from the corresponding locations in the first prediction block, and others can use 100% of the sample values ​​from the corresponding locations in the second prediction block. In the boundary or transition region, the sample values ​​from both prediction blocks can be used proportionally. The weighting in the boundary region also gradually transitions. This means that different partitioning or weight derivation modes can be used to determine the weight of each sample location for different image content scenarios. However, in some cases, such as when the block size is very small, certain GPM modes may not guarantee that some sample locations will use 100% of the sample values ​​from the first prediction block, and others will use 100% of the sample values ​​from the second prediction block. In such cases, GPM can be considered to use two prediction blocks of different sizes than the current block, taking a portion from each and removing the portions with zero weight.

[0118] For example, as shown in Figure 6, which illustrates the weighting of 64 patterns of GPM on a square block, black represents a weight of 0% for the position corresponding to the first prediction block, white represents a weight of 100% for the position corresponding to the first prediction block, and the transition area between the black and white areas can represent a weight value greater than 0% and less than 100% for the position corresponding to the first prediction block, depending on the shade of the color. The weight value for the position corresponding to the second reference block is 100% minus the weight value for the position corresponding to the first reference block.

[0119] GPM can be considered a prediction mode or method because it ultimately produces a prediction block. Alternatively, GPM can be described as a "partitioning" mode that simulates partitioning the prediction block, similar to implementing PU partitioning, but without actually partitioning it. The first and second prediction blocks used in GPM can be generated by intra-frame prediction, inter-frame unidirectional prediction, or inter-frame bidirectional prediction.

[0120] The two parts of GPM can use inter-frame prediction, intra-frame prediction, and intra-block copy (IBC), respectively. That is, GPM can combine prediction blocks from two inter-frame predictions, or it can combine prediction blocks from one inter-frame prediction and one intra-frame prediction, etc.

[0121] In some embodiments, the syntax description of GPM may be as shown in Table 1:

[0122] Table 1

[0123] As shown in Table 1, when encoding or decoding an image using GPM, three pieces of information need to be transmitted in the bitstream: merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1. Here, x0 and y0 can be used to determine the coordinates (x0, y0) of the current block's top-left corner brightness sample relative to the top-left corner brightness sample of the image. merge_gpm_partition_idx can be used to determine the partition shape of GPM (i.e., simulated partitioning); merge_gpm_partition_idx can be used to determine the weight derivation mode, or in other words, merge_gpm_partition_idx can be an index of the weight derivation mode, or an index of the weight matrix derivation mode; merge_gpm_idx0 is the first merge candidate index, which can be used to determine the first motion vector (or the first merge candidate) based on the merge candidate list. merge_gpm_idx1 is the second merge candidate index, which can be used to determine the second motion vector (or second merge candidate) based on the merge candidate list. If MaxNumGpmMergeCand > 2 (i.e., the candidate list length is greater than 2), then merge_gpm_idx1 needs to be decoded; otherwise, the first motion vector to be used can be directly determined based on merge_gpm_idx0.

[0124] It should be noted that in merge mode, if regular_merge_flag is not 1, the current block may use Combined Inter and Intra Prediction (CIIP) or GPM. If the current block does not use CIIP, then it uses GPM, which is the content shown in if(!ciip_flag[x0][y0]).

[0125] In some embodiments, the input information for the GPM decoding process includes: the coordinates (xCb, yCb) of the luminance position of the top-left corner of the current block relative to the top-left corner of the image; the width cbWidth of the luminance component of the current block; the height cbHeight of the luminance component of the current block; luminance motion vectors mvA and mvB with 1 / 16 sample precision; chrominance motion vectors mvCA and mvCB; reference image indices refIdxA and refIdxB; and prediction list flags predListFlagA and predListFlagB.

[0126] In related technologies, motion vectors can be represented by combining motion vectors, reference image indices, and prediction list flags. These technologies can support two reference image lists, each potentially containing multiple reference images. Unidirectional prediction uses only one reference block from one reference image in one of the reference image lists as a reference, while bidirectional prediction uses one reference block from each of the two reference image lists as a reference. In the aforementioned mvA and mvB, mvCA and mvCB, refIdxA and refIdxB, and predListFlagA and predListFlagB, A can be understood as one prediction mode, and B as another. In the following description, X can be used to represent A or B. That is, X in predListFlagX can indicate whether the first or second reference image list is used, X in refIdxX can indicate the reference image index in the used reference image list, X in mvX can indicate the luma motion vector used, and X in mvCX can indicate the chroma motion vector used.

[0127] The output information of the GPM decoding process includes: a luminance prediction sample matrix predSamplesL of (cbWidth) × (cbHeight); a prediction sample matrix (optional) of the Cb chromaticity component of (cbWidth / SubWidthC) × (cbHeight / SubHeightC); and a prediction sample matrix (optional) of the Cr chromaticity component of (cbWidth / SubWidthC) × (cbHeight / SubHeightC).

[0128] For example, the luminance component will be used below. It is understood that the processing of the chrominance component is similar to that of the luminance component, and will not be repeated here.

[0129] Assume that `predSamplesLAL` and `predSamplesLBL` are prediction sample matrices derived from two prediction modes, and that their sizes are (cbWidth) × (cbHeight). `predSamplesLAL` can be derived as follows: `predSamplesLAL` and `predSamplesLBL` are determined based on the luminance motion vectors `mvA` and `mvB`, the chrominance motion vectors `mvCA` and `mvCB`, the reference image indices `refldxA` and `refldxB`, and the prediction list flags `predListFlagA` and `predListFlagB`. That is, predictions are made based on the motion vectors of the two prediction modes; the detailed process is not elaborated here. Since GPM is a sub-mode of the merge mode, both prediction modes of GPM can be considered as merge modes.

[0130] Based on merge_gpm_partition_idx[xCb][yCb], use Table 2 to determine the “partition” angle index variable angleldx and distance index variable distanceldx for GPM.

[0131] Table 2

[0132] Table 2 shows the correspondence between angleIdx and distanceIdx and merge_gpm_partition_idx.

[0133] It should be noted that since all three components (such as Y, Cb, and Cr) can use GPM, some standard texts package the process of generating the GPM prediction sample matrix for one component into a sub-process, namely the weighted sample prediction process for geometric partitioning mode (GPM). All three components will call this process, but with different parameters. Here, we only use the luminance component as an example: the prediction matrix predSamplesL[xL][yL] (where xL = O..cbWidth-1, yL = O..cbHeight-1) is derived from the weighted prediction process of GPM. Here, nCbW can be set to cbWidth, nCbH can be set to cbHeight, and the prediction sample matrices predSamplesLAL and predSamplesLBL generated by the two prediction modes, along with angleldx and distanceldx, can be used as prediction inputs.

[0134] In some embodiments, the weighted prediction derivation process for GPM may include the following steps:

[0135] The inputs to this process are: the width nCbW of the current block, the height nCbH of the current block; two (nCbW)×(nCbH) prediction sample matrices, predSamplesLA and predSamplesLB; the weight-derived modality index variable angleldx of the GPM; the distance index variable distanceldx of the GPM; and the component index variable cIdx. This example uses luminance, so cIdx can be 0 to represent the luminance component.

[0136] The output of this process is a GPM prediction sample matrix pbSamples of (nCbW)×(nCbH).

[0137] For example, the variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor can be derived as follows:

[0138] nW=(cIdx==0)? nCbW:nCbW*SubWidthC;

[0139] nH=(cIdx==0)? nCbH:nCbH*SubHeightC;

[0140] shift1 = Max(5, 17 - BitDepth), where BitDepth is the bit depth for encoding or decoding;

[0141] offsetl = 1 << (shift1 - 1), where "<<" indicates a left shift;

[0142] displacementX = angleldx;

[0143] displacementY=(angleldx+8)%32;

[0144] partFlip=(angleldx>=13&&angleldx<=27)? 0:1;

[0145] shiftHor=(angleldx%16==8||(angleldx%16!=0&&nH>=nW))? 0:1.

[0146] The variables offsetX and offsetY are derived as follows:

[0147] When the value of shiftHor is 0:

[0148] offsetX = (-nW) >> 1,

[0149] offsetY=((-nH)>>1)+(angleldx<16?(distanceldx*nH)>>3:_-((distanceldx nH)>>3)).

[0150] When shiftHor is 1:

[0151] offsetX=((-nW)>>1)+(angleldx<16?(distanceldx*nW)>>3:-((distanceldx*nW)>>3),

[0152] offsetY = (-nH) >> 10.

[0153] Variables xL and yL are derived as follows:

[0154] xL=(cIdx==0)? x:x*SubWidthC,

[0155] yL=(cIdx==0)? y:y*SubHeightC.

[0156] The variable wValue, which represents the weight of the predicted sample at the current position, can be derived as follows: wValue is the weight of the predicted value predSamplesLA[x][y] of the prediction matrix of the first prediction mode at point (x, y), and (8–wValue) is the weight of the predicted value predSamplesLB[x][y] of the prediction matrix of the first prediction mode at point (x, y).

[0157] The distance matrix disLut can be determined according to Table 3:

[0158] Table 3

[0159] weightIdx=(((xL+offsetX)<<1)+1)*disLut[displacement]+(((yL+offsetY)<<1)+1)*disLut[displacementY],

[0160] weightIdxL=partFlip? 32+weightIdx:32–weightIdx,

[0161] wValue=Clip3(0, 8, (weightIdxL+4)>>3).

[0162] The predicted sample values ​​pbSamples[x][y] are derived as follows:

[0163] pbSamples[x][y] = Clip3(0, (1<<BitDepth)–1,(predSamplesLA[x][y]*wValue+predSamplesLB[x][y]*(8-wValue)+offsetl)> >shift1).

[0164] It should be noted that a weight value is derived for each position in the current block, and then a predicted GPM value pbSamples[x][y] is calculated. Because the weight wValue derived in this way is not limited to matrix form, it can be understood that if the wValue of each position is stored in a matrix, then it is a weight matrix. The predicted GPM value can be obtained by calculating the weight of each point separately and then weighting them, or by calculating all the weights and then weighting them uniformly; the principles are similar. The use of the term "weight matrix" in many descriptions in this application is for easier understanding and more intuitive visualization. In actual implementation, it can also be described according to the weight of each position. For example, the weight matrix derivation mode can also be described as a weight derivation mode.

[0165] In some embodiments, as shown in FIG7, which illustrates the GPM decoding process, the decoding process may include: parsing the bitstream to determine whether the current block uses GPM prediction; if the current block uses GPM prediction, determining the weight matrix derivation mode (also known as the partitioning mode or weight derivation mode), the first motion information, and the second motion information; then determining the first prediction block based on the first motion information and the second prediction block based on the second motion information; then determining the weight matrix based on the weight matrix derivation mode, and determining the prediction block of the current block based on the first prediction block, the second prediction block, and the weight matrix.

[0166] It should be noted that the aforementioned first motion information can also be referred to as the first motion vector, and the aforementioned second motion information can also be referred to as the second motion vector. In the embodiments of this application, the first motion information and the second motion information are only used to distinguish different motion information and do not represent the order of the motion information. For example, the first motion information can be the motion information parsed first from the bitstream, and the second motion information can be the motion information parsed later from the bitstream; or, the first motion information can be the motion information parsed later from the bitstream, and the second motion information can be the motion information parsed first from the bitstream. Of course, in actual implementation, the first motion information and the second motion information can also be motion information parsed simultaneously from the bitstream.

[0167] In related technologies, there are two merging modes: ordinary translational motion merging and merging based on affine motion models. These two merging modes can be distinguished by `merge_subblock_flag`. If the value of `merge_subblock_flag` is 1, the current block uses the sub-block merging mode. The decoder can construct a sub-block merging candidate list, which can include candidates based on subblock-based temporal motion vector prediction (SBTMVP) and candidates based on affine motion models. If the value of `merge_subblock_flag` is 0, the current block does not use the sub-block merging mode, and the decoder can construct an ordinary merging candidate list, which can include translational motion candidates. In fact, even when the value of `merge_subblock_flag` is 0, there are still several modes, such as MMVD mode, CIIP mode, or GPM mode. However, these modes all rely on translational motion candidates and also need to obtain candidates from the merge candidate list.

[0168] When constructing the merge candidate list, the length of the merge candidate list can be specified, which is the maximum number of candidates in the merge candidate list. `MaxNumMergeCand` can represent the maximum number of candidates in the normal merge candidate list, and `MaxNumSubblockMergeCand` can represent the maximum number of candidates in the subblock merge candidate list. When constructing the merge candidate list, suitable candidates are first searched from surrounding blocks, but there is no guarantee that there are enough candidates from surrounding blocks to fill the merge candidate list. If the merge candidate list is not full, then constructed candidates or even default motion vectors like (0,0) need to be filled in according to certain rules. When constructing the normal merge candidate list, if the surrounding blocks are encoded using affine mode, only one motion vector is actually used to represent the translational motion. However, when constructing the subblock merge candidate list, if there are not enough affine models from the surrounding blocks, or if enough affine models cannot be constructed, then (0,0) must be filled in at the end.

[0169] In this case, since the candidates in the constructed merge candidate list may be constructed candidates or default motion vectors, and the constructed candidates or default motion vectors may be invalid for the prediction of the current block, it may be meaningless to distinguish between the two modes by using merge_subblock_flag so that the decoder can construct the corresponding merge candidate list. As a result, transmitting merge_subblock_flag in the bitstream wastes transmission resources and is not conducive to improving coding efficiency.

[0170] The following description, in conjunction with the accompanying drawings, provides an exemplary account of the decoding method, encoding method, apparatus, device, storage medium, and program product provided in the embodiments of this application.

[0171] It should be noted that the encoder in any embodiment of this application can be the encoder 112 in Figure 1 or Figure 2 above, or the source device 11 in Figure 1 above. The decoder in any embodiment of this application can be the decoder 122 in Figure 1 or Figure 3 above, or the destination device 12 in Figure 1 above. This application does not limit the specific implementation of the decoder.

[0172] Referring to Figure 8, which shows a flowchart of a decoding method provided in an embodiment of this application, the decoding method can be applied to a decoder. As shown in Figure 8, the decoding method provided in this embodiment of the application may include the following steps 401 to 403.

[0173] Step 401: The decoder parses the bitstream and determines the prediction mode of the current block.

[0174] In some embodiments of this application, the above prediction mode can be used to predict the current block.

[0175] In some embodiments of this application, the prediction mode described above may be a merge mode, or the prediction mode may be GPM.

[0176] In some embodiments of this application, the aforementioned current block may be a CU, or a PU, or other possible forms of coded blocks, etc., and the embodiments of this application are not limited thereto.

[0177] In some embodiments of this application, the encoder can use a prediction mode to predict the current block, obtain a prediction block for the current block, then obtain a residual based on the prediction block and the current block, and encode the residual and the prediction mode to obtain a bitstream, which is then transmitted to the decoder. After receiving the bitstream, the decoder can parse the bitstream to determine the values ​​of the syntax elements, and determine the prediction mode for the current block based on the values ​​of the syntax elements.

[0178] For example, assuming that the syntax element obtained by the decoder from the bitstream is general_merge_flag[x0][y0] and the value of the syntax element is 1, the decoder can determine that the prediction mode of the current block is the merge mode.

[0179] Step 402: The decoder determines the first candidate.

[0180] In some embodiments of this application, the first candidate may be determined from a mixed candidate list.

[0181] In some embodiments of this application, the aforementioned mixed candidate list may be referred to as a mixedCandList, and the candidates in the mixed candidate list may be referred to as candidates. It should be noted that the aforementioned mixed candidate list may also be referred to in other ways, such as a candidate list or a mixed and merged candidate list, etc., and the embodiments of this application do not limit it.

[0182] In some embodiments of this application, the above-mentioned mixed candidate list may include at least one of a first type of candidate and a second type of candidate.

[0183] It is understood that the hybrid candidate list in this application embodiment has the ability to include both the first type of candidate and the second type of candidate, but it is not necessary to include both types of candidates simultaneously. That is, the hybrid candidate list may include the first type of candidate, or the hybrid candidate list may include the second type of candidate, or the hybrid candidate list may include both the first type of candidate and the second type of candidate.

[0184] In other words, the aforementioned mixed candidate list may include one or more candidates. If the mixed candidate list includes one candidate, that candidate may be either a first-type candidate or a second-type candidate. If the mixed candidate list includes multiple candidates, those multiple candidates may include only first-type candidates, or only second-type candidates, or a combination of both types.

[0185] It should be noted that the candidates mentioned in the embodiments of this application refer to motion information, that is, the motion information of the surrounding blocks of the current block, or motion information determined based on the motion information of the surrounding blocks of the current block. This motion information is used to predict the current block to obtain a predicted block for the current block.

[0186] In some embodiments of this application, the first type of candidates and the second type of candidates described above can be determined based on different motion models. In other words, the first type of candidates and the second type of candidates described above can belong to different motion models.

[0187] In one example, the candidates of the first type described above can be determined based on a translational motion model. The candidates of the second type described above can be determined by a sub-block-based motion model. Exemplarily, the candidates of the second type described above can be determined based on an affine motion model, or the candidates of the second type described above can be determined by other sub-block-based motion models.

[0188] Among them, the candidates determined based on the affine motion model can be called affine motion information candidates, and the candidates determined by other sub-block-based motion models can include sub-block-based temporal motion information candidates.

[0189] It should be noted that the above affine motion model can also be called an affine model or an affine model.

[0190] In some embodiments of this application, the first type of candidate described above may be referred to as a translation candidate or a candidate for translational motion. The translation candidate can be understood as a candidate determined based on the motion information of the surrounding blocks of the current block when the surrounding blocks are predicted based on a translational motion model.

[0191] In some embodiments of this application, the candidates of the first type mentioned above may include one or more of the following: spatial motion information candidates, temporal motion information candidates, historical motion information candidates, and pairwise averaged motion information candidates.

[0192] In some embodiments of this application, the aforementioned spatial motion information candidates may include motion information of adjacent blocks of the current block in the spatial domain or motion information of non-adjacent blocks of the current block in the spatial domain. Here, adjacent blocks of the current block in the spatial domain refer to surrounding blocks adjacent to the current block in the same image, and non-adjacent blocks of the current block in the spatial domain refer to surrounding blocks that are not adjacent to the current block in the same image.

[0193] In some embodiments of this application, the aforementioned temporal motion information candidates can be understood as a supplement to the spatial motion information candidates. Generally, the correlation between adjacent regions on the same image is stronger than the correlation between regions on different images. However, there are also cases where temporal motion information is more useful. For example, if the current block and its adjacent surrounding blocks in the current image belong to different objects and have drastically different motions, the motion of a block belonging to the same object as the current block in a reference image can provide better motion information prediction for the current block.

[0194] In some embodiments of this application, the aforementioned temporal motion information candidates may include motion information determined based on the temporal motion information of the co-located blocks of the current block.

[0195] In some embodiments of this application, the aforementioned collocated block is a block in the collocated reference image that corresponds spatially to the current block. The motion vector of the collocated block in the collocated reference image is a vector between the collocated reference image col_pic and the reference image col_ref of the collocated block. The motion vector required for the current block is a vector between the current image curr_pic and the reference image curr_ref of the current block. As shown in Figure 9, assuming the POC distance between col_pic and col_ref is td, and the POC distance between curr_pic and curr_ref is tb, if the motion on the collocated block is constant relative to the motion on the current block, then the scaling ratio can be determined based on td and tb. Assuming the motion vector of the co-occurring block is (col_mv_x, col_mv_y), the temporal motion information (tmvp_x, tmvp_y) can be derived as follows: tmvp_x = col_mv_x * tb / td, tmvp_y = col_mv_y * tb / td. The decoder can then use the derived temporal motion information as candidates for temporal motion information.

[0196] In some embodiments of this application, the co-position reference image is a reference image. The current image can generate prediction values ​​by finding reference blocks in the co-position reference image, that is, predicting the current block by finding reference blocks in the co-position reference image to obtain the predicted block of the current block. Furthermore, the co-position reference image retains its own motion information for prediction; this motion information is the motion information of its reference image referenced by a certain block in the co-position reference image when making predictions. The current image can scale the motion information of the co-position blocks in the co-position reference image to obtain temporal motion information.

[0197] In some embodiments of this application, the smallest unit for storing motion information on the co-position reference image is 8x8. That is, each 8x8 sub-block stores a set of motion information. It is understood that, without considering the cost of hardware implementation, the co-position reference image can also use smaller units such as 4x4, or store a set of motion information per pixel.

[0198] For example, as shown in Figure 10, the black block in Figure 10 represents the current block. Positions 1, 2, 3, 4, and 5 represent the spatial positions of adjacent blocks of the current block, while the other horizontal stripe positions represent the spatial positions of non-adjacent blocks of the current block. Position 6 represents the position used for motion information of adjacent blocks of the current block in the temporal domain. If the position corresponding to the lower right corner of the current block is unavailable, the position corresponding to the center of the current block is used. The other vertical stripe positions represent the positions used for motion information of non-adjacent blocks of the current block in the temporal domain. The decoder can determine the temporal motion information based on the motion information at the corresponding positions in the co-located reference image (i.e., the motion information of the co-located blocks).

[0199] In some embodiments of this application, both the temporal motion information candidates and the spatial motion information candidates are related to the positions of the surrounding blocks of the current block. However, the historical motion information candidates are independent of the positions of the surrounding blocks of the current block, and these historical motion information candidates can be obtained from the motion information list stored in the decoder.

[0200] In some embodiments of this application, the motion information list described above is a first-in-first-out motion information list maintained by the decoder. Each time the decoder decodes a block, it can update the motion information list with the motion information of that block, ensuring that there is no duplicate motion information in the list during the update process.

[0201] In some embodiments of this application, the decoder needs to construct a hybrid candidate list each time it decodes a block. When constructing the hybrid candidate list, motion information can be obtained from the above motion information list as candidates for the hybrid candidate list.

[0202] In some embodiments of this application, the aforementioned pairwise average motion information candidates can be paired average candidates. These paired average motion information candidates can also be referred to as synthesized motion information candidates.

[0203] In some embodiments of this application, the decoder can pair candidates of the first type in the mixed candidate list according to a predefined pairing relationship, and determine the pairwise average motion information candidate based on each pair of candidates of the first type. The predefined pairing relationship can be {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the numbers represent the index values ​​of the candidates of the first type in the mixed candidate list.

[0204] For example, assuming the predefined pairing relationship is {(0, 1), (0, 2), (1, 2)}, the decoder can form a first pair with the candidate of the first type at index 0 and the candidate of the first type at index 1 in the mixed candidate list; a second pair with the candidate of the first type at index 0 and the candidate of the first type at index 2 in the mixed candidate list; and a third pair with the candidate of the first type at index 1 and the candidate of the first type at index 2 in the mixed candidate list. Then, the average motion vector of the two candidates in the first pair is calculated to obtain a motion vector. Next, a reference image index is determined based on the reference image index of the two candidates in the first pair. Then, a flag indicating whether to use the reference image list is determined according to the reference image list indicated by the reference image index. The determined motion vector, reference image index, and flag constitute new motion information, which is used as the candidate for pairwise average motion information. Similarly, the candidate for pairwise average motion information can be calculated based on the second and third pairs. To avoid repetition, this will not be elaborated here.

[0205] In some embodiments of this application, the second type of candidate described above may include one or more of affine motion information candidates and sub-block-based temporal motion information candidates.

[0206] In some embodiments of this application, the aforementioned affine motion information candidates can be affine motion information candidates, and the aforementioned sub-block-based temporal motion information candidates can be sbtmvp candidates.

[0207] In other words, the candidates of the second type mentioned above may include affine motion information candidates, or the candidates of the second type mentioned above may include sub-block-based temporal motion information candidates, or the candidates of the second type mentioned above may include affine motion information candidates and sub-block-based temporal motion information candidates.

[0208] In some embodiments of this application, the aforementioned affine motion information candidates may be referred to as affine candidates or affine motion candidates. These affine motion information candidates can be understood as candidates determined based on the motion information of the surrounding blocks when the current block is predicted using an affine motion model.

[0209] In some embodiments of this application, the above-mentioned candidate temporal motion information based on sub-blocks can be understood as: when the surrounding blocks of the current block are predicted by the motion model based on sub-blocks, the candidates are determined based on the motion information of the surrounding blocks.

[0210] In some embodiments of this application, the aforementioned sub-block-based temporal motion information candidates can be candidates at preset positions in a mixed candidate list.

[0211] In one example, the preset position can be any position in the mixed candidate list.

[0212] In some embodiments of this application, the candidate at the preset position can be the first candidate in the mixed candidate list.

[0213] In other words, the decoder can add sub-block-based temporal motion information candidates to the first position in the hybrid candidate list.

[0214] Thus, when the decoder constructs the hybrid candidate list, translational candidates, affine candidates, and sub-block-based temporal motion information candidates can all be filled into the hybrid candidate list, and sub-block-based temporal motion information candidates can be added to a preset position. In this way, as long as the index points to a candidate at a preset position in the hybrid candidate list, it can be determined that the candidate is a sub-block-based temporal motion information candidate.

[0215] In some embodiments of this application, the principle for constructing the hybrid candidate list is based on any surrounding blocks of the current block. That is, when the decoder constructs the hybrid candidate list, it finds any surrounding block of the current block and constructs the hybrid candidate list based on the motion information of that surrounding block, instead of determining whether the predicted motion model of the surrounding block is a specific motion model when a surrounding block is found, and only using the motion information of the surrounding block to construct the candidate list if the predicted motion model of the surrounding block is a specific motion model.

[0216] In one example, any of the aforementioned surrounding blocks can be understood as surrounding blocks where motion information is available.

[0217] In this way, when constructing the hybrid candidate list, the decoder does not need to parse a flag from the bitstream to determine which type of merged candidate list to construct. Instead, the decoder constructs the list based on the motion information of any surrounding blocks of the current block, saving the transmission resources of transmitting the flag in the bitstream.

[0218] In some embodiments of this application, the specific implementation of the decoder constructing the hybrid candidate list may include the following steps 404 and 405.

[0219] Step 404: The decoder determines the motion model of the surrounding blocks of the current block for prediction.

[0220] In some embodiments of this application, the surrounding blocks of the current block are decoded blocks.

[0221] In some embodiments of this application, the motion model for predicting the surrounding blocks of the current block refers to the motion model used when predicting the surrounding blocks of the current block to obtain the predicted blocks of the surrounding blocks.

[0222] For example, the motion model for predicting the surrounding blocks of the current block can be a translational motion model, an affine motion model, or a sub-block-based motion model.

[0223] Step 405: The decoder uses a motion model based on surrounding blocks to predict motion and constructs a hybrid candidate list.

[0224] In some embodiments of this application, the decoder can make a motion model based on the surrounding blocks to determine the motion information that can be used as a candidate, and add the motion information as a candidate to the mixed candidate list.

[0225] In some embodiments of this application, when constructing a hybrid candidate list, the decoder can add motion information determined based on various motion models as candidates to the hybrid candidate list, instead of adding motion information determined based on different motion models as candidates to different candidate lists (such as the above-mentioned merged candidate list or the above-mentioned sub-block merged candidate list), and instead of searching for surrounding blocks predicted using a specific motion model in the surrounding blocks of the current block, and then constructing the above-mentioned merged candidate list or the above-mentioned sub-block merged candidate list based on the motion information of the surrounding blocks.

[0226] It should be noted that steps 404 and 405 can be performed before step 402.

[0227] In some embodiments of this application, step 405 can be implemented by step 4051 or step 4052 as described below.

[0228] Step 4051: If the surrounding blocks of the current block are predicted based on the translation motion model, the decoder constructs a hybrid candidate list based on the translation motion information of the surrounding blocks.

[0229] The surrounding blocks of the current block are predicted based on the translational motion model. This can be understood as the translational motion model being used when predicting the surrounding blocks of the current block.

[0230] In some embodiments of this application, if the surrounding blocks of the current block are predicted based on a translational motion model, the decoder can use the translational motion information of the surrounding blocks as a candidate in the hybrid candidate list, i.e., a candidate of the first type. Alternatively, the decoder can use motion information identical to the translational motion information of the surrounding blocks as a candidate in the hybrid candidate list, i.e., a candidate of the first type.

[0231] In some embodiments of this application, the decoder may use at least one of the following as candidates of the first type in the mixed candidate list: translation motion information of adjacent blocks in the spatial domain, translation motion information of non-adjacent blocks in the spatial domain, translation motion information of adjacent blocks in the temporal domain, and translation motion information of non-adjacent blocks in the temporal domain.

[0232] For example, the decoder can use the motion information of the current block's neighboring blocks in the spatial domain as spatial motion information candidates in the mixing candidate list. Also for example, the decoder can use the motion information of both the current block's neighboring blocks in the spatial domain and the motion information of the current block's non-neighboring blocks in the spatial domain as spatial motion information candidates in the mixing candidate list. Furthermore, for example, the decoder can use the motion information of the current block's neighboring blocks in the temporal domain as temporal motion information candidates in the mixing candidate list.

[0233] In some embodiments of this application, the translational motion information may include a motion vector (mvLX), a reference image index (refIdxLX), and a flag indicating whether a list of reference images is used (predFlagLX).

[0234] In some embodiments of this application, for a predicted reference direction or a list of reference images, the aforementioned translational motion information can be represented by a motion vector. Therefore, when the current block corresponds to two list of reference images RPL0 and RPL1, list RPL0 corresponds to a flag predFlagL0, a reference image index refIdxL0, and a motion vector mvL0. The flag predFlagL0 indicates whether to use list RPL0, and the reference image index refIdxL0 indicates which image in list RPL0 is used. List RPL1 corresponds to a flag predFlagL1, a reference image index refIdxL1, and a motion vector mvL1. The flag predFlagL1 indicates whether to use list RPL1, and the reference image index refIdxL1 indicates which image in list RPL1 is used.

[0235] Understandably, if the system supports a larger list of reference images, the translational motion information can be increased accordingly. Furthermore, since translational motion is processed on a block-by-block basis during motion compensation, it can be understood that the motion vectors of all sub-blocks or points within the current block are the same.

[0236] Step 4052: If the surrounding blocks of the current block are predicted based on the affine motion model, the decoder constructs a hybrid candidate list based on the affine motion information of the surrounding blocks.

[0237] Typically, motion involves more than just translation; it takes many forms, such as shrinking, magnification, rotation, perspective motion (where objects closer to the camera appear larger and objects farther away appear smaller), and many irregular motion patterns. Therefore, if the surrounding blocks of the current block are predicted based on an affine motion model, the decoder can construct a hybrid candidate list based on the affine motion information of the surrounding blocks. This affine motion information represents affine motion, which can be used to represent the more complex motions mentioned above, in addition to translation.

[0238] In some embodiments of this application, if the surrounding blocks of the current block are predicted based on an affine motion model, the decoder can construct a hybrid candidate list based on the derived affine motion information, which can be derived from the affine motion information of the surrounding blocks. In other words, when the decoder constructs the hybrid candidate list, if the surrounding blocks of the current block are predicted based on an affine motion model, the affine motion information is derived based on the affine motion information of the surrounding blocks, and then the derived affine motion information is used as an affine candidate in the hybrid candidate list.

[0239] In some embodiments of this application, the affine motion information derived above may include a motion vector (cpMvLX), a reference image index (refIdxLX), and a flag indicating whether a list of reference images is used (predFlagLX).

[0240] It should be noted that the process of deriving affine motion information based on the affine motion information of the surrounding blocks can be found in the following description.

[0241] In one example, as shown in Figure 11, the decoder can calculate the motion vector of each sub-block or each pixel in the current block using a linear model based on the motion vector of 2 control points (4 parameters) or 3 control points (6 parameters). The 2 control points can be the control points at the top left and top right corners of the current block in Figure 11, and the 3 control points can be the control points at the top left, top right, and bottom left corners of the current block in Figure 11.

[0242] For a 4-parameter affine model, the motion vector at the current block position (x, y) can be derived according to the following formula (1):

[0243] For a 6-parameter affine model, the motion vector at the current block position (x, y) can be derived using the following formula (2):

[0244] in, This is the motion vector of the control point at the top left corner of the current block. This is the motion vector of the control point at the top right corner of the current block. is the motion vector of the control point at the lower left corner of the current block, W is the width of the current block, H is the height of the current block, and the superscripts h and v represent the horizontal and vertical components of mv, respectively.

[0245] In one example, as shown in Figure 12, which is a schematic diagram of Affine deriving motion vectors based on sub-blocks, the decoder can divide the current block into 4x4 sub-blocks and calculate a motion vector MV for each sub-block based on the motion vectors of the control points at the top left and top right corners of the current block, and perform motion compensation.

[0246] Understandably, with the improvement of hardware processing power, Affine can also achieve pixel-based processing. That is, derive a motion vector for each pixel, and perform motion compensation for a pixel based on the motion vector.

[0247] Thus, Affine only needs a few control points to derive the motion vector for each sub-block or each pixel, enabling more precise predictions compared to block-based motion compensation. Furthermore, Affine significantly reduces overhead compared to dividing the CU into smaller, more manageable units.

[0248] Understandably, affine mapping uses control points, whether two or three, to derive the motion vector for each sub-block or pixel using a single model. More generally, with the top-left corner of the current block at (0, 0), the motion vector at position (x, y) within the current block can be derived using the following formula (3):

[0249] By using formula (3), a set of values ​​for a, b, c, and d, as well as the mv of the upper left corner control point, can be determined, and the motion vector of each sub-block or each pixel can be derived.

[0250] For the above 4-parameter affine model

[0251] For the above 6-parameter affine model

[0252] More generally, once the values ​​of a, b, c, and d are determined, as well as the mv of a certain point, the motion vector of each sub-block or each pixel can be derived. That is, the motion vector of the (x, y) position within the current block can be derived according to the following formula (6):

[0253] In formula (6), the coordinates of the reference point are (x... base y baseThe reference point's mv is mv base The reference point can be any position within the current block, or even outside the current block.

[0254] It should be noted that a, b, c and d above can be represented by the variables dHorX, dVerX, dHorY and dVerY respectively.

[0255] Affine also has a merge mode. Similar to the merge mode for translation, in affine merge mode, the decoder can construct the affine model of the current block based on the affine models of the surrounding blocks.

[0256] For example, as shown in Figure 13, A0, A1, B0, B1, and B2 in Figure 13 are the neighboring blocks of the current block. The decoder checks whether the neighboring blocks of the current block use affine prediction mode. If a neighboring block uses affine prediction mode, the affine model of the current block can be determined based on the affine model of that neighboring block.

[0257] In one example, the decoder can deduce the motion vector of the current block's control point based on the motion vector (CPMV) of the control point of the surrounding blocks. In other words, the current block can inherit the affine model of the surrounding blocks.

[0258] It is understandable that, assuming the current block and the adjacent blocks are in the same affine motion model, using the above formulas (1)-(6) or their variations, the coordinate calculation is not limited to the same block. The decoder can determine the motion vector of the control point of the current block based on the motion vector of the control point of the adjacent block.

[0259] In one example, the decoder can construct the affine model of the current block based on the affine models of its neighboring blocks.

[0260] In the specific implementation, as shown in Figure 14, A0, A1, A2, B0, B1, B2, and B3 in Figure 14 are the surrounding neighboring blocks of the current block. The decoder can determine CPMV0 using the available motion vectors in B2, B3, and A2; CPMV1 using the available motion vectors in B0 and B1; CPMV2 using the available motion vectors in A0 and A1; and CPMV3 using the available motion vectors in T. Here, T represents the temporal motion information determined based on the position of the lower right corner of the current block (i.e., the position of the upper left corner of the current block in the image is (xCb, yCb), and the width of the current block is W and the height is H, so the position of the lower right corner of the current block is (xCb+W, yCb+H)). This temporal motion information is obtained by scaling the motion information used by the block containing the position (xCb+W, yCb+H) in the co-located reference image. The decoder can construct an affine model of the current block based on a combination of some of these candidate CPMVs. For example:

[0261] {CPMV0, CPMV1, CPMV2}

[0262] {CPMV0, CPMV1, CPMV3}

[0263] {CPMV0, CPMV2, CPMV3}

[0264] {CPMV1, CPMV2, CPMV3}

[0265] {CPMV0, CPMV1},

[0266] {CPMV0, CPMV2}.

[0267] The first four groups of three control points can construct a 6-parameter affine model, while the last two groups of two control points can construct a 4-parameter affine model. In one example, the 6-parameter affine model can be represented by {CPMV0, CPMV1, CPMV2}, and the other three cases can be constructed as {CPMV0, CPMV1, CPMV2}. The 4-parameter affine model can be represented by {CPMV0, CPMV1}, and {CPMV0, CPMV2} can be used to construct {CPMV0, CPMV1}.

[0268] Let the motion vector of CPMVidx be Where idx can be 0, 1, 2, or 3. The method for constructing a 6-parameter affine model is as follows:

[0269] Construct {CPMV0, CPMV1, CPMV3} from {CPMV0, CPMV1, CPMV2}, where CPMV2 = CPMV3 + CPMV0 - CPMV1.

[0270] Construct {CPMV0, CPMV1, CPMV2} using {CPMV0, CPMV2, CPMV3}, where CPMV1 = CPMV3 + CPMV0 - CPMV2.

[0271] Construct {CPMV0, CPMV1, CPMV2} from {CPMV1, CPMV2, CPMV3}, where CPMV0 = CPMV1 + CPMV2 - CPMV3.

[0272] The method for constructing a 4-parameter affine model is as follows:

[0273] Construct {CPMV0, CPMV1} using {CPMV0, CPMV2}, where

[0274] It should be noted that the above derivation process is obtained by substituting the known control points into formula (1) and formula (2). In the process of substituting the known control points into formula (1) and formula (2) for calculation, let the upper left control point correspond to CPMV0, and the coordinates of the upper left control point are (0, 0). Let the upper right control point correspond to CPMV1, and the coordinates of the upper right control point are (W, 0). Let the lower left control point correspond to CPMV2, and the coordinates of the lower left control point are (0, H). Let the lower right control point correspond to CPMV3, and the coordinates of the lower right control point are (W, H).

[0275] Thus, if the surrounding blocks of the current block are predicted based on the affine motion model, the decoder can derive the affine model of the current block based on the affine model of the surrounding blocks of the current block through the above derivation process. The affine model of the current block can include motion vectors, which are the motion vectors included in the affine motion information obtained above.

[0276] In some embodiments of this application, the reference image index included in the affine motion information derived above can be determined based on the reference image index of the reference images of the surrounding blocks of the current block, and the flag indicating whether to use the reference image list included in the affine motion information derived above can be determined based on the reference image list in which the reference image indicated by the reference image index is located.

[0277] In some embodiments of this application, the affine motion model may include a 4-parameter affine motion model and a 6-parameter affine motion model, and the affine candidate may also include a 4-parameter affine candidate and a 6-parameter affine candidate.

[0278] It's important to note that Affine currently processes motion compensation based on sub-blocks, but in the future, it may be based on points or pixels. Therefore, when performing motion compensation or preparing for it, the two representations mentioned above can be transformed into the mv matrix for each sub-block, point, or pixel. If these mv matrices are combined, they form a single mv matrix. Thus, for motion compensation, the Affine model can be represented by an mv matrix.

[0279] In some embodiments of this application, step 405 can be implemented by step 4053 or step 4054 as described below.

[0280] Step 4053: The decoder predicts the motion model based on the first surrounding block of the current block as a translational motion model, and adds the translational motion information of the first surrounding block as a first type of candidate to the mixed candidate list.

[0281] The first surrounding block can be any surrounding block of the current block.

[0282] In some embodiments of this application, during the process of the decoder constructing the hybrid candidate list, for the first surrounding block of the current block, if the predicted motion model of the first surrounding block is a translational motion model, the decoder adds the translational motion information of the first surrounding block as a first type of candidate to the hybrid candidate list. In other words, the decoder adds the translational motion information of the first surrounding block as a translational candidate to the hybrid candidate list.

[0283] Step 4054: The decoder predicts the motion model based on the second surrounding block of the current block as an affine motion model, and adds the affine motion information derived from the affine motion information of the second surrounding block as a second type of candidate to the mixed candidate list.

[0284] The aforementioned second surrounding block can be any surrounding block of the current block.

[0285] In some embodiments of this application, during the process of the decoder constructing the hybrid candidate list, for the second surrounding block of the current block, if the predicted motion model of the second surrounding block is an affine motion model, the decoder adds the affine motion information derived from the affine motion information of the second surrounding block as a second type of candidate to the hybrid candidate list. In other words, the decoder adds the affine motion information derived from the affine motion information of the second surrounding block as an affine motion information candidate to the hybrid candidate list.

[0286] In some embodiments of this application, the second type of candidates includes sub-block-based temporal motion information candidates. When the decoder constructs the hybrid candidate list, if the motion model predicted by the surrounding blocks of the current block is based on the sub-block motion model, the decoder adds the motion information determined based on the motion information of the surrounding blocks as a sub-block-based temporal motion information candidate to the hybrid candidate list.

[0287] In the specific implementation, during the process of the decoder constructing the hybrid candidate list, for any surrounding block of the current block, if the motion model predicted by the surrounding block is a sub-block-based motion model, the decoder can add the motion information determined based on the motion information of the surrounding block as a candidate for sub-block-based temporal motion information to the hybrid candidate list.

[0288] In some embodiments of this application, sbtmvp is based on sub-blocks, so sbtmvp can obtain an MVP for each sub-block. In other words, the MVPs of all sub-blocks may not be the same. Furthermore, sbtmvp can determine the position by finding a motion offset based on the motion of the surrounding blocks. For example, as shown in Figure 15, if the block at position A1 references a co-location reference image, then the motion displacement can be set to the motion vector of A1 using the co-location reference image, i.e., the motion vector of A1 relative to A1'. Otherwise, the motion displacement is set to (0, 0). Then, the position is found according to the motion displacement, and the MV corresponding to the position of each sub-block in the "co-location block" is scaled to obtain the MVP of each sub-block.

[0289] In some embodiments of this application, the candidates for sbtmvp may include, in addition to motion vectors, reference image indexes and flags indicating whether a particular list of reference images is used. The reference image indexes may be uniform across the entire block, or a matrix may represent the reference image indexes for each sub-block or location.

[0290] In some embodiments of this application, if a block uses sbtmvp, then the motion within the block can be represented by multiple motion vectors. A candidate sbtmvp can be included in the hybrid candidate list.

[0291] In some embodiments of this application, when SBTMVP performs motion compensation, each sub-block corresponds to a motion value (MV), and the MVs of multiple sub-blocks form an MV matrix. It is understood that, if hardware conditions permit, each position can also correspond to an MV.

[0292] Typically, the decoder needs to construct a merge candidate list mergeCandList or a subblock merge candidate list subblockMergeCandList based on the flags in the bitstream.

[0293] In Example 1 of the relevant technology, the mergeCandList can be constructed as follows:

[0294] i=0

[0295] if(availableFlagB 1)

[0296] mergeCandList[i] = B1

[0297] i = i + 1

[0298] if(availableFlagA1)

[0299] mergeCandList[i] = A1

[0300] i = i + 1

[0301] if(availableFlagB0)

[0302] mergeCandList[i] = B0

[0303] i = i + 1

[0304] if(availableFlagA0)

[0305] mergeCandList[i] = A0

[0306] i = i + 1

[0307] if(availableFlagB2)

[0308] mergeCandList[i] = B2

[0309] i = i + 1

[0310] if(availableFlagCol)

[0311] mergeCandList[i] = Col

[0312] i = i + 1

[0313] If mergeCandList is not full, you can add constructed candidates or default candidates.

[0314] Here, B1, A1, B0, A0, B2, and Col can all be understood as codes or indexes. B1, A1, B0, A0, and B2 represent candidates for the corresponding positions (i.e., spatial motion information candidates), while Col represents temporal motion information candidates.

[0315] In Example 1 of the related technology, when the decoder parses the flag and determines that a merge candidate list needs to be constructed, only the motion information of the translational motion is added to the merge candidate list. This merge candidate list cannot include affine candidates and sub-block-based temporal motion information candidates.

[0316] In Example 2 of the relevant technology, the subblock merge candidate list subblockMergeCandList can be constructed as follows:

[0317] i=0

[0318] if(availableFlagSbCol)

[0319] subblockMergeCandList[i++]=SbCol

[0320] if(availableFlagA&&i <MaxNumSubblockMergeCand)

[0321] subblockMergeCandList[i++]=A

[0322] if(availableFlagB&&i <MaxNumSubblockMergeCand)

[0323] subblockMergeCandList[i++]=B

[0324] if(availableFlagConst1&&i <MaxNumSubblockMergeCand)

[0325] subblockMergeCandList[i++]=Const1

[0326] if(availableFlagConst2&&i <MaxNumSubblockMergeCand)

[0327] subblockMergeCandList[i++]=Const2

[0328] if(availableFlagConst3&&i <MaxNumSubblockMergeCand)

[0329] subblockMergeCandList[i++]=Const3

[0330] if(availableFlagConst4&&i <MaxNumSubblockMergeCand)

[0331] subblockMergeCandList[i++]=Const4

[0332] if(availableFlagConst5&&i <MaxNumSubblockMergeCand)

[0333] subblockMergeCandList[i++]=Const5

[0334] if(availableFlagConst6&&i <MaxNumSubblockMergeCand)

[0335] subblockMergeCandList[i++]=Const6

[0336] If subblockMergeCandList is not full, constructed candidates or default candidates can be added.

[0337] Similar to mergeCandList, subblockMergeCandList also uses designations like SbCol, A, B, Const1, etc. SbCol represents a candidate for sbtmvp, while the others are candidates for affine. Unlike mergeCandList, subblockMergeCandList actually contains two different types of candidates.

[0338] In Example 2 of the related technology, when the decoder parses the flag and determines that a sub-block merging candidate list needs to be constructed, only the motion information of affine motion or the temporal motion information based on the sub-block is added to the sub-block merging candidate list, and the sub-block merging candidate list cannot include translation candidates.

[0339] Thus, in this embodiment of the application, compared to the candidate list construction methods of Examples 1 and 2 above, when the decoder constructs the hybrid candidate list, if the motion model predicted by the surrounding blocks of the current block is a translational motion model, the decoder can use the translational motion information of the surrounding blocks as translational candidates in the hybrid candidate list, or in other words, use the motion information that is the same as the translational motion information of the surrounding blocks as translational candidates in the hybrid candidate list. If the motion model predicted by the surrounding blocks of the current block is an affine motion model, the decoder can use the motion information derived from the affine motion information of the surrounding blocks as affine candidates in the hybrid candidate list. If the motion model predicted by the surrounding blocks of the current block is a sub-block-based motion model, the decoder can use the motion information derived from the motion information of the surrounding blocks as sub-block-based temporal domain motion information candidates in the hybrid candidate list. In other words, when constructing the hybrid candidate list, if a neighboring block of the current block is found, regardless of the type of motion model predicted by that neighboring block, the decoder will determine the motion information that can be used as a candidate based on the motion information of that neighboring block, and include the determined motion information as a candidate in the hybrid candidate list. That is, whichever motion information is determined based on the motion information of the neighboring block is used as a candidate. The hybrid candidate list can include translational candidates, affine candidates, and temporal motion information candidates based on sub-blocks. Therefore, unlike the methods of constructing candidate lists in Examples 1 and 2 above, the method of constructing the hybrid candidate list in this embodiment does not rely on the indication of the flag. Therefore, the flag does not need to be transmitted in the bitstream, which can reduce the waste of transmission resources and improve encoding efficiency.

[0340] It should be noted that, under certain special circumstances, if the same motion model is used to predict multiple surrounding blocks of the current block, the constructed hybrid candidate list may include one type of candidate. However, this does not mean that the hybrid candidate list cannot include other types of candidates. Because if multiple motion models are used to predict the surrounding blocks of the current block, the constructed hybrid candidate list can include other types of candidates. Therefore, even if the hybrid candidate list constructed by the decoder includes one type of candidate, as long as the hybrid candidate list is constructed in accordance with the method of constructing a hybrid list mentioned in the embodiments of this application, then the hybrid candidate list has the ability to include different types of candidates. This hybrid candidate list is different from the merged candidate list in Example 1 and the sub-block merged candidate list in Example 2.

[0341] In some embodiments of this application, when the decoder constructs the hybrid candidate list, candidates of the first type (translation candidates) and candidates of the second type (affine candidates and sub-block-based temporal motion information candidates) are added to the hybrid candidate list. However, since it is impossible to predict whether each candidate in the hybrid candidate list is a translation candidate, an affine candidate, or a sub-block-based temporal motion information candidate, a variable can be used to indicate the motion model or type of the candidate.

[0342] In some embodiments of this application, the motion model corresponding to the candidates in the above-mentioned mixed candidate list can be indicated by a motion model variable. This motion model variable can be referred to as motionModelIdc.

[0343] It should be noted that the candidates in the above mixing list refer to any one candidate in the mixing list. The motion model corresponding to the above candidate refers to the motion model used to predict the surrounding blocks corresponding to the candidate, that is, the motion model used to predict the surrounding blocks corresponding to the motion vectors in the candidate.

[0344] In some embodiments of this application, the motion model described above may include at least one of the following: a translational motion model, an affine motion model, and a sub-block-based motion model. The affine motion model may include a 4-parameter affine motion model and a 6-parameter affine motion model.

[0345] For example, if a candidate is a candidate of the first type, and the motion model corresponding to that candidate is a translational motion model, the value of motionModelIdc can be set to 0. If a candidate is an affine candidate, and the motion model corresponding to that candidate is a 4-parameter affine motion model, the value of motionModelIdc can be set to 1. If a candidate is an affine candidate, and the motion model corresponding to that candidate is a 6-parameter affine motion model, the value of motionModelIdc can be set to 2. If a candidate is a candidate based on sub-block temporal motion information, that is, the motion model corresponding to that candidate is a sub-block motion model, the value of motionModelIdc can be set to 3.

[0346] In some embodiments of this application, the above-mentioned motion model variables can be elements in a motion model array, and the elements in the motion model array correspond one-to-one with the candidates in the above-mentioned mixed candidate list.

[0347] In other words, the decoder can construct a motion model array during the process of building the hybrid candidate list. The elements in the motion model array are motion model variables, and each element in the motion model array corresponds one-to-one with a candidate in the hybrid candidate list.

[0348] In one example, the mixed candidate list is represented by `mixedCandList`, and the above array of motion models can be represented by `mixedCandListMotionModelIdc[]`. The constructed mixed candidate list is then shown in representation one.

[0349] Method 1:

[0350] Here, B1, A1, B0, A0, and B2 are candidates in the mixed candidate list. Assuming B1 is a translation candidate, meaning that the motion information of B1 is obtained based on the translation motion model, then the value of motionModelIdc B1 is 0. Assuming A1 is a 4-parameter affine candidate, meaning that the motion information of A1 is obtained based on the 4-parameter affine motion model, then the value of motionModelIdc A1 is 1.

[0351] Thus, when the decoder uses the candidate with index mix_idx in the mixed candidate list, it can distinguish the processing method of the candidate indicated by mix_idx based on the value of mixedCandListMotionModelIdc[mix_idx], that is, whether it is processed by translational motion, affine motion, or sub-block-based motion.

[0352] In some embodiments of this application, the aforementioned hybrid candidate list may include a first list and a second list, wherein the first list may include motion model variables and the second list may include candidates corresponding to the motion models indicated by the motion model variables.

[0353] In other words, the mixed candidate list includes a first list and a second list. The first list includes multiple motion model variables, and the number of motion model variables in the first list is the same as the number of candidates in the second list. Moreover, the motion model variables in the first list correspond one-to-one with the candidates in the second list.

[0354] In one example, if the motion model corresponding to a candidate is a translational motion model, then the value of the motion model variable corresponding to that candidate is set to 0; if the motion model corresponding to a candidate is a 4-parameter affine motion model, then the value of the motion model variable corresponding to that candidate is set to 1; and if the motion model corresponding to a candidate is a 6-parameter affine motion model, then the value of the motion model variable corresponding to that candidate is set to 2.

[0355] For example, suppose the first list includes three motion model variables: `motionModelIdc A0`, `motionModelIdc B0`, and `motionModelIdc C0`. The second list includes three candidates: a translation candidate A0, a 4-parameter affine candidate B0, and a 6-parameter affine candidate C0, and these three candidates correspond one-to-one with the three motion model variables. The motion model corresponding to translation candidate A0 is a translation motion model, so the value of `motionModelIdc A0` is 0. The motion model corresponding to the 4-parameter affine candidate B0 is a 4-parameter affine motion model, so the value of `motionModelIdc B0` is 1. The motion model corresponding to the 6-parameter affine candidate C0 is a 6-parameter affine motion model, so the value of `motionModelIdc C0` is 2.

[0356] In this way, by adding the motion model variables to the mixed candidate list in the form of a list, the decoder can determine the candidates and their corresponding motion models by parsing the mixed candidate list, so as to perform subsequent processing on the candidates according to the motion models.

[0357] In some embodiments of this application, the candidates in the above-mentioned mixed candidate list may include motion model variables.

[0358] It is understandable that the above motion model variables can be attribute information of candidates in the mixed candidate list, or that the above motion model variables can be included in each candidate along with other motion information, or that the above motion model variables can be part of the motion information corresponding to each candidate.

[0359] In some embodiments of this application, motionModelIdc can be used as attribute information of candidates in the mixed candidate list, that is, the candidates in the mixed candidate list include motionModelIdc. In other words, whether it is a candidate of the first type or a candidate of the second type, a candidate can include motionModelIdc in addition to mvLX, predFlagLX, refIdxLX (where X represents RPL0 or RPL1, and affine motion has multiple mvLX, such as an array cpMvLX[3]). The constructed mixed candidate list is shown in representation method two.

[0360] Method 2:

[0361] i=0

[0362] if(availableFlagB1)

[0363] mixedCandList[i] = B1

[0364] i = i + 1;

[0365] if(availableFlagA1)

[0366] mixedCandList[i] = A1

[0367] i = i + 1;

[0368] if(availableFlagB0)

[0369] mixedCandList[i] = B0

[0370] i = i + 1;

[0371] if(availableFlagA0)

[0372] mixedCandList[i] = A0

[0373] i = i + 1;

[0374] if(availableFlagB2)

[0375] mixedCandList[i] = B2

[0376] i = i + 1;

[0377] Among them, B1, A1, B0, A0 and B2 are candidates in the mixed candidate list.

[0378] In some embodiments of this application, when the decoder uses a candidate with index mix_idx in the mixed candidate list, it can parse the candidate with index mix_idx to obtain motionModelIdc, and then distinguish the processing method of the candidate indicated by mix_idx based on the value of motionModelIdc, that is, whether it is processed by translational motion, or by affine motion, or by sub-block-based motion.

[0379] In some embodiments of this application, representation method two is essentially the same as representation method one, only the representation form is different. In representation method two and representation method one, the data structures for translational candidates, affine candidates, and sub-block-based temporal motion information candidates are different.

[0380] It should be noted that B1, A1, B0, A0, and B2 in the above embodiments are merely examples and are not limited to these positions or this order. In addition to the motion information of adjacent positions in the examples above, motion information of non-adjacent positions, temporal motion information, historical motion information (such as HMVP, History-based Motion Vector Prediction), and constructed motion information (such as Const1 to Const6) can also be used.

[0381] In some embodiments of this application, the decoder can use a data structure that can represent both a first type of candidate and a second type of candidate to represent candidates in a mixed candidate list, and the value of the motion model variable in the candidate indicates the motion model corresponding to the candidate or the type of the candidate, that is, the value of the motion model variable in the candidate indicates whether the candidate is a first type of candidate or a second type of candidate.

[0382] In some embodiments of this application, the first type of candidate is a translation candidate, and the second type of candidate includes affine candidates. The decoder can represent candidates in the mixed candidate list using a data structure that can represent both translation and affine candidates, and the value of `motionModelIdc` in the candidate indicates the motion model corresponding to the candidate or the type of the candidate. For example, the value of `motionModelIdc` in a candidate can indicate whether the candidate is a translation candidate or an affine candidate. Specifically, it can be used to indicate whether the affine candidate is a 4-parameter affine candidate or a 6-parameter affine candidate.

[0383] In one example, the candidates represented by the above data structure include both mvLX and cpMvLX. If the value of motionModelIdc in a candidate is 0, then mvLX is valid, indicating that the candidate is a translation candidate; if the value of motionModelIdc in a candidate is 1, then the first two mv values ​​of cpMvLX are valid, indicating that the candidate is a 4-parameter affine candidate; if the value of motionModelIdc in a candidate is 2, then all three mv values ​​of cpMvLX are valid, indicating that the candidate is a 6-parameter affine candidate.

[0384] In another example, the candidates represented by the above data structure include cpMvLX. If the value of motionModelIdc in a candidate is 0, then the first mv of cpMvLX is valid, indicating that the candidate is a translation candidate; if the value of motionModelIdc in a candidate is 1, then the first two mv of cpMvLX are valid, indicating that the candidate is a 4-parameter affine candidate; if the value of motionModelIdc in a candidate is 2, then all three mv of cpMvLX are valid, indicating that the candidate is a 6-parameter affine candidate.

[0385] In this embodiment, when the decoder constructs the hybrid candidate list, translational and affine candidates are filled into a single hybrid candidate list. Different data structures can be used to represent translational and affine candidates, or the same data structure can be used. Furthermore, regardless of whether the same or different data structures are used, the type of candidate can be distinguished by the value of the motion model variable, i.e., whether a candidate is a translational or affine candidate. This motion model variable can be an element in the motion model array that corresponds one-to-one with the candidates in the hybrid candidate list, or it can be included in the candidates in the hybrid candidate list. Thus, the decoder can construct a hybrid candidate list that includes both translational and affine candidates without requiring the encoder to use a flag indication, and can distinguish different types of candidates by the value of `motionModelIdc`, reducing the amount of data transmitted by the encoder and thus reducing the waste of transmission resources.

[0386] In some embodiments of this application, the first type of candidate is a translation candidate, and the second type of candidate includes affine candidates and sub-block-based temporal motion information candidates. The decoder can use a data structure that can represent translation candidates, affine candidates, and sub-block-based temporal motion information candidates to represent candidates in the mixed candidate list, and the value of motionModelIdc in the candidate indicates the motion model corresponding to the candidate or the type of the candidate. For example, the value of motionModelIdc in the candidate can indicate whether the candidate is a translation candidate, an affine candidate, or a sub-block-based candidate. Specifically, it can be used to indicate whether the affine candidate is a 4-parameter affine candidate or a 6-parameter affine candidate.

[0387] In one example, the candidates represented by the above data structure include mvLX, cpMvLX, and sbtmvpLX. If the value of motionModelIdc in a candidate is 0, then mvLX is valid, indicating that the candidate is a translation candidate; if the value of motionModelIdc in a candidate is 1, then the first two mv values ​​of cpMvLX are valid, indicating that the candidate is a 4-parameter affine candidate; if the value of motionModelIdc in a candidate is 2, then all three mv values ​​of cpMvLX are valid, indicating that the candidate is a 6-parameter affine candidate; if the value of motionModelIdc in a candidate is 3, then the mv values ​​of sbtmvpLX are valid, indicating that the candidate is a sub-block-based temporal motion information candidate.

[0388] In another example, the candidates represented by the above data structure include both mvLX and cpMvLX. If the value of motionModelIdc in a candidate is 0, then mvLX is valid, indicating that the candidate is a temporal motion information candidate based on sub-blocks; if the value of motionModelIdc in a candidate is 1, then the first mv of cpMvLX is valid, indicating that the candidate is a translation candidate; if the value of motionModelIdc in a candidate is 2, then the first two mvs of cpMvLX are valid, indicating that the candidate is a 4-parameter affine candidate; if the value of motionModelIdc in a candidate is 3, then all three mvs of cpMvLX are valid, indicating that the candidate is a 6-parameter affine candidate.

[0389] In another example, the candidates represented by the above data structure include both mvLX and cpMvLX. If the value of motionModelIdc in a candidate is 0, then mvLX is valid, indicating that the candidate is a translation candidate; if the value of motionModelIdc in a candidate is 1, then the first mv of cpMvLX is valid, indicating that the candidate is a temporal motion information candidate based on sub-blocks; if the value of motionModelIdc in a candidate is 2, then the first two mv of cpMvLX are valid, indicating that the candidate is a 4-parameter affine candidate; if the value of motionModelIdc in a candidate is 3, then all three mv of cpMvLX are valid, indicating that the candidate is a 6-parameter affine candidate.

[0390] In this embodiment, when the decoder constructs the hybrid candidate list, translational candidates, affine candidates, and sub-block-based temporal motion information candidates are filled into a single hybrid candidate list. Different data structures can be used to represent these candidates, or the same data structure can be used. Furthermore, regardless of whether the same or different data structures are used, the type of candidate can be distinguished by the value of the motion model variable `motionModelIdc`, indicating whether a candidate is a translational candidate, an affine candidate, or a sub-block-based temporal motion information candidate. `motionModelIdc` can be an element in the motion model array that corresponds one-to-one with the candidates in the hybrid candidate list, or it can be included in the candidates in the hybrid candidate list. Thus, the decoder can construct a hybrid candidate list that includes translational candidates, affine candidates, and sub-block-based temporal motion information candidates without requiring an encoder flag indication, and distinguish different types of candidates by the value of `motionModelIdc`, reducing the amount of data transmitted by the encoder and thus reducing the waste of transmission resources.

[0391] In some embodiments of this application, the first type of candidate in the above-mentioned mixed candidate list may include a translational motion vector.

[0392] In one example, the candidates of the first type mentioned above include translational motion vectors but do not include affine motion vectors.

[0393] In some embodiments of this application, the second type of candidate in the above-mentioned mixed candidate list may include affine motion vectors.

[0394] In one example, the candidates of the second type mentioned above include affine motion vectors but do not include translational motion vectors.

[0395] In some embodiments of this application, the affine motion vector described above can be represented by a first indication information and the motion vector of the control point, wherein the first indication information is used to indicate the number of control points or the affine motion model.

[0396] In one example, the first indication information mentioned above can be numCpMv, used to indicate the number of control points (e.g., 2 or 3) or the affine motion model (e.g., a 4-parameter affine motion model or a 6-parameter affine motion model). The motion vectors of the control points mentioned above can include the motion vectors of 2 or 3 control points.

[0397] For example, the two control points mentioned above can be the top-left and top-right control points of the current block. The three control points mentioned above can be the top-left, top-right, and bottom-left control points of the current block.

[0398] In some embodiments of this application, the motion vector of the affine is represented by the motion vector of the reference point and the rate of change of the motion vector.

[0399] It should be noted that the rate of change of the motion vector mentioned above refers to the rate of change of the motion vector with respect to coordinates within the same affine motion model. Furthermore, given a fixed affine motion model, the rate of change of the motion vector is also fixed.

[0400] It should be noted that the aforementioned reference point can be the reference point recorded in step 4052 above, and the rate of change of the aforementioned motion vector can include parameters a, b, c, and d recorded in step 4052 above. Further descriptions of the aforementioned reference point and the rate of change of the motion vector can be found in the relevant description of step 4052 above; to avoid repetition, they will not be repeated here.

[0401] In some embodiments of this application, the first type of candidates in the above-mentioned mixed candidate list includes translational motion vectors and affine motion vectors, and the second type of candidates in the above-mentioned mixed candidate list includes translational motion vectors and affine motion vectors.

[0402] In other words, for any candidate in the mixed candidate list, the candidate can include both translational motion vectors and affine motion vectors.

[0403] In this embodiment, when each candidate in the hybrid candidate list includes both translational and affine motion vectors, the value of `motionModelIdc` in each candidate can be used to indicate whether the translational motion vector or the affine motion vector in that candidate is valid. In other words, the value of `motionModelIdc` can be used to indicate whether the candidate is a translational candidate or an affine candidate. Specifically, it can also be used to indicate whether the affine candidate is a 4-parameter affine candidate or a 6-parameter affine candidate.

[0404] In a specific implementation, the decoder can use a data structure that can represent both translational and affine candidates to represent candidates in the mixed candidate list.

[0405] In one example, a candidate includes both mvLX and cpMvLX. If the value of motionModelIdc in a candidate is 0, then mvLX is valid, indicating that the candidate is a translation candidate; if the value of motionModelIdc in a candidate is 1, then the first two mv values ​​of cpMvLX are valid, indicating that the candidate is a 4-parameter affine candidate; if the value of motionModelIdc in a candidate is 2, then all three mv values ​​of cpMvLX are valid, indicating that the candidate is a 6-parameter affine candidate.

[0406] In one example, when the affine motion vector is represented by the motion vector and the rate of change of the motion vector through a reference point, cpMvLX can be replaced by the rate of change of the motion vector. That is, a candidate includes both mvLX and the rate of change of the motion vector. If the value of motionModelIdc in a candidate is 0, then mvLX is valid, indicating that the candidate is a translation candidate; if the value of motionModelIdc in a candidate is 1, then both mvLX and the rate of change of the motion vector are valid, indicating that the candidate is an affine candidate.

[0407] Thus, for candidates in the hybrid candidate list, the first type of candidate can include only translational motion vectors, and the second type of candidate can include only affine motion vectors. That is, different data structures can be used to represent the first and second types of candidates in the hybrid candidate list, with each data structure representing either translational or affine motion vectors. Alternatively, the first type of candidate can also include both translational and affine motion vectors, and the second type of candidate can also include both translational and affine motion vectors. In other words, the same data structure can be used to represent both the first and second types of candidates in the hybrid candidate list, with the same data structure representing both translational and affine motion vectors, and the validity of either translational or affine motion vectors indicated by motion model variables. This representation makes the data structure in the hybrid candidate list more unified and easier to manage.

[0408] It should be noted that, based on the above description, the decoder can construct a mixed candidate list. The following section describes the process by which the decoder determines candidates from the mixed candidate list.

[0409] In a first possible implementation of this application, the first candidate may be determined from a mixed candidate list based on a first index in the bitstream.

[0410] In some embodiments of this application, the encoder uses the first candidate indicated by the first index to predict the current block using the prediction mode, and then writes the first index into the bitstream. After the decoder parses the bitstream, it can obtain the first index and determine the first candidate indicated by the first index from the mixed candidate list.

[0411] It should be noted that the above index can be represented using the syntax element `mix_idx`. Taking `mix_idx` as an example, different values ​​of `mix_idx` represent different indices, indicating different candidates. For example, if the value of `mix_idx` is 0, it represents index 0, indicating the candidate with index 0 in the mixing candidate list; if the value of `mix_idx` is 1, it represents index 1, indicating the candidate with index 1 in the mixing candidate list.

[0412] For example, assuming the first index is mix_idx with a value of 2, then the first candidate is the candidate with index 2 in the mix candidate list. Assuming the second index is mix_idx with a value of 10, then the second candidate is the candidate with index 10 in the mix candidate list.

[0413] In some embodiments of this application, when the prediction mode is merge mode, the hybrid candidate list constructed by the decoder can be used to replace the ordinary merge candidate list and sub-block merge candidate list in related technologies. This is one application scenario of the decoding method provided in the embodiments of this application.

[0414] For example, some statements are shown in Table 4 below when making predictions for the current block:

[0415] Table 4

[0416] In Table 4, `merge_data` is a function with five parameters: `x0` and `y0` represent the coordinates of the top-left corner of the current block, `cbWidth` and `cbHeight` represent the width and height of the current block, and `chType` represents the type or characteristic of the current block. `MaxNumMixedCand` represents the length of the mixing candidate list, `merge_idx[x0][y0]` is the index used to indicate the candidate in the mixing candidate list, `mmvd_merge_flag` can be used to indicate the MMVD prediction mode, and `ciip_flag` can be used to indicate the CIIP prediction mode.

[0417] In some embodiments of this application, since the decoder does not distinguish between different types of candidates when constructing the mixed candidate list and adds all different types of candidates to the mixed candidate list, the encoder does not need to transmit merge_subblock_flag to the decoder in the bitstream. Therefore, from a syntactic perspective, the decoder does not need to parse merge_subblock_flag, and the original merge_idx and merge_subblock_idx in related technologies are also merged into mix_idx, reducing the number of syntax elements. Furthermore, since there is no longer a distinction between translation merge and sub-block merge, MMVD and CIIP can be used in any candidate in the mixed candidate list. That is, the decoder can use any candidate in the mixed candidate list to predict the current block using prediction modes such as MMVD and CIIP.

[0418] Example 3: Typically, the decoder can determine candidates from mergeCandList as follows.

[0419] The following assignments are made with N being the candidate at position merge_idx[xCb][yCb]in the merging candidate list mergeCandList(N=mergeCandList[merge_idx[xCb][yCb]]):

[0420] hpelIfIdx=hpelIfIdxN

[0421] bcwIdx=bcwIdxN

[0422] For X=0..1,the following applies:

[0423] refIdxLX=refIdxLXN

[0424] predFlagLX[0][0]=predFlagLXN

[0425] mvLX[0][0][0]=mvLXN[0]

[0426] mvLX[0][0][1]=mvLXN[1]

[0427] mvLX[0][0][1]=mvLXN[1]

[0428] Where N represents the candidate in the candidate list at merge_idx[xCb][yCb], that is, N is one of B1, A1, B0, A0, B2, Col or the candidate added to the candidate list.

[0429] For example, if mergeCandList[0] is B1 and the value of merge_idx[xCb][yCb] is 0, then the current block uses the candidate with index 0 in mergeCandList. When X is 0, refIdxL0 = refIdxL0B1, predFlagL0[0][0] = predFlagL0 B1, mvL0[0][0][0] = mvL0 B1[0]; when X is 1, refIdxL1 = refIdxL1B1, predFlagL1[0][0] = predFlagL1 B1, mvL1[0][0][0] = mvL1 B1[0].

[0430] In its implementation, the decoder can use a data structure {hpelIfIdx, bcwIdx, refIdxL0, predFlagL0, mvL0, refIdxL1, predFlagL1, mvL1} to represent a candidate of mergeCandList. This can be understood as each of B1, A1, B0, A0, B2, and Col being a variable with the aforementioned data structure.

[0431] Example 4: Typically, the decoder can determine candidates from the subblockMergeCandList according to the following, which mainly includes two parts.

[0432] One part specifies how to handle the situation when the value of subblockMergeCandList[merge_subblock_idx[xCb][yCb]] equals SbCol. As you can see in the subsequent for loop, the predFlagLX and mvLX of each child block of the current block are set to equal the predFlagLXSbCol and mvLXSbCol of each child block of SbCol.

[0433] If subblockMergeCandList[merge_subblock_idx[xCb][yCb]] is equal to SbCol, the bi-prediction weight index bcwIdx is set equal to 0and the following applies for X=0..1:

[0434] refIdxLX=refIdxLXSbCol

[0435] For xSbIdx=0..numSbX-1,ySbIdx=0..numSbY-1, the following applies:

[0436] predFlagLX[xSbIdx][ySbIdx]=predFlagLXSbCol[xSbIdx][ySbIdx]

[0437] mvLX[xSbIdx][ySbIdx][0]=mvLXSbCol[xSbIdx][ySbIdx][0]

[0438] mvLX[xSbIdx][ySbIdx][1]=mvLXSbCol[xSbIdx][ySbIdx][1]

[0439] Another part specifies what to do when the value of subblockMergeCandList[merge_subblock_idx[xCb][yCb]] is not equal to SbCol.

[0440] For

[0441] refIdxLX=refIdxLXN

[0442] predFlagLX[0][0]=predFlagLXN

[0443] numCpMv = motionModelIdcN+1

[0444] bcwIdx=bcwIdxN

[0445] For cpIdx=0..numCpMv-1,the following applies:

[0446] cpMvLX[cpIdx]=cpMvLXN[cpIdx]

[0447] For xSbIdx=0..numSbX-1, ySbIdx=0..numSbY-1and X=0..1, the following applies:

[0448] predFlagLX[xSbIdx][ySbIdx]=predFlagLX[0][0]

[0449] For example, if N is A, then we set the values ​​of refIdxLX, predFlagLX, numCpMv, and each cpMvLX of the current block to refIdxLXA, predFlagLXA, and cpMvLXA, respectively. In other words, we replace N with A. After determining the motion vector cpMvLX of each control point, we can derive the mv of each sub-block using an affine model.

[0450] It should be noted that after replacing mergeCandList in Example 3 and subblockMergeCandList in Example 4 with the mixed candidate list mixedCandList, and replacing merge_idx in Example 3 and merge_subblock_idx in Example 4 with mix_idx, the decoder can obtain the first candidate indicated by the first index from the mixed candidate list according to the process described in Examples 3 and 4 above.

[0451] In some embodiments of this application, the prediction mode is a geometric partitioning mode (GPM), and the first candidate may include a first merge candidate and a second merge candidate. The first merge candidate may be determined from the mixed candidate list based on a second index in the bitstream, and the second merge candidate may be determined from the mixed candidate list based on a third index in the bitstream.

[0452] In other words, when the prediction mode is GPM, both sides can use the mixed candidate list in GPM merge. Both candidates needed in GPM merge can be obtained from the mixed candidate list. This is the second application scenario of the decoding method provided in the embodiments of this application.

[0453] In related technologies, the GPM mode can only combine two translation candidates. Furthermore, GPM can use two syntax elements, `merge_gpm_idx0` and `merge_gpm_idx1`, to indicate which candidate from the candidate list each part uses. However, if both parts of GPM can use either translation or affine candidates, then according to the logic of existing technologies, two flags are needed to distinguish whether the two parts use translation or affine candidates. As shown in Table 5, `part0_is_subblock` and `part1_is_subblock` can be used to indicate whether the two parts are translation or affine candidates, respectively.

[0454] Table 5

[0455] In Table 5, merge_gpm_partition_idx represents the partitioning mode of GPM, part0_is_subblock and part1_is_subblock indicate whether the two parts use translational or affine candidates, respectively. merge_subblock_gpm_idx0 and merge_subblock_gpm_idx1 indicate which candidate from the subblock candidate list is used by each part.

[0456] However, since this application uses a hybrid candidate list, which is constructed based on the motion information of any surrounding blocks of the current block, and in principle, it is constructed based on correlation rather than motion model, it can be understood that if there are blocks predicted based on affine motion models among the surrounding blocks of the current block, then there will be affine candidates in the hybrid candidate list, and the current block can select affine candidates without specific indication. Therefore, it is not necessary to use part0_is_subblock and part1_is_subblock to indicate whether to determine translational or affine candidates. The original merge_gpm_idx0 and merge_gpm_idx1 are still used to indicate the candidates to be selected, and merge_gpm_partition_idx represents "partition", that is, it means that two motion vectors are to be used.

[0457] In application scenario two, as shown in Table 6 below, the encoder can use three syntax elements to instruct the decoder to determine the candidates that the encoder will use when predicting the current block from the mixed candidate list.

[0458] Table 6

[0459] In some embodiments of this application, merge_gpm_idx0[x0][y0] in Table 6 is the second index mentioned above, and merge_gpm_idx1[x0][y0] is the third index mentioned above. The decoder can obtain the second merge candidate indicated by merge_gpm_idx0[x0][y0] and the third merge candidate indicated by merge_gpm_idx1[x0][y0] from the merge candidate list in the manner described in Example 4 above.

[0460] Thus, using a hybrid candidate list allows GPM to support prediction of the current block using affine candidates without adding any syntax elements.

[0461] In some embodiments of this application, GPM is used in merge mode. Both motion information needed for predicting the current block can be obtained from the merge candidate list, and MVD is not required. If GPM is understood as a mode that simply combines two predicted blocks using a weight matrix, it can combine any two prediction modes. In inter-frame prediction, if merge mode is used, MVD does not need to be transmitted. MMVD is a special case, which can be understood as a simplified special MVD. The opposite of merge is the prediction mode with MVD. Here, MVD is the MVD described in mvd_coding(), or the complete MVD, which can represent any MVD within the allowed range. The final motion vector is determined by MVP + MVD. For affine, the affine mode with MVD controlled by inter_affine_flag is also referred to as Affine-AMVP mode in some technical descriptions.

[0462] In some embodiments of this application, the prediction mode with MVD needs to transmit MVD in the bitstream. It is understood that this mode requires transmitting more information compared to merge. If the motion information of the current block is the same as or sufficiently similar to the motion information of the already encoded and decoded surrounding blocks, then the merge mode can be used to directly obtain the motion information of the surrounding blocks for the current block, or slightly adjusted using MMVD. If the motion information of the current block differs significantly from the motion information of the already encoded and decoded surrounding blocks—for example, if the object contained in the current block has not appeared in the surrounding blocks—then the prediction mode with MVD can be used to determine the motion information using MVD based on an MVP.

[0463] In some embodiments of this application, if a portion of the content in the current block belongs to a newly appearing object, the motion information of this portion differs significantly from the motion information of the surrounding blocks that have already been encoded and decoded. Conversely, another portion of the content in the current block belongs to the same object as the surrounding blocks, and the motion information of this portion is the same as or similar to that of a certain surrounding block. This scenario actually occurs frequently at the edges of moving objects. Because the edges of moving objects are highly variable, and block partitioning can only divide them into rectangular blocks, and the more detailed the block partitioning, the greater the cost, it is impossible for rectangular block partitioning to perfectly approximate the edges of moving objects.

[0464] In some embodiments of this application, the prediction mode is a geometric partitioning mode, and the first candidate can be determined from the mixed candidate list based on the first index in the bitstream. The first candidate can include a first motion vector for predicting the current block, and the second motion vector for predicting the current block can be determined based on motion vector difference information.

[0465] In some embodiments of this application, the second motion vector for predicting the current block can be determined based on motion vector difference information (MVD) and motion vector prediction (MVP). It should be noted that the motion vector prediction can be determined from the mixed candidate list based on the fourth index, or it can be obtained by parsing the bitstream; this application does not limit this approach.

[0466] In one example, when the prediction mode is geometric partitioning mode, for content in the current block that belongs to the same object as the surrounding blocks, the motion information of this content (i.e., the first motion vector mentioned above) can be determined based on the motion information of the surrounding blocks of the current block. For content in the current block that belongs to a newly appearing object, the motion information of this content (i.e., the second motion vector mentioned above) differs greatly from the motion information of the surrounding blocks of the current block, so it can be determined based on the motion vector difference information.

[0467] It is understandable that when the motion information of the current block differs from that of its surrounding blocks, and the motion information of the current block cannot be represented in any other way, the decoder can determine a second motion vector for predicting the current block based on the motion vector difference information. In other words, when the motion information in the candidates in the mixed candidate list is not suitable as the motion information of the current block, the decoder can determine a second motion vector based on the motion vector difference information to predict the current block.

[0468] In some embodiments of this application, the aforementioned motion vector difference information can be obtained from the bitstream.

[0469] It is understandable that, in addition to writing the motion vector difference information (MVD) into the bitstream, it is also necessary to write syntax elements that indicate the partitioning mode of the GPM, so that the first motion vector and the second motion vector can be processed based on the partitioning mode of the GPM, and finally the prediction block of the current block can be obtained.

[0470] In some embodiments, the above-mentioned GPM partitioning mode can also be called the weight matrix derivation mode or weight derivation mode.

[0471] In some embodiments of this application, the decoder can determine a first candidate from a mixed candidate list based on a first index, identify the motion vector in the first candidate as a first motion vector, and determine a first prediction block based on the first motion vector. This first prediction block can correspond to the portion of the object that belongs to the same object as the surrounding blocks. Furthermore, the decoder can determine a second motion vector based on motion vector difference information and motion vector prediction, and determine a second prediction block based on the second motion vector. This second prediction block can correspond to the portion of the newly appearing object. Then, the decoder can predict the current block based on the GPM partitioning mode, the first prediction block, and the second prediction block to obtain the prediction block for the current block.

[0472] In some embodiments of this application, the amount of the aforementioned motion vector difference information is related to the motion model of the current block.

[0473] In some embodiments of this application, the motion model of the current block is a translational motion model, and the number of the above-mentioned motion vector difference information is 1.

[0474] In other words, if the motion model of the current block is a translational motion model, then an MVD is required in at least one prediction reference direction (such as RPL0 or RPL1) to generate accurate motion vectors.

[0475] In some embodiments of this application, the motion model of the current block is a 4-parameter affine motion model, and the number of the above-mentioned motion vector difference information is 2.

[0476] In other words, if the motion model of the current block is a 4-parameter affine motion model, then two MVDs are needed in at least one prediction reference direction to generate accurate motion vectors.

[0477] In some embodiments of this application, the motion model of the current block is a 6-parameter affine motion model, and the number of the aforementioned motion vector difference information is 3.

[0478] In other words, if the motion model of the current block is a 6-parameter affine motion model, then 3 MVDs are needed in at least one prediction reference direction to generate accurate motion vectors.

[0479] Thus, when using GPM to predict the current block, both translational and affine candidates can be supported without specifying in the bitstream which motion model should be used for the first motion vector. This improves prediction performance by supporting different motion models without requiring additional syntax elements, and better balances the prediction information to be written into the bitstream for the entire block.

[0480] In some embodiments of this application, the above-mentioned mixed candidate list may include a candidate, and the above-mentioned first candidate may be that candidate.

[0481] In some embodiments of this application, a special case occurs when the decoder constructs the mixed candidate list: the length of the mixed candidate list is 1, meaning that only one available candidate needs to be determined and added to the mixed candidate list. In one example, the first candidate can be this single candidate.

[0482] For example, when the length of the mixed candidate list is 1, it can be understood as constructing a mixed candidate list of length 1. Alternatively, it can be understood as determining a motion vector for predicting the current block.

[0483] In this case, the encoder does not need to send a syntax element to the decoder to instruct the decoder which candidate to choose. The encoder can only use this one unique "candidate," and the decoder can only use this one unique "candidate." In this case, it is more appropriate to call the "candidate" a "prediction," such as motion information prediction or motion vector prediction.

[0484] In some embodiments of this application, if the decoder only needs to determine one available motion vector prediction MVP, the decoder can still use the method of this scheme to construct a mixed candidate list, select one candidate from it, and use the motion vector prediction MVP in that candidate to predict the current block.

[0485] It should be noted that although the hybrid candidate list includes one candidate (which can also be understood as including one type of candidate) in the above case, this is because only one candidate is added to the hybrid candidate list, which is a special case. It does not mean that the hybrid candidate list cannot include other types of candidates. In fact, the essence of the hybrid candidate list remains unchanged. As long as the hybrid candidate list is constructed according to the construction method of this application, the resulting hybrid candidate list has the ability to include various types of candidates, which is completely different from the candidate list in related technologies.

[0486] In some embodiments of this application, the above-mentioned mixed candidate list may include multiple candidates. If the above-mentioned prediction mode is a preset prediction mode, then the above-mentioned first candidate may be a candidate at a preset position among multiple candidates.

[0487] In one example, the aforementioned preset prediction mode can be understood as a specific prediction mode, that is, a prediction mode that requires using candidates at a specific position (preset position) in the mixed candidate list. For example, the aforementioned preset prediction mode can be the sbtmvp mode or a sample (pixel) based prediction mode, etc., and this application embodiment does not limit it in this way.

[0488] In one example, the preset position can be any position in the mixed candidate list. For example, the preset position can be the first position in the mixed candidate list.

[0489] In some embodiments of this application, the decoder can construct a mixed candidate list of length N, where N is an integer greater than 1, and then forcibly determine a candidate at a preset position in the mixed candidate list as the first candidate.

[0490] In some embodiments of this application, the aforementioned mixed candidate list includes multiple candidates. When the prediction mode is GPM and the first candidate includes a first merged candidate and a second merged candidate, at least one of the first merged candidate and the second merged candidate can be a candidate at the preset position.

[0491] In one example, the first merge candidate is a candidate at the aforementioned preset position, and the second merge candidate is determined from the mixed candidate list based on an index in the bitstream; or, the second merge candidate is a candidate at the aforementioned preset position, and the first merge candidate is determined from the mixed candidate list based on an index in the bitstream; or, both the first and second merge candidates are candidates at the aforementioned preset positions.

[0492] Thus, since candidates at preset positions in the mixed candidate list are required in certain prediction modes, the decoder can directly obtain the candidates at preset positions in the mixed candidate list when it determines that the prediction mode is a specific prediction mode, without the need for index indication. Therefore, the index used to indicate which candidate the decoder should select is not transmitted in the bitstream, which can save transmission resources.

[0493] In this embodiment, the decoder parses the bitstream, determines the prediction mode, constructs a hybrid candidate list, and determines a first candidate from the hybrid candidate list based on a first index in the bitstream. Alternatively, the decoder parses the bitstream, determines the prediction mode, constructs a hybrid candidate list, and determines a first merged candidate from the hybrid candidate list based on a second index in the bitstream, and a second merged candidate from the hybrid candidate list based on a third index in the bitstream. Alternatively, the decoder parses the bitstream, determines the prediction mode, constructs a hybrid candidate list, and determines a first candidate from the hybrid candidate list based on a first index in the bitstream, obtaining a first motion vector from the first candidate, and determines a second motion vector based on the motion vector difference information in the bitstream.

[0494] Typically, both affine merge mode and sbtmvp (subblock based temporal motion vector prediction) mode are subblock-based merging modes, and a subblock merge candidate list can be constructed based on affine merge and sbtmvp.

[0495] The syntax for determining whether the current block uses a merge mode in the coding unit syntax is shown in Table 7 below:

[0496] Table 7

[0497] In Table 7, `general_merge_flag[x0][y0]` indicates whether the current block uses information from neighboring blocks to derive inter-frame prediction parameters. In other words, it determines whether the current block uses the merge mode. Array indices x0 and y0 represent the coordinates of the brightness sample at the top-left corner of the current block relative to the brightness sample at the top-left corner of the current image.

[0498] If the value of general_merge_flag[x0][y0] is 1, the decoder will parse merge_data(x0,y0,cbWidth,cbHeight,chType), as shown in Table 8 below:

[0499] Table 8

[0500] In `merge_data`, if `MaxNumSubblockMergeCand > 0 && cbWidth >= 8 && cbHeight >= 8`, the decoder can parse `merge_subblock_flag`. Here, `MaxNumSubblockMergeCand` indicates the maximum possible number of candidate subblock merge patterns. `cbWidth` represents the width of the current block, and `cbHeight` represents the height of the current block.

[0501] Here, `merge_subblock_flag[x0][y0]` indicates whether the current block uses information from neighboring blocks to derive inter-frame prediction parameters based on sub-blocks. In other words, it indicates whether the current block uses sub-block merging mode. Array indices x0 and y0 represent the coordinates of the brightness sample at the top-left corner of the current block relative to the brightness sample at the top-left corner of the current image. If `merge_subblock_flag[x0][y0]` does not exist, it is inferred that the current block does not use sub-block merging mode.

[0502] If MaxNumSubblockMergeCand > 1, the decoder needs to parse merge_subblock_idx. merge_subblock_idx[x0][y0] indicates the index of the merge candidate used by the current block in the subblock merge candidate list. Here, array indices x0 and y0 represent the coordinates of the luminance sample at the top-left corner of the current block relative to the luminance sample at the top-left corner of the current image. If merge_subblock_idx[x0][y0] does not exist, it is inferred that the current block did not use any candidate from the subblock merge candidate list.

[0503] In the sub-block merge candidate list, sbtmvp has one candidate, and this candidate is in the first position of the sub-block merge candidate list, followed by affine merge candidates. If the current block uses the sub-block merge mode, i.e., merge_subblock_flag[x0][y0] is 1, then the decoder will construct the sub-block merge candidate list, select the corresponding candidate according to merge_subblock_idx[x0][y0] to determine the motion information of each sub-block of the current block, and make a prediction value based on motion compensation according to the motion information.

[0504] Therefore, in related technologies, the decoder needs to first parse the merge_subblock_flag in the bitstream to determine whether the current block uses the subblock merging mode. If so, the decoder can construct a subblock merging candidate list, then determine the candidate from the subblock merging candidate list according to merge_subblock_idx, and make predictions for the current block based on the candidate and the subblock merging mode.

[0505] In this embodiment, since the decoder constructs a hybrid candidate list capable of including both first-type and second-type candidates, the decoder can retrieve the candidate from the hybrid candidate list based on the first index, regardless of whether the encoder uses a first-type or second-type candidate for predicting the current block. Furthermore, since the decoder does not rely on the `merge_subblock_flag` instruction when constructing the hybrid candidate list, it does not need to parse `merge_subblock_flag`. Compared to existing technologies where the decoder needs to parse `merge_subblock_flag` first to determine which candidate list to construct, this solution not only saves the syntax element `merge_subblock_flag`, reducing transmission resource waste and improving encoding efficiency, but also reduces the decoder's parsing steps, thus improving decoding efficiency.

[0506] Step 403: The decoder predicts the current block based on the prediction mode and the first candidate, and obtains the predicted block of the current block.

[0507] In some embodiments of this application, the prediction mode is merge mode. The decoder can locate the reference in the reference image of the current block based on the motion vector in the first candidate, and perform interpolation processing on the reference block, and use the interpolated reference block as the prediction block of the current block.

[0508] In some embodiments of this application, the prediction mode is GPM. The decoder can first determine the weight derivation mode of the current block, and then determine the first prediction block and the second prediction block of the current block based on the weights, the first motion vector and the second motion vector derived from the weight derivation mode. Then, the first prediction block and the second prediction block are weighted according to the weights derived from the weight derivation mode to obtain the prediction block of the current block.

[0509] The decoding method provided in this application can construct a hybrid candidate list after parsing and determining the prediction mode. This hybrid candidate list not only has the ability to include candidates of the first type, but also the ability to include candidates of the second type. It is not limited to the ability to include only one type of candidate. That is, it does not need to construct a candidate list of a certain type according to the indication of the syntax elements in the bitstream. Therefore, the construction of the hybrid candidate list does not need to be indicated by transmitting the corresponding syntax elements in the bitstream. Thus, it can reduce the syntax elements transmitted in the bitstream, save transmission resources, and improve encoding efficiency.

[0510] In some embodiments of this application, step 403 can be specifically implemented by step 4031 as described below.

[0511] Step 4031: If the first index in the bitstream is less than or equal to the first threshold, the decoder predicts the current block based on the prediction mode and the first candidate, and obtains the prediction block of the current block.

[0512] In some embodiments of this application, the prediction mode described above may be MMVD mode or CIIP mode.

[0513] In some embodiments of this application, the first candidate may be determined from a mixed candidate list based on a first index.

[0514] In some embodiments of this application, since the mixed candidate list does not distinguish between translational and affine candidates, prediction modes such as MMVD and CIIP can also be applied to any candidate in the mixed candidate list. Furthermore, a first threshold can be set; if the value of mix_idx (i.e., the aforementioned first index) is less than or equal to the first threshold, the MMVD mode or CIIP mode can be used to predict the current block.

[0515] In some embodiments of this application, when the decoder constructs the mixed candidate list, the decoding method provided in this application may further include the following steps 406 and 407.

[0516] Step 406: The decoder determines whether the first motion information and the second motion information are duplicated.

[0517] In some embodiments of this application, the first motion information mentioned above may be the motion information of the third surrounding block of the current block.

[0518] In some embodiments of this application, the aforementioned third peripheral block can be any peripheral block of the current block that has already been decoded using inter-frame prediction mode. It is understood that if a peripheral block of the current block has already been decoded using inter-frame prediction mode, the motion information of that peripheral block is available. Therefore, the decoder, after determining that the first motion information is available, then determines whether the first motion information overlaps with the second motion information.

[0519] In some embodiments of this application, the second motion information described above may be motion information from candidates in a mixed candidate list.

[0520] In some embodiments of this application, the second motion information can be the motion information of the fourth surrounding block of the current block. This fourth surrounding block can be a surrounding block of the current block that has been decoded using inter-frame prediction mode and whose motion information is available. In one example, the fourth surrounding block can be one or more surrounding blocks.

[0521] In some embodiments of this application, when the decoder finds a neighboring block of the current block, it first determines whether the motion information of the neighboring block is available. If so, it then determines whether the motion information of the neighboring block (e.g., the first motion information mentioned above) is duplicated with the motion information of other neighboring blocks that have been checked (e.g., the second motion information mentioned above). If they are duplicated, the value of the first variable corresponding to the motion information of the neighboring block is set to a first value, indicating that the motion information of the neighboring block cannot be used as a candidate. If they are not duplicated, the value of the first variable corresponding to the motion information of the neighboring block is set to a second value, indicating that the motion information of the neighboring block can be used as a candidate. Then, the decoder continues to check whether the motion information of the next neighboring block found is available, and so on, until the motion information of all the neighboring blocks of the current block has been checked. Then, the motion information with the value of the first variable being the second value is added to the mixed candidate list. It can be understood that in some embodiments, the decoder may also set the value of the first variable corresponding to the motion information of a neighboring block to the second value and add the motion information of the neighboring block to the mixed candidate list when it finds that the motion information of a neighboring block is not duplicated with the motion information of other neighboring blocks that have been checked.

[0522] In one example, a neighboring block of the current block can be an adjacent neighboring block of the current block or a non-adjacent neighboring block of the current block.

[0523] It should be noted that "motion information of a surrounding block cannot be used as a candidate" means that the motion information of that surrounding block cannot be added to the mixed candidate list. "Motion information of a surrounding block can be used as a candidate" means that the motion information of that surrounding block can be added to the mixed candidate list.

[0524] In one example, the first variable mentioned above can be "available". It should be noted that the values ​​of the first and second values ​​can be set according to actual needs, and this embodiment does not limit this. For example, the first value can be 0, and the second value can be 1.

[0525] For example, assuming the motion information of neighboring block A1 is the same as the motion information of the already checked neighboring block A2, the decoder can set the value of the first variable corresponding to the motion information of neighboring block A1 to 0, indicating that the motion information of neighboring block A1 cannot be used as a candidate. Assuming the motion information of neighboring block A1 is not the same as the motion information of any other checked neighboring blocks, the decoder can set the value of the first variable corresponding to the motion information of neighboring block A1 to 1, indicating that the motion information of neighboring block A1 can be used as a candidate.

[0526] In one example, the decoder can check the motion information of all surrounding blocks in a preset order until a termination condition is met. One possible termination condition is that the mixing candidate list has reached a preset length, i.e., the mixing candidate list is full. Another possible termination condition is that all surrounding blocks have been checked. Furthermore, if all surrounding blocks have been checked but the mixing candidate list is not yet full, constructed candidates or default candidates can be added.

[0527] Step 407: If the first motion information and the second motion information do not overlap, the decoder constructs a hybrid candidate list based on the first motion information.

[0528] In one example of this application, if the first motion information and the second motion information do not overlap, it means that the first motion information can be used as a candidate, and the decoder can fill the first motion information into the mixed candidate list.

[0529] In other words, each time the decoder detects a motion information that can be used as a candidate, it fills that motion information into the mixed candidate list.

[0530] In another example of this application, if the first motion information and the second motion information do not overlap, it means that the first motion information can be used as a candidate. The decoder can set the value of the first variable to the second value, and after checking the motion information of the surrounding blocks of the current block, fill the first motion information into the mixed candidate list.

[0531] In other words, after the decoder has checked the motion information of the surrounding blocks of the current block, it fills the motion information with the value of the first variable as the value of the second value into the mixed candidate list.

[0532] It should be noted that steps 406 to 407 can be performed before step 402.

[0533] In some embodiments of this application, the specific implementation of "the decoder determines whether the first motion information and the second motion information are repeated" in step 406 above can be achieved through steps 4061 to 4063 below.

[0534] Step 4061: The decoder determines whether the first motion model corresponding to the first motion information is the same as the second motion model corresponding to the second motion information.

[0535] It should be noted that, in the embodiments of this application, the motion model corresponding to the motion information refers to the motion model used to predict the surrounding blocks corresponding to the motion information.

[0536] Step 4062: If the first motion model is different from the second motion model, the decoder determines that the first motion information and the second motion information are not repeated.

[0537] Step 4063: If the first motion model is the same as the second motion model, the decoder determines whether the first motion information and the second motion information are duplicated based on the information in the first motion information and the information in the second motion information.

[0538] In some embodiments of this application, the information in a motion information may include motion vectors, reference image indexes, flags indicating whether a reference image list is used, etc. The decoder can compare the motion vectors in the first motion information with the motion vectors in the second motion information, compare the reference image indexes in the first motion information with the reference image indexes in the second motion information, and compare the flags indicating whether a reference image list is used in the first motion information with the flags indicating whether a reference image list is used in the second motion information. If any comparison result is different, it can be determined that the first motion information and the second motion information are not duplicated; if all comparison results are the same, it can be determined that the first motion information and the second motion information are duplicated.

[0539] For example, when constructing a hybrid candidate list, in addition to checking whether motion information for a block containing a specific location is available, it's also necessary to check if this motion information overlaps with other motion information. This overlap refers to two motion information pieces being identical or overly similar. If the block containing a specific location is coded using inter-frame prediction mode, then it has available motion information. This motion information can be compared with other motion information to avoid duplication. This other motion information can be motion information already included in the candidate list, or motion information from blocks containing locations other than the specific location.

[0540] For example, suppose that both the first and second motion information are determined based on translational motion, and the first motion information is motion information M, the second motion information is motion information N, and motion information M is available. The decoder can first determine whether the first motion model corresponding to motion information M is the same as the second motion model corresponding to motion information N, that is, whether the motionModelIdc of motion information M and the motionModelIdc of motion information N are the same. If the motionModelIdc of motion information M and the motionModelIdc of motion information N are different, then it is determined that motion information M and motion information N are not duplicates. If the motionModelIdc of motion information M and the motionModelIdc of motion information N are the same, then it is further checked whether the mvLX, predFlagLX, refIdxLX, etc. of motion information M and motion information N are the same. If they are all the same, then it is determined that motion information M and motion information N are duplicates. If at least one is different, then it is determined that motion information M and motion information N are not duplicates.

[0541] This avoids duplicate candidates in the constructed mixed candidate list, reducing resource waste.

[0542] It should be noted that either step 4062 or step 4063 above can be performed.

[0543] The decoding method provided in this application uses a hybrid candidate list to support both translational and affine motion, without requiring syntax elements in the bitstream to indicate which motion model to use. In other words, it does not require syntax elements in the bitstream to instruct the decoder to construct the corresponding candidate list. This improves prediction performance by supporting different motion models and avoids the need to add additional syntax elements, thus better balancing the prediction information to be written into the bitstream for the entire block and improving coding efficiency.

[0544] Referring to Figure 16, which shows a flowchart of an encoding method provided in an embodiment of this application, the encoding method can be applied to an encoder. As shown in Figure 16, the encoding method provided in this embodiment of the application may include the following steps 501 to 503.

[0545] Step 501: The encoder determines the prediction mode of the current block.

[0546] In some embodiments of this application, the above prediction mode can be used to predict the current block.

[0547] In some embodiments of this application, the prediction mode described above may be a merge mode, or the prediction mode may be GPM.

[0548] In some embodiments of this application, the aforementioned current block may be a CU, or a PU, or other possible forms of coded blocks, etc., and the embodiments of this application are not limited thereto.

[0549] In some embodiments of this application, the encoder can use a prediction mode to predict the current block, obtain a predicted block for the current block, then obtain a residual based on the predicted block and the current block, and encode the residual and the prediction mode to obtain a bitstream, which is then transmitted to the decoder. After receiving the bitstream, the decoder can parse the bitstream, determine the prediction mode, and predict the current block based on the prediction mode to obtain a predicted block for the current block. Then, it can reconstruct the reconstructed block for the current block based on the predicted block and the residual.

[0550] Step 502: The encoder determines the first candidate.

[0551] In some embodiments of this application, the first candidate may be determined from a mixed candidate list.

[0552] In some embodiments of this application, the encoder may use a rate-distortion function to calculate the rate-distortion cost of each candidate in the mixed candidate list, and select the candidate with the smallest rate-distortion cost as the first candidate.

[0553] In some embodiments of this application, the aforementioned mixed candidate list may be referred to as a mixedCandList, and the candidates in the mixed candidate list may be referred to as candidates. It should be noted that the aforementioned mixed candidate list may also be referred to in other ways, such as a candidate list or a mixed and merged candidate list, etc., and the embodiments of this application do not limit it.

[0554] In some embodiments of this application, the above-mentioned mixed candidate list may include at least one of a first type of candidate and a second type of candidate.

[0555] It is understood that the hybrid candidate list in this application embodiment has the ability to include both the first type of candidate and the second type of candidate, but it is not necessary to include both types of candidates simultaneously. That is, the hybrid candidate list may include the first type of candidate, or the hybrid candidate list may include the second type of candidate, or the hybrid candidate list may include both the first type of candidate and the second type of candidate.

[0556] In other words, the aforementioned mixed candidate list may include one or more candidates. If the mixed candidate list includes one candidate, that candidate may be either a first-type candidate or a second-type candidate. If the mixed candidate list includes multiple candidates, those multiple candidates may include only first-type candidates, or only second-type candidates, or a combination of both types.

[0557] It should be noted that the candidates mentioned in the embodiments of this application refer to motion information, that is, motion information of the surrounding blocks of the current block or motion information determined based on the motion information of the surrounding blocks of the current block. This motion information is used to predict the current block to obtain a predicted block for the current block.

[0558] In some embodiments of this application, the first type of candidates and the second type of candidates described above can be determined based on different motion models. In other words, the first type of candidates and the second type of candidates described above can belong to different motion models.

[0559] In one example, the candidates of the first type described above can be determined based on a translational motion model. The candidates of the second type described above can be determined by a sub-block-based motion model. Exemplarily, the candidates of the second type described above can be determined based on an affine motion model, or the candidates of the second type described above can be determined by other sub-block-based motion models.

[0560] Among them, the candidates determined based on the affine motion model can be called affine candidates, and the candidates determined by other sub-block-based motion models can include sub-block-based temporal motion information candidates.

[0561] In some embodiments of this application, the first type of candidate described above may be referred to as a translation candidate or a candidate for translational motion. The translation candidate can be understood as a candidate determined based on the motion information of the surrounding blocks of the current block when the surrounding blocks are predicted based on a translational motion model.

[0562] In some embodiments of this application, the candidates of the first type mentioned above may include one or more of the following: spatial motion information candidates, temporal motion information candidates, historical motion information candidates, and pairwise average motion information candidates.

[0563] It should be noted that other descriptions of the above-mentioned spatial motion information candidates, temporal motion information candidates, historical motion information candidates, and pairwise average motion information candidates can be found in the relevant descriptions of the above-mentioned decoding method embodiments. To avoid repetition, they will not be repeated here.

[0564] In some embodiments of this application, the second type of candidate described above may include one or more of affine motion information candidates and sub-block-based temporal motion information candidates.

[0565] In some embodiments of this application, the aforementioned affine motion information candidates can be affine motion information candidates, and the aforementioned sub-block-based temporal motion information candidates can be sbtmvp candidates.

[0566] In other words, the candidates of the second type mentioned above may include affine motion information candidates, or the candidates of the second type mentioned above may include sub-block-based temporal motion information candidates, or the candidates of the second type mentioned above may include affine motion information candidates and sub-block-based temporal motion information candidates.

[0567] In some embodiments of this application, the aforementioned affine motion information candidates can be referred to as affine candidates or affine motion candidates. These affine motion information candidates can be understood as candidates determined based on the motion information of the surrounding blocks when the current block is predicted based on an affine motion model.

[0568] In some embodiments of this application, the above-mentioned candidate temporal motion information based on sub-blocks can be understood as: when the surrounding blocks of the current block are predicted by the motion model based on sub-blocks, the candidates are determined based on the motion information of the surrounding blocks.

[0569] In some embodiments of this application, the aforementioned sub-block-based temporal motion information candidates can be candidates at preset positions in a mixed candidate list.

[0570] In one example, the preset position can be any position in the mixed candidate list.

[0571] In some embodiments of this application, the candidate at the preset position can be the first candidate in the mixed candidate list.

[0572] In other words, the encoder can add sub-block-based temporal motion information candidates to the first position in the hybrid candidate list.

[0573] In some embodiments of this application, the principle for constructing the hybrid candidate list is based on any surrounding blocks of the current block. That is, when the encoder constructs the hybrid candidate list, it finds any surrounding block of the current block and constructs the hybrid candidate list based on the motion information of that surrounding block, instead of determining whether the predicted motion model of the surrounding block is a specific motion model when the surrounding block is found, and only using the motion information of the surrounding block to construct the candidate list if the predicted motion model of the surrounding block is a specific motion model.

[0574] Thus, since the encoder and decoder construct the merged candidate list in the same way, the encoder does not need to transmit a flag in the bitstream to inform the decoder which type of merged candidate list to construct, saving transmission resources for transmitting the flag in the bitstream.

[0575] In some embodiments of this application, before step 502 above, the encoding method provided in the embodiments of this application may further include steps 504 and 505 as described below.

[0576] Step 504: The encoder determines the motion model of the surrounding blocks of the current block for prediction.

[0577] Step 505: The encoder makes a motion model based on the surrounding blocks and constructs a hybrid candidate list.

[0578] It should be noted that the specific implementation of steps 504 and 505 above can be found in the relevant descriptions of steps 404 and 405 in the above decoding method embodiments. To avoid repetition, they will not be repeated here.

[0579] In some embodiments of this application, step 505 can be specifically implemented by the following steps 5051 and 5052.

[0580] Step 5051: If the surrounding blocks of the current block are predicted based on the translational motion model, the encoder constructs a hybrid candidate list based on the translational motion information of the surrounding blocks.

[0581] The surrounding blocks of the current block are predicted based on the translational motion model. This can be understood as the translational motion model being used when predicting the surrounding blocks of the current block.

[0582] In some embodiments of this application, if the surrounding blocks of the current block are predicted based on a translational motion model, the encoder can use the translational motion information of the surrounding blocks as a candidate in the hybrid candidate list, i.e., a candidate of the first type. Alternatively, the decoder can use motion information identical to the translational motion information of the surrounding blocks as a candidate in the hybrid candidate list, i.e., a candidate of the first type.

[0583] In some embodiments of this application, the translational motion information may include a motion vector (mvLX), a reference image index (refIdxLX), and a flag indicating whether a list of reference images is used (predFlagLX).

[0584] Step 5052: If the surrounding blocks of the current block are predicted based on the affine motion model, the encoder constructs a hybrid candidate list based on the affine motion information of the surrounding blocks.

[0585] In some embodiments of this application, if the surrounding blocks of the current block are predicted based on an affine motion model, the encoder can construct a hybrid candidate list based on the derived affine motion information, which can be derived based on the affine motion information of the surrounding blocks.

[0586] In some embodiments of this application, the affine motion information derived above may include a motion vector (cpMvLX), a reference image index (refIdxLX), and a flag indicating whether a list of reference images is used (predFlagLX).

[0587] It should be noted that for other descriptions of steps 5051 and 5052 above, please refer to the relevant descriptions of steps 4051 and 4052 in the above decoding method embodiments. To avoid repetition, they will not be repeated here.

[0588] In some embodiments of this application, step 505 can be specifically implemented by steps 5053 and 5054 as described below.

[0589] Step 5053: The encoder predicts the motion model based on the first surrounding block of the current block as a translational motion model, and adds the translational motion information of the first surrounding block as a first type of candidate to the mixed candidate list.

[0590] Step 5054: The encoder predicts the motion model based on the second surrounding block of the current block as an affine motion model, and adds the affine motion information derived from the affine motion information of the second surrounding block as a second type of candidate to the mixed candidate list.

[0591] In some embodiments of this application, the second type of candidates includes sub-block-based temporal motion information candidates. When the encoder constructs the hybrid candidate list, if the motion model predicted by the surrounding blocks of the current block is based on the sub-block motion model, the encoder adds the motion information determined based on the motion information of the surrounding blocks as a sub-block-based temporal motion information candidate to the hybrid candidate list.

[0592] It should be noted that for other descriptions of steps 5053 and 5054 above, and the differences between the construction of the hybrid candidate list in this scheme and the construction of the candidate list by related technologies, please refer to the relevant descriptions in the above-described decoding method embodiments. To avoid repetition, they will not be repeated here.

[0593] In some embodiments of this application, when the encoder constructs the hybrid candidate list, candidates of the first type (translation candidates) and candidates of the second type (affine candidates and sub-block-based temporal motion information candidates) are added to the hybrid candidate list. However, since it is impossible to predict whether each candidate in the hybrid candidate list is a translation candidate, an affine candidate, or a sub-block-based temporal motion information candidate, a variable can be used to indicate the motion model or type of the candidate.

[0594] In some embodiments of this application, the motion model corresponding to the candidates in the above-mentioned mixed candidate list is indicated by motion model variables.

[0595] In some embodiments of this application, the motion model corresponding to the candidates in the above-mentioned mixed candidate list can be indicated by a motion model variable. This motion model variable can be referred to as motionModelIdc.

[0596] It should be noted that the candidates in the above mixing list refer to any one candidate in the mixing list. The motion model corresponding to the above candidate refers to the motion model used to predict the surrounding blocks corresponding to the candidate, that is, the motion model used to predict the surrounding blocks corresponding to the motion vectors in the candidate.

[0597] In some embodiments of this application, the motion model described above may include at least one of the following: a translational motion model, an affine motion model, and a sub-block-based motion model. The affine motion model may include a 4-parameter affine motion model and a 6-parameter affine motion model.

[0598] For example, if a candidate is a candidate of the first type, and the motion model corresponding to that candidate is a translational motion model, the value of motionModelIdc can be set to 0. If a candidate is an affine candidate, and the motion model corresponding to that candidate is a 4-parameter affine motion model, the value of motionModelIdc can be set to 1. If a candidate is an affine candidate, and the motion model corresponding to that candidate is a 6-parameter affine motion model, the value of motionModelIdc can be set to 2. If a candidate is a candidate based on sub-block temporal motion information, that is, the motion model corresponding to that candidate is a sub-block motion model, the value of motionModelIdc can be set to 3.

[0599] In some embodiments of this application, the aforementioned motion model variables are elements in a motion model array, and the elements in the motion model array correspond one-to-one with the candidates in the hybrid candidate list.

[0600] In other words, the decoder can construct a motion model array during the process of building the hybrid candidate list. The elements in the motion model array are motion model variables, and each element in the motion model array corresponds one-to-one with a candidate in the hybrid candidate list.

[0601] In some embodiments of this application, the above-mentioned hybrid candidate list includes a first list and a second list, wherein the first list includes motion model variables and the second list includes candidates corresponding to the motion models indicated by the motion model variables.

[0602] In other words, the mixed candidate list includes a first list and a second list. The first list includes multiple motion model variables, and the number of motion model variables in the first list is the same as the number of candidates in the second list. Moreover, the motion model variables in the first list correspond one-to-one with the candidates in the second list.

[0603] In some embodiments of this application, the candidates in the above-mentioned mixed candidate list may include motion model variables.

[0604] It is understandable that the above motion model variables can be attribute information of candidates in the mixed candidate list, or that the above motion model variables can be included in each candidate along with other motion information, or that the above motion model variables can be part of the motion information corresponding to each candidate.

[0605] In some embodiments of this application, motionModelIdc can be used as attribute information of candidates in the mixed candidate list, that is, the candidates in the mixed candidate list include motionModelIdc. In other words, whether it is a candidate of the first type or a candidate of the second type, a candidate may include motionModelIdc in addition to mvLX, predFlagLX, refIdxLX (where X represents RPL0 or RPL1, and there are multiple mvLX for affine motion, such as an array cpMvLX[3]).

[0606] It should be noted that for other descriptions of motion model variables, motion model arrays, the first list, the second list, and related content, please refer to the relevant descriptions in the above-described decoding method embodiments. To avoid repetition, they will not be repeated here.

[0607] In some embodiments of this application, the first type of candidates in the above-mentioned mixed candidate list includes translational motion vectors.

[0608] In some embodiments of this application, the second type of candidate in the above-mentioned mixed candidate list includes affine motion vectors.

[0609] In some embodiments of this application, the affine motion vector is represented by a first indication information and a motion vector of a control point, wherein the first indication information is used to indicate the number of control points or the affine motion model.

[0610] In one example, the first indication information mentioned above can be numCpMv, used to indicate the number of control points (e.g., 2 or 3) or the affine motion model (e.g., a 4-parameter affine motion model or a 6-parameter affine motion model). The motion vectors of the control points mentioned above can include the motion vectors of 2 or 3 control points.

[0611] For example, the two control points mentioned above can be the top-left and top-right control points of the current block. The three control points mentioned above can be the top-left, top-right, and bottom-left control points of the current block.

[0612] In some embodiments of this application, the motion vector of the affine is represented by the motion vector of the reference point and the rate of change of the motion vector.

[0613] In some embodiments of this application, the first type of candidates in the above-mentioned mixed candidate list includes translational motion vectors and affine motion vectors, and the second type of candidates in the above-mentioned mixed candidate list includes translational motion vectors and affine motion vectors.

[0614] In other words, for any candidate in the mixed candidate list, the candidate can include both translational motion vectors and affine motion vectors.

[0615] It should be noted that other descriptions of the translational motion vectors, affine motion vectors, and their representations can be found in the relevant descriptions in the above-described decoding method embodiments. To avoid repetition, they will not be repeated here.

[0616] In some embodiments of this application, the above-described mixed candidate list may include a single candidate.

[0617] In some embodiments of this application, the above-mentioned mixed candidate list may include multiple candidates. If the prediction mode is a preset prediction mode, then the first candidate is a candidate at a preset position among the multiple candidates.

[0618] It should be noted that further descriptions regarding the inclusion of one or more candidates in the aforementioned mixed candidate list can be found in the relevant descriptions in the above decoding method embodiments. To avoid repetition, these will not be repeated here.

[0619] Step 503: The encoder predicts the current block based on the prediction mode and the first candidate, and obtains the prediction block of the current block.

[0620] In some embodiments of this application, the prediction mode is merge mode. The encoder locates a reference in the reference image of the current block based on the motion vector in the first candidate, performs interpolation on the reference block, and uses the interpolated reference block as the prediction block of the current block.

[0621] In some embodiments of this application, the prediction mode is GPM, and the first candidate includes a first merging candidate and a second merging candidate. The encoder can first determine the weight derivation mode of the current block, and then determine the first prediction block and the second prediction block of the current block based on the weights derived from the weight derivation mode, the first motion vector in the first merging candidate and the second motion vector in the second merging candidate. Then, the first prediction block and the second prediction block are weighted according to the weights derived from the weight derivation mode to obtain the prediction block of the current block.

[0622] The encoding method provided in this application has the ability to include not only candidates of the first type, but also candidates of the second type, rather than only candidates of one type. Therefore, it is not necessary to write syntax elements in the bitstream to instruct the decoder to construct candidates of the corresponding type. This reduces the number of syntax elements transmitted in the bitstream, saves transmission resources, and improves encoding efficiency.

[0623] In some embodiments of this application, step 503 can be implemented by step 5031 as described below.

[0624] Step 5031: If the first index is less than or equal to the first threshold, the encoder predicts the current block based on the prediction mode and the first candidate, and obtains the prediction block of the current block.

[0625] In some embodiments of this application, the prediction mode described above may be MMVD mode or CIIP mode.

[0626] In some embodiments of this application, the first index described above may be used to indicate a first candidate.

[0627] In some embodiments of this application, since the mixed candidate list does not distinguish between translational and affine candidates, prediction modes such as MMVD and CIIP can also be applied to any candidate in the mixed candidate list. Furthermore, a first threshold can be set; if the value of mix_idx (i.e., the aforementioned first index) is less than or equal to the first threshold, the MMVD mode or CIIP mode can be used to predict the current block.

[0628] In some embodiments of this application, after step 503 above, the encoding method provided in the embodiments of this application may further include step 506 as described below.

[0629] Step 506: The encoder writes the syntax elements indicating the prediction mode into the bitstream.

[0630] In some embodiments of this application, the encoder may write a syntax element indicating the prediction mode into the bitstream, so that the decoder can determine the prediction mode by parsing the bitstream and subsequently use the prediction mode to predict the current block.

[0631] For example, assuming the prediction mode is merge mode, the syntax element indicating the prediction mode can be general_merge_flag, and the encoder can write this syntax element general_merge_flag into the bitstream.

[0632] In some embodiments of this application, after step 503 above, the encoding method provided in the embodiments of this application may further include step 507 below.

[0633] Step 507: The encoder writes the first index into the bitstream.

[0634] In some embodiments of this application, the first index described above may be used to indicate the first candidate.

[0635] In some embodiments of this application, after the encoder predicts the current block using the merge mode, it writes the first index into the bitstream. The decoder can then parse the bitstream to obtain the first index, thereby determining the first candidate from the mixed candidate list based on the first index.

[0636] In some embodiments of this application, the prediction mode is a geometric partitioning mode, and the first candidate includes a first merged candidate and a second merged candidate; after step 503, the encoding method provided in the embodiments of this application may further include the following step 508.

[0637] Step 508: The encoder writes the second and third indices into the bitstream.

[0638] In some embodiments of this application, the second index described above can be used to indicate a first merge candidate, and the second index described above can be used to indicate a second merge candidate.

[0639] In some embodiments of this application, after the encoder predicts the current block based on GPM, the first merging candidate, and the second merging candidate, it writes a second index indicating the first merging candidate and a third index indicating the second merging candidate into the bitstream so that the decoder, after receiving the bitstream, can parse the second index and the third index, and determine the first merging candidate from the mixed candidate list based on the second index, and determine the second merging candidate from the mixed candidate list based on the second index.

[0640] In some embodiments of this application, the prediction mode is a geometric partitioning mode. After step 503, the encoding method provided in the embodiments of this application may further include step 509.

[0641] Step 509: The encoder writes the first index into the bitstream.

[0642] In some embodiments of this application, the first index is used to indicate a first candidate, which includes a first motion vector for predicting the current block, and a second motion vector for predicting the current block is determined based on motion vector difference information.

[0643] In some embodiments of this application, when the motion information of the current block differs from that of surrounding blocks, and the motion information of the current block cannot be represented in any other way, the decoder can determine a second motion vector for predicting the current block based on the motion vector difference information. In other words, when the motion information in the candidates of the mixed candidate list is unsuitable as the motion information of the current block, the decoder can determine a second motion vector based on the motion vector difference information to predict the current block.

[0644] In some embodiments of this application, the aforementioned motion vector difference information can be obtained from the bitstream.

[0645] In some embodiments of this application, after the encoder predicts the current block using GPM and prediction mode with MVD, it writes a first index and a syntax element for indicating motion vector difference information into the bitstream so that the decoder can parse the first index and motion vector difference information after receiving the bitstream, determine a first candidate from the hybrid candidate list based on the first index, and then determine a first motion vector, and determine a second motion vector based on the motion vector difference information and motion vector prediction.

[0646] In some embodiments of this application, the encoder may write a first index and syntax elements for indicating motion vector difference information into the bitstream.

[0647] In some embodiments of this application, the amount of the aforementioned motion vector difference information is related to the motion model of the current block.

[0648] In some embodiments of this application, the motion model of the current block is a translational motion model, and the number of the above-mentioned motion vector difference information is 1.

[0649] In other words, if the motion model of the current block is a translational motion model, then an MVD is required in at least one prediction reference direction (such as RPL0 or RPL1) to generate accurate motion vectors.

[0650] In some embodiments of this application, the motion model of the current block is a 4-parameter affine motion model, and the number of the above-mentioned motion vector difference information is 2.

[0651] In other words, if the motion model of the current block is a 4-parameter affine motion model, then two MVDs are needed in at least one prediction reference direction to generate accurate motion vectors.

[0652] In some embodiments of this application, the motion model of the current block is a 6-parameter affine motion model, and the number of the aforementioned motion vector difference information is 3.

[0653] In other words, if the motion model of the current block is a 6-parameter affine motion model, then 3 MVDs are needed in at least one prediction reference direction to generate accurate motion vectors.

[0654] In some embodiments of this application, when the encoder constructs a hybrid candidate list, the encoding method provided in this application may further include the following steps 510 and 511.

[0655] Step 510: The encoder determines whether the first motion information and the second motion information are duplicated.

[0656] In some embodiments of this application, the first motion information mentioned above may be the motion information of the third surrounding block of the current block.

[0657] In some embodiments of this application, the aforementioned third peripheral block can be any peripheral block of the current block that has already been encoded using inter-frame prediction mode. It is understood that if a peripheral block of the current block is one that has already been encoded using inter-frame prediction mode, the motion information of that peripheral block is available. Therefore, the encoder, after determining that the first motion information is available, then determines whether the first motion information overlaps with the second motion information.

[0658] In some embodiments of this application, the second motion information described above may be motion information from candidates in a mixed candidate list.

[0659] In some embodiments of this application, the second motion information can be the motion information of the fourth surrounding block of the current block. This fourth surrounding block can be a surrounding block of the current block that has already been encoded using inter-frame prediction mode and whose motion information is available. In one example, the fourth surrounding block can be one or more surrounding blocks.

[0660] In some embodiments of this application, when the encoder finds a neighboring block of the current block, if it determines that the neighboring block has already been encoded using inter-frame prediction mode, it determines that the motion information of the neighboring block is available. Then, it determines whether the motion information of the neighboring block (e.g., the first motion information mentioned above) is duplicated with the motion information of other neighboring blocks that have been checked (e.g., the second motion information mentioned above). If they are duplicated, the value of the first variable corresponding to the motion information of the neighboring block is set to a first value, indicating that the motion information of the neighboring block cannot be used as a candidate. If they are not duplicated, the value of the first variable corresponding to the motion information of the neighboring block is marked as a second value, indicating that the motion information of the neighboring block can be used as a candidate. Then, the encoder continues to check whether the motion information of the next neighboring block found is available, and so on, until the motion information of all the neighboring blocks of the current block has been checked. Then, the motion information with the first variable value of the second value is filled into the mixed candidate list. It is understood that in some embodiments, the encoder may also set the value of the first variable corresponding to the motion information of a certain peripheral block to the second value and fill the motion information of the peripheral block into the mixed candidate list if it finds that the motion information of a certain peripheral block does not overlap with the motion information of other peripheral blocks that have been checked.

[0661] In one example, the first variable mentioned above could be available.

[0662] It should be noted that other descriptions regarding the first variable, the inability of motion information of a surrounding block to be used as a candidate, the possibility of motion information of a surrounding block being used as a candidate, the first value, and the second value can be found in the relevant descriptions of step 406 in the above embodiments. To avoid repetition, they will not be repeated here.

[0663] In one example, the encoder can check the motion information of all surrounding blocks in a preset order until a termination condition is met. One possible termination condition is that the mixing candidate list has reached a preset length, i.e., the mixing candidate list is full. Another possible termination condition is that all surrounding blocks have been checked. Furthermore, if all surrounding blocks have been checked but the mixing candidate list is not yet full, constructed candidates or default candidates can be added.

[0664] Step 511: If the first motion information and the second motion information do not overlap, the encoder constructs a hybrid candidate list based on the first motion information.

[0665] In one example of this application, if the first motion information and the second motion information do not overlap, it means that the first motion information can be used as a candidate, and the encoder can fill the first motion information into the mixed candidate list.

[0666] In other words, each time the encoder detects a motion information that can be used as a candidate, it fills that motion information into the mixed candidate list.

[0667] In another example of this application, if the first motion information and the second motion information do not overlap, it means that the first motion information can be used as a candidate. The encoder can set the value of the first variable corresponding to the first motion information to the second value, and after checking the motion information of the surrounding blocks of the current block, fill the first motion information into the mixed candidate list.

[0668] In other words, after the encoder checks the motion information of the surrounding blocks of the current block, it fills the motion information with the value of the first variable as the value of the second value into the mixed candidate list.

[0669] It should be noted that for other descriptions of steps 510 to 511 above, please refer to the relevant descriptions of steps 406 to 407 in the above decoding method embodiment. To avoid repetition, they will not be repeated here.

[0670] In some embodiments of this application, the "encoder determines whether the first motion information and the second motion information are repeated" in step 510 above can be specifically implemented by steps 5101 to 5103 below.

[0671] Step 5101: The encoder determines whether the first motion model corresponding to the first motion information is the same as the second motion model corresponding to the second motion information.

[0672] Step 5102: If the first motion model is different from the second motion model, the encoder determines that the first motion information and the second motion information are not repeated.

[0673] Step 5103: If the first motion model is the same as the second motion model, the encoder determines whether the first motion information and the second motion information are duplicated based on the information in the first motion information and the information in the second motion information.

[0674] In some embodiments of this application, the information in a motion information may include motion vectors, reference image indexes, flags indicating whether a reference image list is used, etc. The encoder can compare the motion vectors in the first motion information with the motion vectors in the second motion information, compare the reference image indexes in the first motion information with the reference image indexes in the second motion information, and compare the flags indicating whether a reference image list is used in the first motion information with the flags indicating whether a reference image list is used in the second motion information. If any comparison result is different, it can be determined that the first motion information and the second motion information are not duplicated; if all comparison results are the same, it can be determined that the first motion information and the second motion information are duplicated.

[0675] This avoids duplicate candidates in the constructed mixed candidate list, reducing resource waste.

[0676] It should be noted that either step 5102 or step 5103 above may be performed.

[0677] It should be noted that for other descriptions of steps 501 to 511, 5031, 5051 to 5054, and 5101 to 5103, as well as the effects that each step can achieve, please refer to the relevant descriptions of the above-described decoding method embodiments. To avoid repetition, they will not be repeated here.

[0678] Accordingly, this application provides a decoding device, which can be any decoding device in the video decoding process. Based on the above method example, the decoding device can be divided into functional modules. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; in actual implementation, other division methods may be used.

[0679] Figure 17 shows a possible structural diagram of the decoding device involved in the above embodiments, where each functional module is divided according to its corresponding function. As shown in Figure 17, the decoding device 170 includes a parsing module 1701 and a prediction module 1702.

[0680] The parsing module 1701 is used to parse the bitstream and determine the prediction mode of the current block. The prediction module 1702 is used to determine a first candidate, which is determined from a mixed candidate list, which includes at least one of a first type of candidate and a second type of candidate; and to predict the current block based on the prediction mode and the first candidate to obtain the prediction block of the current block.

[0681] In one possible implementation, the first type of candidate and the second type of candidate are determined based on different motion models.

[0682] In one possible implementation, the prediction module 1702 is further used to determine the motion model for predicting the surrounding blocks of the current block; and to construct a hybrid candidate list based on the motion model for predicting the surrounding blocks.

[0683] In one possible implementation, the prediction module 1702 is specifically used to construct a hybrid candidate list based on the translational motion information of the surrounding blocks if the surrounding blocks of the current block are predicted based on the translational motion model; and to construct a hybrid candidate list based on the affine motion information of the surrounding blocks if the surrounding blocks of the current block are predicted based on the affine motion model.

[0684] In one possible implementation, the prediction module 1702 is specifically used to predict the motion model based on the first surrounding block of the current block as a translational motion model, and add the translational motion information of the first surrounding block as a first type of candidate to the mixed candidate list; and to predict the motion model based on the second surrounding block of the current block as an affine motion model, and add the affine motion information derived from the affine motion information of the second surrounding block as a second type of candidate to the mixed candidate list.

[0685] In one possible implementation, the candidates of the first type mentioned above include one or more of the following: spatial motion information candidates, temporal motion information candidates, historical motion information candidates, and pairwise averaged motion information candidates.

[0686] The second type of candidate includes one or more of affine motion information candidates and sub-block-based temporal motion information candidates.

[0687] In one possible implementation, the aforementioned sub-block-based temporal motion information candidates are candidates at preset positions in the hybrid candidate list.

[0688] In one possible implementation, the first candidate is determined from the mixed candidate list based on a first index in the bitstream.

[0689] In one possible implementation, the prediction mode described above is a geometric partitioning mode, where the first candidate includes a first merged candidate and a second merged candidate. The first merged candidate is determined from the mixed candidate list based on a second index in the bitstream, and the second merged candidate is determined from the mixed candidate list based on a third index in the bitstream.

[0690] In one possible implementation, the prediction mode is a geometric partitioning mode, the first candidate is determined from the mixed candidate list based on the first index in the bitstream, the first candidate includes the first motion vector for predicting the current block, and the second motion vector for predicting the current block is determined based on the motion vector difference information.

[0691] In one possible implementation, the motion model corresponding to the candidates in the above-mentioned mixed candidate list is indicated by motion model variables.

[0692] In one possible implementation, the aforementioned motion model variables are elements in a motion model array, and each element in the motion model array corresponds one-to-one with a candidate in the hybrid candidate list.

[0693] In one possible implementation, the aforementioned hybrid candidate list includes a first list and a second list, wherein the first list includes motion model variables and the second list includes candidates corresponding to the motion models indicated by the motion model variables.

[0694] In one possible implementation, the candidates in the aforementioned mixed candidate list include motion model variables.

[0695] In one possible implementation, the above-mentioned mixed candidate list includes one candidate.

[0696] In one possible implementation, the above-mentioned mixed candidate list includes multiple candidates. If the prediction mode is a preset prediction mode, then the first candidate is a candidate at a preset position among the multiple candidates.

[0697] In one possible implementation, the first type of candidates in the above-mentioned mixed candidate list includes translational motion vectors; or, the second type of candidates in the above-mentioned mixed candidate list includes affine motion vectors; or, the first type of candidates in the above-mentioned mixed candidate list includes both translational and affine motion vectors, and the second type of candidates in the mixed candidate list includes both translational and affine motion vectors.

[0698] In one possible implementation, the motion vector of the affine motion is represented by a first indication information and the motion vector of the control point, wherein the first indication information is used to indicate the number of control points or the affine motion model; or, the motion vector of the affine motion is represented by the motion vector of the reference point and the rate of change of the motion vector.

[0699] In one possible implementation, the prediction module 1702 is further configured to determine whether the first motion information and the second motion information are duplicated; and if the first motion information and the second motion information are not duplicated, then a mixed candidate list is constructed based on the first motion information; wherein the first motion information is the motion information of the third surrounding block of the current block; and the second motion information is the motion information of the candidate in the mixed candidate list, or the motion information of the fourth surrounding block of the current block.

[0700] In one possible implementation, the prediction module 1702 is specifically used to determine whether the first motion model corresponding to the first motion information is the same as the second motion model corresponding to the second motion information; and if the first motion model is different from the second motion model, then determine that the first motion information and the second motion information do not overlap; and if the first motion model is the same as the second motion model, then determine whether the first motion information and the second motion information overlap based on the information in the first motion information and the information in the second motion information.

[0701] In one possible implementation, the prediction module 1702 is specifically used to predict the current block based on the prediction mode and the first candidate if the first index in the bitstream is less than or equal to the first threshold, so as to obtain the predicted block of the current block. The prediction mode is MMVD mode or CIIP mode, and the first candidate is determined from the mixed candidate list based on the first index.

[0702] This application provides a decoding apparatus that, after parsing and determining the prediction mode, can construct a hybrid candidate list. This hybrid candidate list not only has the ability to include candidates of the first type, but also the ability to include candidates of the second type, rather than only having the ability to include candidates of one type. That is, it is not necessary to construct a candidate list of a certain type based on the indication of the syntax elements in the bitstream. Therefore, the construction of the hybrid candidate list does not need to be indicated by transmitting the corresponding syntax elements in the bitstream, thus reducing the syntax elements transmitted in the bitstream, saving transmission resources, and improving encoding efficiency.

[0703] Each module of the above-mentioned decoding device can also be used to perform other actions in the above-mentioned method embodiments. All relevant content of each step involved in the above-mentioned method embodiments can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0704] Figure 18 illustrates another possible structural diagram of the decoding device involved in the above embodiments, when using integrated units. As shown in Figure 18, the decoding device 180 provided in this application embodiment may include a processing module 1801 and a communication module 1802. The processing module 1801 can be used to control and manage the operation of the decoding device. For example, the processing module 1801 can be used to support the decoding device in executing the various steps in the embodiments of the above decoding method, and / or other processes used in the technology described herein. The communication module 1802 can be used to support communication between the decoding device and other network entities, such as communication with an encoding device. In one example, as shown in Figure 18, the decoding device may also include a storage module 1803 for storing the program code and data of the decoding device, such as storing the decoded image.

[0705] The processing module 1801 can be a processor, such as the decoder 122 in Figure 1. The communication module 1802 can be a transceiver, transceiver circuit, or communication interface, such as the communication interface 121 in Figure 1. The storage module 1803 can be a memory.

[0706] For more details on how the modules included in the above decoding device implement the above functions, please refer to the descriptions in the previous method embodiments, which will not be repeated here.

[0707] Accordingly, this application provides an encoding device, which can be any encoding device in the video encoding process. Based on the above method example, the encoding device can be divided into functional modules. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; in actual implementation, other division methods may be used.

[0708] With each functional module divided according to its corresponding function, Figure 19 shows a possible structural schematic diagram of the encoding device involved in the above embodiments. As shown in Figure 19, the encoding device 190 includes a prediction module 1901.

[0709] The prediction module 1901 described above is used to determine the prediction mode of the current block; and to determine a first candidate, which is determined from a mixed candidate list, which includes at least one of a first type of candidate and a second type of candidate; and to predict the current block based on the prediction mode and the first candidate to obtain the prediction block of the current block.

[0710] In one possible implementation, the first type of candidate and the second type of candidate are determined based on different motion models.

[0711] In one possible implementation, the prediction module 1901 is further used to determine the motion model for predicting the surrounding blocks of the current block; and to construct a hybrid candidate list based on the motion model for predicting the surrounding blocks.

[0712] In one possible implementation, the prediction module 1901 is specifically used to construct a hybrid candidate list based on the translational motion information of the surrounding blocks if the surrounding blocks of the current block are predicted based on the translational motion model; and to construct a hybrid candidate list based on the affine motion information of the surrounding blocks if the surrounding blocks of the current block are predicted based on the affine motion model.

[0713] In one possible implementation, the prediction module 1901 is specifically used to predict the motion model based on the first surrounding block of the current block as a translational motion model, and add the translational motion information of the first surrounding block as a first type of candidate to the mixed candidate list; and to predict the motion model based on the second surrounding block of the current block as an affine motion model, and add the affine motion information derived from the affine motion information of the second surrounding block as a second type of candidate to the mixed candidate list.

[0714] In one possible implementation, the first type of candidates includes one or more of spatial motion information candidates, temporal motion information candidates, history-based motion information candidates, and pairwise averaged motion information candidates; the second type of candidates includes one or more of affine motion information candidates and sub-block-based temporal motion information candidates.

[0715] In one possible implementation, the temporal motion information candidates based on sub-blocks are candidates at preset positions in the mixed candidate list.

[0716] In one possible implementation, referring to Figure 19 and as shown in Figure 20, the encoding device 190 may further include an encoding module 1902, which is used to write a first index into the bit stream, the first index being used to indicate a first candidate.

[0717] In one possible implementation, the prediction mode is a geometric partitioning mode, and the first candidate includes a first merging candidate and a second merging candidate; the encoding module 1902 is also used to write a second index and a third index into the bitstream, the second index being used to indicate the first merging candidate and the third index being used to indicate the second merging candidate.

[0718] In one possible implementation, the prediction mode is a geometric partitioning mode; the above-mentioned encoding module 1902 is also used to write the first index into the code stream, the first index is used to indicate the first candidate, the first candidate includes the first motion vector for predicting the current block, and the second motion vector for predicting the current block is determined based on the motion vector difference information.

[0719] In one possible implementation, the motion model corresponding to the candidates in the mixed candidate list is indicated by motion model variables.

[0720] In one possible implementation, the motion model variables are elements in the motion model array, and each element in the motion model array corresponds one-to-one with a candidate in the hybrid candidate list.

[0721] In one possible implementation, the hybrid candidate list includes a first list and a second list, the first list including motion model variables and the second list including candidates corresponding to the motion models indicated by the motion model variables.

[0722] In one possible implementation, the candidates in the mixed candidate list include motion model variables.

[0723] In one possible implementation, the mixed candidate list includes a single candidate.

[0724] In one possible implementation, the mixed candidate list includes multiple candidates. If the prediction mode is a preset prediction mode, then the first candidate is a candidate at a preset position among the multiple candidates.

[0725] In one possible implementation, the first type of candidates in the mixed candidate list includes translational motion vectors; or, the second type of candidates in the mixed candidate list includes affine motion vectors; or, the first type of candidates in the mixed candidate list includes both translational and affine motion vectors, and the second type of candidates in the mixed candidate list includes both translational and affine motion vectors.

[0726] In one possible implementation, the affine motion vector is represented by a first indication information and the motion vector of the control point, wherein the first indication information is used to indicate the number of control points or the affine motion model; or, the affine motion vector is represented by the motion vector of the reference point and the rate of change of the motion vector.

[0727] In one possible implementation, the prediction module 1901 is further configured to determine whether the first motion information and the second motion information are duplicated; and if the first motion information and the second motion information are not duplicated, then a mixed candidate list is constructed based on the first motion information; wherein the first motion information is the motion information of the third surrounding block of the current block; and the second motion information is the motion information of the candidate in the mixed candidate list, or the motion information of the fourth surrounding block of the current block.

[0728] In one possible implementation, the prediction module 1901 is specifically used to determine whether the first motion model corresponding to the first motion information is the same as the second motion model corresponding to the second motion information; and if the first motion model is different from the second motion model, then determine that the first motion information and the second motion information do not overlap; and if the first motion model is the same as the second motion model, then determine whether the first motion information and the second motion information overlap based on the information in the first motion information and the information in the second motion information.

[0729] In one possible implementation, the prediction module 1901 is specifically used to predict the current block based on the prediction mode and the first candidate if the first index is less than or equal to the first threshold, so as to obtain the predicted block of the current block. The prediction mode is MMVD mode or CIIP mode, and the first index is used to indicate the first candidate.

[0730] This application provides an encoding device. Since the hybrid candidate list has the ability to include not only candidates of the first type but also candidates of the second type, and is not limited to the ability to include only one type of candidate, it is not necessary to write syntax elements in the bitstream to instruct the decoder to construct candidates of the corresponding type. This reduces the number of syntax elements transmitted in the bitstream, saves transmission resources, and improves encoding efficiency.

[0731] Each module of the above-mentioned encoding device can also be used to perform other actions in the above-mentioned method embodiments. All relevant content of each step involved in the above-mentioned method embodiments can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0732] In the case of using integrated units, FIG21 shows another possible structural schematic diagram of the encoding device involved in the above embodiments. As shown in FIG21, the encoding device 200 provided in this application embodiment may include: a processing module 2001 and a communication module 2002. The processing module 2001 can be used to control and manage the operation of the encoding device. For example, the processing module 2001 can be used to support the encoding device in executing the various steps in the embodiments of the above encoding method, and / or other processes for the technology described herein. The communication module 2002 can be used to support the communication of the encoding device with other network entities, such as communication with the encoding device. In one example, as shown in FIG21, the encoding device may further include a storage module 2003 for storing the program code and data of the encoding device, such as storing encoded images or bitstreams.

[0733] The processing module 2001 can be a processor, such as the encoder 112 in Figure 1. The communication module 2002 can be a transceiver, transceiver circuit, or communication interface, such as the communication interface 121 in Figure 1. The storage module 2003 can be a memory.

[0734] For more details on how the modules included in the above encoding device implement the above functions, please refer to the descriptions in the preceding method embodiments, which will not be repeated here.

[0735] It should be noted that the scope of the methods and apparatus in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0736] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0737] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0738] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0739] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0740] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0741] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0742] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

A decoding method characterized by comprising: include: Analyze the bitstream to determine the prediction mode for the current block; A first candidate is determined from a mixed candidate list, which includes at least one of a first type of candidate and a second type of candidate; Based on the prediction pattern and the first candidate, the current block is predicted to obtain the predicted block of the current block. The method of claim 1, wherein The first type of candidate and the second type of candidate are determined based on different motion models. The method of claim 1, wherein Also includes: Determine the motion model of the surrounding blocks of the current block for prediction; The hybrid candidate list is constructed based on the motion model predicted by the surrounding blocks. The method according to claim 3, characterized in that The motion model that makes predictions based on the surrounding blocks constructs the hybrid candidate list, including: If the surrounding blocks of the current block are predicted based on the translational motion model, then the hybrid candidate list is constructed based on the translational motion information of the surrounding blocks; If the surrounding blocks of the current block are predicted based on an affine motion model, then the hybrid candidate list is constructed based on the affine motion information of the surrounding blocks. The method according to claim 3 or 4, characterized in that The motion model that makes predictions based on the surrounding blocks constructs the hybrid candidate list, including: The motion model predicted based on the first surrounding block of the current block is a translational motion model, and the translational motion information of the first surrounding block is added to the hybrid candidate list as the first type of candidate. The motion model predicted based on the second surrounding block of the current block is an affine motion model. The affine motion information derived from the affine motion information of the second surrounding block is added to the mixed candidate list as the second type of candidate. The method according to claim 1, characterized in that, The first type of candidate includes one or more of the following: spatial motion information candidate, temporal motion information candidate, historical motion information candidate, and pairwise average motion information candidate. The second type of candidate includes one or more of affine motion information candidates and sub-block-based temporal motion information candidates. The method according to claim 6, characterized in that The sub-block-based temporal motion information candidates are candidates at preset positions in the hybrid candidate list. The method of claim 1, wherein The first candidate is determined from the mixed candidate list based on a first index in the bitstream. The method of claim 1, wherein The prediction mode is a geometric partitioning mode. The first candidate includes a first merged candidate and a second merged candidate. The first merged candidate is determined from the mixed candidate list based on a second index in the bitstream, and the second merged candidate is determined from the mixed candidate list based on a third index in the bitstream. The method of claim 1, wherein The prediction mode is a geometric partitioning mode. The first candidate is determined from the mixed candidate list based on the first index in the bitstream. The first candidate includes a first motion vector for predicting the current block, and a second motion vector for predicting the current block is determined based on motion vector difference information. The method of claim 1, wherein The motion model corresponding to the candidates in the hybrid candidate list is indicated by the motion model variable. The method of claim 11, wherein The motion model variables are elements in the motion model array, and each element in the motion model array corresponds one-to-one with a candidate in the hybrid candidate list. The method according to claim 11 or 12, characterized in that The hybrid candidate list includes a first list and a second list, wherein the first list includes the motion model variable and the second list includes candidates corresponding to the motion model indicated by the motion model variable. The method of claim 11, wherein The candidates in the hybrid candidate list include the motion model variables. The method of claim 1, wherein The mixed candidate list includes one candidate. The method of claim 1, wherein The hybrid candidate list includes multiple candidates. If the prediction mode is a preset prediction mode, then the first candidate is a candidate at a preset position among the multiple candidates. The method according to claim 1, characterized in that, The first type of candidates in the hybrid candidate list includes translational motion vectors; or... The second type of candidate in the hybrid candidate list includes affine motion vectors; or... The first type of candidates in the hybrid candidate list includes translational motion vectors and affine motion vectors, and the second type of candidates in the hybrid candidate list includes translational motion vectors and affine motion vectors. The method according to claim 17, characterized in that, The affine motion vector is represented by a first indication information and the motion vectors of the control points, wherein the first indication information is used to indicate the number of control points or the affine motion model; or... The affine motion vector is represented by the motion vector of the reference point and the rate of change of the motion vector. The method of claim 1, wherein The method further includes: Determine whether the first motion information and the second motion information are duplicates; wherein, the first motion information is the motion information of the third surrounding block of the current block; the second motion information is the motion information of the candidate in the mixed candidate list, or the motion information of the fourth surrounding block of the current block; If the first motion information and the second motion information do not overlap, then the hybrid candidate list is constructed based on the first motion information. The method of claim 19, wherein Determining whether the first motion information and the second motion information are duplicated includes: Determine whether the first motion model corresponding to the first motion information is the same as the second motion model corresponding to the second motion information; If the first motion model is different from the second motion model, then it is determined that the first motion information and the second motion information are not duplicated; If the first motion model is the same as the second motion model, then based on the information in the first motion information and the information in the second motion information, it is determined whether the first motion information and the second motion information are duplicated. The method of claim 1, wherein The step of predicting the current block based on the prediction mode and the first candidate to obtain the predicted block of the current block includes: If the first index in the bitstream is less than or equal to the first threshold, then the current block is predicted based on the prediction mode and the first candidate to obtain the predicted block of the current block. The prediction mode is MMVD mode or CIIP mode, and the first candidate is determined from the mixed candidate list based on the first index. An encoding method characterized by comprising: include: Determine the prediction mode for the current block; A first candidate is determined from a mixed candidate list, which includes at least one of a first type of candidate and a second type of candidate; Based on the prediction pattern and the first candidate, the current block is predicted to obtain the predicted block of the current block. The method of claim 22, wherein The first type of candidate and the second type of candidate are determined based on different motion models. The method of claim 22, wherein Also includes: Determine the motion model of the surrounding blocks of the current block for prediction; The hybrid candidate list is constructed based on the motion model predicted by the surrounding blocks. The method of claim 24, wherein The motion model that makes predictions based on the surrounding blocks constructs the hybrid candidate list, including: If the surrounding blocks of the current block are predicted based on the translational motion model, then the hybrid candidate list is constructed based on the translational motion information of the surrounding blocks; If the surrounding blocks of the current block are predicted based on an affine motion model, then the hybrid candidate list is constructed based on the affine motion information of the surrounding blocks. The method according to claim 24 or 25, characterized in that The motion model that makes predictions based on the surrounding blocks constructs the hybrid candidate list, including: The motion model predicted based on the first surrounding block of the current block is a translational motion model, and the translational motion information of the first surrounding block is added to the hybrid candidate list as the first type of candidate. The motion model predicted based on the second surrounding block of the current block is an affine motion model. The affine motion information derived from the affine motion information of the second surrounding block is added to the mixed candidate list as the second type of candidate. The method according to claim 22, characterized in that, The first type of candidate includes one or more of the following: spatial motion information candidate, temporal motion information candidate, historical motion information candidate, and pairwise average motion information candidate. The second type of candidate includes one or more of affine motion information candidates and sub-block-based temporal motion information candidates. The method of claim 27, wherein The sub-block-based temporal motion information candidates are candidates at preset positions in the hybrid candidate list. The method of claim 22, wherein The method further includes: Write the first index into the bitstream; the first index is used to indicate the first candidate. The method of claim 22, wherein The prediction mode is a geometric partitioning mode, and the first candidate includes a first merged candidate and a second merged candidate; The method further includes: The second index and the third index are written into the bitstream, the second index being used to indicate the first merge candidate and the third index being used to indicate the second merge candidate. The method of claim 22, wherein The prediction mode is a geometric partitioning mode; The method further includes: A first index is written into the bitstream. The first index is used to indicate the first candidate. The first candidate includes a first motion vector for predicting the current block. A second motion vector for predicting the current block is determined based on motion vector difference information. The method of claim 22, wherein The motion model corresponding to the candidates in the hybrid candidate list is indicated by the motion model variable. The method of claim 32, wherein The motion model variables are elements in the motion model array, and each element in the motion model array corresponds one-to-one with a candidate in the hybrid candidate list. The method according to claim 32 or 33, characterized in that The hybrid candidate list includes a first list and a second list, wherein the first list includes the motion model variable and the second list includes candidates corresponding to the motion model indicated by the motion model variable. The method of claim 32, wherein The candidates in the hybrid candidate list include the motion model variables. The method of claim 22, wherein The mixed candidate list includes one candidate. The method of claim 22, wherein The hybrid candidate list includes multiple candidates. If the prediction mode is a preset prediction mode, then the first candidate is a candidate at a preset position among the multiple candidates. The method according to claim 22, characterized in that, The first type of candidates in the hybrid candidate list includes translational motion vectors; or... The second type of candidate in the hybrid candidate list includes affine motion vectors; or... The first type of candidates in the hybrid candidate list includes translational motion vectors and affine motion vectors, and the second type of candidates in the hybrid candidate list includes translational motion vectors and affine motion vectors. The method according to claim 38, characterized in that, The affine motion vector is represented by a first indication information and the motion vectors of the control points, wherein the first indication information is used to indicate the number of control points or the affine motion model; or... The affine motion vector is represented by the motion vector of the reference point and the rate of change of the motion vector. The method of claim 22, wherein The method further includes: Determine whether the first motion information and the second motion information are duplicates; wherein, the first motion information is the motion information of the third surrounding block of the current block; the second motion information is the motion information of the candidate in the mixed candidate list, or the motion information of the fourth surrounding block of the current block; If the first motion information and the second motion information do not overlap, then the hybrid candidate list is constructed based on the first motion information. The method of claim 40, wherein Determining whether the first motion information and the second motion information are duplicated includes: Determine whether the first motion model corresponding to the first motion information is the same as the second motion model corresponding to the second motion information; If the first motion model is different from the second motion model, then it is determined that the first motion information and the second motion information are not duplicated; If the first motion model is the same as the second motion model, then based on the information in the first motion information and the information in the second motion information, it is determined whether the first motion information and the second motion information are duplicated. The method of claim 22, wherein The step of predicting the current block based on the prediction mode and the first candidate to obtain the predicted block of the current block includes: If the first index is less than or equal to the first threshold, then the current block is predicted based on the prediction mode and the first candidate to obtain the predicted block of the current block. The prediction mode is MMVD mode or CIIP mode, and the first index is used to indicate the first candidate. A decoding device characterized by comprising: It includes a parsing module and a prediction module; The parsing module is used to parse the bitstream and determine the prediction mode of the current block; The prediction module is used to determine a first candidate, which is determined from a mixed candidate list, the mixed candidate list including at least one of a first type of candidate and a second type of candidate; Furthermore, based on the prediction mode and the first candidate, the current block is predicted to obtain the predicted block of the current block. An encoding device characterized by comprising: Includes a prediction module; The prediction module is configured to determine the prediction mode of the current block; and to determine a first candidate, which is determined from a mixed candidate list, the mixed candidate list including at least one of a first type of candidate and a second type of candidate; Furthermore, based on the prediction mode and the first candidate, the current block is predicted to obtain the predicted block of the current block. A decoder, characterized in that It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the decoding method as described in any one of claims 1 to 21. An encoder characterized by It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the encoding method as described in any one of claims 22 to 42. A readable storage medium characterized by, The readable storage medium stores a bitstream, which is a bitstream encoded based on the encoding method described in any one of claims 22 to 42. A readable storage medium characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the decoding method as described in any one of claims 1 to 21, or the steps of the encoding method as described in any one of claims 22 to 42. A computer program product stored in a storage medium, the computer program product being executed by at least one processor to implement the steps of the decoding method as claimed in any one of claims 1 to 21, or to implement the steps of the encoding method as claimed in any one of claims 22 to 42.

Images