An encoder, a decoder and corresponding methods for inter prediction

By employing a prediction method with triangle and geometric modes and sub-block-based techniques, the method addresses the challenge of achieving high compression ratios with minimal quality loss in video coding, enhancing efficiency and quality in digital video applications.

KR103045416B1Active Publication Date: 2026-11-06HUAWEI TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
KR1020247005335
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-09-24
Filing Date
2019-11-22
Publication Date
2026-11-06
Estimated Expiration
2039-11-22

AI Technical Summary

Technical Problem

Existing video coding technologies face challenges in achieving high compression ratios with minimal loss in image quality, particularly in scenarios with limited network bandwidth and memory resources.

Method used

The implementation of a prediction method for image blocks that includes parsing and adjusting prediction indices, utilizing triangle and geometric prediction modes, and employing sub-block-based prediction techniques to optimize index coding and reduce redundancy.

Benefits of technology

This approach enhances coding efficiency by minimizing redundancy and improving compression ratios without sacrificing image quality, making it suitable for various digital video applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112024017747826-PAT00052_ABST
    Figure 112024017747826-PAT00052_ABST
Patent Text Reader

Abstract

A prediction method for an image block comprising a first prediction sub-block and a second prediction sub-block comprises: a step of parsing a first index from a bitstream, wherein the first index is used to obtain prediction information of the first prediction sub-block; a step of parsing a second index from the bitstream; a step of comparing the first index with the second index; a step of adjusting the second index at an event where the second index is greater than or equal to the first index; and a step of obtaining prediction information of the second prediction sub-block according to the adjusted second index.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention claims priority to U.S. Provisional Application No. 62 / 770,826 filed November 22, 2018, U.S. Provisional Application No. 62 / 787,678 filed January 2, 2019, U.S. Provisional Application No. 62 / 816,897 filed March 11, 2019, and U.S. Provisional Application No. 62 / 905,367 filed September 24, 2019, the contents of which are incorporated herein by reference in their entirety.

[0002] The embodiments of the present invention generally relate to the field of image processing, and in particular to inter prediction. Background Technology

[0003] Video coding (video encoding and decoding) is used in a wide range of digital video applications, such as broadcast digital TV, video transmission over the Internet and mobile networks, video chatting, video conferencing, DVD and Blu-ray discs, video content acquisition and editing systems, and real-time conversation applications like camcorders in security applications.

[0004] The amount of video data required to depict a relatively short video can be substantial, which can cause difficulties when data is streamed or transmitted over communication networks with limited bandwidth. Therefore, video data is typically compressed before being transmitted over modern communication networks. Since memory resources may be limited, the size of the video can also become an issue when it is stored on a storage device. Video compression devices often use software and / or hardware at the source to code video data before transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is received at the destination by a video decompression device that decodes the video data. As network resources remain limited and the demand for higher video quality continues to grow, improved compression and decompression technologies that enhance compression ratios with little to no sacrifice in image quality are desirable.

[0005] Embodiments of the present invention provide an apparatus and method for encoding and decoding according to independent claims.

[0006] In the first aspect of the present application, a prediction method for an image block comprises, wherein the current block includes a first prediction sub-block and a second prediction sub-block, and the prediction method comprises: a step of parsing a first index from a bitstream, wherein the first index is used to obtain prediction information of the first prediction sub-block; a step of parsing a second index from the bitstream; a step of comparing the first index with the second index; a step of adjusting the second index at an event where the second index is greater than or equal to the first index; and a step of obtaining prediction information of the second prediction sub-block according to the adjusted second index.

[0007] In a possible implementation, the step of adjusting the second index includes: increasing the second index by m, where m is a positive integer.

[0008] In a possible implementation, m is 1.

[0009] In a possible implementation, prior to the step of parsing a first index from a bitstream, the prediction method further comprises the step of parsing at least one index to determine the prediction mode of the current block, wherein the prediction mode is a triangle prediction mode or a geometric prediction mode. The prediction mode may be other sub-block-based prediction modes, including a rectangular or non-rectangular (trapezoidal) mode. The triangle prediction mode and the geometric prediction mode may be combined into a single prediction mode, which may also be included in an executable implementation.

[0010] In a possible implementation, the prediction method further includes the step of obtaining a candidate list for the current block.

[0011] In a possible implementation, the prediction information of the first prediction sub-block is obtained from a candidate list according to the first index.

[0012] In a possible implementation, the prediction information of the second prediction sub-block is obtained from the candidate list according to the adjusted second index.

[0013] In a possible implementation, the above candidate list is a candidate list for Merge Mode.

[0014] In a possible implementation, the prediction method further comprises: a step of parsing a first number to determine a maximum allowed candidate index from the candidate list; and a step of obtaining a maximum index based on the maximum allowed candidate index, wherein the first index is not greater than the maximum index.

[0015] In a possible implementation, the step of obtaining a maximum index based on the maximum allowed candidate index includes: obtaining the maximum index by calculating between the maximum allowed candidate index and a predetermined number.

[0016] In a possible implementation, the step of obtaining a maximum index based on the maximum allowed candidate index comprises: a step of parsing a second number to derive the difference between the maximum allowed candidate index and the maximum index; and a step of obtaining the maximum index by calculating between the maximum allowed candidate index and the difference.

[0017] In a possible implementation, the prediction method further includes the step of parsing a third number to determine the maximum index.

[0018] In a possible implementation, the maximum allowed candidate index is not smaller than the maximum index.

[0019] In a possible implementation, after the step of obtaining prediction information of the second prediction sub-block according to the adjusted second index, the prediction method further includes the step of obtaining a prediction value of the current block based on one of the prediction information of the first prediction sub-block and the prediction information of the second prediction sub-block.

[0020] In a possible implementation, the first index or the second index is binaryized according to a truncated unary code.

[0021] In a possible implementation, the first bin of the binary-coded first index or second index is coded using the normal coding mode of CABAC.

[0022] In a possible implementation, the non-first bin of the binarized first index or second index is coded using the bypass coding mode of CABAC.

[0023] In a possible implementation, a step of parsing a direction indicator from the bitstream—the direction indicator is used to indicate the splitting direction of the current block.

[0024] In a second aspect of the present application, an inter-prediction method for an image block comprises: acquiring a prediction indicator; determining whether the prediction indicator indicates that a sub-block prediction will be applied to the block; acquiring two different indicators when the prediction indicator indicates that a sub-block prediction will be applied to the block—wherein the two different indicators represent two distinct entries in a motion information candidate list for two sub-blocks within the block; and performing an inter-prediction for the block based on the two different indicators.

[0025] In an executable implementation, the step of obtaining two different indicators includes: obtaining two initial indicators including an initial first indicator and an initial second indicator; comparing the initial second indicator with the initial first indicator; adjusting the initial second indicator to obtain an updated second indicator when the initial second indicator is greater than or equal to the initial first indicator—the updated second indicator is different from the initial first indicator—; and determining the initial first indicator and the updated second indicator as two different indicators.

[0026] In a possible implementation, the step of adjusting the initial second indicator to obtain the updated second indicator further includes the step of increasing the initial second indicator by m, where m is a predefined number and is preferably set to 1.

[0027] In an executable implementation, the method further includes the step of setting the maximum value of the initial first indicator to M and the maximum value of the initial second indicator to Mm, where M is not greater than N and N is the size of the motion information candidate list.

[0028] In an executable implementation, N is a positive integer determined based on the indicator signaled in the received bitstream.

[0029] In an executable implementation, the method comprises: comparing the size of a block with a specified threshold; if the size of the block is not greater than the specified threshold, setting the maximum value of an initial first indicator to M and the maximum value of an initial second indicator to Mm, wherein M is not greater than N and N is the size of the motion information candidate list; if the size of the block is greater than the specified threshold, setting the maximum value of an initial first indicator to P and the maximum value of an initial second indicator to Pm, wherein P is greater than M and not greater than N, and N is the size of the motion information candidate list.

[0030] In the executable implementation, here M and P are predefined positive integers.

[0031] In a feasible implementation, if P is determined to be greater than N, P is updated to be equal to N, or if M is determined to be greater than N, M is updated to be equal to N.

[0032] In a feasible implementation, the method further comprises: the step of obtaining a single indicator when the prediction indicator indicates that a sub-block prediction will not be applied to the block—wherein the single indicator represents an entry in the merge candidate list for the block—; and the step of performing an inter-prediction for the block based on said single indicator.

[0033] In a feasible implementation, the method further includes the step of setting the maximum value of the initial first indicator to M, where N is the size of the merge candidate list that is not identical to the motion information candidate list.

[0034] In a feasible implementation, the method further includes a step of determining a splitting direction indicator, and the splitting direction indicator indicates the splitting direction for the block.

[0035] In a feasible implementation, the step of obtaining two different indicators comprises: a step of adjusting an initial second indicator to obtain an updated second indicator when the division direction indicator indicates a first division direction, wherein the updated second indicator is different from the initial first indicator; and a step of determining the initial first indicator and the updated second indicator as two different indicators; or further comprising a step of adjusting the initial first indicator to obtain an updated first indicator when the division direction indicator indicates a second division direction, wherein the updated first indicator is different from the initial second indicator; and a step of determining the updated first indicator and the initial second indicator as two different indicators.

[0036] In an executable implementation, the method comprises: selecting motion information from a list of motion information candidates based on two different indicators; and performing a sub-block prediction for the current block based on the selected motion information.

[0037] In an executable implementation, the method comprises: selecting a first motion information and a second motion information from a list of motion information candidates according to two different indicators; performing a sub-block prediction for a first sub-block based on the first motion information; and performing a sub-block prediction for a second sub-block based on the second motion information.

[0038] In an executable implementation, the first sub-block is assigned to the part where the geometric center is closer to the left boundary of the current block.

[0039] In a feasible implementation, the method further includes the step of binarizing two different indices according to a truncated unary code.

[0040] In an executable implementation, the method comprises: coding a first bin of an index of two different indices using a coding mode of context adaptive binary arithmetic coding (CABAC); and coding another bin of an index of two different indices using a bypass mode of CABAC.

[0041] In a third aspect of the present application, the current block comprises a first sub-unit and a second sub-unit, and includes the step of parsing a first indicator—wherein the first indicator is a pattern of the current block decoding method for the block used to determine the partition; the step of parsing a second indicator and a third indicator; the step of determining prediction information of the first sub-unit based on the value of the second indicator; the step of determining the value of the third indicator—wherein if the value of the third indicator is greater than or equal to the second indicator, the value of the third indicator is added by a target value—; and the step of determining prediction information of the second sub-unit based on the determined value of the third indicator.

[0042] In a feasible implementation, the maximum allowable value of the second indicator is M, and the maximum allowable value of the third indicator is Mm, where M is a positive integer and m is a preset positive integer.

[0043] In a feasible implementation, the number of entries in the list of predicted information candidates is N.

[0044] In a possible implementation, here M is less than or equal to N.

[0045] In a viable implementation, here M is greater than or equal to N.

[0046] In a feasible implementation, a step of parsing a fourth indicator is further included, said fourth indicator is used to display the value of N.

[0047] In possible implementations, here N is a predetermined value.

[0048] In a feasible implementation, a step of parsing a fifth indicator is further included, where the fifth indicator is used to display the value of M.

[0049] In an executable implementation, the value of M is determined by the value of N.

[0050] In a fourth aspect of the present application, an inter-prediction device is provided, wherein the current block comprises a first prediction sub-block and a second prediction sub-block, and the inter-prediction device comprises: a parsing module configured to parse a first index from a bitstream, wherein the first index is used to obtain prediction information of the first prediction sub-block; and to parse a second index from the bitstream; a locating module configured to compare the first index with the second index; an adjustment module configured to adjust the second index at an event where the second index is greater than or equal to the first index; and an acquisition module configured to obtain prediction information of the second prediction sub-block according to the adjusted second index.

[0051] In an executable implementation, the locating module is configured to increase the second index by m, where m is a positive integer.

[0052] In a viable implementation, m is 1.

[0053] In an executable implementation, before parsing a first index from a bitstream, the parsing module is further configured to parse at least one index to determine the prediction mode of the current block, wherein the prediction mode is a triangle prediction mode or a geometric prediction mode. The prediction mode may be other sub-block-based prediction modes, including a rectangular or non-rectangular (trapezoidal) mode. The triangle prediction mode and the geometric prediction mode may be combined into a single prediction mode, which may also be included in the executable implementation.

[0054] In an executable implementation, the locating module is further configured to obtain a list of candidates for the current block.

[0055] In an executable implementation, the prediction information of the first prediction sub-block is obtained from a candidate list according to the first index.

[0056] In an executable implementation, the prediction information of the second prediction sub-block is obtained from the candidate list according to the adjusted second index.

[0057] In an executable implementation, the above candidate list is a candidate list for Merge Mode.

[0058] In an executable implementation, the parsing module is further configured to: parse the first number to determine the maximum allowed candidate index from the candidate list; and obtain the maximum index based on the maximum allowed candidate index, wherein the first index is not greater than the maximum index.

[0059] In an executable implementation, the parsing module is configured to obtain the maximum index by calculating between the maximum allowed candidate index and a predetermined number.

[0060] In an executable implementation, the parsing module is configured to: parse a second number to derive the difference between the maximum allowed candidate index and the maximum index; and obtain the maximum index by calculating the difference between the maximum allowed candidate index and the maximum index.

[0061] In an executable implementation, the parsing module is configured to parse a third number to determine the maximum index.

[0062] In an executable implementation, the maximum allowed candidate index is not smaller than the maximum index.

[0063] In an executable implementation, after obtaining prediction information of the second prediction sub-block according to the adjusted second index, the acquisition module is further configured to obtain a prediction value of the current block based on one of the prediction information of the first prediction sub-block and the prediction information of the second prediction sub-block.

[0064] In an executable implementation, the first index or the second index is binaryized according to a truncated unary code.

[0065] In an executable implementation, the first bin of the binary-coded first index or second index is coded using the regular coding mode of CABAC.

[0066] In an executable implementation, the non-first bin of the binarized first index or second index is coded using the bypass coding mode of CABAC.

[0067] In an executable implementation, the parsing module is configured to parse a direction indicator from the bitstream, and the direction indicator is used to indicate the splitting direction of the current block.

[0068] In the fifth aspect of the present application, a computer program product is provided and includes program code for performing a method according to any one of the first to fourth aspects when executed on a computer or processor.

[0069] In a sixth aspect of the present application, a decoder is provided, the decoder comprises: one or more processors; and a non-transient computer-readable storage medium coupled to said processors and storing a program for execution by said processors, said program being configured such that when executed by said processors, the decoder performs a method according to any one of the first to fourth aspects.

[0070] In the seventh aspect of the present application, an encoder is provided, the encoder comprises: one or more processors; and a non-transient computer-readable storage medium coupled to said processors and storing a program for execution by said processors, wherein the encoder is configured to perform a method according to any one of the first to fourth aspects when the program is executed by said processors.

[0071] In the eighth aspect of the present application, a non-transient computer-readable medium is provided and includes program code that, when executed by a computer device, causes the computer device to perform any one of the methods of the first to fourth aspects.

[0072] The above and other objectives are achieved by the subject matter of the independent claim. Additional forms of implementation are apparent from the dependent claims, description, and drawings.

[0073] Certain embodiments are described in the appended independent claims, and other embodiments are set forth in the dependent claims.

[0074] Detailed information on one or more embodiments is described in the attached drawings and the description below. Other features, purposes, and benefits will become apparent from the description, drawings, and claims.

[0075] In this application, regarding the triangle prediction mode, it is noted that redundancy exists when two prediction sub-blocks within the block have the same prediction information, compared to a prediction mode where the entire block integrates prediction information. To avoid redundancy, this application designs a prediction index coding method. Bits for signaling the prediction index are stored, and coding efficiency is improved. Furthermore, the maximum amount of candidate prediction information for the triangle prediction mode is derived based on the maximum amount of candidate prediction information for the merge mode. Coding bits are also stored and are signaled independently by comparing them with the maximum amount of candidate prediction information for the triangle prediction mode. Brief explanation of the drawing

[0076] In the following embodiments, embodiments of the present invention are described in more detail with reference to the attached drawings. FIG. 1a is a block diagram showing an example of a video coding system configured to implement an embodiment of the present invention. FIG. 1b is a block diagram showing another example of a video coding system configured to implement embodiments of the present invention. FIG. 2 is a block diagram illustrating an example of a video encoder configured to implement an embodiment of the present invention. FIG. 3 is a block diagram illustrating an exemplary structure of a video decoder configured to implement an embodiment of the present invention. Figure 4 is a block diagram showing an example of an encoding device or a decoding device. Figure 5 is a block diagram showing another example of an encoding device or a decoding device. Figure 6 is a diagram showing an example of the location of spatial merger candidates. Figure 7 is a diagram showing an example of a pair of candidates considered for checking the overlap of spatial merge candidates. Figure 8 is a diagram showing an example of motion vector scaling for temporal merge candidates. Figure 9 is a diagram showing an example of a location for a temporal candidate. Figure 10a is an example of dividing a block into two triangle prediction units. Figure 10b is another example of dividing a block into two triangle prediction units. Figure 11 is an example of another sub-block partitioning method. Figure 12 is an example of deriving a single predicted motion vector from a list of merge candidates. Figure 13 is an example of a block with a blending filter applied. Figure 14 is a schematic block diagram illustrating the process of CABAC. Figure 15 is a block diagram showing an example of a prediction method. Figure 16 is a block diagram showing another example of a prediction method. FIG. 17 is a block diagram showing an example of a prediction device configured to implement embodiments of the present application. FIG. 18 is a block diagram showing an example of an encoding device or a decoding device. FIG. 19 is a block diagram illustrating an exemplary structure of a content supply system (3100) that realizes a content transmission service. FIG. 20 is a block diagram showing the structure of an example of a terminal device. In the following identical reference numerals, it represents the same or at least functionally equivalent function unless otherwise explicitly specified. Specific details for implementing the invention

[0077] In the following description, reference is made to the accompanying drawings, which form part of the present disclosure and, by way of example, illustrate specific aspects of embodiments of the present invention or specific aspects in which embodiments of the present invention may be used. It is understood that embodiments of the present invention may be used in other aspects and may include structural or logical variations not illustrated in the drawings. Accordingly, the following detailed description should not be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0078] For example, it is understood that a disclosure relating to a described method may also be true for a corresponding device or system configured to perform the method, and vice versa. For example, where one or more specific method steps are described, the device may include one or more units. It may include a functional unit for performing the described one or more method steps (e.g., one unit performing one or more steps, or multiple units each performing one or more of the multiple steps), and such one or more units may not be explicitly described or illustrated in the drawings. Meanwhile, for example, where a specific device is described based on one or more units, for example, a corresponding method in a functional unit may include a step that performs the function of one or more units (e.g., one step that performs the function of one or more units, or multiple steps each performing one or more functions), and such one or more steps may not be explicitly described or illustrated in the drawings. Furthermore, it is understood that features of various exemplary embodiments and / or aspects described herein may be combined with one another unless specifically stated otherwise.

[0079] Video coding generally refers to the processing of a series of pictures that form a video or video sequence. The terms "frame" or "image" may be used as synonyms in the field of video coding instead of the term "video." Video coding (or coding in general) consists of two parts: video encoding and video decoding. Video encoding is performed at the source side and typically involves processing the original video image to reduce the amount of data required to represent the video image (e.g., by compression) for more efficient storage and / or transmission. Video decoding is performed at the destination side and generally involves reverse processing relative to the encoder to reconstruct the video image. Examples referring to the "coding" of a video image (or image in general) should be understood to relate to the "encoding" or "decoding" of the video image or each video sequence. The combination of the encoding and decoding parts is also referred to as a Codec (Coding and Decoding, CODEC).

[0080] In the case of lossless video coding, the original video image can be reconstructed. That is, the reconstructed video image has the same quality as the original video image (assuming there is no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, additional compression is performed, for example by quantization, to reduce the amount of data representing the video image that cannot be fully reconstructed by the decoder; that is, the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0081] Several video coding standards belong to the "lossy hybrid video codec" group (i.e., combining spatial and temporal prediction in the sample domain with 2D transformation coding to apply quantization in the transformation domain). Each image in a video sequence is typically divided into a set of non-overlapping blocks, and coding is typically performed at the block level. In other words, in the encoder, video is typically encoded at the block (video block) level, for example, by generating a prediction block using spatial (intra-image) prediction and / or temporal (inter-image) prediction, obtaining a residual block by subtracting the prediction block from the current block (currently processed / block to be processed), transmitting the residual block to reduce the amount of data to be transmitted (compression), and encoding by quantizing the residual block in the transformation domain; on the other hand, in the decoder, compared to the encoder, inverse processing is applied to the encoded or compressed block to reconstruct the current block. Furthermore, the encoder will duplicate the decoder processing loop to generate a reconstruction for both to process the same predictions (e.g., intra and inter predictions) and / or subsequent blocks, i.e., for coding.

[0082] In the following embodiment of the video coding system (10), the video encoder (20) and the video decoder (30) are described based on FIGS. 1 to 3.

[0083] FIG. 1a is a schematic block diagram illustrating, for example, an exemplary coding system (10), a video coding system (10) (or abbreviated as coding system (10)) capable of utilizing the technology of the present application. The video encoder (20) (or abbreviated as encoder (20)) and video decoder (30) (or abbreviated as decoder (30)) of the video coding system (10) represent examples of devices that may be configured to perform the technology according to various examples described in the present application.

[0084] As illustrated in FIG. 1a, the coding system (10) includes, for example, a source device (12) configured to provide encoded image data (21). The encoded image data (13) is transmitted to a destination device (14) for decoding.

[0085] The source device (12) includes an encoder (20) and additionally, i.e. optionally, may include, for example, a video source (16), a preprocessor (or preprocessing unit) (18), for example, a video preprocessor (18), and a communication interface or communication unit (22).

[0086] The image source (16) may include any type of image capture device, e.g., a camera for capturing real images, and / or any type of image generation device, e.g., a computer graphics processor for generating computer animations, and / or real images, computer-generated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage for storing any of the aforementioned images.

[0087] In distinction from the processing performed by the preprocessor (18) and the preprocessing unit (18), the image or image data (17) may also be referred to as a raw image or raw image data (17).

[0088] The free processor (18) is configured to receive (original) image data (17) and perform preprocessing on the image data (17) to obtain a preprocessed image (19) or preprocessed image data (19). The preprocessing performed by the preprocessor (18) may include, for example, trimming, color format conversion (e.g., from RGB to YCbCr), color correction, or noise removal. It may be understood that the preprocessing unit (18) may be an optional component.

[0089] A video encoder (20) is configured to receive preprocessed video data (19) and provide encoded video data (21) (details will be explained below, for example, based on FIG. 2).

[0090] The communication interface (22) of the source device (12) may be configured to receive encoded image data (21) and transmit the encoded image data (21) (or any additionally processed version thereof) to another device, for example, a destination device (14) for storage or direct reconstruction, or any other device through the communication channel (13).

[0091] The receiving device (14) includes a decoder (30) (e.g., a video decoder (30)) and additionally, i.e. optionally, may include a communication interface or communication unit (28), a post-processor (32) (or a post-processing unit (32)) and a display device (34).

[0092] The communication interface (28) of the destination device (14) is configured to receive, for example, encoded image data (21) (or any additionally processed version thereof), receive it, for example, directly from the source device (12) or from another source, for example, a storage device, for example, an encoded image data storage device, and provide the encoded image data (21) to the decoder (30).

[0093] The communication interface (22) and the communication interface (28) may be configured to transmit or receive encoded video data (21) or encoded data (13) through, for example, a direct communication link between a source device (12) and a destination device (14), for example, a direct wired or wireless connection or any kind of network, for example, a wired or wireless network or a combination thereof, or any kind of private and public network, or any kind of combination thereof.

[0094] The communication interface (22) may be configured to process the encoded video data (21) by, for example, packaging the encoded video data (21) into a suitable format, for example, a packet, and / or using any kind of transmission encoding or processing for transmission through a communication link or communication network.

[0095] A communication interface (28) forming a counterpart to a communication interface (22) may be configured, for example, to receive transmitted data and process the transmitted data using any kind of corresponding transmission decoding or processing and / or depackaging to obtain encoded image data (21).

[0096] Both the communication interface (22) and the communication interface (28) may be configured as a unidirectional communication interface or a bidirectional communication interface indicated by the arrow for the communication channel (13) of FIG. 1a pointing from the source device (12) to the destination device (14), for example, sending and receiving messages, for example, establishing a connection, and verifying and exchanging other information related to communication links and / or data transmission, for example, transmission of encoded video data.

[0097] The decoder (30) is configured to receive encoded image data (21) and provide decoded image data (31) or decoded image (31) (more details will be described later, for example, based on FIG. 3 or FIG. 5).

[0098] The post-processor (32) of the destination device (14) post-processes, for example, decoded image data (31) (also referred to as reconstructed image data), for example, the decoded image (31), to obtain post-processed image data (33), for example, the post-processed image (33). The post-processing performed by the post-processing unit (32) may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, trimming or resampling, or any other processing to prepare the decoded image data (31) for, for example, a display, for example, a display device (34).

[0099] The display device (34) of the destination device (14) is configured to receive post-processed image data (33) to display, for example, an image to a user or viewer. The display device (34) may include any type of display, integrated or external display or monitor for displaying, for example, a reconstructed image. The display is composed of, for example, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0100] FIG. 1a illustrates the source device (12) and the destination device (14) as separate devices, but embodiments of the device may include the source device (12) or the corresponding function and the destination device (14) or the corresponding function, or both. In such embodiments, the source device (12) or the corresponding function and the destination device (14) or the corresponding function may be implemented by the same hardware and / or software, or by separate hardware and / or software, or any combination thereof.

[0101] As is obvious to those skilled in the art based on the description, the presence and (precise) division of different units or functions within the source device (12) and / or target device (14), as illustrated in FIG. 1a, may vary depending on the actual device and application.

[0102] The encoder (20) (e.g., video encoder (20)) or decoder (30) (e.g., video decoder (30)) or both the encoder (20) and the decoder (30) may be implemented through one or more processing circuits, such as a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), discrete logic, hardware, dedicated to video coding, or a combination thereof, as illustrated in FIG. 1b. The encoder (20) may be implemented through a processing circuit (46) and / or any other encoder system or subsystem described herein to implement various modules, as discussed in relation to the encoder (20) of FIG. 2. The decoder (30) may be implemented through a processing circuit (46) and / or any other decoder system or subsystem described herein to implement various modules, as discussed in relation to the decoder (30) of FIG. 3. The processing circuit may be configured to perform various operations as described below. As illustrated in FIG. 5, where the technology is partially implemented in software, the device may store instructions for the software in a suitable non-transient computer-readable storage medium and execute instructions in hardware using one or more processors to perform the technology of the present disclosure. One of the video encoder (20) and the video decoder (30) may be integrated as part of an encoder / decoder (CODEC) combined in a single device, for example, as illustrated in FIG. 1b.

[0103] The source device (12) and the destination device (14) are any type of handheld or fixed device, for example, a laptop or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (for example, a content service server or a content transmission server), a broadcast receiver device, a broadcast transmitter device, etc., and may not use an operating system at all or of any type. In some cases, the source device (12) and the destination device (14) may be equipped for wireless communication. Thus, the source device (12) and the destination device (14) may be wireless communication devices.

[0104] In some cases, the video coding system (10) illustrated in FIG. 1a is merely an example, and the technology of the present application may be applied to video coding setups (e.g., video encoding or video decoding) that do not necessarily involve any data communication between encoding and decoding devices. In other examples, data is retrieved from local memory or streamed over a network, etc. A video encoding device may encode data and store it in memory, and / or a video decoding device may retrieve data from memory and decode it. In some examples, encoding and decoding are performed by devices that simply encode data into memory and / or retrieve and decode data from memory without communicating with each other.

[0105] For convenience of explanation, embodiments of the present invention are described herein with reference, for example, to High-efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), and the next-generation video coding standard was developed by the Joint Cooperation Team for Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG). Those skilled in the art will understand that embodiments of the present invention are not limited to HEVC or VVC.

[0106] Encoder and Encoding Method

[0107] FIG. 2 illustrates a schematic block diagram of an exemplary video encoder (20) configured to implement the technology of the present application. In the example of FIG. 2, the video encoder (20) includes an input (201) (or input interface (201)), a residual calculation unit (204), a transformation processing unit (206), a quantization unit (208), an inverse quantization unit (210) and an inverse transformation processing unit (212), a reconstruction unit (214), a loop filter unit (220), a decoded image buffer (DPB) (230), a mode selection unit (260), an entropy encoding unit (270), and an output (272) (or output interface (272)). The mode selection unit (260) may include an inter prediction unit (244), an intra prediction unit (254), and a splitting unit (262). The inter prediction unit (244) may include a motion estimation unit and a motion compensation unit (not shown). The video encoder (20) illustrated in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder based on a hybrid video codec.

[0108] The residual calculation unit (204), the conversion processing unit (206), the quantization unit (208), and the mode selection unit (260) may be referred to as forming the forward signal path of the encoder (20), while the inverse quantization unit (210), the inverse conversion processing unit (212), the reconstruction unit (214), the buffer (216), the loop filter (220), the decoded image buffer (DPB) (230), the inter prediction unit (244), and the intra prediction unit (254) may be referred to as forming the reverse signal path of the video encoder, wherein the reverse signal path of the video encoder (20) corresponds to the signal path of the decoder (see video decoder (30) in FIG. 3). The inverse quantization unit (210), inverse transform processing unit (212), reconstruction unit (214), loop filter (220), decoding picture buffer (DPB) (230), inter prediction unit (244) and intra prediction unit (254) are also referred to as forming the "built-in decoder" of the video encoder (20).

[0109] Image and image segmentation (image and block)

[0110] The encoder (20) may be configured to receive, for example, an image (17) (or image data 17), for example, a series of images forming a video or video sequence, via an input (201). The received image or image data may also be a preprocessed image (19) (or preprocessed image data (19)). For simplicity, the following description refers to the image (17). The image (17) may also be referred to as the current image or the image to be coded (particularly in video coding to distinguish the current image from other images, for example, the same video sequence, for example, previously encoded and / or decoded images of a video sequence that also includes the current image).

[0111] A (digital) image can be considered as a two-dimensional array or sample matrix containing intensity values. Samples in an array are also called pixels (an abbreviation for image elements) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or image defines the size and / or resolution of the image. For color representation, three color elements are typically used; that is, the image may be represented or contain three sample arrays. In the RGB format or color space, an image consists of corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in the luminance and chroma format or color space, e.g., YCbCr, which includes a luminance component denoted by Y (sometimes L is used instead) and two chroma components denoted by Cb and Cr. The luminance (or abbreviated as lumin) component Y represents brightness or gray level intensity (as in, for example, a grayscale image), while the two chroma (or abbreviated as chroma) components Cb and Cr represent chromaticity or color information components. Therefore, an image in YCbCr format consists of a luminance sample array of luminance sample values ​​(Y) and two chroma sample arrays of chroma values ​​(Cb and Cr). An image in RGB format can be converted or switched to YCbCr format, and vice versa. This process is also called color conversion or conversion. If the image is monochromatic, the image may contain only a luminance sample array. Thus, the image may be, for example, a luminance sample array in a monochromatic format or a luminance sample array and two corresponding chroma sample arrays in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0112] An embodiment of the video encoder (20) may include a video splitting unit (not shown in FIG. 2) configured to split a video (17) into a plurality of (typically non-overlapping) video blocks (203). These blocks may also be referred to as root blocks, macro blocks (H.264 / AVC) or coding tree blocks (CTB) or coding tree units (CTU) (H.265 / HEVC and VVC). The video splitting unit may be configured to use the same block size for all videos in a video sequence and a corresponding grid that defines the block size, or to change the block size between videos, subsets, or groups of videos and split each video into the corresponding blocks.

[0113] In an additional embodiment, the video encoder may be configured to directly receive, for example, a block (203) of the image (17), for example, one, several, or all blocks forming the image (17). The image block (203) may also be referred to as the current image block to be coded or the image block.

[0114] As with the image (17), the image block (203) may be considered as a two-dimensional array or a matrix of samples having intensity values ​​(sample values), although having a smaller dimension than the image (17). In other words, the block (203) may include, for example, one sample array (e.g., a luminance array for the grayscale image (17), a luminance or chroma array for the color image (17)) or three sample arrays (e.g., a luminance and two chroma arrays for the color image (17)) or different numbers and / or types of arrays depending on the applied color format. The number of samples in the horizontal and vertical directions (or axes) of the block (203) defines the size of the block (203). Thus, the block may be, for example, an MxN (M-column x N-row) array of samples or an MxN array of transformation factors.

[0115] An embodiment of the video encoder (20) as illustrated in FIG. 2 may be configured to encode the image (17) in blocks, for example, and, for example, encoding and prediction are performed for each block (203).

[0116] An embodiment of the video encoder (20) as illustrated in FIG. 2 may also be configured to divide and / or encode an image using slices (also called video slices), wherein the image may be divided into one or more slices or encoded using one or more slices (typically non-overlapping), and each slice may include one or more blocks (e.g., CTU) or one or more groups of blocks (e.g., tiles (H.265 / HEVC and VVC) or bricks (VVC)).

[0117] An embodiment of the video encoder (20) as illustrated in FIG. 2 may be further configured to divide and / or encode an image using slice / tile groups (also called video tile groups) and / or tiles (also called video tiles), wherein the image may be divided or encoded using one or more slice / tile groups (typically non-overlapping), and each slice / tile group may include, for example, one or more blocks (e.g., CTUs) or one or more tiles, and each tile may be, for example, rectangular in shape and may include one or more blocks (e.g., CTUs), for example, complete or fractional blocks.

[0118] Residual calculation

[0119] The residual calculation unit (204) may be configured to calculate a residual block (205) (also referred to as residual (205)) based on, for example, an image block (203) and a prediction block (265) (further details regarding the prediction block (265) will be provided later), and obtain the residual block (205) in the sample domain, for example, by subtracting the sample value of the prediction block (265) from the sample value of the image block (203).

[0120] conversion

[0121] The transformation processing unit (206) may be configured to apply a transformation, for example, a discrete cosine transform (DCT) or a discrete sine transform (DST), to sample values ​​of the residual block (205) to obtain transformation coefficients (207) in the transformation domain. The transformation coefficients (207) may also be referred to as transformation residual coefficients and may represent the residual block (205) in the transformation domain.

[0122] The conversion processing unit (206) may be configured to apply integer approximations of DCT / DST, such as the conversion specified for H.265 / HEVC. Compared to the orthogonal DCT conversion, these integer approximations are generally adjusted by specific factors. Additional scaling factors are applied as part of the conversion process to preserve the norm of the residual blocks processed by the forward and inverse conversions: the scaling factors are selected based on specific constraints, such as a scaling factor that is a power of 2 for the shift operation, the bit depth of the conversion factor, and a balance between accuracy and implementation cost. Specific scaling factors are specified, for example, for the inverse conversion, for example by the inverse conversion processing unit (212) (and by the corresponding inverse conversion, for example by the inverse conversion processing unit (312) of the video decoder (30)), and the corresponding scaling factor for the forward conversion in the encoder (20), for example by the conversion processing unit (206), may be specified accordingly.

[0123] An embodiment of the video encoder (20) (each conversion processing unit (206)) may be configured to output conversion parameters, for example, directly or through the entropy encoding unit (270) to output a type of encoded or compressed conversion or conversions so that the video decoder (30) can receive and use the conversion parameters for decoding.

[0124] Quantization

[0125] The quantization unit (208) may be configured to quantize the transformation coefficient (207) to obtain the quantized coefficient (209) by applying, for example, scalar quantization or vector quantization. The quantized coefficient (209) may also be referred to as the quantized transformation coefficient (209) or the quantized residual coefficient (209).

[0126] The quantization process can reduce the bit depth associated with some or all of the conversion factors (207). For example, an n-bit conversion factor can be rounded to an m-bit conversion factor during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, in the case of scalar quantization, different scaling may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. Applicable quantization step sizes can be denoted by the quantization parameter (QP). The quantization parameter may be, for example, an index to a predefined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to fine quantization (small quantization step size) and a large quantization parameter may correspond to coarser quantization (large quantization step size), or vice versa. Quantization may involve division by the quantization step size, and, for example, the corresponding and / or inverse quantization by the inverse quantization unit (210) may involve multiplication by the quantization step size. An embodiment according to some standards, for example HEVC, may be configured to use quantization parameters to determine the quantization step size. Generally, the quantization step size may be calculated based on the quantization parameters using a fixed-point approximation of an expression involving division. Additional scaling factors may be introduced in the quantization and inverse quantization to restore the standard of the residual block, which may be modified by the scaling used in the fixed-point approximation of the equation for the quantization step size and the quantization parameters. In one exemplary implementation, the scaling of the inverse transform and inverse quantization may be combined. Alternatively, a custom quantization table may be used to send the signal from the encoder to the decoder as a bitstream. Quantization is a lossy operation, and the loss increases as the quantization step size increases.

[0127] An embodiment of the video encoder (20) (each quantization unit (208)) may be configured to output quantization parameters (QP), for example, directly or through an entropy encoding unit (270) so that the video decoder (30) can receive and apply the quantization parameters for decoding.

[0128] Inverse quantization

[0129] The inverse quantization unit (210) is configured to apply the inverse quantization of the quantization unit (208) to the quantized coefficients in order to obtain the inverse quantized coefficients (211) by applying the inverse of the quantization method applied by the quantization unit (208) based on, for example, the same quantization step size as the quantization unit (208) or using it. The inverse quantized coefficients (211) may also be referred to as inverse quantized residual coefficients (211) and may correspond to the transformation coefficients (207)—typically not identical to the transformation coefficients due to loss by quantization.

[0130] Inverse transformation

[0131] The inverse transform processing unit (212) is configured to obtain a reconstructed residual block (213) (or a corresponding inverse quantized coefficient (213)) in the sample domain by applying an inverse transform of the transform applied by the transform processing unit (206), for example, an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST) or another inverse transform. The reconstructed residual block (213) may also be referred to as the transform block (213).

[0132] Reconstruction

[0133] A reconstruction unit (214) (e.g., an adder or summer (214)) is configured to obtain a reconstruction block (215) in a sample domain by adding a transformation block (213) (i.e., a reconstruction residual block (213)) to a prediction block (265) by adding, for example, a sample value of a reconstruction residual block (213) and a sample value of a prediction block (265) in sample units.

[0134] Filtering

[0135] A loop filter unit (220) (or abbreviated as "loop filter" (220)) is configured to filter a reconstructed block (215) to obtain a filtered block (221), or generally to filter a reconstructed sample to obtain a filtered sample value. The loop filter unit is configured, for example, to smooth pixel transitions or improve video quality. The loop filter unit (220) may include one or more loop filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, or one or more other filters, for example, an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination thereof. In one example, the loop filter unit (220) may include a deblocking filter, an SAO filter, and an ALF filter. The order of the filtering process may be the deblocking filter, SAO, and ALF. In another example, a process called lumina mapping with chroma scaling (LMCS) (i.e., an adaptive in-loop reshaper) is added. This process is performed before deblocking. In another example, the deblocking filter process may also be applied to internal sub-block edges, for example, affine sub-block edges, ATMVP sub-block edges, sub-block transformation (SBT) edges, and internal sub-partition (ISP) edges. Although the loop filter unit (220) is shown in FIG. 2 as an in-loop filter, in other configurations, the loop filter unit (220) may be implemented as a post-loop filter. The filtered block (221) may also be referred to as a filtered reconstructed block (221).

[0136] An embodiment of the video encoder (20) (each loop filter unit (220)) may be configured to output loop filter parameters (e.g., SAO filter parameters or ALF filter parameters or LMCS parameters) directly or through an entropy encoding unit (270) so that the decoder (30) can receive and apply the same loop filter parameters or each loop filter for decoding.

[0137] Decoded video buffer

[0138] The decoded image buffer (DPB) (230) may be a memory that stores a reference image or generally reference image data for encoding video data by the video encoder (20). The DPB (230) may be formed by various memory devices such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The decoded image buffer (DPB) (230) may be configured to store one or more filtered blocks (221). The decoded image buffer (230) may be further configured to store, for example, other previously filtered blocks, for example, previously reconstructed and filtered blocks (221) of the same current image or different images, and may provide, for example, a completely previously reconstructed, i.e., decoded image (and corresponding reference blocks and samples) and / or a partially reconstructed current image (and corresponding reference blocks and samples) for inter-prediction. The decoded image buffer (DPB) (230) may also be configured to store one or more unfiltered reconstructed blocks (215), or generally, for example, unfiltered reconstructed samples, where the reconstructed block (215) is not filtered by the loop filter unit (220) or other additional processed versions of the reconstructed block or sample.

[0139] Mode Selection (Partition and Prediction)

[0140] The mode selection unit (260) includes a partitioning unit (262), an inter-prediction unit (244), and an intra-prediction unit (254) and is configured to receive or acquire, for example, original image data, and is configured to receive or acquire, for example, original block (203) (current block (203) of the current image (17)) and reconstructed image data, for example, filtered and / or unfiltered reconstructed samples or blocks of the same (current) image and / or one or more previously decoded images from, for example, a decoded image buffer (230) or another buffer (e.g., a line buffer, not shown). The reconstructed image data is used as reference image data for, for example, prediction, inter-prediction, or intra-prediction to acquire a prediction block (265) or a predictor (265).

[0141] The mode selection unit (260) may be configured to determine or select partitioning for a current block prediction mode (including no partition) and a prediction mode (e.g., intra or inter prediction mode), and to generate a corresponding prediction block (265) used therein, the prediction block (265) being used for the calculation of residual blocks (205) and the reconstruction of reconstructed blocks (215). An embodiment of the mode selection unit (260) may be configured to select a partitioning and prediction mode (e.g., from those supported or available by the mode selection unit (260)) which provides the best match, that is, the minimum residual (the minimum residual means better compression for transmission or storage), or the minimum signaling overhead (the minimum signaling overhead means better compression for transmission or storage), or this considers both or balances them. The mode selection unit (260) may be configured to determine the split and prediction modes based on rate distortion optimization (RDO), that is, to select the prediction mode that provides the minimum rate distortion. In this context, terms such as “best,” “minimum,” “optimal,” etc. do not necessarily refer to the entire “best,” “minimum,” “optimal,” etc., but selection criteria such as values ​​that exceed or fall below a termination or threshold or other constraints potentially lead to “below-optimal selection” but reduce complexity and processing time.

[0142] In other words, the partitioning unit (262) may be configured to divide the image of the video sequence into a sequence of coding tree units (CTU), and the CTU (203) may be further divided into smaller block partitions or sub-blocks (which again form blocks) by repeatedly using, for example, quad-tree-partitioning (QT), binary partitioning (BT), or triple-tree-partitioning (TT), or any combination thereof, and may perform, for example, prediction for each block partition or sub-block, and the mode selection includes the selection of the tree structure of the partitioned block (203), and the prediction mode is applied to each block partition or sub-block.

[0143] In the following, the segmentation and prediction processing (by the inter-prediction unit (244) and intra-prediction unit (254)) performed by the exemplary video encoder (20) (e.g. by the segmentation unit (260)) will be described in more detail.

[0144] Partitioning

[0145] The partitioning unit (262) may be configured to partition images from a video sequence into a sequence of coding tree units (CTUs), and the partitioning unit (262) may partition (or split) the coding tree units (CTUs) (203) into smaller partitions, for example, small blocks of square or rectangular size. For an image with three sample arrays, the CTU consists of N x N blocks of luminance samples along with two corresponding blocks of chroma samples. The maximum allowable size of the luminance blocks in the CTU is specified as 128 x 128 in the Multipurpose Video Coding for Development (VVC), but may be specified as a value other than 128 x 128, for example, 256 x 256 in the future. The CTUs of the image may be clustered / grouped into slice / tile groups, tiles, or bricks. A tile covers a rectangular area of ​​the image, and a tile may be divided into one or more bricks. A brick consists of multiple rows of CTUs within a tile. A tile that is not divided into multiple bricks can be called a brick. However, a brick is a true subset of a tile and is not referred to as a tile. VVC supports two tile group modes: raster-scan slice / tile group mode and rectangular slice mode. In raster-scan tile group mode, a slice / tile group contains a series of tiles within a tile raster scan of the image. In rectangular slice mode, a slice contains multiple bricks of the image that collectively form a rectangular area of ​​the image. The bricks within a rectangular slice are in the order of the brick raster scans of the slice. These small blocks (also called subblocks) can be further divided into smaller partitions.This is also called tree partitioning or hierarchical tree partitioning, where the root block, for example, at root tree level 0 (hierarchy level 0, depth 0), can be recursively partitioned into, for example, two or more blocks of nodes at the next subtree level, for example, tree level 1 (hierarchy level 1, depth 1), where these blocks can be further partitioned into two or more blocks at the next lower level, for example, tree level 2 (hierarchy level 2, depth 2), and so on, until the partitioning ends, for example, because a termination criterion is met, for example, because the maximum tree depth or minimum block size is reached. Blocks that are no longer partitioned are also called leaf blocks or leaf nodes of the tree. A tree partitioned into two partitions is called a binary tree (BT), a tree partitioned into three partitions is called a ternary tree (TT), and a tree partitioned into four partitions is called a quad tree (QT).

[0146] For example, a coding tree unit (CTU) may be or may include a CTB composed of luminance samples, two corresponding CTBs composed of chroma samples of an image having three sample arrays, or a CTB composed of samples of an image coded using three distinct color planes and syntax structures used to code a monochrome image or samples. Correspondingly, a coding tree block (CTB) may be an NxN block of samples for some values ​​of N such that partitioning the components into CTBs is partitioning. A coding unit (CU) may be or may include a coding block composed of luminance samples, two corresponding coding blocks composed of chroma samples of an image having three sample arrays, or a coding block composed of samples of an image coded using three distinct color planes and syntax structures used to code a monochrome image or samples. Correspondingly, a coding block (CB) may be an MxN block of samples for some values ​​of M and N such that partitioning the CTB into coding blocks is partitioning.

[0147] For example, according to HEVC, a Coding Tree Unit (CTU) can be divided into CUs using a quadtree structure represented as a coding tree. Whether to code image regions using inter-image (temporal) or intra-image (spatial) prediction is determined at the leaf CU level. Each leaf CU can be further divided into one, two, or four PUs depending on the PU division type. Within a single PU, the same prediction process is applied, and the relevant information is transmitted to the decoder at the PU level. After obtaining the remaining blocks by applying the prediction process according to the PU division type, the leaf CU can be divided into Transform Units (TUs) according to another quadtree structure similar to the coding tree for the CU.

[0148] In the embodiments, for example, according to the latest Video Coding Standard (VVC) currently under development, a combined quad tree nested multi-type tree coding tree unit using binary and ternary splitting structures used for splitting, for example. In the coding tree structure within the coding tree unit, the CU can have a square or rectangular shape. For example, the coding tree unit (CTU) is first split by a quaternary tree. Then, the quaternary tree leaf nodes can be further split into a multi-type tree structure. The multi-type tree structure has four splitting types: vertical binary splitting (SPLIT_BT_VER), horizontal binary splitting (SPLIT_BT_HOR), vertical ternary splitting (SPLIT_TT_VER), and horizontal ternary splitting (SPLIT_TT_HOR). The multi-type tree leaf nodes are called coding units (CUs), and if the CU is not too large compared to the maximum transform length, this splitting is used for prediction and transform processing without further splitting. This means that in most cases, CUs, PUs, and TUs have the same block size in the quadtree using a nested multi-type tree coding block structure. An exception occurs when the maximum supported transformation length is smaller than the width or height of the CU's color component. VVC develops a unique signaling mechanism for partition splitting information in the quadtree using a nested multi-type tree coding tree structure. In the signaling mechanism, the Coding Tree Unit (CTU) is treated as the root of a 4th-order tree and is first split by the 4th-order tree structure. Then, each 4th-order tree leaf node (when large enough to be allowed) is further split by the multi-type tree structure.In a multi-type tree structure, a first flag (mtt_split_cu_flag) is signaled to indicate whether the node is further split; if the node is further split, a second flag (mtt_split_cu_vertical_flag) is signaled to indicate the direction of the split, and then a third flag (mtt_split_cu_binary_flag) is signaled to indicate whether the split is a binary split or a binary split. Based on the values ​​of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree splitting mode (MttSplitMode) of the CU can be derived by the decoder based on predefined rules or tables. For specific designs, such as the 64x64 luminance block and 32x32 chroma pipelining design of the VVC hardware decoder, TT splitting is prohibited if the width or height of the luminance coding block is greater than 64, as shown in FIG. 6. TT partitioning is prohibited even if the width or height of the chroma coding block is greater than 32. Pipelining designs partition the image into Virtual Pipeline Data Units (VPDUs), which are defined as non-overlapping units within the image. In hardware decoders, consecutive VPDUs are processed simultaneously across multiple pipeline stages. Since the VPDU size is nearly proportional to the buffer size in most pipeline stages, it is important to keep the VPDU size small. In most hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in VVC, the VPDU size can increase due to ternary tree (TT) and binary tree (BT) partitioning.

[0149] In addition, it should be noted that if part of a tree node block extends beyond the bottom or right image boundary, the tree node block is forcibly split until all samples of all coded CUs are located within the image boundary.

[0150] For example, the Intra Sub-Partitions (ISP) tool can divide the Luma Intra Prediction block into two or four sub-partitions vertically or horizontally depending on the block size.

[0151] In one example, the mode selection unit (260) of the video encoder (20) may be configured to perform any combination of the splitting techniques described herein.

[0152] As described above, the video encoder (20) is configured to determine or select the best or optimal prediction mode from a set of prediction modes (e.g., predetermined). The set of prediction modes may include, for example, intra-prediction modes and / or inter-prediction modes.

[0153] Internal prediction

[0154] The set of intra prediction modes may include 35 different intra prediction modes, such as non-directional modes like DC (or average) mode and planar modes, or directional modes as defined in HEVC, for example, or 67 different intra prediction modes, such as non-directional modes like DC (or average) mode and planar modes, or directional modes as defined in VVC. For example, several conventional angle intra prediction modes are adaptively replaced by wide-angle intra prediction modes for non-square blocks as defined in VVC, for example. As another example, only the longer side is used to calculate the average of non-square blocks to avoid partitioning operations for DC prediction. Additionally, the intra prediction results of planar modes may be further modified by a Position Dependent Intra Prediction Combination (PDPC) method.

[0155] The intra prediction unit (254) is configured to use a reconstructed sample of a neighbor block of the same current image to generate an intra prediction block (265) according to the intra prediction mode of the set of intra prediction modes.

[0156] The intra-prediction unit (254) (or generally the mode selection unit (260)) is further configured to output intra-prediction parameters (or general information indicating the intra-prediction mode selected for the block) to the entropy encoding unit (270) in the form of syntax elements (266) to be included in the encoded image data (21) so that, for example, the video decoder (30) can receive and use the prediction parameters for decoding.

[0157] Mutual prediction

[0158] The set of (or possible) inter-prediction modes depends on the available reference image (i.e., the previously at least partially decoded image stored in, for example, DBP (230)) and other inter-prediction parameters. Whether the entire reference image or a part thereof, for example, a search window area around the current block area of ​​the reference image is used to search for the most matching reference block, and / or depending on, for example, whether pixel interpolation is applied, for example, half / semi-pel, quarter-pel, and / or 1 / 16 pel interpolation.

[0159] In addition to the above prediction modes, skip mode, direct mode, and / or other inter-prediction modes may be applied.

[0160] For example, in extended merge prediction, the merge candidate list for this mode is composed of five types of candidates in order: spatial MVP of spatially adjacent CUs, temporal MVP of placed CUs, history-based MVP of the FIFO table, pairwise, average MVP, and zero MV. Additionally, to improve the accuracy of the MV in the merge mode, decoder-side motion vector refinement (DMVR) based on bidirectional matching may be applied. The merge mode using MVD (M-mVD) originates from a merge mode with a motion vector difference. The M-mVD flag is signaled immediately after sending the skip flag and merge flag to specify whether the M-mVD mode is used for the CU. Furthermore, an Adaptive Motion Vector Resolution (AMVR) method at the CU level may be applied. Using AMVR allows the CU's MVD to be coded with different precision. The current CU's MVD can be adaptively selected based on the current CU's prediction mode. When a CU is coded in merge mode, the Combined Inter / Intra Prediction (CIIP) mode combined with the current CU can be applied. A weighted average of the inter and intra prediction signals is performed to obtain the CIIP prediction. For affine motion compensation prediction, the affine motion field of a block is described by motion information of two control points (4-parameters) or three control point motion vectors (6-parameters). Subblock-based temporal motion vector prediction (SbTMVP) is similar to HEVC's Temporal Motion Vector Prediction (TMVP) but predicts the motion vectors of sub-CUs within the current CU.Bi-directional optical flow (BDOF), formerly known as BIO, is a simpler version that requires significantly fewer computations, particularly in terms of the number of multiplications and the size of the multiplier. In the triangular splitting mode, the CU is divided evenly into two triangular-shaped splits using diagonal splitting or diagonal splitting. Furthermore, the bidirectional prediction mode extends beyond a simple average to allow for a weighted average of two prediction signals.

[0161] The inter-prediction unit (244) may include a motion estimation (ME) unit and a motion compensation (MC) unit (both not shown in 2). The motion estimation unit may be configured to receive or acquire an image block (203) (the current image block (203) of the current image (17)) and a decoded image (231), or at least one or more previously reconstructed blocks, for example, for motion estimation, one or more reconstructed blocks of other / other / previously decoded images (231). For example, the video sequence may include the current image and the previously decoded image (231), that is, the current image and the previously decoded image (231) may be part of or form a sequence of images that form the video sequence.

[0162] The encoder (20) may be configured to select a reference block from a plurality of reference blocks of the same or different images among a plurality of different images, for example, and to provide a reference image (or reference image index) and / or an offset (spatial offset) between the position (x, y coordinates) of the reference block and the position of the current block as an inter-prediction parameter for the motion estimation unit. This offset is also called a motion vector (MV).

[0163] The motion compensation unit is configured to obtain an inter prediction block (265) by, for example, receiving an inter prediction parameter and performing an inter prediction based on or using the inter prediction parameter. Motion compensation performed by the motion compensation unit may include fetching or generating a prediction block based on a motion / block vector determined by motion estimation, and, if possible, performing interpolation for subpixel precision. Since interpolation filtering can generate additional pixel samples from known pixel samples, it can potentially increase the number of candidate prediction blocks that can be used to code an image block. When the motion compensation unit receives a motion vector for the PU of the current image block, it can find the prediction block pointed to by the motion vector in one of the reference image lists.

[0164] The motion compensation unit may also generate a block and a syntax element associated with the video slice that is used by the video decoder (30) when decoding a video block of the video slice. In addition to or alternative to the slice and each syntax element, a tile group and / or a tile and each syntax element may be generated or used.

[0165] Entropy Coding

[0166] The entropy encoding unit (270) is configured to apply, for example, an entropy encoding algorithm or method (e.g., variable length coding (VLC) method, context adaptive VLC scheme (CAVLC), arithmetic coding method, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding or other entropy encoding methodology or technique) or bypass (uncompressed) for quantized coefficients, inter-prediction parameters, intra-prediction parameters, loop filter parameters and / or other syntax elements to obtain encoded image data (21) that can be output through an output (272) in the form of, for example, an encoded bitstream (21), and accordingly, for example, the video decoder (30) Parameters for decoding can be received and used. The encoded bitstream (21) can be transmitted to a video decoder (30) or stored in memory for later transmission or retrieval by the video decoder (30).

[0167] Other structural variations of the video encoder (20) may be used to encode a video stream. For example, a non-transform-based encoder (20) may quantize the residual signal directly without a transform processing unit (206) for a specific block or frame. In other implementations, the encoder (20) may have a quantization unit (208) and an inverse quantization unit (210) combined into a single unit.

[0168] Decoder and Decoding Method

[0169] FIG. 3 illustrates an example of a video decoder (30) configured to implement the technology of the present application. The video decoder (30) is configured to receive encoded video data (21) (e.g., an encoded bitstream (21)). It obtains a video (331) that has been encoded and decoded by an encoder (20). The encoded video data or bitstream includes information for decoding the encoded video data, for example, data representing a video block of an encoded video slice (and / or tile group or tile) and associated syntax elements.

[0170] In the example of the image of FIG. 3, the decoder (30) includes an entropy decoding unit (304), an inverse quantization unit (310), an inverse transform processing unit (312), a reconstruction unit (314) (e.g., summer (314)), a loop filter (320), a decoded image buffer (DBP) (330), a mode application unit (360), an inter prediction unit (344), and an intra prediction unit (354). The inter prediction unit (344) may be a motion compensation unit or may include it. The video decoder (30) may, in some examples, perform a decoding pass that is generally the inverse of the encoding pass described for the video encoder (100) of FIG. 2.

[0171] As described in relation to the encoder (20), inverse quantization unit (210), inverse transformation processing unit (212), reconstruction unit (214), loop filter (220), and decoding image buffer (DPB) (230), the inter prediction unit (344) and the intra prediction unit (354) are also referred to as forming the "built-in decoder" of the video encoder (20). Accordingly, the inverse quantization unit (310) may be functionally identical to the inverse quantization unit (110), the inverse transformation processing unit (312) may be functionally identical to the inverse transformation processing unit (212), the reconstruction unit (314) may be functionally identical to the reconstruction unit (214), the loop filter (320) may be functionally identical to the loop filter (220), and the decoded image buffer (330) may be functionally identical to the decoded image buffer (230). Accordingly, the description provided for each unit and function of the video (20) encoder is applied in correspondence with each unit and function.

[0172] Entropy decoding

[0173] The entropy decoding unit (304) is configured to parse the bitstream (21) (or generally the encoded image data (21)) and perform entropy decoding on the encoded image data (21) to obtain, for example, some or all of the quantized coefficients (309) and / or decoded coding parameters (not shown in FIG. 3), for example, inter-prediction parameters (e.g., reference image index and motion vector), intra-prediction parameters (e.g., intra-prediction mode or index), transformation parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit (304) may be configured to apply a decoding algorithm or method corresponding to the encoding method described in relation to the entropy encoding unit (270) of the encoder (20). The entropy decoding unit (304) may be further configured to provide inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the mode application (360) and to provide other parameters to other units of the decoder (30). The video decoder (30) may receive syntax elements at the video slice level and / or video block level. In addition to or alternative to the slice and each syntax element, a tile group and / or a tile and each syntax element may be received and / or used.

[0174] Inverse quantization

[0175] The inverse quantization unit (310) may be configured to receive quantization parameters (QP) (or information generally related to inverse quantization) from encoded image data (21) (e.g., by parsing and / or decoding by the entropy decoding unit (304)) and to apply inverse quantization to decoded quantized coefficients (309) based on the quantization parameters to obtain inverse quantized coefficients (311), which may also be referred to as transform coefficients (311). The inverse quantization process may determine the degree of quantization and the degree of inverse quantization to be applied for each video block of a video slice (or tile or group of tiles), including the use of quantization parameters determined by the video encoder (20).

[0176] Inverse transformation

[0177] The inverse transformation processing unit (312) may be configured to receive inverse quantized coefficients (311), also referred to as transformation coefficients (311), and to apply a transformation to the inverse quantized coefficients (311) to obtain a reconstructed residual block (213) in the sample domain. The reconstructed residual block (213) may also be referred to as a transformation block (313). The transformation may be an inverse transformation, e.g., inverse DCT, inverse DST, inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transformation processing unit (312) may be further configured to receive transformation parameters or corresponding information from the encoded image data (21) to determine the transformation to be applied to the inverse quantized coefficients (311) (e.g., by parsing and / or decoding by the entropy decoding unit (304)).

[0178] Reconstruction

[0179] The reconstruction unit (314) (e.g., an adder or summer (314)) may be configured to obtain the reconstructed block (315) in the sample domain by adding the reconstructed residual block (313) to the prediction block (365), for example, by adding the sample value of the reconstructed residual block (313) and the sample value of the prediction block (365).

[0180] Filtering

[0181] A loop filter unit (320) (in or after the coding loop) is configured to filter the reconstructed block (315) to obtain, for example, a filtered block (321), for example, to smooth pixel transitions or otherwise improve video quality. The loop filter unit (320) may include one or more loop filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, or, for example, one or more other filters, such as an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination thereof. In one example, the loop filter unit (220) may include a deblocking filter, an SAO filter, and an ALF filter. The order of the filtering process may be the deblocking filter, SAO, and ALF. In another example, a process called lumina mapping with chroma scaling (LMCS) (i.e., an adaptive in-loop reshaper) is added. This process is performed before deblocking. In another example, the deblocking filter process may also be applied to internal subblock edges, for example, affine subblock edges, ATMVP subblock edges, sub-block transform (SBT) edges, and intra sub-partition (ISP) edges. Although the loop filter unit (320) is shown in FIG. 3 as an in-loop filter, in other configurations, the loop filter unit (320) may be implemented as a post-loop filter.

[0182] Decoded video buffer

[0183] The decoded video block (321) of the video is stored in the decoded video buffer (330), and the decoded video (331) is stored as a reference video for subsequent motion compensation for other videos and / or for each display.

[0184] The decoder (30) is configured to output, for example, a decoded image (311) through an output (312) to display or show to the user.

[0185] prediction

[0186] The inter prediction unit (344) may be identical to the inter prediction unit (244) (particularly for the motion compensation unit), and the intra prediction unit (354) may be functionally identical to the inter prediction unit (254), and may perform partitioning or partitioning decision and prediction basis based on partitioning and / or prediction parameters or respective information received from the encoded image data (21) (e.g., by parsing and / or decoding, e.g. by the entropy decoding unit (304)). The mode application unit (360) may be configured to perform prediction per block (intra or inter prediction) based on the reconstructed image, block, or respective sample (filtered or unfiltered) to acquire the prediction block (365).

[0187] When a video slice is coded as an intra-coding (I) slice, the intra-prediction unit (354) of the mode application unit (360) is configured to generate a prediction block (365) for a video block of the current video slice based on the intra-prediction mode and data signaled from a previously decoded block of the current video. When a video image is coded as an inter-coding (i.e., B or P) slice, the inter-prediction unit (e.g., motion compensation unit) of the mode application unit (360) is configured to generate a prediction block (365) for a video block of the current video slice based on motion vectors and other syntax elements received from the entropy decoding unit (304). For inter-prediction, the prediction block may be generated from one of the reference images within one of the reference image lists. The video decoder (30) may configure the reference frame list, List 0 and List 1 using a default configuration technique based on the reference images stored in the DPB (330). The same or similar may apply to embodiments using a tile group (e.g., a video tile group) and / or a tile (e.g., a video tile) in addition to or alternative to a slice (e.g., a video slice), and, for example, a video may be coded using I, P, or B tile groups and / or tiles.

[0188] The mode application unit (360) is configured to determine prediction information for a video block of a current video slice by parsing motion vectors or related information and other syntax elements, and to generate a prediction block for the current video block being decoded using the prediction information.

[0189] For example, the mode application unit (360) determines a prediction mode (e.g., intra or inter prediction) used to code video blocks of a video slice using some of the received syntax elements, an inter prediction slice type (e.g., B slice, P slice, or GPB slice), configuration information for one or more of the reference image lists for the slice, motion vectors for each inter-encoded video block of the slice, an inter prediction state for each inter-encoded video block of the slice, and other information for decoding video blocks of the current video slice. The same or similar may be applied to embodiments that additionally or alternatively use a tile group (e.g., video tile group) and / or tiles (e.g., video tiles) in addition to or alternatively to the slice (e.g., video slice), for example, the video may be coded using an I, P, or B tile group and / or tiles.

[0190] An embodiment of the video decoder (30) illustrated in FIG. 3 may be configured to divide and / or decode an image using slices (also called video slices), wherein the image may be divided or decoded using one or more slices (typically non-overlapping), and each slice may include one or more blocks (e.g., CTU) or one or more groups of blocks (e.g., tiles (H.265 / HEVC and VVC) or bricks (VVC)).

[0191] An embodiment of the video decoder (30) illustrated in FIG. 3 may be configured to divide and / or decode an image using slice / tile groups (also called video tile groups) and / or tiles (also called video tiles), wherein the image may be divided or decoded using one or more slice / tile groups (typically non-overlapping), and each slice / tile group may include, for example, one or more blocks (e.g., CTUs) or one or more tiles, and each tile may be, for example, rectangular in shape and may include one or more blocks (e.g., CTUs), for example, complete or fractional blocks.

[0192] Other variations of the video decoder (30) may be used to decode encoded video data (21). For example, the decoder (30) may generate an output video stream without a loop filtering unit (320). For example, a non-transform-based decoder (30) may directly de-quantize the residual signal for a specific block or frame without an inverse transformation processing unit (312). In other implementations, the video decoder (30) may combine the inverse quantization unit (310) and the inverse transformation processing unit (312) into a single unit.

[0193] It should be understood that the encoder (20) and decoder (30) may output the processing result of the current stage to the next stage after further processing. For example, after interpolation filtering, motion vector derivation, or loop filtering, additional operations such as Clip or Shift may be performed on the processing result of interpolation filtering, motion vector derivation, or loop filtering.

[0194] Note that additional operations may be applied to the derived motion vectors of the current block (including, but not limited to, control point motion vectors in affine mode, affine, plane, ATMVP mode, temporal motion vectors, etc.). For example, the value of a motion vector is limited to a predefined range according to its representative bit. If the representative bit of the motion vector is bitDepth, the range is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where "^" represents the exponent. For example, if bitDepth is set to 16, the range is -32768 to 32767. If bitDepth is set to 18, the range is -131072 to 131071. For example, the value of a derived motion vector (e.g., the MV of four 4x4 sub-blocks within a single 8x8 block) is limited such that the maximum difference between the integer parts of the four 4x4 sub-block MVs is not greater than N pixels, e.g., not greater than 1 pixel. Here, two methods for limiting motion vectors based on bitDepth are provided.

[0195] FIG. 4 is a schematic diagram of a video coding device (400) according to an embodiment of the present disclosure. The video coding device (400) is suitable for implementing the disclosed embodiments as described herein. In one embodiment, the video coding device (400) may be an encoder such as the video decoder (30) of FIG. 1a or the video encoder (20) of FIG. 1a.

[0196] A video coding device (400) includes a receiving port (410) (or input port (410)) and a receiver unit (Rx) (420) for receiving data; a processor, logic unit, or central processing unit (CPU) (430) for processing data; a transmitter unit (Tx) (440) and an exit port (450) (or output port (450)) for transmitting data; and a memory (460) for storing data. The video coding device (400) may also include an optical-electric (OE) component and an electro-optical (EO) component coupled to the input port (410), receiver unit (420), transmitter unit (440), and transmission port (450) for the outflow or inflow of optical or electrical signals.

[0197] The processor (430) is implemented in hardware and software. The processor (430) may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor (430) communicates with an inlet port (410), a receiver unit (420), a transmitter unit (440), an outlet port (450), and memory (460). The processor (430) includes a coding module (470). The coding module (470) implements the disclosed embodiments described above. For example, the coding module (470) implements, processes, prepares, or provides various coding operations. Thus, the inclusion of the coding module (470) provides substantial improvements to the functionality of the video coding device (400) and influences the transition of the video coding device (400) to different states. Alternatively, the coding module (470) is implemented as instructions stored in memory (460) and executed by the processor (430).

[0198] The memory (460) may include one or more disks, tape drives and solid-state drives and may be used as an overflow data storage device, storing the program when it is selected for execution and reading it during program execution. The memory (460) may be, for example, volatile and / or non-volatile and may be read-only memory (ROM), random access memory (RAM), terrestrial content addressable memory (TCAM) and / or static random access memory (SRAM).

[0199] FIG. 5 is a simplified block diagram of a device (500) that can be used as either or both of the source device (12) and destination device (14) of FIG. 1 according to an exemplary embodiment.

[0200] The processor (502) of the device (500) may be a central processing unit. Alternatively, the processor (502) may be any other type of device or multiple devices capable of manipulating or processing information that is currently existing or will be developed in the future. The disclosed implementation may be executed with a single processor, e.g., the processor (502), as illustrated, but the benefits of speed and efficiency may be achieved by using one or more processors.

[0201] The memory (504) of the device (500) may be a read-only memory (ROM) device or a random access memory (RAM) device in the implementation. Any other suitable type of storage device may be used as memory (504). Memory (504) may contain code and data (506) accessed by the processor (502) using the bus (512). Memory (504) may further include an operating system (508) and an application program (510). The application (510) includes at least one program that allows the processor (502) to perform the method described herein. For example, the application program (510) may include applications 1 through N, which further include a video coding application that performs the method described herein.

[0202] The device (500) may also include one or more output devices, such as a display (518). The display (518) may, in one example, be a touch-sensing display that combines the display with a touch-sensing element operable to detect touch input. The display (518) may be connected to the processor (502) via a bus (512).

[0203] Although shown here as a single bus, the bus (512) of the device (500) may be configured as multiple buses. Additionally, the auxiliary storage (514) may be directly coupled to other components of the device (500) or accessed via a network, and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Thus, the device (500) can be implemented in a wide variety of configurations.

[0204] Several technologies that can be implemented as the current solution of the present application are introduced as follows.

[0205] Merge candidate lists

[0206] The process of constructing a merge motion candidate list was introduced by the ITU-T H.265 standard. In another embodiment, the merge motion candidate list is used by extended merge prediction in Versatile Video Coding (VVC).

[0207] Block merging is a special mode (also called "merge mode") for motion data coding. Block merging allows the current block to use the same motion information as neighboring blocks. Motion information includes motion data, which contains not only whether one or two reference image lists are used, but also reference indices and motion vectors for each reference image list. Block merging is particularly useful when two adjacent blocks correspond to the same immutable object in an image frame. In this case, since the two blocks can be predicted using the same motion vector and the same reference image, the total motion information is identical for both blocks.

[0208] In one implementation, after checking whether neighboring blocks are available and contain motion information, some additional duplicate checks are performed before taking all motion data from neighboring blocks as motion information candidates.

[0209] In the implementation, the merge candidate list is composed of the following five types of candidates in order.

[0210] 1) Spatial MVP of adjacent CU

[0211] 2) Temporary MVP of the assigned CU

[0212] 3) History-based MVP of FIFO Tables

[0213] 4) Pairwise Average MVP

[0214] 5) Zero MV.

[0215] For example, the size of the merge list is indicated in the slice header, and the maximum allowed size of the merge list is 6. For each CU code of the merge mode, the best merge candidate index is encoded.

[0216] A generation process is provided for each category of merge candidates:

[0217] Inducing spatial candidates

[0218] In one implementation, up to four merge candidates are selected from the candidates located at the positions shown in FIG. 6. The derivation order is B1, A1, B0, A0, and B2. Position B2 is considered only if the CU of positions A0, B0, B1, and A1 cannot be used (e.g., because it belongs to a different slice or tile) or is intra-coded. After the candidate for position B1 is added, the addition of the remaining candidates is subjected to a duplicate check to exclude candidates with identical motion information from the list, thereby improving coding efficiency. To reduce computational complexity, not all possible pairs of candidates are considered in the aforementioned duplicate check. Instead, only pairs connected by arrows in FIG. 7 are considered, and a candidate is added to the list only if the corresponding candidate used in the duplicate check does not have identical motion information.

[0219] The order, location, and number of spatial neighbors considered changeable during implementation cannot be regarded as limitations in the above example.

[0220] Temporal candidate induction

[0221] In a single implementation, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the colocated CU belonging to the collated reference picture. The list of reference pictures to be used for deriving the coexisting CU is explicitly signaled in the slice header. The scaled motion vector for the temporal merge candidate is obtained as indicated by the dashed line in Fig. 8, which is scaled from the motion vector of the colocated CU using POC distances tb and td, where tb is defined as the difference between the reference picture of the current picture and the POC of the current picture, and td is defined as the difference between the reference picture of the same location picture and the same location picture. The reference picture index of the temporal merge candidate is set to 0. The location for the temporal candidate is selected between candidates C0 and C1 as shown in Fig. 9. If the CU at location C0 is unavailable, intra-coded, or outside the current row of the CTU, location C1 is used. Otherwise, position C0 is used for the derivation of temporal merge candidates.

[0222] History-based merge candidate induction

[0223] History-based MVP (HMVP) merge candidates are added to the merge list following the spatial MVP and TMVP. Motion information of previously coded blocks in an implementation is stored in a table and used as the MVP for the current CU. A table containing multiple HMVP candidates is maintained during the encoding / decoding process. When a new CTU row is found, the table is reset (emptied). Whenever there is a non-subblock intercoded CU, the associated motion information is added to the last entry of the table as a new HMVP candidate.

[0224] The HMVP table size S is set to 5, which indicates that, for example, up to 5 History-based MVP (HMVP) candidates can be added to the table. When inserting a new motion candidate into the table, a restricted First-in-First-Out (FIFO) rule is used, and a duplicate check is applied first to find if there is a duplicate HMVP in the table. If found, the duplicate HMVP is removed from the table, and all subsequent HMVP candidates are moved forward.

[0225] HMVP candidates can be used in the merge candidate list construction process. The most recent HMVP candidates are identified in order from the table and inserted into the candidate list following the TMVP candidates. Duplicate checks are applied to the spatial or temporal merge candidates for the HMVP candidates.

[0226] Other simplifications may be introduced to reduce the number of duplicate check tasks. Generally, when the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the merge candidate list configuration process in HMVP is terminated.

[0227] Pair-wise average merge candidate derivation

[0228] Pairwise average candidates are generated by averaging predefined pairs of candidates from an existing list of merge candidates, where the predefined pairs are defined, for example, as (0, 1), where the number represents the merge index for the list of merge candidates. Average motion vectors are calculated separately for each list of references. If both motion vectors are available in a list, these two motion vectors are averaged even if they point to different reference images; if only one motion vector is available, that one is used directly; and if no motion vectors are available, this list is kept invalid.

[0229] The motion information candidate list is output during the merge candidate list construction process. The term "motion information" refers to the collected information required to perform the inter-prediction process. Motion information generally refers to the following information:

[0230] 1) Whether the block applies a single prediction or a double prediction (prediction direction).

[0231] 2) Motion vectors (2 motion vectors if the block applies double prediction)

[0232] 3) Reference image index used for prediction. (If the block applies double prediction, there are 2 indices, each index corresponds to one reference image list, the first reference image list (L0) or the second reference image list (L1).

[0233] In some executable implementations, motion information may also refer to a switchable interpolation filter index used to represent a specific interpolation filter for motion compensation of the inter-prediction unit.

[0234] In the present application, motion information may be one or more of the above items or any other information required to perform an inter-prediction process according to another embodiment.

[0235] The reference image index is used to represent an entry in the reference image list used in the prediction process of the coding block. For example, the first motion vector may point to the first image of L0, and the second motion vector may point to the first image of L1. Two reference image lists may be maintained, and the image pointed to by the first motion vector in L0 is selected, and the image pointed to by the second motion vector in L1 is selected.

[0236] Each reference image list L0 and L1 may contain one or more reference images, each identified by a picture order count (POC). The association between each reference index and its POC value may be signaled in the bitstream. For example, the L0 and L1 reference image lists may contain the following reference images:

[0237]

[0238] In the example above, the first entry of reference image list L1 (indicated by reference index 0) is a reference image with a POC value of 13.

[0239] POC is a variable associated with each image, uniquely identifying the associated image among all images in the Coded Video Sequence (CVS), and indicates the position of the associated image in the output order relative to the output order positions of other images in the same CVS to be output from the decoded image buffer when the associated image is output.

[0240] Triangular prediction mode

[0241] When using the triangular prediction mode (TP-m), the CU is evenly divided into two triangular-shaped partitions using diagonal partitioning or diagonal partitioning as in FIG. 10a or FIG. 10b. Note that either FIG. 10a or FIG. 10b is an example. The positions of PU1 and PU2 are not limited in this application. Each triangular partition of the CU is mutually predicted using its own motion; only a single prediction is allowed for each partition, that is, each partition has one motion vector and one reference index. A single prediction motion constraint is applied so that only two motion compensation predictions are required for each CU, just like conventional dual prediction. The single prediction motion for each partition is derived directly from the merge candidate list configured for the extended merge prediction described above.

[0242] When the triangle partition mode is used for the current CU, a flag indicating the direction of the triangle partition (diagonal or counter-diagonal) and two merge indices (one for each partition) are additionally signaled. After predicting each triangle partition, sample values ​​along the diagonal or diagonal corners are adjusted using adaptive weighted blending processing. This is the prediction signal for the entire CU, and as in other prediction modes, the transformation and quantization processes are applied to the entire CU. Finally, the motion field of the CU predicted using the triangle partition mode is stored in 4x4 units.

[0243] TP-m is a special case of sub-block partitioning in which a block is divided into two blocks. In the example above, two block partitioning directions (45-degree and 135-degree partitions) are illustrated. However, it should be noted that other partition angles and partition ratios are also possible, as illustrated in Fig. 11. For example, depending on the partition angle, the sub-block can be rectangular (e.g., the middle and right diagrams of Fig. 11) or non-rectangular (trapezoidal, e.g., the left diagram of Fig. 11). In some examples, the current block consists of two prediction units, and the two prediction units are divided by a virtual dividing line. In this case, the current block is said to be predicted in geometric prediction mode. For example, the virtual dividing line can be a straight line that is the boundary between the first sub-block and the second sub-block.

[0244] More specifically, the prediction procedure using TP-m includes the following:

[0245] Configuration of a single prediction candidate list

[0246] Given a merge candidate index, a single predicted motion vector is derived from a list of merge candidates configured for extended merge prediction, as exemplified in FIG. 12. For a candidate in the list, the LX motion vector, where X is equal to the parity of the merge candidate index value, is used as the single predicted motion vector for the triangle splitting mode. This motion vector is indicated as "x" in FIG. 12. If a corresponding LX motion vector does not exist, the L(1-X) motion vector of the same candidate from the extended merge prediction candidate list is used as the single predicted motion vector for the triangle splitting mode.

[0247] Note that in an executable implementation, a single predicted motion vector can be directly derived from the merge candidate list used in the regular merge mode without explicitly constructing a special separate single predicted candidate list.

[0248] Blending along the edges of the triangular partition

[0249] After predicting each triangle partition using self-motion, blending is applied to the two predicted signals to derive samples around the diagonal or diagonal edges. The weights used in the blending process are as follows:

[0250] As shown in Fig. 13, for Luma {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and for Chroma {6 / 8, 4 / 8, 2 / 8}.

[0251] Compared to the general prediction mode, motion information of different sub-blocks must be signaled to the blocks. Therefore, the overhead of the additional information used to represent motion information in the sub-block prediction mode is higher. To improve the efficiency of additional information coding for the sub-block prediction mode, another embodiment is introduced in this application.

[0252] Example 1:

[0253] When a sub-block prediction is determined to be applied to a block:

[0254] 1. Maximum value M(M is an integer, M An initial first index is included in a bitstream (on the encoder side) that can have N (where N is the number of candidates in the motion information candidate list).

[0255] 2. Mm(m is an integer and m <M, m은 미리 정의된 값)의 최대 값을 가질 수 있는 비트스트림(인코더 측에서)에 초기 제2 인덱스가 포함된다.

[0256] 3. If the value of the initial second index is greater than or equal to the value of the initial first index, the value of the initial second index is increased by a predefined number to obtain the updated second index (e.g., the predefined number may be 1).

[0257] 4. The first sub-block portion of the block is predicted by the application of motion candidate information determined based on the initial first index.

[0258] 5. A second sub-block portion of the block is predicted by applying motion candidate information determined based on the updated second index.

[0259] In an executable implementation, the initial first index and the initial second index may be included in the bitstream by the encoder and parsed by the decoder.

[0260] In an executable implementation, the operation of increasing the initial second index by m is performed on both the encoder side and the decoder side, and this operation aims to maintain consistency between the encoder side and the decoder side.

[0261] In an executable implementation, an initial first index and an updated second index are used to select an entry from a list of motion information candidates (corresponding motion information candidates may be selected based on the initial first index and the updated second index as entries), and the selected motion information candidates are used in a first sub-block portion of a block (e.g., PU1 in FIG. 10a or FIG. 10b) and a second sub-block portion (e.g., PU2 in FIG. 10a or FIG. 10b) to perform a prediction.

[0262] In a feasible implementation, the motion information candidate list may consist only of single-predicted motion information candidates. Note that the merge candidate list (e.g., the merge candidate list of ITU-T H.265) may consist of single-predicted and dual-predicted motion information candidates. Therefore, the motion information candidate list used in this embodiment may differ from the merge candidate list of ITU-T H.265.

[0263] The motion information candidate list may not match the merge candidate list, as the merge candidate list may contain double prediction candidates, which may be prohibited when it is determined that a block will apply a sub-block (e.g., triangle) prediction. In this case, since each sub-block must apply single prediction motion information, the initial first index and the updated second index point to entries in the motion information candidate list that contain only single prediction candidates. The motion information candidate list may be constructed using the same spatially and temporally adjacent blocks used in the construction of the merge candidate list. In another example, the motion information candidate list may be constructed based on the merge candidate list by converting the double prediction candidates of the merge candidate list into single prediction candidates.

[0264] Note that the initial first index and initial second index do not need to follow a specific order in the bitstream structure.

[0265] Note that the comparison operation between the initial first index and the initial second index is performed in both the encoder and the decoder. On the encoder side, the indices (e.g., the initial first index and the initial second index) are included in the bitstream. On the decoder side, the indices (e.g., the initial first index and the initial second index) are parsed from the bitstream.

[0266] If the value of the initial second index is greater than or equal to the value of the initial first index, the value of the initial second index is increased by a predefined number (e.g., 1). Generally, the same motion information candidate list (composed of motion information candidates) is used, and if two indices point to the same motion information in the motion information candidate list, this corresponds to having a single motion information for the entire block. Therefore, the initial second index is increased by 1 to prevent obtaining the same index. If the initial second index is not increased, the first and second indices may point to the same motion candidate in the motion information candidate list (because the same list is used to select motion information for two sub-block parts). In this case, each sub-block part will apply the same motion information for prediction, which means there is no point to divide the block into two sub-blocks. If the value of the initial second index is greater than or equal to the initial first index, the initial second index is increased to avoid duplicate representations. Therefore, it should be noted that the motion information candidate list contains at least two sets of motion information. By eliminating duplicate motion information representations of sub-blocks according to the present invention, compression efficiency is improved.

[0267] Note that even if the result of the increment operation does not exceed the number of candidates in the motion information candidate list, the initial second index can be increased by a predefined number (e.g., 1, 2, 3, etc.).

[0268] In a specific implementation of the first embodiment, it is assumed that the motion information candidate list includes motion information candidates having six entries. Additionally, it is assumed that the first motion candidate of the motion information candidate list is applied to the first sub-block portion of the block for prediction and the fifth motion candidate of the motion information candidate list is applied to the second sub-block portion of the block for prediction.

[0269] On the encoder side:

[0270] 1. A value of 0 is included in (or signaled) the bitstream to represent the value of the initial first index. (Index value 0 corresponds to the first entry of the motion information candidate list, value 1 corresponds to the second entry, and so on.)

[0271] 2. Value 3 is included in (or signaled) the bitstream to represent the value of the initial second index.

[0272] 3. The value of the updated second index is calculated by increasing the value of the initial second index by, for example, 1, so the value 4 is obtained.

[0273] 4. The initial first index is determined to point to the first motion candidate in the motion information candidate list. It is applied to predict the first sub-block portion of the block.

[0274] 5. The updated second index is determined to point to the fifth motion candidate in the motion information candidate list. It is applied to predict the second sub-block portion of the block.

[0275] On the decoder side:

[0276] 1. The value 0 is parsed from the bitstream and represents the value of the initial first index.

[0277] 2. Value 3 is parsed from the bitstream and represents the value of the initial second index.

[0278] 3. The value of the updated second index is calculated by increasing the value by, for example, 1 (since 3 is greater than 0), and thus the value 4 is obtained.

[0279] 4. An initial first index is determined to point to the first motion candidate in the motion information candidate list. It is applied to predict the first sub-block portion of the block.

[0280] 5. The updated second index is determined to point to the fifth motion candidate in the motion information candidate list. It is applied to predict the second sub-block portion of the block.

[0281] In this implementation, the initial second index is also updated by incrementing by 1 on the encoder side, and this operation aims to maintain consistency with similar operations on the decoder side.

[0282] It can be understood that even if the result of the increment operation does not exceed the number of candidates in the motion information candidate list, the increment number can be a predefined number, such as 1, 2, 3, etc.

[0283] In another specific implementation of the first embodiment, assuming the motion information candidate list includes the following three entries:

[0284] (1) 11th motion information candidate (1st candidate)

[0285] (2) Second motion information candidate (second candidate)

[0286] (3) Third motion information candidate (third candidate)

[0287] The values ​​of the first and second indices are as follows:

[0288]

[0289] In the table above, the maximum value of the first index is 3, and the maximum value of the second index is 2. If the initial second index in the table is greater than the initial first index, the initial second index continues to increment by 1. This is to maintain consistent behavior for scenarios where the initial second index is greater than or equal to the initial first index (e.g., incrementing by 1). For example, looking at rows 1 through 3 of the table above, for rows 1 and 3, the initial first index and the initial second index are identical; to make them different, the initial second index increments by 1. For row 2, the initial second index is greater than the initial first index, and if no incremental operation is performed, this would be an exception and increase complexity. To keep it simple, when it is determined that the initial second index is greater than the initial first index, the initial second index still increments by 1.

[0290] Example 2:

[0291] When a sub-block prediction is determined to be applied to a block:

[0292] 1. An indicator representing the direction of division is included in the bitstream.

[0293] 2. An initial first index is included in a bitstream that can have a maximum value of M (M is an integer and M N, N is the number of candidates in the motion information candidate list).

[0294] 3. An initial second index is included in a bitstream that can have a maximum value of Mm (m is an integer and m <M, m은 미리 정의된 값이다).

[0295] 4. If the value of the initial second index is greater than or equal to the value of the initial first index, the value of the initial second index is increased by a predefined number to obtain the updated second index (e.g., the predefined number may be 1).

[0296] 5. The first sub-block portion of the block is predicted by the application of motion candidate information determined based on the initial first index.

[0297] 6. A second sub-block portion of the block is predicted by applying motion candidate information determined based on the updated second index.

[0298] Note that the partition direction indicator, the initial first index, and the initial second index do not need to follow a specific order in the bitstream structure.

[0299] In an executable implementation, there can be the following two directions of partitioning:

[0300] (1) Dividing the block from the top-left corner to the bottom-right corner (see left drawing of FIG. 10a or FIG. 10b)

[0301] (2) Divide the block from the top right corner to the bottom left corner (refer to the right drawing in FIG. 10a or FIG. 10b)

[0302] In a feasible implementation, there can be 4 partition directions.

[0303] (1) Split from the top-left corner to the bottom-right corner of the block

[0304] (2) Split from the top right corner to the bottom left corner of the block

[0305] (3) Split from the center top point of the block to the center bottom point

[0306] (4) Divide the block from the right center point to the left center point.

[0307] Example 3:

[0308] When a sub-block prediction is determined to be applied to a block:

[0309] 1. An indicator representing the direction of division is included in the bitstream.

[0310] 2. Where the indicator indicates a specific division direction (e.g., division from the top-left corner to the bottom-right corner of a block, see the left drawing in FIG. 10a or FIG. 10b)

[0311] 2.1 The bitstream contains the first index that can have the maximum value of M (M is an integer and M N, N is the number of candidates in the motion information candidate list).

[0312] 2.2 The initial second index is contained in a bitstream that can have a maximum value of Mm (where m is an integer and m <M, m은 미리 정의된 값이다).

[0313] 2.3. If the value of the initial second index is greater than or equal to the value of the initial first index, the value of the initial second index is increased by a predefined number to obtain the updated second index (e.g., the predefined number may be 1).

[0314] 3. Otherwise (where the indicator indicates a different division direction, for example, dividing from the top-right corner to the bottom-left corner of the block (see the right drawing in FIG. 10a or FIG. 10b)

[0315] 3.1 The initial first index is contained in a bitstream that can have a maximum value of Mm (where m is an integer and m <M, m은 미리 정의된 값이다).

[0316] 3.2 A bitstream that can have a maximum value of M includes an initial second index (M is an integer and M N, N is the number of candidates in the motion information candidate list).

[0317] 3.3. If the value of the initial second index is greater than or equal to the value of the initial first index, the value of the initial second index is increased by a predefined number to obtain an updated second index (e.g., the predefined number may be 1).

[0318] 4. The first sub-block portion of the block is predicted by the application of motion information candidates determined based on the first index.

[0319] 5. The second sub-block portion of the block is predicted by the application of motion information candidates determined based on the second index.

[0320] In an executable implementation, there can be two directions of partitioning.

[0321] (1) Split from the top-left corner to the bottom-right corner of the block

[0322] (2) Divide the block from the top right corner to the bottom left corner.

[0323] In a feasible implementation, there can be 4 partition directions.

[0324] (1) Split from the top-left corner to the bottom-right corner of the block

[0325] (2) Divide the block from the top right corner to the bottom left corner.

[0326] (3) Split from the center top point of the block to the center bottom point

[0327] (4) Divide the block from the right center point to the left center point.

[0328] Example 4:

[0329] When it is decided to apply the block based on sub-block prediction:

[0330] 1. An indicator representing the direction of division is included in the bitstream.

[0331] 2. An initial first index is included in a bitstream that can have a maximum value of M (M is an integer and M N, N is the number of candidates in the motion information candidate list).

[0332] 3. An initial second index is included in a bitstream that can have a maximum value of Mm (m is an integer and m <M, m은 미리 정의된 값이다).

[0333] 4. If the value of the initial second index is greater than or equal to the value of the initial first index, the value of the initial second index is increased by a predefined number to update the second index (e.g., the predefined number may be 1).

[0334] 5. The first sub-block portion of the block is predicted by the application of motion candidate information determined based on the initial first index.

[0335] 6. A second sub-block portion of the block is predicted by applying motion candidate information determined based on the updated second index.

[0336] The first sub-block portion is designated as the part where the geometric center is closer to the left boundary of the block. Taking FIG. 10a or FIG. 10b as an example, PU1 represents the first sub-block portion and PU2 represents the second sub-block portion.

[0337] In Example 4, the index corresponding to the sub-block portion having a geometric center closer to the left boundary of the block is included in the bitstream first. The construction of the motion information candidate list generally considers the motion information of neighboring blocks in the following order: motion information of the left block, motion information of the top block, motion information of the top right block ... The above spatial neighbor order is taken as an example in HEVC. Because the motion information of the left neighbor is considered first, the index representing the motion information of the left neighbor generally has fewer bits. Since the sub-block portion closer to the left neighbor (geometric mean) is coded first and the second index cannot point to the same motion information (i.e., the same item in the merged list), the total number of bits to represent the first index and the second index is generally lower. In other feasible implementations, the index corresponding to the sub-block portion having a geometric center closer to the top boundary of the block is included in the bitstream first. The order is not limited by the present application.

[0338] In a specific embodiment, it is assumed that the size of the motion information candidate list is 3 and includes the following motion information candidates: motion information of a left neighbor block (first entry), motion information of an upper neighbor block (second entry), and a block where motion information is placed (third entry, temporally adjacent block).

[0339] Also, it is assumed that the division direction and the first sub-block (PU1) and the second sub-block (PU2) are given as shown in the left side of FIG. 10a or FIG. 10b. Generally, PU1 will be predicted based on the first entry of the motion information candidate list (corresponding to the motion information of the left neighbor block) and PU2 will be predicted based on the second entry of the motion information candidate list according to spatial proximity (corresponding to the motion information of the upper neighbor block).

[0340] According to the present invention, a value of 0 is included in the bitstream to represent a first index (parsed by a decoder), and a value of 0 is included in the bitstream to represent a second index (the smallest index that can be assumed) since the second index is equal to the first index (parsed by a decoder), and the second index is incremented by 1 before selecting a motion candidate from a motion information candidate list. In this case, the first and second indices have the smallest possible values ​​according to the present invention, which will require the minimum total number of bits to represent the first index and the second index in the bitstream.

[0341] Note that the splitting direction indicator, the first index, and the second index do not need to follow a specific order in the bitstream structure.

[0342] In certain implementations, the sub-block prediction mode is the triangular prediction mode.

[0343] A first index (e.g., an initial first index) and a second index (e.g., an updated second index) are used to select motion information to be applied to a first sub-block and a second sub-block. Motion information is selected from the same list of motion information candidates. To prevent the same motion information from being selected for two sub-block parts (equivalent to having no sub-block partition), the second index is incremented by a predefined number (e.g., 1) if it is greater than or equal to the first index.

[0344] Examples 1-4 provide different efficient methods for signaling motion information for each sub-block of a block to which sub-block prediction is applied.

[0345] Example 5:

[0346] The maximum value of the first index and the second index (indicated as M in Examples 1 to 4) is smaller than or equal to the size of the motion information candidate list N.

[0347] Also, note that the maximum values ​​of the first and second indices describe the number of entries in the motion information candidate list. For example, if the maximum value of the first index is 6 (assuming counting starts from 1, the index can be assumed to have values ​​of 1, 2, 3, 4, 5, and 6), the size of the motion information candidate list is 6.

[0348] The merge candidate list can be configured according to the methods of ITU-T H.265 and VVC, and refer to the above examples and disclosures regarding the merge list configuration process in HEVC and VVC.

[0349] The maximum value of the first and second indices (given as M and equal to the size of the motion information candidate list) is less than or equal to the size of the merge candidate list N. If it is determined that a block does not apply the sub-block prediction mode, the block can be predicted based on one of the entries in the merge candidate list. However, if the block is predicted using the sub-block prediction mode, the block is predicted using an entry in the motion information candidate list.

[0350] For example, when a prediction indicator parsed from a bitstream indicates that a sub-block prediction will be applied to a block, two different indicators are obtained, and these two different indicators represent two different entries in two motion information candidate lists, and an inter-prediction is performed for the block based on the two different indicators. When a prediction indicator parsed from a bitstream indicates that a sub-block prediction will not be applied to a block, a single indicator is obtained, and the single indicator indicates an entry in the merge candidate list for the block (e.g., can be configured according to the methods of ITU-T H.265 and VVC); and (e.g., for non-sub-block prediction) an inter-prediction is performed for the block based on the single indicator.

[0351] If M is less than or equal to N, the maximum value of the first index is set to be equal to M, where N is the size (number of candidates) of the merge candidate list. Otherwise (if N is less than M), the maximum value of the first merge index is set to be equal to N.

[0352] In an executable implementation, N can be derived from an indicator contained in the bitstream, and M can be a predefined number.

[0353] For example, the value M (describes the maximum value of the first index) can be a predetermined 5. And the value of N (size merge candidate list) can be signaled from the sequence parameter set (SPS). If the value of N is signaled as 6, the maximum value of the first index is 5 because N is greater than M (it is 4 if counting starts from 0). In another scenario, for the SPS, if N is signaled as 3, the maximum value of the first index is 3 (it is 2 if counting starts from 0).

[0354] N is the size of the merge candidate list that can be constructed according to the methods of ITU-T H.265 and VVC. The first and second indices used in the embodiments point to different lists, namely, a motion information candidate list consisting only of single prediction candidates. The motion information candidate list may not coincide with the merge candidate list, because the merge candidate list may contain double prediction candidates, which may be prohibited from use when it is determined that a block applies sub-block (or triangular) prediction. In this case, since each sub-block must apply single prediction motion information, the first and second indices point to entries in the motion information candidate list containing only single prediction candidates. The motion information candidate list may be constructed using the same spatially and temporally adjacent blocks used in the construction of the merge candidate list. Alternatively, the motion information candidate list may be constructed based on entries in the merge candidate list. In a feasible implementation, the motion information candidate list may not be explicitly constructed. For example, single prediction candidates may be derived directly from the merge candidate list. For block-based prediction, only one set of motion information is signaled (in one implementation, it is signaled in the form of a merge index). For sub-block prediction, since two sets of motion information are required (increasing signaling overhead), the index value is not expected to be greater than the maximum value of the merge index (the maximum value of the merge index is equal to the size of the merge candidate list).

[0355] Since the motion information candidate list is expected to be constructed based on the candidates or entries of the merge candidate list used to construct the merge candidate list, the size of the motion information candidate list will not be larger than the size of the merge candidate list.

[0356] Therefore, the size of the motion information candidate list (and thus the maximum value of the first and second indices) is set to be equal to or smaller than the merge candidate list.

[0357] In another executable implementation, N can be derived from an indicator contained in the bitstream and M can be derived from an indicator contained in the bitstream. In this case, the indicator used to derive the value of M cannot represent a value of M greater than N.

[0358] In HEVC, the size of the motion information candidate list is N, and N can be modified depending on the syntax elements included in the bitstream. The value of N can be a positive integer (typically between 2 and 5) and is signaled in the SPS. The size of the merge list is fixed for the entire video sequence.

[0359] The maximum value of the first index and the second index cannot be larger than the size of the motion information candidate list. The first index and the second index are used to select motion information from a different list (a motion information candidate list that is not identical to the merge candidate list), but generally, the motion information candidate list can be constructed using the same spatial adjacency blocks and temporal adjacency blocks (provided that different construction rules are applied from the motion information candidate list).

[0360] In a specific implementation, the motion information candidate list can be constructed by converting the double prediction candidates of the merge candidate list into single prediction candidates. Therefore, setting the maximum values ​​of the first and second indices to be smaller than the size of the merge candidate list ensures that the motion information candidate list constructed based on the merge candidate list can be used to select motion information for each sub-block of the current block.

[0361] Example 6:

[0362] The first and second indices use truncated unary binary codes and are binarized (converted from decimal to binary representation) based on the maximum value of the index. The maximum value of the index is used in the process of mapping the decimal value of the index to its binary representation. The value codeword assignment for the truncated unary binary code (truncated unary code with a maximum decimal value of 4) is as follows:

[0363]

[0364] In the table above, each decimal value requires one additional bit in its binary representation when compared to the preceding decimal value (one less than the decimal value), excluding the last decimal value corresponding to the maximum value of the index. In this case (where the index value is the maximum value, 4), it is represented by the same amount of bits in the binary representation of the preceding decimal value (which is 3). If the maximum value of the first index is 4 (=M) and the maximum value of the second index is 3 (=M-1), the following binary representation applies:

[0365]

[0366] Example 7:

[0367] The first bin of the first index is coded using the regular coding mode of Context-based Adaptive Binary Arithmetic Coding (CABAC) (using probability estimates that are coded and updated whenever the first index occurs), while the other bin is coded using the bypass mode of CABAC (using the same probability that is not updated). The regular coding mode of CABAC is described as the "Regular Arithmetic Encoder" in FIG. 14. The bypass mode is described as the "Bypass Arithmetic Encoder" in FIG. 14.

[0368] For example, the first bin of the second index is coded using the CABAC's normal coding mode (using probability estimates that are updated after coding all occurrences of the second index), while the other bin is coded using the CABAC's bypass mode (using the same unupdated probabilities). In this case, the first index is included in the bitstream by the CABAC (or parsed from the bitstream by a decoder), where the first bin of the binary first index is coded by the CABAC using a first probability estimation model that uses the CABAC's normal mode coding mode. The remaining bin of the binary first index is coded using the CABAC's bypass mode. The second index is included in the bitstream by the CABAC (or parsed from the bitstream by a decoder), where the first bin of the binary second index is coded by the CABAC using a second probability estimation model that uses the CABAC's normal mode coding mode. The remaining bin of the binary second index is coded using the CABAC's bypass mode.

[0369] As another example, the first bin of the first index and the first bin of the second index are coded using the normal coding mode of CABAC, and the same probability estimation model is used. The probability estimation model is updated after coding the first index and after coding the second index. In this case, the first index is included in the bitstream by CABAC (or parsed from the bitstream by a decoder), where the first bin of the binarized first index is coded by CABAC using a first probability estimation model that uses the normal mode coding mode of CABAC. The remaining bin of the binarized first index is coded using the bypass mode of CABAC. The second index is included in the bitstream by CABAC (or parsed from the bitstream by a decoder), where the first bin of the binarized second index is coded by CABAC using a first probability estimation model that uses the normal mode coding mode of CABAC. The remaining bin of the binarized second index is coded using the bypass mode of CABAC.

[0370] The probability estimation model explains the probability of bins having a value of "1" instead of "0". The probability estimation model is updated to adapt to statistics, for example, when the probability of observing "1" is 0.8 (meaning "0" is 0.2), bins with a value of "1" are coded by CABAC using fewer actual bits than "0".

[0371] Bin 1 is the first symbol of the binary representation, and examples are shown in the table below:

[0372]

[0373] Example 8:

[0374] If the sub-block prediction determines that the block will be applied,

[0375] 1. An indicator representing the direction of division is included in the bitstream.

[0376] 2. When the block size is less than or equal to the specified threshold

[0377] 2.1 The initial first index is contained in a bitstream that can have a maximum value of M (M is a positive integer and M N, N is the number of candidates in the merge candidate list).

[0378] 2.2 The initial second index is contained in a bitstream that can have a maximum value of Mm (where m is a positive integer and m <M, m은 미리 정의된 값이다).

[0379] 3. Otherwise (if the block size is larger than the specified threshold)

[0380] 3.1 The bitstream includes a first index that can have a maximum value of P (P is a positive integer and m <P N, N is the number of candidates in the motion information candidate list).

[0381] 3.2 The second index is contained in a bitstream that can have the maximum value of Pm (m is a positive integer and m <P, m은 미리 정의된 값이다).

[0382] 4. If the value of the initial second index is greater than or equal to the value of the initial first index, the value of the initial second index is increased by a predefined number to obtain the updated second index (e.g., the predefined number may be 1).

[0383] 5. The first sub-block portion of the block is predicted by the application of motion information candidates determined based on the initial first index.

[0384] 6. A second sub-block portion of the block is predicted by applying motion information candidates determined based on the updated second index.

[0385] Here, P and M are positive integer values ​​and m <P It is N.

[0386] The initial first index and the initial second index can be binarized as follows:

[0387] If the block size is less than or equal to the specified threshold:

[0388] (1) The first index is based on the maximum value M and is converted from decimal to binary by applying truncated binary coding.

[0389] (2) The second index is converted from decimal to binary by applying binary coding truncated based on the maximum value M-1.

[0390] If the block size is larger than the specified threshold:

[0391] (1) The first index is converted from decimal to binary by applying binary coding that is truncated based on the maximum value P.

[0392] (2) The second index is converted from decimal to binary by applying binary coding truncated based on the maximum value P-1.

[0393] The threshold value can be a positive integer signaled in the bitstream or a predefined number.

[0394] To determine if a block is larger than a threshold, the product of the block's width and height can be compared to the threshold (Width x Height > Threshold). In other examples, both the width and height can be compared to the threshold, and if both are greater than the threshold (Width > Threshold and Height > Threshold), the block can be considered larger than the threshold. In other examples, if either the width or the height is greater than the threshold (Width > Threshold or Height > Threshold), the block can be considered larger than the threshold.

[0395] Note that the splitting direction indicator, the first index, and the second index do not need to follow a specific order in the bitstream structure.

[0396] In an executable implementation, there can be two directions of partitioning:

[0397] (1) Split from the top-left corner to the bottom-right corner of the block

[0398] (2) Divide the block from the top right corner to the bottom left corner.

[0399] In other viable implementations, there may be four partition directions.

[0400] (1) Split from the top-left corner to the bottom-right corner of the block

[0401] (2) Divide the block from the top right corner to the bottom left corner.

[0402] (3) Split from the center top point of the block to the center bottom point

[0403] (4) Divide the block from the right center point to the left center point.

[0404] In an executable implementation, the split direction indicator is not included in the bitstream (or is not parsed from it).

[0405] Example 9:

[0406] If the sub-block prediction determines that the block will be applied,

[0407] 1. An initial first index is included in a bitstream that can have a maximum value of M. (M is an integer and M N, N is the number of candidates in the motion information candidate list.)

[0408] 2. An initial second index is included in a bitstream that can have a maximum value of M.

[0409] 3. If the value of the initial second index is greater than or equal to the value of the first index, the value of the initial second index is increased by a predefined number to obtain the updated second index (e.g., the predefined number may be 1).

[0410] 4. The first sub-block portion of the block is predicted by the application of motion candidate information determined based on the initial first index.

[0411] 5. A second sub-block portion of the block is predicted by applying motion candidate information determined based on the updated second index.

[0412] The maximum value of the initial first and second indices is set to M if M is less than or equal to N-1, where N is the size of the motion information candidate list (number of motion candidates). Otherwise, the maximum value of the first and second indices is set to N-1.

[0413] N is the size of the merge candidate list that can be constructed according to the methods of ITU-T H.265 and VVC. The first index and the second index used in the embodiment point to another motion information candidate list consisting only of single prediction candidates. The motion information candidate list does not correspond to the merge candidate list, as the merge candidate list may contain double prediction candidates, which are prohibited from use when it is determined that a block applies sub-block (or triangular) prediction. In this case, since each sub-block must apply single prediction motion information, the first index and the second index point to an entry in the motion information candidate list containing only single prediction candidates. The motion information candidate list can be constructed using the same spatially and temporally adjacent blocks used in the construction of the merge candidate list. In another example, the motion information candidate list can be constructed based on the merge candidate list by converting the double prediction candidates of the merge candidate list into single prediction candidates.

[0414] In an executable implementation, N can be derived from an indicator contained in the bitstream, and M can be a predefined number.

[0415] In another executable implementation, N can be derived from an indicator contained in the bitstream and M can be derived from an indicator contained in the bitstream. In this case, the indicator used to derive the value of M cannot represent a value of M greater than N-1.

[0416] In a specific example, it is assumed that the motion information candidate list includes three entries: a first motion information candidate (first candidate), a second motion information candidate (second candidate), and a third motion information candidate (third candidate).

[0417] The values ​​of the 1st and 2nd indices are shown as examples in the table below:

[0418]

[0419] Note that the third motion information candidate cannot be selected to be applied to the first sub-block of the block. The advantage is that the maximum values ​​of the first index and the second index included in the bitstream are the same (1 in the example above). Therefore, the same binarization method (binary coding truncated based on the maximum value 1) can be applied to binarize both the first index and the second index, and the maximum values ​​of the first index and the second index are set to be the same. This feature has the additional advantage of using the same binarization method for both the first and second merged indices when using truncated binary coding.

[0420] Example 10:

[0421] If the sub-block prediction decides to apply the block,

[0422] 1. The bitstream contains the first index that can have the maximum value of M. ((M is an integer and M N, N is the number of candidates in the motion information candidate list)

[0423] 2. An initial second index is included in a bitstream that can have a maximum value of M.

[0424] 3. The first sub-block portion of the block is predicted by the application of motion candidate information determined based on the initial first index.

[0425] 4. The second sub-block portion of the block is predicted by the application of motion candidate information determined based on the initial second index.

[0426] In a feasible implementation, the first index and the second index are used to select items from the motion information candidate list (the same list is used to select motion information with the first index and the second index), and the selected motion information candidate is applied to the first sub-block and the second sub-block of the block for predicting the block. In a feasible implementation, the motion information candidate list may consist only of single-predicted motion information candidates. Note that the merge candidate list (e.g., the merge candidate list of ITU-T H.265) may consist of single-predicted and double-predicted motion information candidates. Therefore, the motion information candidate list used in this embodiment may differ from the merge candidate list of ITU-T H265.

[0427] The motion information candidate list does not match the merge candidate list, which may contain double prediction candidates, and which are prohibited from use when it is determined that a block applies sub-block (or triangular) prediction. In this case, since each sub-block must apply single prediction motion information, the first and second indices point to entries in the motion information candidate list that contain only single prediction candidates. The motion information candidate list may be constructed using the same spatially and temporally adjacent blocks used in the construction of the merge candidate list. In another example, the motion information candidate list may be constructed based on the merge candidate list by converting the double prediction candidates of the merge candidate list into single prediction candidates.

[0428] Note that the first and second indices do not need to follow a specific order in the bitstream structure.

[0429] The same operations are performed in the encoder and decoder, except that the index is included in the bitstream. In the decoder, the index is parsed from the bitstream, whereas the index is included in the bitstream by the encoder.

[0430] In a specific example, assume a motion information candidate list containing motion information candidates with six items. Also, assume that the first motion candidate of the motion information candidate list is applied to the first sub-block for prediction and the fifth motion candidate of the motion information candidate list is applied to the second sub-block for prediction.

[0431] On the encoder side:

[0432] 1. A value of 0 is included in the bitstream to represent the value of the first index. (Index value 0 corresponds to the first entry of the motion information candidate list, value 1 corresponds to the second entry, and so on.)

[0433] 2. Value 3 is included in the bitstream to represent the value of the second index.

[0434] 3. The first index is determined to point to the first motion candidate in the motion information candidate list. It is applied to predict the first sub-block of the block.

[0435] 4. The second index is determined to point to the fourth motion candidate in the motion information candidate list. It is applied to predict the second sub-block of the block.

[0436] On the decoder side:

[0437] 1. The value 0 is parsed from the bitstream to represent the value of the first index.

[0438] 2. Value 3 is parsed from the bitstream to represent the value of the second index.

[0439] 3. The first index is determined to point to the first motion candidate in the motion information candidate list. It is applied to predict the first sub-block of the block.

[0440] 4. The second index is determined to point to the fourth motion candidate in the motion information candidate list. It is applied to predict the second sub-block of the block.

[0441] In another specific example, it is assumed that the motion information candidate list includes three entries: a first motion information candidate (first candidate), a second motion information candidate (second candidate), and a third motion information candidate (third candidate).

[0442] The values ​​of the first and second indices are as follows:

[0443]

[0444] Example 11:

[0445] If the sub-block prediction determines that the block will be applied,

[0446] 1. An indicator representing the direction of division is included in the bitstream.

[0447] 2. The bitstream contains the first index that can have the maximum value of M. (M is an integer and M N, N is the number of candidates in the motion information candidate list)

[0448] 3. An initial second index is included in a bitstream that can have a maximum value of Mm (m is an integer and m <M, m은 미리 정의된 값이다).

[0449] 4. If the value of the initial second index is greater than or equal to the value of the first index, the value of the initial second index is increased by a predefined number to obtain the updated second index (e.g., the predefined number may be 1).

[0450] 5. The first sub-block portion of the block is predicted by the application of motion candidate information determined based on the initial first index.

[0451] 6. A second sub-block portion of the block is predicted by applying motion candidate information determined based on the updated second index.

[0452] N is the size of the merge candidate list that can be configured according to the methods of ITU-T H.265 and VVC. The first index and the second index used in the embodiment point to another motion information candidate list composed only of single prediction candidates. The motion information candidate list does not correspond to the merge candidate list, as the merge candidate list may contain double prediction candidates, which are prohibited from use when it is determined that a block applies sub-block (or triangular) prediction. In this case, since each sub-block must apply single prediction motion information, the first index and the second index point to an entry in the motion information candidate list containing only single prediction candidates. The motion information candidate list can be configured using the same spatially and temporally adjacent blocks used in the configuration of the merge candidate list. In another example, the motion information candidate list can be configured based on the merge candidate list by converting the double prediction candidates in the merge candidate list into single prediction candidates.

[0453] The maximum value of the initial first index is set to be equal to M.

[0454] In an executable implementation, N can be derived from an indicator contained in the bitstream, and M can be a predefined number.

[0455] In another executable implementation, N can be derived from an indicator contained in the bitstream and M can be derived from an indicator contained in the bitstream. In this case, the indicator used to derive the value of M cannot represent a value of M greater than N-1.

[0456] In other executable implementations, N is derived from the indices contained in the bitstream, and M can be derived from N. For example, M can be derived from N as follows.

[0457] If N is 1, M is equal to 0 (sub-block prediction is not used and the syntax element corresponding to the sub-block prediction is not signaled).

[0458] If N ≤ 2, M is equal to N.

[0459] For example, M can be derived from N according to the following table:

[0460]

[0461] In other executable implementations, N is derived from the indices contained in the bitstream, and M can be derived from N. For example, M can be derived from N as follows:

[0462] If N is 1, M is equal to 0 (sub-block prediction is not used and the syntax element corresponding to the sub-block prediction is not signaled).

[0463] N 2 and N In the case of K, M is equal to N, where K is a predefined integer value (for example, K can be equal to 5).

[0464] If N > K, then M is equal to K.

[0465] For example, M can be derived from N according to the following table (K is equal to 5):

[0466]

[0467] Example 12:

[0468] If the sub-block prediction decides to apply the block,

[0469] 1. An indicator representing the direction of division is included in the bitstream.

[0470] 2. An initial first index is included in a bitstream that can have a maximum value of M (M is an integer).

[0471] 3. An initial second index is included in a bitstream that can have a maximum value of Mm (m is an integer and m <M, m은 미리 정의된 값이다).

[0472] 4. If the value of the initial second index is greater than or equal to the value of the first index, the value of the initial second index is increased by a predefined number to obtain the updated second index (e.g., the predefined number may be 1).

[0473] 5. The first sub-block portion of the block is predicted by the application of motion candidate information determined based on the initial first index.

[0474] 6. A second sub-block portion of the block is predicted by applying motion candidate information determined based on the updated second index.

[0475] The first and second indices used in the embodiment point to a different motion information candidate list consisting only of single prediction candidates. The motion information candidate list does not correspond to the merge candidate list, which may include double prediction candidates, and which is prohibited from use when it is determined that a block applies sub-block (or triangular) prediction. In this case, since each sub-block must apply single prediction motion information, the first and second indices point to an entry in the motion information candidate list containing only single prediction candidates. The motion information candidate list may be constructed using the same spatially and temporally adjacent blocks used in the construction of the merge candidate list. In another example, the motion information candidate list may be constructed based on the merge candidate list by converting the double prediction candidates of the merge candidate list into single prediction candidates.

[0476] The maximum value of the initial first index is M.

[0477] In an executable implementation, M can be derived from the indicator contained in the bitstream.

[0478] M can depend on an integer value N, which is the size of the merge candidate list that can be configured according to the methods of ITU-T H.265 and VVC.

[0479] In other executable implementations, N can be derived from an indicator contained in the bitstream, and M can be a predefined number.

[0480] In other executable implementations, N is derived from the indices contained in the bitstream, and M can be derived from N. For example, M can be derived from N as follows.

[0481] If N is 1, M is equal to 2 (in some examples, sub-block mode may require an initial first index that is not equal to the updated second index).

[0482] If N≤2, M is equal to N.

[0483] For example, M can be derived from N according to the following table:

[0484]

[0485] In other executable implementations, N is derived from the index contained in the bitstream, and M can be derived from N. For example, M can be derived from N as follows: if N is 1, M is equal to 2. (In some examples, sub-block mode may require an initial first index that is not equal to the updated second index);

[0486] N≥2 and N If K, then M is equal to N. Here, K is a predefined integer value (for example, K can be equal to 5).

[0487] If N > K, then M is equal to K.

[0488] For example, M can be derived from N according to the following table (K is equal to 5):

[0489]

[0490] Note that the value of N (size of the merge candidate list) may be smaller than the value of M (max value of the initial first index).

[0491] FIG. 15 illustrates the inter prediction method of the present application. The inter prediction method is performed on image blocks in a sub-block based prediction method, such as a triangle prediction mode, for example.

[0492] In the triangle prediction mode, the current block includes a first prediction sub-block and a second prediction sub-block, for example, PU1 and PU2 of FIG. 10a or FIG. 10b. It is noted that the present application may also be implemented based on different sub-block-based prediction methods, for example, the prediction mode illustrated in FIG. 11.

[0493] S1501: Parse at least one indicator to determine the prediction mode of the current block.

[0494] Generally, inter prediction involves multiple inter prediction modes. The target inter prediction mode is selected on the encoder side using different criteria, such as the RDO procedure, for example, and is encoded in the bitstream as one or more indicators. The decoder side analyzes the bitstream to achieve the values ​​of one or more indicators and determines the target inter prediction mode based on the values ​​of one or more indicators.

[0495] In an executable implementation, the metric can be a prediction mode index.

[0496] In another viable implementation, multiple indicators are combined to determine the prediction mode.

[0497] If the determined prediction mode of the current block is a triangle prediction mode, the procedure of the present method continues.

[0498] S1502: Obtain a list of candidates for the current block.

[0499] The candidate list is obtained from the merge mode candidate list. For example, the configuration of the merge mode candidate list for triangle prediction of the current block and the configuration of the single prediction candidate list can be referenced in the section above.

[0500] Note that the candidate list for triangle prediction is derived from the merge mode candidate list. In a viable implementation, the candidate list may not be an independent list. Candidates in the candidate list can be represented as indicators pointing to candidates in the merge mode candidate list.

[0501] Note that step S1502 may be implemented after parsing the prediction information index of the first prediction sub-block and / or the second prediction sub-block. This application is not limited to this.

[0502] S1503: Parse the first index from the bitstream.

[0503] The first index is used to obtain prediction information of the first prediction sub-block.

[0504] For example, parse a syntax element representing a first index in a bitstream; find an item in a candidate list according to the first index; and obtain said item as prediction information of said first prediction sub-block.

[0505] S1504: Parse the second index from the bitstream.

[0506] The second index is used to obtain prediction information for the second prediction sub-block.

[0507] For example, parse another syntax element representing the second index in the bitstream.

[0508] S1505: Compare the first index and the second index.

[0509] S1506A: In an event where the second index is smaller than the first index, find an item in the candidate list according to the second index; and obtain said item as prediction information of said second prediction sub-block.

[0510] S1506B: In the event that the second index is greater than or equal to the first index, the second index is adjusted, and prediction information of the second prediction sub-block is obtained according to the adjusted second index.

[0511] Similar to step S1506A, the step of obtaining prediction information of a second prediction sub-block according to an adjusted second index comprises: a step of finding an item in a candidate list according to an adjusted second index; and a step of obtaining said item as prediction information of said second prediction sub-block.

[0512] In an executable implementation, adjusting the second index can increase the second index by m, where m is a positive integer.

[0513] In a viable implementation, m can be 1.

[0514] In other executable implementations, adjusting the second index may be a different calculation based on the parsed value of the second index, and the adjusted value of the second index is different from the parsed value.

[0515] In a specific implementation, steps S1505, S1506A, and S1506B may be described as follows: Assume that variables idxm and idxn, which are (adjusted) first and second indices, are derived as follows using the parsed value of the first index (merge_triangle_idx0) and the parsed value of the second index (merge_triangle_idx1) for each of the first prediction sub-block and the second prediction sub-block:

[0516] idxm = merge_triangle_idx0

[0517] idxn = merge_triangle_idx1 +(merge_triangle_idx1 >= idxm)? 1: 0

[0518] In other implementations, idxn may also be derived as merge_triangle_idx1 + (merge_triangle_idx1 > idxm). Similarly, if the second index is the same as the first index, the action of S1506A or S1506B may be performed alternatively based on different embodiments not limited to the present application.

[0519] In an executable implementation, the first index is binarized according to the truncated unary code, and the second index is binarized according to the truncated unary code.

[0520] In an executable implementation, the first bin of the first binary index is coded using the normal coding mode of CABAC, and the first bin of the second binary index is coded using the normal coding mode of CABAC.

[0521] In an executable implementation where the non-first bin of the binarized first index is coded using the CABAC bypass coding mode, the non-first bin of the binarized second index is coded using the CABAC bypass coding mode. A non-first bin refers to any other bin of the binarized first index (or binarized second index) excluding the first bin.

[0522] S1507: Obtain the predicted value of the current block based on one of the predicted information of the first predicted sub-block and the predicted information of the second predicted sub-block.

[0523] After obtaining the prediction information of the first prediction sub-block and the prediction information of the second prediction sub-block, the prediction value of the current block can be obtained based on the configuration method of the triangle prediction method described in the section above.

[0524] In a possible implementation, the prediction method further includes the step of parsing a direction indicator from a bitstream, wherein the direction indicator is used to indicate the splitting direction of the current block. For example, if the direction indicator is 0, PU1 and PU2 are split in the splitting direction as shown in the left diagram of FIG. 10a or FIG. 10b, and if the direction indicator is 1, PU1 and PU2 are split in the splitting direction as shown in the right diagram of FIG. 10a or FIG. 10b.

[0525] In an executable implementation, a direction indicator is parsed from the bitstream before parsing a first index from the bitstream, and in another executable implementation, a direction indicator is parsed from the bitstream after deriving a second index adjusted. The order of implementation is not limited in this application, which means that the direction indicator may be transmitted to a different location by the bitstream.

[0526] FIG. 16 illustrates another inter prediction method of the present application. Note is made of the codeword design of the first index and / or the second index based on the maximum allowable value of the first index and / or the second index. The decoder side cannot successfully parse the first index and / or the second index without obtaining the maximum allowable value of the first index and / or the second index.

[0527] In a possible implementation, the maximum allowable values ​​of the first index and / or the second index are obtained by both the encoder and decoder sides according to a preset protocol, for example, a preset value of a standard. In this case, the indicator representing the maximum allowable value is not signaled.

[0528] In other executable implementations, one or more indicators representing the maximum allowable value are signaled in the bitstream. Therefore, the decoder side can parse the bitstream to obtain the same value as the encoder side.

[0529] For example, note that the first prediction sub-block and the second prediction sub-block share the same candidate list, and the maximum allowable value of the first index and / or the second index can be considered as the length of the candidate list.

[0530] Note that the length of the candidate list can be encoded in the bitstream as a high-level syntax that may be included, for example, in a sequence parameter set, image parameter set, image header, or slice header. In this case, the length of the candidate list can be determined prior to step S1501.

[0531] The length of the candidate list may be encoded at the block or PU level. In this case, the length of the candidate list can be determined between steps S1502 and S1501.

[0532] The length of the candidate list is determined as follows:

[0533] S1508: Parse the first digit to determine the length of the first candidate list.

[0534] In the executable implementation, the first number is parsed directly from the bitstream.

[0535] In another executable implementation, multiple phrases are parsed from the bitstream, and the parsed phrases are combined to determine the first number.

[0536] For example, the first length is the maximum quantity of candidate prediction information for Merge Mode in the candidate list.

[0537] As described above, the list of candidates for the triangular prediction mode is derived from the list of candidates for the merge mode. The merge mode index can also be used as a first index and / or a second index. In this case, the list of candidates for the triangular prediction mode can be considered as part of the list of candidates for the merge mode, and, for example as shown in Figure 12, each candidate in the list of candidates for the triangular prediction mode corresponds to a single prediction motion information in the list of candidates for the merge mode. And the maximum number of candidate prediction information for the merge mode and the triangular prediction mode may be different.

[0538] S1509: Derive the second length of the candidate list based on the first number.

[0539] The second length is the maximum amount of candidate prediction information for a sub-block-based prediction mode in the candidate list, and the sub-block-based prediction mode is a triangular prediction mode or a geometric prediction mode. The prediction mode may be other sub-block-based prediction modes, including rectangular or non-rectangular (trapezoidal) modes. Furthermore, the triangular prediction mode and the geometric prediction mode may be integrated into a single prediction mode, which may also be included in the executable implementation.

[0540] Since the candidate list for the triangular prediction mode is derived from the candidate list for the merge mode, the first length is greater than or equal to the second length.

[0541] In an executable implementation, the second length can be obtained by subtracting a preset delta value from the first number.

[0542] In another executable implementation, the second length can be obtained by subtracting the delta value from the first number, and the delta value is parsed from the bitstream.

[0543] Note that, as illustrated in FIG. 12, a single prediction candidate list is configured for the triangular prediction mode, and a double prediction candidate list is configured for the merge mode. In the present application, each motion information of the single prediction candidate list indicated by the index is a candidate for TP-m; and each set of motion information of the bidirectional prediction candidate list indicated by the index (motion information of List0 and motion information of List1) becomes a candidate for the merge mode.

[0544] Examples can also be described as follows.

[0545] Assuming the candidate list is a candidate list in merge mode, the merge mode index is used to mark candidates in the candidate list.

[0546] S1508': Parse the first number to determine the maximum allowed candidate index in the candidate list.

[0547] The maximum allowed candidate index can be the maximum index in merge mode. In other words, it is the maximum allowed value of the merge mode index.

[0548] S1509': Obtain the maximum value index based on the maximum allowed candidate index.

[0549] The maximum value index is used to indicate the maximum limit for the first index and the second index (the same index as described in the embodiment of FIG. 15).

[0550] It is noted according to the present application that when the first index and the second index are identical, there is redundancy for the sub-block-based prediction mode. Thus, when the first index has a maximum limit MAX, the second index has a maximum limit MAX-m when parsed from the bitstream. m is a positive integer, and for example, m may be 1 in the present embodiment. Also, it is noted that the second index adjusted in the present embodiment may be the same as the maximum limit.

[0551] In possible implementations, the maximum index is obtained by calculating between the maximum allowed candidate index and a predetermined number. For example, the maximum index is equal to the maximum allowed candidate index minus the predetermined number. In another example, the maximum index is equal to the predetermined number minus the maximum allowed candidate index plus the offset value.

[0552] In another possible implementation, the second number is parsed to derive the difference between the maximum allowed candidate index and the maximum index; and the maximum index is obtained by calculating the difference between the maximum allowed candidate index and the difference. For example, the maximum index is equal to the allowed maximum candidate index minus the difference. In another example, the maximum index is equal to the difference minus the maximum allowed candidate index plus the offset value.

[0553] In another embodiment, the maximum index and the maximum allowed candidate index are signaled independently. For example, similar to S1508 or S1508', the maximum index is determined by parsing the third number.

[0554] And even if the maximum index and the maximum allowed candidate index are signaled independently or dependently, in an executable implementation, the maximum allowed candidate index, which is the maximum index in merge mode, is not smaller than the maximum index.

[0555] In a specific implementation, max_num_merge_cand_minus_max_num_triangle_cand specifies the maximum number of supported triangular merge mode candidates in the slice subtracted from MaxNuM-mergeCand (or in the image according to other embodiments). six_minus_max_num_merge_when specifies the maximum number of merge motion vector prediction (MVP) candidates supported in the slice subtracted from 6. MaxNuM-mergeCand, the maximum number of merge MVP candidates, is derived as follows: MaxNuM-mergeCand = 6 - six_minus_max_num_merge_when MaxNumTriangleMergeCand, the maximum number of triangular merge mode candidates, is derived as follows:

[0556] MaxNumTriangleMergeCand =

[0557] MaxNuM-mergeCand - max_num_merge_cand_minus_max_num_triangle_cand.

[0558] max_num_merge_cand_minus_max_num_triangle_cand can be passed by a bitstream (existence), so the value of max_num_merge_cand_minus_max_num_triangle_cand can be parsed. The value of MaxNumTriangleMergeCand must be in the range from 2 to MaxNuM-mergeCand.

[0559] Under other conditions, max_num_merge_cand_minus_max_num_triangle_cand may not be transmitted by the bitstream (does not exist). In this case, when sps_triangle_enabled_flag is 0 or MaxNuM-mergeCand is less than 2, MaxNumTriangleMergeCand is set to 0, which means that triangle merging mode is not allowed for the current slice (or current image according to other embodiments); When sps_triangle_enabled_flag is 1 and MaxNuM-mergeCand is greater than or equal to 2, max_num_merge_cand_minus_max_num_triangle_cand is estimated to be equal to pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 - 1, where sps_triangle_enabled_flag is a syntax element included in the sequence parameter set and indicates whether TP-m is allowed, and pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 is a syntax element included in the image parameter set.

[0560] pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 being equal to 0 specifies that max_num_merge_cand_minus_max_num_triangle_cand exists in the slice header of a slice referencing a set of image parameters (or the image header of an image according to another embodiment). pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 being greater than 0 specifies that max_num_merge_cand_minus_max_num_triangle_cand does not exist in the slice header of a slice referencing a set of image parameters (or the image header of an image according to another embodiment). The value of pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 must be between 0 and MaxNuM-mergeCand-1.

[0561] Alternatively, the first length and the second length can be signaled independently, which means that step S1509 can parse the second number to determine the second length of the candidate list.

[0562] Similarly, in an executable implementation, the second number is parsed directly from the bitstream. In another executable implementation, multiple phrases are parsed from the bitstream and the parsed phrases are combined to determine the second number.

[0563] Note that any information parsed from the bitstream to directly or indirectly indicate the first length and / or the second length (e.g., the difference value between the two lengths) may be transmitted by the bitstream in a sequence parameter set, image parameter set, photo header, or slice header, etc.

[0564] FIG. 17 illustrates an inter prediction device (1700) of the present application.

[0565] As a current block comprising a first prediction sub-block and a second prediction sub-block, the device (1700) comprises: a parsing module (1701) configured to parse a first index from a bitstream, wherein the first index is a first prediction sub-block; a locating module (1702) configured to parse a second index from a bitstream and compare the first index with the second index; an acquisition module (1703) configured to adjust the second index if the second index is greater than or equal to the first index; and an acquisition module (1703) configured to acquire prediction information of the second prediction sub-block according to the adjusted second index.

[0566] In an executable implementation, the locating module (1702) is configured to increase the second index by m, where m is a positive integer.

[0567] In a feasible implementation, here m is 1.

[0568] In an executable implementation, before parsing a first index from a bitstream, the parsing module (1701) is further configured to parse at least one index to determine the prediction mode of the current block, wherein the prediction mode is a triangular prediction mode or a geometric prediction mode. The prediction mode may be other sub-block-based prediction modes, including a rectangular or non-rectangular (trapezoidal) mode. And the triangular prediction mode and the geometric prediction mode may be combined into a single prediction mode, which may also be included in the executable implementation.

[0569] In an executable implementation, the locating module (1702) is further configured to obtain a list of candidates for the current block.

[0570] In a feasible implementation, the prediction information of the first prediction sub-block is obtained from a candidate list according to the first index.

[0571] In a possible implementation, the prediction information of the second prediction sub-block is obtained from the candidate list according to the adjusted second index.

[0572] In a possible implementation, the candidate list is a candidate list in merge mode.

[0573] In a possible implementation, the parsing module (1701) parses a first number to determine the maximum allowed candidate index in the candidate list; obtains a maximum index based on the maximum allowed candidate index, wherein the first index is not greater than the maximum index.

[0574] In a possible implementation, the parsing module (1701) is configured to obtain the maximum index by calculating between the maximum allowed candidate index and a predetermined number.

[0575] In a possible implementation, the parsing module (1701) parses a second number to derive the difference between the maximum allowed candidate index and the maximum index; and obtains the maximum index by calculating the difference between the maximum allowed candidate index and the maximum index.

[0576] In an executable implementation, the parsing module (1701) is configured to parse a third number to determine the maximum index.

[0577] In possible implementations, the maximum allowed candidate index is not smaller than the maximum index.

[0578] In a feasible implementation, after obtaining prediction information of a second prediction sub-block according to an adjusted second index, the acquisition module (1703) is also configured to: obtain a prediction value of the current block based on the prediction information of the first prediction sub-block and the prediction information of the second prediction sub-block.

[0579] In an executable implementation, the first index or the second index is binaryized according to a truncated unary code.

[0580] In a possible implementation, the first bin of the binary first index or second index is coded using the normal coding mode of CABAC.

[0581] In a possible implementation, the non-first bin of the binarized first index or second index is coded using the bypass coding mode of CABAC.

[0582] In a possible implementation, the parsing module (1701) is configured to parse a direction indicator from the bitstream, and the direction indicator is used to indicate the splitting direction of the current block.

[0583] FIG. 18 illustrates an inter-prediction device (1800) of the present application, wherein the device (1800) may be a decoder or an encoder. The device (1800) comprises one or more processors (1801); and a non-transient computer-readable storage medium (1802) coupled to the processor and storing a program for execution by the processor, wherein the program is configured such that when executed by the processor, the decoder performs the method of FIG. 15 or FIG. 16.

[0584] In another embodiment, a computer program product is provided that includes program code for performing the method of FIG. 15 or FIG. 16 when executed on a computer or processor.

[0585] In another embodiment, a non-transient computer-readable medium is provided that carries program code that causes the computer device to perform the method of FIG. 15 or FIG. 16 when executed by the computer device.

[0586] The following is a description of the application of the encoding and decoding methods shown in the aforementioned embodiments and the system using them.

[0587] FIG. 19 is a block diagram illustrating a content supply system (3100) for realizing a content distribution service. The content supply system (3100) includes a capture device (3102) and a terminal device (3106), and optionally includes a display (3126). The capture device (3102) communicates with the terminal device (3106) via a communication link (3104). The communication link may include the communication channel (13) described above. The communication link (3104) includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination of any kind thereof.

[0588] The capture device (3102) may generate data and encode the data using an encoding method as illustrated in the above embodiment. Alternatively, the capture device (3102) may distribute the data to a streaming server (not illustrated in the drawing), and the server may encode the data and transmit the encoded data to a terminal device (3106). The capture device (3102) includes, but is not limited to, a camera, a smartphone or pad, a computer or laptop, a video conferencing system, a PDA, a vehicle-mounted device, or a combination thereof. For example, the capture device (3102) may include a source device (12) as described above. If the data includes video, the video encoder (20) included in the capture device (3102) may actually perform video encoding processing. If the data includes audio (i.e., voice), the audio encoder included in the capture device (3102) may actually perform audio encoding processing. In some actual scenarios, the capture device (3102) multiplexes the encoded video and audio data together and distributes them. For example, in other real-world scenarios such as video conferencing systems, the encoded audio data and encoded video data are not multiplexed. The capture device (3102) distributes the encoded audio data and encoded video data individually to the terminal device (3106).

[0589] In the content supply system (3100), the terminal device (310) receives and plays encoded data. The terminal device (3106) may be a device equipped with data reception and recovery functions, such as a smartphone or pad (3108), a computer or laptop (3110), a network video recorder (NVR) / digital video recorder (DVR) (3112), a TV (3114), a set-top box (STB) (3116), a video conferencing system (3118), a video surveillance system (3120), a PDA (Personal Digital Assistant) (3122), a vehicle-mounted device (3124), or any combination thereof, capable of decoding the above-described encoded data. For example, the terminal device (3106) may include a destination device (14) as described above. If the encoded data includes video, the video decoder (30) included in the terminal device has priority to perform video decoding. If the encoded data contains audio, the audio decoder included in the terminal device has priority to perform audio decoding processing.

[0590] In the case of a terminal device with a display, for example, a smartphone or pad (3108), a computer or laptop (3110), a network video recorder (NVR) / digital video recorder (DVR) (3112), a TV (3114), a personal digital assistant (PDA) (3122), or a vehicle-mounted device (3124), the terminal device can supply decoded data to the display. In the case of a terminal device without a display (for example, an STB (3116), a video conferencing system (3118), or a video surveillance system (3120)), an external display (3126) contacts the inside to receive and display the decoded data.

[0591] When each device of the present system performs encoding or decoding, an image encoding device or an image decoding device may be used as described in the above-described embodiment.

[0592] FIG. 20 is a diagram illustrating an example structure of a terminal device (3106). After the terminal device (3106) receives a stream from a capture device (3102), a protocol processing unit (3202) analyzes the transmission protocol of the stream. The protocol includes, but is not limited to, a Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real-time Transport Protocol (RTTP), Real Time Messaging Protocol (RTMP), or any combination of any kind thereof.

[0593] After the protocol processing unit (3202) processes the stream, a stream file is generated. The file is output to the demultiplexing unit (3204). The demultiplexing unit (3204) can separate the multiplexed data into encoded audio data and encoded video data. As described above, in some actual scenarios, such as video conferencing systems, the encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder (3206) and audio decoder (3208) without passing through the demultiplexing unit (3204).

[0594] Through demultiplexing, a video ES (Elementary Stream), an audio ES, and optionally subtitles are generated. A video decoder (3206), including the video decoder (30) described in the above-described embodiment, decodes the video ES by the decoding method shown in the above-described embodiment to generate a video frame and supplies this data to a synchronization unit (3212). An audio decoder (3208) decodes the audio ES to generate an audio frame and supplies this data to the synchronization unit (3212). Alternatively, the video frame may be stored in a buffer (not shown in FIG. Y) before being supplied to the synchronization unit (3212). Similarly, the audio frame may be stored in a buffer (not shown in FIG. Y) before being supplied to the synchronization unit (3212).

[0595] The synchronization unit (3212) synchronizes video frames and audio frames and supplies video / audio to a video / audio display (3214). For example, the synchronization unit (3212) synchronizes the presentation of video and audio information. The information can be coded in syntax using a time stamp related to the representation of the coded audiovisual data and a time stamp related to the delivery of the data stream itself.

[0596] If the stream contains subtitles, the subtitle decoder (3210) decodes the subtitles and synchronizes them with the video frame and audio frame to provide the video / audio / subtitles to the video / audio / subtitle display (3216).

[0597] The present invention is not limited to the aforementioned system, and the image encoding device or image decoding device in the aforementioned embodiment may be integrated into other systems, for example, automotive systems.

[0598] mathematical operators

[0599] The mathematical operators used in this application are similar to the operators used in the C programming language. However, the results of integer division and arithmetic shift operations are more precisely defined, and additional operations such as exponentiation and real number division are defined. Numbering and calculation rules generally start from 0. For example, "1st" corresponds to the 0th and "2nd" corresponds to the 1st.

[0600] arithmetic operators

[0601] The following arithmetic operators are defined as follows:

[0602]

[0603] Logical Operators The following logical operators are defined as follows:

[0604] x && y Boolean logic "logical OR" of x and y

[0605] Boolean logical "CONT" of x | | y

[0606] ! Logical Boolean "not"

[0607] x? y: zx evaluates to the value of y if x is TRUE or not 0. Otherwise, it evaluates to the value of z.

[0608] relational operators

[0609] The following relational operators are defined as follows:

[0610] Larger than

[0611] >= greater than or equal to

[0612] <

[0613] <= less than or equal to

[0614] = = Equals

[0615] ! = not equal to

[0616] When a relational operator is applied to a specified syntax element or variable with the value "na" (not applicable), the value "na" is treated as a unique value for the syntax element or variable. The value "na" is considered not to be equal to any other value.

[0617] bitwise operators

[0618] The following bitwise operators are defined as follows:

[0619] & bitwise "and". When operating on integer arguments, the operation is performed on the two's complement representation of the integer value. When operating on binary arguments containing fewer bits than other arguments, the shorter argument is extended by adding more significant bits, such as 0.

[0620] | Bitwise "or". When operating on integer arguments, the operation is performed on the two's complement representation of the integer value. When operating on binary arguments containing fewer bits than other arguments, the shorter argument is extended by adding more significant bits, such as 0.

[0621] ^ Bitwise "exclusive or". When operating on an integer argument, the operation is performed on the two's complement representation of the integer value. When operating on a binary argument containing fewer bits than other arguments, the short argument is extended by adding more significant bits, such as 0.

[0622] x >> y: Arithmetic right shift of the two's complement integer representation of x in binary. This function is defined only for non-negative integer values ​​of y. As a result of the right shift, the bit shifted to the MSB (most significant bit) has the same value as the MSB of x before the shift operation.

[0623] Arithmetic left shift of x << yy to the binary two's complement integer representation of x. This function is defined only for non-negative integer values ​​of y. As a result of the left shift, the value of the bit shifted to the least significant bit (LSB) is 0.

[0624] assignment operator

[0625] The following arithmetic operators are defined as follows:

[0626] = assignment operator

[0627] ++ increment, i.e., x++, is equivalent to x=x+1. When used for array indices, it evaluates to the variable value before the increment operation.

[0628] -- Decrement, i.e., x-- is equivalent to x=x-1. When used in array indices, it evaluates to the variable value before the decrement operation.

[0629] += increases by a specified amount, that is, x+ = 3 is equal to x=x+3 and x+ = (-3) is equal to x = x+(-3).

[0630] -= decreases by a specified amount, that is, x-= 3 is equal to x=x-3 and x- =(-3) is equal to x=x-(-3).

[0631] Range notation

[0632] The following notation is used to specify a value range:

[0633] x = y..z x takes integer values ​​from y to z, where x, y, and z are integers and z is greater than y.

[0634] mathematical functions

[0635] The following mathematical function is defined:

[0636] Abs(x) =

[0637] Asin(x) is a function that is the inverse of a trigonometric function, operates on an argument x in the range of 1.0 to 1.0, and uses an output value in radians in the range of -π÷2 to π÷2.

[0638] Atan(x) is a trigonometric inverse tangent function that operates on the argument x, and uses output values ​​in radians ranging from -π÷2 to π÷2.

[0639] Atan2( y, x ) =

[0640] Ceil(x) The smallest integer greater than or equal to x.

[0641] Clip1 Y (x) = Clip3( 0, ( 1 << BitDepth Y ) - 1,x)

[0642] Clip1 C (x) = Clip3( 0, ( 1 << BitDepth C ) - 1,x)

[0643] Clip3( x, y, z ) =

[0644] Cos(x) is a trigonometric cosine function that operates on an argument x in radians.

[0645] Floor(x) The largest integer less than or equal to x.

[0646] GetCurrMsb( a, b, c, d ) =

[0647] Ln(x) is the natural logarithm of x (logarithm with base e, where e is the basic constant of the natural logarithm, 2.718 281 828 ...).

[0648] Log2(x) is the logarithm of x with base 2.

[0649] Log10(x) is the logarithm of x with base 10.

[0650] Min( x, y ) =

[0651] Max( x, y ) =

[0652] Round(x) = Sign(x) * Floor( Abs(x) + 0.5 )

[0653] Sign(x) =

[0654] Sin(x) is a trigonometric sine function that operates on an argument x in radians.

[0655] Sqrt(x) =

[0656] Swap( x, y ) =( y,x)

[0657] Tan(x) is a trigonometric tangent function that operates on an argument x in radians.

[0658] Operation precedence

[0659] If the precedence of an expression is not explicitly indicated using parentheses, the following rules apply:

[0660] - Higher-priority operations are evaluated before lower-priority tasks.

[0661] Operations with the same priority are evaluated sequentially from left to right.

[0662] The table below assigns the priority of tasks from highest to lowest. In the table, higher positions indicate higher priority.

[0663] For operators also used in the C programming language, the precedence used in this specification is the same as the order used in the C programming language.

[0664] Table: Task priorities from highest (top of table) to lowest (bottom of table)

[0665]

[0666] Text description of logical operations

[0667] Description of logical operations mathematically explained in the text in the following format:

[0668] if(condition 0)

[0669] Question 0

[0670] else if(condition 1)

[0671] Question 1

[0672] ...

[0673] else / * Explanation of information regarding the remaining condition * /

[0674] Question n

[0675] It can be explained in the following way:

[0676] ... as follows / ... the following applies:

[0677] - If condition 0, door 0

[0678] - Otherwise, if condition 1, question 1

[0679] - ...

[0680] - Otherwise (information on residual state), document n

[0681] Each "If ... otherwise, if ... otherwise, ..." statement in the text is followed immediately by "... as follows" or "... applies to:". The final condition of "If ... otherwise ... otherwise..." is always "otherwise...". Interleaved "If ... otherwise, if ... otherwise..." statements can be identified by matching "... as follows" or "... applies to:" with the ending "otherwise, ...".

[0682] Description of logical operations mathematically explained in the text in the following format:

[0683] if(condition 0a && condition 0b)

[0684] Question 0

[0685] else if(condition 1a | | condition 1b)

[0686] Question 1 ...

[0687] else

[0688] Question n

[0689] It can be explained in the following way.

[0690] ... as follows / ... the following applies:

[0691] - If all of the following conditions are true, then question 0:

[0692] - Condition 0a

[0693] - Condition 0b

[0694] - Otherwise, if one or more of the following conditions are true, Question 1:

[0695] - Condition 1a

[0696] - Condition 1b

[0697] - ...

[0698] - Otherwise, door n

[0699] Description of logical operations mathematically explained in the text in the following format:

[0700] if(condition 0)

[0701] Question 0

[0702] if(condition 1)

[0703] Question 1

[0704] It can be explained in the following way.

[0705] When condition 0, statement 0

[0706] Condition 1, Question 1

[0707] For example, with reference to embodiments of the encoder (20) and decoder (30), and the functions described herein, for example, the encoder (20) and decoder (30), may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on a computer-readable medium or transmitted as one or more instructions or codes via a communication medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a type of medium such as a data storage medium, or a communication medium including any medium that facilitates the transmission of a computer program from one place to another according to a communication protocol, for example. In this way, the computer-readable medium may generally correspond to (1) a non-transient type of computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for the implementation of the technology described in this disclosure. A computer program product may include a computer-readable medium.

[0708] As an example rather than a limitation, such computer-readable storage media may be used to store desired program code in the form of RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disc storage devices or other magnetic storage devices, flash memory, or instructions or data structures, and may be accessible by a computer. Additionally, all connections are referred to as computer-readable media. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other transient media, but instead refer to non-transient types of storage media. The discs used herein include compact discs (CDs), laser discs, optical discs, Digital Versatile Discs (DVDs), floppy discs, and Blu-ray discs. While discs generally reproduce data magnetically, discs reproduce data optically using lasers. The above combination must also be included within the scope of computer-readable media.

[0709] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the term “processor” as used herein may refer to any other structure suitable for implementing the structure described above or the technology described herein. Additionally, in some respects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or integrated into a combined codec. Furthermore, the technology may be fully implemented in one or more circuits or logic elements.

[0710] The technology of the present disclosure may be implemented in a wide variety of devices or equipment, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). While various components, modules, or units are described in the present disclosure to highlight the functional aspects of a device configured to perform the disclosed technology, implementation by different hardware units is not necessarily required. Rather, as described above, various units may be coupled to a codec hardware unit or provided by a set of interoperable hardware units comprising one or more processors described above, together with appropriate software and / or firmware.

Claims

Claim 1 An encoding method for an image, wherein the current block of the image comprises a first prediction sub-block and a second prediction sub-block, and the encoding method comprises: a step of encoding a first index into a bitstream, wherein the value of the first index is used to select a first entry for the first prediction sub-block from a motion information candidate list; and a step of encoding a second index having a first value into the bitstream, wherein when the first value of the second index is equal to or greater than the value of the first index, a second value of the second index determined based on the first value is used to select a second entry for the second prediction sub-block from the motion information candidate list. Claim 2 An encoding method according to claim 1, wherein the second value of the second index is the same as the first value of the second index plus 1. Claim 3 An encoding method according to claim 1, wherein when the first value of the second index is smaller than the value of the first index, the first value of the second index is used to select a second entry for the second prediction sub-block from the motion information candidate list. Claim 4 An encoding method according to claim 1, further comprising the step of encoding one or more indicators into the bitstream—wherein the one or more indicators are used to determine the prediction mode of the current block, and the prediction mode is a triangular prediction mode or a geometric prediction mode. Claim 5 An encoding method according to claim 1, wherein the prediction information of the first prediction sub-block is obtained based on the first entry and the motion information candidate list. Claim 6 In paragraph 5, the encoding method wherein the prediction information of the second prediction sub-block is obtained based on the second entry and the motion information candidate list. Claim 7 In claim 1, the above motion information candidate list is an encoding method in which the candidate list is a Merge Mode. Claim 8 In claim 6, the encoding method further comprises the step of obtaining a predicted value of the current block based on the predicted information of the first predicted sub-block and the predicted information of the second predicted sub-block. Claim 9 A program stored on a computer-readable storage medium, which causes a computer to perform a method according to any one of claims 1 through 8. Claim 10 An encoding device comprising: one or more processors; and a computer-readable storage medium coupled to the one or more processors and storing a program for execution by the one or more processors, wherein the program configures the encoding device to perform a method according to any one of claims 1 to 8 when executed by the one or more processors. Claim 11 A non-transient type of computer-readable storage medium, wherein the non-transient type of computer-readable storage medium carries program code that, when executed by a computer device, causes the computer device to perform a method according to any one of claims 1 through 8. Claim 12 A bitstream storage device comprising a computer-readable medium and a transmitter, wherein the computer-readable medium is configured to store a video bitstream, the video bitstream includes data representing a current block of an image, a first index, and a second index having a first value, the current block includes a first prediction sub-block and a second prediction sub-block, the value of the first index is used to select a first entry of the first prediction sub-block from a motion information candidate list, and when the first value of the second index is equal to or greater than the value of the first index, a second value of the second index determined based on the first value is used to select a second entry for the second prediction sub-block from the motion information candidate list. Claim 13 A bitstream storage device according to claim 12, wherein the second value of the second index is the same as the first value of the second index plus 1. Claim 14 In paragraph 12 or 13, the bitstream storage device configured such that the transmitter transmits the video bitstream to another device. Claim 15 A non-transient type computer-readable storage medium carrying a bitstream, wherein the bitstream comprises data representing a current block of an image, a first index, and a second index having a first value, the current block comprises a first prediction sub-block and a second prediction sub-block, the value of the first index is used to select a first entry of the first prediction sub-block from a motion information candidate list, and when the first value of the second index is equal to or greater than the value of the first index, a second value of the second index determined based on the first value is used to select a second entry for the second prediction sub-block from the motion information candidate list.

Citation Information

Patent Citations

  • Locating motion vectors for video data units

    US20100080296A1

  • Parsing dependency reduction for palette index coding

    US20160094851A1