An encoder, a decoder and a corresponding method using history-based motion vector prediction

Abstract

The embodiment of the present invention provides an image coding method and device (encoder and / or decoder). A history-based motion vector prediction (HMVP) list corresponding to a current coding tree unit (CTU) row in a block of an image is initialized, and a CTU in the current CTU row is processed according to the initialized HMVP list.

Description

[0001] This application is a divisional application. The application number of the original application is 201980061144.9, and the original application date is December 20, 2019. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] Embodiments of the present application (invention) generally relate to the field of image processing, and more specifically, to processing images using history-based motion vector prediction (HMVP). Background Art

[0003] Video coding (video encoding and decoding) is widely used in digital video applications such as broadcast digital television, video transmission over the Internet and mobile networks, real-time conversation applications such as video chat and video conferencing, DVD and Blu-ray Discs, video content acquisition and editing systems, and camcorders for security applications.

[0004] Even in the case of short videos, large amounts of video data need to be described, which can cause difficulties when the data is to be sent or otherwise transmitted over a communication network with limited bandwidth capacity. Therefore, video data is often compressed before being sent over modern telecommunications networks. Since memory resources may be limited, the size of the video may also become an issue when storing the video in a storage device. Video compression devices typically use software and / or hardware on the source side to encode the video data before transmission or storage, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received on the destination side by a video decompression device that is used to decode the video data. With limited network resources and a growing demand for higher video quality, there is a need for improved compression and decompression techniques that can increase the compression ratio with little impact on image quality. Summary of the invention

[0005] In view of the above problems, the present invention provides a solution to alleviate or even eliminate the above problems.

[0006] The embodiments of the present application provide the encoding and decoding methods and devices described in the independent claims.

[0007] Embodiments facilitate efficient processing of images using history-based motion vector prediction, such as parallel processing. The scope of protection is defined by the claims.

[0008] Embodiments of the invention are defined by the features of the independent claims, and further advantageous implementations of the embodiments are defined by the features of the dependent claims, the description and the drawings.

[0009] Specific embodiments are outlined in the attached independent claims, further embodiments are outlined in the dependent claims.

[0010] In the present invention, the term "tile group" may also be spelled as "tilegroup". The meanings of these two terms are the same in the present invention.

[0011] The present invention provides an image decoding method implemented by a decoding device. The method comprises: initializing a history-based motion vector prediction (HMVP) list corresponding to a current coding tree unit (CTU) row in a block of an image; and processing a CTU in the current CTU row according to the initialized HMVP list.

[0012] Here, the HMVP list is initialized by setting the size of the list size to 0. This can be done by setting the variable NumHmvpCand to 0 via NumHmvpCand=0. Basically, the variable NumHmvpCand is used to track the number of valid candidates in the HMVP list, and only HMVP candidates starting with an index in the range 0...NumHmvpCand are also used to build a fusion list. Among them, initialization is performed according to predefined conditions. A block group may include one or more blocks, and a block includes one or more CTUs. The processing of CTUs is completed row by row in units of blocks from left to right. The general hierarchy is: the image contains block groups, and then the block groups contain blocks. Blocks contain CTUs, and each group of CTUs starting from the left column boundary of the block to the right column boundary of the block constitutes a CTU row inside the block.

[0013] In other words, the first CTU in each CTU row in the tile can be identified by checking whether the address of a given CTU is the same as the address of a CTU in a list, where the list stores the address of the left tile column boundary in units of CTB, which indirectly indicates the first CTU in each CTU row in the tile.

[0014] According to the aforementioned aspect, in a possible implementation manner of the method, initializing the HMVP list corresponding to the current CTU row may include the following steps:

[0015] Reset the HMVP list corresponding to the current CTU row; and / or

[0016] Setting a default value for the HMVP list corresponding to the current CTU row; and / or

[0017] The HMVP list corresponding to the current CTU row is initialized based on the HMVP list of the CTU in the previous CTU row in the coding and / or scanning order.

[0018] Here, the HMVP list may be initialized when the following conditions are met:

[0019] CtbAddrX==CtbToTileColBd[CtbAddrX].

[0020] The list CtbToTileColBd[ctbAddrX] includes the conversion from the horizontal CTB address to the address of the left tile column boundary in units of CTB, and the range of "ctbAddrX" is "0 to PicWidthInCtbsY" (including the end value). Therefore, when the horizontal CTB address CtbAddrX is the same as the address of the left tile column boundary, the following equation can be satisfied:

[0021] CtbAddrX==CtbToTileColBd[CtbAddrX].

[0022] The address of the left block column boundary is basically the first CTU in each CTU row in the block. When the above conditions are met, NumHmvpCand is set to 0 (NumHmvpCand=0). This is equivalent to resetting or clearing the HMVP list.

[0023] According to the aforementioned aspect or any aforementioned implementation of the aforementioned aspect, in one possible implementation of the method, the previous CTU row in the coding and / or scanning order is the previous CTU row adjacent to the current CTU row in the coding and / or scanning order.

[0024] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in one possible implementation manner of the method, the previous CTU row that is adjacent to the current CTU row in encoding and / or scanning order is a CTU row that is spatially adjacent or directly adjacent to the current CTU row.

[0025] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the operation of initializing the HMVP list corresponding to the current CTU row is performed at the beginning of a tile group (TG), at the beginning of the tile, or at the beginning of each CTU row in the tile.

[0026] Here, the three situations may be consistent. The HMVP list may be initialized when the following conditions are met:

[0027] CtbAddrX==CtbToTileColBd[CtbAddrX].

[0028] The list CtbToTileColBd[ctbAddrX] includes the conversion from the horizontal CTB address to the address of the left tile column boundary in units of CTB, and the range of "ctbAddrX" is "0 to PicWidthInCtbsY" (including the end value). Therefore, when the horizontal CTB address CtbAddrX is the same as the address of the left tile column boundary, the following conditions can be met:

[0029] CtbAddrX==CtbToTileColBd[CtbAddrX].

[0030] Among them, the address of the left block column boundary is basically the first CTU in each CTU row in the block. Because the HMVP list is reset at the beginning of each row, the list is of course initialized at the beginning of each block and each block group, see the above description of the processing of block groups or blocks and CTUs. Then, when the condition NumHmvpCand=0 is met, the HMVP list can be reset or cleared.

[0031] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the operation of initializing the HMVP list corresponding to the current CTU row is performed before processing the first CTU in the coding and / or scanning order in the current CTU row, wherein the first CTU in the current CTU row is or includes:

[0032] The first CTU in the coding and / or scanning order in a tile group (TG); and / or

[0033] The first CTU in the coding and / or scanning order of the partition; and / or

[0034] The first CTU in the CTU row in coding and / or scanning order.

[0035] Here, the HMVP list may be initialized when the following condition is met: CtbAddrX==CtbToTileColBd[CtbAddrX]).

[0036] The above condition is verified to be true for the beginning of a tile group or tile or the beginning of each CTU row in a tile. It should be noted that at the beginning of a tile group or tile, the verification of CtbAddrX==CtbToTileColBd[CtbAddrX] is also true. Then, when the condition NumHmvpCand=0 is met, the HMVP list can be reset or cleared.

[0037] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, initializing the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the current CTU row includes:

[0038] The length of the HMVP list is set to zero to process the first CTU in the coding or scanning order in the current CTU row.

[0039] Here, the HMVP list may be initialized when the following condition is met: CtbAddrX==CtbToTileColBd[CtbAddrX]).

[0040] The above condition is verified to be true for the beginning of a partition group or a partition or the beginning of each CTU row within a partition. Then, when the condition NumHmvpCand=0 is met, the HMVP list can be reset or cleared.

[0041] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, initializing the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the first CTU in the block allows parallel processing or decoding of the blocks.

[0042] According to the aforementioned aspect, in a possible implementation of the method, initializing the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the first CTU in each block allows parallel processing or decoding of blocks.

[0043] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the operation of initializing the HMVP list corresponding to the current CTU row is performed for each CTU row in the block before processing the first CTU in each CTU row.

[0044] According to the aforementioned aspect, in a possible implementation manner of the method, resetting the HMVP list corresponding to the current CTU row further includes:

[0045] When the block of the first CTU is encoded using the motion vector, ie, inter-frame encoding is performed, the length of the HMVP list is increased by 1, and the motion vector is added to the HMVP list.

[0046] According to any of the aforementioned implementations of the aforementioned aspect, in a possible implementation of the method, the length of the HMVP list is increased and the motion vector is added only when the HMVP list does not contain motion information.

[0047] According to any of the aforementioned implementations of the aforementioned aspect, in a possible implementation of the method, the motion information includes at least one of a motion vector and a reference image index.

[0048] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, only the length of the HMVP list is increased until the maximum length of the HMVP list is reached.

[0049] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the method further includes:

[0050] The current block of the current CTU is encoded considering only the reduced number of HMVP list candidates according to the current length of the HMVP list, wherein the current length of the HMVP list is less than the maximum length of the HMVP list.

[0051] According to any of the aforementioned implementations of the aforementioned aspect, in a possible implementation of the method, the reduced number of HMVP list candidates corresponds to or is equal to a current length of the HMVP list.

[0052] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, regardless of the size of the block, the HMVP list is reset for each CTU row in the block.

[0053] According to the aforementioned aspect or any aforementioned implementation of the aforementioned aspect, in one possible implementation of the method, when the size of the block is greater than or equal to a threshold size, the HMVP list is reset for each CTU row in the block, and / or when the size of the block is less than the threshold size, the HMVP list is not reset for each CTU row in the block.

[0054] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, setting a default value for the HMVP list corresponding to the current CTU row includes:

[0055] Filling the motion vector (MV) in the HMVP list with the MV in the unidirectional prediction mode;

[0056] Wherein, the MV in the unidirectional prediction mode is a zero motion vector or a non-zero motion vector;

[0057] The reference image comprises the first reference image in list 0 (list 0, L0); and / or

[0058] The MV in the HMVP list is filled with the MV in the bidirectional prediction mode, wherein the MV in the bidirectional prediction mode is a zero motion vector or a non-zero motion vector, and the reference image includes the first reference image in list L0 and the first reference image in list 1 (list 1, L1).

[0059] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, each concatenated image stores a time-domain HMVP list corresponding to each CTU row or corresponding to the entire image, and setting a default value for the HMVP list corresponding to the current CTU row includes:

[0060] The HMVP list corresponding to the current CTU row is initialized according to the time-domain HMVP list.

[0061] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the previous CTU row is a CTU row that is immediately adjacent to the current CTU row and is located above the current CTU row;

[0062] The CTU in the previous CTU row is the second CTU in the previous CTU row, or the CTU in the previous CTU row is the first CTU in the previous CTU row.

[0063] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the CTUs in the CTU row have corresponding addresses, and the address of the first CTU in the CTU row in the partition is identified by determining the first CTB in the CTU row by the following expression:

[0064] FirstCtbInCtuRowInTile=((ctbAddrInTs-FirstCtbAddrTs[tileIdx]-NumCtusInTile[tileIdx])%ColWidth[tileIdx]==0)? 1:0,

[0065] If((tile_group_type!=I)&&(FirstCtbInCtuRowInTile))

[0066] HMVPCandNum=0

[0067] Among them, FirstCtbInCtuRowInTile represents the first CTU in the CTU row in the tile, ctbAddrInTx represents the address of the CTB in the tile, FirstCtbAddrTs represents the first address of the CTB in the tile, tileIdx represents the index of each tile, NumCtusInTile represents the number of CTUs in the tile, ColWidth represents the width of the tile relative to the number of CTUs, "%" is the modulus operator, and tile_group_type represents the type of tile group.

[0068] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the CTUs in the CTU row have corresponding addresses, and the address of the first CTU in the CTU row is identified by determining whether a modulo operation of the address j relative to the current block produces 0 by the following expression:

[0069] if(j%ColWidth[tileIdx]==0),

[0070] Wherein, J is the address of the current CTU, ColWidth[tileIdx] represents the width of the block relative to the number of CTUs, tileIdx represents the index of each block, and "%" is the modulus operator.

[0071] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the CTUs in the CTU row have corresponding addresses, and the address of the first CTU in each CTU row in the block is identified by determining whether the address of the current CTU is the same as the address of the CTU in the list or array, thereby deriving the first CTU in each CTU row in the block, wherein the list or array stores the address of the left block column boundary in units of CTB.

[0072] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, it is determined whether the address of the current CTU is the same as the address of the CTU in the list by the following expression, wherein the list stores the address of the left partition column boundary in units of CTB:

[0073] if(CtbAddrX==CtbToTileColBd[CtbAddrX]),

[0074] Among them, the list or array CtbToTileColBd[ctbAddrX] includes the conversion from the horizontal CTB address to the address of the left tile column boundary in units of CTB, and the range of "ctbAddrX" is "0 to PicWidthInCtbsY" (including the end value), where ctbAddrX represents the index of the horizontal CTB.

[0075] On the other hand, there is provided an image encoding method implemented by an encoding device, comprising:

[0076] Initialize the History-based Motion Vector Prediction (HMVP) list corresponding to the current Coding Tree Unit (CTU) row in the image block;

[0077] The CTUs in the current CTU row are processed according to the initialized HMVP list.

[0078] According to the aforementioned aspect, in a possible implementation manner of the method, initializing the HMVP list corresponding to the current CTU row includes the following steps:

[0079] Reset the HMVP list corresponding to the current CTU row; and / or

[0080] Setting a default value for the HMVP list corresponding to the current CTU row; and / or

[0081] The HMVP list corresponding to the current CTU row is initialized based on the HMVP list of the CTU in the previous CTU row in the coding and / or scanning order.

[0082] According to the aforementioned aspect, in a possible implementation manner of the method, the previous CTU row in the coding and / or scanning order is the previous CTU row that is immediately adjacent to the current CTU row in the coding and / or scanning order.

[0083] According to the aforementioned aspect, in a possible implementation manner of the method, the previous CTU row that is immediately adjacent to the current CTU row in coding and / or scanning order is a CTU row that is spatially adjacent or directly adjacent to the current CTU row.

[0084] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the operation of initializing the HMVP list corresponding to the current CTU row is performed at the beginning of a tile group (TG), at the beginning of the tile, or at the beginning of each CTU row in the tile.

[0085] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the operation of initializing the HMVP list corresponding to the current CTU row is performed before processing the first CTU in the coding and / or scanning order in the current CTU row, wherein the first CTU in the current CTU row is or includes:

[0086] The first CTU in the coding and / or scanning order in a tile group (TG); and / or

[0087] The first CTU in the coding and / or scanning order of the partition; and / or

[0088] The first CTU in the CTU row in coding and / or scanning order.

[0089] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, initializing the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the current CTU row includes:

[0090] The length of the HMVP list is set to zero to process the first CTU in the coding or scanning order in the current CTU row.

[0091] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, initializing the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the first CTU in the block allows parallel processing or encoding of the blocks.

[0092] According to the aforementioned aspect, in a possible implementation of the method, initializing the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the first CTU in each block allows parallel processing or encoding of blocks.

[0093] According to the aforementioned aspect or any aforementioned implementation of the aforementioned aspect, in one possible implementation of the method, the operation of initializing the HMVP list corresponding to the current CTU row is performed for each CTU row in the block before processing the first CTU in each CTU row.

[0094] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, resetting the HMVP list corresponding to the current CTU row further includes:

[0095] When the block of the first CTU is encoded using the motion vector, ie, inter-frame encoding is performed, the length of the HMVP list is increased by 1, and the motion vector is added to the HMVP list.

[0096] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the length of the HMVP list is increased and the motion vector is added only when the HMVP list does not contain motion information.

[0097] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the motion information includes at least one of a motion vector and a reference image index.

[0098] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, only the length of the HMVP list is increased until the maximum length of the HMVP list is reached.

[0099] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the method further includes:

[0100] The current block of the current CTU is encoded by considering only the reduced number of HMVP list candidates according to the current length of the HMVP list, wherein the current length of the HMVP list is less than the maximum length of the HMVP list.

[0101] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the reduced number of HMVP list candidates corresponds to or is equal to a current length of the HMVP list.

[0102] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, regardless of the size of the block, the HMVP list is reset for each CTU row in the block.

[0103] According to the aforementioned aspect or any aforementioned implementation of the aforementioned aspect, in one possible implementation of the method, when the size of the block is greater than or equal to a threshold size, the HMVP list is reset for each CTU row in the block, and / or when the size of the block is less than the threshold size, the HMVP list is not reset for each CTU row in the block.

[0104] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, setting a default value for the HMVP list corresponding to the current CTU row includes:

[0105] Filling the motion vector (MV) in the HMVP list with the MV in the unidirectional prediction mode;

[0106] Wherein, the MV in the unidirectional prediction mode is a zero motion vector or a non-zero motion vector;

[0107] The reference image comprises the first reference image in list 0 (list 0, L0); and / or

[0108] The MV in the HMVP list is filled with the MV in the bidirectional prediction mode, wherein the MV in the bidirectional prediction mode is a zero motion vector or a non-zero motion vector, and the reference image includes the first reference image in list L0 and the first reference image in list 1 (list 1, L1).

[0109] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, each concatenated image stores a time-domain HMVP list corresponding to each CTU row or corresponding to the entire image, and setting a default value for the HMVP list corresponding to the current CTU row includes:

[0110] The HMVP list corresponding to the current CTU row is initialized according to the time-domain HMVP list.

[0111] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the previous CTU row is a CTU row that is immediately adjacent to the current CTU row and is located above the current CTU row;

[0112] The CTU in the previous CTU row is the second CTU in the previous CTU row, or the CTU in the previous CTU row is the first CTU in the previous CTU row.

[0113] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the CTUs in the CTU row have corresponding addresses, and the address of the first CTU in the CTU row is identified by determining whether a modulo operation of the address j relative to the current block produces 0 by the following expression:

[0114] if(j%ColWidth[tileIdx]==0),

[0115] Wherein, J is the address of the current CTU, ColWidth[tileIdx] represents the width of the block relative to the number of CTUs, tileIdx represents the index of each block, and "%" is the modulus operator.

[0116] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, the CTUs in the CTU row have corresponding addresses, and the address of the first CTU in each CTU row in the block is identified by determining whether the address of the current CTU is the same as the address of the CTU in the list or array, thereby deriving the first CTU in each CTU row in the block, wherein the list or array stores the address of the left block column boundary in units of CTB.

[0117] According to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect, in a possible implementation manner of the method, it is determined whether the address of the current CTU is the same as the address of the CTU in the list by the following expression, wherein the list stores the address of the left partition column boundary in units of CTB:

[0118] if(CtbAddrX==CtbToTileColBd[CtbAddrX]),

[0119] Among them, the list or array CtbToTileColBd[ctbAddrX] includes the conversion from the horizontal CTB address to the address of the left tile column boundary in units of CTB, and the range of "ctbAddrX" is "0 to PicWidthInCtbsY" (including the end value), where ctbAddrX represents the index of the horizontal CTB.

[0120] On the other hand, a decoder is provided, comprising a processing circuit for executing the method according to the aforementioned aspect or any aforementioned implementation of the aforementioned aspect.

[0121] On the other hand, an encoder is provided, comprising a processing circuit for executing a method according to the aforementioned aspect or any aforementioned implementation of the aforementioned aspect.

[0122] On the other hand, a computer program product is provided, comprising program code for executing the method according to the aforementioned aspect or any aforementioned implementation manner of the aforementioned aspect.

[0123] On the other hand, a computer-readable non-transitory medium is provided, wherein the computer-readable non-transitory medium stores a program, wherein the program includes instructions. When the instructions are executed on a processor, the processor performs the method according to the aforementioned aspect or any of the aforementioned implementations of the aforementioned aspect.

[0124] In another aspect, a decoder is provided, comprising:

[0125] one or more processors;

[0126] A non-transitory computer-readable storage medium is coupled to the processor and stores a program executed by the processor, wherein when the processor executes the program, the program configures the decoder to perform a method according to the aforementioned aspect or any aforementioned implementation of the aforementioned aspect.

[0127] In another aspect, an encoder is provided, comprising:

[0128] one or more processors;

[0129] A non-transitory computer-readable storage medium is coupled to the processor and stores a program executed by the processor, wherein when the processor executes the program, the program configures the encoder to perform a method according to the aforementioned aspect or any aforementioned implementation of the aforementioned aspect.

[0130] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages are apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0131] The following is a detailed description of the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. In the accompanying drawings:

[0132] Figure 1A A block diagram of an example of a video decoding system for implementing an embodiment of the present invention;

[0133] Figure 1B is a block diagram of another example of a video decoding system for implementing an embodiment of the present invention;

[0134] Figure 2 A block diagram of an example of a video encoder for implementing an embodiment of the present invention;

[0135] Figure 3 A block diagram of an example of a video decoder for implementing an embodiment of the present invention;

[0136] Figure 4 A block diagram of an example encoding device or decoding device;

[0137] Figure 5 A block diagram of another example of an encoding device or a decoding device;

[0138] Figure 6 An example of an image comprising or divided into 12 blocks is shown, wherein the 12 blocks are divided into 3 block groups;

[0139] Figure 7 Shown for Figure 6 Example of resetting of tile groups, tiles, and CTU rows within tiles;

[0140] Figure 8 An example syntax description of HMVP list reset for a partition group, a partition, and a CTU row within a partition is shown;

[0141] Fig. 9 HMVP list reset for tile groups, tiles, and CTU rows within tiles is shown;

[0142] Fig.10 An example of the location of spatially neighboring blocks for fusion candidate list construction and AMVP candidate list construction is shown;

[0143] Fig.11 Flowchart for example coding using HMVP;

[0144] Fig.12 An example of updating the HMVP table is shown;

[0145] Fig.13 An example of Wavefront Parallel Processing (WPP) is shown;

[0146] Fig.14 Another example syntax description of HMVP list reset for partition groups, partitions, and CTU rows within partitions is shown.

[0147] In the following, unless explicitly stated otherwise, identical reference signs refer to identical or at least functionally equivalent features. DETAILED DESCRIPTION

[0148] In the following description, reference is made to the accompanying drawings that form part of the present invention and that illustrate specific aspects of embodiments of the present invention or specific aspects in which embodiments of the present invention may be used by way of illustration. It should be understood that embodiments of the present invention may be used in other aspects and may include structural changes or logical changes not described in the accompanying drawings. Therefore, the following detailed description should not be understood in a restrictive sense, and the scope of the present invention is defined by the appended claims.

[0149] For example, it should be understood that the disclosure in conjunction with the described method may also be equally applicable to a corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units (e.g., functional units) to perform the described one or more method steps (e.g., one unit performs one or more steps, or each of multiple units performs one or more of multiple steps), even if the one or more units are not explicitly described or illustrated in the drawings. On the other hand, for example, if a specific device is described based on one or more units (e.g., functional units), the corresponding method may include a step to implement the function of one or more units (e.g., one step implements the function of one or more units, or each of multiple steps implements the function of one or more units in multiple units), even if the one or more steps are not explicitly described or illustrated in the drawings. In addition, it should be understood that, unless otherwise stated, the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0150] Video decoding generally refers to the processing of a sequence of images of a video or video sequence. In the field of video decoding, the terms "frame" or "picture / image" can be used as synonyms. Video decoding (or commonly referred to as decoding) includes two parts: video encoding and video decoding. Video encoding is performed on the source side and generally includes processing (e.g., compressing) the original video image to reduce the amount of data required to represent the video image (thereby storing and / or sending it more efficiently). Video decoding is performed on the destination side and generally includes inverse processing relative to the encoder to reconstruct the video image. The "encoding" of the video image (or commonly referred to as the image) involved in the embodiment should be understood as the "encoding" or "decoding" of the video image or the corresponding video sequence. The encoding part and the decoding part are also collectively referred to as codec (encoding and decoding, CODEC).

[0151] In the case of lossless video decoding, the original video image can be reconstructed, that is, the reconstructed video image has the same quality as the original video image (assuming that there is no transmission loss or other data loss during storage or transmission). In the case of lossy video decoding, further compression is performed by quantization, etc. to reduce the amount of data required to represent the video image, but the video image cannot be completely reconstructed at the decoder, that is, the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0152] Several video coding standards belong to the group of "lossy hybrid video codecs" (i.e., combining spatial prediction and temporal prediction in the pixel domain with 2D transform coding in the transform domain for applying quantization). Each image in a video sequence is typically partitioned into a set of non-overlapping blocks, and decoding is typically performed on a block-level basis. In other words, at the encoder, the video is typically processed (i.e., encoded) at the block (video block) level, for example, a prediction block is generated by spatial (intra-image) prediction and temporal (inter-image) prediction; the prediction block is subtracted from the current block (currently processed / to be processed block) to obtain a residual block; the residual block is transformed in the transform domain and quantized to reduce the amount of data to be transmitted (compressed), while at the decoder, the inverse process relative to the encoder is applied to the coded or compressed block to reconstruct the current block for representation. In addition, the encoder replicates the processing loop of the decoder so that both generate the same prediction (e.g., intra-frame prediction and inter-frame prediction) and / or reconstruction for processing (i.e., decoding) subsequent blocks.

[0153] In the following embodiment of the video decoding system 10, the video encoder 20 and the video decoder 30 are configured according to FIG. 1 to FIG. Figure 3 Give a description.

[0154] Figure 1A As a schematic block diagram, an exemplary decoding system 10 is shown, such as a video decoding system 10 (or simply a decoding system 10) that can utilize the techniques of the present application. The video encoder 20 (or simply encoder 20) and the video decoder 30 (or simply decoder 30) of the video decoding system 10 represent examples of devices that can be used to perform various techniques according to various examples described in the present application.

[0155] like Figure 1A As shown, the decoding system 10 includes a source device 12 for providing encoded image data 21 such as encoded image data 13 to a destination device 14 for decoding the encoded data 13 .

[0156] The source device 12 includes an encoder 20 and may additionally or optionally include an image source 16 , a preprocessor (or preprocessing unit) 18 such as an image preprocessor 18 , and a communication interface or communication unit 22 .

[0157] The image source 16 may include or may be any type of image acquisition device, such as a camera for acquiring real-world images, and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images, or any other device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory (memory / storage) for storing any of the above images.

[0158] In distinction from the processing performed by the pre-processor 18 and the pre-processing unit 18 , the image or image data 17 may also be referred to as a raw image or raw image data 17 .

[0159] The preprocessor 18 is used to receive (raw) image data 17 and preprocess the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. The preprocessing performed by the preprocessor 18 may include cropping, color format conversion (e.g., from RGB to YCbCr), color correction or denoising, etc. It can be understood that the preprocessing unit 18 may be an optional component.

[0160] The video encoder 20 is used to receive the pre-processed image data 19 and provide encoded image data 21 (hereinafter referred to as Figure 2 The communication interface 22 in the source device 12 may be used to receive the encoded image data 21 and transmit the encoded image data 21 (or any other processed version thereof) to another device such as the destination device 14 or any other device via the communication channel 13 for storage or direct reconstruction.

[0161] Destination device 14 includes a decoder 30 (eg, video decoder 30 ) and, in addition or alternatively, may include a communication interface or communication unit 28 , a post-processor 32 (or post-processing unit 32 ), and a display device 34 .

[0162] The communication interface 28 in the destination device 14 is used to receive the encoded image data 21 (or any other processed version) directly from the source device 12 or from any other source device such as a storage device, for example, the storage device is a coded image data storage device, and provide the encoded image data 21 to the decoder 30.

[0163] Communication interface 22 and communication interface 28 can be used to send or receive encoded image data 21 or encoded data 13 via a direct communication link (e.g., a direct wired or wireless connection) between source device 12 and destination device 14, or via any type of network (e.g., a wired or wireless network or any combination thereof, or any type of private and public networks), or any combination thereof.

[0164] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or process the encoded image data using any type of transmission coding or processing for transmission over a communication link or network.

[0165] For example, the communication interface 28 corresponding to the communication interface 22 may be used to receive the transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain the encoded image data 21 .

[0166] Both the communication interface 22 and the communication interface 28 can be configured as follows Figure 1A The unidirectional communication interface indicated by the arrow pointing from the source device 12 to the corresponding communication channel 13 of the destination device 14, or configured as a bidirectional communication interface, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded image data transmission, etc.

[0167] The decoder 30 is used to receive the encoded image data 21 and provide decoded image data 31 or decoded image 31 (hereinafter referred to as Figure 3 or Figure 5 etc. for further description).

[0168] The post-processor 32 of the destination device 14 is used to post-process the decoded image data 31 (also referred to as reconstructed image data) (e.g., decoded image 31) to obtain post-processed image data 33 (e.g., post-processed image 33). The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., conversion from YCbCr to RGB), color matching, cropping or resampling, or any other processing for generating decoded image data 31 for display by a display device 34 or the like.

[0169] The display device 34 of the destination device 14 is used to receive the post-processed image data 33 to display the image to a user or viewer, etc. The display device 34 can be or include any type of display for displaying the reconstructed image, such as an integrated or external display screen or display. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0170] although Figure 1A The source device 12 and the destination device 14 are described as separate devices, but the device embodiment may also include two devices or two functions, namely the source device 12 or the corresponding function and the destination device 14 or the corresponding function. In such embodiments, the source device 12 or the corresponding function and the destination device 14 or the corresponding function may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.

[0171] According to the description, Figure 1A The presence and (exact) division of different units or functionalities in the source device 12 and / or destination device 14 shown may vary depending on the actual device and application, as will be apparent to the skilled person.

[0172] The encoder 20 (eg, video encoder 20) or the decoder 30 (eg, video decoder 30), or both the encoder 20 and the decoder 30, may be configured to receive the video signal via a processor such as a decoder. Figure 1B The processing circuits shown are implemented, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, video decoding dedicated processors, or any combination thereof. The encoder 20 may be implemented by processing circuits 46 to include reference Figure 2 The various modules discussed in encoder 20 and / or any other encoder system or subsystem described herein. Decoder 30 may be implemented by processing circuit 46 to include reference Figure 3The processing circuits may be used to perform the various operations described below. Figure 5 As shown, if part of the technology is implemented in software, the device can store the instructions of the software in a suitable non-transitory computer-readable storage medium and use one or more processors to execute the instructions in hardware to perform the technology of the present invention. Any of the video encoder 20 and the video decoder 30 can be integrated into a single device as part of a combined codec (encoder / decoder, CODEC), such as Figure 1B shown.

[0173] Source device 12 and destination device 14 may include any of a variety of devices, including any type of handheld or fixed device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet / tabletcomputer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content distribution server), a broadcast receiver device, a broadcast transmitter device, etc., and may not use or use any type of operating system. In some cases, source device 12 and destination device 14 may be equipped with components for wireless communication. Therefore, source device 12 and destination device 14 may be wireless communication devices.

[0174] In some cases, Figure 1A The video decoding system 10 shown is merely an example, and the techniques of the present application may be applicable to video decoding arrangements (e.g., video encoding or video decoding) that do not necessarily include any data communication between the encoding device and the decoding device. In other examples, data is retrieved from a local memory, sent over a network, and so on. The video encoding device may encode the data and store the data in a memory, and / or the video decoding device may retrieve the data from the memory and decode the data. In some examples, the encoding and decoding are performed by devices that do not communicate with each other but only encode the data to the memory and / or retrieve the data from the memory and decode the data.

[0175] For ease of description, the embodiments of the present invention are described with reference to the Versatile video coding (VVC) reference software or the High-Efficiency Video Coding (HEVC) developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG). It should be understood by those skilled in the art that the embodiments of the present invention are not limited to HEVC or VVC.

[0176] Encoders and encoding methods

[0177] Figure 2 FIG. 2 is a schematic block diagram of an exemplary video encoder 20 for implementing the technology of the present application. Figure 2 In the example of , the video encoder 20 includes an input terminal 201 (or an input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filtering unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output terminal 272 (or an output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a segmentation unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The illustrated video encoder 20 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[0178] The residual calculation unit 204, the transform processing unit 206, the quantization unit 208, and the mode selection unit 260 may be referred to as a forward signal path constituting the encoder 20, and the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244, and the intra-frame prediction unit 254 may be referred to as a reverse signal path constituting the video encoder 20. The reverse signal path of the video encoder 20 is similar to that of the decoder (see Figure 3The inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244 and the intra-frame prediction unit 254 also constitute a "built-in decoder" of the video encoder 20.

[0179] Images and Image Segmentation (Images and Blocks)

[0180] The encoder 20 may be used to receive, via an input 201 or the like, an image 17 (or image data 17), e.g., an image in a sequence of images forming a video or a video sequence. The received image or image data may also be a pre-processed image 19 (or pre-processed image data 19). For simplicity, the following description uses image 17. Image 17 may also be referred to as a current image or an image to be encoded (particularly when distinguishing the current image from other images in video decoding, such as previously encoded images and / or decoded images in the same video sequence, i.e., a video sequence also including the current image).

[0181] A (digital) image is or can be considered as a two-dimensional array or matrix of pixels with intensity values. The pixels in the array can also be called pixels (pixel or pel) (short for picture elements). The size and / or resolution of the image is defined by the number of pixels in the array or image in the horizontal and vertical directions (or axes). In order to represent color, three color components are usually used, that is, the image can be represented as or includes three pixel arrays. In the RGB format or color space, the image includes corresponding red, green and blue pixel arrays. However, in video decoding, each pixel is usually represented by a brightness and chrominance format or in a color space, for example, YCbCr, including a brightness component represented by Y (sometimes also indicated by L) and two chrominance components represented by Cb and Cr. The brightness (or luma for short) component Y represents the brightness or grayscale intensity (for example, in a grayscale image), while the two chrominance (or chroma for short) components Cb and Cr represent the chrominance or color information components. Therefore, an image in YCbCr format includes a luminance pixel array of luminance pixel values ​​(Y) and two chrominance pixel arrays of chrominance values ​​(Cb and Cr). An image in RGB format can be converted or transformed into YCbCr format and vice versa, a process also referred to as color conversion or conversion. If the image is monochrome, the image may include only a luminance pixel array. Accordingly, for example, the image may be a luminance pixel array in monochrome format or a luminance pixel array and two corresponding chrominance pixel arrays in 4:2:0, 4:2:2 and 4:4:4 color formats.

[0182] In one embodiment, an embodiment of the video encoder 20 may include an image segmentation unit ( Figure 2 203) for partitioning the image 17 into a plurality of (usually non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264 / AVC) or coding tree blocks (CTB), or coding tree units (CTU) (H.265 / HEVC and VVC). The partitioning unit may be used to use the same block size for all images in a video sequence and a corresponding grid of defined block sizes, or to change the block size between images or subsets or groups of images, and to partition each image into corresponding blocks.

[0183] In other embodiments, the video encoder may be configured to directly receive a block 203 of the image 17, for example, one, several or all blocks constituting the image 17. The image block 203 may also be referred to as a current image block or an image block to be decoded.

[0184] Like the image 17, the image block 203 is also or can be considered as a two-dimensional array or matrix composed of pixels with intensity values ​​(pixel values), but the image block 203 is smaller than that of the image 17. In other words, the block 203 may include one pixel array (e.g., a brightness array in the case of the monochrome image 17 or a brightness array or a color difference array in the case of the color image) or three pixel arrays (e.g., one brightness array and two color difference arrays in the case of the color image 17) or any other number and / or type of arrays according to the color format adopted. The number of pixels in the horizontal and vertical directions (or axes) of the block 203 defines the size of the block 203. Therefore, the block can be an M×N (M columns×N rows) pixel array, or an M×N transform coefficient array, etc.

[0185] Figure 2 The illustrated embodiment of the video encoder 20 may be used to encode the image 17 block by block, for example, by encoding and predicting the block 203 .

[0186] Residual calculation

[0187] The residual calculation unit 204 can be used to calculate the residual block 205 (also referred to as residual 205) based on the image block 203 and the prediction block 265 (the prediction block 265 will be described in detail below) in the following manner: subtract the pixel point value of the prediction block 265 from the pixel point value of the image block 203 pixel by pixel (pixel by pixel) to obtain the residual block 205 in the pixel point domain.

[0188] Transform

[0189] The transform processing unit 206 may be used to apply a transform such as discrete cosine transform (DCT) or discrete sine transform (DST) to the pixel values ​​of the residual block 205 to obtain a transform coefficient 207 in a transform domain. The transform coefficient 207 may also be referred to as a transform residual coefficient, representing the residual block 205 in the transform domain.

[0190] The transform processing unit 206 can be used to apply an integerized approximation of the DCT / DST, such as the transform specified for H.265 / HEVC. Compared to the orthogonal DCT transform, the integer approximation is usually scaled by a certain factor. In order to maintain the norm of the residual block processed by the forward transform and the inverse transform, other scaling factors are used as part of the transform process. The scaling factor is usually selected based on certain constraints, such as whether the scaling factor is a power of 2 for the shift operation, the bit depth of the transform coefficients, the trade-off between accuracy and implementation cost, etc. For example, a specific scaling factor is specified for the inverse transform by the inverse transform processing unit 212, etc. (and for the corresponding inverse transform at the video decoder 30 by the inverse transform processing unit 312, etc.), and accordingly, the corresponding scaling factor can be specified for the forward transform at the encoder 20 by the transform processing unit 206, etc.

[0191] Embodiments of the video encoder 20 (specifically the transform processing unit 206) may be used to encode or compress output transform parameters (e.g., one or more transform types) directly or via an entropy coding unit 270, such that, for example, the video decoder 30 may receive and use the transform parameters for decoding.

[0192] Quantification

[0193] The quantization unit 208 may be configured to quantize the transform coefficients 207 by applying scalar quantization or vector quantization, etc., to obtain quantized coefficients 209. The quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209.

[0194] The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different degrees of scaling may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to a finer quantization, while a larger quantization step size corresponds to a coarser quantization. A suitable quantization step size may be represented by a quantization parameter (QP). For example, a quantization parameter may be an index of a predefined set of suitable quantization step sizes. For example, a smaller quantization parameter may correspond to fine quantization (a smaller quantization step size), a larger quantization parameter may correspond to coarse quantization (a larger quantization step size), and vice versa. Quantization may include dividing by a quantization step size, while corresponding and / or inverse quantization performed by an inverse quantization unit 210 or the like may include multiplying by a quantization step size. Embodiments according to some standards such as HEVC may be used to determine a quantization step size using a quantization parameter. In general, the quantization step size may be calculated using a fixed-point approximation of an equation containing division according to the quantization parameter. Quantization and dequantization may introduce additional scaling factors to restore the norm of the residual block, which may be modified due to the use of scaling in the fixed-point approximation of the equations for the quantization step size and the quantization parameter. In an exemplary implementation, the scales of the inverse transform and dequantization may be combined. Alternatively, a custom quantization table may be used and signaled from the encoder to the decoder, for example in the bitstream. Quantization is a lossy operation, where the larger the quantization step size, the greater the loss.

[0195] In one embodiment, the video encoder 20 (correspondingly, the quantization unit 208) may be configured to output a quantization parameter (QP), for example, directly output or output after being encoded or compressed by the entropy coding unit 270, for example, so that the video decoder 30 may receive and use the quantization parameter for decoding.

[0196] Dequantization

[0197] The inverse quantization unit 210 is used to perform inverse quantization of the quantization unit 208 on the quantized coefficients to obtain dequantized coefficients 211, for example, performing an inverse quantization scheme of the quantization scheme performed by the quantization unit 208 according to or using the same quantization step size as the quantization unit 208. The dequantized coefficients 211 may also be referred to as dequantized residual coefficients 211, corresponding to the transform coefficients 207, but due to the loss caused by quantization, the dequantized coefficients 211 are usually not completely the same as the transform coefficients.

[0198] Inverse Transform

[0199] The inverse transform processing unit 212 is used to apply the inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain a reconstructed residual block 213 (or a corresponding dequantized coefficient 213) in the pixel domain. The reconstructed residual block 213 may also be referred to as a transform block 213.

[0200] reconstruction

[0201] The reconstruction unit 214 (e.g., the summer 214) is used to add the transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain the reconstructed block 215 in the pixel point domain, for example, by adding the pixel point values ​​of the reconstructed residual block 213 and the pixel point values ​​of the prediction block 265.

[0202] Filtering

[0203] The loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstruction block 215 to obtain the filter block 221, or is generally used to filter the reconstructed pixels to obtain the filtered pixels. For example, the loop filter unit is used to smooth pixel transitions or improve video quality. The loop filter unit 220 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening, a smoothing filter, or a collaborative filter or any combination thereof. Although the loop filter unit 220 is Figure 2 2. Although shown as a loop filter in FIG. 2, in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filter block 221 may also be referred to as a filter reconstruction block 221.

[0204] An embodiment of the video encoder 20 (specifically the loop filter unit 220) can be used to encode output loop filter parameters (such as pixel adaptive offset information) directly or via the entropy coding unit 270, so that, for example, the decoder 30 can receive and apply the same loop filter parameters or the corresponding loop filter for decoding.

[0205] Decoded Image Buffer

[0206] The decoded picture buffer (DPB) 230 may be a reference picture memory for storing reference picture data for use by the video encoder 20 when encoding video data. The DPB 230 may be formed by any of a variety of memory devices, such as a dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of storage devices. The decoded picture buffer (DPB) 230 may be used to store one or more filter blocks 221. The decoded picture buffer 230 may also be used to store other previous filter blocks of the same current image or a different image such as a previous reconstructed image, such as a previously reconstructed and filtered block 221, and may provide a complete previously reconstructed, i.e., decoded image (and corresponding reference blocks and pixels) and / or a partially reconstructed current image (and corresponding reference blocks and pixels), such as for inter-frame prediction. The decoded image buffer 230 may also be used to store one or more unfiltered reconstructed blocks 215, or generally to store unfiltered reconstructed pixels, for example, reconstructed blocks 215 that have not been filtered by the loop filter unit 220, or reconstructed blocks or reconstructed pixels that have not undergone any other processing.

[0207] Mode selection (segmentation and prediction)

[0208] The mode selection unit 260 includes a segmentation unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, which are used to receive or obtain original image data such as an original block 203 (current block 203 of the current image 17) and reconstructed image data from the decoded image buffer 230 or other buffers (e.g., a column buffer, not shown in the figure), such as filtered and / or unfiltered reconstructed pixels or reconstructed blocks of the same (current) image and / or one or more previously decoded images. The reconstructed image data is used as reference image data required for prediction such as inter-frame prediction or intra-frame prediction to obtain a prediction block 265 or a prediction value 265.

[0209] The mode selection unit 260 may be used to determine or select a partition type for a current block prediction mode (including non-partitioning) and a prediction mode (eg, intra-frame or inter-frame prediction mode), and to generate a corresponding prediction block 265 to calculate the residual block 205 and to reconstruct the reconstruction block 215.

[0210] In one embodiment, the mode selection unit 260 may be configured to select a segmentation and prediction mode (e.g., from prediction modes supported or available by the mode selection unit 260) that provides the best match or minimum residual (minimum residual means better compression in transmission or storage), or provides minimum signaling overhead (minimum signaling overhead means better compression in transmission or storage), or considers or balances both of the above. The mode selection unit 260 may be configured to determine the segmentation and prediction mode based on rate distortion optimization (RDO), i.e., select the prediction mode that provides the minimum rate distortion. In this context, terms such as "best", "minimum", "optimal", etc. do not necessarily mean "best", "minimum", "optimal", etc. in general, but may also refer to meeting termination or selection criteria, for example, a value exceeding or falling below a threshold or other constraint, which may result in a "suboptimal selection", but reduces complexity and processing time.

[0211] In other words, the partitioning unit 262 can be used to partition the block 203 into smaller partitions or sub-blocks (forming blocks again), for example, iteratively using quad-tree (QT) partitioning, binary-tree (BT) partitioning or triple-tree (TT) partitioning or any combination thereof, and for example, predict each partition or sub-block, wherein the mode selection includes selecting a tree structure of the partition 203 and applying a prediction mode to each partition or sub-block.

[0212] The segmentation (eg, performed by segmentation unit 260) and prediction processes (eg, performed by inter-prediction unit 244 and intra-prediction unit 254) performed by video encoder 20 will be described in detail below.

[0213] segmentation

[0214] The segmentation unit 262 can segment (or divide) the current block 203 into smaller parts, such as small square or rectangular blocks. These smaller blocks (also referred to as sub-blocks) can be further segmented into smaller parts. This is also called tree segmentation or hierarchical tree segmentation, where a root block at root tree level 0 (hierarchical level 0, depth 0) or the like can be recursively segmented into two or more blocks at the next lower tree level, such as nodes at tree level 1 (hierarchical level 1, depth 1). These blocks can be segmented into two or more blocks at the next lower level, such as tree level 2 (hierarchical level 2, depth 2), etc., until the segmentation ends (because the end criteria are met, such as reaching the maximum tree depth or the minimum block size). Blocks that are not further segmented are also referred to as leaf blocks or leaf nodes of the tree. A tree segmented into two parts is called a binary tree (binary-tree, BT), a tree segmented into three parts is called a ternary tree (ternary-tree, TT), and a tree segmented into four parts is called a quadtree (quad-tree, QT).

[0215] As mentioned above, the term "block" as used herein may be a portion of an image, in particular a square or rectangular portion. For example, in conjunction with HEVC and VVC, a block may be or correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU) and a transform unit (TU), and / or correspond to a corresponding block, for example, a coding tree block (CTB), a coding block (CB), a transform block (TB) or a prediction block (PB).

[0216] For example, a coding tree unit (CTU) may be or include a CTB of a luminance pixel, two corresponding CTBs of a chrominance pixel of an image with an array of three pixels, or a CTB of a pixel of a monochrome image, or a CTB of a pixel of an image encoded using three independent color planes and a syntax structure (for encoding pixels). Accordingly, a coding tree block (CTB) may be an N×N pixel block, where N may be set to a certain value so that the component is divided into CTBs, which is segmentation. A coding unit (CU) may be or include a decoding block of luminance pixels, two corresponding decoding blocks of chrominance pixels of an image with an array of three pixels, or a decoding block of a pixel of a monochrome image, or a decoding block of a pixel of an image encoded using three independent color planes and a syntax structure (for encoding pixels). Accordingly, a decoding block (CB) may be an M×N pixel block, where M and N may be set to a certain value so that the CTB is divided into decoding blocks, which is segmentation.

[0217] In an embodiment, for example according to HEVC, a coding tree unit (CTU) can be divided into multiple CUs by a quadtree structure represented as a coding tree. It is decided at the CU level whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to decode the image area. Each CU can be further divided into one, two or four PUs according to the PU partition type. The same prediction process is used within a PU, and relevant information is sent to the decoder in units of PUs. After the residual block is obtained by applying the prediction process according to the PU partition type, the CU can be divided into transform units (TUs) according to another quadtree structure similar to the coding tree for the CU.

[0218] In an embodiment, for example, according to the latest video decoding standard currently under development called Versatile Video Coding (VVC), quad-tree and binary tree (QTBT) segmentation is used to segment the decoding block. In the QTBT block structure, the CU can be a square or a rectangle. For example, the coding tree unit (CTU) is first segmented by a quadtree structure. The quadtree leaf nodes are further segmented by a binary tree or a ternary tree (or ternary tree) structure. The segmented leaf nodes are called coding units (CUs), and such segmentation is used for prediction and transformation processing without any other segmentation. This means that in the QTBT decoding block structure, the block sizes of CU, PU, ​​and TU are the same. At the same time, it is also proposed to combine multiple segmentations such as ternary tree segmentation with the QTBT block structure.

[0219] In one example, mode select unit 260 of video encoder 20 may be used to perform any combination of the segmentation techniques described herein.

[0220] As described above, the video encoder 20 is used to determine or select the best or optimal prediction mode from a (predetermined) prediction mode set. The prediction mode set may include intra-frame prediction mode and / or inter-frame prediction mode, etc.

[0221] Intra prediction

[0222] The intra-frame prediction mode set may include 35 different intra-frame prediction modes, for example, non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in HEVC, or may include 67 different intra-frame prediction modes, for example, non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in VVC.

[0223] The intra prediction unit 254 is configured to generate an intra prediction block 265 using reconstructed pixels of neighboring blocks of the same current image according to an intra prediction mode in the intra prediction mode set.

[0224] The intra-frame prediction unit 254 (or generally the mode selection unit 260) is also used to output intra-frame prediction parameters (or generally information representing the selected intra-frame prediction mode of the block) to the entropy coding unit 270 in the form of syntax elements 266 to be included in the encoded image data 21, so that the video decoder 30 can perform operations, such as receiving and using the prediction parameters for decoding.

[0225] Inter prediction

[0226] In a possible implementation, the set of inter-frame prediction modes depends on the available reference image (i.e., at least part of the previously decoded image stored in DBP230 as mentioned above) and other inter-frame prediction parameters, for example, on whether to use the entire reference image or only a part of the reference image, such as a search window area near the area of ​​the current block, to search for the best matching reference block, and / or for example on whether to perform pixel interpolation, such as half-pixel and / or quarter-pixel interpolation.

[0227] In addition to the above prediction modes, skip mode and / or direct mode may also be employed.

[0228] The inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (both in Figure 2 ). The motion estimation unit may be configured to receive or acquire an image block 203 (the current image block 203 of the current image 17) and a decoded image 231, or at least one or more previously reconstructed blocks, e.g., reconstructed blocks of one or more other / different previously decoded images 231, for motion estimation. For example, a video sequence may include a current image and a previously decoded image 231, or in other words, the current image and the previously decoded image 231 may be part of or form a sequence of images forming the video sequence.

[0229] For example, the encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same or different images in a plurality of other images, and provide the reference image (or reference image index) and / or the offset (spatial offset) between the position (x coordinate, y coordinate) of the reference block and the position of the current block as an inter-frame prediction parameter to the motion estimation unit. The offset is also referred to as a motion vector (MV).

[0230] The motion compensation unit is used to obtain, for example, receive, inter-frame prediction parameters, and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain an inter-frame prediction block 265. The motion compensation performed by the motion compensation unit may include extracting or generating a prediction block based on a motion / block vector determined by motion estimation, and may also include performing interpolation with sub-pixel accuracy. Interpolation filtering can generate pixel points of other pixels from pixel points of known pixels, thereby potentially increasing the number of candidate prediction blocks that can be used to encode the image block. Once a motion vector corresponding to the PU of the current image block is received, the motion compensation unit can locate the prediction block pointed to by the motion vector in one of the reference image lists.

[0231] The motion compensation unit may also generate syntax elements associated with blocks and video slices for use by the video decoder 30 when decoding image blocks of the video slices. Partitions and partition groups and corresponding syntax may be used in addition to or instead of video slices and corresponding syntax elements.

[0232] Entropy decoding

[0233] For example, the entropy coding unit 270 is used to apply an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) decoding or other entropy coding methods or techniques) or a bypass entropy coding algorithm or scheme (without compression) to the quantization coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters and / or other syntax elements to obtain the encoded image data 21 that can be output in the form of an encoded bitstream 21 via the output terminal 272, so that, for example, the video decoder 30 can receive and use the parameters for decoding. The encoded codestream 21 may be transmitted to the video decoder 30 , or stored in a memory for later transmission or retrieval by the video decoder 30 .

[0234] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, a non-transform based encoder 20 may directly quantize the residual signal of certain blocks or frames without the transform processing unit 206. In another implementation, in the encoder 20, the quantization unit 208 and the inverse quantization unit 210 may be combined into one unit.

[0235] Decoder and decoding method

[0236] Figure 3 An exemplary video decoder 30 for implementing the technology of the present application is shown. The video decoder 30 is used to receive the encoded image data 21 (e.g., the encoded bit stream 21) encoded by the encoder 20, for example, to obtain a decoded image 331. The encoded image data or bit stream includes information for decoding the encoded image data, such as data representing image blocks of the encoded video slice (and / or partition or partition group) and related syntax elements.

[0237] exist Figure 3 In the example of FIG. 3 , the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a loop filter 320, a decoded picture buffer (DPB) 330, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. The inter-frame prediction unit 344 may be or include a motion compensation unit. In some examples, the video decoder 30 may perform substantially the same as the reference Figure 2 The decoding process of the video encoder 100 is the reverse of the encoding process described above.

[0238] As described for the encoder 20, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 344, and the intra-frame prediction unit 354 also constitute the "built-in decoder" of the video encoder 20. Accordingly, the function of the inverse quantization unit 310 may be the same as that of the inverse quantization unit 110; the function of the inverse transform processing unit 312 may be the same as that of the inverse transform processing unit 212; the function of the reconstruction unit 314 may be the same as that of the reconstruction unit 214; the function of the loop filter 320 may be the same as that of the loop filter 220; and the function of the decoded picture buffer 330 may be the same as that of the decoded picture buffer 230. Therefore, the description of the corresponding units and functions of the video encoder 20 is correspondingly applicable to the corresponding units and functions of the video decoder 30.

[0239] Entropy decoding

[0240] The entropy decoding unit 304 is used to parse the code stream 21 (or generally the encoded image data 21) and, for example, perform entropy decoding on the encoded image data 21 to obtain quantization coefficients 309 and / or decoded encoding parameters ( Figure 3 The entropy decoding unit 304 may be used to apply a decoding algorithm or scheme corresponding to the encoding scheme described for the entropy encoding unit 270 of the encoder 20. The entropy decoding unit 304 may also be used to provide the inter-frame prediction parameters, intra-frame prediction parameters and / or other syntax elements to the mode selection unit 360, and to provide other parameters to other units of the decoder 30. The video decoder 30 may receive syntax elements at the video slice level and / or the video block level. Blocks and block groups and corresponding syntax may be used in addition to or instead of video slices and corresponding syntax elements.

[0241] Dequantization

[0242] The inverse quantization unit 310 may be configured to receive a quantization parameter (QP) (or generally information related to inverse quantization) and quantization coefficients from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.), and apply inverse quantization to the decoded quantization coefficients 309 according to the quantization parameter to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311. The inverse quantization process may include using the quantization parameter calculated by the video encoder 20 for each video block in the video slice to determine a degree of quantization, and also determine a degree of inverse quantization that needs to be performed.

[0243] Inverse Transform

[0244] The inverse transform processing unit 312 may be used to receive the dequantized coefficients 311, also referred to as transform coefficients 311, and apply a transform to the dequantized coefficients 311 to obtain a reconstructed residual block 213 in the pixel domain. The reconstructed residual block 213 may also be referred to as a transform block 313. The transform may be an inverse transform, such as an inverse DCT, an inverse DST, an inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 may also be used to receive transform parameters or corresponding information from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304) to determine the transform applied to the dequantized coefficients 311.

[0245] reconstruction

[0246] The reconstruction unit 314 (e.g., adder or summer 314) can be used to add the reconstructed residual block 313 to the prediction block 365 by, for example, adding the pixel point values ​​of the reconstructed residual block 313 and the pixel point values ​​of the prediction block 365 to obtain the reconstructed block 315 in the pixel point domain.

[0247] Filtering

[0248] The loop filter unit 320 (in the decoding loop or after the decoding loop) is used to filter the reconstruction block 315 to obtain a filter block 321 to smooth pixel transitions or otherwise improve video quality. The loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening, a smoothing filter, or a collaborative filter, or any combination thereof. Although the loop filter unit 320 is in Figure 3 3. Although shown as a loop filter in FIG. 3, in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0249] Decoded Image Buffer

[0250] The decoded video blocks 321 in one picture are then stored in a decoded picture buffer 330 which stores the decoded pictures 331 as reference pictures for subsequent motion compensation of other pictures and / or for respective output display.

[0251] The decoder 30 is used to output the decoded image 311 via an output 312 or the like, for presentation to a user or for viewing by the user.

[0252] predict

[0253] The function of the inter-frame prediction unit 344 can be the same as that of the inter-frame prediction unit 244 (especially the motion compensation unit), and the function of the intra-frame prediction unit 354 can be the same as that of the inter-frame prediction unit 254, and decides to divide or partition and perform prediction based on the partition and / or prediction parameters or corresponding information received from the encoded image data 21 (for example, parsed and / or decoded by the entropy decoding unit 304, etc.). The mode selection unit 360 can be used to perform prediction (intra-frame or inter-frame prediction) of each block based on the reconstructed image, block or corresponding pixel point (filtered or unfiltered) to obtain a prediction block 365.

[0254] When the video slice is encoded as an intra-coded (I) slice, the intra-prediction unit 354 of the mode selection unit 360 is used to generate a prediction block 365 of an image block of the current video slice based on the indicated intra-prediction mode and data from a previously decoded block of the current image. When the video image is encoded as an inter-coded (i.e., B or P) slice, the inter-prediction unit 344 (e.g., motion compensation unit) of the mode selection unit 360 is used to generate a prediction block 365 of a video block of the current video slice based on motion vectors and other syntax elements received from the entropy decoding unit 304. For inter-prediction, these prediction blocks may be generated from one of the reference images in one of the reference image lists. The video decoder 30 may construct the reference frame lists: List 0 and List 1 using a default construction technique based on the reference images stored in the DPB 330. The same or similarities may apply to embodiments using partitions (video partitions) or partition groups (video partition groups) as a supplement or alternative to slices (video slices).

[0255] The mode selection unit 360 is used to determine the prediction information of the video block of the current video slice by parsing the motion vector and other syntax elements, and use the prediction information to generate a prediction block for the decoded current video block. For example, the mode selection unit 360 uses some received syntax elements to determine the prediction mode (e.g., intra prediction or inter prediction) for encoding the video block of the video slice, the inter prediction slice type (e.g., B slice, P slice or GPB slice), the construction information of one or more reference picture lists of the slice, the motion vector of each inter-frame coded video block of the slice, the inter-frame prediction state of each inter-frame coded video block of the slice, and other information to decode the video block of the current video slice. The same or similar points can be applied to embodiments using blocks (video blocks) or block groups (video block groups) as a supplement or replacement for slices (video slices).

[0256] Other variations of the video decoder 30 may be used to decode the encoded image data 21. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, a non-transform based decoder 30 may directly inverse quantize the residual signal of certain blocks or frames without the inverse transform processing unit 312. In another implementation, in the video decoder 30, the inverse quantization unit 310 and the inverse transform processing unit 312 may be combined into a single unit.

[0257] It should be understood that the processing result of the current step can be further processed in the encoder 20 and the decoder 30 and then output to the next step. For example, after interpolation filtering, motion vector derivation or loop filtering, the processing result of interpolation filtering, motion vector derivation or loop filtering can be further operated, such as clipping or shifting operation.

[0258] It should be noted that the derived motion vector of the current block (including but not limited to the control point motion vector of the affine mode, the sub-block motion vector of the affine, plane, ATMVP mode, the time domain motion vector, etc.) can be further operated. For example, the value of the motion vector is limited to a predefined range according to the representation bit of the motion vector. If the representation bit of the motion vector is bitDepth, the range is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where "^" represents the power. For example, if bitDepth is set to 16, the range is -32768 to 32767; if bitDepth is set to 18, the range is -131072 to 131071. For example, the value of the derived motion vector (e.g., the MV of 4 4×4 sub-blocks in an 8×8 block) is limited so that the maximum difference between the integer parts of the 4 4×4 sub-block MVs does not exceed N pixels, for example, not more than 1 pixel. Two methods of limiting motion vectors according to bitDepth are provided here.

[0259] Method 1: Remove the most significant bit (MSB) of the overflow by smoothing

[0260] ux=(mvx+2 bitDepth )%2 bitDepth (1)

[0261] mvx=(ux>=2 bitDepth-1 )? (ux-2 bitDepth ):ux(2)

[0262] uy=(mvy+2 bitDepth )%2 bitDepth (3)

[0263] mvy=(uy>=2 bitDepth-1 )? (uy-2 bitDepth ):uy(4)

[0264] Among them, mvx is the horizontal component of the motion vector of an image block or sub-block, mvy is the vertical component of the motion vector of an image block or sub-block, and ux and uy represent intermediate values.

[0265] For example, if the value of mvx is –32769, the value obtained by using formulas (1) and (2) is 32767. In computer systems, decimal numbers are stored in the form of two's complement. The two's complement of -32769 is 1,0111,1111,1111,1111 (17 bits), and if the MSB is discarded, the two's complement obtained is 0111,1111,1111,1111 (32767 in decimal), which is the same as the output obtained by using formulas (1) and (2).

[0266] ux=(mvpx+mvdx+2 bitDepth )%2 bitDepth (5)

[0267] mvx=(ux>=2 bitDepth-1 )? (ux-2 bitDepth ):ux(6)

[0268] uy=(mvpy+mvdy+2 bitDepth )%2 bitDepth (7)

[0269] mvy=(uy>=2 bitDepth-1 )? (uy-2 bitDepth ):uy(8)

[0270] These operations can be performed in the process of summing mvp and mvd, as shown in equations (5) to (8).

[0271] Method 2: Clip the value to remove the overflowed MSB

[0272] vx=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vx)

[0273] vy=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vy)

[0274] Where vx is the horizontal component of the motion vector of an image block or sub-block, vy is the vertical component of the motion vector of an image block or sub-block; x, y and z correspond to the three input values ​​of the MV clipping process respectively, and the function Clip3 is defined as follows:

[0275]

[0276] Figure 4 Schematic diagram of a video decoding device 400 provided in an embodiment of the present invention. The video decoding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video decoding device 400 may be a decoder, such as Figure 1A The video decoder 30 in the embodiment may also be an encoder, for example Figure 1A The video encoder 20 in.

[0277] The video decoding device 400 includes: an input port 410 (or input port 410) and a receiving unit (Rx) 420 for receiving data; a processor, a logic unit or a central processing unit (CPU) 430 for processing data; a transmitting unit (Tx) 440 and an output port 450 (or output port 450) for transmitting data; and a memory 460 for storing data. The video decoding device 400 may also include an optical-to-electrical (OE) component and an electrical-to-optical (EO) component coupled to the input port 410, the receiving unit 420, the transmitting unit 440 and the output port 450, which are used as an outlet or an inlet of an optical signal or an electrical signal.

[0278] The processor 430 is implemented by hardware and software. The processor 430 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the input port 410, the receiving unit 420, the sending unit 440, the output port 450, and the memory 460. The processor 430 includes a decoding module 470. The decoding module 470 implements the embodiments disclosed above. For example, the decoding module 470 performs, processes, prepares, or provides various decoding operations. Therefore, the inclusion of the decoding module 470 significantly improves the function of the video decoding device 400 and realizes the conversion of different states of the video decoding device 400. Alternatively, the decoding module 470 is implemented by instructions stored in the memory 460 and executed by the processor 430.

[0279] The memory 460 includes one or more disks, tape drives, and solid-state hard disks, and can be used as an overflow data storage device for storing such programs when such programs are selected for execution, and for storing instructions and data read during program execution. For example, the memory 460 can be volatile and / or non-volatile, and can be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random-access memory (SRAM).

[0280] Figure 5A simplified block diagram of an apparatus 500 is provided for an exemplary embodiment, wherein the apparatus 500 may be used as either or both of the source device 12 and the destination device 14 in FIG. 1 .

[0281] The processor 502 in the apparatus 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices that are currently available or will be developed in the future and are capable of manipulating or processing information. Although a single processor such as the processor 502 shown in the figure may be used to implement the disclosed implementation, using more than one processor may improve speed and efficiency.

[0282] In one implementation, the memory 504 in the apparatus 500 may be a read only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 accessed by the processor 502 via a bus 512. The memory 504 may also include an operating system 508 and an application 510, wherein the application 510 includes at least one program that allows the processor 502 to perform the methods described herein. For example, the application 510 may include applications 1 to N, and also include a video decoding application that performs the methods described herein.

[0283] The apparatus 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element that may be used to sense touch input. The display 518 may be coupled to the processor 502 via the bus 512.

[0284] Although the bus 512 of the device 500 is described herein as a single bus, the bus 512 may include multiple buses. In addition, the auxiliary storage 514 may be directly coupled to other components of the device 500 or accessed through a network, and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Therefore, the device 500 may have a variety of configurations.

[0285] Mathematical operators

[0286] The mathematical operators used in this application are similar to those used in the C programming language. However, the results of integer division and arithmetic shift operations are more precisely defined, and other operations such as exponentiation and real-valued division are defined. Numbering and counting specifications generally start at zero, for example, "first" is equivalent to 0, "second" is equivalent to 1, and so on.

[0287] Arithmetic operators

[0288] Arithmetic operators are defined as follows:

[0289] +Addition

[0290] - Subtraction (as a two-argument operator) or negation (as a unary prefix operator)

[0291] * Multiplication, including matrix multiplication

[0292]

[0293]

[0294] ÷ is used to represent division in arithmetic equations, but no truncation or rounding is performed.

[0295] Used to represent division in arithmetic equations, but without truncation or rounding.

[0296] The sum of f(i), where i takes all integer values ​​from x to y (inclusive).

[0297] x%y modulus operation, the remainder when x is divided by y, where both x and y must be integers, and x≥0 and y>0.

[0298] Logical operators

[0299] Logical operators are defined as follows:

[0300] x&&y Boolean logic "AND" operation of x and y

[0301] x||y Boolean logical "OR" operation of x and y

[0302] ! Boolean logic "NOT" operation

[0303] x? y:z If x is true (TRUE) or not equal to 0, then return the value of y, otherwise, return the value of z.

[0304] Relational operators

[0305] The relational operators are defined as follows:

[0306] > greater than

[0307] > = greater than or equal to

[0308] <

[0309] <= less than or equal to

[0310] == is equal to

[0311] ! = not equal to

[0312] When a relational operator is applied to a syntactic element or variable that has been assigned the value na (not applicable), the na value is treated as a distinct value for that syntactic element or variable. The na value is considered not equal to any other value.

[0313] Bitwise operators

[0314] The bitwise operators are defined as follows:

[0315] & Bitwise AND. When operating on an integer variable, the two's complement representation of the integer value is operated on. When operating on binary arguments, if it contains fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.

[0316] | Bitwise OR. When operating on an integer variable, the two's complement representation of the integer value is operated on. When operating on binary arguments, if it contains fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.

[0317] ^ Bitwise XOR. When operating on an integer variable, the two's complement representation of the integer value is operated on. When operating on binary arguments, if it contains fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.

[0318] x >> y The two's complement integer representation of x is arithmetically shifted right by y binary bits. This function is only defined when y is positive. The result of the right shift is that the bit shifted into the most significant bit (MSB) should be equal to the MSB of x before the shift operation.

[0319] x << y The two's complement integer representation of x is arithmetically shifted left by y binary bits. This function is only defined when y is positive. The result of the left shift is that the bit shifted into the least significant bit (LSB) should be equal to 0.

[0320] Assignment operators

[0321] The arithmetic operators are defined as follows:

[0322] = Assignment operator

[0323] ++ Increment. For example, x++ is equal to x = x + 1; when used in an array index, it should be equal to the value of the variable before the increment operation.

[0324] -- Decrement. For example, x-- is equal to x = x - 1; when used in an array index, it should be equal to the value of the variable before the decrement operation.

[0325] + = increase the specified amount, for example: x+ = 3 is equal to x = x + 3, x+ = (-3) is equal to x = x + (-3).

[0326] - = Reduces the specified amount, for example: x– = 3 is equal to x = x – 3, x– = (-3) is equal to x = x – (-3).

[0327] Range Notation

[0328] The following symbols are used to indicate a range of values:

[0329] x=y..zx takes integer values ​​in the range of y to z, including y and z, where x, y and z are all integers and z is greater than y.

[0330] Operation priority order

[0331] When parentheses are not used to explicitly indicate precedence, the following rules apply:

[0332] –Higher priority operations are evaluated before lower priority operations.

[0333] – Operations of the same priority are evaluated from left to right.

[0334] The following table describes the priority of operations from highest to lowest, with the higher the position in the table, the higher the priority.

[0335] For operators that are also used in the C programming language, the operator precedence order in this specification is the same as the precedence order in the C programming language.

[0336] Table: Operation priority is sorted from highest (top of table) to lowest (bottom of table)

[0337]

[0338] Text description of logical operation

[0339] In this paper, the logical operation statements are described in mathematical form as follows:

[0340]

[0341] It can be described in the following ways:

[0342] ...as follows / ...the following applies:

[0343] –If condition 0,statement 0

[0344] –Otherwise,if condition 1,statement 1

[0345] –...

[0346] –Otherwise(informative remark on remaining condition),statement n

[0347] "If..." followed by "...as follows" or "...the following applies" introduces each "If...Otherwise,if...Otherwise,..." statement in the text. The last condition of an "If...Otherwise,if...Otherwise,..." is always "Otherwise,...". Intervening "If...Otherwise,if...Otherwise,..." statements can be identified by matching "...as follows" or "...the following applies" with the ending "Otherwise,...".

[0348] In this paper, the logical operation statements are described in mathematical form as follows:

[0349]

[0350] It can be described in the following ways:

[0351]

[0352] In this paper, the logical operation statements are described in mathematical form as follows:

[0353] if(condition 0)

[0354] statement 0

[0355] if(condition 1)

[0356] statement 1

[0357] It can be described in the following ways:

[0358] When condition 0, statement 0

[0359] When condition 1, statement 1

[0360] Chunk groups and chunks

[0361] The embodiments of the present application can be used to implement the concept of blocks and block groups, in other words, to process or encode images using blocks and block groups. A more detailed description is provided below.

[0362] Block: The rectangular area in the image where the CTU is located within a specific block column and a specific block row. The size of the block may be different in different images and in the same image.

[0363] Block column: A rectangular area where a CTU is located, with a height equal to the picture height and a specified width. For example, the width is specified by a syntax element in a picture parameter set.

[0364] Tiled group: An integer number of tiles of a picture in the tiled raster scan of the picture. Tiles in a tile group can only be contained in a single NAL unit.

[0365] Block group header: The part of a decoded block group containing data elements related to the first or all blocks represented in the block group.

[0366] Block group data: contains the actual decoded data for all blocks belonging to a given block group.

[0367] Tile line: A rectangular area where a CTU is located, with a specified height and a width equal to the picture width. For example, the height is specified by a syntax element in a picture parameter set.

[0368] Block scan: A specific sequential arrangement of CTUs that partition an image, where CTUs are arranged consecutively in a block-based CTU raster scan and blocks in an image are arranged consecutively in a block-based raster scan of the image.

[0369] Raster Scan: The mapping of a rectangular two-dimensional pattern to a one-dimensional pattern so that the first entry in the one-dimensional pattern is from the top row of the two-dimensional pattern scanned from left to right, followed by the second row, third row, etc., of the pattern (downwards) scanned from left to right.

[0370] For other terms and definitions, if not detailed herein, reference is made to JVET-L1001-v10 dated December 14, 2018 and JVET-L0686-V2 dated October 12, 2018, the entire contents of which are incorporated herein by reference, especially the portions of JVET-L1001-v10 (also referred to as clauses) that are specifically referenced herein.

[0371] Segmentation of images into tile groups and tiles:

[0372] For example, Figure 6As shown, an image can be divided into tile groups and tiles. For example, in a tiled raster scan of an image, a tile group is a column of tiles. A tile is a column of CTUs that cover a rectangular area in the image. Figure 6 In , the image is divided into 12 blocks (blocks 1 to 12) and 3 block groups (block groups 1 to 3). Figure 6 In FIG. , the image includes 3 block columns and 4 block rows, blocks 1 and 2 belong to block group 1, blocks 3 to 7 belong to block group 2, and blocks 8 to 12 belong to block group 3. Figure 6 Each block depicted in comprises 18 CTUs, distributed in 6 CTU columns and 3 CTU rows. The cross-block (block scanning order within the image) raster scan may be a raster scan, for example, scanning the image horizontally block by block row starting from block 1 located in the upper left corner, and ending the scan at block 12 located in the lower right corner. This is represented by the dotted arrows representing the block group scanning order (i.e., cross-block raster scan). Within the block, another raster scan (intra-block raster scan) (intra-block CTU scanning order) may be performed, for example, scanning the block horizontally CTU row by CTU row starting from the CTU located in the upper left corner, and ending the scan at the CTU located in the lower right corner. The intra-block raster scan is indicated by a solid arrow, which indicates the CTU scanning order within the block.

[0373] Therefore, if Figure 6 As shown, the block group shown is composed of blocks, and the blocks include an integer number of CTUs. Figure 6 It should also be noted that the Versatile Test Model (VTM) 3.1 version also uses block groups.

[0374] Figure 6 and Figure 7 The tile groups (TG) involved in the raster scan order are shown. This can be compared with Fig. 9 In comparison, Fig. 9 A rectangle TG is shown. A tile group can also be considered as a strip comprising tiles.

[0375] CTB raster and block scanning process

[0376] The list ColWidth[i] represents the width of the i-th tile column in CTB units, where i ranges from 0 to num_tile_columns_minus1 (including the end value). The derivation process of the list ColWidth[i] is as follows:

[0377]

[0378]

[0379] The list RowHeight[j] represents the height of the jth tile row in CTB units, where j ranges from 0 to num_tile_rows_minus1 (including the end value). The derivation process of the list RowHeight[j] is as follows:

[0380]

[0381] List ColBd[i] represents the position of the i-th tile column boundary in CTB units, where i ranges from 0 to num_tile_columns_minus1+1 (including the end value). The derivation process of the list ColBd[i] is as follows:

[0382] for(ColBd[0]=0,i=0;i<=num_tile_columns_minus1;i++)

[0383] ColBd[i+1]=ColBd[i]+ColWidth[i](3)

[0384] The list RowBd[j] represents the position of the jth tile row boundary in CTB units, where j ranges from 0 to num_tile_rows_minus1+1 (including the end value). The derivation process of the list RowBd[j] is as follows:

[0385] for(RowBd[0]=0,j=0;j<=num_tile_rows_minus1;j++)

[0386] RowBd[j+1]=RowBd[j]+RowHeight[j](4)

[0387] The list CtbAddrRsToTs[ctbAddrRs] represents the conversion from the CTB address in the CTB raster scan of the image to the CTB address in the block scan, where ctbAddrRs ranges from 0 to PicSizeInCtbsY-1 (including the end value). The derivation process of the list CtbAddrRsToTs[ctbAddrRs] is as follows:

[0388]

[0389]

[0390] The list CtbAddrTsToRs[ctbAddrTs] represents the conversion from the CTB address in the block scan to the CTB address in the CTB raster scan of the image, where ctbAddrTs ranges from 0 to PicSizeInCtbsY-1 (including the end value). The derivation process of the list CtbAddrTsToRs[ctbAddrTs] is as follows:

[0391] for(ctbAddrRs=0;ctbAddrRs <PicSizeInCtbsY;ctbAddrRs++)(6)

[0392] CtbAddrTsToRs[CtbAddrRsToTs[ctbAddrRs]]=ctbAddrRs

[0393] The list TileId[ctbAddrTs] represents the conversion from the CTB address in the block scan to the block ID, where the range of ctbAddrTs is 0 to PicSizeInCtbsY-1 (including the end value). The derivation process of the list TileId[ctbAddrTs] is as follows:

[0394] for(j=0,tileIdx=0;j<=num_tile_rows_minus1;j++)

[0395] for(i=0;i<=num_tile_columns_minus1;i++,tileIdx++)

[0396] for(y=RowBd[j];y <RowBd[j+1];y++)(7)

[0397] for(x=ColBd[i];x <ColBd[i+1];x++)

[0398] TileId[CtbAddrRsToTs[y*PicWidthInCtbsY+x]]=ileIdx

[0399] The list NumCtusInTile[tileIdx] represents the conversion from tile index to the number of CTUs in the tile, where tileIdx ranges from 0 to PicSizeInCtbsY-1 (including the end value). The derivation process of the list NumCtusInTile[tileIdx] is as follows:

[0400] for(j=0,tileIdx=0;j<=num_tile_rows_minus1;j++)

[0401] for (i = 0; i <= num_tile_columns_minus1; i++, tileIdx++) (8)

[0402] NumCtusInTile[tileIdx] = ColWidth[i] * RowHeight[j]

[0403] The list FirstCtbAddrTs[tileIdx] represents the conversion from the tile ID to the CTB address in the tile scan of the first CTB in the tile, where the range of tileIdx is from 0 to NumTilesInPic - 1 (including the end values). The derivation process of this list FirstCtbAddrTs[tileIdx] is as follows:

[0404]

[0405] The value of ColumnWidthInLumaSamples[i] represents the width of the i-th tile column in terms of luma pixels and is set to be equal to ColWidth[i] << CtbLog2SizeY, where the range of i is from 0 to num_tile_columns_minus1 (including the end values).

[0406] The value of RowHeightInLumaSamples[j] represents the height of the j-th tile row in terms of luma pixels and is set to be equal to RowHeight[j] << CtbLog2SizeY, where the range of j is from 0 to num_tile_rows_minus1 (including the end values).

[0407] Image Parameter Set Raw Byte Sequence Payload (RBSP) Syntax

[0408] The picture parameter set syntax for tiles and tile groups is as follows.

[0409]

[0410] Chunk Group Header Syntax

[0411]

[0412]

[0413] Chunk Group Data Syntax

[0414]

[0415]

[0416] Image Parameter Set Semantics

[0417] single_tile_in_pic_flag equal to 1 indicates that there is only one tile in each picture referring to the PPS. single_tile_in_pic_flag equal to 0 indicates that there is more than one tile in each picture referring to the PPS.

[0418] Codestream conformance requires that the value of single_tile_in_pic_flag shall be the same for all PPSs activated within a CVS.

[0419] num_tile_columns_minus1 plus 1 indicates the number of tile columns in the segmented image. num_tile_columns_minus1 should be in the range 0 to PicWidthInCtbsY-1 (inclusive). If not present, the value of num_tile_columns_minus1 is inferred to be equal to 0.

[0420] num_tile_rows_minus1 plus 1 indicates the number of tile rows in the segmented image. num_tile_rows_minus1 should be in the range 0 to PicHeightInCtbsY-1 (inclusive). If not present, the value of num_tile_rows_minus1 is inferred to be equal to 0.

[0421] Set the variable NumTilesInPic equal to (num_tile_columns_minus1+1)*(num_tile_rows_minus1+1).

[0422] When single_tile_in_pic_flag is equal to 0, NumTilesInPic should be greater than 1.

[0423] uniform_tile_spacing_flag equal to 1 indicates that tile column boundaries and tile row boundaries are uniformly distributed throughout the image. uniform_tile_spacing_flag equal to 0 indicates that tile column boundaries and tile row boundaries are not uniformly distributed throughout the image, but are explicitly indicated using the syntax elements tile_column_width_minus1[i] and tile_row_height_minus1[i]. If not present, the value of uniform_tile_spacing_flag is inferred to be equal to 1.

[0424] tile_column_width_minus1[i] plus 1 represents the width of the i-th tile column in CTB units.

[0425] tile_row_height_minus1[i] plus 1 represents the height of the i-th tile row in CTB units.

[0426] As described in the "CTB Raster and Block Scan Process" section, the following variables are derived by calling the CTB raster and block scan conversion process:

[0427] – A list ColWidth[i] representing the width of the i-th tile column in CTB units, where i ranges from 0 to num_tile_columns_minus1 (including the end value);

[0428] – A list RowHeight[j] representing the height of the j-th tile row in CTB units, where j ranges from 0 to num_tile_rows_minus1 (including the end value);

[0429] – A list ColBd[i] indicating the position of the i-th tile column boundary in CTB units, where i ranges from 0 to num_tile_columns_minus1+1 (including the end value);

[0430] – A list RowBd[j] indicating the position of the j-th tile row boundary in CTB units, where j ranges from 0 to num_tile_rows_minus1+1 (including the end value);

[0431] – a list CtbAddrRsToTs[ctbAddrRs] representing the conversion from CTB addresses in the CTB raster scan of the image to CTB addresses in the block scan, where ctbAddrRs ranges from 0 to PicSizeInCtbsY-1 (including the end value);

[0432] – a list CtbAddrTsToRs[ctbAddrTs] representing the conversion from CTB addresses in the block scan to CTB addresses in the CTB raster scan of the picture, where ctbAddrTs ranges from 0 to PicSizeInCtbsY-1 (including the end value);

[0433] – A list TileId[ctbAddrTs] representing the conversion from CTB addresses in tile scan to tile IDs, where ctbAddrTs ranges from 0 to PicSizeInCtbsY-1 (including the end value);

[0434] – a list NumCtusInTile[tileIdx] representing the conversion from tile index to the number of CTUs in the tile, where tileIdx ranges from 0 to PicSizeInCtbsY-1 (inclusive);

[0435] – a list FirstCtbAddrTs[tileIdx] representing the conversion from tile ID to the CTB address in the tile scan of the first CTB in the tile, where tileIdx ranges from 0 to NumTilesInPic-1 (inclusive);

[0436] – A list ColumnWidthInLumaSamples[i] representing the width of the i-th tile column in units of luminance pixels, where i ranges from 0 to num_tile_columns_minus1 (including the end value);

[0437] – A list of RowHeightInLumaSamples[j] representing the height of the j-th tile row in units of luminance pixels, where j ranges from 0 to num_tile_rows_minus1 (including the end value).

[0438] The value of ColumnWidthInLumaSamples[i] and the value of RowHeightInLumaSamples[j] should both be greater than 0, where i ranges from 0 to num_tile_columns_minus1 (inclusive) and j ranges from 0 to num_tile_rows_minus1 (inclusive).

[0439] loop_filter_across_tiles_enabled_flag equal to 1 indicates that in-loop filtering operations can be performed across tile boundaries in images referenced to the PPS. loop_filter_across_tiles_enabled_flag equal to 0 indicates that in-loop filtering operations are not performed across tile boundaries in images referenced to the PPS. In-loop filtering operations include deblocking filtering operations, pixel adaptive offset filtering operations, and adaptive loop filtering operations. If not present, the value of loop_filter_across_tiles_enabled_flag is inferred to be 1.

[0440] Chunked group header semantics

[0441] tile_group_pic_parameter_set_id indicates the value of pps_pic_parameter_set_id of the PPS currently used. The value of tile_group_pic_parameter_set_id should be in the range of 0 to 63 (inclusive).

[0442] tile_group_address represents the tile address of the first tile in the tile group. The length of tile_group_address is Ceil(Log2(NumTilesInPic)) bits. The value of tile_group_address shall be in the range 0 to NumTilesInPic-1 (inclusive), and the value of tile_group_address shall not be equal to the value of tile_group_address of any other coded tile group NAL unit in the same coded picture. When tile_group_address does not exist, its value is inferred to be equal to 0.

[0443] num_tiles_in_tile_group_minus1 plus 1 indicates the number of tiles in the tile group. The value of num_tiles_in_tile_group_minus1 should be in the range 0 to NumTilesInPic-1, inclusive. If not present, the value of num_tiles_in_tile_group_minus1 is inferred to be equal to 0.

[0444] According to Table 1, tile_group_type indicates the encoding type of the tile group.

[0445] Table 1: Name association of tile_group_type

[0446] tile_group_type The name of the tile_group_type 0 B (B block group) 1 P(P block group) 2 I (I block group)

[0447] offset_len_minus1 plus 1 indicates the length of the entry_point_offset_minus1[i] syntax element in bits. The value of offset_len_minus1 shall be in the range of 0 to 31, inclusive.

[0448] entry_point_offset_minus1[i] plus 1 represents the i-th entry point offset in bytes, which is represented by offset_len_minus1 plus 1 bit. The tile group data following the tile group header consists of num_tiles_in_tile_group_minus1+1 subsets, with subset index values ​​ranging from 0 to num_tiles_in_tile_group_minus1 (including 0 and num_tiles_in_tile_group_minus1). The first byte of the tile group data is considered to be byte 0. If present, the anti-aliasing byte appearing in the tile group data portion of the coded tile group NAL unit is counted as part of the tile group data for the purpose of subset identification. Subset 0 consists of byte 0 to byte entry_point_offset_minus1[0] of the coded tile group data (including the end value), and subset k consists of byte firstByte[k] to byte lastByte[k] of the coded tile group data (including the end value), where k ranges from 1 to num_tiles_in_tile_group_minus1-1 (including the end value). Bytes firstByte[k] and lastByte[k] are defined as follows:

[0449]

[0450] lastByte[k]=firstByte[k]+entry_point_offset_minus1[k](11)

[0451] The last subset (whose subset index is equal to num_tiles_in_tile_group_minus1) consists of the remaining bytes of the coded tile group data.

[0452] Each subset shall consist of all coded bits of all CTUs in the same partition of the partition group.

[0453] History-based motion vector prediction (HMVP)

[0454] The embodiments of the present application may be used to implement history-based motion vector prediction, or in other words, to process or decode an image using history-based motion vector prediction. A more detailed description is provided below.

[0455] For example, in Versatile Video Coding (VVC), the motion vector of an inter-coded block can be indicated in two ways: Advanced motion vector prediction (AMVP) mode or fusion mode. For AMVP mode, the difference between the actual motion vector and the motion vector prediction (MVP), the reference index, and the MVP index of the reference AMVP candidate list are indicated. For fusion mode, the fusion index of the reference fusion candidate list is indicated, and all motion information associated with the fusion candidate is inherited.

[0456] For both the AMVP candidate list and the fused candidate list, motion candidates are derived from temporal neighboring decoding blocks or spatial neighboring decoding blocks. More specifically, the fused candidate list can be constructed by checking the following four types of fused MVP candidates in the following order:

[0457] 1. Spatial fusion candidates (from five spatially adjacent blocks, such as Fig.10 shown).

[0458] 2. Temporal MVP (TMVP) fusion candidate.

[0459] 3. Merged bidirectional prediction fusion candidates.

[0460] 4. Zero motion vector fusion candidate.

[0461] Once the number of available fusion candidates reaches the indicated maximum number of allowed fusion candidates (eg, 5 under normal testing conditions), the fusion candidate list construction process is terminated.

[0462] Similarly, for the AMVP candidate list, three types of MVP candidates are checked in the following order:

[0463] 1. Up to two spatial MVP candidates (one from B0, B1 and B2, the other from A0 and A1, such as Fig.10 shown).

[0464] 2. Temporal MVP (TMVP).

[0465] 3. Zero MVP candidates.

[0466] therefore, Fig.10 The locations of spatially neighboring blocks used for fused candidate list construction and AMVP candidate list construction are shown.

[0467] In the history-based MVP (HMVP) method, the HMVP candidate is defined as the motion information of the previously decoded block. A table with multiple HMVP candidates is maintained during the decoding (i.e., encoding and / or decoding) process. Whenever there is an inter-frame decoding block, the relevant motion information is added to the last entry of the table as a new HMVP candidate. Fig.11 The entire decoding process is shown.

[0468] The size of the HMVP table (also referred to as the length L) is set to a given value HMVPCand (HMVPCand may be a positive integer, such as 16), which indicates that up to 16 HMVP candidates may be added to the table. If there are more than 16 HMVP candidates from a previously decoded block, for example, a first in first out (FIFO) rule is used so that the table always contains the most recent 16 previously encoded motion vector candidates. Fig.12 An example of updating the HMVP table in the HMVP method is shown. Fig.12 (a) shows an example where the FIFO rule is used to remove HMVP candidates and add new candidates to the table used in the proposed method.

[0469] In order to further improve the decoding efficiency, the following Fig.12 (b) shows the constrained FIFO rule, where, when inserting an HMVP candidate into the table, a redundancy check is first performed to search whether there is an identical HMVP candidate in the table. If so, the identical HMVP candidate is deleted from the table, and all subsequent HMVP candidates (in FIFO order) are moved, i.e., their HMVP candidate index is reduced by 1.

[0470] HMVP candidates can be used in the construction process of the fusion candidate list. For example, after the TMVP candidate, all HMVP candidates from the last entry to the first entry in the insert table are pruned. Once the total number of available fusion candidates reaches the indicated maximum number of allowed fusion candidates, the fusion candidate list construction process is terminated.

[0471] Similarly, HMVP candidates can also be used in the construction of the AMVP candidate list. For example, the motion vectors of the last K HMVP candidates in the table are inserted after the TMVP candidates. Only HMVP candidates whose reference image is the same as the AMVP target reference image are used to construct the AMVP candidate list. HMVP candidates are pruned. K can be set to an integer value, for example, K can be 4.

[0472] In addition, when the total number of fusion candidates is greater than or equal to 15, the fusion index can be encoded by using a truncated unary plus fixed length (3 bits) binarization method. The total number of fusion candidates is represented by N mrg , the binarization method is shown in the table below.

[0473] Table: Binary string of fused index (assuming N mrg for 15)

[0474]

[0475]

[0476] The fusion candidate list derived by the "HMVP tool" is generated through the following process:

[0477] This process is called only when merge_flag[xCb][yPb] is equal to 1, where (xCb, yCb) represents the upper left pixel of the current luma decoding block relative to the upper left luma pixel of the current image.

[0478] The inputs to this process are:

[0479] – The luminance position (xCb, yCb) of the upper left pixel of the current luminance decoding block relative to the upper left luminance pixel of the current image;

[0480] – The variable cbWidth indicates the width of the current decoding block calculated in luminance pixels;

[0481] – The variable cbHeight indicates the height of the current decoding block calculated in luminance pixels;

[0482] – Bidirectional prediction weight index gbiIdx.

[0483] The output of this process is:

[0484] – Luma motion vectors mvL0[0][0] and mvL1[0][0] with 1 / 16 fractional pixel accuracy;

[0485] – reference indices refIdxL0 and refIdxL1;

[0486] – Prediction table utilization flags predFlagL0[0][0] and predFlagL1[0][0].

[0487] The bidirectional prediction weight index gbiIdx is set to 0.

[0488] The motion vectors mvL0[0][0] and mvL1[0][0], the reference indexes refIdxL0 and refIdxL1, and the prediction table utilization flags predFlagL0[0][0] and predFlagL1[0][0] are obtained by the following sequential steps:

[0489] 1. Call the process of obtaining merging candidates from adjacent coding units described in clause 8.3.2.3 with the luma coding block position (xCb, yCb), luma coding block width cbWidth and luma coding block height cbHeight as input. The outputs of this process are availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1 and availableFlagB2, reference indices refIdxLXA0, refIdxLXA1, refIdxLXB0, refIdxLXB1 and refIdxLXB2, prediction table utilization flags predFlagLXA0, predFlagLXA1, predFlagLXB0, predFlagLXB1 and predFlagLXB2, motion vectors mvLXA0, mvLXA1, mvLXB0, mvLXB1 and mvLXB2, and bidirectional prediction weight indices gbiIdxA0, gbiIdxA1, gbiIdxB0, gbiIdxB1 and gbiIdxB2, where X is 0 or 1.

[0490] 2. The reference index refIdxLXCol and the bidirectional prediction weight index gbiIdxCol of the time domain fusion candidate Col are set to 0, where X is 0 or 1.

[0491] 3. Call the temporal luma motion vector prediction derivation process described in clause 8.3.2.11 with the luma position (xCb, yCb), luma coding block width cbWidth, luma coding block height cbHeight and variable refIdxL0Col as input. The output of this process is the availability flag availableFlagL0Col and the temporal motion vector mvL0Col. The derivation process of the variables availableFlagCol, predFlagL0Col and predFlagL1Col is as follows:

[0492] availableFlagCol=availableFlagL0Col(12)

[0493] predFlagL0Col=availableFlagL0Col(13)

[0494] predFlagL1Col=0(14)

[0495] 4. When tilegroup_type is B, the temporal luma motion vector prediction derivation process described in clause 8.3.2.11 is called with the luma position (xCb, yCb), luma coding block width cbWidth, luma coding block height cbHeight and variable refIdxL1Col as input. The output of this process is the availability flag availableFlagL1Col and the temporal motion vector mvL1Col. The derivation process of the variables availableFlagCol and predFlagL1Col is as follows:

[0496] availableFlagCol=availableFlagL0Col||availableFlagL1Col(15)

[0497] predFlagL1Col=availableFlagL1Col(16)

[0498] 5. The construction process of the fusion candidate list mergeCandList is as follows:

[0499] i=0

[0500] if(availableFlagA1)

[0501] mergeCandList[i++]=A1

[0502] if(availableFlagB1)

[0503] mergeCandList[i++]=B1

[0504] if(availableFlagB0)

[0505] mergeCandList[i++]=B0(17)

[0506] if(availableFlagA0)

[0507] mergeCandList[i++]=A0

[0508] if(availableFlagB2)

[0509] mergeCandList[i++]=B2

[0510] if(availableFlagCol)

[0511] mergeCandList[i++] = Col

[0512] 6. Set the variables numCurrMergeCand and numOrigMergeCand to the number of merge candidates in mergeCandList.

[0513] 7. When numCurrMergeCand is less than (MaxNumMergeCand-1) and NumHmvpCand is greater than 0, perform the following operations:

[0514] – Call the process of deriving history-based merge candidates described in the next section entitled (“Derivation Process of History-Based Merge Candidates”) with mergeCandList and numCurrMergeCand as input. The output of this process is the modified mergeCandList and numCurrMergeCand.

[0515] – Set numOrigMergeCand equal to numCurrMergeCand.

[0516] The derivation process of history-based fusion candidates

[0517] The inputs to this process are:

[0518] –Merge candidate list mergeCandList;

[0519] – Number of available fusion candidates in list numCurrMergeCand.

[0520] The output of this process is:

[0521] – The modified fusion candidate list mergeCandList;

[0522] – The number of fusion candidates in the modified list numCurrMergeCand.

[0523] Set each element in the array isPruned[i] to FALSE, where i=0...numCurrMergeCand-1. Set the variable numOrigMergeCand equal to numCurrMergeCand.

[0524] For each candidate in HmvpCandList[hMvpIdx], the following ordered steps are repeated until numCurrMergeCand is equal to (MaxNumMergeCand-1), where index hMvpIdx=1..NumHmvpCand.

[0525] 1. The derivation process of the variable sameMotion is as follows:

[0526] – For any value of i = 0 ... numOrigMergeCand-1, if the following conditions are true, set sameMotion and isPruned[i] to true:

[0527] – Candidate HmvpCandList[NumHmvpCand-hMvpIdx] is equal to merge candidate mergeCandList[i];

[0528] –isPruned[i] is equal to false (FALSE).

[0529] – Otherwise, set sameMotion to FALSE.

[0530] 2. When sameMotion is equal to false (FALSE), add the candidate HmvpCandList[NumHmvpCand-hMvpIdx] to the fusion candidate list, as follows:

[0531] mergeCandList[numCurrMergeCand++]=

[0532] HmvpCandList[NumHmvpCand-hMvpIdx](18)

[0533] The motion vector prediction candidate list derivation comprising the "HMVP tool" is generated by the following process:

[0534] The inputs to this process are:

[0535] – The luminance position (xCb, yCb) of the upper left pixel of the current luminance decoding block relative to the upper left luminance pixel of the current image;

[0536] – The variable cbWidth indicates the width of the current decoding block calculated in luminance pixels;

[0537] – The variable cbHeight indicates the height of the current decoding block calculated in luminance pixels;

[0538] – The reference index refIdxLX of the current coding unit partition, where X is 0 or 1.

[0539] The output of this process is a motion vector prediction candidate list mvpListLX with 1 / 16 fractional pixel accuracy, where X is 0 or 1.

[0540] The motion vector prediction candidate list mvpListLX is derived by the following ordered steps, where X is 0 or 1:

[0541] 1. Call the process of obtaining spatial motion vector prediction value candidates from adjacent coding unit partitions described in clause 8.3.2.10 with the luma coding block position (xCb, yCb), luma coding block width cbWidth, luma coding block height cbHeight and refIdxLX as input, where X is 0 or 1. The output of this process is the availability flag availableFlagLXN and the motion vector mvLXN, where N can be replaced by A or B.

[0542] 2. Invoke the motion vector rounding process described in clause 8.3.2.14 with mvX set to mvLXN, rightShift set to MvShift+2, and leftShift set to MvShift+2 as input, where N can be replaced by A or B. The output of this process is the rounded mvLXN, where N can be replaced by A or B.

[0543] 3. If both availableFlagLXA and availableFlagLXB are equal to 1, and mvLXA is not equal to mvLXB, set availableFlagLXCol to 0.

[0544] 4. Otherwise, take the following steps:

[0545] – Call the temporal luma motion vector prediction value derivation process described in clause 8.3.2.11 with the luma coding block position (xCb, yCb), luma coding block width cbWidth, luma coding block height cbHeight and refIdxLX as input, where X is 0 or 1. The output of this process is the availability flag availableFlagLXCol and the temporal motion vector prediction value mvLXCol.

[0546] – Invoke the motion vector rounding process described in clause 8.3.2.14 with mvX set to mvLXCol, rightShift set to MvShift + 2, and leftShift set to MvShift + 2. The output of this process is the rounded mvLXCol.

[0547] 5. The construction process of the motion vector prediction value candidate list mvpListLX is as follows:

[0548]

[0549] 6. When numCurrMvpCand is less than 2 and NumHmvpCand is greater than 0, perform the following operations for i=1..Min(4,NumHmvpCand) until numCurrMvpCand is equal to 2:

[0550] – For each reference picture list LY (where Y=0...1) perform the following until numCurrMvpCand equals 2:

[0551] – When all of the following conditions are true, set mvpListLX[numCurrMvpCand++] equal to the LY motion vector of candidate HmvpCandList[NumHmvpCand-i].

[0552] –The LY reference index of the history-based motion vector prediction value candidate HmvpCandList[i] is equal to refIdxLX.

[0553] –The motion vector prediction candidate list is empty (numCurrMvpCand is equal to 0), or the LY motion vector of candidate HmvpCandList[NumHmvpCand-i] is not equal to mvpListLX[j], where j=0…numCurrMvpCand-1.

[0554] The updating of the history-based motion vector predictor list is described in the section entitled "Updating of History-Based Motion Vector Predictor Candidate List", and is performed after the motion information for a given coding unit is obtained.

[0555] Update of the motion vector prediction candidate list based on history:

[0556] The inputs to this process are:

[0557] – Luma motion vectors mvL0 and mvL1 with 1 / 16 fractional pixel accuracy;

[0558] – reference indices refIdxL0 and refIdxL1;

[0559] – Prediction table utilization flags predFlagL0 and predFlagL1.

[0560] The MVP candidate hMvpCand includes luma motion vectors mvL0 and mvL1, reference indexes refIdxL0 and refIdxL1, prediction table utilization flags predFlagL0 and predFlagL1, and a bidirectional prediction weight index gbiIdx set to 0.

[0561] Use the candidate hMvpCand to modify the candidate list HmvpCandList in the following sequential steps:

[0562] 1. Set the variable identicalCandExist to FALSE and the variable removeIdx to 0.

[0563] 2. When NumHmvpCand is greater than 0, for each index hMvpIdx of hMvpIdx=0...NumHmvpCand-1, perform the following steps until identicalCandExist is equal to TRUE:

[0564] – When hMvpCand is equal to HmvpCandList[hMvpIdx], set identicalCandExist equal to TRUE and removeIdx equal to hMvpIdx.

[0565] 3. The candidate list HmvpCandList is updated as follows:

[0566] – If identicalCandExist is equal to TRUE or NumHmvpCand is equal to 6, then take the following steps:

[0567] – For each index i where i = (removeIdx+1) ... (NumHmvpCand-1), set HmvpCandList[i-1] equal to HmvpCandList[i].

[0568] – Set HmvpCandList[NumHmvpCand–1] equal to mvCand.

[0569] – If (identicalCandExist is equal to FALSE and NumHmvpCand is less than 6), then take the following steps:

[0570] – Set HmvpCandList[NumHmvpCand++] equal to mvCand.

[0571] The following table exemplarily shows the syntax for resetting the HMVP list at the beginning of a new partition group and at the beginning of each CTU row.

[0572]

[0573] At the beginning of a new tile group, the HMVP list is reset using the following syntax: When CtbAddrInRs == tilegroup_address, ie the beginning of a new tile group, NumHmvpCand is set to 0.

[0574] At the beginning of each CTU row, that is, the first CTU of each CTU row, the HMVP list is also reset. The syntax CtbAddrInRs%PicWidthInCtbsY==0 points to the first CTU of each CTU row, and the HMVP list is reset by setting the variable NumHmvpCand to 0.

[0575] Wavefront parallel processing (WPP)

[0576] In order to improve processing efficiency, a processing mode called wavefront parallel processing (WPP) is introduced, where the WPP mode allows parallel processing of CTUs of each row. In WPP mode, each CTU row is processed relative to the previous (immediately adjacent) CTU row by delaying, for example, two (or one, three or more) consecutive CTUs. Fig.13 shows wavefront parallel processing. See, for example, Fig.13 , the image is divided into CTUs, and each thread (row) includes 11 CTUs, that is, thread 1 includes CTU0 to CTU10, thread 2 includes CTU11 to CTU21, thread 3 includes CTU22 to CTU32, thread 4 includes CTU33 to CTU43, and so on. Therefore, in WPP mode, when the encoding and decoding process of CTU1 in thread 1 is completed, the encoding and decoding process of CTU11 in thread 2 can be started. Similarly, when the encoding and decoding process of CTU12 in thread 2 is completed, the encoding and decoding process of CTU22 in thread 3 can be started, when the encoding and decoding process of CTU23 in thread 3 is completed, the encoding and decoding process of CTU33 in thread 4 can be started, and when the encoding and decoding process of CTU34 in thread 4 is completed, the encoding and decoding process of CTU44 in thread 5 can be started.

[0577] For example, when history-based motion vector prediction (HMVP) is used, the CTUs in the image can be processed in parallel in the embodiment of the present application. The embodiment of the present application allows the use of blocks or block groups in combination with HMVP.

[0578] A tile scan is defined as an ordered arrangement of CTUs that partition a picture, where the CTUs are arranged consecutively in a CTU raster scan within a tile, and the tiles within a picture are arranged consecutively in a tile raster scan of the picture. A tile group header is defined as a portion of a coded tile group containing data elements related to the first or all tiles represented in the tile group.

[0579] Thus, a given image may contain one or more tile groups, and each tile group may contain one or more tiles. Parallel processing of tiles is allowed, for example, different tile groups or tiles are processed in parallel. At a finer granularity, wavefront parallel processing (WPP) or other tools may also be used in each tile in a given tile group to allow further parallel processing, wherein the other tools, for example, use CTU row-based parallelism.

[0580] Embodiments of the present invention are used, for example, to reset the HMVP list at the beginning of a new partition group and / or at the beginning of a new partition and / or at the beginning of each CTU row within a given partition. Therefore, parallel processing of CTUs is facilitated.

[0581] Figure 6 An image is shown that is divided into 3 different partition groups and 12 different partitions. The HMVP tool maintains a motion vector candidate list and maintains and updates it after processing each decoded block. In this way, the HMVP list is maintained and continuously updated until the last CTU of the CTU row. Therefore, the wavefront parallel processing tool cannot be executed within a given image, partition group or partition. This has been further explained above.

[0582] Figure 7 and Figure 8 The idea of ​​resetting the HMVP list at the beginning of each CTU row for both tile groups and tiles is shown.

[0583] Figure 7 Involved in Figure 6 That is, the number of tiles, the three tile groups, and the tile group scanning order (i.e., cross-tile raster scanning) correspond to those already given for Figure 6 Explained content. Figure 7 Also shown is the raster scan within a tile, as indicated by the solid arrows representing the CTU scan order within the tile. Figure 6 In addition, Figure 7 Instructions passed Figure 7 The arrows corresponding to block 1, block 2, and block 3 reset the HMVP list.

[0584] exist Figure 7For all of the following combinations, perform an HMVP list reset:

[0585] · At the start of each tile group (equivalent to the start of a stripe);

[0586] · At the start of each tile within a tile group;

[0587] · The first CTU of each CTU row within a given tile.

[0588] The first alternative of the possible syntax description is provided in the following table. The program tile_group_data consists of the encoded data of all tiles belonging to a given tile group. The for loop "for(i = 0; i <= num_tiles_in_tile_group_minus1; i++, tileIdx++)" iterates over all tiles in the given tile group, and the for loop "for(j = 0; j < NumCtusInTile[tileIdx]; j++, ctbAddrInTs++)" iterates over all CTUs in the given tile.

[0589]

[0590]

[0591] This syntax description is as Figure 8 shown. That is, Figure 8 shows the possible syntax description for the HMVP list reset for tile groups, tiles, and CTU rows within a tile.

[0592] The following provides a second alternative of the syntax for resetting the HMVP list (e.g., at the start of each CTU row within a tile):

[0593]

[0594] This alternative is as Fig.14 shown. That is, Fig.14 shows another possible syntax description for the HMVP list reset for tile groups, tiles, and CTU rows within a tile.

[0595] Fig. 9 Examples of three different tiles with non-uniform spacing (e.g., having different sizes) are given. Fig. 9 Can be compared with Figure 5 and Figure 6 for comparison. Figure 5 and Figure 6 show the tile group (TG) with respect to the raster scan order, Fig. 9 shows a rectangular TG and shows the tile boundaries represented by dashed lines. Fig. 9 A Coding Tree Block (CTB) is also shown.

[0596] Fig. 9 The HMVP list reset is shown for tile groups, tiles, and the start of a CTU row within a tile.

[0597] For example, reset the HMVP list based on the formula specified in the previous syntax table. The formula is as follows:

[0598]

[0599] The variable FirstCtbInCtuRowInTile is set to true only if the ctbAddrInTs of a given CTU is equal to the start of a tile group or tile, or if the address of a given CTU is the same as the first CTU of a given CTU row within a tile. The expression HMVPCandNum=0 is used to initialize or reset the HMVP list. When the variable FirstCtbInCtuRowInTile is set to true, HMVPCandNum is set to 0.

[0600] right Fig. 9 Using this formula in block 1 will produce the following results. Fig. 9 Block 1 in contains 12 CTUs, numbered 0 to 11. Its ColWidth is 4 CTBs, and the value of FirstCtbAddrTs is 0.

[0601] Therefore, for CTB0, CTB4, and CTB8, the value of FirstCtbInCtuRowInTile is 1, so the HMVP lists of these CTBs are reset.

[0602] Fig. 9 Block 2 in contains 21 CTBs, ColWidth is 7 CTBs, and the value of FirstCtbAddrTs is 12.

[0603] Therefore, for CTB12, CTB19, and CTB26, the value of FirstCtbInCtuRowInTile is 1, so the HMVP lists of these CTBs are reset.

[0604] Similarly, Fig. 9 Block 3 in contains a total of 12 CTUs, its ColWidth is 4 CTBs, and the value of FirstCtbAddrTs is 33.

[0605] Therefore, for CTB33, CTB37, and CTB41, the value of FirstCtbInCtuRowInTile is 1, so the HMVP lists of these CTBs are reset.

[0606] Therefore, by resetting (ie, "initializing") the HMVP list at the beginning of each CTU row (the first CTU) within a tile, operation of tools such as wavefront parallel processing or similar parallel processing can be achieved.

[0607] Another syntax for resetting the HMVP list at the beginning of each CTU row within a partition is provided below.

[0608]

[0609]

[0610] Another syntax for resetting the HMVP list (eg, at the beginning of each CTU row within a partition) is provided below.

[0611]

[0612] Initialize or reset the HMVP list by the following formula:

[0613] FirstCtbInCtuRowInTile=(j%ColWidth[tileIdx]==0)? 1:0

[0614] If((tile_group_type!=I)&&(FirstCtbInCtuRowInTile))

[0615] HMVPCandNum=0

[0616] The variable FirstCtbInCtuRowInTile is set to true only if the ctbAddrInTs of a given CTU (variable j) is equal to the start of a tile group or tile, or if the address of a given CTU is the same as the first CTU of a given CTU row within a tile. The expression (j%ColWidth[tileIdx]==0) is used to check if the address of a CTU is the same as the address of the first CTU of a given CTU row within a tile. The expression HMVPCandNum=0 is used to initialize or reset the HMVP list. When the variable FirstCtbInCtuRowInTile is set to true, HMVPCandNum is set to 0.

[0617] Embodiments of encoders 20, decoders 30, etc., and the functions described herein related to encoders 20, decoders 30, etc. may be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the various functions may be stored as one or more instructions or codes in a computer-readable medium or transmitted via a communication medium, and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media (such as data storage media), or communication media including any medium that facilitates the transfer of computer programs from one place to another (for example, according to a communication protocol). In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product may include a computer-readable medium.

[0618] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwave are used to transmit instructions from a website, server or other remote source, then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other transient media, but are directed to non-transient tangible storage media. Disks and optical disks used herein include compact discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks and Blu-ray discs, where disks usually reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0619] Instructions may be executed by one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. In addition, in some aspects, the various functions described herein may be provided in dedicated hardware and / or software modules for encoding and decoding, or incorporated into a combined codec. Moreover, the techniques may be implemented entirely in one or more circuits or logic elements.

[0620] The techniques of the present invention may be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this application to emphasize the functional aspects of the apparatus for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units may be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by a collection of interoperable hardware units (including one or more processors as described above).

[0621] The present invention includes the following aspects, wherein the numbering is not necessarily consistent with the numbering used in other parts of this application.

[0622] In a first aspect, an image decoding method implemented by a decoding device includes: initializing (or constructing) a history-based motion vector prediction (History-based Motion Vector Prediction, HMVP) list corresponding to a current CTU row in a block of an image; and processing a CTU in the current CTU row according to the initialized HMVP list.

[0623] According to the method described in the first aspect, in the second aspect, the HMVP list corresponding to the current CTU row is initialized in the following manner: resetting (or clearing) the HMVP list corresponding to the current CTU row; and / or setting a default value for the HMVP list corresponding to the current CTU row; and / or initializing the HMVP list corresponding to the current CTU row according to the HMVP list of the CTU in the previous CTU row in the decoding and / or scanning order (that is, the previous one refers to the previous one in the decoding and / or scanning order. In one embodiment, the previous CTU row may be the previous CTU row that is immediately adjacent to the current CTU row in the decoding and / or scanning order, for example, a CTU row that is spatially adjacent or directly adjacent to the current CTU row).

[0624] According to the method described in the first aspect or the second aspect, in the third aspect, the operation of initializing (e.g., resetting) the HMVP list corresponding to the current CTU row is performed at the beginning of a tile group (TG), at the beginning of a tile, or at the beginning of each CTU row in the tile.

[0625] According to the method described in any one of the first to third aspects, in the fourth aspect, the operation of initializing (e.g., resetting) the HMVP list corresponding to the current CTU row is performed when (e.g., before) processing (e.g., decoding) the first CTU in the current CTU row in the decoding and / or scanning order, wherein the first CTU in the current CTU row is or includes: the first CTU in the decoding and / or scanning order in a tile group (TG); and / or the first CTU in the decoding and / or scanning order in a tile; and / or the first CTU in the decoding and / or scanning order in a CTU row. Resetting the HMVP list corresponding to the first CTU in a tile (e.g., the first CTU in each tile) allows parallel processing or decoding of the tiles. Resetting the HMVP list corresponding to the first CTU in a CTU row (e.g., the first CTU of each CTU row in the tile) allows parallel processing of independent CTU rows in the same tile. It has been found that, especially for larger CTUs, the impact of the reset corresponding to each CTU row on compression performance is negligible, but the processing or decoding time can be reduced.

[0626] According to the method described in any one of aspects 1 to 4, in aspect 5, the operation of initializing (e.g., resetting) the HMVP list corresponding to the current CTU row is performed when (e.g., before) processing (e.g., decoding) the first CTU in each CTU row, for example, before processing or decoding the first block (e.g., CB, such as the first leaf node CB) of the first CTB (e.g., luminance or chrominance CTB) of the first CTU in the decoding order, for each CTU row in the partition.

[0627] According to the method described in any one of the first to fifth aspects, in the sixth aspect, the initializing the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein the resetting the HMVP list corresponding to the current CTU row includes: setting the length of the HMVP list (which may also be a length indicator or a length parameter, such as HMVPCandNum) to zero (e.g., HMVPCandNum=0) to process (e.g., decode) the first CTU in the decoding or scanning order in the current CTU row. By resetting the HMVP list by setting the length of the HMVP list to zero (and only gradually expanding the HMVP list), the number of candidates checked for prediction of the block to be decoded can be reduced compared to the case of having a full-length (i.e., maximum-length) HMVP list.

[0628] According to the method described in the sixth aspect, in the seventh aspect, the resetting of the HMVP list corresponding to the current CTU row also includes: when a block of the first CTU (for example, the first block or a continuous block, where the block can be a CTB or CB, such as a leaf node CB) is decoded (i.e., inter-frame decoding) using a motion vector, the length of the HMVP list is increased by 1, and the motion vector is added to the HMVP list.

[0629] According to the method of the seventh aspect, in the eighth aspect, the length of the HMVP list is increased and the motion vector is added only when the HMVP list does not contain motion information (for example, the motion information includes a motion vector or a reference picture index) (pruning embodiment).

[0630] According to the method of any one of aspects 6 to 8, in aspect 9, only the length of the HMVP list is increased until the maximum length of the HMVP list is reached.

[0631] According to the method described in any one of aspects 6 to 9, in aspect 10, the method further includes: decoding the current block (e.g., CTB or CB) of the current CTU only by considering the reduced number of HMVP list candidates based on the current length of the HMVP list, wherein the current length of the HMVP list is less than the maximum length of the HMVP list (e.g., the reduced number of HMVP list candidates corresponds to or is equal to the current length of the HMVP list).

[0632] According to the method of any one of the second aspect to the tenth aspect, in the eleventh aspect, regardless of the size of the block, the HMVP list is reset for each CTU row in the block.

[0633] According to the method described in any one of aspects 2 to 10, in aspect 12, when the size of a block is greater than (or greater than or equal to) a threshold size, the HMVP list is reset for each CTU row in the block, and / or when the size of the block is less than (or less than or equal to) the threshold size, the HMVP list is not reset for each CTU row in the block.

[0634] According to the method described in any one of aspects 2 to 12, in aspect 13, setting a default value for the HMVP list corresponding to the current CTU row includes: filling the MV in the HMVP list with the MV in the unidirectional prediction mode, wherein the MV in the unidirectional prediction mode is a zero motion vector or a non-zero motion vector, and the reference image includes the first reference image in list L0; and / or filling the MV in the HMVP list with the MV in the bidirectional prediction mode, wherein the MV in the bidirectional prediction mode is a zero motion vector or a non-zero motion vector, and the reference image includes the first reference image in list L0 and the first reference image in list L1.

[0635] According to the method described in any one of aspects 2 to 12, in aspect 14, each juxtaposed image may store a time domain HMVP list corresponding to each CTU row or corresponding to the entire image, and setting a default value for the HMVP list corresponding to the current CTU row includes: initializing the HMVP list corresponding to the current CTU row according to the time domain HMVP list.

[0636] According to the method described in any one of aspects 2 to 12, in aspect 15, the previous CTU row is a CTU row that is adjacent to the current CTU row and located above the current CTU row.

[0637] According to the method of aspect 15, in aspect 16, the CTU in the previous CTU row is the second CTU in the previous CTU row.

[0638] According to the method of aspect 15, in aspect 17, the CTU in the previous CTU row is the first CTU in the previous CTU row.

[0639] In an eighteenth aspect, an image encoding method implemented by an encoding device includes: initializing (or constructing) an HMVP list corresponding to a current CTU row in a block of an image; and processing a CTU in the current CTU row according to the constructed / initialized HMVP list.

[0640] According to the method described in aspect 18, in aspect 19, the HMVP list corresponding to the current CTU row is initialized in the following manner: resetting (or clearing) the HMVP list corresponding to the current CTU row; and / or setting a default value for the HMVP list corresponding to the current CTU row; and / or constructing / initializing the HMVP list corresponding to the current CTU row based on the HMVP list of the CTU in the previous CTU row in the coding and / or scanning order (that is, the previous one refers to the previous one in the coding and / or scanning order. In one embodiment, the previous CTU row may be the previous CTU row that is immediately adjacent to the current CTU row in the coding and / or scanning order, for example, a CTU row that is spatially adjacent or directly adjacent to the current CTU row).

[0641] According to the method described in aspect 18 or aspect 19, in aspect 20, the operation of initializing (e.g., resetting) the HMVP list corresponding to the current CTU row is performed at the beginning of a tile group (TG), at the beginning of a tile, or at the beginning of each CTU row in the tile.

[0642] According to the method described in any one of aspects 18 to 20, in aspect 21, the operation of initializing (e.g., resetting) the HMVP list corresponding to the current CTU row is performed when (e.g., before) processing (e.g., encoding) the first CTU in the coding and / or scanning order in the current CTU row, wherein the first CTU in the current CTU row is or includes: the first CTU in the coding and / or scanning order in a tile group (TG); and / or the first CTU in the coding and / or scanning order in a tile; and / or the first CTU in the coding and / or scanning order in a CTU row.

[0643] According to the method described in any one of aspects 18 to 21, in aspect 22, the operation of initializing (e.g., resetting) the HMVP list corresponding to the current CTU row is performed when (e.g., before) processing (e.g., encoding) the first CTU in each CTU row, for example, before processing or encoding the first block (e.g., CB, such as the first leaf node CB) of the first CTB (e.g., luminance or chrominance CTB) of the first CTU in the coding order, for each CTU row in the partition.

[0644] According to the method described in any one of aspects 18 to 22, in aspect 23, the initialization of the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the current CTU row includes: setting the length of the HMVP list (which may also be a length indicator or a length parameter, such as HMVPCandNum) to zero (for example, HMVPCandNum=0) to process (for example, encode) the first CTU in the encoding or scanning order in the current CTU row.

[0645] According to the method described in aspect 23, in aspect 24, resetting the HMVP list corresponding to the current CTU row also includes: when a block of the first CTU (for example, the first block or a continuous block, where the block can be a CTB or CB, such as a leaf node CB) is encoded (i.e., inter-frame encoding) using a motion vector, the length of the HMVP list is increased by 1, and the motion vector is added to the HMVP list.

[0646] According to the method of aspect 24, in aspect 25, the length of the HMVP list is increased and the motion vector is added only when the HMVP list does not contain motion information (for example, the motion information includes a motion vector or a reference picture index) (pruning embodiment).

[0647] According to the method of any one of aspects 23 to 25, in aspect 26, only the length of the HMVP list is increased until the maximum length of the HMVP list is reached.

[0648] According to the method described in any one of aspects 23 to 26, in aspect 27, the method further includes: encoding the current block (e.g., CTB or CB) of the current CTU only by considering the reduced number of HMVP list candidates based on the current length of the HMVP list, wherein the current length of the HMVP list is less than the maximum length of the HMVP list (e.g., the reduced number of HMVP list candidates corresponds to or is equal to the current length of the HMVP list).

[0649] According to the method of any one of aspects 19 to 27, in aspect 28, regardless of the size of the block, the HMVP list is reset for each CTU row in the block.

[0650] According to the method described in any one of aspects 19 to 27, in aspect 29, when the size of a block is greater than (or greater than or equal to) a threshold size, the HMVP list is reset for each CTU row in the block, and / or when the size of the block is less than (or less than or equal to) the threshold size, the HMVP list is not reset for each CTU row in the block.

[0651] According to the method described in any one of aspects 19 to 29, in aspect 30, setting a default value for the HMVP list corresponding to the current CTU row includes: filling the MV in the HMVP list with the MV in the unidirectional prediction mode, wherein the MV in the unidirectional prediction mode is a zero motion vector or a non-zero motion vector, and the reference image includes the first reference image in list L0; and / or filling the MV in the HMVP list with the MV in the bidirectional prediction mode, wherein the MV in the bidirectional prediction mode is a zero motion vector or a non-zero motion vector, and the reference image includes the first reference image in list L0 and the first reference image in list L1.

[0652] According to the method described in any one of aspects 19 to 29, in the thirty-first aspect, each juxtaposed image may store a time domain HMVP list corresponding to each CTU row or corresponding to the entire image, and setting a default value for the HMVP list corresponding to the current CTU row includes: initializing / constructing the HMVP list corresponding to the current CTU row according to the time domain HMVP list.

[0653] According to the method described in any one of aspects 19 to 29, in aspect 32, the previous CTU row is a CTU row that is adjacent to the current CTU row and located above the current CTU row.

[0654] According to the method described in aspect 32, in aspect 33, the CTU in the previous CTU row is the second CTU in the previous CTU row.

[0655] According to the method described in aspect 32, in aspect 34, the CTU in the previous CTU row is the first CTU in the previous CTU row.

[0656] In aspect 35, an encoder (20) comprises a processing circuit for executing the method described in any one of aspects 19 to 34.

[0657] In a thirty-sixth aspect, a decoder (30) comprises a processing circuit for executing the method described in any one of aspects 1 to 17.

[0658] In aspect 37, a computer program product comprises a program code for executing the method described in any one of aspects 1 to 34.

[0659] In aspect thirty-eight, a decoder comprises: one or more processors; a non-transitory computer-readable storage medium, coupled to the processor and storing a program executed by the processor, wherein, when the processor executes the program, the decoder executes the method described in any one of aspects one to seventeen.

[0660] In aspect thirty-ninth, an encoder comprises: one or more processors; a non-transitory computer-readable storage medium, coupled to the processor and storing a program executed by the processor, wherein, when the processor executes the program, the encoder executes any one of the methods described in aspects nineteen to thirty-four.

Claims

1. An image encoding method implemented by an encoding device, characterized in that: include: The History-based Motion Vector Prediction (HMVP) list performs redundancy detection when updating and inserting HMVP candidates. If the same HMVP candidate exists, the same HMVP candidate is deleted from the table, and all HMVP candidate indexes after the same HMVP candidate are reduced by 1. Under the condition that CtbAddrX = = CtbToTileColBd[ CtbAddrX ], the HMVP list corresponding to the current Coding Tree Unit (CTU) row in the image block is initialized. The CtbAddrX represents the address of the horizontal CTB, and the CtbToTileColBd[ ctbAddrX ] represents the conversion from the address of the horizontal CTB to the address of the left block column boundary in units of CTB; The CTUs in the current CTU row are processed according to the initialized HMVP list.

2. The method according to claim 1, characterized in that Initializing the HMVP list corresponding to the current CTU row includes the following steps: Reset the HMVP list corresponding to the current CTU row; and / or Setting a default value for the HMVP list corresponding to the current CTU row; and / or Based on the HMVP list of the CTU in the previous CTU row in the coding and / or scanning order, the HMVP list corresponding to the current CTU row is initialized.

3. The method according to claim 2, characterized in that The previous CTU row in the coding and / or scanning order is a previous CTU row that is immediately adjacent to the current CTU row in the coding and / or scanning order.

4. The method according to claim 3, characterized in that The previous CTU row that is immediately adjacent to the current CTU row in the coding and / or scanning order is a CTU row that is spatially adjacent to or directly adjacent to the current CTU row.

5. The method according to any one of claims 1 to 4, characterized in that The operation of initializing the HMVP list corresponding to the current CTU row is performed at the beginning of a tile group (TG), or at the beginning of the tile, or at the beginning of each CTU row in the tile.

6. The method according to any one of claims 1 to 4, characterized in that The operation of initializing the HMVP list corresponding to the current CTU row is performed before processing the first CTU in the encoding and / or scanning order in the current CTU row, wherein the first CTU in the current CTU row is or includes: The first CTU in the coding and / or scanning order of a tile group (TG); and / or The first CTU in the coding and / or scanning order of the partition; and / or The first CTU in the CTU row in coding and / or scanning order.

7. The method according to any one of claims 1 to 4, characterized in that Initializing the HMVP list corresponding to the current CTU row includes: resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the current CTU row includes: The length of the HMVP list is set to zero to process the first CTU in the coding or scanning order in the current CTU row.

8. The method according to any one of claims 1 to 4, characterized in that Initializing the HMVP list corresponding to the current CTU row includes resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the first CTU in the block allows parallel processing or encoding of the blocks.

9. The method according to claim 8, characterized in that Initializing the HMVP list corresponding to the current CTU row includes resetting the HMVP list corresponding to the current CTU row, wherein resetting the HMVP list corresponding to the first CTU in each block allows parallel processing or encoding of blocks.

10. The method according to any one of claims 1 to 4, characterized in that The operation of initializing the HMVP list corresponding to the current CTU row is performed for each CTU row in the block before processing the first CTU in each CTU row.

11. The method according to claim 8, characterized in that The resetting of the HMVP list corresponding to the current CTU row further includes: When the block of the first CTU is encoded using the motion vector, ie, inter-frame encoding is performed, the length of the HMVP list is increased by 1, and the motion vector is added to the HMVP list.

12. The method according to claim 11, characterized in that Only if the HMVP list does not contain motion information, the length of the HMVP list is increased and the motion vector is added.

13. The method according to claim 12, characterized in that The motion information includes at least one of a motion vector and a reference image index.

14. The method according to claim 6, characterized in that The length of the HMVP list is only increased until the maximum length of the HMVP list is reached.

15. The method according to claim 6, characterized in that The method further comprises: The current block of the current CTU row is encoded considering only the reduced number of HMVP list candidates according to a current length of the HMVP list, wherein the current length of the HMVP list is less than a maximum length of the HMVP list.

16. The method according to claim 15, characterized in that The reduced number of HMVP list candidates corresponds to or is equal to the current length of the HMVP list.

17. The method according to any one of claims 2 to 4, characterized in that Regardless of the size of a partition, the HMVP list is reset for each CTU row in the partition.

18. The method according to any one of claims 2 to 4, characterized in that When the size of the partition is greater than or equal to the threshold size, the HMVP list is reset for each CTU row in the partition, and / or when the size of the partition is less than the threshold size, the HMVP list is not reset for each CTU row in the partition.

19. The method according to any one of claims 2 to 4, characterized in that The setting of a default value for the HMVP list corresponding to the current CTU row includes: Filling the motion vector (MV) in the HMVP list with the MV in the unidirectional prediction mode; Wherein, the MV in the unidirectional prediction mode is a zero motion vector or a non-zero motion vector; The reference image comprises the first reference image in list 0 (L0); and / or The MV in the HMVP list is filled with the MV in the bidirectional prediction mode, wherein the MV in the bidirectional prediction mode is a zero motion vector or a non-zero motion vector, and the reference image includes the first reference image in list L0 and the first reference image in list 1 (list1, L1).

20. The method according to any one of claims 2 to 4, characterized in that Each concatenated image stores a time domain HMVP list corresponding to each CTU row or corresponding to the entire image, and setting a default value for the HMVP list corresponding to the current CTU row includes: The HMVP list corresponding to the current CTU row is initialized according to the time-domain HMVP list.

21. The method according to any one of claims 2 to 4, characterized in that The previous CTU row is a CTU row that is immediately adjacent to the current CTU row and is located above the current CTU row; The CTU in the previous CTU row is the second CTU in the previous CTU row, or the CTU in the previous CTU row is the first CTU in the previous CTU row.

22. The method according to any one of claims 1 to 4, characterized in that The CTUs in a CTU row have corresponding addresses. The address of the first CTU in a CTU row is identified by determining whether the modulo operation of address j relative to the current block produces 0 using the following expression: if ( j % ColWidth[tileIdx] == 0 ), Among them, j is the address of the current CTU, ColWidth[tileIdx] represents the width of the block relative to the number of CTUs, tileIdx represents the index of each block, and "%" is the modulus operator.

23. The method according to any one of claims 1 to 4, characterized in that The CTUs in a CTU row have corresponding addresses, and the address of the first CTU in each CTU row in the block is identified by determining whether the address of the current CTU is the same as the address of the CTU in the list or array, thereby deriving the first CTU in each CTU row in the block, wherein the list or array stores the address of the left block column boundary in units of CTB.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a video code stream obtained by the method according to any one of claims 1 to 23 executed by one or more processors.

25. An encoder (20), characterized in that Comprising processing circuitry for performing a method according to any one of claims 1 to 23.

26. A computer program product, characterized in that The method comprises a program code for executing the method according to any one of claims 1 to 23.

27. A computer-readable non-transitory medium storing a program, characterized in that: The computer-readable non-transitory medium includes instructions; when the instructions are executed on a processor, the processor performs the method according to any one of claims 1 to 23.

28. An encoder, characterized in that include: one or more processors; A non-transitory computer-readable storage medium is coupled to the processor and stores a program executed by the processor, wherein when the processor executes the program, the encoder is caused to perform the method according to any one of claims 1 to 23.

Images