Video processing method and apparatus

Abstract

The embodiments of the present application provide a video processing method and apparatus, which relate to the field of video encoding and decoding. The method includes: a coder side can predict motion information of a current block based on the absolute position of the block in a current video frame in a lookup table, thereby improving MV encoding / decoding efficiency and further improving video coding efficiency.

Description

[Technical Field]

[0001] This application claims priority to Chinese Patent Application No. 202211517674.X, entitled "VIDEO PROCESSING METHOD AND APPARATUS," filed with the State Intellectual Property Office of the People's Republic of China on November 30, 2022, which is incorporated herein by reference in its entirety.

[0002] TECHNICAL FIELD Embodiments of the present application relate to the field of video encoding and decoding, and in particular to video processing methods and apparatus. [Background technology]

[0003] Digital video functionality may be applied to a variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite wireless telephones (referred to as "smartphones"), video conferencing devices, video streaming devices, etc. Digital video devices implement video compression technologies such as those described in MPEG-2, MPEG-4, ITU-T H.263, and ITU-T H.264 / MPEG-4 Part 10 advanced video coding (AVC), video coding standard H.265 / high efficiency video coding (HEVC) standards, and extensions to such standards. By implementing such video compression technologies, video devices can more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0004] In video compression techniques, spatial (intra) prediction and / or temporal (inter) prediction are performed to reduce or remove inherent redundancy in video sequences. In block-based video coding, a video slice (i.e., a video frame or a portion of a video frame) may be divided into picture blocks, which may also be referred to as tree blocks, coding units (CUs), and / or coding nodes. Picture blocks in an (I) slice of a picture to be intra-coded are coded through spatial prediction of reference samples in neighboring blocks in the same picture. Picture blocks in a (P or B) slice of a picture to be inter-coded may be coded through spatial prediction of reference samples in neighboring blocks in the same picture or temporal prediction of reference samples in another reference picture. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame. Existing video coding methods have low coding efficiency. Summary of the Invention [Means for solving the problem]

[0005] In order to solve the above-mentioned technical problems, the present application provides a video processing method and apparatus, in which a coder side can predict the motion information of a current block according to the absolute position of the block in a current video frame in a lookup table, thereby improving MV encoding / decoding efficiency and further improving video coding efficiency.

[0006] According to a first aspect, the present application provides a video processing method. The method may be applied to an encoder side or a decoder side. The encoder side or the decoder side may be referred to as a coder side. The method may include: obtaining a lookup table for a current block in a current video frame, the lookup table including at least one candidate position, the candidate position being within the current video frame and relative to a sample in an upper-left corner of the current video frame; obtaining motion information for the current block based on the lookup table; and coding (encoding or decoding) the current block based on the motion information.

[0007] For example, the candidate position may be an absolute position of the block within the current video frame, where the absolute position is within the video frame and may be expressed as a position relative to a sample in the top left corner of the video frame.

[0008] For example, the motion information may include at least one of reference block information and motion vectors (MVs).

[0009] For example, the reference block information may include at least one of unidirectional or bidirectional prediction information, a reference block list, and a reference block index corresponding to the reference block list.

[0010] In this way, the coder side in this application can predict the motion information of the current block based on the absolute position of the block in the current video frame in the lookup table to reduce the complexity of coding the motion information, thereby improving the MV encoding / decoding efficiency and further improving the video coding efficiency.

[0011] According to a first aspect, the current video frame may include text content, which is content that includes characters and is different from a natural picture.

[0012] In this way, when a video contains text content, the motion information of the current block can be predicted based on the lookup table, which helps predict the motion information of characters and can improve the coding efficiency of the MV of repeated character blocks.

[0013] According to the first aspect or any one of the above-described implementations of the first aspect, the candidate locations include locations of recurring blocks within the current video frame.

[0014] For example, the word "Qin" (Chinese character) appears many times in a video frame, and the block position of the word "Qin" is written into a lookup table. In this case, when the next word "Qin" is coded or decoded, the MVP of the second word "Qin" can be obtained based on the position of the word "Qin" in the lookup table, so that the MVP is more accurate and the MVD is smaller. This improves MV coding efficiency.

[0015] There may be repeated blocks (which may be blocks with repeated characters or may be repeated picture blocks) in a video frame. The coder side can add the absolute positions of the repeated blocks in the video frame to a lookup table to improve the prediction efficiency and coding efficiency of the MV of the repeated blocks in the video frame.

[0016] According to the first aspect or any one of the above-described implementation forms of the first aspect, the candidate locations include locations of the coding block within the current video frame and / or locations of reference blocks for the coding block.

[0017] The coder side can use the position of the coding block (the position is also an absolute position) and / or the position of the reference block of the coding block (also referred to as the reference position), where the reference position is the absolute position of the block to which the MV of the coding block points in the corresponding video frame. In this way, when the motion information of the current block is predicted based on the lookup table, the position information of the coding block can be referenced, so that the motion information of the repeatedly appearing block can be accurately predicted, thereby reducing the amount of coding bits for the motion information of the block and improving the MV coding efficiency.

[0018] According to the first aspect or any one of the above-described implementations of the first aspect, the candidate locations include locations of first occurrences of the repeated block within the current video frame.

[0019] The coder side can write the position (which is also the absolute position) where the repeated character or picture block first appears in the current video frame as a candidate position in the lookup table, so that when coding the character or picture block that reappears in the video frame, the coder side can refer to the position where the character or picture block first appears in the lookup table to predict the motion vector of the reappearing character or picture block. In this way, the positions of blocks that are easily referenced multiple times can be stored in the lookup table to improve MV prediction efficiency.

[0020] According to the first aspect or any one of the above-mentioned implementation forms of the first aspect, obtaining a lookup table for a current block in a current video frame includes: updating the lookup table by selecting, based on motion information of at least one coding block in the current video frame, candidate positions to be added to the lookup table from positions corresponding to at least one coding block, wherein the positions corresponding to the at least one coding block include positions of the coding block in the current video frame or positions of reference blocks of the coding block in the current video frame; and obtaining the updated lookup table.

[0021] In example a1, when the encoder side encodes a block based on the actual motion information of the block to be encoded (e.g., in AMVP mode), after encoding the block, the encoder side can add the block's position information (e.g., the coordinates (x, y) of the block's absolute position in the current video frame) to a position lookup table.

[0022] In example b1, when the first character among repeated characters in a video frame is coded in AMVP mode, the MVP determined from the neighboring MVPs is significantly different from the actual MV of the character, and the MVD is large. Therefore, when the character at the character start position in the video frame is coded, a large amount of bits are consumed for the motion information (e.g., MV) of the block of the character.

[0023] In example c1, if the encoder determines that the absolute value of the horizontal component and / or the absolute value of the vertical component of the MVD of the coding block is greater than a second preset threshold, it indicates that a large amount of bits will be consumed when the encoder encodes the MVD of the block, and it may also indicate that the coding block is likely to be a character block appearing for the first time in the video frame. In this case, the encoder may add the position information (x, y) of the coding block within the video frame to the position lookup table.

[0024] In example d1, if the encoder determines that the sum of the absolute value of the horizontal component and the absolute value of the vertical component of the MVD of the coding block is greater than a third preset threshold, it indicates that a large amount of bits will be consumed when the encoder encodes the MVD of the block, and it may also indicate that the coding block is likely to be a character block appearing for the first time in the video frame. In this case, the encoder may add the position information (x, y) of the coding block within the video frame to the position lookup table.

[0025] In example f1, the encoder side can further add the reference position of the coding block to the position lookup table.

[0026] According to the first aspect or any one of the above-mentioned implementation forms of the first aspect, obtaining a lookup table for a current block in a current video frame includes obtaining a global lookup table for at least one region in the current video frame, wherein the at least one region includes a first region, the first region includes the current block, and the global lookup table includes at least one first position, wherein the at least one first position is within the region and is a position relative to a sample of the upper left corner of the current video frame; and obtaining a lookup table for the current block in the current video frame based on the global lookup table for the at least one region, wherein the first position includes a candidate position in the lookup table.

[0027] The encoder side can generate a global lookup table for each of at least one region in the current video frame and encode the global lookup table for the region into a bitstream for transmission to the decoder side.

[0028] The global lookup table for the region may include at least one first location. The first location may also be an absolute location within the region and within the current video frame. The first location may be within the region and relative to a sample in the upper left corner of the current video frame. The global lookup table may include absolute locations of blocks within the region.

[0029] In one example, when obtaining the lookup table of the current block based on the global lookup table of at least one region, the coder side can use the global lookup table of the first region including the current block as the lookup table of the current block, so that each first position in the global lookup table of the first region is a candidate position in the lookup table of the current block.

[0030] In another example, when obtaining the lookup table of the current block based on the global lookup table of at least one region, the coder side can delete some of the first positions (which are absolute positions of uncoded blocks in the current video frame) in the global lookup table of the first region including the current block, and use the global lookup table of the first region obtained through the deletion operation as the lookup table of the current block. Thus, each first position in the global lookup table of the first region and obtained through the deletion operation is a candidate position in the lookup table of the current block.

[0031] In this way, the data in the generated lookup table is more accurate, which helps to accurately predict the MV and reduce the amount of coding bits for the MV.

[0032] According to the first aspect or any one of the above-described implementation forms of the first aspect, the global lookup table for each of the at least one region is a lookup table generated based on the frequency with which blocks in the region repeatedly appear before the blocks in the region are coded.

[0033] For example, the first location may be the absolute location of a repeated block of text or a repeated picture block within the current video frame.

[0034] The absolute position represents the position relative to the pixel in the top left corner of the current video frame.

[0035] According to the first aspect or any one of the above-mentioned implementation forms of the first aspect, obtaining motion information of the current block based on the lookup table includes: determining at least one target candidate position based on the lookup table, wherein the candidate positions in the lookup table include the target candidate position; determining at least one motion vector predictor MVP (e.g., at least one MVP candidate) for the current block based on the at least one target candidate position; and obtaining motion information of the current block based on the at least one MVP.

[0036] According to the first aspect or any one of the above-described implementation forms of the first aspect, there is one target candidate location, and determining at least one target candidate location based on the lookup table includes determining, based on the lookup table and an index of the candidate location, a target candidate location that is in the lookup table and corresponds to the index.

[0037] At the encoder side, an index of the candidate position (i.e., the target candidate position) corresponding to the MV with the lowest coding cost in the lookup table may be determined based on the lookup table, and the index is encoded into the bitstream for transmission to the decoder side.

[0038] The decoder side can obtain the index of the target candidate position from the bitstream and obtain the target candidate position indicated by the index from the lookup table on the decoder side.

[0039] According to the first aspect or any one of the above-described implementation forms of the first aspect, determining at least one target candidate location based on a lookup table includes selecting a predetermined amount of candidate locations from the lookup table as target candidate locations, wherein the predetermined amount is an integer greater than 0.

[0040] When predicting the motion information of the current block based on the lookup table, the coder side can select a certain number of candidate positions from the lookup table, convert the candidate positions into candidate MVPs, and obtain the motion information of the current block based on a predetermined amount of candidate MVPs.

[0041] According to the first aspect or any one of the above-mentioned implementation forms of the first aspect, obtaining motion information of the current block based on at least one MVP includes constructing an MVP candidate list for the current block based on the at least one MVP, and obtaining motion information of the current block based on the MVP candidate list.

[0042] For example, when the current block is coded in AMVP mode, the MVP candidate list may include MVP candidates in the AMVP candidate list and may further include MVPs (which are also MVP candidates) obtained by transforming a predetermined amount of candidate positions.

[0043] For example, when the current block is coded in merge mode, the MVP candidate list may include merge candidates (which are also MVPs) in the merge candidate list, and may further include MVPs (which are also MVP candidates) obtained by transforming a predetermined amount of candidate positions.

[0044] For example, the coder side can obtain the motion information of the current block based on the MVP candidate list in the AMVP mode or the merge mode.

[0045] According to the first aspect or any one of the above-mentioned implementation forms of the first aspect, the predetermined amount is related to at least one of the coding mode of the motion information of the current block, the size of the current block, and the shape of the current block.

[0046] According to a second aspect, the present application provides a video processing device, comprising: a first acquisition module configured to acquire a lookup table for a current block within a current video frame, the lookup table including at least one candidate location, the candidate location being within the current video frame and being a location relative to a sample in an upper left corner of the current video frame; a second obtaining module configured to obtain motion information of the current block based on the lookup table; a coding module configured to code the current block based on the motion information.

[0047] According to a second aspect, the current video frame includes text content.

[0048] According to the second aspect or any one of the above-described implementations of the second aspect, the candidate locations include locations of recurring blocks within the current video frame.

[0049] According to the second aspect or any one of the above-described implementation forms of the second aspect, the candidate locations include locations of the coding block within the current video frame and / or locations of reference blocks for the coding block.

[0050] According to the second aspect or any one of the above-described implementations of the second aspect, the candidate locations include locations of first occurrences of the repeated block within the current video frame.

[0051] According to the second aspect or any one of the above-mentioned implementation forms of the second aspect, the first acquisition module is specifically configured to: select, based on motion information of at least one coding block in a current video frame, a candidate position to be added to the lookup table from positions corresponding to the at least one coding block, to update the lookup table, wherein the positions corresponding to the at least one coding block include a position of the coding block in the current video frame or a position of a reference block of the coding block in the current video frame; and obtain an updated lookup table.

[0052] According to the second aspect or any one of the above-mentioned implementation forms of the second aspect, the first acquisition module is specifically configured to: acquire a global lookup table for at least one region in a current video frame, wherein the at least one region includes a first region, the first region includes a current block, and the global lookup table includes at least one first position, wherein the at least one first position is within the region and is a position relative to a sample of the upper left corner of the current video frame; and acquire a lookup table for a current block in the current video frame based on the global lookup table of the at least one region, wherein the first position includes a candidate position in the lookup table.

[0053] According to the second aspect or any one of the above-described implementation forms of the second aspect, the global lookup table for each of the at least one region is a lookup table generated based on the frequency with which blocks in the region repeatedly appear before the blocks in the region are coded.

[0054] According to the second aspect or any one of the above-mentioned implementation forms of the second aspect, the second acquisition module is specifically configured to determine at least one target candidate position based on a lookup table, where the candidate positions in the lookup table include the target candidate position; determine at least one motion vector predictor MVP of the current block based on the at least one target candidate position; and acquire motion information of the current block based on the at least one MVP.

[0055] According to the second aspect or any one of the above-mentioned implementation forms of the second aspect, the second acquisition module is specifically configured to determine, based on the lookup table and an index of the candidate location, a target candidate location that is in the lookup table and corresponds to the index, and there is one target candidate location.

[0056] According to the second aspect or any one of the above-mentioned implementation forms of the second aspect, the second acquisition module is specifically configured to select a predetermined amount of candidate locations from the lookup table as target candidate locations, where the predetermined amount is an integer greater than 0.

[0057] According to the second aspect or any one of the above-mentioned implementation forms of the second aspect, the second acquisition module is specifically configured to construct an MVP candidate list for the current block based on the at least one MVP, and acquire motion information of the current block based on the MVP candidate list.

[0058] According to the second aspect or any one of the above-mentioned implementation forms of the second aspect, the predetermined amount is related to at least one of the coding mode of the motion information of the current block, the size of the current block, and the shape of the current block.

[0059] The effect of the video processing device in the implementation is similar to the effect of the video processing method in the implementation, and the details will not be described again herein.

[0060] According to a third aspect, the present application provides an encoder that may be configured to implement the method in any one of the above-mentioned implementations.

[0061] According to a third aspect, the encoder is further configured to encode the global lookup table into a bitstream of the current video frame.

[0062] According to the third aspect or any one of the above-described implementations of the third aspect, the encoder is further configured to encode an index of the target candidate position in the lookup table into a bitstream of the current video frame.

[0063] According to the third aspect or any one of the above-described implementations of the third aspect, the encoder is further configured to encode the predetermined amount into a bitstream of the current video frame.

[0064] The effect of the encoder in the implementation is similar to the effect of the video processing method in the implementation, and the details will not be described again here.

[0065] According to a fourth aspect, the present application provides a decoder that may be configured to implement the method in any one of the above-mentioned implementations.

[0066] According to a fourth aspect, the decoder is further configured to obtain a global lookup table for at least one region in the current video frame from the bitstream.

[0067] According to the fourth aspect or any one of the above-described implementations of the fourth aspect, the encoder is further configured to obtain an index of the target candidate position from the bitstream.

[0068] According to the fourth aspect or any one of the above-described implementations of the fourth aspect, the decoder is further configured to obtain a predetermined amount from the bitstream.

[0069] The effect of the decoder in the implementation is similar to the effect of the video processing method in the implementation, and the details will not be described again here.

[0070] According to a fifth aspect, the present application provides a video processing device, the video processing device including one or more interface circuits and one or more processors, the interface circuits configured to receive signals from a memory and send signals to the processors, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, the processor can implement the method in any one of the above-mentioned implementations.

[0071] The effect of the video processing device in this implementation is similar to that of the video processing method in the implementation, and the details will not be described again here.

[0072] According to a sixth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed on a computer or processor, the computer or processor is enabled to perform the method in any one of the above-described implementations.

[0073] The effect of the computer-readable storage medium in this implementation is similar to that of the video processing method in the implementation, and the details will not be described again herein.

[0074] According to a seventh aspect, the present application provides a computer program product, the computer program product including a software program, which, when executed by a computer or processor, performs the method in any one of the above-described implementations.

[0075] The effect of the computer program product in this implementation is similar to the effect of the video processing method in the implementation, and the details will not be described again herein.

[0076] According to an eighth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium storing a video bitstream obtained by executing, by one or more processors, the method in any one of the above-mentioned implementations.

[0077] According to a ninth aspect, the present application provides a video data encoder, the encoder including a memory configured to store video data in the form of a bitstream, the encoder configured to perform a method in any one of the above-mentioned implementation forms.

[0078] According to a tenth aspect, the present application provides a video data decoder, the decoder including a memory configured to store video data in the form of a bitstream, the decoder configured to perform a method in any one of the above-mentioned implementation forms.

[0079] In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly describe the accompanying drawings required to describe the embodiments of the present application. Obviously, the accompanying drawings in the following description only illustrate some embodiments of the present application, and those skilled in the art can still derive other accompanying drawings from these accompanying drawings without creative efforts. [Brief explanation of the drawings]

[0080] [Figure 1A] 1 is a block diagram of an example video encoding and decoding system according to one embodiment; [Figure 1B] 1 is a block diagram of an example video coding system according to one embodiment; [Figure 2] FIG. 2 is a block diagram illustrating an example of a configuration of an encoder according to an embodiment. [Figure 3] FIG. 2 is a block diagram of an example configuration of a decoder according to an embodiment. [Figure 4] 1 is a block diagram of an example of a video coding device according to one embodiment. [Figure 5] FIG. 10 is a block diagram of another example of an encoding device or a decoding device according to an embodiment. [Figure 6] FIG. 2 is a diagram of a frame according to one embodiment. [Figure 7] 1 is a process diagram of a video transmission system according to one embodiment. [Figure 8] FIG. 2 is a diagram of a video coding process according to one embodiment. [Figure 9a] FIG. 1 is a diagram of a MV prediction process according to one embodiment. [Figure 9b] FIG. 1 is a diagram of a MV prediction process according to one embodiment. [Figure 10] FIG. 31 is a block diagram of an example configuration of a content supply system 3100 for implementing a content distribution service. [Figure 11] FIG. 2 is a block diagram of an example configuration of a terminal device. DETAILED DESCRIPTION OF THE INVENTION

[0081] The following clearly describes the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. It is clear that the described embodiments are only a part, but not all, of the embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0082] The term "and / or" in this specification describes only a relational relationship between related objects and indicates that three relationships may exist. For example, A and / or B can indicate the following three cases: when only A exists, when both A and B exist, and when only B exists.

[0083] In the description and claims of the embodiments of the present application, terms such as "first" and "second" are intended to distinguish between different objects, but do not indicate a specific order of the objects. For example, "first target object," "second target object," etc. are used to distinguish between different target objects, but are not used to describe a specific order of the target objects.

[0084] In the embodiments of the present application, words such as "exemplary," "for example," and the like are used to denote providing an example, illustration, or explanation. Any embodiment or design scheme described in the embodiments of the present application as an "example" or "for example" should not be described as being preferred or having more advantages over another embodiment or design scheme. Rather, the use of words such as "exemplary," "for example," and the like is intended to present the relevant concept in a particular way.

[0085] In describing the embodiments of this application, unless otherwise specified, "plurality" means two or more than two. For example, a plurality of processing units is two or more processing units, and a plurality of systems is two or more systems.

[0086] The following describes embodiments of the present invention with reference to the accompanying drawings in which embodiments of the present invention are described. In the following description, reference is made to the accompanying drawings, which form a part of this invention and which show, by way of illustration, certain aspects of example embodiments of the present invention or in which embodiments of the present invention may be used. It should be understood that embodiments of the present invention may be used in other ways and may include structural or logical changes not shown in the accompanying drawings. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present invention is defined by the appended claims. For example, it should be understood that disclosure referring to a described method may also apply to a corresponding device or system for performing the method, and vice versa. For example, when one or more particular method steps are described, the corresponding device may include one or more units, such as functional units, for performing the described one or more method steps (e.g., one unit that performs one or more steps, or multiple units that each perform one or more of the steps), even if such one or more units are not explicitly described or shown in the accompanying drawings. Correspondingly, for example, when a particular apparatus is described based on one or more units, such as functional units, a corresponding method may include one or more steps for performing the functions of the one or more units (e.g., one step for performing the functions of one or more units, or multiple steps each performing the functions of one or more of the units), even if one or more steps are not explicitly described or shown in the accompanying drawings. Furthermore, it should be understood that the features of various exemplary embodiments and / or aspects described herein may be combined with each other, unless specifically stated otherwise.

[0087] The technical solutions in the embodiments of the present invention can be applied not only to existing video coding standards (e.g., H.264 standard and HEVC standard), but also to future video coding standards (e.g., H.267 and AV2). The terms used in the implementation form part of the present invention are merely intended to describe specific embodiments of the present invention, and are not intended to limit the present invention. The following will first briefly describe some concepts that can be used in the embodiments of the present invention.

[0088] Video coding typically refers to the processing of a sequence of pictures that form a video or a video sequence. In the field of video coding, the terms "frame" or "picture" may be used synonymously. Video coding, as used herein, refers to video encoding or video decoding. Video encoding is performed on the source side and typically involves processing the original video picture (e.g., by compression) to reduce the amount of data required to represent the video picture for more efficient storage and / or transmission. Video decoding is performed on the destination side and typically involves the reverse processing compared to the encoder's processing to reconstruct the video picture. "Coding" of a video picture in embodiments may be understood as "encoding" or "decoding" of a video sequence. The combination of the encoding and decoding parts is also referred to as a codec (encoding and decoding).

[0089] A video sequence includes a series of pictures. Pictures are further divided into slices, and each slice is further divided into blocks. In video coding, each block is coded. Some new video coding standards further expand the concept of a "block." For example, the H.264 standard uses macroblocks (MBs), which can be further divided into multiple predictive blocks (or partitions) that can be used for predictive coding. The high-efficiency video coding (HEVC) standard uses basic concepts such as coding units (CUs), prediction units (PUs), and transform units (TUs). Multiple types of block units are obtained through functional partitioning, and these units are described based on a new tree-based structure. For example, a CU may be divided into smaller CUs based on a quadtree, and each of the smaller CUs may be further divided to form a quadtree structure. A CU is the basic unit for dividing and coding a coding picture. PUs and TUs also have a similar tree structure. A PU may correspond to a prediction block and is a basic unit for predictive coding. A CU is further divided into multiple PUs in a partitioned manner. A TU may correspond to a transform block and is a basic unit for transforming a prediction residual. Essentially, CUs, PUs, and TUs are all concepts of a block (or picture block).

[0090] For example, in HEVC, a CTU is divided into multiple CUs based on a quadtree structure, represented as a coding tree. The decision of whether to code a picture region through inter (temporal) prediction or intra (spatial) prediction is made at the CU level. Each CU may be further divided into one, two, or four PUs based on the PU partition type. The same prediction process is performed in one PU, and related information is transmitted to the decoder on a PU-by-PU basis. After a residual block is obtained by performing the prediction process, the CU may be divided into TUs based on the PU partition type and another quadtree structure similar to the coding tree used for the CU. In the latest development of video compression technology, quadtree and binary tree (QTBT) partitioned frames are used to divide coding blocks. In the QTBT block structure, CUs may have square or rectangular shapes.

[0091] In this specification, for ease of explanation and understanding, a picture block to be coded in a current coding picture may be referred to as a current block. For example, in encoding, the current block is the block currently being coded, and in decoding, the current block is the block currently being decoded. A decoded picture block in a reference picture that is used to predict a current block is referred to as a reference block. In other words, a reference block is a block that provides a reference signal for the current block, where the reference signal indicates pixel values ​​in the picture block. A block in a reference picture that provides a prediction signal for the current block may be referred to as a prediction block, where the prediction signal indicates pixel values, sample values, or sample signals in the prediction block. For example, an optimal reference block is found after traversing multiple reference blocks, and the optimal reference block provides a prediction for the current block, and this block is referred to as a prediction block.

[0092] In the case of lossless video coding, the original video picture can be reconstructed. This means that the reconstructed video picture has the same quality as the original video picture (assuming there is no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed in a manner such as quantization to reduce the amount of data required to represent the video picture, and the video picture cannot be perfectly reconstructed at the decoder side. This means that the quality of the reconstructed video picture is lower or worse than the quality of the original video picture.

[0093] Some video coding standards, starting with H.261, use "lossy hybrid video coding" (i.e., spatial and temporal prediction in the sample domain are combined with 2D transform coding to apply quantization in the transform domain). Each picture of a video sequence is typically divided into a set of non-overlapping blocks, and coding is typically performed at the block level. In other words, at the encoder side, video is typically processed, i.e., encoded, at the block (or video block) level. For example, a predictive block is generated through spatial (intra) prediction and temporal (inter) prediction, and the predictive block is subtracted from a current block (the block currently being processed or to be processed) to obtain a residual block, which is then transformed and quantized in the transform domain to reduce the amount of data to be transmitted (compressed). At the decoder side, a reverse process, compared to that of the encoder, is performed on the coded or compressed block to reconstruct the current block for representation. Furthermore, the encoder performs processing based on a decoder processing loop, so that the encoder and decoder generate the same prediction (e.g., intra-prediction and inter-prediction) and / or reconstruction for processing, i.e., coding, subsequent blocks.

[0094] The following describes a system architecture applicable to an embodiment of the present invention. FIG. 1A is a block diagram of an example of a video encoding and decoding system according to one embodiment. As shown in FIG. 1A, the video encoding and decoding system 10 may include a source device 12 and a destination device 14. The source device 12 generates encoded video data. Accordingly, the source device 12 may be referred to as a video encoding device. The destination device 14 can decode the encoded video data generated by the source device 12. Accordingly, the destination device 14 may be referred to as a video decoding device. Embodiments of the source device 12, the destination device 14, or both the source device 12 and the destination device 14 may include one or more processors and memories coupled to the processors. Memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other medium that can be used to store desired program code in the form of programmable instructions or computer-accessible data structures, as described herein. Source device 12 and destination device 14 may include a variety of devices, including desktop computers, mobile computing devices, notebook computers (e.g., laptop computers), tablet computers, set-top boxes, telephone handsets (such as "smartphones"), televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, wireless communication devices, etc.

[0095] 1A depicts source device 12 and destination device 14 as separate devices, an embodiment may alternatively include both source device 12 and destination device 14, or the functionality of both source device 12 and destination device 14, i.e., source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such an embodiment, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented by the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0096] The communication connection between source device 12 and destination device 14 may be performed through link 13, and destination device 14 may receive encoded video data from source device 12 through link 13. Link 13 may include one or more media or devices capable of moving encoded video data from source device 12 to destination device 14. In one example, link 13 may include one or more communication media that enable source device 12 to transmit encoded video data directly to destination device 14 in real time. In this example, source device 12 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to destination device 14. The one or more communication media may include wireless communication media and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network. For example, the packet-based network is a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication mediums may include a router, a switch, a base station, or another device that facilitates communication from source device 12 to destination device 14.

[0097] Source device 12 includes an encoder 20. In one embodiment, source device 12 may further include a picture source 16, a picture preprocessor 18, and a communication interface 22. In one embodiment, encoder 20, picture source 16, picture preprocessor 18, and communication interface 22 may be hardware components within source device 12 or may be software programs within source device 12. A separate description follows.

[0098] Picture source 16 may include or be, for example, any type of picture capture device configured to capture real-world pictures and / or any type of device configured to generate pictures or comments (in the case of screen content coding, even a portion of text on the screen is considered part of the picture to be coded), for example, a computer graphics processor configured to generate computer-animated pictures, or any type of device configured to acquire and / or provide real-world pictures or computer-animated pictures (e.g., screen content or virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). Picture source 16 may be a camera lens configured to capture pictures or a memory configured to store pictures. Picture source 16 may further include any type of interface (internal or external) for storing previously captured or generated pictures and / or acquiring or receiving pictures. For example, when picture source 16 is a camera lens, picture source 16 may be a local camera lens or an integrated camera lens integrated into the source device. If picture source 16 is a memory, picture source 16 may be a local memory, such as an integrated memory integrated into the source device. For example, if picture source 16 includes an interface, the interface may be an external interface for receiving pictures from an external video source. For example, the external video source may be an external picture capture device such as a camera lens, an external memory, or an external picture generation device. For example, the external picture generation device may be an external computer graphics processor, computer, or server.The interface may be any type of interface, for example a wired or wireless interface or an optical interface according to any proprietary or standardized interface protocol.

[0099] A picture can be considered a two-dimensional array or matrix of pixels (picture elements). A pixel in the array is sometimes referred to as a sample. The amount of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. Typically, three color components are used to represent color. A picture may be represented as or contain three sample arrays. For example, in an RBG format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is usually represented in a luminance / chrominance format or color space. For example, a picture in YUV format contains a luminance component represented by Y (or sometimes represented by L) and two chrominance components represented by U and V. The luminance (luma) component Y represents brightness or grayscale intensity (e.g., both are the same in a grayscale picture), and the two chrominance (chroma) components U and V represent chrominance or color information components. Thus, a picture in YUV format includes a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(U and V). A picture in RGB format may be transformed or converted to a picture in YUV format, and vice versa. This process is also referred to as color transformation or conversion. If the picture is monochrome, the picture may include only a luminance sample array. In one embodiment, the picture sent by picture source 16 to picture processor may be referred to as original picture data 17.

[0100] The picture preprocessor 18 is configured to receive original picture data 17 and process the original picture data 17 to obtain a preprocessed picture 19 or preprocessed picture data 19. The preprocessing performed by the picture preprocessor 18 may include, for example, cropping, color format conversion (e.g., conversion from RGB format to YUV format), color correction, or noise removal.

[0101] The encoder 20 (also referred to as video encoder 20) is configured to receive pre-processed picture data 19 and process the pre-processed picture data 19 in an associated prediction mode (e.g., a prediction mode in an embodiment of the present invention) to provide encoded picture data 21. (The structure of the encoder 20 is described in more detail below with reference to FIG. 2, FIG. 4, or FIG. 5.) In some embodiments, the encoder 20 may be configured to perform various embodiments described below to implement an encoder-side application of the motion vector (MV) prediction method described in the present invention.

[0102] The communication interface 22 may be configured to receive the encoded picture data 21 and transmit the encoded picture data 21 via the link 13 to the destination device 14 or any other device (e.g., a memory) for storage or direct reconstruction. The other device may be any device for decoding or storage. For example, the communication interface 22 may be configured to encapsulate the encoded picture data 21 in an appropriate format, e.g., data packets, for transmission via the link 13.

[0103] Destination device 14 includes a decoder 30. In one embodiment, destination device 14 may further include a communication interface 28, a picture post-processor 32, and a display device 34, each of which is described below.

[0104] Communications interface 28 may be configured to receive encoded picture data 21 from source device 12 or any other source. The other source may be a storage device, such as an encoded picture data storage device. Communications interface 28 may be configured to transmit or receive encoded picture data 21 over link 13 between source device 12 and destination device 14 or over any type of network. Link 13 may be a direct wired or wireless connection, such as a wired or wireless network, or any combination thereof, or any type of private or public network, or any combination thereof. For example, communications interface 28 may be configured to decapsulate data packets transmitted over communications interface 22 to obtain encoded picture data 21.

[0105] Both communication interface 28 and communication interface 22 may be used as unidirectional or bidirectional communication interfaces and may be configured, for example, to send and receive messages to establish a connection, acknowledge and exchange any other information related to the communication link and / or data transmission (e.g., coded picture data transmission).

[0106] The decoder 30 (also referred to as video decoder 30) is configured to receive the coded picture data 21 and to provide decoded picture data 331 or decoded pictures 331 (the structure of the decoder 30 is described in more detail below with reference to Figure 3, 4 or 5). In some embodiments, the decoder 30 may be configured to perform various embodiments described below in order to implement decoder-side applications of the video processing methods described in the present invention.

[0107] The picture post-processor 32 is configured to perform post-processing on the decoded picture data 331 (also referred to as reconstructed picture data) to obtain post-processed picture data 33. The post-processing performed by the picture post-processor 32 may include color format conversion (e.g., conversion from YUV format to RGB format), color correction, cropping, resampling, or any other processing. The picture post-processor 32 may further be configured to transmit the post-processed picture data 33 to a display device 34.

[0108] A display device 34 is configured to receive the post-processed picture data 33, for example, to display the picture to a user or viewer. The display device 34 may be or include any type of display for presenting the reconstructed picture, e.g., an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCOS), a digital light processor (DLP), or any type of other display.

[0109] FIG. 1A shows the source device 12 and the destination device 14 as separate devices, but in one embodiment, both the source device 12 and the destination device 14, or the functions of both the source device 12 and the destination device 14, i.e., the source device 12 or the corresponding function and the destination device 14 or the corresponding function, may alternatively be included. In such an embodiment, the source device 12 or the corresponding function and the destination device 14 or the corresponding function may be implemented by the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0110] As will be apparent to those skilled in the art based on the description, the presence and (exact) functional partitioning of the different units or functions of the source device 12 and / or destination device 14 shown in FIG. 1A can vary depending on the actual device and application. The source device 12 and the destination device 14 can include any one of a variety of devices, such as any type of handheld or fixed device, e.g., a notebook computer or laptop computer, mobile phone, smartphone, tablet computer, camera, desktop computer, set-top box, television, camera, in-vehicle device, display device, digital media player, video game console, video streaming device (such as a content service server or content delivery server), broadcast receiver device, or broadcast transmitter device, and may use no operating system or any type of operating system.

[0111] Both the encoder 20 and the decoder 30 may be implemented as any one of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If these techniques are implemented in part in software, the device may store instructions for the software on a suitable non-transitory computer-readable storage medium and execute these instructions in hardware by one or more processors to perform the techniques of the present invention. Any one of the above (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors.

[0112] In some cases, the video encoding and decoding system 10 shown in FIG. 1A is merely an example, and the techniques of the present invention are applicable to video coding settings (e.g., video encoding or video decoding) that do not necessarily include data communication between encoding and decoding devices. In other examples, data may be retrieved from local memory, streamed over a network, or the like. A video encoding device may encode data and store the encoded data in memory, and / or a video decoding device may retrieve data from memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but simply encode data to memory and / or retrieve data from memory and decode data.

[0113] FIG. 1B is an explanatory diagram illustrating an example of a video coding system according to an embodiment. As shown in FIG. 1B, according to an embodiment, a video coding system 40 includes the encoder 20 of FIG. 2 and / or the decoder 30 of FIG. 3. The video coding system 40 may implement a combination of various techniques in embodiments of the present invention. In FIG. 1B, the video coding system 40 may include an imaging device 41, the encoder 20, the decoder 30 (and / or a video encoder / decoder implemented by logic circuitry 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0114] 1B, imaging device 41, antenna 42, processing unit 46, logic circuitry 47, encoder 20, decoder 30, processor 43, memory 44, and / or display device 45 may be in communication with one another. As discussed above, video coding system 40 is shown with encoder 20 and decoder 30, but video coding system 40 may include only encoder 20 or only decoder 30 in different examples.

[0115] In some embodiments, antenna 42 may be configured to transmit or receive an encoded bitstream of video data. Further, in some embodiments, display device 45 may be configured to present the video data. In some examples, logic circuitry 47 may be implemented by processing unit 46. Processing unit 46 may include ASIC logic, a graphics processing unit, a general-purpose processor, etc. Video coding system 40 may further include optional processor 43. Similarly, optional processor 43 may include ASIC logic, a graphics processing unit, a general-purpose processor, etc. In some embodiments, logic circuitry 47 may be implemented by hardware, e.g., dedicated video coding hardware, and processor 43 may be implemented by general-purpose software, an operating system, etc. Additionally, memory 44 may be any type of memory, e.g., volatile memory (e.g., static random access memory (SRAM) or dynamic random access memory (DRAM)) or non-volatile memory (e.g., flash memory). In one non-limiting example, memory 44 may be implemented by cache memory. In some embodiments, logic 47 may have access to memory 44 (e.g., for a picture buffer implementation). In other embodiments, logic 47 and / or processing unit 46 may include memory (e.g., a cache) for a picture buffer or similar implementation.

[0116] In some embodiments, encoder 20 implemented by logic circuitry may include a picture buffer (e.g., implemented by processing unit 46 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include encoder 20 implemented by logic circuitry 47 to implement the various modules described with reference to FIG. 2 and / or any other encoder system or subsystem described herein. The logic circuitry may be configured to perform various operations described herein.

[0117] In some embodiments, decoder 30 may be implemented in a similar manner by logic circuitry 47 to implement the various modules described with reference to decoder 30 of FIG. 3 and / or any other decoder system or subsystem described herein. In some embodiments, decoder 30 implemented by logic circuitry may include a picture buffer (e.g., implemented by processing unit 46 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include decoder 30 implemented by logic circuitry 47 to implement the various modules described with reference to FIG. 3 and / or any other decoder system or subsystem described herein.

[0118] In some embodiments, antenna 42 may be configured to receive an encoded bitstream of video data. As described above, the encoded bitstream may include data associated with video frame coding and described herein, indicators, index values, mode selection data, etc., such as data related to coding partitions (e.g., transform coefficients or quantized transform coefficients, optional indicators (as described), and / or data defining the coding partitions). Video coding system 40 may further include a decoder 30 coupled to antenna 42, the decoder 30 configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.

[0119] In this embodiment of the present invention, for the examples described with reference to encoder 20, it should be understood that decoder 30 may be configured to perform the reverse process. With respect to indicating (signaling) syntax elements, decoder 30 may be configured to receive and parse such syntax elements and, correspondingly, decode associated video data. In some embodiments, encoder 20 may entropy encode the syntax elements into the encoded video bitstream. In such examples, decoder 30 may parse the syntax elements and, correspondingly, decode associated video data.

[0120] It should be noted that the method described in the embodiment of the present invention is mainly applied to an inter-prediction process, and the process exists in both the encoder 20 and the decoder 30. The encoder 20 and the decoder 30 in the embodiment of the present invention may be an encoder and a decoder compatible with video standard protocols such as H.263, H.264 / AVC, H.265 / HEVC, H.266 / VVC, MPEG-2, MPEG-4, VP8, VP9, ​​and AV1, or next-generation video standard protocols (e.g., H.267 and AV2).

[0121] 2 is a schematic / conceptual block diagram of an example of an encoder according to an embodiment. In FIG. 2, the encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). According to a hybrid video codec, the encoder 20 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder.

[0122] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form a forward signal path of the encoder 20, while the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the DPB 230, and the prediction processing unit 260 form a backward signal path of the encoder 20. The backward signal path of the encoder corresponds to the signal path of a decoder (e.g., the decoder 30 of FIG. 3).

[0123] For example, encoder 20 receives picture 201 or a picture block 203 of picture 201 via input 202. Picture 201 is, for example, a picture in a sequence of pictures forming a video or a video sequence. Picture block 203 may also be referred to as a current picture block or a picture block to be coded, and picture 201 may also be referred to as a current picture or a picture to be coded (particularly in video coding, to distinguish the current picture from another picture, which may, for example, be a previously coded and / or decoded picture of the same video sequence (i.e., the video sequence that also includes the current picture)).

[0124] In one embodiment, the encoder 20 may include a partitioning unit (not shown in FIG. 2) configured to partition a picture 201 into multiple blocks, such as picture blocks 203. The picture 201 is typically partitioned into multiple non-overlapping blocks. The partitioning unit may be configured to use the same block size and corresponding raster defining the block size for all pictures of the video sequence, or to vary the block size between pictures or subsets or groups of pictures, and to partition each picture into the corresponding blocks.

[0125] In one example, prediction processing unit 260 of encoder 20 may be configured to perform any combination of the above-mentioned partitioning techniques.

[0126] The size of the picture block 203 is smaller than the size of the picture 201, but like the picture 201, the picture block 203 may also be, or may be considered to be, a two-dimensional array or matrix of sample values. In other words, the picture block 203 may include, for example, one sample array (e.g., a luminance array for a black-and-white picture 201), three sample arrays (e.g., one luminance array and two chrominance arrays for a color picture), or any other quantity and / or type of array depending on the color format used. The amount of samples in the horizontal and vertical directions (or axes) of the picture block 203 defines the size of the picture block 203.

[0127] The encoder 20 shown in FIG. 2 is configured to encode a picture 201 block by block, eg, to encode and predict each picture block 203.

[0128] The residual calculation unit 204 is configured to calculate the residual block 205 based on the picture block 203 and the prediction block 265 (the prediction block 265 will be described in further detail below), for example, by subtracting sample values ​​of the prediction block 265 from sample values ​​of the picture block 203 on a sample-by-sample (pixel-by-pixel) basis to obtain the residual block 205 in the sample domain.

[0129] The transform processing unit 206 is configured to apply a transform, for example, a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207, sometimes referred to as transform residual coefficients, represent the residual block 205 in the transform domain.

[0130] The transform processing unit 206 may be configured to apply an integer approximation of a DCT / DST, such as the transform specified in HEVC / H.265. Compared to an orthogonal DCT, such an integer approximation is typically scaled based on a coefficient. To preserve the norm of the residual block processed by the forward transform and the inverse transform, a separate scaling factor is used as part of the transform process. The scaling factor is typically selected based on several constraints, such as the scaling factor being a power of two for shift operations, the bit depth of the transform coefficients, and a trade-off between accuracy and implementation cost. For example, the inverse transform processing unit 212 specifies a particular scaling factor for the inverse transform at the decoder side 30 (and, for example, the inverse transform processing unit 212 specifies a scaling factor for the corresponding inverse transform at the encoder side 20). Correspondingly, the transform processing unit 206 can specify a corresponding scaling factor for the forward transform at the encoder side 20.

[0131] The quantization unit 208 is configured to quantize the transform coefficients 207 (e.g., by performing scalar quantization or vector quantization) to obtain quantized transform coefficients 209. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be truncated to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, in the case of scalar quantization, different scaling may be performed to implement finer or coarser quantization. A smaller quantization step indicates finer quantization, and a larger quantization step indicates coarser quantization. The appropriate quantization step may be indicated by the QP. For example, the quantization parameter may be an index into a predefined set of appropriate quantization steps. For example, a smaller quantization parameter may correspond to finer quantization (e.g., a smaller quantization step), and a larger quantization parameter may correspond to coarser quantization (e.g., a larger quantization step), or vice versa. Quantization may involve division by the quantization step and corresponding quantization or inverse quantization (e.g., performed by the inverse quantization unit 210), or may involve multiplication by the quantization step. According to standards such as HEVC, in one embodiment, the quantization parameter may be used to determine the quantization step. Generally, the quantization step may be calculated based on the quantization parameter through a fixed-point approximation of an equation involving division. To restore the norm of the residual block, other scaling factors may be introduced for quantization and inverse quantization, where the norm may be modified for the scaling used in the fixed-point approximation of the equation for the quantization step and quantization parameter. In an exemplary embodiment, the scaling of the inverse transform may be combined with the scaling of the inverse quantization. Alternatively, customized quantization tables may be used, e.g., signaled from the encoder to the decoder in the bitstream.Quantization is a lossy operation, with larger quantization steps indicating larger losses.

[0132] The inverse quantization unit 210 is configured to perform inverse quantization on the quantized coefficients of the quantization unit 208 to obtain inverse quantized coefficients 211, e.g., applying the inverse of the quantization scheme used by the quantization unit 208 based on or by using the same quantization step as that of the quantization unit 208. The inverse quantized coefficients 211, which may also be referred to as inverse quantized residual coefficients 211, correspond to the transform coefficients 207 but are typically not identical to the transform coefficients due to losses caused by quantization.

[0133] The inverse transform processing unit 212 is configured to perform an inverse transform (e.g., an inverse DCT or an inverse DST) on the transform performed by the transform processing unit 206 to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transformed inverse quantization block 213 or an inverse transformed residual block 213.

[0134] The reconstruction unit 214 (e.g., adder 214) is configured to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265, for example, by adding the sample values ​​of the reconstructed residual block 213 to the sample values ​​of the prediction block 265, to obtain the reconstructed block 215 in the sample domain.

[0135] In one embodiment, a buffer unit (or buffer) 216, e.g., a line buffer 216, is configured to buffer or store the reconstructed blocks 215 and corresponding sample values, e.g., for intra-prediction. In other embodiments, the encoder may use the unfiltered reconstructed blocks and / or corresponding sample values ​​stored in the buffer unit 216 for any type of estimation and / or prediction, e.g., intra-prediction.

[0136] For example, in one embodiment, encoder 20 may use buffer unit 216 not only for storing reconstructed block 215 for intra prediction, but also for loop filter unit 220 (not shown in FIG. 2), and / or may have buffer unit 216 and decoded picture buffer unit 230 form one buffer, for example. In other embodiments, filtered block 221 and / or blocks or samples from decoded picture buffer 230 (blocks or samples not shown in FIG. 2) may be used as input or basis for intra prediction.

[0137] The loop filter unit (or loop filter) 220 is configured to filter the reconstructed block 215 to obtain a filtered block 221 to smooth pixel transitions or improve video quality. The loop filter unit 220 represents one or more loop filters, including a deblocking filter, a sample-adaptive offset (SAO) filter, and another filter, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or collaborative filtering. Although the loop filter unit 220 is illustrated in FIG. 2 as an in-loop filter, in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as a filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed coding block after the loop filter unit 220 performs a filtering operation on the reconstructed coding block.

[0138] For example, in one embodiment, encoder 20 (correspondingly, loop filter unit 220) may directly output loop filter parameters (e.g., sample adaptive offset information), or may output loop filter parameters after entropy encoding unit 270 or any other entropy encoding unit performs entropy encoding, so that, for example, decoder 30 can receive and apply the same loop filter parameters for decoding.

[0139] The DPB 230 may be a reference picture memory that stores reference picture data, which is used by the encoder 20 to encode the video data. The DPB 230 may be formed by any one of a variety of memory devices, such as DRAM (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or another type of memory device. The DPB 230 and the buffer 216 may be provided by the same memory device or by different memory devices. In one embodiment, the DPB 230 is configured to store the filtered block 221. DPB 230 may further be configured to store other previously filtered blocks (e.g., previously reconstructed filtered block 221) of the same current picture or different pictures (e.g., previously reconstructed pictures), e.g., to provide a fully previously reconstructed (i.e., decoded) picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples) for inter-prediction. In one embodiment, if reconstructed block 215 is reconstructed but in-loop filtering is not performed, DPB 230 is configured to store reconstructed block 215.

[0140] The prediction processing unit 260, also referred to as block prediction processing unit 260, is configured to receive or obtain a picture block 203 (e.g., the current picture block 203 of the current picture 201) and reconstructed picture data, e.g., reference samples of the same (or current) picture from the buffer 216 and / or reference picture data 231 of one or more previously decoded pictures from the decoded picture buffer 230, and to process such data for prediction, i.e., to provide a prediction block 265, which may be an inter-prediction block 245 or an intra-prediction block 255.

[0141] The mode selection unit 262 may be configured to select a prediction mode (e.g., intra or inter prediction mode) and / or select the corresponding prediction block 245 or 255 as the prediction block 265 for calculation of the residual block 205 and reconstruction of the reconstructed block 215.

[0142] In one embodiment, mode selection unit 262 may be configured to select a prediction mode (e.g., from prediction modes supported by prediction processing unit 260). The prediction mode provides the best match, in other words, the smallest residual (which means better compression for transmission or storage), or the smallest signaling overhead (which means better compression for transmission or storage), or considers or balances both. Mode selection unit 262 may be configured to determine the prediction mode based on rate-distortion optimization (RDO), i.e., select a prediction mode that provides the smallest rate-distortion optimization, or select a prediction mode whose associated rate-distortion at least meets a prediction mode selection criterion.

[0143] The prediction processing (eg, by prediction processing unit 260) and mode selection (eg, by mode selection unit 262) performed by exemplary encoder 20 are described in more detail below.

[0144] As explained above, the encoder 20 is configured to determine or select an optimal prediction mode from a set of prediction modes (which may be predetermined), which may include, for example, intra-prediction modes and / or inter-prediction modes.

[0145] The set of intra-prediction modes may include 35 different intra-prediction modes, e.g., non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.265, or may include 67 different intra-prediction modes, e.g., non-directional modes such as DC (or average) mode and planar mode, and directional modes such as those defined in H.266, or directional modes such as those defined in the currently developing H.267.

[0146] In one embodiment, the set of inter prediction modes depends on available reference pictures (e.g., at least partially decoded pictures stored in DBP 230) and other inter prediction parameters, such as whether to use the entire reference picture or only a portion of the reference picture, such as whether a search window region around the region of the current block is used to search for the best-matching reference block, and / or whether to apply pixel interpolation, such as half-pixel and / or quarter-pixel interpolation. For example, the set of inter prediction modes may include an advanced motion vector predictor (AMVP) mode and a merge mode. In one embodiment, the set of inter prediction modes may include a control point-based AMVP mode and a control point-based merge mode, which are improved in embodiments of the present invention. In one example, intra prediction unit 254 may be configured to perform any combination of the intra prediction techniques described below.

[0147] In addition to the prediction modes mentioned above, skip mode and / or direct mode may also be applied to embodiments of the present invention.

[0148] The prediction processing unit 260 may be further configured to divide the picture block 203 into smaller block partitions or sub-blocks, for example, by repeatedly using quadtree (QT) partitioning, binary-tree (BT) partitioning, triple-tree (TT) partitioning, or any combination thereof, and perform prediction on each of the block partitions or sub-blocks, for example, where the mode selection includes selecting a tree structure of the divided picture block 203 and selecting a prediction mode to be used for each block partition or sub-block.

[0149] The inter prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 2) and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is configured to receive or obtain a picture block 203 (e.g., a current picture block 203 of the current picture 201) and a decoded picture 231, or at least one or more previously reconstructed blocks (e.g., a reconstructed block of one or more other / different previously decoded pictures 231) for motion estimation. For example, a video sequence may include the current picture and the previously decoded picture 331, or in other words, the current picture and the previously decoded picture 331 can be part of or form a sequence of pictures that form a video sequence.

[0150] For example, the encoder 20 may be configured to select a reference block from multiple reference blocks of the same picture or different pictures among multiple other pictures, and provide an offset (spatial offset) between the position (coordinates X and Y) of the reference picture and / or reference block and the position of the current block as an inter-prediction parameter to a motion estimation unit (not shown in FIG. 2 ). This offset is also referred to as a motion vector (MV).

[0151] The motion compensation unit is configured to obtain inter prediction parameters and perform inter prediction based on or by using the inter prediction parameters to obtain an inter prediction block 245. The motion compensation performed by the motion compensation unit (not shown in FIG. 2) may include extracting or generating a prediction block based on a motion / block vector determined through motion estimation (interpolation may be performed to improve sub-pixel accuracy). Interpolation filtering can generate additional samples from known samples, potentially increasing the amount of candidate prediction blocks that can be used to code the picture block. Upon receiving the motion vector of the PU of the current picture block, the motion compensation unit 246 can locate the prediction block to which the motion vector points within the reference picture list. The motion compensation unit 246 may further generate syntax elements associated with the block and the video slice, where the syntax elements are used by the decoder 30 to decode the picture block of the video slice.

[0152] In one embodiment, the inter prediction unit 244 may send a syntax element to the entropy coding unit 270, where the syntax element includes inter prediction parameters (e.g., an indication of a selection of an inter prediction mode to be used for predicting a current block after traversing multiple inter prediction modes). In one embodiment, if there is only one inter prediction mode, the inter prediction parameters may not be carried in the syntax element. In this case, the decoder side 30 may directly perform decoding in a default prediction mode. It may be understood that the inter prediction unit 244 may be configured to perform any combination of inter prediction techniques.

[0153] The intra prediction unit 254 is configured to obtain (e.g., receive) the picture block 203 (current picture block) and one or more previously reconstructed blocks (e.g., reconstructed neighboring blocks) of the same picture for intra estimation. For example, the encoder 20 may be configured to select an intra prediction mode from multiple intra prediction modes (which may be predetermined).

[0154] In one embodiment, the encoder 20 may be configured to select an intra prediction mode according to an optimization criterion, for example, based on the minimum residual (e.g., the intra prediction mode that provides the predicted block 255 that is most similar to the current picture block 203) or the minimum rate distortion.

[0155] The intra prediction unit 254 is further configured to determine the intra-predicted block 255 based on the intra prediction parameters (e.g., the selected intra prediction mode). In either case, after selecting the intra prediction mode for the block, the intra prediction unit 254 is also configured to provide the intra prediction parameters, i.e., information indicating the selected intra prediction mode for the block, to the entropy coding unit 270. In one example, the intra prediction unit 254 may be configured to perform any combination of intra prediction techniques.

[0156] In one embodiment, the intra prediction unit 254 can send a syntax element to the entropy coding unit 270, where the syntax element includes intra prediction parameters (e.g., an indication of the selection of the intra prediction mode used to predict the current block after traversing multiple intra prediction modes). In one embodiment, if there is only one intra prediction mode, the intra prediction parameters may not be carried in the syntax element. In this case, the decoder side 30 can directly perform decoding in the default prediction mode.

[0157] The entropy encoding unit 270 is configured to apply an entropy encoding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, a context adaptive binary arithmetic coding (CABAC) scheme, a syntax-based context-adaptive binary arithmetic coding (SBAC) scheme, a probability interval partitioning entropy (PIPE) coding, or another entropy encoding method or technique) to the quantized residual coefficients 209, the inter-prediction parameters, the intra-prediction parameters, and / or the loop filter parameters, individually or together (or not at all), to obtain encoded picture data 21, which may be output via an output terminal 272, e.g., in the form of an encoded bitstream 21. The encoded bitstream may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30. Entropy encoding unit 270 may further be configured to entropy encode another syntax element for the current video slice being encoded.

[0158] Other structural modifications of the video encoder 20 may be used to encode the video stream. For example, for some blocks or frames, the non-transform-based encoder 20 may directly quantize the residual signal without the transform processing unit 206. In another embodiment, the encoder 20 may have the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.

[0159] In one embodiment, encoder 20 may be configured to implement the video processing methods described in the following embodiments.

[0160] It should be understood that other structural variations of the video encoder 20 may be used to encode the video stream. For example, for some picture blocks or picture frames, the video encoder 20 may directly quantize the residual signal without processing by the transform processing unit 206, and correspondingly, processing by the inverse transform processing unit 212 is not required. Alternatively, for some picture blocks or picture frames, the video encoder 20 does not generate residual data, and correspondingly, processing by the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 is not required. Alternatively, the video encoder 20 may directly store the reconstructed picture block as a reference block without processing by the filter 220. Alternatively, the quantization unit 208 and the inverse quantization unit 210 in the video encoder 20 may be combined. The loop filter 220 is optional, and in the case of lossless compression coding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 are optional. It should be understood that in different application scenarios, the inter prediction unit 244 and the intra prediction unit 254 may be selectively used.

[0161] Figure 3 is a schematic / conceptual block diagram of an example decoder according to one embodiment. As shown in Figure 3, video decoder 30 is configured to receive coded picture data (e.g., coded bitstream) 21, e.g., coded by encoder 20, to obtain decoded picture 331. In the decoding process, video decoder 30 receives video data from video encoder 20, e.g., a coded video bitstream representing picture blocks of coded video slices and associated syntax elements.

[0162] 3, decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., summer 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. Prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. In some examples, video decoder 30 may perform a decoding path that is generally the reverse of the encoding path described with respect to video encoder 20 shown in FIG.

[0163] Entropy decoding unit 304 is configured to entropy decode coded picture data 21 to obtain, e.g., quantized coefficients 309 and / or decoded coding parameters (not shown in FIG. 3), e.g., any one or all of inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other (decoded) syntax elements. Entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to prediction processing unit 360. Video decoder 30 may receive syntax elements at the video slice level and / or the video block level.

[0164] The inverse quantization unit 310 may have the same function as the inverse quantization unit 110, the inverse transform processing unit 312 may have the same function as the inverse transform processing unit 212, the reconstruction unit 314 may have the same function as the reconstruction unit 214, the buffer 316 may have the same function as the buffer 216, the loop filter 320 may have the same function as the loop filter 220, and the decoded picture buffer 330 may have the same function as the decoded picture buffer 230.

[0165] Prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354. Inter prediction unit 344 may be similar in function to inter prediction unit 244, and intra prediction unit 354 may be similar in function to intra prediction unit 254. Prediction processing unit 360 typically performs block prediction and / or obtains prediction blocks 365 from coded data 21 and is configured to receive or obtain (explicitly or implicitly) prediction-related parameters and / or information regarding a selected prediction mode, e.g., from entropy decoding unit 304.

[0166] When a video slice is coded as an intra-coded (I) slice, intra prediction unit 354 of prediction processing unit 360 is configured to generate a predictive block 365 for a picture block of the current video slice based on an indicated intra prediction mode and data from previously decoded blocks of the current frame or picture. When a video frame is coded as an inter-coded (i.e., B or P) slice, inter prediction unit 344 (e.g., a motion compensation unit) of prediction processing unit 360 is configured to generate a predictive block 365 for a video block of the current video slice based on a motion vector and another syntax element received from entropy decoding unit 304. For inter prediction, the predictive block may be generated from one of the reference pictures in the reference picture list. Video decoder 30 may construct the reference frame lists, i.e., List 0 and List 1, by using a default construction technique and based on the reference pictures stored in DPB 330.

[0167] Prediction processing unit 360 is configured to determine prediction information for video blocks of the current video slice by parsing motion vectors or another syntax element and use the prediction information to generate predictive blocks for the current video block being decoded. In one example of the invention, prediction processing unit 360 determines, by using several received syntax elements to decode the video blocks of the current video slice, a prediction mode (e.g., intra prediction or inter prediction) for encoding the video blocks of the video slice, an inter-prediction slice type (e.g., B slice, P slice, or GPB slice), configuration information for one or more of a reference picture list for the slice, a motion vector for each inter-coded video block of the slice, an inter-prediction status for each inter-coded video block of the slice, and other information. In another embodiment, the syntax elements received by video decoder 30 from the bitstream include syntax elements in one or more of an adaptive parameter set (APS), a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header.

[0168] Inverse quantization unit 310 may be configured to inverse quantize (i.e., de-quantize) the quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 304. The inverse quantization process may include determining a quantization order to be applied and a dequantization order to be applied based on quantization parameters calculated by video encoder 20 for each video block of a video slice.

[0169] Inverse transform processing unit 312 is configured to apply an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients, producing residual blocks in the pixel domain.

[0170] The reconstruction unit 314 (e.g., adder 314) is configured to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365, for example, by adding the sample values ​​of the reconstructed residual block 313 to the sample values ​​of the prediction block 365, to obtain the reconstructed block 315 in the sample domain.

[0171] A loop filter unit 320 (either during the coding loop or after the coding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321 to smooth pixel transitions or improve video quality. In one example, the loop filter unit 320 may be configured to perform any combination of the filtering techniques described below. The loop filter unit 320 represents one or more loop filters including a deblocking filter, a sample-adaptive offset (SAO) filter, and another filter, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or collaborative filtering. While the loop filter unit 320 is shown in FIG. 3 as an in-loop filter, in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0172] The decoded filtered blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

[0173] The decoder 30 is arranged to output the decoded pictures 331, for example via an output 332, for presentation to or viewing by a user.

[0174] Other variations of the video decoder 30 may be used to decode the compressed bitstream. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, for some blocks or frames, the non-transform-based decoder 30 may directly inverse quantize the residual signal without the inverse transform processing unit 312. In another implementation, the video decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0175] In one embodiment, decoder 30 is configured to implement the video processing methods described in the following embodiments.

[0176] It should be understood that other structural variations of the video decoder 30 may be used to decode the encoded video bitstream. For example, the video decoder 30 may generate an output video stream without processing by the filter 320. Alternatively, for some picture blocks or picture frames, the entropy decoding unit 304 of the video decoder 30 does not obtain quantized coefficients through decoding, and correspondingly, the inverse quantization unit 310 and the inverse transform processing unit 312 are not required for processing. The loop filter 320 is optional, and in the case of lossless compression, the inverse quantization unit 310 and the inverse transform processing unit 312 are optional. It should be understood that inter prediction units and intra prediction units may be selectively used in different application scenarios.

[0177] It should be understood that in the encoder 20 and the decoder 30, the processing result of a procedure may be further processed and then output to the next process. For example, after a procedure such as interpolation filtering, motion vector derivation, or loop filtering, an operation such as clip or shift is further performed on the processing result of the corresponding process.

[0178] For example, a motion vector derived based on the motion vectors of the control points of the current picture block and neighboring affine coding blocks may be further processed. This is not limited herein. For example, the value range of a motion vector is limited to a specific bit depth. Assuming that the allowed bit depth of a motion vector is bitDepth, the range of a motion vector is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where the symbol "^" represents exponentiation. When bitDepth is 16, the value range is -32768 to 32767. When bitDepth is 18, the value range is -131072 to 131071. The value range may be limited in one of the following two ways:

[0179] Method 1: The overflowing most significant bits of the motion vector are removed. ux=(vx+2 bitDepth )%2 bitDepth vx=(ux>=2 bitDepth-1 )?(ux-2 bitDepth ):ux uy=(vy+2 bitDepth )%2 bitDepth vy=(uy>=2 bitDepth-1 )?(uy-2 bitDepth ):uy

[0180] For example, if the value of vx is -32769, 32767 is obtained according to the formula above. This value is stored in the computer as two's complement. The two's complement of -32769 is 1, 01111, 1111, 1111, 1111 (17 bits). The computer handles overflow by discarding the most significant bit. Therefore, the value of vx is 0111, 1111, 1111, 1111, or 32767, which is the same result obtained by processing according to the formula.

[0181] Method 2: The motion vector is corrected as shown in the following equation. vx=Clip3(-2 bitDepth-1 ,2bitDepth-1 -1,vx) vy=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vy)

[0182] Clip3 is defined to indicate clipping of z values ​​to the range [x,y].

number

[0183] 4 is a diagram of a structure of a video coding device according to one embodiment. In FIG. 4, video coding device 400 (e.g., a video encoding device or a video decoding device) is applicable to implementing embodiments described herein. In one embodiment, video coding device 400 may be a video decoder (e.g., decoder 30 of FIG. 1A) or a video encoder (e.g., encoder 20 of FIG. 1A). In another embodiment, video coding device 400 may be one or more components of decoder 30 of FIG. 1A or encoder 20 of FIG. 1A.

[0184] Video coding device 400 includes an ingress port 410 and a receiver unit (Rx) 420 configured to receive data, a processor, logic unit, or central processing unit (CPU) 430 configured to process the data, an egress port 450 and a transmitter unit (Tx) 440 configured to transmit the data, and a memory 460 configured to store the data. Video coding device 400 may also include optical-to-electrical and electrical-to-optical (EO) components coupled to ingress port 410, receiver unit 420, transmitter unit 440, and egress port 450 for the egress or ingress of optical or electrical signals.

[0185] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the ingress port 410, the receiver unit 420, the transmitter unit 440, the egress port 450, and the memory 460. The processor 430 includes a coding module 470 (e.g., an encoding module or a decoding module). The coding module 470 implements embodiments disclosed herein and implements the video processing methods provided in the embodiments of the present invention. For example, the coding module 470 implements, processes, or provides various coding operations. Therefore, including the coding module 470 significantly improves the functionality of the video coding device 400 and affects the switching of the video coding device 400 to different states. Alternatively, the coding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0186] Memory 460 may include one or more disks, tape drives, and solid state drives, may be used as overflow data storage devices to store programs when such programs are selectively executed, and to store instructions and data read during program execution. Memory 460 may be volatile and / or non-volatile and may be ROM, RAM, ternary content-addressable memory (TCAM), and / or static random access memory (SRAM).

[0187] FIG. 5 is a simplified block diagram of an apparatus according to one embodiment. In FIG. 5, the apparatus 500 provided in this embodiment can be used as either or both of the source device 12 and the destination device 14 of FIG. 1A. The apparatus 500 can implement the techniques of the present invention. In other words, FIG. 5 is a block diagram of an embodiment of an encoding or decoding device (referred to as a coding device) according to one embodiment. Continuing to refer to FIG. 5, the coding device 500 may include a processor 510, a memory 530, and a bus system 550. The processor is connected to the memory via the bus system, the memory is configured to store instructions, and the processor is configured to execute the instructions stored in the memory. The memory of the coding device stores program code. The processor can invoke the program code stored in the memory to perform the video encoding or decoding methods, particularly various video processing methods, described herein. To avoid repetition, details will not be described again herein.

[0188] In one embodiment, processor 510 may be a central processing unit (CPU), or processor 510 may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0189] The memory 530 may include a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may also be used as the memory 530. The memory 530 may include code and data 531 accessed by the processor 510 via the bus 550. The memory 530 may further include an operating system 533 and an application 535. The application 535 includes at least one program that enables the processor 510 to perform the video encoding or decoding methods described herein (particularly the video processing methods described herein). For example, the application 535 may include Application 1 through Application N and may further include a video encoding or decoding application (referred to as a video coding application) that performs the video encoding or decoding methods described herein.

[0190] In addition to a data bus, the bus system 550 may further include a power bus, a control bus, a status signal bus, etc. However, for clarity of explanation, various buses are illustrated in the figures as the bus system 550.

[0191] In one embodiment, coding device 500 may further include one or more output devices, such as a display 570. In one example, display 570 may be a touch display that combines a display and a touch unit, where the touch unit is capable of operatively sensing touch input. Display 570 may be connected to processor 510 via bus 550.

[0192] A block-based hybrid video coding / compression structure is used in advanced video coding standards such as HEVC, where "hybrid" refers to the combination of prediction, transform, and quantization coding methods to reduce redundancy in video signals. Prediction and transform coding removes redundancy in video signals by removing correlations between the temporal, spatial, and frequency domains. Quantization reduces the precision of the transform coefficients of the prediction residual to reduce the amount of bits required to represent the residual coefficients. Quantization can remove high-frequency details imperceptible to the human eye to reduce the amount of information to be coded. In addition to prediction, transform, and quantization, hybrid video coding structures also include features such as block partitioning, in-loop filtering, and entropy coding.

[0193] A hybrid video encoder first divides an input video frame or video signal into coding tree units (CTUs) of similar size, predicts each CTU, and obtains a prediction residual. The prediction residual is transformed into the frequency domain through a transform module, and then a quantization module performs lossy compression on the frequency domain transform coefficients. One quantized transform coefficient is output to a bitstream through an entropy coding module, and the other transform coefficients undergo steps such as inverse quantization, inverse transform, and loop filtering to obtain a reconstructed value of the current frame, which is used as a reference object for subsequent coding units.

[0194] The hybrid video coding structure described above is typically used to process natural video captured by a camera. In recent years, screen content video, including computer-generated text, graphics, and animation, has become widespread. Screen content video exhibits different colors and other signal characteristics from those of natural video. To meet the requirements for efficient coding of such content, many screen content coding (SCC) tools have been developed, thus significantly improving coding efficiency. HEVC SCC is the first standard to include SCC functions, including coding tools such as Intra Block Copy (IBC) and palette. This can improve the compression efficiency of screen content by 30% to 40%.

[0195] IBC is an important tool for screen content coding in the HEVC SCC standard. It is a block-based prediction technique, and its mechanism is similar to inter-prediction or motion compensation. Motion compensation means that for a current prediction unit (which may be a current picture block, coding block, coding unit, prediction unit, etc.), the encoder uses a motion search algorithm to find the best matching block in a previously coded reference picture as a predictor, and a motion vector (MV) indicates the matching relationship. The difference obtained by subtracting the predictor from the pixel value of the current prediction unit is used as the prediction residual. The prediction residual is processed by modules such as a transform module, a quantization module, and an entropy coding module before being output to a bitstream.

[0196] The main difference between IBC and motion compensation is that the reference samples in IBC are obtained from within the current picture (the reconstructed part), and block vectors (BVs), similar to motion vectors, are used to indicate block matching relationships. IBC can better handle screen content with multiple similar graphics or words within a video frame; that is, the current block can reference a block with a similar graphic within the current picture, and a prediction residual whose pixel value is close to 0 can be obtained. This residual occupies a very small bitrate in the bitstream. The encoding process of the syntax structure and related information contained in the remaining part of IBC is almost the same as that of motion compensation.

[0197] The key to the IBC technology is to obtain coding of block vectors BV. In the related technology, the MV predictive coding method in inter prediction is still used for coding of BV.

[0198] In the case of MV predictive coding method, the motion vector MV is used in the encoder to describe the reference relationship of similar blocks to remove redundancy in the time domain or the space domain.However, since the MV of the current block is usually related to the MV of the neighboring block in the current picture or the neighboring coding picture, the conventional encoder can use the neighboring block as a predictor to reduce the signal amplitude of the MV of the current block.This is the MV predictive coding method.

[0199] During the evolution of video standards, the MV predictive coding method has been significantly optimized. MV prediction in H.261 is simple differential coding. In H.264 / AVC, the MVP is implicitly derived through median prediction. The median value of three spatially neighboring motion vectors is used as the MVP, and information about the predictor does not need to be explicitly transmitted. A list competition scheme is introduced in H.265 / HEVC to collect an MVP list and explicitly transmit the index of the optimal MVP. Here, spatially and temporally neighboring MVs are added to a candidate list, the MV with the best MV cost and SAD (sum of absolute difference) cost is selected, and the MV index is transmitted. There are two inter-prediction modes for MVP derivation in H.265: advanced motion vector predictor (AMVP) mode and merge mode. AMVP mode can handle the flexible block structure in HEVC, and merge mode can reduce segmentation redundancy. In H.266, history-based MV prediction (HMVP) is proposed to add the MV of the most recent coding block to the candidate list.

[0200] In AMVP mode, a candidate motion vector list is first constructed based on the motion information of coding blocks spatially or temporally neighboring the current coding block. Then, the rate-distortion cost of the motion information in the candidate motion vector list is calculated to select an optimal motion vector with the lowest rate-distortion cost from the candidate motion vector list as the motion vector predictor (MVP) for the current coding block. The rate-distortion cost is calculated according to Equation (1), where J denotes the rate-distortion cost (RD Cost), SAD denotes the sum of absolute differences (SAD) between the predicted pixel value obtained by performing motion estimation based on the candidate motion vector predictor and the original pixel value, R denotes the bit rate, and λ denotes the Lagrange multiplier. The encoder side obtains the index value and reference frame index value of the selected MVP in the candidate motion vector list and sends the index value and reference frame index value to the decoder side. In addition, in AMVP mode, a motion search may be further performed in a neighborhood centered on the MVP to obtain the actual motion vector of the current coding block. The encoder side transmits the difference between the MVP and the actual motion vector (eg, the motion vector difference (MVD)) to the decoder side. J = SAD + λR(1)

[0201] In merge mode, a candidate motion information list is first constructed based on the motion information of coding blocks spatially or temporally neighboring the current coding block, then the optimal motion information is determined from the candidate motion information list based on the rate-distortion cost (Equation (1) above) and used as the motion information of the current coding block, and then the index value (represented as merge index) of the position of the optimal motion information in the candidate motion information list is obtained and sent to the decoder side.

[0202] The motion information includes at least one of reference picture information and a motion vector. The reference picture information may include at least one of unidirectional / bidirectional prediction information, a reference picture list, and / or a reference picture index corresponding to the reference picture list. The motion vector is a position offset in the horizontal and vertical directions.

[0203] In the merge candidate list, a history-based motion vector predictor (MVP) (HMVP) candidate set is introduced. HMVP is described as follows:

[0204] In the history-based vector predictor (HMVP) method for inter-coding, an HMVP candidate list is maintained and temporally updated. After a non-affine inter-coding block is decoded, the list is updated by adding the associated new motion information as a new HMVP candidate to the last entry of the list. A FIFO rule for constraints is used to remove entries and add entries to the list. HMVP candidates can be used in merge candidate lists and can also be used in AMVP candidate lists.

[0205] In versatile video coding (VVC), the motion vectors of non-affine inter-coded blocks can be indicated in two ways: advanced motion vector predictor (AMVP) mode or merge mode. In AMVP mode, the difference (MVD) between the actual motion vector and the motion vector predictor (MVP), a reference index, and an MVP index referencing an AMVP candidate list are signaled. In merge mode, a merge index referencing a merge candidate list is signaled, and all motion information associated with the merge candidate is inherited.

[0206] Both the AMVP candidate list and the merge candidate list are from temporally or spatially neighboring coding blocks. Under conventional testing conditions, up to six merge candidates and two AMVP candidates can be added to the motion vector predictor candidate list.

[0207] MV predictive coding techniques such as AMVP and Merge are proposed based on natural pictures, in which the MV of a current picture block is predicted by referring to neighboring MVs.

[0208] However, with the development of multimedia technology, video applications such as online meetings and online education have become widespread. In these applications, text typically accounts for a large portion of the transmitted video content, and viewers tend to focus primarily on the text portion. Compared with natural scenarios and other types of screen content, text content such as electronic documents and PPTs has the distinct characteristic of text characters being complex in texture and dense in placement. Therefore, text compression is important for such scenarios.

[0209] Text generally has a high character repetition rate. This means that MV is more likely to exist between the same characters (e.g., characters such as Chinese characters and Chinese characters, which are not limited herein and may be blocks of any unit size). However, characters may be randomly combined, and the same characters will not appear in groups. Except for some phrases that appear together (e.g., "Zhuang Xiangwang" in FIG. 6), the MVs of characters within a phrase may be consistent, and most neighboring characters in the text have different MVs.

[0210] 6 shows a frame of a picture to be coded. This video frame 104 contains multiple lines of text, and a grid in the picture is used to divide the blocks to be coded in the picture. Three adjacent characters in the frame of the picture ("Qin", "Wang", and "." (Chinese symbol)) are used as an example. The MV of the picture block 101 in "Qin" is MV 1, the MV of the picture block 102 in "Wang" is MV 2, and the MV of the picture block 103 corresponding to "." is MV 3. As can be seen from FIG. 6, the MVs of adjacent characters are different.

[0211] Therefore, when MV prediction is performed on text content in screen display content by using existing MV predictive coding techniques such as AMVP and Merge, the MVs of the neighboring blocks of the current block to be coded are selected as the predictors of the MVs of the current block, which results in a large difference between the predictors of the MVs and the actual MVs, and the accuracy of the MV prediction is low, which results in a large MVD (motion vector differential), a large amount of coding bits for the MVs, and low MV coding efficiency.

[0212] Therefore, the present application provides a video transmission system. As shown in Figure 7, the video transmission system may include an encoder side and a decoder side. The encoder side may include, but is not limited to, an encoding module and an encapsulation module, and the decoder side may include, but is not limited to, a decapsulation module and a decoding module.

[0213] As shown in Figure 7, video transmission mainly includes the following steps: an encoding module performs video encoding on the original frames of the video; an encapsulation module performs video encapsulation on the encoded frames; the encoder side transmits the bitstream obtained through encapsulation to the decoder side through a channel; a decapsulation module on the decoder side decapsulates the bitstream; and a decoding module performs video decoding on the decapsulated bitstream. The purpose of video encoding and decoding is to obtain an encoded file with a high compression rate for transmission, thereby reducing network transmission load. Video encapsulation converts the encoded video file into a file in a format that is easy to transmit. In this application, the encoding module and decoding module are improved so that the encoder side can maintain a positional lookup table for predictive coding of IBC MVs (motion vectors), and the decoder side establishes the same positional lookup table to restore the encoded MVs.

[0214] The position lookup table can include at least one candidate position, where each candidate position is an absolute position of a pixel within the current video frame being coded.

[0215] The absolute positions are defined as follows: Each candidate position is within a video frame and is relative to the sample in the top left corner of the video frame.

[0216] For the definition of absolute position, please refer to the video frame 104 shown in Figure 6 for understanding. In Figure 6, a planar Cartesian coordinate system is established by using the pixel in the upper left corner of the video frame 104 as the origin (0,0), and the candidate positions in the position lookup table of the video frame 104 are coordinate positions relative to the origin (0,0).

[0217] For example, the coordinate position (x1, y1) of block 105 in the word "Qin" that appears multiple times in Figure 6 may be a candidate position in the position lookup table. In this way, the candidate position may be the absolute position of the block in video frame 104.

[0218] 6, the coordinates of the geometric center of the block 105 are used herein as the coordinates of the block 105. In another embodiment, the location of the block, such as the boundary of the block or the geometric center of the block, is used as the coordinate location of the block in this application, and this is not limited thereto.

[0219] Additionally, in this application, the coordinate system established based on the origin of the video frame 104 is not limited to the planar Cartesian coordinate system shown in Figure 6, or may be another coordinate system. However, the absolute position of a pixel in a video frame as defined in this application is still a position relative to a sample in the upper left corner of the video frame (e.g., the origin (0,0) in Figure 6).

[0220] It should be understood that FIG. 6 shows only one candidate position in one position lookup table, and in other embodiments, the candidate positions in the position lookup table of video frame 104 shown in FIG. 6 are not limited to the coordinate position of block 105 shown in FIG. 6.

[0221] For example, the video frame may include text content, and the candidate locations may include absolute locations (ie, locations relative to the sample top left corner of the video frame) of repeating blocks within the video frame.

[0222] In a possible implementation, when a specified identifier is identified in a bitstream, the coder side can encode or decode a video frame by using a position lookup table, and the specified identifier indicates that the video frame corresponding to the bitstream contains text content.

[0223] In this way, the position lookup table can buffer the absolute position of a block in a video frame, for example, the position of a character or picture block that appears repeatedly, and the MV of the same character in text content is usually the same. In this way, the position of a block that is referenced multiple times for IBC in a video frame can be added to the position lookup table and used as an optional prediction item during MV predictive coding to improve the MV prediction efficiency of IBC, thereby improving the coding efficiency of the text content in the video frame while consuming the lowest possible bandwidth / bitrate, and reconstructing high-quality text content for viewing on a terminal, thereby improving the user's viewing experience.

[0224] 8 shows an example of a video processing method in the present application. The method may be applied to the encoder side of FIG. 7 (e.g., encoder 20 of FIG. 2) to encode video frames, and may be applied to the decoder side of FIG. 7 (e.g., decoder 30 of FIG. 2) to decode video frames.

[0225] As shown in FIG. 8, the video processing method may include the following steps.

[0226] S601: Obtain a position lookup table of the current block in the video frame.

[0227] In one embodiment, the encoder and decoder sides can generate position lookup tables in real time during the encoding and decoding process. The position lookup tables generated by the encoder and decoder sides are the same. For details, see Example 1 below.

[0228] In another embodiment, the encoder side can generate the position lookup table before encoding the object to be encoded, and encode the position lookup table into the bitstream for transmission to the decoder side. For details, see Example 2 below.

[0229] For example, the object to be coded may be a picture, a subpicture, a slice, a Largest Coding Unit (LCU), a coding tree unit (CTU), a row, a patch, a coding tree block (CTB), a coding unit (CU), a coding block (CB), a prediction unit (PU), or a region of a particular size (e.g., 8x8 or 16x16), which is not limited herein.

[0230] The position lookup table may include one or more position candidates, each of which includes position coordinates of an absolute position within a video frame.

[0231] In one embodiment, the encoder or decoder side can assign a counter to each position lookup table to keep track of the total amount of candidate positions in the position lookup table.

[0232] In one example, the encoder or decoder side may initialize a counter to 0 before encoding or decoding a picture, subpicture, slice, LCU, CTU, row, or patch.

[0233] In one example, the encoder or decoder may update the counter after encoding or decoding a CTU, CTB, CU, CB, PU, ​​or a region of a particular size (e.g., 8x8 or 16x16).

[0234] In one example, each time the encoder or decoder adds a candidate position to the position lookup table, the value of the counter in the position lookup table is incremented by one.

[0235] In one embodiment, the size of the position lookup table (total amount of candidate positions allowed) and / or the amount of the position lookup table may be fixed or adaptive, and the size of the position lookup table for different objects may be the same or different, which is not limited herein.

[0236] In one example, the different position lookup tables may be of different sizes.

[0237] In one example, the encoder or decoder side may pre-specify the size of the position lookup table and / or the quantity of the position lookup table.

[0238] In one example, the size of the position lookup table and / or the quantity of the position lookup table may be transmitted in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, a patch header, a CTU, a CTB, a CU, a PU, or an area covering multiple CTUs, CTBs, CUs, or PUs, which is not limited in this application.

[0239] Example 1 The encoder side and the decoder side can generate a position lookup table in real time in the encoding and decoding process. The position lookup tables generated by the encoder side and the decoder side are the same, and the principle of the process of generating a position lookup table in real time by the encoder side is the same as that of the decoder side. The encoder side is used as an example for explanation in this specification, and the same is true for the decoder side. The details will not be described again in this specification.

[0240] After encoding a block, the encoder can add the position of this block (i.e., the encoding block) or the reference position of this block (i.e., the encoding block) to the position lookup table, which is initially an empty table.

[0241] The reference position is the absolute position of the block (called "reference block" for short) to which the motion information of the coding block points within the video frame to which the coding block belongs.

[0242] After encoding a current block (e.g., a block to be encoded) in a video frame using the IBC technique, the encoder side can update the position of this block or the reference position of this block in a position lookup table.

[0243] In example a1, when the encoder side encodes a block based on the actual motion information of the block to be encoded (e.g., in AMVP mode), after encoding the block, the encoder side can add the block's position information (e.g., the coordinates (x, y) of the block's absolute position in the current video frame) to a position lookup table.

[0244] In some embodiments, the motion vectors of blocks in the neighborhood cannot be used as motion vectors for blocks in the character because no motion vector can be found in the neighborhood that matches the first occurrence of the character in the video frame. Therefore, the first occurrence of a character among repeated characters in a video frame is typically coded in AMVP mode rather than merge mode.

[0245] In this case, the position of the block coded in the AMVP mode of IBC in a video frame is usually the position where the repeated character first appears in the video frame (abbreviated as "character start position"). The character start position is also usually a position that is frequently used as a reference position. Therefore, only the positions of the blocks coded in the AMVP mode are added to the position lookup table, so that the position where the character with a high frequency first appears in the video frame can be more accurately identified.

[0246] In example b1, when the first character among repeated characters in a video frame is coded in AMVP mode, the MVP determined from the neighboring MVPs is significantly different from the actual MV of the character, and the MVD is large. Therefore, when the character at the character start position in the video frame is coded, a large amount of bits are consumed for the motion information (e.g., MV) of the block of the character.

[0247] Therefore, when the encoder determines that the amount of bits consumed for the motion information (MV) of the block is greater than a first preset threshold after encoding the block, it indicates that the accuracy of the MVP of the coded block is low, and the coded block is likely to be a character block that first appears in the picture. In this case, the encoder can add the position information (x, y) of the coded block in the video frame to a position lookup table.

[0248] In example c1, if the encoder determines that the absolute value of the horizontal component and / or the absolute value of the vertical component of the MVD of the coding block is greater than the second preset threshold, it indicates that a large amount of bits will be consumed when the encoder encodes the MVD of the block, and it may also indicate that the coding block is likely to be a character block that appears for the first time in the video frame. In this case, the encoder can add the position information (x, y) of the coding block in the video frame to the position lookup table.

[0249] In example d1, if the encoder determines that the sum of the absolute value of the horizontal component and the absolute value of the vertical component of the MVD of the coding block is greater than a third preset threshold, it indicates that a large amount of bits will be consumed when the encoder encodes the MVD of the block, and it may also indicate that the coding block is likely to be a character block that appears for the first time in the video frame. In this case, the encoder may add the position information (x, y) of the coding block within the video frame to the position lookup table.

[0250] In example e1, if the encoder side determines that the motion information (eg, MV) of the coding block is derived based on a position lookup table, the position of the block in the video frame may not be added to the position lookup table.

[0251] For example, in Figure 6, the position coordinates of the block of the first character "Qin" in the first row in the frame of the picture shown in Figure 6 are stored in a position lookup table. In this case, the encoder side can predict the motion vectors (MVs) of other words "Qin" appearing in Figure 6 based on the position coordinates of the first character "Qin" stored in the position lookup table, in order to avoid redundancy of characters corresponding to candidate positions in the position lookup table, and the position coordinates of words "Qin" other than the first "Qin" in the frame of the picture do not need to be used to predict the motion vector of the word "Qin" at another position.

[0252] In example f1, the encoder side can further add the reference position of the coding block to the position lookup table.

[0253] The reference position is the absolute position of a block (abbreviated as "reference block") that is referenced to predict the motion information of the current block.

[0254] In example g1, the encoder side updates the position lookup table only when it determines that the size of the coding block is equal to a particular size (eg, 8x8 or 16x16).

[0255] In example h1, when the encoder side determines that the value of the counter in the position lookup table has reached its maximum value, the encoder side may stop updating the position lookup table. Alternatively, when the encoder side determines that the position lookup table has reached its maximum allowable counter value, the encoder side may delete the first candidate position (the candidate position with an index equal to 0) from the position lookup table, subtract 1 from the indexes of all subsequent candidate positions in the position lookup table, and then add the new candidate positions to the position lookup table.

[0256] It should be understood that example a1 to example h1 can be randomly combined to form a new example. In addition, the process of updating the position lookup table in real time in this application is not limited to the examples in the above examples. In addition, the above process of updating the position lookup table in real time can also be applied to the decoder side. After decoding a current block (e.g., a block to be decoded) in a video frame by using the IBC technology, the decoder side can update the position of this block or the reference position of this block in the position lookup table. The specific process and exemplary principles are the same as those described for the encoder side. The details will not be described again in this specification.

[0257] In this manner, the encoder and decoder sides in this application can each maintain a position lookup table. The position lookup table may include at least one candidate position, and each candidate position is an absolute position of a block within a video frame. Although the above description is provided by using an example in which a block within a video frame is an object to be coded, it should be understood that the candidate position in the position lookup table may alternatively be an absolute position of an object to be coded within another unit within the video frame (e.g., CTU, CTB, CU, CB, PU, ​​or a region of a specific size, this is not limited herein).

[0258] In this way, the candidate positions in the position lookup table maintained by each of the encoder and decoder sides of the present application during the encoding or decoding process can be absolute positions that can be frequently used as reference positions within the video frame. In this case, for blocks of frequently occurring characters within the video frame, the candidate positions in the position lookup table are used, resulting in more accurate predicted MVs and smaller MVDs. This improves MV coding efficiency.

[0259] In another example, the candidate locations in the location lookup table may alternatively be locations where the repeated character does not first occur in the video frame.

[0260] In another example, the candidate locations in the location lookup table may alternatively be the locations where the character first appears in the video frame, and the character is the same as the character and is not limited to characters that are in the video frame.

[0261] In Example 1, the encoder and decoder can generate position lookup tables in real time during the encoding and decoding process. The position lookup tables can be updated with the coding of coding decisions. In addition, both the encoder and decoder can maintain the same position lookup table to accurately encode and decode MVs. In addition, according to the method in this embodiment, before encoding and decoding, region division does not need to be performed on the video frame, and a lookup table for each region does not need to be generated. This operating method has lower encoding and decoding complexity.

[0262] Example 2 The encoder side can pre-generate a global position lookup table (global lookup table for short) for at least one region in the current video frame and encode the global position lookup table into a bitstream for transmission to the decoder side.

[0263] In this way, the encoder side and the decoder side can obtain a position lookup table for a current block (e.g., a block to be coded or a block to be decoded) in a current video frame based on the global position lookup table.

[0264] The global lookup table for the region may include at least one candidate position (also referred to herein as a first position), where the first position is an absolute position within the current video frame, and the at least one first position is within the region and is a position relative to a sample in the upper left corner of the current video frame (the current video frame to be coded).

[0265] The following describes the process in step 1 and step 2 in which the encoder side obtains the lookup table for the current block in the current video frame.

[0266] Step 1: Before encoding the object to be encoded, the encoder side can pre-analyze at least one region in the current video frame to obtain a global position lookup table of at least one region in the current video frame, and each region can have a global position lookup table.

[0267] For example, a region may be a picture, a subpicture, a slice, a patch, a CTU, or a row, which is not limited herein.

[0268] In example a2, the encoder side may scan the pixels of each M×M block (e.g., but not limited to, 4×4, 8×8, or 16×16) in the region in raster scan order (e.g., from left to right or from top to bottom). It should be understood that the scan order is not limited in this application. The encoder side may then calculate a hash value for each scanned M×M block in the region and collect statistics regarding the amount of blocks having the same hash value. The encoder side may then sort the amount of blocks obtained through the statistics collection in descending order of block amount, and determine N hash values ​​corresponding to the first N amounts of blocks from the sorting result. The encoder side may obtain the absolute position, i.e., coordinate position, of the first scanned M×M block (not limited to the first M×M block in this specification, but may be the second M×M block, the third M×M block, etc.) corresponding to each of the N hash values ​​in the current video frame to generate a global position lookup table for the region.

[0269] For example, the first pixel in the upper left corner of the current video frame is used as the origin of a planar Cartesian coordinate system to determine the coordinate position of each MxM block within the region.

[0270] In this way, the global position lookup table of the region may include one or more candidate positions (N candidate positions in this specification), and each candidate position may be the position where a block of frequently occurring characters in the region first appears, i.e., the character start position. Of course, if the absolute positions of the M×M blocks corresponding to the N hash values ​​obtained by the encoder side in the above process are not the first scanned M×M blocks corresponding to each hash value, the candidate position in the global position lookup table may alternatively be a position where the block of frequently occurring characters in the region does not first appear (for example, a position where the block appears for the second or third time). This is not limited in this specification.

[0271] Optionally, the candidate positions in the global position lookup table may also include at least two display positions (i.e., absolute positions) of the same character within the region, i.e., the absolute positions of at least two M×M blocks having the same hash value within the region may be used as candidate positions in the global position lookup table.

[0272] In this example, when pre-analyzing a region, the encoder can identify frequently occurring characters in the region by collecting statistics on the hash values ​​of the characters in the region, and generate a global position lookup table for the region based on the absolute positions of the characters in the region.

[0273] It should be understood that when the current video frame is divided into multiple regions and each region is pre-analyzed to obtain a respective global position lookup table, there are no overlapping pixels within the multiple regions.

[0274] In another example b2, the encoder side may further convert the display content of the region into text according to an optical character recognition (OCR) method for the region. The encoder side may then extract the semantics of the characters in the region from the converted text and sort the amount of characters with the same semantics in descending order. The encoder side may then determine N characters (high-frequency characters) corresponding to the amount of the first N characters from the sorting result. Finally, the encoder side obtains the positions at which each of the N characters appears in the region (e.g., the positions at which each of the N characters first appears, which is not limited herein). When determining the order of the positions at which the characters appear, the encoder side may determine the order of the appearance of the same characters in the region in raster scanning order. In this case, the encoder side may use the positions at which the N characters first appear in the region (which are also absolute positions within the current video frame, i.e., positions relative to the sample in the upper left corner of the current video frame) as candidate positions to generate a global position lookup table for the region.

[0275] It should be noted that the manner of obtaining the global position lookup table in the present application is not limited to the implementation forms of Example a2 and Example b2, and may further include another implementation form for determining the absolute position of a high-frequency character in a region in a video frame to obtain the global position lookup table. The implementation form is not limited in the present application.

[0276] In some embodiments, the candidate locations in the global position lookup table are not limited to locations of characters (or picture blocks) that frequently occur within the region. The frequency with which characters or picture blocks corresponding to candidate locations in the global position lookup table occur within the region is not limited by this application.

[0277] In a possible implementation, the encoder side may transmit at least one global position lookup table, each corresponding to at least one region, to the decoder side (e.g., encoding the at least one global position lookup table into a bitstream for transmission).

[0278] For example, before encoding a region, the encoder side may obtain the global position lookup table of the region in advance, and may send both the global position lookup table and the encoded data of the region to the decoder side, which can obtain the global position lookup table of the region by analyzing the bitstream.

[0279] For example, the encoder side may transmit a global position lookup table within a VPS, SPS, PPS, Picture Header, Slice Header, Patch Header, CTU, CTB, CU, PU, ​​or an area covering multiple CTUs, CTBs, CUs, or PUs, which is not limited in this application.

[0280] In one example, the syntax design of the global position lookup table is shown in Table 1.

[0281] [Table 1]

[0282] In Table 1, table_size_minus1 represents the amount of the global position lookup table minus 1. Therefore, the amount of the global position lookup table is TableSize=table_size_minus1+1.

[0283] table_pos_x[i] represents the x-coordinate of the ith position (i.e., candidate position) in the global position lookup table. Therefore, the x-coordinate of the ith position in the global position lookup table is TablePosX[i]=table_pos_x[i].

[0284] table_pos_y[i] represents the y coordinate of the ith position in the global position lookup table. Therefore, the y coordinate of the ith position in the global position lookup table is TablePosY[i]=table_pos_y[i].

[0285] The encoder side can transmit the global position lookup table based on the syntax, and the decoder side can obtain the global position lookup table by parsing the bitstream.

[0286] In another example, a syntax design for a global position lookup table is shown in Table 2.

[0287] [Table 2]

[0288] In Table 2, table_size_minus1 represents the amount of the global position lookup table minus 1. Therefore, the amount of the global position lookup table is TableSize=table_size_minus1+1.

[0289] table_pos_start_x represents the x-coordinate of the 0th position (i.e., candidate position) in the global position lookup table. Therefore, the x-coordinate of the 0th position in the global position lookup table is TablePosX[0]=table_pos_start_x.

[0290] table_pos_start_y represents the y coordinate of the 0th position in the global position lookup table. Therefore, the y coordinate of the 0th position in the global position lookup table is TablePosY[0]=table_pos_start_y.

[0291] table_pos_diff_x[i] represents the difference between the x-coordinate of the ith position and the x-coordinate of the (i-1)th position in the global position lookup table. Therefore, the x-coordinate of the ith position in the global position lookup table is TablePosX[i]=table_pos_x[i-1]+table_pos_diff_x[i].

[0292] table_pos_diff_y[i] represents the difference between the y coordinate of the ith position and the y coordinate of the (i-1)th position in the global position lookup table. Therefore, the y coordinate of the ith position in the global position lookup table is TablePosY[i] = table_pos_y[i-1] + table_pos_diff_y[i].

[0293] The encoder side can transmit the global position lookup table based on the syntax, and the decoder side can obtain the global position lookup table by parsing the bitstream.

[0294] Step 2: The encoder side obtains a position lookup table of a current block (eg, a current block to be coded) in a current video frame based on the global position lookup table of at least one region.

[0295] In example a3, when encoding a current block in a region, the encoder side can use the global position lookup table of the region where the current block is located as the position lookup table of the current block.

[0296] In another example b3, the global position lookup table is a position lookup table generated before the encoder encodes the region, so blocks at candidate positions in the global position lookup table (e.g., absolute positions of characters that frequently appear in the region) are not encoded (or decoded). When the position lookup table of a block is used to predict the motion information of the block, the encoder mainly refers to the motion information of the coded block (the decoder refers to the motion information of the decoded block). Therefore, when encoding the current block in the region, the encoder can delete the absolute positions of uncoded blocks in the current video frame in the global position lookup table of the region to obtain the position lookup table of the current block, i.e., delete the candidate positions of uncoded blocks in the global position lookup table. Therefore, to obtain the motion information of the current block, only the positions of coded blocks in the region can be used as reference positions.

[0297] In the above process, when encoding a region, the encoder side may perform encoding in a preset order (e.g., raster scan order). In this case, the encoder side can determine candidate positions corresponding to uncoded blocks in the global position lookup table of the region based on the absolute position of the current block in the current video frame, and delete some candidate positions (candidate positions of uncoded blocks) in the global position lookup table of the region to obtain the position lookup table of the current block.

[0298] In yet another example c3, the current video frame may be divided into multiple regions, each having a global position lookup table, and different regions may have the same character. In this case, when encoding a block in one of the multiple regions (e.g., region 2), the encoder side may further obtain the global position lookup table (e.g., global position lookup table 1) of the region (e.g., region 1) that was encoded before region 2 was encoded. In this way, when encoding a block in region 2, the encoder side may combine the global position lookup table 1 of region 1 with the global position lookup table 2 of region 2 to obtain the position lookup table of the current block in region 2 (the specific combination method is not limited). In this way, when the position lookup table of a block is used to predict the motion information of the block, the encoder side can use the absolute position of the block having the same character as the block in the position lookup table as the reference position of the current block to accurately predict the motion information of the current block.

[0299] It should be understood that example c3 in step 2 may be combined with example a3 or example b3, and the details will not be described again in this specification.

[0300] The specific implementation process of step 2 will be described by using the encoder side as an example. When the decoder side obtains the position lookup table of the current block (herein referred to as the block to be decoded) in the region by using the global position lookup table of the region obtained by analyzing the bitstream, the implementation principle of the process is the same as that of the implementation process of step 2 for the encoder side. When decoding the current block (herein referred to as the block to be decoded), the decoder side deletes the absolute position of the undecoded block in the global position lookup table of the region including the current block to obtain the global position lookup table of the current block; and / or in combination with example c3, the decoder side may combine the global position lookup table of the decoded region 1 with the global position lookup table of the region 2 where the current block (herein referred to as the block to be decoded) is located in various ways to obtain the global position lookup table of the current block. For details, please refer to the description of the encoder side in step 2. The details will not be described again in this specification.

[0301] In Example 2, the encoder side can pre-generate a global position lookup table for at least one region in the current video frame and encode the global position lookup table into a bitstream for transmission to the decoder side. In this solution, candidate positions that can be used as reference positions with high probability within the video frame, such as the absolute positions of picture blocks or text characters that appear multiple times within the video frame, can be pre-determined before the video frame is encoded and decoded, and the absolute positions are stored in the global position lookup table. Furthermore, the encoder side can transmit the global position lookup table to the decoder side in a bitstream manner. In this way, the candidate positions in the global position lookup table are more accurate, and when predicting the motion information of a block by using the global position lookup table, the encoder side and the decoder side can more easily find the accurate MV of the block.

[0302] S602: Obtain the motion information of the current block based on the position lookup table.

[0303] The encoder side or the decoder side (sometimes collectively referred to as the coder side) may obtain motion information of the current block (a block to be coded or a block to be decoded) by performing S602.

[0304] In this specification, an example in which the motion information is a motion vector (MV) is used for explanation, which is not limited in this specification.

[0305] When executing S602, the coder side can determine at least one target candidate position in the position lookup table based on the position lookup table of the current block, where the position lookup table includes at least one candidate position, and the target candidate position is the candidate position determined in the position lookup table. Then, the coder side can determine at least one motion vector predictor (MVP) of the current block based on the at least one target candidate position. Finally, the coder side can obtain motion information of the current block based on the at least one MVP.

[0306] In example a4, the encoder side may determine a target candidate position in a position lookup table for the current block, where the target candidate position is a candidate position in the position lookup table that corresponds to the MV with the minimum coding cost. The encoder side may convert the coordinates of the target candidate position into the MVP of the current block. In one embodiment, the encoder side may use the MVP as the MV of the current block, and the encoder side may encode the index pos_idx of the target candidate position into a bitstream for transmission to the decoder side to encode the MV of the current block. Alternatively, in another embodiment, the encoder side may perform a motion search in a neighborhood centered on the MVP to obtain the actual motion vector (MV) of the current block, and the encoder side may obtain the difference between the MVP and the MV of the current block (e.g., the motion vector difference (MVD)). Thus, MV = MVP + MVD. In this case, the encoder side may encode the index pos_idx of the target candidate position and the MVD into a bitstream for transmission to the decoder side to encode the MV of the current block.

[0307] Note that in IBC mode, MV is also called block vector (BV), MVP is also called BVP, and MVD is also called BVD. MV, MVP, and MVD are used for the following description. Details will not be described again.

[0308] In example a4, the decoder side may obtain an index pos_idx of a target candidate position from the bitstream, determine a target candidate position corresponding to the index from a position lookup table on the decoder side based on the index, and convert the target candidate position into an MVP of a current block (a block to be decoded). In one embodiment, the decoder side can determine the MVP as the MV of the current block. In another embodiment, the decoder side can further obtain the MVD of the current block from the bitstream through decoding to obtain the MV of the current block according to MV=MVP+MVD.

[0309] In example b4, the encoder side may check a predetermined amount K of candidate positions in the position lookup table of the current block in a predetermined order, and the value of the predetermined amount K may be any integer equal to or greater than 0. This is not specifically limited. The predetermined amount of candidate positions are also referred to as target candidate positions. The encoder side may convert each checked target candidate position (absolute position of the block) into an MVP (relative position of the block), and then the encoder side may construct a candidate motion vector list of the MVPs obtained through the conversion. The candidate motion vector list may include the MVPs obtained by converting the predetermined amount of target candidate positions.

[0310] The policy of the predetermined order is not limited in this application and may be checking from front to back, or checking from back to front, or checking at intervals. It should be understood that the policy for the predetermined order used by the encoder side is the same as the policy used by the decoder side.

[0311] In one embodiment, the candidate motion vector list only includes MVPs obtained by transforming a predetermined amount of target candidate positions. The encoder side may determine the MVP with the lowest rate-distortion cost of the current block in the candidate motion vector list according to the above formula (1), and determine the index of the target candidate position corresponding to the MVP. The subsequent process is the same as in the above example a4. The details will not be described again in this specification.

[0312] In one embodiment, the candidate motion vector list optionally further includes two AMVP candidates in the AMVP candidate list. Alternatively, in another embodiment, the candidate motion vector list optionally further includes merge candidates in the merge candidate list. In this case, the merge candidate list may not only include the MVP obtained by transforming the target candidate position, but also neighboring MVPs in the merge candidate list. The encoder side can determine the MV of the current block based on the candidate motion vector list. In a specific implementation process, the MVP with the lowest coding cost can be selected from the candidate motion vector list according to existing methods. For example, in merge mode, the encoder side may determine the MVP with the lowest rate-distortion cost in the candidate motion vector list according to the above formula (1) and use the MVP as the MV of the current block. In another example, in AMVP mode, the encoder side can determine the MVP with the lowest rate-distortion cost in the candidate motion vector list according to the above formula (1), and perform motion search in the neighborhood centered on the MVP to obtain the actual motion vector MV of the current block, and obtain the MVD of the current block, where MVD is the difference between MV (actual motion vector MV) and the MVP with the lowest rate-distortion cost.It should be understood that the method of determining the MVP with the lowest coding cost in this application is not limited to the above-mentioned rate-distortion cost method, which is not limited in this application.

[0313] In this case, when the candidate motion vector list further includes an AMVP candidate in the AMVP candidate list, or optionally further includes a merge candidate in the merge candidate list, the bitstream of encoded data transmitted by the encoder side to the decoder side may include encoded data of the index of the MVP having the lowest coding cost selected from the candidate motion vector list, optionally further include encoded data of the MVD, and optionally further include encoded data of the values ​​of candidate positions to be checked for a predetermined amount K in the position lookup table.

[0314] In the above example b4, the decoder side can analyze the bitstream to obtain the value of a predetermined amount K (or the value of K is preset on the decoder side) and the index of the MVP with the lowest coding cost. For a current block (a block to be decoded), the target candidate positions of the predetermined amount K in the position lookup table of the current block can be checked in a predetermined order and converted into the MVP (block relative position) of the current block, and then the decoder side can build a candidate motion vector list for the MVP obtained through conversion. The candidate motion vector list can include the MVP obtained by converting the target candidate positions of the predetermined amount.

[0315] In one embodiment, if the coding mode of the current block is Merge mode, the decoder side further obtains a Merge candidate list of the current block to use the MVP in the Merge candidate list as the MVP in the candidate motion vector list, and determines the MVP pointed to by the index from the candidate motion vector list based on the index of the MVP obtained by analyzing the bitstream, and can use the MVP as the MV of the current block.

[0316] In another embodiment, if the coding mode of the current block is AMVP mode, the decoder side can further obtain an AMVP candidate list of the current block, use the MVP in the AMVP candidate list as the MVP in the candidate motion vector list, determine the MVP pointed to by the index from the candidate motion vector list based on the index of the MVP obtained by analyzing the bitstream, obtain the MVD of the current block by analyzing the bitstream, and obtain the MV (MV=MVP+MVD) of the current block based on the MVP and MVD.

[0317] In the above example b4, the amount K of candidate locations to be checked in the location lookup table may be predefined.

[0318] In one example, the quantity K may be related to at least one of the coding information of the blocks (having different list lengths), the block size, the block shape, and the like.

[0319] For example, when the coding mode is the AMVP mode, the value of K may be small, for example, K = 2. When the coding mode is the Merge mode, the value of K may be large, for example, K = 44.

[0320] In another example, when the size of the current block is large, the value of K may be large, and when the size of the current block is small, the value of K may be small.

[0321] In one example, the encoder side may transmit the quantity K within a VPS, SPS, PPS, Picture Header, Slice Header, Patch Header, CTU, CTB, CU, PU, ​​or an area covering multiple CTUs, CTBs, CUs, or PUs, which is not limited in this application.

[0322] The above-mentioned examples a4 and b4 describe two methods of using a position lookup table. When the position lookup table is used based on the position index of the target candidate position as described in example a4, the process will be described in detail below with reference to example 3. The process of predicting the MV of the current block based on the index of the MVP in the candidate motion vector list by checking the quantity K of the candidate positions and building a candidate motion vector list as described in example b4 will be described in detail with reference to example 4.

[0323] Example 3 The process of using the position lookup table to perform MV prediction based on the candidate positions in the position lookup table is as follows.

[0324] FIG. 9a shows an example of a process in which the coder side performs MV prediction of the current block by using a position lookup table.

[0325] The process shown in Figure 9a may be applied to the encoder side or the decoder side. The encoder side and the decoder side are collectively referred to as the coder side. As shown in Figure 9a, this process may include the following steps:

[0326] S700: Table 3 is generated.

[0327] Table 3 may include a location lookup table, and may optionally include AMVP candidates in the AMVP candidate list. The specific contents of Table 3 are shown in Table 3 below.

[0328] When Table 3 is generated, the length of the AMVP candidate list may be extended when the coder side performs coding (encoding or decoding) on ​​the current block. As explained above, the original length of the AMVP candidate list is 2, and the AMVP candidate list may be used to store two AMVP candidates, for example, MVP 0 and MVP 1 in Table 3, where MVP 0 and MVP 1 are MVs of coding blocks that are temporally or spatially neighboring the current block. In this specification, the coder side may extend the length of the AMVP candidate list to 3, where the position lookup table may be represented as the third MVP candidate in the AMVP candidate list. As shown in Table 3, when mvp_idx is 2, it indicates that the position lookup table is used to encode and decode the bitstream.

[0329] It should be understood that the position of the location lookup table in the AMVP candidate list is not limited in this application. Although the mvp_idx corresponding to the location lookup table in Table 3 is 2, in another embodiment, the mvp_idx of the location lookup table in the AMVP candidate list may alternatively be 0 (specifically, the location lookup table is used as the first MVP candidate in the AMVP candidate list, and the original MVP candidate in the AMVP candidate list is placed after the location lookup table) or 1. In other words, the order in which the location lookup table and the original MVP candidate are placed in the AMVP candidate list is not limited in this application.

[0330] It should be understood that the two MVP candidates in the AMVP candidate list are also obtained and updated by the coder side in real time, and the two MVP candidates also change based on different current blocks.

[0331] In a possible implementation, when adding a position lookup table to the AMVP candidate list, the coder side may further convert each candidate position in the position lookup table into an MVP candidate for the current block, and check whether the MVP candidate obtained through the conversion and the original MVP candidate in the AMVP candidate list (e.g., MVP 0 and MVP 1 in Table 3) have the same MVP candidate. If they have the same MVP candidate, the redundant MVP candidate may be deleted. Specifically, the original MVP candidate may be deleted, or the candidate position corresponding to the corresponding redundant MVP candidate in the position lookup table may be deleted. This is not limited in this specification. In this way, the AMVP candidate list with the newly added position lookup table, for example, Table 3, can avoid redundancy of MVP candidates.

[0332] In this embodiment, the coder side can use the position lookup table mode as a new IBC mode.

[0333] [Table 3]

[0334] In Example 3, an extended AMVP candidate list is used as an example to explain the process of using a position lookup table. In another embodiment, the position lookup table can alternatively be used as a merge candidate in the merge candidate list (and the MV of the coding block in the neighborhood of the current block). For example, the merge candidate list may include five merge candidates, and the position lookup table may be used as a newly added merge candidate in the merge candidate list. The arrangement order of the original five merge candidates (the amount is not limited herein) in the merge candidate list and the position lookup table added to the merge candidate list is also not limited herein, and a redundancy check can also be performed on the original five merge candidates and the MVP candidate in the position lookup table. The method is similar. The details will not be described again herein.

[0335] Table 3 shows an example of coder-side use of the position lookup table syntax.

[0336] As shown in Table 3, mvp_idx represents the index of the MVP candidate in the AMVP candidate list, pos_idx represents the index of the optimal candidate position in the position lookup table (i.e., the target candidate position described above) to be used for the current block when the MV of the current block is predicted by using the position lookup table, and Candidate represents the MVP candidate.

[0337] In this embodiment, the position lookup table may include n candidate positions, and the absolute position coordinates of the n candidate positions in the current video frame are represented as Pos0, Pos1, Pos2, ..., and Pos(n-1), respectively.

[0338] For a current block to be encoded, the encoder side can determine the MVP candidate or the optimal candidate position with the lowest coding cost in Table 3 based on the position coordinates of the current block in the current video frame. In one embodiment, if MVP 0 or MVP 1 in Table 3 has the lowest coding cost, the position lookup table does not need to be used this time to predict the MV of the current block, or if the MVP corresponding to one candidate position in Table 3 has the lowest coding cost, the position lookup table needs to be used this time to predict the MV of the current block.

[0339] In embodiment 1, once the coding costs corresponding to all candidates in Table 3 (including two MVP candidates and n candidate positions) are determined, to calculate the coding cost of the MVD of each candidate, an MVP corresponding to each candidate may be constructed, and MVD=MV-MVP may be calculated. MV in this specification is the actual motion vector of the current block obtained by performing motion search in a neighborhood centered on the MVP.

[0340] Each candidate position in Table 3 can be transformed into an MVP candidate for the current block. For example, if the position coordinates of the current block are (x, y) and the coordinates of the candidate position in the position lookup table are (pred_x, pred_y), the coordinates of the MVP candidate obtained through the transformation of the candidate position are (mvp_x, mvp_y), where mvp_x = pred_x - x and mvp_y = pred_y - y.

[0341] When MVP 0 or MVP 1 in Table 3 is used to predict the MV of the current block, the coding cost may include the amount of information bits for coding mvp_idx and the amount of information bits for coding MVD.

[0342] When the position lookup table in Table 3 is used to predict the MV of the current block, the coding cost may include the amount of information bits for coding mvp_idx, the amount of information bits for coding MVD, and the amount of information bits for coding pos_idx.

[0343] When the content of the current block is a character, MVP 0 and MVP 1 in Table 3 are the MVs of the coding blocks adjacent to the current block. Since the MVs of adjacent characters are usually different, the difference between each of MVP 0 and MVP 1 and the actual motion vector MV of the current block is large, and the value of MVD is large. Therefore, when MVP 0 or MVP 1 in Table 3 is used to predict the MV of the current block, the obtained MVD is large and the coding cost of MVD is high.

[0344] When the content of the current block is a character, and the position lookup table of Table 3 is used to predict the MV of the current block, the candidate positions in the position lookup table are the absolute positions of characters that frequently appear in the current video frame. Therefore, if the character of the current block is a high-frequency character, such as "Qin" in Figure 6, the MVP of the current block obtained by using the candidate positions in the position lookup table is closer to the MV of the current block, and the value of the MVD is smaller. Therefore, the coding cost of the MVD is lower than the coding cost required by using MVP 0 or MVP 1.

[0345] Therefore, if the content of the current block is a character, the encoder side can predict the MV of the current block by using a position lookup table, and determine the index pos_idx of the optimal candidate position with the lowest coding cost from the position lookup table.

[0346] When record 3 of Table 3 (i.e., the position lookup table) is used to predict the MV of the current block, the coding cost includes mvp_idx, pos_idx, and MVD. When record 1 or record 2 of Table 3 (i.e., the neighboring MV) is used to predict the MV of the current block, the coding cost includes mvp_idx and MVD. Although the coding cost of the candidate position index pos_idx increases by using the position lookup table, the coding cost of the MVD obtained by using the position lookup table is significantly reduced compared to the coding cost of the MVD obtained by using record 1 or record 2. Therefore, in most cases, when MV prediction is performed on repeated picture blocks or characters in the current video frame, the position lookup table is used for prediction. This improves MV coding efficiency.

[0347] In embodiment 2, once the coding costs corresponding to all candidates in Table 3 (including two MVP candidates and n candidate positions) are determined, the rate-distortion cost corresponding to each candidate may be calculated according to the above-mentioned formula (1), the candidate with the lowest rate-distortion cost is selected, and the MV of the current block is predicted based on the optimal candidate.

[0348] S701: The index of the best candidate position is obtained.

[0349] S702: Obtain the MVP of the current block based on the index of the best candidate position.

[0350] S703: The MV of the current block is obtained based on the MVP of the current block.

[0351] In the above process of S701 to S703, the explanation is provided separately from the encoder side and the decoder side.

[0352] Encoder side The encoder side may select a candidate with the lowest coding cost, for example, a candidate position in a position lookup table, by comparing the coding costs corresponding to different candidates in Table 3 (the coding cost may be determined in embodiment 1 or embodiment 2 or another method, which is not limited in this specification), and predict the MV of the current block based on the candidate. In this case, the encoder side may obtain the index of the MVP candidate in Table 3, i.e., mvp_idx=2 in this specification, and obtain the value of the index pos_idx corresponding to the optimal candidate position (for example, pos_idx=1).

[0353] In addition, the encoder side can further obtain from Table 3 the coordinate Pos1 of the candidate position corresponding to the case where mvp_idx=2 and pos_idx=1.

[0354] In addition, the encoder side may obtain the coordinates (mvp_x, mvp_y) of the MVP of the current block based on the coordinates (pred_x, pred_y) of the optimal candidate position Pos1 and the position coordinates (x, y) of the current block, where mvp_x=pred_x-x and mvp_y=pred_y-y.

[0355] Then, the encoder side performs a motion search in the neighborhood centered on MVP (whose coordinates are (mvp_x, mvp_y)) to obtain the actual motion vector MV of the current block, and obtains MVD=MV-MVP through calculation.

[0356] In this case, the coding information of the MV of the current block sent by the encoder side to the decoder side may include mvp_idx, pos_idx, and the MVD obtained through calculation to encode the MV of the current block.

[0357] In this specification, if mvp_idx is 2, mvp_idx may be used as a designated identifier to indicate that the current block contains text content, and the decoder side may decode the coding information of the current block based on a position lookup table.

[0358] In the above process, if mvp_idx corresponding to the candidate with the lowest coding cost determined in Table 3 is 0 or 1, the encoder side can perform MV prediction using the original MVP 0 or MVP 1 in the AMVP candidate list. In this way, the value of the index mvp_idx of the MVP candidate can be used to indicate that the current block is not encoded by using a position lookup table. If mvp_idx corresponding to the candidate with the lowest coding cost is 2, the encoder side uses the position lookup table to perform MV prediction of the current block, truncated unary coding is used for mvp_idx, and the maximum symbol is changed from 1 to 2. In addition, if mvp_idx=2, the index pos_idx of the optimal candidate position in the position lookup table (i.e., the candidate position used to encode the current block) is additionally transmitted in the MV coding information, where truncated binary code (TBC) may be used for pos_idx, and the maximum symbol is the current length of the position lookup table.

[0359] Decoder side The decoder side also extends the length of the AMVP candidate list at S700 and writes the position lookup table (for specific generation methods, see the relevant explanations in Example 1 or Example 2) into the AMVP candidate list as a new mode, and also obtains Table 3.

[0360] The decoder side receives the encoder side bitstream and can parse the bitstream to obtain the MV coding information of the current block. For example, the coding information may include mvp_idx, pos_idx, and MVD. For example, when decoding the bitstream, the decoder side may first parse mvp_idx. If mvp_idx is 2, the decoder side may determine that the current block is coded using a position lookup table, for example, determine that the current block contains text content. In this case, the decoder side needs to use the position lookup table to decode the current block. In S700, the decoder side can write the position lookup table into the AMVP candidate list to extend the length of the AMVP candidate list for the current block and obtain Table 3. In addition, the decoder side further parses the index pos_idx of the candidate position in the bitstream.

[0361] Then, the decoder side can obtain the coordinates of the optimal candidate position (e.g., the coordinates (pred_x, pred_y) of Pos1) from the position lookup table based on the values ​​of mvp_idx and pos_idx in Table 3 maintained by the decoder side, and convert the coordinates of the optimal candidate position into the MVP (mvp_x, mvp_y) of the current block based on the position coordinates (x, y) of the current block, where mvp_x=pred_x-x and mvp_y=pred_y-y.

[0362] The decoder side may then further parse the bitstream to obtain the MVD of the current block, and decode the MVD to obtain MV=MVD+MVP of the current block to decode the MV of the current block.

[0363] With reference to Example 1 or Example 2, in Example 3, the present application provides another position prediction method different from the original neighborhood MV prediction by establishing a position lookup table. In the present application, the complexity of using the position lookup table can be reduced by limiting the length of the position lookup table. In addition, in a scenario where a repeated picture block or character block in a current video frame is encoded or decoded, the position lookup table is used to perform MV prediction for the current block, and the encoder side can compare more MVP candidates. As a result, the complexity of MV prediction can be reduced, the MVD is smaller, and the coding bit cost of the candidate position index pos_idx is lower. Compared with the prior art, in the present application, the amount of bits for coding MV can be further reduced, and MV coding efficiency can be improved.

[0364] Example 4 The position lookup table is used by converting the K candidate positions in the position lookup table into MVPs in the candidate motion vector list, and the MV of the current block is predicted based on the index of the MVP with the lowest coding cost in the candidate motion vector list.

[0365] FIG. 9b shows an example of a process in which the coder side performs MV prediction of the current block by using a position lookup table.

[0366] The process shown in Figure 9b may be applied to the encoder side or the decoder side. The encoder side and the decoder side are collectively referred to as the coder side. As shown in Figure 9b, this process may include the following steps:

[0367] S801: The coder side generates a candidate motion vector list based on a position lookup table.

[0368] The coder side may select K candidate positions from the position lookup table in a predetermined order (e.g., from front to back or back to front, where the order selection policy is not limited herein). The coder side may convert the coordinates of the K candidate positions into K MVP candidates based on the position coordinates (x, y) of the current block (also referred to as the current coding block). The coder side can then add the K MVP candidates to a candidate motion vector list, which is herein an AMVP candidate list.

[0369] In this way, the candidate motion vector list may not only include MVP candidates in the AMVP candidate list (e.g., MVP 0 and MVP 1 in Table 3), but also K MVP candidates obtained by transforming K candidate positions. In this specification, MV prediction for the current block is implemented in AMVP mode.

[0370] In another embodiment, the K MVP candidates may alternatively be added to the Merge candidate list, in which case MV prediction for the current block is implemented in Merge mode.

[0371] S802: The coder side determines the index of the optimal MVP based on the candidate motion vector list.

[0372] S803: The coder side obtains the MV of the current block based on the index of the optimal MVP.

[0373] In the cases of S802 and S803, the encoder side can determine the MVP candidate with the lowest coding cost in the candidate motion vector list as the optimal MVP. In this specification, the index mvp_idx of the optimal MVP in the candidate motion vector list can be determined.

[0374] When determining the MVP candidate with the lowest coding cost, the encoder may determine the MVP candidate with the lowest rate-distortion cost in the candidate motion vector list as the optimal MVP according to the above formula (1).The specific solution for determining the MVP candidate with the lowest coding cost is not limited in this specification.

[0375] To obtain the actual motion vector MV of the current block, the encoder side can perform a motion search in a neighborhood centered on the optimal MVP and determine MVD=MV-MVP (optimal MVP in this specification).

[0376] The coding information of the MV of the current block sent by the encoder side to the decoder side may include mvp_idx (the index of the optimal MVP), the MVD obtained through calculation, and the value of K to encode the MV of the current block.

[0377] When the encoder side predicts the MV in Merge mode, the optimal MVP may be used as the MV, and the coding information sent by the encoder side to the decoder side does not need to carry the MVD, and the coding information may include mvp_idx (the index of the optimal MVP) and the value of K to encode the MV of the current block.

[0378] For S802 and S803, the decoder side may receive the encoder side bitstream and analyze the bitstream to obtain coding information of the current block. For example, the coding information may include mvp_idx (the index of the optimal MVP), the MVD obtained through calculation, and the value of K.

[0379] The decoder side may select K candidate positions from the decoder side position lookup table (the specific generation process may be the method in Example 1 or Example 2, which is not limited in this specification) in a predetermined order based on the value of K obtained by analyzing the bitstream, and convert the K candidate positions into K MVP candidates based on the position coordinates of the current block.

[0380] Then, when the bitstream is coded in AMVP mode, the decoder side can add the K MVP candidates to the decoder side's AMVP candidate list to build a candidate motion vector list for the current block.

[0381] In another embodiment, when the bitstream is coded in Merge mode, the decoder side can add K MVP candidates to the Merge candidate list on the decoder side to build a candidate motion vector list for the current block.

[0382] Then, the decoder side may obtain, based on the index mvp_idx of the optimal MVP obtained by analyzing the bitstream, the MVP candidate pointed to by the index mvp_idx as the MVP of the current block from the candidate motion vector list.

[0383] Finally, if the bitstream is coded in AMVP mode, the decoder side can parse the bitstream to obtain the MVD of the current block, add the MVD of the current block to the MVP of the current block to obtain the MV of the current block, and decode the MV of the current block.

[0384] Alternatively, in another embodiment, when the bitstream is coded in Merge mode, the decoder side may use the MVP of the current block as the MV of the current block to decode the MV of the current block.

[0385] As shown in FIG. 8, after S602, S603: Coding the current block based on the motion information of the current block.

[0386] The coder side can copy the reconstructed pixels in the current video frame as a predicted block based on the acquired motion information (e.g., MV) of the current block. For the specific process, please refer to the description of the reconstructed video picture, which is not limited in this specification.

[0387] In video coding, MV represents the relative motion of an object. In natural scenarios, the motion vectors of moving objects have high consistency. Therefore, the MV of the block to be coded can be predicted with high accuracy based on the MV of the coding block, and the amount of bits for coding the MV can be effectively reduced.

[0388] It should be understood that the similar contents in the above-mentioned implementations are not described one by one in the implementations, but can be referenced to each other. In addition, the above-mentioned implementations can be combined with each other to form new embodiments. The details will not be described again in this specification.

[0389] According to the same inventive concept as the method, one embodiment of the present invention further provides a video processing device, including: a first acquisition module configured to acquire a lookup table for a current block in a current video frame, the lookup table including at least one candidate location, the candidate location being a location within the current video frame relative to a sample in an upper left corner of the current video frame; a second acquisition module configured to acquire motion information of the current block based on the lookup table; and a coding module configured to code the current block based on the motion information.

[0390] It should be noted that the first acquisition module, the second acquisition module, and the coding module may be used in the video encoding or decoding process on the encoder side or the decoder side. Specifically, on the encoder side, these modules may be used in the encoder 20, and on the decoder side, these modules may be used in the decoder 30.

[0391] It is further noted that for the specific contents of the first acquisition module, the second acquisition module and the coding module, please refer to the above description of the contents including the overview.In order to simplify the present invention, details are not described in this specification.

[0392] In one embodiment, the first acquisition module, the second acquisition module, and the coding module may be implemented by hardware, software, or any combination thereof.

[0393] In the following, application examples of the encoding method and decoding method shown in the above embodiment and a system using these methods will be described.

[0394] 10 is a block diagram of a content delivery system 3100 for implementing a content distribution service. The content delivery system 3100 includes a capture device 3102 and a terminal device 3106, and optionally includes a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the above-mentioned link 13. The communication link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0395] The capture device 3102 can generate data and encode the data according to the encoding method described in the above-described embodiment. Alternatively, the capture device 3102 can deliver the data to a stream server (not shown), which then encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 may include, but is not limited to, a camera, a smartphone or tablet computer, a computer or notebook computer, a video conferencing system, a PDA, an in-vehicle device, or any combination thereof. For example, the capture device 3102 may include the source device 12 described above. If the data includes video, the video encoder 20 included in the capture device 3102 may actually perform the video encoding process. If the data includes audio (i.e., voice), the audio encoder included in the capture device 3102 may actually perform the audio encoding process. In some practical scenarios, the capture device 3102 multiplexes the encoded video and audio data together for distribution. In another practical scenario, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. The capture device 3102 delivers the encoded audio data and the encoded video data separately to the terminal device 3106 .

[0396] In the content supply system 3100, a terminal device 3106 receives and plays encoded data. The terminal device 3106 may be a device capable of receiving and restoring data, such as a smartphone or tablet computer 3108, a computer or notebook computer 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination thereof, or any device capable of decoding encoded data. For example, the terminal device 3106 may include the destination device 14 described above. If the encoded data includes video, the video decoder 30 included in the terminal device prioritizes video decoding. If the encoded data includes audio, the audio decoder included in the terminal device prioritizes audio decoding.

[0397] In the case of a terminal device with a display, such as a smartphone or tablet computer 3108, a computer or notebook computer 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device may provide the decoded data to the display of the terminal device. In the case of a terminal device without a display (e.g., an STB 3116, a video conferencing system 3118, or a video surveillance system 3120), an external display 3126 is connected to receive and display the decoded data.

[0398] When each device in the system performs encoding or decoding, the picture encoding device or picture decoding device shown in the above embodiments may be used.

[0399] 11 is a diagram of an example configuration of the terminal device 3106. When the terminal device 3106 receives a stream from the capture device 3102, a protocol processing unit (protocol forwarding unit) 3202 analyzes the transport protocol of the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG-DASH, Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any combination thereof.

[0400] After processing the stream, the protocol processing unit 3202 generates a stream file. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As mentioned above, in some practical scenarios, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. In this case, the encoded data is sent to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.

[0401] A video elementary stream (ES), an audio ES, and optional subtitles are generated through demultiplexing. The video decoder 3206 includes the video decoder 30 described in the above embodiment, decodes the video ES according to the decoding method shown in the above embodiment to generate video frames, and supplies such data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate audio frames and supplies such data to the synchronization unit 3212. Alternatively, the video frames may be stored in a buffer (not shown in FIG. 11 ) before being supplied to the synchronization unit 3212. Similarly, the audio frames may be stored in a buffer (not shown in FIG. 11 ) before being supplied to the synchronization unit 3212.

[0402] The synchronization unit 3212 synchronizes video and audio frames and provides the video / audio to the video display / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video and audio information. The information may be syntactically coded based on timestamps associated with the presentation of the coded audio and video data and timestamps associated with the transmission of the data stream.

[0403] If the stream contains subtitles, the subtitle decoder 3210 decodes the subtitles, synchronizes them with the video and audio frames, and provides the video / audio / subtitles to the video display / audio display / subtitle display 3216.

[0404] The present invention is not limited to the above-mentioned system, and the picture encoding device or the picture decoding device in the above-mentioned embodiments may be combined into another system, for example, an automobile system.

[0405] Those skilled in the art will understand that the illustrative logical blocks, modules, and algorithm steps disclosed and described herein, and the functions described in combination, can be implemented by hardware, software, firmware, or any combination thereof. When software is used for implementation, the illustrative logical blocks, modules, and steps, and the functions described in combination, can be stored on or transmitted through a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media include computer-readable storage media and may correspond to tangible media, such as data storage media, or may include communication media (e.g., communication protocols) that facilitate the transmission of a computer program from one place to another. In this manner, computer-readable media can typically correspond to non-transitory tangible computer-readable storage media (1) or communication media, such as signals or carrier waves (2). Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the techniques described herein. A computer program product may include computer-readable media.

[0406] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store necessary program code in the form of instructions or data structures and that can be accessed by a computer. Additionally, any connection may be referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or another remote source via coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (e.g., infrared, radio, or microwave), the definition of medium would include the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (e.g., infrared, radio, and microwave). However, it should be understood that computer-readable storage media and data storage media are actually non-transitory tangible storage media and do not include connections, carriers, signals, or other transitory media. As used herein, magnetic disks and optical disks include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs. Magnetic disks typically reproduce data magnetically, while optical disks reproduce data optically using laser light. Combinations of the above are also intended to be included within the scope of computer-readable media.

[0407] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein, may refer to any one of the above structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the exemplary logic blocks, modules, and steps described herein, and the functionality described in combination, may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or may be combined into a composite codec. Furthermore, these techniques may be implemented entirely in one or more circuits or logic elements.

[0408] The techniques of the present invention may be implemented in a number of apparatuses or devices, including a wireless handheld mobile phone, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Although various components, modules, or units are described herein to highlight functional aspects of an apparatus configured to perform the disclosed techniques, the components, modules, or units are not necessarily implemented by different hardware units. In practice, as described above, the various units may be combined into a codec hardware unit in combination with appropriate software and / or firmware, or may be provided by interoperable hardware units (including one or more processors, as described above).

[0409] In the above-mentioned embodiments, the description of the embodiments has different focus, and for the parts not described in detail in this embodiment, please refer to the related descriptions in other embodiments.

[0410] The above description is merely an example of a specific implementation of the present invention and is not intended to limit the protection scope of the present invention. Any variations or replacements conceived by those skilled in the art within the technical scope disclosed herein shall fall within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims. [Explanation of symbols]

[0411] 12 source device, 13 link, 14 destination device, 16 picture source, 17 original picture data, 18 picture preprocessor, 19 preprocessed picture, 20 encoder, 21 coded picture data, 22 communication interface, 28 communication interface, 30 decoder, 32 picture postprocessor, 34 display device, 40 video coding system, 41 imaging device, 42 antenna, 43 processor, 44 memory, 45 display device, 46 processing unit, 47 logic circuit, 101 picture block, 102 picture block, 104 video frame, 105 block, 110 inverse quantization unit, 201 picture, 202 input end, 203 picture block, 204 residual calculation unit, 205 residual block, 206 transform processing unit, 207 transform coefficients, 208 quantization unit, 209 Quantized transform coefficients, 210, inverse quantization unit, 211, inverse quantized coefficients, 212, inverse transform processing unit, 213, reconstructed residual block, inverse quantization block, 214, reconstruction unit, adder, 215, reconstructed block, 216, buffer unit, 220, loop filter unit, 221, filtered block, 230, decoded picture buffer, 231, decoded picture, 244, inter prediction unit, 245, inter prediction block, 246, motion compensation unit, 254, intra prediction unit, 255, intra prediction block, 260, block prediction processing unit, 262, mode selection unit, 265, prediction block, 270, entropy coding unit, 272, output terminal, 304, entropy decoding unit, 309, quantized coefficients, 310, inverse quantization unit, 312, inverse transform processing unit, 313, reconstructed residual block, 314 Reconstruction unit, adder, 315 reconstructed block, 316 buffer, 320 loop filter, 321 filtered block, 330 decoded picture buffer, 331 decoded picture, 332 output terminal, 344 inter prediction unit, 354 intra prediction unit, 360 prediction processing unit, 362 mode selection unit, 365 prediction block, 400Video coding device, 410 ingress port, 420 receiver, 430 processor, 440 transmitter, 450 egress port, 460 memory, 470 encoding / decoding module, coding module, 500 apparatus, 510 processor, 530 memory, 531 data, 533 operating system, 535 application, 550 bus system, 570 display, 3100 content delivery system, 3102 capture device, 3104 communication link, 3106 terminal device, 3108 smartphone or tablet computer, 3110 computer or notebook computer, 3112 network video recorder / digital video recorder, 3114 TV, 3116 set-top box, 3118 video conferencing system, 3120 video surveillance system, 3122 personal digital assistant, 3124 in-vehicle device, 3126 external display, 3202 protocol processing unit, 3204 Demultiplexing unit, 3206 video decoder, 3208 audio decoder, 3210 subtitle decoder, 3212 synchronization unit, 3214 video display / audio display, 3216 video display / audio display / subtitle display

Claims

1. 1. A video processing method, the method comprising: obtaining a lookup table for a current block in a current video frame, the lookup table including at least one candidate location, the candidate location being within the current video frame and being a location relative to a sample in an upper left corner of the current video frame; obtaining motion information of the current block based on the lookup table; coding the current block based on the motion information.

2. The method of claim 1 , wherein the current video frame includes text content.

3. The method of claim 2 , wherein the candidate locations include locations of recurring blocks within the current video frame.

4. The method of claim 1 , wherein the candidate locations include locations of coding blocks in the current video frame and / or locations of reference blocks for the coding blocks.

5. The method of claim 1 , wherein the candidate locations include the location of the first occurrence of the repeated block in the current video frame.

6. obtaining a lookup table for a current block in a current video frame; selecting a candidate position to be added to the lookup table to update the lookup table based on motion information of at least one coding block in the current video frame from a position corresponding to the at least one coding block, the location corresponding to the at least one coding block comprises a location of the coding block in the current video frame or a location of a reference block of the coding block in the current video frame; and obtaining an updated look-up table.

7. obtaining a lookup table for a current block in a current video frame; obtaining a global lookup table for at least one region in the current video frame, the at least one region including a first region, the first region including the current block, the global lookup table including at least one first location, the at least one first location being within the region and being a location relative to the sample in the upper left corner of the current video frame; 6. The method of claim 1, further comprising: obtaining the lookup table for the current block in the current video frame based on the global lookup table for the at least one region, wherein the first position comprises the candidate position in the lookup table.

8. 8. The method of claim 7, wherein the global lookup table for each of the at least one region is a lookup table generated based on the frequency with which the blocks in the region recur before the blocks in the region are coded.

9. obtaining motion information of the current block based on the lookup table, determining at least one target candidate location based on the lookup table, the candidate locations in the lookup table including the target candidate location; determining at least one motion vector predictor MVP for the current block based on the at least one target candidate location; and obtaining the motion information of the current block based on the at least one MVP.

10. When one target candidate location exists, the step of determining at least one target candidate location based on the lookup table includes: The method of claim 9 , further comprising determining, based on the lookup table and an index of the candidate location, the target candidate location that is in the lookup table and corresponds to the index.

11. determining at least one target candidate location based on the lookup table; 10. The method of claim 9, comprising selecting a predetermined amount of candidate locations from the lookup table as target candidate locations, wherein the predetermined amount is an integer greater than zero.

12. The step of obtaining the motion information of the current block based on the at least one MVP includes: constructing an MVP candidate list for the current block based on the at least one MVP; and obtaining the motion information of the current block based on the MVP candidate list.

13. The method according to claim 11 or 12, wherein the predetermined amount is related to at least one of a coding mode of the motion information of the current block, a size of the current block, and a shape of the current block.

14. 1. A video processing device, comprising: a first acquisition module configured to acquire a lookup table for a current block within a current video frame, the lookup table including at least one candidate location, the candidate location being within the current video frame and being a location relative to a sample in an upper left corner of the current video frame; a second obtaining module configured to obtain motion information of the current block based on the lookup table; a coding module configured to code the current block based on the motion information.

15. An encoder configured to perform the method of any one of claims 1 to 13.

16. The encoder of claim 15 , wherein the encoder is further configured to encode the global lookup table into a bitstream of the current video frame.

17. The encoder of claim 15 or 16, wherein the encoder is further configured to encode an index of the target candidate location in the lookup table into the bitstream of the current video frame.

18. The encoder of claim 15 or 16, wherein the encoder is further configured to encode the predetermined amount into the bitstream of the current video frame.

19. A decoder configured to perform the method of any one of claims 1 to 13.

20. 20. The decoder of claim 19, wherein the decoder is further configured to obtain a global lookup table for at least one region in a current video frame from a bitstream.

21. 21. A decoder according to claim 19 or 20, wherein the encoder is further configured to obtain an index of the target candidate position within the bitstream.

22. 21. A decoder according to claim 19 or 20, wherein the decoder is further configured to obtain the predetermined amount in the bitstream.

23. 14. A computer readable storage medium containing a computer program, which when run on a computer or processor enables the computer or processor to carry out the method of any one of claims 1 to 13.

24. 14. A video processing device comprising one or more interface circuits and one or more processors, the interface circuits configured to receive signals from a memory and send the signals to the processor, the signals comprising computer instructions stored in the memory, and wherein, when the processor executes the computer instructions, the processor is configured to perform the method of any one of claims 1 to 13.

25. 14. A computer program product, the computer program product comprising a software program which, when executed by a computer or processor, performs the steps of the method of any one of claims 1 to 13.

26. 14. A computer-readable storage medium storing a video bitstream obtained by executing the method of any one of claims 1 to 13 by one or more processors.

27. 14. A video data encoder, the encoder comprising a memory configured to store the video data in the form of a bitstream, the encoder configured to perform the method of any one of claims 1 to 13.

28. 14. A video data decoder, the decoder comprising a memory configured to store the video data in the form of a bitstream, the decoder being configured to perform a method according to any one of claims 1 to 13.

Images