Video data processing method by decoder and encoder, video data processing device, and program

Abstract

FIELD This disclosure relates to video coding / decoding, and more particularly to enhancing block adaptive weighted prediction. One method includes receiving a video bitstream including a current block and a reference block, where the reference block is used to predict the current block and is identified by a motion vector associated with the current block; receiving from the video bitstream a syntax element indicating a scale factor (α); determining the scale factor (α) based on the value of the syntax element; predicting the current block using the formula p'(x') = α × p(x) + β, where p'(x') is a predicted sample in the current block, p(x) is a co-located reference sample in the reference block corresponding to p'(x'), α is the determined scale factor based on the value of the syntax element, and β is an offset; and reconstructing the current block based on the predicted current block.

Description

[Technical Field]

[0001] This disclosure describes a suite of advanced video / streaming coding / decoding techniques. More specifically, the disclosed techniques include enhancements to Block Adaptive Weighted Prediction (BAWP) and Local Illumination Compensation (LIC) to compensate for local illumination changes. [Background technology]

[0002] Uncompressed digital video can contain a series of pictures and may specify bit rate requirements for storage and data processing, as well as transmission bandwidth in streaming applications. One goal of video coding and decoding can be the reduction of redundancy in the uncompressed input video signal by various compression techniques. Summary of the Invention

[0003] This disclosure describes various embodiments of a method, apparatus, and computer-readable medium that enhances block adaptive weighted prediction (BAWP) to model local illumination compensation (LIC).

[0004] According to one aspect, an embodiment of the present disclosure provides a method for decoding a current block of a current frame in a coded video bitstream, the method including the steps of: receiving a video bitstream including a current block in the current frame and a reference block in a reference frame, where the reference block is used to predict the current block and is identified by a motion vector associated with the current block; receiving from the video bitstream a syntax element indicating a scale factor (α); determining the scale factor (α) based on the value of the syntax element; predicting the current block using the formula p'(x') = α × p(x) + β, where p'(x') is a predicted sample in the current block, p(x) is a co-located reference sample in the reference block corresponding to p'(x'), α is the determined scale factor based on the value of the syntax element, and β is an offset; and reconstructing the current block based on the predicted current block.

[0005] According to another aspect, embodiments of the present disclosure provide an apparatus for processing a current block of a current frame in a coded video bitstream, the apparatus having a memory that stores instructions and a processor in communication with the memory, the processor being configured, when executing the instructions, to cause the apparatus to perform the above-described method for video decoding and / or encoding.

[0006] In another aspect, embodiments of the present disclosure provide a non-transitory computer-readable medium storing instructions that, when executed by a computer for video decoding and / or encoding, cause the computer to perform the above-described method of video decoding and / or encoding.

[0007] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] 1 shows a simplified block diagram schematic of a communication system (100) according to an example embodiment. [Figure 2] 1 shows a simplified block diagram schematic of a communication system (200) according to an example embodiment. [Figure 3] 1 shows a simplified block diagram schematic of a video decoder according to an example embodiment; [Figure 4] 1 shows a simplified block diagram schematic of a video encoder according to an example embodiment; [Figure 5] 1 shows a block diagram of a video encoder according to another example embodiment. [Figure 6] 1 shows a block diagram of a video decoder according to another exemplary embodiment; [Figure 7] 1 illustrates a coding block partitioning scheme according to an exemplary embodiment of the present disclosure. [Figure 8] 10 illustrates another scheme for coding block partitioning according to an exemplary embodiment of the present disclosure. [Figure 9] 10 illustrates another scheme for coding block partitioning according to an exemplary embodiment of the present disclosure. [Figure 10] represents composite motion compensation. [Figure 11] 10A and 10B represent examples of interpolated reference frames for motion compensation. [Figure 12] Examples of templates for the current block and the reference block are shown below. [Figure 13] 1 illustrates an example logic flow for the method of the present disclosure. [Figure 14] 1 illustrates a schematic diagram of a computer system according to an example embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will now be described in detail hereinafter with reference to the accompanying drawings, which form a part hereof and which show, by way of illustration, specific examples of embodiments. It should be noted that the present invention may, however, be embodied in many different forms and that the subject matter covered or claimed should not be construed as being limited to any of the embodiments shown below. It should also be noted that the present invention may be embodied as a method, device, component, or system. Thus, embodiments of the present invention may take the form of, for example, hardware, software, firmware, or any combination thereof.

[0010] Throughout the specification and claims, terms may have meanings full of nuances suggested or implied in context beyond those explicitly stated. The phrases "in one embodiment" or "in some embodiments" used in the specification do not necessarily refer to the same embodiment, and the phrases "in another embodiment" or "in other embodiments" used in the specification do not necessarily refer to different embodiments. Similarly, the phrases "in one implementation" or "in some implementations" used in the specification do not necessarily refer to the same implementation, and the phrases "in another implementation" or "in other implementations" used in the specification do not necessarily refer to different implementations. For example, claimed subject matter is intended to include combinations of example embodiments / implementations, in whole or in part.

[0011] Generally, terms can be understood, at least in part, from their usage in context. For example, terms such as "and," "or," or "and / or" used in the specification can have a variety of meanings that can depend, at least in part, on the context in which such terms are used. Typically, when "or" is used to relate a list, such as A, B, or C, it is intended to mean A, B, and C, which is used here in an inclusive sense, or A or B or C, which is used here in an exclusive sense. Furthermore, as used in the specification, "one or more" or "at least one" may be used to describe any feature, structure, or characteristic in the singular sense, or may be used to describe a combination of features, structures, or characteristics in the plural sense, depending, at least in part, on the context. Similarly, terms such as "a," "an," or "the" may be understood to convey the singular or the plural, again depending, at least in part, on the context. Furthermore, the terms "based on" or "determined by" are not necessarily intended to convey an exclusive set of factors, but instead, depending at least in part on the context, may again allow for the existence of additional factors not necessarily explicitly recited.

[0012] As shown in FIG. 1 , terminal devices may be implemented as servers, personal computers, and smartphones, although the applicability of the underlying principles of the present disclosure may not be so limited. Embodiments of the present disclosure may be implemented in desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated videoconferencing equipment, and / or the like. Network 150 represents any number or type of network that carries coded video data between terminal devices, such as, for example, wireline and / or wireless communication networks. Communication network 150 may exchange data over circuit-switched, packet-switched, and / or other types of channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0013] 2 illustrates the arrangement of a video encoder and a video decoder in a video streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter may be similarly applicable to other video applications, such as, for example, video conferencing, digital TV, broadcasting, gaming, virtual reality, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0014] As shown in Figure 2, a video streaming system may include a video capture subsystem (213) that may include a video source (201), such as a digital camera, that generates a stream of uncompressed video pictures or images (202). By way of example, the stream of video pictures (202) may include samples recorded by the digital camera of the video source (201). The stream of video pictures (202) is represented by a bold line to emphasize its high data volume compared to the encoded video data (204) (or coded video bitstream) and may be processed by an electronic device (220) that includes a video encoder (203) coupled to the video source (201). The video encoder (203) may include hardware, software, or a combination thereof for enabling or implementing aspects of the disclosed subject matter, as described in further detail below. The encoded video data 204 (or encoded video bitstream 204), represented by thin lines to emphasize its lower data volume compared to the uncompressed video picture stream 202, may be stored on the streaming server 205 for future use or sent directly to a downstream video device (not shown). One or more streaming client subsystems, such as the client subsystems 206 and 208 of FIG. 2, can access the streaming server 205 to retrieve copies 207 and 209 of the encoded video data 204. The client subsystem 206 may include a video decoder 210, for example, in an electronic device 230. The video decoder 210 decodes the incoming copy of the encoded video data 207 to generate an outgoing stream of video pictures 211 that is uncompressed and can be rendered on a display 212 (e.g., a display screen) or other rendering device (not shown).

[0015] 3 shows a block diagram of a video decoder (310) of an electronic device (330) according to any embodiment of the present disclosure. The electronic device (330) may include a receiver (331) (e.g., a receiving circuit). The video decoder (310) may be used in place of the video decoder (210) of FIG. 2.

[0016] As shown in FIG. 3, the receiver (331) may receive one or more coded video sequences from the channel (301). To combat network jitter and / or handle playback timing, a buffer memory (315) may be disposed between the receiver (331) and an entropy decoder / parser (320) (hereinafter "parser (320)"). The parser (320) may reconstruct symbols (321) from the coded video sequences. Categories of these symbols include information used to manage the operation of the video decoder (310) and, potentially, information for controlling a rendering device such as a display (312) (e.g., a display screen). The parser (320) may parse / entropy decode the coded video sequences. The parser (320) may extract from the coded video sequences a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder. Subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (320) may also extract information from the coded video sequence, such as transform coefficients (e.g., Fourier transform coefficients), quantization parameter values, motion vectors, etc. Reconstruction of the symbols (321) may require a number of different processing or functional units. The units involved and how they participate may be controlled by subgroup control information parsed by the parser (320) from the coded video sequence.

[0017] The first unit may include a scalar / inverse transform unit (351), which may receive quantized transform coefficients as symbols (321) from the parser (320) along with control information including information such as which transform to use, block size, quantization coefficients / parameters, quantization scaling matrix, etc. The scalar / inverse transform unit (351) may output blocks containing sample values ​​that may be input to an aggregator (355).

[0018] In some cases, the output samples of the scaler / inverse transform unit (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by the intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) may generate blocks of the same size and shape as the block being reconstructed using surrounding block information that has already been reconstructed and stored in the current picture buffer (358). The current picture buffer (358), for example, buffers partially reconstructed and / or fully reconstructed current pictures. In some implementations, the aggregator (355) may add, on a sample-by-sample basis, prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351).

[0019] In other cases, the output samples of the scaler / inverse transform unit (251) may relate to an inter-coded, and potentially motion-compensated, block. In such cases, the motion-compensated prediction unit (353) can access a reference picture memory (357) based on the motion vector to fetch samples used for intra-picture prediction. After motion-compensating the fetched samples according to the block-related symbols (321), the samples can be appended by an aggregator (355) to the output of the scaler / inverse transform unit (351) (the output of unit 351 may be referred to as residual samples or a residual signal) to generate output sample information.

[0020] The output samples of the aggregator (355) can undergo various loop filtering techniques in a loop filter unit (356), which includes several types of loop filters. The output of the loop filter unit (356) can be a sample stream that can be output to a render device (312) or further stored in a reference picture memory (357) for use in future inter-picture prediction.

[0021] 4 shows a block diagram of a video encoder (403) according to an example embodiment of the present disclosure. The video encoder (403) is included in an electronic device (420). The electronic device (420) may further include a transmitter (440) (e.g., a transmitting circuit). The video encoder (403) can be used in place of the video encoder (203) in the example of FIG. 2.

[0022] The video encoder (403) may receive video samples from a video source (401). According to some example embodiments, the video encoder (403) may code and compress pictures of a source video sequence into a coded video sequence (443) in real time or under any other time constraint required by the application. Imposing an appropriate coding rate constitutes one function of the controller (450). In some embodiments, the controller (450) may be operatively coupled to and control other functional units, as described below. Parameters set by the controller (450) may include parameters related to rate control (picture skip, quantizer, lambda value for rate-distortion optimization techniques, etc.), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc.

[0023] In some example embodiments, the video encoder (403) may be configured to operate in a coding loop. The coding loop may include a source coder (430) and a (local) decoder (433) embedded in the video encoder (403). Even though the embedded decoder 433 processes the video stream coded by the source coder 430 without entropy coding (because any compression between the coded video bitstream and the symbols in entropy coding can be lossless in the video compression techniques contemplated by the disclosed subject matter), the decoder (433) reconstructs the symbols to generate sample data in a manner similar to that which a (distant) decoder would also generate. At this point, it can be observed that all decoder technologies other than parsing / entropy decoding, which may only be present in the decoder, may also need to be present in the corresponding encoder in substantially identical functional form. For this reason, the disclosed subject matter may focus on decoder operations related to the decoding portion of the encoder. Therefore, the description of the encoder technology may be simplified, since it is the inverse of the decoder technology, which is described generically. Only certain areas or aspects of the encoder are described in more detail below.

[0024] In some example implementations, during operation, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from the video sequence designated as "reference pictures."

[0025] The local video decoder (433) may decode coded video data for pictures that may be designated as reference pictures. The local video decoder (433) may replicate the decoding process that may be performed by a video decoder on the reference pictures, causing the reconstructed reference pictures to be stored in a reference picture memory (434). In this way, the video encoder (403) may locally store copies of reconstructed reference pictures that have content in common with reconstructed reference pictures that would be obtained by a far-end (distant) video decoder (without transmission errors).

[0026] The predictor (435) may perform a predictive search for the coding engine (432). That is, for a new picture to be coded, the predictor (435) may look for specific metadata, such as reference picture motion vectors, block shapes, or sample data (as candidate reference pixel blocks) from the reference picture memory (434) that may serve as a suitable prediction basis for the new picture.

[0027] The controller (450) may manage the coding operations of the source coder (430), such as setting the parameters and subgroup parameters used to encode the video data.

[0028] The output of all of the above functional units may undergo entropy coding in an entropy coder (445). The transmitter (440) may buffer the coded video sequence produced by the entropy coder (445) to prepare it for transmission over a communication channel (460), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (440) may merge the coded video data from the video coder (403) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0029] The controller (450) may manage the operation of the video encoder (403). During coding, the controller (450) may assign a particular coding picture type to each coded picture, which may affect the coding technique that may be applied to each picture. For example, pictures may often be assigned as one of the following picture types: intra picture (I picture), predicted picture (P picture), bidirectionally predicted picture (B picture), or multi-predicted picture. A source picture may generally be spatially subdivided into multiple sample coding blocks, as described in more detail below.

[0030] 5 shows a diagram of a video encoder (503) according to another example embodiment of this disclosure. The video encoder (503) is configured to receive a processed block (e.g., a predictive block) of sample values ​​in a current video picture in a sequence of video pictures and encode the processed block into a coded picture that is part of the coded video sequence. The example video encoder (503) can be used in place of the video encoder (403) in the example of FIG. 4.

[0031] For example, the video encoder (503) receives a matrix of sample values ​​for a processing block. The video encoder (503) then determines, for example, using rate-distortion optimization (RDO), whether the processing block is best encoded using intra-mode, inter-mode, or bi-predictive mode.

[0032] In the example of Figure 5, the video encoder (503) includes an inter-encoder (530), an intra-encoder (522), a residual calculator (523), a switch (526), ​​a residual encoder (524), a general controller (521), and an entropy encoder (525), coupled in the exemplary arrangement shown in Figure 5.

[0033] The inter-encoder (530) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in previous and subsequent pictures in display order), generate inter-prediction information (e.g., a description of redundancy information, motion vectors, and merge mode information according to an inter-encoding technique), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique.

[0034] The intra encoder (522) is configured to receive samples of a current block (e.g., a processing block), compare the block with previously coded blocks in the same picture, generate transformed and quantized coefficients, and in some cases also generate intra prediction information (e.g., intra prediction direction information according to one or more intra encoding techniques).

[0035] The general controller (521) may be configured to determine general control data and control other components of the video encoder (503) based on the general control data, for example, to determine a prediction mode for a block and provide a control signal to the switch (526) based on the prediction mode.

[0036] The residual calculation unit (523) may be configured to calculate the difference (residual data) between the received block and a prediction result of the block selected from the intra-encoder (522) or inter-encoder (530). The residual encoder (524) may be configured to encode the residual data to generate transform coefficients. The transform coefficients then undergo a quantization process to obtain quantized transform coefficients. In various exemplary embodiments, the video encoder (503) also includes a residual decoder (528). The residual decoder (528) is configured to perform an inverse transform and generate decoded residual data. The entropy encoder (525) may be configured to format a bitstream to include the encoded block and perform entropy coding.

[0037] 6 shows a diagram of an example video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is configured to receive coded pictures that are part of a coded video sequence and decode the coded pictures to generate reconstructed pictures. By way of example, the video decoder (610) can be used in place of the video decoder (410) in the example of FIG. 4.

[0038] In the example of Figure 6, the video decoder (610) includes an entropy decoder (671), an inter-decoder (680), a residual decoder (673), a reconstruction module (674), and an intra-decoder (672), coupled as shown in the example arrangement of Figure 6.

[0039] The entropy decoder (671) may be configured to reconstruct, from a coded picture, specific symbols representing syntax elements of which the coded picture is composed. The inter decoder (680) may be configured to receive inter prediction information and generate inter prediction results based on the inter prediction information. The intra decoder (672) may be configured to receive intra prediction information and generate prediction results based on the intra prediction information. The residual decoder (673) may be configured to perform inverse quantization to extract inverse quantized transform coefficients and process the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The reconstruction module (674) may be configured to combine, in the spatial domain, the residual output by the residual decoder (673) and the prediction results (possibly output by the inter prediction module or the intra prediction module) to form reconstructed blocks that form portions of the reconstructed picture as part of the reconstructed video.

[0040] It should be noted that the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using any suitable technology. In some example embodiments, the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using one or more integrated circuits. In other embodiments, the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using one or more processors executing software instructions.

[0041] With regard to block partitioning for coding and decoding, typical partitioning begins with a base block and may follow a predefined set of rules, a specific pattern, a partition tree, or any partition structure or scheme. Partitioning can be hierarchical and recursive. After dividing or partitioning the base block according to any of the example partitioning procedures described below or other procedures, a final set of partitions or coding blocks can be obtained. Each of these partitions can be at one of various partitioning levels within the partitioning hierarchy and can have various shapes. Each partition may be referred to as a coding block (CB). For various example partitioning implementations described further below, each resulting CB can be of any allowed size and partitioning level. Such partitions are referred to as coding blocks because they form the units within which basic coding / decoding decisions are made, coding / decoding parameters are optimized, determined, and signaled in the encoded video bitstream. The highest or deepest level among the final partitions represents the depth of the coding block partitioning structure of the tree. A coding block may be a luma coding block or a chroma coding block. The CB tree structure for each color may be referred to as a coding block tree (CBT). The coding blocks for all color channels may be collectively referred to as a coding unit (CU). The hierarchical structure for all color channels may be collectively referred to as a coding tree unit (CTU). The partitioning pattern or structure of various color channels within a CTU may or may not be the same.

[0042] In some implementations, the partition tree scheme or structure used for the luma channel and the chroma channel need not be the same. In other words, the luma channel and the chroma channel may have separate coding tree structures or patterns. Furthermore, whether the luma channel and the chroma channel use the same or different coding partition tree structures, and the actual coding partition tree structure used, may depend on whether the slice being coded is a P slice, a B slice, or an I slice. For example, for an I slice, the chroma channel and the luma channel may have separate coding partition tree structures or coding partition tree structure modes, while for a P slice or a B slice, the luma channel and the chroma channel may share the same coding partition tree scheme. When separate coding partition tree structures or modes are applied, the luma channel may be partitioned into CBs by one coding partition tree structure, and the chroma channel may be partitioned into chroma CBs by another coding partition tree structure.

[0043] FIG. 7 shows examples of 10 predefined partitioning structures / patterns that enable recursive partitioning to form a partitioning tree. A root block can start at a predefined level (e.g., from a base block at the 128x128 or 64x64 level). The example partitioning structure of FIG. 7 includes various 2:1 / 1:2 and 4:1 / 1:4 rectangular partitions. In some exemplary implementations, none of the rectangular partitions in FIG. 7 can be further subdivided. A coding tree depth can be further defined to indicate the partition depth from the root node or root block. For example, the coding tree depth of the root node or root block can be set to 0, and after the root block is further divided once according to FIG. 7, the coding tree depth is increased by 1. In some implementations, only 710 full square partitions can be allowed for recursive partitioning to the next level of the partitioning tree according to the pattern of FIG. 7.

[0044] In some example implementations of coding tree partitioning, a quadtree structure may be used. Such quadtree division may be applied hierarchically and recursively to any square partition. Whether a base block or intermediate block or partition is further quadtree divided may be adapted to various local characteristics of the base block or intermediate block / partition.

[0045] In yet another example, a ternary tree partitioning scheme may be used to partition the base block or any intermediate blocks, as shown in FIG. 8. The ternary tree pattern may be implemented vertically, as shown at 802, or horizontally, as shown at 804. While the exemplary division ratio in FIG. 8 is shown as 1:2:1, other ratios may be predefined. In some implementations, two or more different ratios may be predefined. In some implementations, the width and height of the partitions in the exemplary ternary tree are always powers of two to avoid further transformations.

[0046] The above partitioning schemes can be combined in any manner at different partitioning levels. As an example, the above-described quadtree partitioning scheme and binary tree partitioning scheme can be combined to partition a base block into a quadtree-binary-tree (QTBT) structure. In such a scheme, the base block or intermediate blocks / partitions can be quadtree- or binary tree-partitioned according to a set of predefined conditions, if specified. A specific example is shown in FIG. 9, in which the base block is first quadtree-partitioned into four partitions indicated by 902, 904, 906, and 908. Each of the resulting partitions is then quadtree-partitioned into four further partitions (e.g., 908) or binary tree-partitioned into two further partitions at the next level (e.g., 902 if horizontal or 906 if vertical, both symmetric), or not partitioned (e.g., 904). Binary tree or quad tree partitioning may be recursively permitted in the case of square partitions, as shown by the overall example partition pattern in 910 and the corresponding tree structure / representation in 920, where solid lines represent quad tree partitioning and dashed lines represent binary tree partitioning. A flag may be used in the case of each binary tree partition node (non-leaf binary tree partition) to indicate whether the binary tree partitioning is horizontal or vertical. For example, as shown in 920, a "0" can represent a horizontal binary tree partition and a "1" can represent a vertical binary tree partition, consistent with the partitioning structure in 910. In the case of quad tree partitioning, it is not necessary to indicate the partition type, since quad tree partitioning always divides a block or partition both horizontally and vertically to generate four sub-blocks / partitions of equal size. In some implementations, a flag "1" can represent a horizontal binary tree partition and a flag "0" can represent a vertical binary tree partition.

[0047] In some example implementations of QTBT, the quadtree and binary tree splitting rule sets can be expressed by the following predefined parameters and their associated corresponding functions: - CTU size: quadtree root node size (base block size) - MinQTSize: The minimum allowed quadtree leaf node size - MaxBTSize: Maximum allowed binary tree root node size - MaxBTDepth: Maximum allowed binary tree depth - MinBTSize: The minimum allowed binary tree leaf node size

[0048] In some example implementations of the QTBT partitioning structure, the CTU size may be set as 128x128 luma samples and two corresponding 64x64 blocks of chroma samples (if example chroma subsampling is considered and used), MinQTSize may be set as 16x16, MaxBTSize may be set as 64x64, MinBTSize (for both width and height) may be set as 4x4, and MaxBTDepth may be set as 4. Quad-tree partitioning can have sizes from its minimum allowed size of 16x16 (i.e., MinQTSize) to 128x128 (i.e., CTU size). If a node is 128x128, it will not be split by the binary tree first because its size exceeds MaxBTSize. Otherwise, nodes that do not exceed MaxBTSize may be partitioned by the binary tree. In the example of FIG. 9, the base block is 128x128. The base block can only be quadtree partitioned according to a predefined set of rules. The base block has a partitioning depth of 0. Each of the resulting four partitions is 64x64, does not exceed MaxBTSize, and can be further quadtree or binary tree partitioned at level 1. The process continues. When the binary tree depth reaches MaxBTDepth (i.e., 4), no further divisions are possible. If a binary tree node has a width equal to MinBTSize (i.e., 4), no further horizontal divisions are possible. Similarly, if a binary tree node has a height equal to MinBTSize, no further vertical divisions are possible.

[0049] In some example implementations, the above QTBT scheme can be configured to support the flexibility of luma and chroma having the same or different QTBT structures. For example, for P slices and B slices, the luma CTB and chroma CTB in one CTU may share the same QTBT structure. However, for I slices, the luma CTB may be partitioned into CBs by a QTBT structure, and the chroma CTB may be partitioned into chroma CBs by a different QTBT structure. This means that a CU can be used to refer to different color channels within an I slice; for example, an I slice can consist of a coding block for a luma component or a coding block for two chroma components, and a CU in a P slice or B slice can consist of coding blocks for all three color components.

[0050] The various CB partitioning schemes described above and the further partitioning of CB into PBs may be combined in any way. The following specific examples are given as non-limiting examples:

[0051] Inter prediction may be performed, for example, in a single reference mode or a mixed reference mode. In some implementations, a skip flag may first be included in the bitstream of the current block (or at a higher level) to indicate whether the current block is inter-coded and should not be skipped. If the current block is inter-coded, another flag may be included in the bitstream to indicate whether a single reference mode or a mixed reference mode is used for predicting the current block. In the single reference mode, one reference block may be used to generate a prediction block for the current block. In the mixed reference mode, two or more reference blocks may be used to generate a prediction block, for example, by weighted averaging. One reference block or two or more reference blocks may be identified using one or more reference frame indexes and one or more corresponding motion vectors that indicate a shift between the reference block and the current block in position relative to the frame (e.g., horizontal and vertical pixels). For example, an inter-predicted block for a current block may be generated from a single reference block identified by one motion vector in a reference frame as the predictive block in a single reference mode, while in a mixed reference mode, the predictive block may be generated by a weighted average of two reference blocks in two reference frames indicated by two reference frame indices and two corresponding motion vectors. The motion vectors may be coded and included in the bitstream in various ways.

[0052] In some example implementations, one or more reference picture lists, including identification of short-term and long-term reference frames for inter prediction, may be formed based on information in a Reference Picture Set (RPS). For example, for unidirectional inter prediction, a single picture reference list may be formed and denoted as the L0 reference (or reference list 0), while for bidirectional inter prediction, two picture reference lists may be formed and denoted as the L0 (or reference list 0) and the L1 (or reference list 1), for each of the two prediction directions. The reference frames included in the L0 and L1 lists may be ordered in various predetermined manners. The lengths of the L0 and L1 lists may be signaled in the video bitstream. Unidirectional inter prediction may be in a single reference mode or in a mixed reference mode, where multiple references for generating a prediction block by weighted averaging in a mixed prediction mode are on the same side of the frame in which the block to be predicted is located. Bidirectional inter prediction can only be a mixed mode in that bidirectional inter prediction involves at least two reference blocks.

[0053] In some implementations, a merge mode (MM) for inter prediction may be implemented. Generally, in merge mode, one or more motion vectors in single reference prediction or mixed reference prediction for a current PB may be derived from other motion vectors rather than being calculated and signaled independently. For example, in an encoding system, a current motion vector for a current PB may be represented by a difference between the current motion vector and one or more other already-encoded motion vectors (referred to as reference motion vectors). Such a difference, rather than the entire current motion vector, may be encoded and included in the bitstream and linked to the reference motion vector. Correspondingly, in a decoding system, a motion vector corresponding to the current PB may be derived based on a decoded motion vector difference and a decoded reference motion vector linked thereto. As a specific form of general merge mode (MM) inter prediction, such inter prediction based on a motion vector difference may be referred to as merge mode with motion vector difference (MMVD). Thus, MM in general, or MMVD in particular, may be implemented to exploit correlation between motion vectors associated with different PBs to improve coding efficiency. For example, adjacent PBs have similar motion vectors, resulting in smaller MVDs and more efficient coding. As another example, motion vectors may be temporally correlated (between frames) for blocks that are similarly positioned / located in space.

[0054] In some example implementations of MMVD, a list of reference motion vector (RMV) or MV predictor candidates for motion vector prediction may be formed for a block being predicted. The RMV candidate list may include a predetermined number (e.g., two) of MV predictor candidate blocks having motion vectors that can be used to predict the current motion vector. The RMV candidate blocks may include blocks selected from neighboring blocks within the same frame and / or temporal block (e.g., blocks at the same location in a frame before or after the current frame). These choices correspond to blocks at spatial or temporal locations relative to the current block that may have similar or the same motion vector as the current block. The size of the MV predictor candidate list may be predetermined. For example, the list may include two or more candidates. To be on the RMV candidate list, a candidate block must, for example, have the same reference frame(s) as the current block, must exist (e.g., a boundary check must be performed when the current block is near the edge of a frame), and must have already been encoded during the encoding process and / or decoded during the decoding process. In some implementations, the list of merge candidates may first be populated with spatially adjacent blocks (scanned in a specific predefined order) if they are available and satisfy the above conditions, and then with temporal blocks if there is still free space in the list. Neighboring RMV candidate blocks may be selected, for example, from the blocks to the left and above the current block. The list of RMV predictor candidates may be dynamically formed at various levels (sequence, picture, frame, slice, sub-block, etc.) as a Dynamic Reference List (DRL).

[0055] In some implementations, the actual MV predictor candidate used as the reference motion vector to predict the motion vector of the current block may be signaled. When the RMV candidate list includes two candidates, a one-bit flag called a merge candidate flag may be used to indicate the selection of the reference merge candidate. When the current block is predicted in decoding mode, each of the multiple motion vectors predicted using the MV predictor may be associated with a reference motion vector from the merge candidate list. The encoder may determine which RMV candidate more accurately predicts the MV of the current coding block and signal the selection as an index to the DRL.

[0056] In some example implementations of MMVD, after an RMV candidate is selected and used as a base motion vector predictor for predicting a motion vector, a motion vector differential (MVD or delta MV representing the difference between the motion vector to be predicted and a reference candidate motion vector) may be calculated in the encoding system. Such MVD may include information representing the magnitude and direction of the MV differential. Both the magnitude and direction may be signaled in the bitstream in various ways.

[0057] In some example implementations of MMVD, a distance index may be used to specify the magnitude information of the motion vector differential and to indicate one of a set of predefined offsets that represent a predefined motion vector differential from a starting point (reference motion vector). The MV offset according to the signaled index may then be added to either the horizontal or vertical component of the starting (reference) motion vector. An example of a predefined relationship between the distance index and the predefined offsets is set forth in Table 1. [Table 1]

[0058] In some example implementations of MMVD, a direction index may be further signaled and used to indicate the direction of the MVD relative to the reference motion vector. In some implementations, the direction may be limited to either the horizontal or vertical direction. An example 2-bit direction index is shown in Table 2. In the example of Table 2, the interpretation of the MVD may vary depending on the information of the start / reference MV. For example, if the start / reference MV corresponds to a uni-predictive block or a bi-predictive block, and both reference frame lists point to the same side of the current picture (i.e., the POCs of the two reference pictures are both greater than or both less than the POC of the current picture), the code in Table 2 may specify the sign (direction) of the MV offset added to the start / reference MV. If the start / reference MV corresponds to a bi-predictive block with two reference pictures on different sides of the current picture (i.e., the POC of one reference picture is larger than the POC of the current picture and the POC of the other reference picture is smaller than the POC of the current picture), and the difference between the reference POC in picture reference list 0 and the current frame is larger than the difference between the reference POC in picture reference list 1 and the current frame, then the code in Table 2 may specify the sign of the MV offset added to the reference MV corresponding to the reference picture in reference picture list 0, and the sign of the offset of the MV corresponding to the reference picture in picture reference list 1 may have the opposite value (opposite sign to the offset). Otherwise, if the difference between the reference POC in picture reference list 1 and the current frame is larger than the difference between the reference POC in picture reference list 0 and the current frame, then the code in Table 2 may specify the sign of the MV offset added to the reference MV associated with picture reference list 1, and the sign of the offset for the reference MV associated with picture reference list 0 has the opposite value. [Table 2]

[0059] In some embodiments, the MVD may be scaled according to the POC difference for each direction. If the POC difference in both lists is the same, no scaling is required. Otherwise, if the POC difference in Reference List 0 is larger than the POC difference in Reference List 1, the MVD of Reference List 1 is scaled. If the POC difference in Reference List 1 is larger than the POC difference in Reference List 0, the MVD of Reference List 0 may be scaled in the same way. If the starting MV is unidirectionally predicted, the MVD is added to the available MV or reference MV.

[0060] In some embodiments of coding and signaling MVDs for bidirectional mixed prediction, in addition to or as an alternative to separately coding and signaling two MVDs, symmetric MVD coding may be implemented, whereby only one MVD needs to be signaled, and the other MVD may be derived from the signaled MVD. In such an implementation, no motion information, including reference picture indices for list 0 and list 1, is signaled. Specifically, at the slice level, a flag may be included in the bitstream, called "mvd_l1_zero_flag," which indicates whether reference list 1 is not signaled in the bitstream. If this flag is 1, indicating that reference list 1 is equal to 0 (and thus not signaled), a bidirectional prediction flag, called "BiDirPredFlag," may be set to 0, i.e., no bidirectional prediction is present. Otherwise, if mvd_l1_zero_flag is 0, then BiDirPredFlag may be set to 1 when the closest reference picture in list 0 and the closest reference picture in list 1 form a forward-backward pair of reference pictures or a backward-forward pair of reference pictures, and the reference pictures in both list 0 and list 1 are short-term reference pictures. Otherwise, BiDirPredFlag may be set to 0. A BiDirPredFlag of 1 may indicate that a symmetric mode flag is also signaled in the bitstream. A decoder may extract the symmetric mode flag from the bitstream when BiDirPredFlag is 1. The symmetric mode flag may, for example, be signaled at the CU level (optionally), and it may indicate whether symmetric MVD coding mode is used for the corresponding CU. When the symmetric mode flag is 1, it indicates the use of symmetric MVD coding mode, and indicates that only the reference picture indices of both list 0 and list 1 (called "mvp_l0_flag" and "mvp_l1_flag") are signaled along with the MVD associated with list 0 (called "MVD0"), and that the other motion vector differential "MVD1" should be derived without being signaled.For example, MVD1 may be derived as -MVD0. Thus, only one MVD is signaled in the exemplary target MVD mode.

[0061] In other embodiments of MV prediction, a harmonized scheme may be used to implement general merge mode, MMVD, and other types of MV prediction for both single-reference and mixed-reference modes. Various syntax elements may be used to signal how the MV of the current block is predicted. For example, for single-reference mode, the following MV prediction modes may be signaled:

[0062] NEARMV-DRL (Dynamic Reference List) uses one of the motion vector predictors (MVPs) in the list pointed to by the index.

[0063] One of the motion vector predictors (MVPs) in the list signaled by the NEWMV-DRL index is used as a reference and the delta is applied to the MVP.

[0064] GLOBALMV - Uses motion vectors based on frame-level global motion parameters.

[0065] Similarly, for mixed reference inter prediction modes using two reference frames corresponding to the two MVs to be predicted, the next MV prediction mode may be signaled.

[0066] NEAR_NEARMV - Use one of the motion vector predictors (MVP) in the list signaled by the DRL index.

[0067] NEAR_NEWMV - Use one of the motion vector predictors (MVPs) in the list signaled by the DRL index as a reference to transmit the delta MV for the second MV.

[0068] NEW_NEARMV - Use one of the motion vector predictors (MVPs) in the list signaled by the DRL index as a reference to transmit the delta MV for the first MV.

[0069] NEW_NEWMV - Use one of the motion vector predictors (MVPs) in the list signaled by the DRL index as a reference and send delta MVs for both MVs.

[0070] GLOBAL_GLOBALMV - Use the MV from each reference based on their frame-level global motion parameters.

[0071] The term "NEAR" above refers to MV prediction using a reference MV without MVD as a general merge mode, while the term "NEW" refers to MV prediction using a reference MV and offsetting it with a signaled motion vector differential (MVD), as in the MMVD mode. In the case of decoding inter prediction, both the reference base motion vector and the motion vector differential may generally be different or independent between the two references, although they may be correlated and such correlation may be utilized to reduce the amount of information required to signal the two motion vector differences. In such a situation, joint signaling of the two MVDs may be implemented and indicated in the bitstream.

[0072] The dynamic reference list (DRL) described above is dynamically maintained and can be used to hold a set of indexed motion vectors that are considered as candidate motion vector predictors.

[0073] Motion Vector Differential Coding

[0074] In some embodiments, in coding techniques such as AV1, fractional motion vector precision (or accuracy) such as 1 / 8 pixel (i.e., one-eighth of a pixel) is allowed or supported, and the following syntax is used to signal motion vector differentials for reference frame list 0 (L0) or list 1 (L1): mv_joint specifies which components of the motion vector differential are non-zero ○0 indicates no non-zero MVD in either the horizontal or vertical direction ○ 1 indicates non-zero MVD in the horizontal direction only ○2 indicates that there is non-zero MVD only in the vertical direction ○3 indicates that there is non-zero MVD in both the horizontal and vertical directions mv_sign specifies whether the motion vector differential is positive or negative mv_class specifies the class of the motion vector differential. As shown in Table 3, the higher the class, the larger the magnitude of the motion vector differential. [Table 3] mv_bit specifies the integer part of the offset between the motion vector differential and the starting magnitude for each MV class mv_fr specifies the first two fractional bits of the motion vector differential mv_hp specifies the third fractional bit of the motion vector differential

[0075] Adaptive MVD Resolution

[0076] In some embodiments, the resolution of the MVD for various MVD size classes may be differentiated. For example, a high-resolution MVD for large MVD values ​​of higher MVD classes may not statistically significantly improve compression efficiency. Therefore, the MVD may be coded with reduced resolution (integer pixel resolution or fractional pixel resolution) for larger MVD size ranges corresponding to higher MVD size classes. Similarly, the MVD may generally be coded with reduced resolution (integer pixel resolution or fractional pixel resolution) for larger MVD values. Such MVD class-dependent or MVD size-dependent MVD resolution may generally be referred to as adaptive MVD resolution.

[0077] When adaptive MVD resolution is used, the number of signaling bits saved by targeting a low-precision MVD may be larger than the number of additional bits required for coding the inter-prediction residual resulting from such a low-precision MVD. This is due to the statistical observation that non-adaptively treating the MVD resolution of a large magnitude or high class MVD at the same level as the MVD resolution of a low magnitude or low class MVD may not significantly improve the inter-prediction residual coding efficiency of blocks with large magnitude or high class MVDs. In other words, using a higher MVD resolution for a large magnitude or high class MVD may not result in significant coding gain compared to using a lower MVD resolution.

[0078] In some embodiments, further constraints may be imposed on mixed reference modes, such as the NEW_NEWMV and NEAR_NEWMV modes described above. Specifically, the accuracy of the motion vector differential (MVD) depends on the associated class and the magnitude of the MVD.

[0079] In some embodiments, fractional MVDs are allowed if the MVD size is 1 pixel or less. Alternatively, or additionally, in some embodiments, only one MVD value is allowed if the value of the associated MV class is MV_CLASS_1 or greater, and the MVD value for each MV class is derived as 4, 8, 16, 32, or 64 for MV class 1 (MV_CLASS_1), 2 (MV_CLASS_2), 3 (MV_CLASS_3), 4 (MV_CLASS_4), or 5 (MV_CLASS_5), respectively.

[0080] As an example, the allowed MVD values ​​for each MV class are shown in Table 4. [Table 4] Additionally, if the current block is coded as NEW_NEARMV or NEAR_NEWMV mode, one context is used to signal mv_joint or mv_class. Otherwise, another different context is used to signal mv_joint or mv_class. Note that in AV1, mv_class specifies the class of the motion vector differential. The higher the class, the larger the motion vector differential represents the update, and mv_joint specifies which components of the motion vector differential are non-zero.

[0081] Adaptive MVD resolution improvement

[0082] In some embodiments, further improvements to adaptive MVD may be implemented. A new inter-coding mode called AMVDMV may be added for the single reference case. When the AMVDMV mode is selected and / or flagged, it indicates that AMVD is applied to signal the MVD so that adaptive MVD resolution is used.

[0083] In one solution, a flag called amvd_flag is added under the JOINT_NEWMV mode to indicate whether AMVD is applied to the joint MVD coding mode. When adaptive MVD resolution is applied to the joint MVD coding mode (sometimes also called the joint AMVD coding mode), the MVDs of two reference frames are jointly signaled, and the accuracy of the MVD is implicitly determined, for example, by the size of the MVD. Otherwise, the MVDs of two (or more) reference frames are jointly signaled, and conventional MVD coding is applied.

[0084] Alternatively, the MVDs of two (or more) reference frames may be jointly signaled, and conventional MVD coding is applied. In this case, instead of adding amvd_flag as described above, one new inter-prediction mode called JOINT_AMVDNEWMV is added to indicate that AMD is applied to the joint MVD coding mode.

[0085] Adaptive Motion Vector Resolution (AMVR)

[0086] In some embodiments, AMVR may be implemented with various coding techniques, such as AV1. For example, a total of seven MV precisions (e.g., 8, 4, 2, 1, 1 / 2, 1 / 4, 1 / 8) are supported. For each prediction block, the encoder (e.g., AMV, AV1 encoder) searches through all supported precision values ​​and signals the best precision to the decoder.

[0087] To reduce the runtime complexity of the encoder, two precision sets are supported. Each precision set may, for example, include four predefined precisions. One precision set is adaptively selected at the frame level based on the maximum precision value of the frame. As an example, the maximum precision may be signaled in the frame header. Table 5 summarizes the supported precision values ​​based on the frame-level maximum precision. [Table 5]

[0088] In some embodiments, there is a frame-level flag to indicate whether the MV of a frame includes sub-pel (i.e., sub-pixel) precision. AMVR is only valid if the flag cur_frame_force_integer_mv has a value of 0. Under AMVR, if the block precision is less than the maximum precision, the motion model and interpolation filter are not signaled. If the block precision is less than the maximum precision, the motion mode can be inferred as translation motion, and the interpolation filter is inferred as a regular interpolation filter. Similarly, if the block precision is either 4-pel or 8-pel, the inter-intra mode is not signaled and is inferred as 0.

[0089] Joint MVD Coding (JMVD)

[0090] In some embodiments, in coding techniques such as AV1, an inter-coding mode called JOINT_NEWMV is applied to indicate whether the MVDs of two reference lists are jointly signaled. When the inter-prediction mode is equal to the JOINT_NEWMV mode, the MVDs of reference list 0 and reference list 1 are jointly signaled. In this case, only one MVD called joint_mvd is signaled and transmitted to the decoder, and the delta MVs of reference list 0 and reference list 1 are derived from joint_mvd.

[0091] In some embodiments, the JOINT_NEWMV mode is signaled along with the NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, NEW_NEWMV, and GLOBAL_GLOBALMV modes. No additional context needs to be added.

[0092] When JOINT_NEWMV mode is signaled and the Picture Order Count (POC) distance between the two frames and the current frame is different, the MVD is scaled for reference list 0 or reference list 1 based on the POC distance. Specifically, the distance between reference frame list 0 and the current frame is denoted as td0, and the distance between reference frame list 1 and the current frame is denoted as td1. If td0 is greater than or equal to td1, then joint_mvd is the joint-signaled MVD and is used directly for reference list 0, and mvd for reference list 1 is derived from joint_mvd based on equation (1):

number

[0093] Otherwise, if td1 is greater than or equal to td0, then joint_mvd is used directly for reference list 1, and mvd for reference list 0 is derived from joint_mvd according to equation (2):

number

[0094] Improved Joint MVD Coding

[0095] In some embodiments, in coding techniques such as AV1, when a block is coded as a joint MVD coding mode, for example, JOINT_NEWMV or JOINT_AMVDNEWMV, a new syntax called mvd_scaling_factor_idx is signaled in the bitstream to indicate the MVD scaling factor between reference frame 0 and reference frame 1.

[0096] As an example, two predefined lookup tables may be used to store the supported / allowed scaling factors for JOINT_NEWMV or JOINT_AMVDNEWMV, respectively, as shown in Tables 6 and 7. The associated entry index of the selected scaling factor in the lookup table is signaled in the bitstream. For JOINT_AMVDNEWMV mode, the same scaling factor is applied to both the vertical and horizontal components of the MVD of reference frame lists 0 and / or 1. For JOINT_NEWMV mode, the scaling factor of one component of the MVD (either the vertical or horizontal component) is restricted to be 1, and the scaling factor of the other component of the MVD can be another value, such as 2 or 1 / 2. In one example, the MVD of reference frame lists 0 and 1 is calculated by the following formula:

number

[0097] Here, mvd_ref0 and mvd_ref1 represent the MVD of reference frame list 0 and the MVD of reference frame list 1, respectively. The distance between reference frame list 0 and the current frame is denoted as td0, the distance between reference frame list 1 and the current frame is denoted as td1, joint_mvd represents the joint signaled MVD, and jmvd_scale represents the scaling coefficient. [Table 6] [Table 7]

[0098] Bi-prediction with CU-level weights (BCW)

[0099] In some embodiments, in video coding techniques such as HEVC, a bi-predictive signal is generated by averaging two prediction signals obtained from two different reference pictures and / or by using two different motion vectors. In VVC, the bi-predictive mode is extended beyond simple averaging to allow for a weighted averaging of the two prediction signals. For example, P bi-pred The prediction using bidirectional, denoted as , can be calculated by using the following equation (5):

number

[0100] As an example, weighted average bi-prediction allows five weights w∈{-2,3,4,5,10}. When w is equal to 4, equal weighting factors are used to perform the weighted average of the two predicted samples. For each bi-predicted CU, the weight w is determined in one of two ways: 1) for non-merged CUs, the weight index is signaled after the motion vector differential; 2) for merged CUs, the weight index is inferred from neighboring blocks based on merge candidate indexes.

[0101] In some embodiments, BCW is applied only to CUs that contain 256 or more luma samples (CU width × CU height is 256 or more). For low latency pictures, all five weights are used. For non-low latency pictures, only three weights (w∈{3,4,5}) are used.

[0102] Local Illumination Compensation (LIC)

[0103] LIC is a video coding tool that can be utilized by video encoders and video decoders. In some embodiments, LIC may be applied based on a linear model to compensate for illumination changes between one or more temporal reference pictures and a current picture (e.g., in a motion compensation stage). The linear model is based on LIC parameters, including a scaling factor α and an offset β, which are described in more detail in the block adaptive weighted prediction section below. LIC may be enabled or disabled, for example, by high-level signaling at various levels.

[0104] In an embodiment, a bi-predictive reference template may be generated for a template sample associated with a current block. The reference template sample may be identified based on one or more motion vectors associated with the current block. For example, the reference template sample may include a temporal reference CU neighboring the current CU and may correspond to a template sample for the current CU. The reference template samples may be jointly considered (e.g., averaged) in LIC parameter derivation.

[0105] In some embodiments, a least mean-squared-error algorithm may be applied to derive the LIC parameters. For example, a calculation based on LMSE may be performed to determine the LIC parameters such that the difference between the bi-predicted reference template samples and the template samples for the current CU is minimized.

[0106] In some embodiments, a similar approach may be used under uni-prediction, in which case the LIC parameters may be determined such that the difference between the uni-predicted reference template reference sample and the template sample for the current CU may be minimized.

[0107] It should be noted that the LMSE algorithm described here is only an example for deriving the LIC parameters; one or more other approaches / algorithms may also be used.

[0108] Block Adaptive Weighted Prediction (BAWP)

[0109] In some embodiments, the BAWP is used to model local illumination variations.

[0110] Referring to Figure 12, for example, BAWP may be a block-level weighted prediction that models the local illumination change between a current block and its predicted block as a function of that between a current block template (or causal samples of the current block) and a reference block template. A template for a current block (1212) (also referred to as a current template 1210) and a template for a reference block (1222) (also referred to as a reference template 1220) are shown in Figure 12. The reference block may be indicated or determined by a motion vector (MV 1230). The current block may be in a current picture (or current frame), and the reference block may be in a reference picture (or reference frame). In some implementations, the function may be a linear function. The parameters of the function may be represented by a scale factor α and an offset β that form a linear equation. The scale factor may also be called a scale, an alpha coefficient, or an alpha value.

[0111] An example linear function used to compensate for illumination changes when a block is coded in BAWP mode is given below:

number

[0112] where p'(x') is a predicted sample at position x' in the current block (or a predicted sample in a prediction unit (PU) in the current block), p(x) is a sample corresponding to p'(x') at position x in the reference block, alpha is a scale factor (or scale), and β is an offset value. Note that the reference block can be identified or derived from the MV associated with the current block, and p(x) is a reference sample indicated by the MV at position x on the reference picture. Note that in equation (6), the reference sample and the predicted sample may have the same coordinates (i, j) within each block (i.e., the reference block and the current block, or the reference block and the prediction unit). Alternatively, the coordinates of the reference sample in the reference block may be based on the coordinates of the predicted sample in the current block. For example, the coordinates of the predicted sample may be adjusted by a delta value to obtain the coordinates of the corresponding reference sample (in the reference picture). In some embodiments, p'(x') and p(x) may be said to be collocated, where p(x) is the reference sample at position x in the reference block that corresponds to p'(x') at position x' in the current block. In some embodiments, the correspondence between p'(x') and p(x) may be established by motion information such as a motion vector.

[0113] In some embodiments, α and β can be derived based on a current block template and a reference block template, so signaling overhead for them is unnecessary, except that a BAWP flag (e.g., bawp_flag) is signaled in the case of a single inter-prediction mode to indicate the use of BAWP. In some embodiments, the BAWP method is applied only to blocks having a size of 8×8 or larger and coded in a single inter-prediction mode. In some embodiments, the BAWP method is applied only to the luma component.

[0114] Although deriving both the scale factor and the offset (α and β) will save some signaling overhead, there may be some potential decisions in certain scenarios. For example, the effectiveness and / or accuracy of the scale factor depends on the similarity between the current block and its template in the reference frame. If the correlation between the current block and its template in the reference frame is high, the derived scale factor can significantly improve prediction accuracy. However, if the current block and its template in the reference frame are significantly different, the scale factor may be inaccurate or even useless, and prediction accuracy may be reduced.

[0115] This disclosure discloses various embodiments for improving video encoding / decoding techniques in BAWP mode and / or LIC mode, with the aim of improving prediction accuracy while minimizing signaling overhead. Specifically, various methods are described for signaling and / or deriving the scale factors and offsets (α and β) measured in equation (6). Furthermore, adaptive precision of the scale factors and / or offsets is used to widen the supported parameter range.

[0116] In this disclosure, the term "block" may refer to a transform block, a coded block, a prediction block, a coding block, etc. The term "chroma block" may refer to any block of a chrominance (color) channel. The direction of a reference frame is determined by whether the reference frame is before the current frame in display order or after the current frame in display order.

[0117] In this disclosure, a sample may be interpreted as a pixel value of a pixel, which may generally refer to any component (luma or chroma).

[0118] In this disclosure, the terms "x-axis" and "y-axis" refer to the horizontal and vertical components of a two-dimensional (2D) value. They may be replaced by two other axes along two predefined directions that are orthogonal to each other, and similar embodiments apply in this case. That is, the x-axis and y-axis may be rotated by a certain angle. For example, the x-axis and y-axis may be replaced by a 45-degree axis and a 135-degree axis.

[0119] In this disclosure, conventional JMVD may refer to JMVD with normal full MV resolution or JMVD with AMVR.

[0120] For the embodiments described below, a coding block or coded block may be coded in BAWP (or LIC, or compound weighted prediction (CWP)) mode, which will be referred to hereafter as BAWP for ease of description. In some examples, a BAWP flag (e.g., bawp_flag) is signaled in the video bitstream to indicate whether the BAWP mode is used for the current block. Based on the value of BAWP (e.g., bawp_flag) being equal to 1, a scale factor and / or offset is further signaled in the video bitstream to indicate the degree of illumination change of the current block.

[0121] In one embodiment, when a current block is predicted from its reference block using a linear function including a scale factor and an offset β (e.g., the linear equation (6) described above), the selection of the scale factor and / or the offset may be signaled in the bitstream and parsed at the decoder side to reconstruct the predicted block. The reference block may be specified, for example, by a motion vector associated with the current block. That is, the scale factor α and / or the offset β may be explicitly signaled rather than derived by the decoder. In some embodiments, only the scale factor may be signaled, and the offset β may be derived based on the scale factor, or vice versa. In this disclosure, the scale factor may also be referred to as a scale, a scale factor, a scale factor α, or an alpha coefficient.

[0122] Based on research and statistical observations, non-adaptively processing (coding, signaling) large scale factor values ​​with the same or the same precision as small scale factor values ​​may not significantly improve prediction accuracy and efficiency. In other words, applying the same precision to large scale factor values ​​may not provide significant coding benefits compared to using lower precision.

[0123] In one embodiment, rather than using a uniform precision, the signaled precision of the scale factor α may depend on the magnitude of the scale factor.

[0124] In one embodiment, the signaled precision of a scale factor may decrease as the magnitude (or absolute value of the magnitude) of the scale factor increases. The precision may be represented by the step size between two adjacent scale factors.

[0125] As an example, the list of supported scale factors may be {1.0, 1.2, 1.6, 2.2, 3.0, 4.0}. The step sizes between two adjacent scale factors are 0.2, 0.4, 0.6, 0.8, 1.0, with decreasing precision. As another alternative, the step size may be the same for a first scale factor range and increased for a second scale factor range if the second scale factor range covers scale factors with larger values ​​or larger absolute values.

[0126] In one embodiment, to further improve coding (e.g., entropy coding) gain and efficiency and to reduce signaling overhead, the supported range of scale factors may be divided into multiple groups. Each group may be assigned a different precision according to various methods. For example, the precision of the signaled scale factors may decrease as the group index increases. The group index and the values ​​of the scale factors within each group are signaled in the bitstream and parsed at the decoder side. The group index may be used to identify the group, and another index may be used to identify a specific scale factor within the group. The groups may be implemented as a lookup table, a list, a set, etc. The precision of the signaled scale factor follows the precision of the group to which the scale value belongs.

[0127] In this disclosure, the term "list" for storing scale values ​​may be used synonymously with the term "lookup table." Both lists and lookup tables can support sorting functionality, and both can be sorted based on specific sorting rules discussed below.

[0128] In some embodiments, the supported scale factor ranges are divided into four groups, as shown in Table 8 below. The second column of the table lists the range (or range of absolute values ​​of the scale factor) for each group, while the third column lists the step size (or precision) of the signaled values ​​within each group. In the table, parentheses "(" or ")" indicate that the endpoint values ​​are exclusive. Brackets "[" or "]" indicate that the endpoint values ​​are inclusive. [Table 8]

[0129] In one embodiment, the precision of the signaled scale factor may decrease as the difference between the magnitude of the scale factor and a threshold (TH) increases. The absolute difference, denoted Diff, represents the scale and distance from the threshold. In some examples, TH may be set or preset to 1.

[0130] An example is shown in Table 9 below. The first column of the table lists the range of Diff per group, while the second column lists the precision of the signaled scale factor within each group. [Table 9]

[0131] For example, if TH is set to 1, and the scale factor calculated by the encoder is 1.2, Diff will be 0.2 and the signaled scale factor will be assigned a step size of 1 / 8; if the scale factor calculated by the encoder is 5.3, Diff will be 4.3 and the signaled scale factor will be assigned a step size of 2.

[0132] Alternatively, in one embodiment, to reduce encoder / decoder complexity, a uniform scale factor precision may be used, i.e., a fixed precision may be used for all supported values ​​of the scale factor.

[0133] In one embodiment, all supported values ​​of the scale factor are sorted based on the absolute difference (or distance) between the magnitude of the scale factor and a threshold TH. The magnitude threshold can be supported by the encoder and / or decoder, and the TH used to sort the scale values ​​can be dynamically updated, for example, by signaling or syntax elements. The selection and dynamic update of the TH can help improve the efficiency of entropy encoding. After each update of the TH value, the decoder can resort (reorder) the supported scale values ​​based on the updated TH. Thus, the decoder maintains an ordered data structure (e.g., an ordered list, an ordered lookup table) for storing the scale factors. If multiple lookup tables are used, the scale factors can be shuffled to different lookup tables (e.g., with different table indexes) during the re-sorting process.

[0134] In one embodiment, the index of the selected scale factor for the current block in the reordered scale factor list (eg, a lookup table) is signaled directly in the bitstream and parsed at the decoder side.

[0135] In one embodiment, the reordered scale factor list may be divided into multiple groups (e.g., multiple lookup tables). The group index of the target group containing the selected scale factor to be used for the current block, along with an index identifying the selected scale factor within the target group, is signaled in the bitstream and parsed at the decoder side. For example, the reordered scale factor list may contain 16 scale factors, which may be divided into four groups (sublists), each of which may contain four scale factors and be assigned a group index. The reordered scale factor list is used merely as an example. In other implementations, the decoder may not need to use the reordered scale factor list and may directly maintain multiple groups.

[0136] It should be noted that in the present disclosure, unless otherwise specified, a signaling may include one or more sub-signals. The one or more sub-signals may be transmitted together or separately. In the above example, a group index identifying a target group and an index identifying a selected scale factor within the target group may be signaled together in one signaling, or they may be signaled by two sub-signals that may be signaled separately. The signaling of the present disclosure may be carried, for example, by syntax elements.

[0137] In one embodiment, the value of TH is signaled in a high-level syntax (or syntax element), including a sequence-level syntax, a frame-level syntax, or a slice-level syntax.

[0138] In one embodiment, instead of signaling the value of the scale factor, a scale factor difference (delta scale factor), which is the difference between the scale factor predictor and the scale factor, may be signaled in the bitstream and parsed at the decoder side. The decoder may first derive or predict a predicted scale factor to obtain the scale factor predictor, and then apply the delta scale factor to the scale factor predictor to derive the final scale factor.

[0139] In some embodiments, the decoder can derive a scale factor predictor based on local illumination changes between the current block template 1210 (or causal samples of the current block) and the reference block template 1220, and the delta scale factors are signaled in the bitstream and parsed at the decoder side.

[0140] In one embodiment, the decoder may keep an ordered list of all supported scale values, and the sorting or reordering may be based on the absolute difference between the scale factor magnitude and the scale factor predictor, as described above.

[0141] In one embodiment, the index of the selected scale factor for the current block in the reordered scale factor list is signaled directly in the bitstream and parsed at the decoder side.

[0142] In one embodiment, the reordered scale factor list may be divided into multiple groups, and the group index of the selected scale factor for the current block, along with the index within each group, is signaled in the bitstream and parsed at the decoder side.

[0143] In one embodiment, causal samples to the left or above (top) of the current block and / or reference block may be used to derive the predicted scale factor.

[0144] In some embodiments, if the width is greater than the height for the current block, only the upper causal samples (of the current block and / or the reference block) (e.g., 1214 in FIG. 12) are used to derive the predicted scale factor.

[0145] In some embodiments, if the height is greater than the width for the current block, only the left causal sample (of the current block and / or reference block) (e.g., 1216 in FIG. 12) may be used to derive the predicted scale factor.

[0146] In some embodiments, the bottom-left and / or top-right samples (of the current block and / or reference block) may be used to derive the predicted scale factor.

[0147] In one embodiment, the minimum supported value of the scale factor α is greater than zero.

[0148] In one embodiment, the offset value β may be derived as the average value of the template for the current block.

[0149] As an example, if the width is greater than the height for the current block, only the upper causal samples (of the current block and / or the reference block) (e.g., 1214 in Figure 12) may be used to derive the offset value β.

[0150] As an example, if the height is greater than the width for the current block, only the left causal samples (e.g., 1216 in Figure 12) (of the current block and / or reference block) may be used to derive the offset value β.

[0151] In one embodiment, the offset value β may be derived as the average value of the reference block (or a template of the reference block).

[0152] In one embodiment, the offset value β may be derived as the difference between the template of the current block and the template of the reference block. By way of example, the average of each template may be used in calculating the difference.

[0153] In one embodiment, a flag is signaled in and parsed from the bitstream to indicate whether an explicit signal of the scale factor and / or offset β is selected for the current block. In some embodiments, this flag is a signal for BAWP / LIC / CWP modes. In some embodiments, this flag is signaled only under BAWP mode, or this flag is signaled only under LIC mode, or this flag is signaled only under CWP mode. Note that the explicit signal may include a real value or an index to look up a real value from a lookup table.

[0154] In one embodiment, if the flag indicates that explicit signaling is applied, the scale factor is explicitly signaled and the offset β is set as 0 (so no access to neighboring samples is required) or the mean of the template (necessary access to neighboring samples).

[0155] In one embodiment, one high-level syntax is signaled to indicate whether explicit signaling of the scale factor and / or offset β is supported. The syntax can be signaled at at least one of the following levels: sequence level, frame level, slice level, or superblock level.

[0156] In one embodiment, the scale factor and the offset β may be jointly signaled.

[0157] In one embodiment, a group of combinations of α and β may be predefined or implicitly derived for each block, and an index indicating the selected combination (α and β) is signaled. For example, the group could be {(1.1, 0.2), (1.6, 0.1), (2.3, 0.3)}, where each pair in parentheses forms a combination of α and β.

[0158] In one embodiment, a group of predicted combinations of α and β are predefined or implicitly derived for each block, and the difference between the selected combination of α and β and the predicted combination of α and β is signaled.

[0159] In this disclosure, signaling (e.g., syntax elements) may indicate values ​​such as the scale factor and / or offset β in an explicit or implicit manner. Explicit indication may be done by sending an index to a lookup table to look up the value or by sending the value directly. If the value is sent in an implicit manner, the decoder may need to perform further derivation based on the signal syntax element. In the case of implicit indication, the syntax element may also be said to be "associated" with the value to be derived.

[0160] The various embodiments and / or implementations described in this disclosure may be combined or performed separately in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). The one or more processors execute a program stored on a non-transitory computer-readable medium. In this disclosure, the term "block" may be interpreted as a prediction block, a coding block, or a coding unit (CU).

[0161] 13 shows a flowchart 1300 of an exemplary method according to the principles underlying the above implementations for indicating scale factor and offset values. The exemplary decoding method flow may include some or all of the following steps: S1310, receiving a video bitstream including a current block and a reference block, the reference block being used to predict the current block and identified by a motion vector associated with the current block; S1320, receiving from the video bitstream a syntax element indicating a scale factor (α); S1330, determining the scale factor (α) based on the value of the syntax element; S1340, predicting the current block using the following formula: p'(x') = α × p(x) + β, where p'(x') is a predicted sample within the current block, α is a scale factor determined based on the value of the syntax element, and β is an offset; S1350, reconstructing the current block based on the predicted current block.

[0162] In some embodiments, the current block may be in a current frame and the reference block may be in a reference frame.

[0163] In some embodiments, S1320 may alternatively include receiving from the video bitstream a syntax element indicating or associated with a scale factor (α) that is the slope of a linear equation (e.g., p'(x') = α × p(x) + β).

[0164] In this disclosure, the orientation of a reference frame may be determined by whether the reference frame precedes the current frame in display order or follows the current frame in display order.

[0165] The above operations may be combined or arranged in any quantity or order as desired. Two or more steps and / or operations may be performed in parallel. The embodiments and implementations of the present disclosure may be combined or used separately in any order. The steps of an embodiment / method may be divided into multiple substeps, and each substep may be independent of other steps in the embodiment to form a standalone solution. Furthermore, each of the method (or embodiment), encoder, and decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). As an example, the one or more processors execute a program stored on a non-transitory computer-readable medium. The embodiments of the present disclosure may be applied to luma blocks or chroma blocks. The term "block" may be interpreted as a prediction block, a coding block, or a coding unit, i.e., a CU. The term "block" herein may also be used to refer to a transform block. In the following sections, block size may refer to either the width or height of the block, or the maximum width and height, or the minimum width and height, or the area size (width x height), or the aspect ratio of the block (width:height or height:width).

[0166] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 14 illustrates a computer system (1800) suitable for implementing certain embodiments of the disclosed subject matter.

[0167] The computer software may be coded in any suitable machine code or computer language that may be subject to mechanisms such as assembly, compilation, linking, etc. to produce code containing instructions that may be executed by one or more central processing units (CPUs), graphics processing units (GPUs), etc., directly or through interpretation, microcode execution, etc.

[0168] The instructions may be executable by various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming consoles, Internet of Things devices, and the like.

[0169] 14 for computer system 1800 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement regarding any one or combination of components described in the exemplary embodiment of computer system 1800.

[0170] The computer system 1800 may include certain human interface input devices, which may include one or more of a keyboard 1801, a mouse 1802, a trackpad 1803, a touchscreen 1810, a data glove (not shown), a joystick 1805, a microphone 1806, a scanner 1807, and a camera 1808 (only one of each is shown).

[0171] The computer system 1800 may also include certain human interface output devices that may stimulate one or more of the user's senses through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1810), data gloves (not shown), or joystick (1805), although haptic feedback devices that do not function as input devices may also exist), audio output devices (e.g., speakers (1809), headphones (not shown)), visual output devices (e.g., screens (1810) including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capability, each with or without haptic feedback capability, some of which are capable of outputting two-dimensional visual output or output in more than three dimensions by means of stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0172] The computer system (1800) may also include human-accessible storage devices and their associated media, such as CD / DVD or similar media (1821), including CD / DVD ROM / RW (1820), thumb drives (1822), removable hard disks or solid state drives (1823), legacy magnetic media, such as tape and floppy disks (not shown), dedicated ROM / ASIC / PLD-based devices, such as security dongles (not shown), and the like.

[0173] Those skilled in the art will also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0174] The computer system (1800) may also include an interface (1854) to one or more communications networks (1855). The networks may be, for example, wireless, wireline, or optical. The networks may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks such as Ethernet; wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, and the like; TV wireline or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; and vehicle and factory networks including CAN buses.

[0175] The above-mentioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core 1840 of the computer system 1800.

[0176] The core (1840) may include one or more central processing units (CPUs) (1841), graphics processing units (GPUs) (1842), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (1843), hardware accelerators for specific tasks (1844), graphics adapters (1850), etc. These devices may be connected through a system bus (1848), along with read-only memory (ROM) (1845), random access memory (RAM) (1846), internal mass storage devices such as internal non-user-accessible hard drives, SSDs, etc. (1847). In some computer systems, the system bus (1848) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached to the core's system bus (1848) directly or through a peripheral bus (1849). In an example, a display 1810 may be connected to a graphics adapter 1850. Architectures for peripheral buses include PCI, USB, and the like.

[0177] The computer-readable medium can bear computer code for performing various computer-implemented operations. The medium and computer code can be those specially designed and constructed for the purposes of the present disclosure, or they can be of the kind well known and available to those skilled in the computer software arts.

[0178] While this disclosure has described several exemplary embodiments, alternatives, permutations, and various substitute equivalents exist and are included within the scope of this disclosure. Thus, it will be apparent to those skilled in the art that numerous systems and methods, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

[0179] [Incorporated by reference] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 459,565, filed April 14, 2023, which claims the benefit of priority to U.S. Patent Application No. 18 / 461,785, filed September 6, 2023, each of which is incorporated herein by reference in its entirety.

Claims

1. 1. A method for a decoder to process video data, comprising: receiving a video bitstream including a current block in a current frame and a reference block in a reference frame, the reference block being used to predict the current block and identified by a motion vector associated with the current block; receiving, from the video bitstream, a syntax element indicating a scale factor (α); determining the scale factor (α) based on the value of the syntax element; Next p'(x')=α×p(x)+β where p′(x′) is a predicted sample in the current block, p(x) is a co-located reference sample in the reference block that corresponds to p′(x′), α is the determined scale factor based on the value of the syntax element, and β is an offset; reconstructing the current block based on the predicted current block; A method having the following.

2. the decoder maintains a lookup table storing candidate scale factors, and the value of the syntax element includes an index that identifies the scale factor within the lookup table. The method of claim 1.

3. the candidate scale factors in the lookup table are sorted based on the difference between each of the candidate scale factors and a threshold value; The method of claim 2.

4. the decoder maintains two or more lookup tables storing candidate scale factors; each of the two or more lookup tables supporting a different scale factor range; the value of the syntax element includes a first index that identifies a target lookup table among each of the two or more lookup tables, and a second index that identifies the scale factor within the target lookup table. The method of claim 1.

5. the candidate scale factors in each of the two or more lookup tables are sorted based on a difference between each of the candidate scale factors and a threshold value; The method of claim 4.

6. the threshold value is equal to 1; The method of claim 5.

7. receiving, from the video bitstream, high-level syntax indicating the threshold; The high level syntax is: Sequence level, Frame level, or Slice Level signaled at at least one level of The method of claim 5.

8. and further comprising updating and sorting each of the two or more lookup tables in response to the threshold being updated. The method of claim 5.

9. For any pair of a first lookup table and a second lookup table among the two or more lookup tables, a first precision of the first lookup table storing candidate scale coefficients having a higher magnitude is lower than a second precision of the second lookup table storing candidate scale coefficients having a lower magnitude, each of the first precision and the second precision being measured by a step size between two adjacent scale coefficients in each lookup table. The method of claim 4.

10. for any pair of a first lookup table and a second lookup table among the two or more lookup tables, a first precision of the first lookup table storing candidate scale coefficients having a long distance to a predefined threshold is lower than a second precision of the second lookup table storing candidate scale coefficients having a short distance to the predefined threshold; the distance between the scale factor and the predefined threshold is determined by the absolute value of the difference between the scale factor and the predefined threshold; each of the first accuracy and the second accuracy being measured by a step size between two adjacent scale factors in a lookup table; The method of claim 4.

11. The current block is coded in one of the following modes: a block adaptive weighted prediction (BAWP) mode or a local illumination compensation (LIC) mode. The method of claim 1.

12. deriving the offset as an average value of a template for the current block. The method of claim 1.

13. determining the offset based on an upper sample of a template of the current block in response to the width of the current block being greater than the height of the current block; determining the offset based on a left sample of the template of the current block in response to the height of the current block being greater than the width of the current block; Further comprising: The method of claim 1.

14. deriving the offset as an average value of one of an average value of the reference block or an average value of a template of the reference block. The method of claim 1.

15. receiving high-level syntax indicating whether the syntax element indicating the scale factor (α) is used in the video bitstream; The high level syntax is: Sequence level, Frame level, Slice level, or Superblock Level transmitted at least one level of The method of claim 1.

16. in response to the high level syntax indicating that the syntax element is used in the video bitstream, determining that the scale factor is explicitly indicated by the syntax element; setting the offset to one of an average value of the template for the current block or zero; Further comprising:

16. The method of claim 15.

17. the decoder maintains a lookup table storing candidate value pairs formed by a scale factor and an offset; the value of the syntax element includes an index that identifies a target value pair in the lookup table; The target value pair includes a scale factor and an offset used in a linear equation. The method of claim 1.

18. the syntax element indicates, explicitly or implicitly, the scale factor; p(x) is derived from p'(x') based on the motion vector; The method of claim 1.

19. 1. A device for processing video data, comprising: a memory for storing computer instructions and a processor in communication with the memory; The computer instructions, when executed by the processor, cause the processor to perform the method of any one of claims 1 to 18. device.

20. A program comprising computer readable instructions, The computer readable instructions, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 18. program.

21. 1. A method for an encoder to process video data, comprising: predicting a current block in a current frame using a reference block in a reference frame, the reference block being identified by a motion vector associated with the current block, the current block being a next p'(x')=α×p(x)+β where p′(x′) is a predicted sample in the current block, p(x) is a collocated reference sample in the reference block corresponding to p′(x′), α is a scale factor, and β is an offset; encoding a syntax element indicating said scale factor (α); A method having the following.

Images