Motion vector derivation based on block boundary distortion

Abstract

A method for obtaining a current block of a motion compensated prediction sample is provided. The method comprises selecting (s902) a motion vector from a set of candidate motion vectors, the set of candidate motion vectors comprising a first candidate motion vector and a second candidate motion vector. Selecting a motion vector from a set of candidate motion vectors includes calculating a first misaligned distortion value for a first candidate motion vector. Calculating the first misaligned distortion value for the first candidate motion vector is based on a difference in sample values on a boundary between the first neighboring block and the current block. The method comprises using (s904) the selected motion vector to generate a block of motion compensated prediction samples.

Description

Motion vector derivation based on block boundary distortion Technical Field

[0001] This disclosure relates to methods and apparatus for deriving motion vectors. Background Technology

[0002] VVC and ECM

[0003] Multifunctional Video Coding (VVC) is a block-based video codec standardized by ITU-T and MPEG. Enhanced Coding Model (ECM) is an exploratory codec currently under development. ECM aims to demonstrate and attempt to prove video coding capabilities beyond VVC. The current version of ECM is ECM-10.0.

[0004] Videos and pictures

[0005] A video (also known as a "video sequence") consists of a series of images. In VVC, each image is identified by a Picture Order Count (POC) value. The POC value also indicates the order in which the images are displayed. An image with a smaller POC value is displayed before another image with a larger POC value.

[0006] Quantity

[0007] Typically, each image consists of three components: a luminance component (Y), where the sample values ​​are luminance values; and two chrominance components (Cb and Cr), where the sample values ​​are chrominance values. Each component can be described as a two-dimensional rectangular array of sample values. Furthermore, the size of the chrominance components is usually half that of the luminance components in each dimension. For example, the luminance component of a high-resolution image would have a size of 1920x1080, while the chrominance components would each have a size of 960x540. These components are sometimes referred to as color components.

[0008] Blocks, sub-blocks, and units

[0009] A block is a two-dimensional (2D) matrix of sample values ​​(or simply "samples"). A block can be divided into two or more sub-blocks, each of which is a sample matrix. In video coding, each component of an image is divided into blocks, and the encoded video bitstream consists of a series of coded blocks. In video coding, images are typically divided into units that cover specific regions of the image.

[0010] Each unit comprises all blocks from all components that make up that particular region of the image, and each block belongs entirely to one unit. A coding unit (CU) in VVC is an example of a unit. In VVC, a CU can be recursively divided into smaller CUs. The CU at the top level is called a coding tree unit (CTU).

[0011] A CU typically contains three coding blocks: one coding block for luminance and two coding blocks for chrominance.

[0012] In VVC, the CU size can range from 4x4 to 128x128. In the current ECM, the CU size can range from 4x4 to 256x256.

[0013] Parameter set, slice header and image header

[0014] VVC specifies three types of parameter sets: Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS). PPS contains data common to all units of a picture, SPS contains data common to the Coding Layer Video Sequence (CLVS), and VPS contains data common to multiple CLVS, such as data for multiple layers in a bitstream.

[0015] The concept of slicing divides an image into independently encoded slices, where decoding one slice of an image is independent of decoding other slices of the same image. Each slice has a slice header that includes syntax elements. When decoding a slice, the decoded slice header values ​​from these syntax elements are used.

[0016] In VVC, an encoded image includes an image header. The image header contains parameters common to all slices of the encoded image.

[0017] Intra-frame prediction

[0018] In intra-frame prediction (also known as spatial prediction), previously decoded blocks within the same image are used to predict the current block. Samples from previously decoded blocks within the same image are used to predict samples within the current block. An image that includes only intra-frame predicted blocks is called an intra-frame image.

[0019] Inter-frame prediction

[0020] In inter-frame prediction (also known as temporal prediction), blocks from previously decoded images (called reference blocks) are used to predict the current block of the current image. Samples from reference blocks in previously decoded images are used to predict samples within the current block. An image that includes one or more inter-frame prediction blocks is called an inter-frame image. The previously decoded image used for inter-frame prediction is called a reference image.

[0021] Vectors (i.e., sets of values) (called "motion vectors (MVs)") are used to indicate the location of a reference block within a reference image. Each MV consists of two values: an x-value (also called the x-component) and a y-value (also called the y-component), representing the displacement between the current block and the reference block in the x or y dimension. The component values ​​can have a finer resolution than integer positions. In this case, filtering (typically interpolation) is performed to compute the values ​​used for prediction. Figure 1 shows an example of an MV for the current block. The example MV includes values ​​2,1.

[0022] An inter-frame image can use multiple reference images. These reference images are typically placed in two reference image lists, L0 and L1. The reference image output before the current image is usually the first image in L0. The reference image output after the current image is usually the first image in L1.

[0023] Inter-frame prediction blocks can use one of two prediction types (single prediction and double prediction). A single prediction block uses either L0 or L1 to predict from a single reference image. A double prediction block predicts from two reference images (one from L0 and the other from L1). Figure 2 shows an example of the prediction types.

[0024] Image encoding types (low-latency images and non-low-latency images)

[0025] A low-latency image is one in which all its reference images are displayed before it. In other words, for a low-latency image, all its reference images have a lower POC value than the current POC.

[0026] A non-low-latency image is one whose at least one reference image is displayed after it. In other words, at least one reference image of a non-low-latency image has a POC value that is larger than the current POC.

[0027] Fractional MV, interpolation filter and MV rounding

[0028] The values ​​of the x or y components of MV can correspond to sample locations with a finer granularity than integer (sample) locations. These locations are also called fractional (sample) locations.

[0029] In VVC and the current ECM, MV can be at the 1 / 16 sample position. Figure 3 shows multiple fractional positions in the horizontal (x) dimension. Solid squares represent integer positions. Circles represent 1 / 16 positions. For example, MV = (4, 10) means that the x component is at the 4 / 16 position and the y component is at the 10 / 16 position.

[0030] In video encoding, the MV rounding process is sometimes used to transform an MV at one location into another target location. An example of rounding is rounding a fractional MV position to the nearest integer position.

[0031] When the MV is at a fractional position, filtering (usually interpolation) is performed to calculate the sample values ​​at those positions. In VVC, the length (number of filter taps) of the interpolation filter used for the luminance component is 8, as shown in Table 1 below. In ECM, the length of the interpolation filter used for the luminance component has increased to 12. Table 1

[0032] Residuals, Transformations and Quantization

[0033] Residual blocks are generated by calculating the difference between samples from a source block (also known as an "input block") containing the original samples and samples from a prediction block (e.g., an inter-frame prediction block or an intra-frame prediction block). To further eliminate redundancy, the residual block is then typically compressed using a spatial transform, resulting in transform coefficients. These transform coefficients are then quantized based on quantization parameters (QP) to control the fidelity of the residual block, and thus also the bit rate required to compress the block. A coded block flag (CBF) is used to indicate the presence of any non-zero quantized transform coefficients. All coding parameters are then entropy-encoded at the encoder and decoded at the decoder. If the coded block flag is 1, a reconstructed block can be derived by inverse quantization and inverse transform of the quantized transform coefficients, and this reconstructed block is then added to the prediction block. If the coded block flag is 0, the reconstructed block is identical to the prediction block.

[0034] Inter-frame prediction information / motion information

[0035] For inter-frame blocks within an inter-frame image in VVC, the inter-frame prediction information includes the following three elements:

[0036] (1) Reference image list flag (RefPicListFlag), which indicates which reference image list was used for the block (when the value of the flag is equal to 0, it means that only L0 was used to predict the current block; when the value of the flag is equal to 1, it means that only L1 was used to predict the current block; and when the value of the flag is equal to 2, it means that both L0 and L1 were used to predict the current block).

[0037] (2) The reference image index (RefPicIdx) for each reference image list used (this index indicates which reference image in the reference list will be used to predict the current block); and

[0038] (3) The motion vector (MV) of each reference image used represents the position of the current block within the reference image.

[0039] Inter-frame prediction information is also known as motion information. The decoder stores the motion information for each inter-frame block. In other words, each inter-frame block maintains its own motion information.

[0040] Encoder decision and rate distortion (RD) costs

[0041] In practice, in order for the encoder to determine the best prediction mode for the current block, the encoder will evaluate all possible prediction modes for the current block and select the prediction mode that produces the minimum rate distortion (RD) cost.

[0042] RD costs are calculated as D (distortion) measures the difference between the reconstructed block and the corresponding source block. A common metric used to calculate D is the sum of squared errors. ,in and These are the sample values ​​from blocks F and G, respectively. R (rate) is typically an estimate of the number of bits required to encode the pattern, and It is a trade-off parameter between R and D.

[0043] motion information signaling

[0044] VVC and ECM include various methods for implicit signaling of motion information for each block, including merging and sub-block merging methods. The common motivation behind these implicit methods is to inherit or reuse motion information from adjacent coded blocks. This is often effective in practice due to the spatial correlation between adjacent blocks (i.e., the fact that adjacent blocks often behave similarly).

[0045] Merge (block merge) methods and merge modes

[0046] The merging method derives a set of motion information from the previously decoded block and uses the derived motion information to generate a sample of the entire block. This merging method is sometimes referred to as the block merging method.

[0047] The merging method first generates a candidate list of motion information. This list is also called the merging list. Candidates are derived from previously encoded blocks. These previously encoded blocks can be spatially adjacent blocks or temporally collocated blocks relative to the current block. Figure 4 shows spatially adjacent blocks: left (L), top (T), top right (TR), bottom left (LB), and top left (TL).

[0048] The merge list construction process typically checks previously coded blocks in a predefined order, such as T, L, TR, LB, and then TL. For each previously coded block checked, if the previous coded block was inter-coded and its motion information is not copied in the list, then the motion information of that previous coded block is added to the merge list.

[0049] After the merge list is generated, one of the candidates in the list is used to derive the motion information for the current block. The candidate selection process takes place on the encoder side. The encoder selects the best candidate from the list and encodes the index (merge_index) in a bitstream to signal the decoder. The decoder receives this index, follows the same merge list derivation process as the encoder, and uses the index to retrieve the correct candidate. Blocks using the block merging method are sometimes referred to as blocks in merge mode.

[0050] In the current ECM, non-adjacent spatial blocks are also considered sources of motion information during the construction of the merge list. Figure 5 shows some examples of these non-adjacent spatial blocks (labeled NA1, NA2, and NA3).

[0051] Sub-block merging method

[0052] VVC and ECM also include a sub-block merging method. It divides the current block into multiple sub-blocks, allowing each sub-block to have its own motion information. Figure 6 shows an example of the current block and its sub-blocks. Each sub-block maintains its own motion information. It should be noted that the sub-blocks are rectangular.

[0053] Overlapping Block Motion Compensation (OBMC)

[0054] OBMC is a tool included in ECM that operates at the block or sub-block boundaries of the current inter-frame block. OBMC blends the predicted sample P_CUR (generated using currently associated motion information) of the current block or sub-block with another set of predicted samples P_NB (generated using motion information from neighboring blocks or neighboring sub-blocks at the current block or sub-block boundary). The OBMC blending process weights P_CUR and P_NB to produce the OBMC-modified predicted sample set P_OBMC. In other words, the sets P_CUR or P_NB are each associated with their respective weighting factors, and the OBMC-modified predicted sample set P_OBMC is derived as w1*P_CUR + w2*P_NB (where w1 + w2 = 1). OBMC can provide better predictions for samples closer to block or sub-block boundaries.

[0055] Bidirectional optical flow (BDOF)

[0056] BDOF is a tool included in VVC and the current ECM, which can be used to refine the predicted samples generated from the dual MV. BDOF relies on optical flow estimation to derive a pair of refinement parameters (Vx, Vy), which can be further used to refine the predicted samples.

[0057] Decoder-side motion vector refinement (DMVR)

[0058] DMVR is a tool included in VVC and the current ECM for refining the motion vectors of dual motion vectors (MVs). DMVR operates at the sub-block level (typically 16x16). Unlike BDOF, which relies on optical flow estimation, DMVR relies on bilateral matching of two reference blocks to refine the dual MV. This matching is based on SAD (sum of absolute differences). DMVR searches within a window around the dual MV to find if another dual MV (dual MV') exists that gives a better match between the L0 and L1 reference blocks. If it exists, this dual MV' is also used alternatively to generate prediction samples for the current block. Sub-pixel adjustments are then made based on the SAD cost around the motion with the minimum SAD to determine the sub-block motion with sub-pixel accuracy.

[0059] Multi-channel DMVR

[0060] In ECM, DMVR has been further developed to use multi-channel optimization. First, bilateral block matching is performed in the search region to find the refinement of the merged motion that provides the best match. Then, bilateral 16x16 sub-block matching is performed, followed by bidirectional optical flow for 8x8 sub-blocks to further refine the motion. Multi-channel DMVR also allows modification of only one dual-predictive motion. DMVR is also allowed when using BCW (with weights other than the average).

[0061] Template Matching (TM)

[0062] In ECM, block or sub-block motion can be refined by matching templates outside the current block with corresponding templates on a reference image. The search is limited to a small range to find better motion without signaling additional motion information. This can also be cascaded with bilateral matching in a multi-channel DMVR. Summary of the Invention

[0063] Specific challenges exist. For example, DMVR and methods that determine motion based solely on distortion within the current block (bilateral block matching) or distortion only outside the current block (template matching) risk obtaining inaccurate motion, leading to poor matching between the current block and surrounding blocks. Visual artifacts may appear at block (e.g., sub-block) boundaries and may also cause some sub-blocks to appear completely off-center from surrounding sub-blocks, due to completely incorrect sub-block motion in cases where the actual motion is too large or where one of the reference blocks is too smooth to find a good match.

[0064] Therefore, according to a first aspect of this disclosure, a method for obtaining a current block of motion-compensated prediction samples is provided. The method includes: selecting motion vectors from a set of candidate motion vectors, the set including a first candidate motion vector and a second candidate motion vector. The method includes: generating the block of motion-compensated prediction samples using the selected motion vectors, wherein selecting motion vectors from the set of candidate motion vectors includes: calculating a first misalignment distortion value CM_Dist1 for the first candidate motion vector. Calculating the first misalignment distortion value for the first candidate motion vector includes: calculating a boundary difference using n and c, and using the boundary difference to calculate the first misalignment distortion value, wherein n is a sample value from a first adjacent block adjacent to the current block and on the boundary between the first adjacent block and the current block, and c is a first candidate motion-compensated prediction sample value determined using the first candidate motion vector and located in the current block and on the boundary between the first adjacent block and the current block; and / or calculating a first difference d1 between n1 and n2; calculating a second difference d2 between c1 and c2; calculating d... The third difference d3 between d1 and d2; and using d3 to calculate the first misalignment distortion value, where n1 and n2 are sample values ​​from the first neighboring block and on the boundary between the first neighboring block and the current block, c1 is a first candidate motion compensation prediction sample value determined using the first candidate motion vector and located in the current block and on the boundary between the first neighboring block and the current block, and c2 is a second candidate motion compensation prediction sample value determined using the first candidate motion vector and located in the current block and on the boundary between the first neighboring block and the current block.

[0065] According to a second aspect of this disclosure, a computer program including instructions is provided that, when executed by processing circuitry, causes the processing circuitry to perform the method according to the first aspect.

[0066] According to a third aspect of this disclosure, a carrier comprising a computer program according to a second aspect is provided, wherein the carrier is one of an electrical signal, an optical signal, a radio signal, and a computer-readable storage medium.

[0067] According to a fourth aspect of this disclosure, an apparatus configured to perform the method according to the first aspect is provided.

[0068] Compared to methods that derive motion on the decoder side (e.g., in a DMVR), the embodiments disclosed herein have the advantage of providing more coherent subblocks and avoiding misaligned subblocks. In some cases, it can also provide better objective performance in terms of Bjontegaard incremental rate (BDR) (bit rate reduction at equivalent quality), as well as better subjective quality. Attached Figure Description

[0069] The accompanying drawings, which are incorporated herein and form part of this specification, illustrate various embodiments.

[0070] Figure 1 shows an example of a motion vector (MV);

[0071] Figure 2 illustrates single-frame prediction and two-frame prediction;

[0072] Figure 3 shows multiple score positions in the horizontal (x) dimension;

[0073] Figure 4 shows spatially adjacent blocks;

[0074] Figure 5 shows some examples of non-adjacent spatial blocks;

[0075] Figure 6 shows an example of the current block and its sub-blocks;

[0076] Figure 7 illustrates a system according to some embodiments;

[0077] Figure 8 shows an example of motion compensation values ​​for the current sub-block and its neighboring sub-blocks;

[0078] Figure 9 is a flowchart illustrating a process according to some embodiments;

[0079] Figure 10 is a schematic block diagram of an encoder according to an embodiment;

[0080] Figure 11 is a schematic block diagram of a decoder according to an embodiment;

[0081] Figure 12 is a block diagram of an apparatus according to some embodiments. Detailed Implementation

[0082] Figure 7 illustrates a system 700 according to an embodiment. System 700 includes an encoder 702 and a decoder 704, wherein the encoder 702 communicates with the decoder 704 via a network 110 (e.g., the Internet or another network). The encoder 702 encodes a source video sequence 701 (e.g., encodes block-sized image units of the video sequence 701) into a bitstream comprising an encoded video sequence (e.g., encoded blocks), and transmits the bitstream to the decoder 704 via a network 708. In some embodiments, the encoder 702 does not communicate with the decoder 704, and in such embodiments, instead of transmitting the bitstream to the decoder 704, the bitstream is stored in a data storage unit for later use. The decoder 704 decodes the images included in the encoded video sequence to generate video data for display and / or for further image processing (e.g., machine vision tasks). Therefore, the decoder 704 may be part of a device 703 having an image processor 705 and / or a display 706. The image processor 705 may perform machine vision tasks on the decoded images. Device 703 can be a mobile device, a set-top box device, a head-mounted display, or any other device.

[0083] As mentioned above, existing DMVR schemes may result in inaccurate motion, which leads to a poor match between the current block and its surrounding blocks.

[0084] Therefore, this disclosure provides a method for penalizing the misalignment of at least one boundary of the currently predicted sub-block relative to neighboring samples of adjacent sub-blocks. When determining the total distortion for each given candidate motion vector of the current sub-block, the penalty can be performed by taking into account the cumulative misalignment distortion (sometimes referred to as "sub-block boundary distortion"). Then, for a given range of motion, the candidate motion vector that minimizes the total distortion is selected and used to generate the sub-block containing the motion-compensated prediction samples. Any known method can be used to perform the generation of the motion-compensated prediction samples from the sub-block.

[0085] In one embodiment, cumulative misalignment distortion is based on misalignment distortion, which is based on the difference in properties (texture, sample activity variation) between the current sub-block boundary and the boundaries of adjacent sub-blocks. The difference in boundary properties is preferably phase-sensitive (motion-sensitive) so that not only is identical texture required, but also phase alignment is necessary. An example is calculating the pixel-by-pixel difference for pairs of samples that are both within the current sub-block and at the block boundary, and similarly for corresponding pairs of samples that are in adjacent sub-blocks. For example, misalignment distortion can be based on a value equal to the absolute value of ((N0-N1)-(C1-C2)), where N0 and N1 are samples along the block boundary at the bottom of the upper adjacent sub-block, and C1 and C2 are samples along the block boundary at the top of the current block. Any difference indicates some kind of texture misalignment between the current sub-block and its adjacent sub-blocks. That is, to obtain low distortion, the same relationship needs to be obtained between sample values ​​C0 and C1 as well as between N0 and N1. If the sample value between N0 and N1 increases, then the sample value between C0 and C1 should also increase.

[0086] When boundary samples are close to a constant value (low spatial activity), the characteristic difference between the samples at the current sub-block boundary and those at the boundaries of adjacent sub-blocks cannot distinguish the correct phase / motion. In this case, a constant difference may still exist between sub-block boundaries, which can lead to visual artifacts. In this situation, another misalignment distortion criterion that penalizes any constant difference between boundary samples can be advantageous. Therefore, in another embodiment, misalignment distortion is calculated based on the pixel-wise difference between at least one sample at the block boundary but within the current sub-block and its neighboring sample in the adjacent sub-block. Any difference indicates some degree of misalignment between the current sub-block and its neighboring sub-blocks.

[0087] The embodiments disclosed herein can be used only at the encoder, or they can be used at both the encoder and the decoder. The embodiments can be used for both blocks and sub-blocks. The embodiments described below are primarily for sub-blocks, but similar methods can generally be used for blocks. For samples, the embodiments primarily refer to predicted samples from previously coded images that have undergone motion compensation. The predicted samples used in the embodiments can be samples after double prediction, or samples from separate unidirectional predictions performed before double prediction. Adjacent samples can also be reconstructed samples.

[0088] Additional details

[0089] Determining the motion-compensated prediction sample for the current sub-block (e.g., the current input sub-block corresponding to the current region of the image) can involve the following three steps:

[0090] Step 1: Determine whether the current sub-block has adjacent sub-blocks that are adjacent to one of the boundaries of the current sub-block.

[0091] Step 2: As a result of determining that the neighboring sub-block is adjacent to one of the boundaries of the current sub-block, then for each candidate motion vector (MV) pair of the current sub-block (e.g., for each candidate MV offset used to derive the MV pair): i) determine at least two samples from the neighboring sub-block (these samples may be predicted or reconstructed samples of the neighboring sub-block), ii) determine at least two samples from the current sub-block based on the candidate MV pair (e.g., based on the motion vector offset), iii) calculate the cumulative misalignment distortion for the candidate MV pair based on these samples.

[0092] Step 3: Select one of the candidate MV pairs based on the distortion criterion (e.g., select the candidate MV offset with the minimum total distortion, where the total distortion is a function of the cumulative misalignment distortion).

[0093] For example, in one embodiment, selecting candidate MV offsets includes: i) for each candidate MV offset in the candidate MV offset set, determining a total distortion value (denoted as "T_Dist") for the MV offset, where the total distortion value is a function of the current block distortion (denoted as "CB_Dist") (also referred to as block matching distortion) and the cumulative misalignment distortion (denoted as "CM_Dist"); ii) selecting the candidate with the lowest T_Dist from the candidate set. MV offsets can be vectors (e.g., vectors of length 2 (also referred to as 2-tuples)) or scalars. For example, in one embodiment, each candidate MV offset is a 2-tuple that includes a first value (xOffset) specifying a horizontal offset (also referred to as the horizontal component) and a second value (yOffset) specifying a vertical offset (also referred to as the vertical component).

[0094] Step 4: Use the selected candidate to generate motion-compensated prediction samples. For example, if the selected MV candidate is an MV offset, at least one MV can be derived from the selected MV offset and the "default" MV, and this at least one derived MV is used in the process of generating motion-compensated prediction samples. For example, a pair of MVs can be derived from the selected MV offset and the default MV or MV pair, and this pair of derived MVs is used to generate motion-compensated prediction samples.

[0095] Misalignment distortion indicates the difference in boundary properties (texture, sample activity variations) on either side of a sub-block boundary. This difference can be expressed as the sum of absolute differences in boundary properties across the sub-block boundary. The distortion measure is preferably sensitive to phase / movement in directions parallel to the boundary, but insensitive to variations perpendicular to the boundary. In this way, minimizing distortion preserves the natural structure / details parallel to and across the sub-block boundary.

[0096] An example of the boundary characteristic difference for a horizontal sub-block boundary is the sum of the absolute differences of the sample differences across the horizontal boundary, and for a vertical sub-block boundary, it is the sum of the absolute differences of the sample differences across the vertical sub-block boundary. At least two samples at the boundary of the current sub-block and two neighboring samples from adjacent sub-blocks are used to calculate the sample difference on the corresponding side of the boundary.

[0097] If the boundary samples are close to constant values ​​(e.g., with low spatial activity), misalignment may still exist between the current sub-block and its neighboring sub-blocks. This can be accounted for in the cumulative misalignment distortion by including a metric that measures the constant difference between the boundary samples. An example metric is the sample difference across the sub-block boundary. At least one sample at the boundary of the current sub-block and a neighboring sample in an adjacent sub-block can be used to calculate the sample difference across the boundary.

[0098] In one embodiment, at least two distinct distances between samples are used to calculate cumulative misalignment distortion, such as short distances between samples and longer distances between samples. For example, the sample difference between neighboring samples and the sample difference between non-neighboring samples. The sample difference between samples closer to each other focuses on high-frequency variations, while the sample difference between samples with longer distances focuses on low-frequency variations. Both can be used to determine misalignment distortion.

[0099] In some embodiments, lower-frequency variations are given higher importance, such as differences between samples with greater distances between them. This is because misalignment of low-frequency samples is more visible than misalignment of high-frequency samples, as human vision is more sensitive to low-frequency artifacts.

[0100] In some embodiments, cumulative misalignment distortion is calculated using the sample differences between samples that are equally spaced and do not overlap. As an example of sample differences that are equally spaced, do not overlap, and cover samples P0, P1, P2, and P3, one set is P0-P1, and another non-overlapping set with the same sample distance is P2-P3. Examples of overlapping sets covering the same range would be P0-P1, P1-P2, and P2-P3. Compared to the case where samples are equally spaced and overlap, the case where samples are equally spaced and do not overlap allows for covering a larger sample range with fewer sets.

[0101] Example for calculating cumulative misalignment distortion (CM_Dist)

[0102] In one embodiment, the CM_Dist used for the candidate MV is based on i) the misalignment distortion (MD) (hMD) at the horizontal (top or bottom) boundary between the current sub-block and the top / bottom sub-block (denoted as "B"), and ii) the misalignment distortion (MD) (vMD) at the vertical boundary between the current sub-block and the left / right sub-block (denoted as "D"): CM_Dist = SF*(hMD + vMD), where SF > 0 (Equation 1)

[0103] In another embodiment: CM_Dist = SF1*hMD + SF2*vMD, where SF1 > 0 and SF2 > 0 (Equation 2).

[0104] In one embodiment, vMD = vMD1 + vMD2 + vMD3 + vMD4, and hMD = hMD1 + hMD2 + hMD3 + hMD4.

[0105] Using Figure 8 as an example, in one embodiment, hMD1, hMD2, hMD3, and hMD4 are calculated as follows:

[0106] In this particular example, B is the sub-block above the current sub-block, and the size of the sub-block is MxN, and A is a set of candidate motion compensation prediction sample values ​​generated using candidate motion vectors (e.g., using one or two candidate motion vectors derived from candidate motion vector offsets).

[0107] Using the example shown in Figure 8 again, in one embodiment, vMD1, vMD2, vMD3, and vMD4 are calculated as follows:

[0108] In this particular example, D is the child block to the left of the current child block.

[0109] This metric can also be used for other block and sub-block sizes, where M and N are multiples of 8. For example, 8xN, Mx8, 16xN, Mx16, 32xN, Mx32, 64xN, Mx64, 128xN, Mx128, where N and M equal 8, 16, 32, 64, or 128. For smaller blocks or sub-blocks (e.g., 4x4), the first three metrics (vMD1, vMD2, vMD3 and hMD1, hMD2, hMD3) can be used.

[0110] In alternative embodiments, right and bottom block or sub-block boundaries may also be considered in place of left and top boundaries or as a supplement to left and top boundaries.

[0111] Regarding the scaling factor (SF) in Equation 1 above, in one embodiment, when the sub-block misalignment distortion is based on a total of 19*2=38 spatial differences, SF=8, which corresponds to 38*8. This is close to the number of samples (256) derived for a block-matching motion for a 16x16 sub-block size. This means that the importance of misalignment distortion is similar to the importance of block matching with 256 samples. More generally, SF=N*M / number of spatial differences is used. In one example, the number of spatial differences for 4 sets is (M-2) / 2+(M-2) / 2-1+M / 4+M / 8+(M-2) / 2+(N-2) / 2-1+N / 4+N / 8. For an 8x8 sub-block, SF will be 8x8 / 16=4, for a 4x4 sub-block, SF will be 4x4 / 4=4, and for a 32x32 sub-block, SF will be 32x32 / 82=12.

[0112] In some embodiments, if the current sub-block has only one neighboring sub-block to its left or above its top, the SF is smaller because in this case, the accurate misalignment distortion can only be determined vertically or horizontally, not simultaneously. In this case, an example SF is SF=2, which corresponds to 19*2. This means that the importance of the misalignment distortion is approximately 1 / 6 (19*2 / 256) of the importance of a block match with 256 samples. This typically means that SF is (N*M / 6) / number of spatial differences, where examples of the number of spatial differences are given by vMD1, vMD2, vMD3, and vMD4 if the current sub-block has only one neighboring sub-block to its left, and by hMD1, hMD2, hMD3, and hMD4 for a single neighboring sub-block above the current sub-block, as described above.

[0113] As shown in Equation 2, separate scaling factors can also be applied to the corresponding misalignment distortions (left or top). Thus, the left sub-block misalignment distortion has a scaling factor SFleft, while the top sub-block misalignment distortion has a scaling factor SFtop. One example of having separate scaling factors for different boundaries (e.g., top and left sub-block boundaries) is to make them dependent on the spatial activity of the boundary samples, such that boundaries with spatial activity less than th2 are given a higher scaling factor compared to boundaries with spatial activity equal to or greater than a threshold (th2), because regions with low spatial activity are generally more visually sensitive to artifacts. Alternatively, if both the top and left sub-block boundaries have samples with low spatial activity, the scaling factor for the current block is increased.

[0114] In one embodiment, different importance levels are assigned to boundary characteristics determined by sample differences measured using different distances between samples.

[0115] Consider four sets of sample activity differences, each set comprising a set of sample activities computed using the same distances between samples. Each set is then given a local scaling factor (f) to control the importance of the corresponding set. That is, for example,

[0116] hMD = f1*hMD1+f2*hMD2+f3*hMD3+f4*hMD4, and

[0117] vMD = f1*vMD1+f2*vMD2+f3*vMD3+f4*vMD4.

[0118] The scaling factor fi (i=1, 2, 3, 4) can be implemented through multiplication or by right or left shifting the corresponding unaligned distortion. Multiplying by a scaling factor greater than 1 will increase the importance of the set, while scaling factors less than 1 will decrease the importance of the set. Left shifting (e.g., hMD1 << f1) will similarly increase importance, while right shifting (e.g., hMD1 >> f1) will decrease importance.

[0119] For example, f1 and f2 can be 1, which would mean that no scaling factor is needed for hMD1, hMD2, vMD1, and vMD2. f3 can be multiplied by 2 or shifted left by 1, thus doubling the importance of the corresponding hMD3 and vMD3. f4 can be multiplied by 4 or shifted left by 2.

[0120] One reason for controlling importance could be to balance the influence of different sets. For example, if hMD1 and hMD2 correspond to 8 absolute differences in boundary characteristics, hMD3 to 4, and hMD4 to 2, then applying a scaling factor of 2 for f3 and a scaling factor of 4 for f4 will make all sets have the same effect on the total boundary distortion hMD1 + hMD2 + 2*hMD3 + 4*hMD4 (the same applies to vMD). Another reason could be to prioritize low-frequency boundary characteristic differences over high-frequency boundary characteristic differences.

[0121] Low-space activity examples

[0122] When spatial activity is low (or the proxy for spatial activity is low), the utility of using texture (boundary property) difference is low because it cannot detect the DC / constant difference between the boundary samples of adjacent sub-blocks and the current sub-block. Therefore, in one embodiment, if the spatial activity on the boundary samples of at least the adjacent sub-block boundary samples is less than a threshold (th3), the difference between the boundary samples of adjacent sub-blocks and the current sub-block sample is used as part of the misalignment distortion.

[0123] One example is when the spatial activity in adjacent boundary samples is less than th3, only the difference between boundary samples is used.

[0124] In other words, for example, instead of hMD being equal to (hMD1 + hMD2 + hMD3 + hMD4) (or simply hMD_a), when the space activity measurement is less than th3, hMD is calculated as:

[0125] hMD = fb * hMD_b, where fb is the scaling factor and hMD_b is the boundary difference.

[0126] Otherwise, when the spatial activity measurement is not less than th3, hMD equals hMD_a (i.e., hMD = hMD1 + hMD2 + hMD3 + hMD4). In other words, misalignment distortion is based solely on boundary characteristic differences.

[0127] Similarly, instead of vMD being equal to (vMD1 + vMD2 + vMD3 + vMD4) (or simply vMD_a), vMD is calculated as follows when the space activity measurement is less than th3:

[0128] vMD = fb*vMD_b, where fb is the scaling factor and vMD_b is the boundary difference.

[0129] Otherwise, when the spatial activity measurement is not less than th3, vMD equals vMD_a (i.e., vMD1 + vMD2 + vMD3 + vMD4). In other words, misalignment distortion is based solely on boundary characteristic differences.

[0130] In one embodiment, ,as well as .

[0131] Note that in this case, the boundary differences are summed before taking the absolute value. Since the focus here is on capturing the constant difference between the current sample and its neighbors, this is sufficient, as long as other complementary boundary feature differences are used, since these complementary boundary feature differences will capture the non-constant difference. It would also be possible to take the absolute value in each individual case, but this would increase complexity. When calculating misalignment distortion also via boundary feature differences, a small number of boundary differences (but at least one) is usually sufficient.

[0132] An alternative way to calculate the boundary difference is to use the Laplace operator, which is insensitive to the slope across the sub-block boundary (linearly increasing or decreasing sample 1 2 3) and only gives a non-zero difference for a constant difference across the sub-block boundary (1 1 2), which is determined for the top boundary between the adjacent sub-block A and the current sub-block B as:

[0133] And for the left boundary between sub-blocks D and B, it is determined as: .

[0134] Note that in this case, the boundary differences are summed before taking the absolute value. Since the focus here is on capturing the constant difference between the current sample and its neighbors, this is sufficient, as long as other complementary boundary feature differences are used, since these complementary boundary feature differences will capture the non-constant difference. It would also be possible to take the absolute value in each individual case, but this would increase complexity. When calculating misalignment distortion also via boundary feature differences, a small number of boundary differences (but at least one) is usually sufficient.

[0135] In another embodiment, when the spatial activity value in adjacent boundary samples is less than th3, the difference between boundary samples is added to the misalignment distortion based on the boundary characteristic difference; otherwise, the misalignment distortion is based solely on the boundary characteristic difference. That is, for example:

[0136] Measurement of space activities

[0137] As a proxy for spatial activity in neighboring samples, the boundary characteristic difference used for boundaries can be used. This can be useful if the boundary characteristic difference is calculated anyway. Note that this does not measure the spatial activity itself, but if there is good alignment based on the boundary characteristics, the difference can only be a constant value that cannot be detected by the boundary characteristics.

[0138] For the top boundary between adjacent sub-block A and the current sub-block B, the misalignment distortion can be given as follows:

[0139] For the left boundary between adjacent sub-block D and the current sub-block B, the misalignment distortion can be given as follows:

[0140] Spatial activity is measured using only boundary properties based on neighboring boundary samples. This is also relatively simple if the boundary property differences are calculated anyway. Spatial activity for neighboring samples of sub-block B can be based on: saH = saTop = saTop1 + saTop2 + saTop3 + saTop4

[0141] Here, B is the child block above the current child block, and the size of the child block is MxN.

[0142] Spatial activity of neighboring samples of sub-block D can be based on: saV = saLeft = saLeft1 + saLeft2 + saLeft3 + saLeft4

[0143] Where D is the sub-block to the left of the current sub-block.

[0144] Alternatively, spatial activity for neighboring samples can be based on:

[0145] Alternatively, an average spatial activity value can be taken for the aforementioned spatial activities to reduce sensitivity to noise. For example, for the case where M=N=16:

[0146] More generally, a right shift of 4 (similar to division by 16) can be given by log2(M) and log2(N).

[0147] Alternatively, an exact average could be calculated, but this is more complex. Since M-1 and N-1 spatial differences are calculated separately in this example, the exact average is given by dividing by M-1 and N-1 respectively.

[0148] Alternatively, spatial activity for neighboring samples can be based on:

[0149] Among them, C M-1,N-1 It is the lower right sample in the upper left sub-block C of the current block A. It is included to avoid aligning the boundary samples of the current sub-block and the adjacent sub-block B when there is a real edge between sub-blocks C and D, and similarly to avoid aligning the boundary samples of the current sub-block and the adjacent sub-block D when there is a real edge between sub-blocks C and B.

[0150] Alternatively, spatial activity for neighboring samples can be based on:

[0151] Alternatively, spatial activity for neighboring samples can be based on: And saLeft = saTop

[0152] This includes considering spatial activities around the current block above, to the left, and to the upper left.

[0153] The method described in other embodiments can be determined for horizontal and / or vertical sub-block boundaries when the criterion for the absolute difference between the motion of the current sub-block and the motion of the upper or left adjacent sub-block is greater than or equal to a threshold (th1). An example th1 is 8 when the motion is given with 16-pixel precision (e.g., motion corresponding to 1 sample).

[0154] When the criterion based on true edge detection is less than the true edge threshold, the method described in other embodiments can be determined for horizontal sub-block boundaries and / or vertical sub-block boundaries. True edge checking for the top horizontal sub-block boundary can be determined as abs(C). M-1,N-1 – D M-1,0 ), and the true edge check for the left vertical sub-block boundary can be determined as abs(C M-1,N-1 – B 0,N-1 ).

[0155] The method described in other embodiments can be determined for horizontal and / or vertical sub-block boundaries when the criterion for determining spatial activity for at least one reference sub-block centered on the default / initial MV is below a threshold. In another embodiment, the reference sub-block may be identified by candidate MVs. If the spatial activity is below the threshold (e.g., there is a probability of unreliable motion determination), one of the above embodiments is implemented; otherwise, it is not implemented.

[0156] Figure 9 is a flowchart illustrating a process 900 for obtaining a current block of motion-compensated prediction samples (e.g., obtaining an inter-frame prediction block for the input block) according to some embodiments. On the encoder side, the inter-frame prediction block and the input block are used to obtain a residual block (the residual block represents the difference between the prediction block and the input block). On the encoder and decoder sides, the inter-frame prediction block is added to the reconstructed residual block to derive the reconstructed block.

[0157] Process 900 can begin from step s902.

[0158] Step s902 includes performing a process for selecting motion vectors (e.g., motion vector offset pairs) (e.g., horizontal component (x) and vertical component (y)) from a candidate motion vector set (e.g., an offset tuple set), the candidate motion vector set including a first candidate motion vector and a second candidate motion vector.

[0159] Step s904 includes generating blocks of motion-compensated prediction samples using the selected motion vectors. Step s904 uses well-known methods for generating motion-compensated prediction samples. For example, if one of the motion-compensated prediction samples is represented as "A(x,y)", then step s904 may include calculating: A(x,y) = (ref0(x,y) + ref1(x,y) + 1) >> 1, where ref0 and ref1 are reference blocks derived as:

[0160] ref0(x,y) = refP0(xPic + x + MvX + OffsetX,yPic + y + MvY + OffsetY),

[0161] ref1(x,y) = refP1(xPic + x + MvX - OffsetX,yPic + y + MvY - OffsetY),

[0162] x and y are the positions within the current block.

[0163] xPic and yPic are in the image coordinates and are used to locate the current block (e.g., to locate the current block within the image or another block).

[0164] MvX and MvY are the horizontal and vertical components of the default motion vector.

[0165] OffsetX is the horizontal component of the candidate MV offset.

[0166] OffsetY is the vertical component of the candidate MV offset.

[0167] MvX + OffsetX, MvY + OffsetY, MvX – OffsetX, and MvY – OffsetY can use integer sample precision, or fractional sample precision. However, to maintain low complexity, fractional sample precision is preferably rounded to integer sample precision; otherwise, interpolation of samples in the reference image is required to obtain motion compensation prediction samples ref0 and / or ref1. For example, if the motion (mv + offset) has 1 / 16 fractional precision, integer precision can be obtained by using mvFrac >> 4, so the 16 in 1 / 16 precision becomes 1 in integer precision.

[0168] refP0 is the first reference image, and

[0169] refP1 is the second reference image.

[0170] In another example, A(x,y) = (ref0(x,y)*a + ref1(x,y)*b + r) >> s, where, for example, a = 1, b = 7, r = 4 and s = 3. For example, the specific weights a, b, r and s can be derived from BCW, for example.

[0171] Figure 10 illustrates the functional components of encoder 702 according to some embodiments. It should be noted that encoders can be implemented in different ways, and therefore implementations other than this particular example may be used. Encoder 702 employs subtractor 241 to generate a residual block, which is the sample value difference between the input block and the prediction block (i.e., the output of selector 251, which is either an inter-prediction block output by inter-frame predictor 250 (also referred to as a motion compensator) or an intra-prediction block output by intra-frame predictor 249). A forward transform 242 is then performed on the residual block to generate a transform block including transform coefficients. Quantization unit 243 quantizes the transform coefficients based on quantization parameter (QP) values ​​(e.g., QP values ​​obtained based on a picture QP value for an image in which the input block is a part and a block-specific QP offset value for the input block), thereby generating quantized transform coefficients. The quantized transform coefficients are then encoded into a bitstream by encoder 244 (e.g., an entropy encoder), and the bitstream with the encoded transform coefficients is output from encoder 702. Next, encoder 702 uses quantized transform coefficients to generate a reconstructed block. This is done by first applying inverse quantization 245 and inverse transform 246 to the transform coefficients to generate a reconstructed residual block, and then using adder 247 to add the predicted block to the reconstructed residual block to generate a reconstructed block, which is stored in reconstructed picture buffer (RPB) 266. Loop filtering of loop filter (LF) stage 267 is applied, and the final decoded picture is stored in decoded picture buffer (DPB) 268. Inter-frame predictor 250 can then use this decoded picture in DPB 268 to generate inter-frame prediction blocks for the next picture to be processed. LF stage 267 may include three sub-stages: i) a deblocking filter, ii) a sample adaptive offset (SAO) filter, and iii) an adaptive loop filter (ALF).

[0172] Figure 11 illustrates the functional components of a decoder 704 according to some embodiments. It should be noted that the decoder 704 can be implemented in different ways, and therefore implementations other than this particular example can be used. The decoder 704 includes a decoder module 361 (e.g., an entropy decoder) that decodes the quantization transform coefficient values ​​of blocks from the bitstream. The decoder 704 also includes a reconstruction stage 398, where the quantization transform coefficient values ​​undergo an inverse quantization process 362 and an inverse transform process 363 to produce a residual block. This residual block is input to an adder 364, which adds the residual block to a prediction block output from a selector 390 to form a reconstructed block. The selector 390 selects to output either an inter-frame prediction block or an intra-frame prediction block. The reconstructed block is stored in an RPB 365. The inter-frame prediction block is generated by an inter-frame prediction module 350, while the intra-frame prediction block is generated by an intra-frame prediction module 369. After reconstruction stage 398, loop filter stage 367 applies loop filtering, and the final decoded image can be stored in decoded image buffer (DPB) 368 and output to image processor 105. The image is stored in DPB for two main reasons: 1) to await image output, and 2) to be used as a reference when decoding future images.

[0173] Figure 12 is a block diagram of an apparatus 1200 for implementing encoder 702 and / or decoder 704 according to some embodiments. When apparatus 1200 implements encoder 702, apparatus 1200 may be referred to as encoder apparatus, and when apparatus 1200 implements decoder 704, apparatus 1200 may be referred to as decoder apparatus. As shown in Figure 12, apparatus 1200 may include: processing circuitry (PC) 1202, which may include one or more processors (P) 1255 (e.g., one or more general-purpose microprocessors and / or one or more other processors, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), which may be co-located in a single housing or a single data center, or may be geographically distributed (i.e., apparatus 1200 may be a distributed computing apparatus); at least one network interface 1248 (e.g., physical interface or air interface), which includes transmitter (Tx) 1245 and A receiver (Rx) 1247 enables device 1200 to send and receive data to and from other nodes connected to network 110 (e.g., an Internet Protocol (IP) network); a network interface 1248 is (physically or wirelessly) connected to network 110 (e.g., network interface 1248 may be coupled to an antenna device including one or more antennas to enable device 1200 to wirelessly send / receive data); and a storage unit (also referred to as a “data storage system”) 1208, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 1202 includes a programmable processor, a computer-readable storage medium (CRSM) 1242 may be provided. CRSM 1242 may store a computer program (CP) 1243 including computer-readable instructions (CRI) 1244. CRSM 1242 may be a non-transitory computer-readable medium, such as a magnetic medium (e.g., a hard disk), an optical medium, a storage device (e.g., random access memory, flash memory), etc. In some embodiments, the CRI 1244 of the computer program 1243 is configured such that, when executed by the PC 1202, the CRI causes the device 1200 to perform the steps described herein (e.g., the steps described herein with reference to the flowcharts). In other embodiments, the device 1200 may be configured to perform the steps described herein without requiring code. That is, for example, the PC 1202 may comprise only one or more ASICs. Therefore, the features of the embodiments described herein may be implemented in hardware and / or software.

[0174] Overview of various embodiments (as used in the embodiments below, the term "block" is intended to be understood broadly to refer to a block or sub-block.)

[0175] A1. A method for obtaining a current block (e.g., an inter-frame prediction block, for obtaining a reconstructed block by adding the inter-frame prediction block and the reconstructed residual block) of motion-compensated prediction samples, the method comprising:

[0176] Perform a process for selecting motion vectors (e.g., motion vector offsets) from a set of candidate motion vectors (e.g., an offset set), the candidate motion vector set including a first candidate motion vector and a second candidate motion vector; and

[0177] The selected motion vectors are used to generate blocks of motion-compensated prediction samples (e.g., to generate inter-frame prediction blocks), where...

[0178] The process for selecting motion vectors from the candidate motion vector set includes: calculating a first unaligned distortion value (denoted as CM_Dist1) for the first candidate motion vector, and

[0179] Calculating the first misalignment distortion value for the first candidate motion vector includes:

[0180] The boundary difference is calculated using n and c, and the first misalignment distortion value is calculated using the boundary difference.

[0181] n is a sample value that comes from the first neighboring block adjacent to the current block, and on the boundary between the first neighboring block and the current block.

[0182] c is the first candidate motion compensation prediction sample value, which is determined using the first candidate motion vector and is located in the current block and on the boundary between the first neighboring block and the current block; and / or

[0183] Calculate the first difference (d1) between n1 and n2; calculate the second difference (d2) between c1 and c2; calculate the third difference (d3) between d1 and d2 (e.g., d3 = d1 - d2, or d3 = d2 - d1); and use d3 to calculate the first misalignment distortion value, where,

[0184] n1 and n2 are sample values ​​that come from the first neighboring block and are located on the boundary between the first neighboring block and the current block.

[0185] c1 is the first candidate motion compensation prediction sample value, which is determined using the first candidate motion vector, and is located in the current block and on the boundary between the first adjacent block and the current block.

[0186] c2 is the second candidate motion compensation prediction sample value, which is determined using the first candidate motion vector and is located in the current block and on the boundary between the first adjacent block and the current block.

[0187] A2. The method according to embodiment A1, wherein using n and c to calculate the boundary difference includes: calculating nc or calculating cn.

[0188] A3. The method according to embodiment A1, wherein calculating the boundary difference using n and c includes: calculating n² – 2*n + c, where,

[0189] n2 is a sample value that comes from the first neighboring block adjacent to the current block and is not on the boundary between the first neighboring block and the current block.

[0190] A4. The method according to any one of embodiments A1-A3, wherein,

[0191] The candidate motion vector (MV) set includes the candidate MV offset set.

[0192] Selecting the motion vector for the current block involves selecting the MV offset for the current block from the set of candidate MV offsets.

[0193] A5. The method according to any one of embodiments A1-A4, wherein,

[0194] The process for selecting motion vectors from the set of candidate motion vectors also includes:

[0195] The first distortion value (e.g., T-Distortion value) is calculated using the first unaligned distortion value (CM_Dist1).

[0196] Calculate the second distortion value for the second candidate motion vector;

[0197] The first distortion value is compared with the second distortion value to determine which one is lower.

[0198] A6. The method according to embodiment A5, wherein the first distortion value T-Distortion-1 for the first candidate MV is a function of i) the current block distortion value (referred to as "CurrentDistortion") and ii) the first unaligned distortion value CM_Dist1.

[0199] A7. The method according to embodiment A6, wherein T-Distortion-1 = CurrentDistortion +CM_Dist1.

[0200] A8. The method according to embodiment A7, wherein CM_Dist1 is a function of the following:

[0201] Vertical misalignment distortion value vMD and / or horizontal misalignment distortion value hMD (e.g., CM_Dist = SF(vMD + hMD), or CM_Dist = SF1*vMD + SF2*hMD, where SF, SF1, and SF2 are pre-configured factors).

[0202] A9. The method according to embodiment A8, wherein,

[0203] CM_Dist1 is a function of hMD.

[0204] hMD is a function of hMD_a and / or hMD_b.

[0205] hMD_a is a function of the first level of misaligned distortion (hMD1) and the second level of misaligned distortion (hMD2), and

[0206] hMD_b is the boundary difference.

[0207] A10. The method according to embodiment A9 further includes: determining CM_Dist1, wherein determining CM_Dist1 includes:

[0208] Determine whether the first condition is met (e.g., determine whether the spatial activity value is less than a threshold, or determine whether hMD_a is less than a threshold), and

[0209] As a result of determining that the first condition is met, hMD is set to be equal to hMD_a, or hMD is set to be equal to hMD_b, or hMD is set to be equal to hMD_b + hMD_a.

[0210] A11. The method according to embodiment A9 or A10, wherein,

[0211] hMD_a is a function of hMD1, hMD2, hMD3, and hMD4, where, as well as

[0212] hMD_b is i) or ii) The function, and

[0213] A is the set of candidate motion compensation prediction sample values ​​generated using the first candidate motion vector, and

[0214] B is the block above the current block, and the size of the block is M x N.

[0215] A12. The method according to any one of embodiments A8-A11, wherein,

[0216] CM_Dist1 is a function of vMD.

[0217] vMD is a function of vMD_a and / or vMD_b.

[0218] vMD_a is a function of the first vertical misalignment distortion value (vMD1) and the second vertical misalignment distortion value (vMD2), and

[0219] vMD_b is the boundary difference.

[0220] A13. The method according to embodiment A12 further includes: determining CM_Dist1, wherein determining CM_Dist1 includes:

[0221] To determine whether the second condition is met, as a result of determining that the second condition is met, set vMD to be equal to vMD_a, or set vMD to be equal to vMD_b, or set vMD to be equal to vMD_b + vMD_a.

[0222] A14. The method according to embodiment A12 or A13, wherein,

[0223] vMD_a is a function of vMD1, vMD2, vMD3, and vMD4, where,

[0224] vMD_b is i) or ii) The function, and

[0225] A is the set of candidate motion compensation prediction sample values ​​generated using the first candidate motion vector, and

[0226] D is the block to the left of the current block, and the size of the block is M x N.

[0227] A15. The method according to any one of embodiments A5-A14, wherein,

[0228] Calculating the second distortion value for the second candidate motion vector includes determining whether the second distortion value should be determined using the second misalignment distortion value (CM_Dist2).

[0229] Determining whether the second distortion value should be used with the second misaligned distortion value (CM_Dist2) includes:

[0230] Determine whether the space activity value (e.g., surrogate value) used for the first reference block meets a first criterion (e.g., determine whether the space activity is less than a first threshold).

[0231] Determine whether i) the first absolute difference between the first component of the second candidate MV and ii) the corresponding component of the MV of the first neighboring block (e.g., the left neighbor or the top neighbor) satisfies the second criterion (e.g., whether the absolute difference is greater than a second threshold); and / or

[0232] Determine whether the boundary between the current block and its neighboring blocks is a true edge.

[0233] A16. The method according to embodiment A15, wherein,

[0234] Determining whether the second distortion value should be used to determine the second misalignment distortion value (CM_Dist2) includes:

[0235] Determine whether the first absolute difference between i) the first component of the second candidate MV and ii) the corresponding component of the MV of the first neighboring block (e.g., the left neighbor or the top neighbor) satisfies the second criterion (e.g., whether the absolute difference is greater than a second threshold); and

[0236] Determine whether the second absolute difference between i) the first component of the second candidate MV and ii) the corresponding component of the MV of the second adjacent block (if the first adjacent block is a neighbor, then it is the left neighbor; if the first adjacent block is a left neighbor, then it is the top neighbor) satisfies a second criterion (e.g., determining whether the absolute difference is greater than a second threshold), wherein,

[0237] If either the first absolute difference or the second absolute difference meets the second criterion, then the second distortion value should be determined using the second misalignment distortion value.

[0238] A17. The method according to embodiment A11 or A14, wherein A is derived based on CPB1 and / or CPB2, wherein CPB1 is a first candidate prediction block (e.g., candidate inter-frame prediction block) derived using a first candidate MV, and CPB2 is a second candidate prediction block derived using the first candidate MV.

[0239] A18. The method according to embodiment A17, wherein,

[0240] A equals the average of CPB1 and CPB2.

[0241] A equals the weighted average of CPB1 and CPB2.

[0242] A equals CPB1, or

[0243] A equals CPB2.

[0244] B1. A computer program (1243) including instructions (1244) that, when executed by a processing circuit (1202), cause the processing circuit (1202) to perform the method according to any of the above embodiments.

[0245] B2. A carrier comprising a computer program according to embodiment B1, wherein the carrier is one of an electrical signal, an optical signal, a radio signal, and a computer-readable storage medium (1242).

[0246] C1. An apparatus (1200) for encoding an image, the apparatus being configured to perform a method comprising:

[0247] C2. The apparatus according to embodiment C1, wherein the apparatus is further configured to perform the method according to any one of embodiments A2-A18.

[0248] D1. An apparatus (1200) comprising:

[0249] Memory (1242); and

[0250] A processing circuit (1202) coupled to a memory (1242), wherein the device (1200) is configured to perform the method described according to any one of embodiments A1-A18.

[0251] in conclusion

[0252] Although various embodiments have been described herein, it should be understood that they are provided by way of example only and not as limiting. Therefore, the breadth and scope of this disclosure should not be limited to any of the exemplary embodiments described above. Furthermore, unless otherwise indicated herein or otherwise clearly contradicted by the context, this disclosure includes any combination of all possible variations of the foregoing elements.

[0253] Additionally, although the process described above and shown in the accompanying drawings is presented as a series of steps, this is for illustrative purposes only. Therefore, it is conceivable that some steps may be added, some steps may be omitted, the order of steps may be rearranged, and some steps may be performed in parallel.

[0254] References

[0255] [1] K. Andersson, R, Yu, “AHG10: Encoder MV selections and DMVR revisited”, JVET-AD0045, Antalya, Turkey, April 2023.

Claims

1. A method for obtaining the current block of motion-compensated prediction samples, the method comprising: A motion vector is selected from a set of candidate motion vectors, wherein the set of candidate motion vectors includes a first candidate motion vector and a second candidate motion vector. The block for generating motion-compensated prediction samples using the selected motion vectors, wherein selecting motion vectors from the set of candidate motion vectors includes: calculating a first misalignment distortion value CM_Dist1 for the first candidate motion vectors, and calculating the first misalignment distortion value for the first candidate motion vectors includes: calculating a boundary difference using n and c, and using the boundary difference to calculate the first misalignment distortion value, wherein n is a sample value that comes from a first neighboring block adjacent to the current block and on the boundary between the first neighboring block and the current block, and c is a first candidate motion-compensated prediction sample value that is determined using the first candidate motion vectors and is located in the current block and between the first neighboring block and the current block. On the boundary; and / or calculate a first difference d1 between n1 and n2; calculate a second difference d2 between c1 and c2; calculate a third difference d3 between d1 and d2; and use d3 to calculate the first misalignment distortion value, wherein n1 and n2 are sample values ​​from the first neighboring block and on the boundary between the first neighboring block and the current block, c1 is a first candidate motion compensation prediction sample value determined using the first candidate motion vector and located in the current block and on the boundary between the first neighboring block and the current block, and c2 is a second candidate motion compensation prediction sample value determined using the first candidate motion vector and located in the current block and on the boundary between the first neighboring block and the current block.

2. The method according to claim 1, wherein, Calculating the boundary difference using n and c includes calculating nc or calculating cn.

3. The method according to claim 1, wherein, Calculating the boundary difference using n and c includes: calculating n2 – 2*n + c, where n2 is a sample value that comes from the first neighboring block adjacent to the current block and is not on the boundary between the first neighboring block and the current block.

4. The method according to any one of claims 1-3, wherein, The candidate motion vector MV set includes a candidate MV offset set, and selecting a motion vector for the current block includes selecting an MV offset for the current block from the candidate MV offset set.

5. The method according to any one of claims 1-4, wherein, The process for selecting a motion vector from the set of candidate motion vectors further includes: calculating a first distortion value T-Distortion-1 using the first unaligned distortion value CM_Dist1; calculating a second distortion value for the second candidate motion vector; and comparing the first distortion value with the second distortion value to determine which one is lower.

6. The method according to claim 5, wherein, The first distortion value T-Distortion-1 for the first candidate MV is a function of i) the current block distortion value "CurrentDistortion" and ii) the first unaligned distortion value CM_Dist1.

7. The method according to claim 6, wherein, T-Distortion-1 = CurrentDistortion + CM_Dist1.

8. The method according to claim 7, wherein, CM_Dist1 is a function of the following: vertical misalignment distortion value vMD and / or horizontal misalignment distortion value hMD, CM_Dist = SF(vMD + hMD), or CM_Dist = SF1*vMD + SF2*hMD, where SF, SF1, and SF2 are pre-configured factors.

9. The method according to claim 8, wherein, CM_Dist1 is a function of hMD, hMD is a function of hMD_a and / or hMD_b, hMD_a is a function of the first level unaligned distortion value hMD1 and the second level unaligned distortion value hMD2, and hMD_b is the boundary difference.

10. The method of claim 9, further comprising: Determine CM_Dist1, wherein determining CM_Dist1 includes: determining whether a first condition is satisfied, and as a result of determining that the first condition is satisfied, setting hMD to be equal to hMD_a, or setting hMD to be equal to hMD_b, or setting hMD to be equal to hMD_b + hMD_a.

11. The method according to claim 9 or 10, wherein, hMD_a is a function of hMD1, hMD2, hMD3, and hMD4. in, And hMD_b is i) or ii) The function is defined as follows: A is a set of candidate motion compensation prediction samples generated using the first candidate motion vector; B is a block above the current block, and the size of the block is MxN.

12. The method according to any one of claims 8-11, wherein, CM_Dist1 is a function of vMD, vMD is a function of vMD_a and / or vMD_b, vMD_a is a function of the first vertical misaligned distortion value vMD1 and the second vertical misaligned distortion value vMD2, and vMD_b is the boundary difference.

13. The method of claim 12, further comprising: Determine CM_Dist1, wherein determining CM_Dist1 includes: determining whether the second condition is satisfied, and as a result of determining that the second condition is satisfied, setting vMD to be equal to vMD_a, or setting vMD to be equal to vMD_b, or setting vMD to be equal to vMD_b + vMD_a.

14. The method according to claim 12 or 13, wherein, vMD_a is a function of vMD1, vMD2, vMD3, and vMD4. in, vMD_b is i) or ii) The function is defined as follows: A is a set of candidate motion compensation prediction sample values ​​generated using the first candidate motion vector; D is a block to the left of the current block; and the size of the block is M x N.

15. The method according to any one of claims 5-14, wherein, Calculating the second distortion value for the second candidate motion vector includes: determining whether the second distortion value should be determined using the second unaligned distortion value CM_Dist2. Determining whether the second distortion value should be determined using the second unaligned distortion value CM_Dist2 includes: determining whether the spatial activity value used for the first reference block meets a first criterion; determining whether the first absolute difference between i) the first component of the second candidate MV and ii) the corresponding component of the MV of the first adjacent block meets a second criterion; and / or determining whether the boundary between the current block and the adjacent block is a true edge.

16. The method according to claim 15, wherein, Determining whether the second distortion value should be determined using the second unaligned distortion value CM_Dist2 includes: determining whether i) the first absolute difference between the first component of the second candidate MV and ii) the corresponding component of the MV of the first adjacent block satisfies the second criterion; and determining whether i) the second absolute difference between the first component of the second candidate MV and ii) the corresponding component of the MV of the second adjacent block satisfies the second criterion, wherein if either the first absolute difference or the second absolute difference satisfies the second criterion, then it is determined that the second distortion value should be determined using the second unaligned distortion value.

17. The method according to claim 11 or 14, wherein, A is derived based on CPB1 and / or CPB2, where CPB1 is a first candidate prediction block derived using the first candidate MV, and CPB2 is a second candidate prediction block derived using the first candidate MV.

18. The method according to claim 17, wherein, A equals the average of CPB1 and CPB2, A equals the weighted average of CPB1 and CPB2, A equals CPB1, or A equals CPB2.

19. A computer program (1243) comprising instructions (1244) that, when executed by a processing circuit (1202), cause the processing circuit (1202) to perform the method according to any one of the preceding claims.

20. A carrier comprising the computer program according to claim 19, wherein, The carrier is one of electrical signals, optical signals, radio signals, and computer-readable storage media (1242).

21. An apparatus configured to perform the method according to any one of claims 1-18.

Images