Method and apparatus for decoder-side motion vector correction in video coding

Abstract

To provide a method and an apparatus for a decoder side motion vector correction in video coding.SOLUTION: A method for video coding includes the steps of deriving an initial motion vector (MV) for a current coding unit (CU), deriving a plurality of motion vector (MV) candidates for decoder-side motion vector correction (DMVR), determining a cost value for each of the initial MV and the MV candidates, obtaining an updated cost value by adjusting at least one of the cost values to favor the initial MV, and deriving a corrected motion vector (MV) on the basis of the initial MV and the updated cost values of the MV candidates.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 62 / 862,674, entitled “Decoder-side Motion Vector Refinement for Video Coding,” filed June 17, 2019, which is incorporated by reference in its entirety for all purposes.

[0002] This application relates generally to video encoding and compression, and more particularly, but not exclusively, to a method and apparatus for Decoder-side Motion Vector Refinement (DMVR) in video encoding. [Background technology]

[0003] Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video gaming consoles, smartphones, video teleconferencing devices, video streaming devices, etc. The electronic devices transmit, receive, encode, decode, and / or store digital video data by performing video compression / decompression. The digital video devices implement video coding techniques, such as those described in standards defined by Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), and extensions to such standards.

[0004] Video coding generally employs prediction methods (e.g., inter-prediction, intra-prediction) that exploit redundancy present in a video image or sequence. An important goal of video coding techniques is to compress video data into a format that uses a lower bit rate while avoiding or minimizing degradation of video quality. As more advanced video services become available, encoding techniques that provide better coding efficiency are needed.

[0005] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video coding, a video frame is divided into one or more slices, each having multiple video blocks, which may also be referred to as coding tree units (CTUs). The CTUs may be divided into coding units (CUs) using a quadtree with nested types of tree structures. A CU defines a region of pixels that share the same prediction mode. Each CTU may contain one coding unit (CU) or may be recursively divided into smaller CUs until a predetermined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more transform units (TUs), which in turn contain one or more prediction units (PUs). Each CU may be coded in intra mode, inter mode, or IBC mode. Video blocks in an intra-coded (I) slice of a video frame are coded using spatial prediction with respect to reference samples in neighboring blocks in the same video frame. Video blocks within an inter-coded (P or B) slice of a video frame may use spatial prediction with respect to reference samples in neighboring blocks in the same video frame, or temporal prediction with respect to reference samples in other previous and / or future reference video frames.

[0006] In some examples of the present disclosure, the term "unit" defines a region of an image containing all components such as luma and chroma, and the term "block" is used to define a region containing a particular component (e.g., luma), and taking into account a chroma sampling scheme such as 4:2:0, blocks of different components (e.g., luma vs. chroma) may have different spatial locations.

[0007] A prediction block for a current video block to be coded is obtained from a previously coded reference block, e.g., spatial or temporal prediction based on neighboring blocks. The process of finding the reference block can be realized by a block matching algorithm. Residual data representing pixel differences between the current block to be coded and the prediction block is called a residual block or prediction error. Inter-coded blocks are coded according to a motion vector pointing to a reference block in a reference frame forming the prediction block, and the residual block. The process of determining the motion vector is typically called motion estimation. Intra-coded blocks are coded according to an intra-prediction mode and a residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, e.g., the frequency domain, from which residual transform coefficients are obtained, which can then be quantized. The quantized transform coefficients are initially arranged in a two-dimensional array, which can be scanned to yield a one-dimensional vector of transform coefficients, which can then be entropy coded into a video bitstream to achieve further compression.

[0008] The encoded video bitstream may then be stored in a computer-readable storage medium (e.g., flash memory) and accessed by another electronic device having digital video capabilities, or transmitted directly to the electronic device via wired or wireless connection. The electronic device may then perform video decompression (the reverse process of video compression described above), for example, by parsing the encoded video bitstream to obtain syntax elements from the bitstream, reconstructing the digital video data from the encoded video bitstream to its original form based at least in part on the syntax elements obtained from the bitstream, and rendering the reconstructed digital video data on a display of the electronic device. Summary of the Invention [Problem to be solved by the invention]

[0009] As digital video quality improves from high definition to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded increases exponentially. A constant challenge is how to encode / decode video data more efficiently while maintaining the image quality of the decoded video data. [Means for solving the problem]

[0010] In general, this disclosure describes example techniques related to decoder-side motion vector compensation (DMVR) in video coding.

[0011] According to a first aspect of the present disclosure, a method for video coding is provided, the method including: deriving an initial motion vector (MV) for a current coding unit (CU); deriving a plurality of motion vector (MV) candidates for decoder-side motion vector compensation (DMVR); determining cost values ​​for the initial MV and each of the MV candidates; obtaining an updated cost value by adjusting at least one of the cost values ​​to prioritize the initial MV; and deriving a corrected motion vector (MV) based on the updated cost values ​​of the initial MV and the MV candidates.

[0012] According to a second aspect of the present disclosure, an apparatus for video coding is provided, the apparatus including one or more processors and a memory configured to store instructions executable by the one or more processors, wherein the one or more processors, when executing the instructions, are configured to: derive an initial motion vector (MV) for a current coding unit (CU); derive a plurality of motion vector (MV) candidates for decoder-side motion vector compensation (DMVR); determine cost values ​​for the initial MV and each of the MV candidates, obtain updated cost values ​​by adjusting at least one of the cost values ​​to prioritize the initial MV, and derive a compensated MV based on the updated cost values ​​of the initial MV and the MV candidates.

[0013] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium is provided that includes stored instructions, which, when executed by one or more processors, cause the one or more processors to perform operations including: deriving an initial motion vector (MV) for a current coding unit (CU); deriving a plurality of motion vector (MV) candidates for decoder-side motion vector correction (DMVR); determining cost values ​​for the initial MV and each of the MV candidates; obtaining an updated cost value by adjusting at least one of the cost values ​​to favor the initial MV; and deriving a corrected motion vector (MV) based on the updated cost values ​​of the initial MV and the MV candidates.

[0014] A more particular description of examples of the present disclosure will be provided by reference to specific examples illustrated in the accompanying drawings, which examples will be described and explained with additional specificity and detail using the accompanying drawings, provided that these drawings illustrate only some examples and therefore should not be considered limiting in scope. [Brief description of the drawings]

[0015] [Figure 1] 1 is a block diagram illustrating an example video encoder according to some implementations of the present disclosure. [Diagram 2] 1 is a block diagram illustrating an example video decoder according to some implementations of the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram illustrating an example of decoder-side motion vector correction (DMVR) according to some implementations of the present disclosure. [Figure 4] 1 is a schematic diagram illustrating an example of a DMVR search procedure according to some implementations of the present disclosure. [Diagram 5] FIG. 1 is a schematic diagram illustrating an example of a DMVR integer luminance sample search pattern according to some implementations of the present disclosure. [Figure 6] 1 is a block diagram illustrating an example apparatus for video coding according to some implementations of the present disclosure. [Figure 7] 1 is a flowchart illustrating an example process of decoder-side motion vector compensation (DMVR) in video coding according to some implementations of this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Reference will now be made in detail to specific implementations. Examples of implementations are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one skilled in the art that various alternatives can be used. For example, it will be apparent to one skilled in the art that the subject matter presented herein can be implemented in many types of electronic devices having digital video capabilities.

[0017] Throughout this specification, references to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language mean that a particular feature, structure, or characteristic described is included in at least one embodiment or example. Unless expressly specified otherwise, a feature, structure, element, or characteristic described in connection with one or some embodiments is also applicable to other embodiments.

[0018] Throughout this disclosure, unless expressly specified otherwise, all terms such as "first," "second," "third," etc. are used solely as terms to refer to associated elements, e.g., devices, components, compositions, steps, etc., and do not imply any spatial or chronological order. For example, a "first device" and a "second device" can refer to two separately formed devices or two parts, components, or operational states of the same device, and may be arbitrarily named.

[0019] As used herein, the terms "if" or "when" may be understood to mean "upon" or "in response to," depending on the context. These terms, when recited in the claims, may not indicate that the associated limitation or feature is conditional or optional.

[0020] The terms "module," "sub-module," "circuit," "sub-circuit," "circuitry," "sub-circuitry," "unit," or "sub-unit" may include memory (shared, dedicated, or group) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits with or without stored code or instructions. A module or circuit may include one or more components that are directly or indirectly connected. These components may or may not be physically attached to each other, or may or may not be adjacent to each other.

[0021] A unit or module can be implemented purely by software, purely by hardware, or by a combination of hardware and software. For example, in a purely software implementation, a unit or module can include functionally related code blocks or software components, which are directly or indirectly linked together to perform a specific function.

[0022] FIG. 1 shows a block diagram illustrating an exemplary block-based hybrid video encoder 100 that can be used with many video coding standards that use block-based processing. In the encoder 100, a video frame is divided into multiple video blocks for processing. For each given video block, a prediction is formed based on an inter-prediction technique or an intra-prediction technique. In inter-prediction, one or more predictors are formed based on pixels of a previously reconstructed frame by motion estimation and motion compensation. In intra-prediction, a predictor is formed based on reconstructed pixels in a current frame. A mode decision can select the best predictor to predict the current block.

[0023] A prediction residual, which represents the difference between the current video block and its predictor, is sent to a transform circuit 102. The transform coefficients are then sent from the transform circuit 102 to a quantization circuit 104 for entropy reduction. The quantized coefficients are then sent to an entropy coding circuit 106 to generate a compressed video bitstream. As shown in FIG. 1, prediction related information 110 from an inter prediction circuit and / or an intra prediction circuit 112, such as video block partition information, motion vectors, reference picture indexes, and intra prediction modes, are also sent by the entropy coding circuit 106 and stored in the compressed video bitstream 114.

[0024] In the encoder 100, decoder-related circuitry is also required to reconstruct pixels for prediction purposes. First, a prediction residual is reconstructed by an inverse quantization circuit 116 and an inverse transform circuit 118. This reconstructed prediction residual is combined with a block predictor 120 to generate unfiltered reconstructed pixels for the current video block.

[0025] Spatial prediction (or "intra prediction") predicts a current video block using pixels from samples of already coded neighboring blocks (also called reference samples) in the same video frame as the current video block.

[0026] Temporal prediction (also called "inter prediction") predicts a current video block using reconstructed pixels from an already coded video picture. Temporal prediction reduces the temporal redundancy inherent in video signals. A temporal prediction signal for a given coding unit (CU) or coding block is typically signaled by one or more motion vectors (MVs) that indicate the amount and direction of motion between the current CU and its temporal references. In addition, if multiple reference pictures are supported, one reference picture index is also signaled, which is used to identify which reference picture in the reference picture store the temporal prediction signal comes from.

[0027] After spatial and / or temporal prediction is performed, an intra / inter mode decision circuit 121 in the encoder 100 chooses the best prediction mode, for example based on a rate-distortion optimization method. The block predictor 120 is then subtracted from the current video block, and the resulting prediction residual is decorrelated using the transform circuit 102 and quantization circuit 104. The resulting quantized residual coefficients are inverse quantized by the inverse quantization circuit 116 and inverse transformed by the inverse transform circuit 118 to form a reconstructed residual, which is then added back to the predictive block to form a reconstructed signal for the CU. Further in-loop filtering 115, such as a deblocking filter, sample adaptive offset (SAO), and / or an adaptive in-loop filter (ALF), may be applied to the reconstructed CU before it is placed into the reference picture store of the picture buffer 117 and used to encode future video blocks. To form the output video bitstream 114, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit 106 for further compression and packing to form the bitstream.

[0028] For example, deblocking filters are available in AVC, HEVC, and the current version of VVC. HEVC specifies an additional in-loop filter called SAO (Sample Adaptive Offset) to further improve coding efficiency. Another in-loop filter called ALF (Adaptive Loop Filter) is also being actively researched.

[0029] These in-loop filter operations are optional. Performing these operations helps improve coding efficiency and visual quality. These operations can also be turned off as a decision provided by the encoder 100 to save computational complexity.

[0030] It should be noted that when these filter options are turned on by the encoder 100, intra prediction is typically based on unfiltered reconstructed pixels, whereas inter prediction is based on filtered reconstructed pixels.

[0031] FIG. 2 is a block diagram showing an exemplary block-based video decoder 200 that can be used with many video coding standards. The decoder 200 is similar to the reconstruction-related parts present in the encoder 100 of FIG. 1. In the decoder 200, an incoming video bitstream 201 is first decoded by entropy decoding 202 to derive quantized coefficient levels and prediction-related information. The quantized coefficient levels are then processed by inverse quantization 204 and inverse transform 206 to obtain a reconstructed prediction residual. A block predictor mechanism implemented in an intra / inter mode selector 212 is configured to perform intra prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by adding the reconstructed prediction residual from the inverse transform 206 and the prediction output generated by the block predictor mechanism using an adder 214.

[0032] The reconstructed block can further receive an in-loop filter 209 and is then stored in a picture buffer 213 that functions as a reference picture memory unit. The reconstructed video in the picture buffer 213 can be transmitted to drive a display device and can also be used to predict future video blocks. In the situation where the in-loop filter 209 is turned on, a filtering operation is performed on these reconstructed pixels to derive a final reconstructed video output 222.

[0033] The above-mentioned video encoding / decoding standards such as VVC, JEM, HEVC, and MPEG-4 Part 10 are conceptually similar. For example, they all use block-based processing. At the Joint Video Expert Team (JVET) meetings, JVET defined the drafts of the Versatile Video Coding (VVC) and VVC Test Model 1 (VTM1) encoding methods. It was decided to include a quadtree with multiple nested types of trees that use a 2-partition and 3-partition encoding block structure as the first new encoding feature of VVC.

[0034] <Decoder-side Motion Vector Refinement (DMVR) in VVC>

[0035] Decoder-side Motion Vector Refinement (DMVR) is a technique for blocks encoded in the bi-directional prediction merge mode. Under this mode, bilateral matching (BM) prediction can be used to further correct the two motion vectors (MVs) of a block.

[0036] Figure 3 is a schematic diagram showing an example of decoder-side motion vector compensation (DMVR). As shown in Figure 3, a bidirectional matching method is used to compensate the motion information of a current CU 322 by searching for the closest correspondence between the two reference blocks 302, 312 along the moving trajectory of the current CU in its two associated reference pictures, i.e., refPic in list L0 300 and refPic in list L1 310. The patterned square blocks 322, 302 and 312 show the current CU and the two reference blocks based on the initial motion information from merge mode. The patterned square blocks 304, 314 show a pair of reference blocks based on the MV candidates used in the motion compensation search process, i.e., the motion vector compensation process.

[0037] The MV difference between the MV candidates, i.e., MV0′ and MV1′, and the initial MVs, i.e., MV0 and MV1 (also called the original MVs), is MV diff and -MV diff Both the MV candidate and the initial MV are bidirectional motion vectors. During DMVR, multiple such MV candidates around the initial MV can be identified. Specifically, for each given MV candidate, its two related reference blocks can be identified in the reference pictures of list 0 and list 1, respectively, and the difference between these reference blocks can be calculated.

[0038] The block difference can also be called a cost value, and is usually measured in terms of the sum of absolute differences (SAD) or column subsample SAD (i.e., the SAD calculated for every other column of the relevant block). In some other examples, the average removed SAD or the sum of squared differences (SSD) can also be used as the cost value. The MV candidate with the lowest cost value or SAD between the two reference blocks becomes the correction MV and is used to generate the bidirectional prediction signal as the actual prediction for the current CU.

[0039] In VVC, DMVR is applied to a CU that satisfies the following conditions. · The CU is encoded by CU-level merge mode (not sub-block merge mode) with bi-predictive MVs. · For the current picture, one reference picture of the CU is past (i.e., has a POC smaller than the POC of the current picture), and the other reference picture is future (i.e., has a POC larger than the POC of the current picture). · The POC distance (i.e., the absolute POC difference) from both reference pictures to the current picture is the same. · The CU has a size larger than 64 luminance samples, and the height of the CU is larger than 8 luminance samples.

[0040] The corrected MV derived by the DMVR process is used to generate inter-predicted samples and is also used for temporal motion vector prediction for future picture coding. When the original MV is used in the non-blocking process, it is also used for spatial motion vector prediction for future CU coding.

[0041] <Search Method in DMVR>

[0042] As shown in FIG. 3, MV candidates (or search points) surround the initial MV, and the MV offset follows the MV difference mirroring rule. In other words, any point confirmed by DMVR is indicated by an MV candidate pair (MV0, MV1) and follows the following two equations.

Equation

[0043] In the above equation, MV diff represents the correction offset between the initial MV and the corrected MV in one of the reference pictures. In the current VVC, the correction search range is two integer luminance samples from the initial MV.

[0044] An example of a search process for a DMVR is shown in Figure 4. As shown in Figure 4, the search process includes an integer sample offset search stage 402 and a fractional sample correction stage 404.

[0045] In order to reduce the complexity of the search, a fast search method with an early termination mechanism is applied in the integer sample offset search stage 402. Instead of searching the full 25 locations, a two-iteration search scheme is applied to reduce the number of SAD confirmation points. Figure 5 shows an example of a search pattern of DMVR integer luminance samples for the integer sample offset search stage 402. Each square in Figure 5 represents a point (MV). As shown in Figure 5, according to the fast search method, in the first iteration, up to six SADs (SADs for the center and P1-P5) are confirmed. In the first iteration, the initial MV is the center. First, the SADs of five points (center and P1-P4) are compared. If the SAD of the center (i.e., the center position) is the smallest, the DMVR integer sample offset search stage 402 ends. If not, another position P5 (determined based on the SAD distribution of P1-P4) is confirmed. Then, the position (among P1-P5) with the smallest SAD is selected as the center position of the second iteration search. The process of the second iteration search is the same as that of the first iteration search. The SAD calculated in the first iteration can be used again in the second iteration, so in the second iteration, we only need to calculate the SAD of three additional points. Note that when the SAD of the center point in the first iteration is smaller than the number of samples used to calculate the SAD (equal to w*h / 2, where w and h represent the width and height of the DMVR operation unit, respectively), the whole DMVR process will terminate early without further search.

[0046] The integer sample search 402 is followed by fractional sample correction 404. To reduce computational complexity, the fractional sample correction 404 is derived using a parametric error surface equation rather than an additional search with SAD comparison. The fractional sample correction 404 is conditionally invoked based on the output of the integer sample search stage. When the integer sample search stage 402 ends with the center having the smallest SAD in the first or second iteration search, the fractional sample correction is further applied.

[0047] In fractional sample correction based on parametric error surface, the SAD costs (or cost values) of the center location and its four neighboring locations are used to fit a two-dimensional parabolic error surface equation of the form:

[0048]

number

[0049]

number

[0050] (x min ,y min)'s solver can be determined by the division-free processes shown in Tables 1 and 2 below. In this process, the values of E(-1,0) and E(0,-1) are substituted into sadMinus when solving equations (1) and (2) respectively, E(1,0) and E(0,1) are substituted into sadPositive when solving equations (1) and (2) respectively, and E(0,0) is substituted into sadCenter. When deriving equations (1) or (2) respectively, the output of this process dMvC is the solution x min or y min is.

[0051]

Table 1

[0052]

Table 2

[0053] x min and y min values are further restricted to -8 to 8, which corresponds to a half (1 / 2) pel offset from the center point at 1 / 16 pel MV accuracy. By adding the calculated fractional offset (x min , y min ) to the integer-distance MV correction, an MV correction with sub-pel accuracy is obtained.

[0054] <Bilinear Interpolation and Sample Padding for DMVR>

[0055] In VVC, the resolution of the MV is 1 / 16 luma samples. Samples at fractional positions are interpolated using an 8-tap interpolation filter. In DMVR search, when MV point candidates point to positions less than 1 pel, it is necessary to interpolate their related fractional position samples. To reduce the computational complexity, a bilinear interpolation filter is used to generate fractional samples in the DMVR search process.

[0056] Another advantage of using a bilinear filter for interpolation is that due to the two-sample search range, the DVMR search process does not access more reference samples than the normal motion compensation process. After the compensated MV is achieved by the DMVR search process, a normal 8-tap interpolation filter is applied to generate the final prediction. Again, in this 8-tap interpolation process, sample padding is used to avoid accessing more reference samples than the normal motion compensation process. More specifically, in the 8-tap interpolation process based on the compensated MV, samples beyond those required for motion compensation based on the original MV are padded from adjacent available samples.

[0057] <Maximum DMVR processing unit>

[0058] When the width and / or height of a CU is greater than 16 luma samples, the DMVR operation on the CU is performed based on a DMVR processing unit with a maximum width and / or height equal to 16 samples. In other words, in such a case, the original CU is divided into sub-blocks with width and / or height equal to 16 luma samples for the DMVR operation. The maximum processing unit size for the DMVR search process is limited to 16x16.

[0059] In the current VVC design, there is no control flag to control whether to enable DMVR. However, it is not guaranteed that the DMVR-corrected MV is always better than the MV before correction. In some cases, the DMVR correction process may even result in a corrected MV that is worse than the original MV. According to some examples of the present disclosure, some methods are proposed to reduce the penalty caused by such uncertainty of DMVR MV correction.

[0060] <Updated cost value for DMVR by adjusting cost value>

[0061] Several exemplary approaches are proposed for prioritizing the original MV during the DMVR process. Note that these different approaches can be applied independently or jointly.

[0062] The terms "initial MV" and "original MV" may be used interchangeably in some examples of this disclosure.

[0063] In some examples, during the DMVR process, the cost values ​​for the initial MV and each of the MV candidates can be adjusted or updated to favor the initial MV. That is, after calculating the cost values ​​(e.g., SAD) of the search points in the DMVR process, the cost values ​​can be adjusted to increase the probability that the initial MV has the smallest cost value among the updated cost values, that is, to favor the initial MV.

[0064] Therefore, after obtaining the updated cost value, the initial MV is more likely to be selected as the MV with the lowest cost during the DMVR process.

[0065] Here, the SAD value is used as an exemplary cost value for illustrative purposes. Other values ​​such as column subsample SAD, mean removed SAD, or sum of squared residuals (SSD) can also be used as the cost value.

[0066] In some examples, the SAD value between the reference blocks referenced by the initial MV (or original MV) is compared with the SAD values ​​of other MV candidates to obtain a first value Offset calculated by a predetermined process. SAD Therefore, the initial MV is preferred over other candidate MVs because its SAD value is decreased.

[0067] In one example, Offset SAD The value of may be determined as 1 / N of the SAD value associated with the initial MV, where N is an integer (eg, 4, 8, or 16).

[0068] In another example, Offset SAD The value of can be determined as a constant value M.

[0069] In yet another example, Offset SAD The value of can be determined according to the coding information of the current CU, where the coding information includes at least one of the following: a coding block size, a motion vector magnitude, a SAD of an initial MV, and a relative position of a DMVR process unit, or a combination thereof. For example, Offset SAD The value of Offset may be determined as 1 / N of the SAD value associated with the initial MV, where N is an integer value (e.g., 4, 8, or 16) selected based on the block size of the current CU. When the current block size is equal to or greater than a predetermined size (e.g., 16×16), the value of N is set to 8, otherwise the value of N is set to 4. For example, Offset SAD The value of may be determined as 1 / N of the SAD value associated with the initial MV, where N is an integer value (e.g., 4, 8, or 16) selected based on the distance between the center position of the DMVR process unit and the center position of the current CU. When this distance is greater than or equal to a predetermined threshold, N is set to one value (e.g., 8), otherwise N is set to another value (e.g., 4).

[0070] In these examples, we set the SAD value associated with the initial MV to a specific value, Offset SAD In practice, this concept can be implemented in different ways. For example, instead of decreasing the SAD value associated with the initial MV, this Offset SAD The values ​​of can be added to those SADs associated with the other MV candidates, and the results in these two cases are equivalent.

[0071] In some other examples, the SAD value between the reference blocks referenced by the non-initial MV candidates is calculated by a second value Offset calculated by a predetermined process. SADThe second value, Offset SAD ' and the first value Offset SAD can be the same or different. Thus, the SAD values ​​of the non-initial MVs are increased so that the initial MVs are preferred.

[0072] In one example, Offset SAD The value of ' may be determined as 1 / N of the SAD value associated with the non-initial MV, where N is an integer (e.g., 4, 8, or 16).

[0073] In another example, Offset SAD The value of ' can be determined as a constant value M.

[0074] In yet another example, Offset SAD The value of Offset ' may be determined according to the coding information of the current CU, where the coding information may include a coding block size, a motion vector magnitude, a SAD value of a non-initial MV, and / or a relative position of a DMVR process unit within the current CU. For example, the value may be determined as 1 / N of the SAD value from the BM using the non-initial MV, where N is an integer (e.g., 4, 8, or 16) selected based on the block size. When the current block size is equal to or larger than a predetermined size (e.g., 16×16), the value of N is set to 8, otherwise, the value of N is set to 4. For example, Offset SAD The value of ' may be determined as 1 / N of the SAD value from the BM using the non-initial MV, where N is an integer value (e.g., 4, 8, or 16) selected based on the distance between the center position of the DMVR process unit and the center position of the current CU. When this distance is greater than or equal to a predetermined threshold, N is set to one value (e.g., 8), otherwise N is set to another value (e.g., 4).

[0075] In these examples, we set the SAD values ​​associated with the non-initial MV candidates to a specific value, Offset SADIn practice, this concept can be implemented in different ways. For example, instead of increasing the SAD value associated with the non-initial MV, this Offset SAD The value of ' can be subtracted from the SAD associated with the initial MV and the results are equivalent.

[0076] In some further examples, the BM SAD associated with the initial MV is calculated based on a proper subset of samples used for the SAD calculation associated with the non-initial MV, i.e., the SAD value of the initial MV is determined using fewer samples compared to the SAD value of the MV candidate, which is similar to decreasing the SAD value of the initial MV.

[0077] In some other examples, x for fractional sample correction based on parameter error surfaces min and y min The process of deriving or obtaining is simplified by eliminating the comparison between sadMinus and sadCenter and the comparison between sadPlus and sadCenter shown in Table 3. That is, the fractional sample correction is simplified by eliminating the comparison between sadMinus and sadCenter and the comparison between sadPlus and sadCenter when deriving the correction MV. Here, sadMinus represents the value of E(-1,0) or E(0,-1), sadPlus represents the value of E(1,0) or E(0,1), and sadCenter represents the value of E(0,0) when solving the following two-dimensional parabolic equation:

number

[0078] [Table 3]

[0079] In some other examples, x for fractional sample correction based on parameter error surfaces min and ymin The process of obtaining x is done by restricting dMvC to -4 or 4 for the corner cases shown in Table 4. min and y min and reducing the value of the iteration counter shown in Table 5 to 2 (eg, the counter is initialized to a value of 2).

[0080] [Table 4]

[0081] [Table 5]

[0082] Following the above example, the DMVR process is modified to prioritize the initial MV over other MV candidates during the integer sample offset search stage and / or the fractional sample correction stage, thereby reducing the penalty due to the possible scenario where the correction MV is degraded from the original MV.

[0083] 6 is a block diagram illustrating an exemplary apparatus for video coding according to some implementations of the present disclosure. The apparatus 600 may be a terminal such as a mobile phone, a tablet computer, a digital broadcast terminal, a tablet device, or a personal digital assistant.

[0084] As shown in FIG. 6, the device 600 may include one or more of the following components: a processing component 602, a memory 604, a power component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0085] The processing component 602 typically controls the overall operation of the device 600, such as operations related to display, making and receiving calls, data communication, camera operation, and recording operations. The processing component 602 may include one or more processors 620 that execute instructions to complete all or a portion of the steps of the methods described above. Additionally, the processing component 602 may include one or more modules to facilitate interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate interaction between the multimedia component 608 and the processing component 602.

[0086] The memory 604 is configured to store different types of data to support the operation of the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or combination thereof, and the memory 604 can be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or a compact disk.

[0087] The power component 606 provides power for the different components of the device 600. The power component 606 can include a power management system, one or more power sources, and other components associated with the generation, management, and distribution of power to the device 600.

[0088] The multimedia component 608 includes a screen that provides an output interface between the device 600 and a user. In some examples, the screen can include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, the screen can be implemented as a touch screen that receives input signals from a user. The touch panel can include one or more touch sensors for sensing touches, slides, and gestures on the touch panel. The touch sensor can sense the boundaries of a touch or slide motion, as well as detect the duration and pressure associated with the touch or slide motion. In some examples, the multimedia component 608 can include a front camera and / or a rear camera. When the device 600 is in an operation mode, such as a photography mode or a video mode, the front camera and / or the rear camera can receive multimedia data from the outside.

[0089] The audio component 610 is configured to output and / or input an audio signal. For example, the audio component 610 includes a microphone (MIC). When the device 600 is in an operation mode such as an incoming / outgoing mode, a recording mode, and an audio recognition mode, the microphone is configured to receive an audio signal from the outside. The received audio signal can be further stored in the memory 604 or can be transmitted via the communication component 616. In some examples, the audio component 610 further includes a speaker for outputting the audio signal.

[0090] The I / O interface 612 provides an interface between the processing component 602 and a peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0091] The sensor component 614 includes one or more sensors for providing a status assessment in different embodiments of the device 600. For example, the sensor component 614 can detect the on / off state of the device 600 and the relative position of the components. For example, the components can include the display and keypad of the device 600. The sensor component 614 can also detect position changes of the device 600 or components of the device 600, the presence or absence of a user's touch on the device 600, the orientation or acceleration / deceleration of the device 600, and temperature changes of the device 600. The sensor component 614 can include a proximity sensor configured to detect the presence of a nearby object without physical contact. The sensor component 614 can further include an optical sensor, such as a CMOS or CCD image sensor used in imaging applications. In some examples, the sensor component 614 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0092] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard such as WiFi, 4G, or a combination thereof. In one example, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In one example, the communication component 616 can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0093] In one example, the apparatus 600 may be implemented by one or more of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic elements to perform the methods described above.

[0094] The non-transitory computer-readable storage medium may be, for example, a hard disk drive (HDD), a solid-state drive (SSD), a flash memory, a hybrid drive or a solid-state hybrid drive (SSHD), a read-only memory (ROM), a compact disk read-only memory (CD-ROM), a magnetic tape, a floppy disk, etc.

[0095] FIG. 7 is a flowchart illustrating an example process of decoder-side motion vector compensation in video coding according to some implementations of this disclosure.

[0096] In step 702, the processor 620 derives an initial motion vector (MV) for the current coding unit (CU).

[0097] In step 704, the processor 620 derives multiple motion vector (MV) candidates for decoder-side motion vector compensation (DMVR).

[0098] At step 706, the processor 620 determines a cost value for the initial MV and each of the candidate MVs.

[0099] In step 708, processor 620 obtains updated cost values ​​by adjusting at least one of the cost values ​​in favor of the initial MV.

[0100] At step 710, the processor 620 derives a corrected MV based on the updated cost values ​​of the initial MV and the candidate MVs.

[0101] Adjusting at least one of the plurality of cost values ​​may include decreasing a cost value of the initial MV by a first value determined using a predetermined process or increasing a cost value of the candidate MV by a second value determined using a predetermined process.

[0102] In some examples, an apparatus for video coding is provided, which includes one or more processors 620 and a memory 604 configured to store instructions executable by the one or more processors, the one or more processors being configured to, when executing the instructions, perform the method shown in FIG.

[0103] In some other examples, a non-transitory computer-readable storage medium 604 having instructions stored thereon is provided that, when executed by one or more processors 620, cause the processors to perform the method described in FIG.

[0104] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or to be limiting of the present disclosure. Many modifications, variations and alternative implementations will be apparent to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the accompanying drawings.

[0105] These examples have been chosen and described to explain the principles of the disclosure and to enable those skilled in the art to understand the disclosure with respect to various implementations and to best utilize the underlying principles and various implementations with various modifications adapted to the particular applications contemplated. Thus, it should be understood that the scope of the disclosure is not limited to the specific examples of implementations disclosed, but that modifications and other implementations are intended to be encompassed within the scope of the disclosure.

Claims

1. 1. A method of video decoding, comprising: deriving an initial motion vector (MV) for a current block; determining a cost value for each of a plurality of MV candidates and the Initial MV, wherein the Initial MV and each MV among the MV candidates are associated with two reference blocks, and each of the cost values ​​is determined based on a difference between a reference sample in one of the two reference blocks and a reference block in the other of the two reference blocks; obtaining updated cost values ​​by decreasing a cost value for the initial MV or by increasing a cost value for the candidate MV; deriving a corrected MV based on the updated cost value; A method comprising:

2. 2. The method of claim 1, wherein decreasing the cost value for the initial MV comprises decreasing the cost value for the initial MV by a first value determined using a predetermined process.

3. The method of claim 2 , wherein the first value is determined as 1 / N of the cost value for the initial MV, where N is an integer value.

4. The method of claim 2 , wherein the first value is determined as a constant value.

5. 3. The method of claim 2, wherein the first value is determined according to coding information of the current block, the coding information including at least one or a combination of a coding block size, a motion vector magnitude, a sum of absolute difference (SAD) of the initial MV, and a relative position of a decoder-side motion vector correction (DMVR) process unit.

6. 6. The method of claim 5, wherein the first value is determined as 1 / N of the cost value for the Initial MV, where N is an integer value derived based on a block size of the current block.

7. 6. The method of claim 5, wherein the first value is determined as 1 / N of the cost value for the initial MV, where N is an integer value derived based on a distance between a center position of the DMVR process unit and a center position of the current block.

8. 2. The method of claim 1, wherein increasing the cost value for the MV candidate comprises increasing the cost value for the MV candidate by a second value determined using a predetermined process.

9. The method of claim 1 , wherein the cost value for the initial MV is determined using fewer samples than the cost value for the candidate MV.

10. the step of deriving the corrective motion vector includes fractional sample correction, the fractional sample correction being simplified by eliminating a comparison between sadMinus and sadCenter and a comparison between sadPlus and sadCenter; [0010] 2. The method of claim 1, wherein sadMinus represents a value of E(-1,0) or E(0,-1), sadPlus represents a value of E(1,0) or E(0,1), and sadCenter represents a value of E(0,0).

11. 2. The method of claim 1, wherein the step of deriving the corrective motion vector includes fractional sample correction, which is simplified by limiting fractional positions to -4 or 4 for corner cases with 1 / 16 pel MV accuracy and reducing the iteration count value to 2 in a division-free solver.

12. The method of claim 1 , wherein the cost value is a sum of absolute differences (SAD).

13. 1. An apparatus for video decoding, comprising: one or more processors; a memory configured to store instructions executable by the one or more processors; Equipped with When the one or more processors execute the instructions, Deriving an initial motion vector (MV) for a current block; determining a cost value for each of a plurality of MV candidates and the initial MV, wherein the initial MV and each MV among the MV candidates are associated with two reference blocks, and each of the cost values ​​is determined based on a difference between a reference sample in one of the two reference blocks and a reference sample in the other of the two reference blocks; Obtaining updated cost values ​​by decreasing a cost value for the initial MV or by increasing a cost value for the candidate MV; deriving a corrected MV based on the updated cost value; An apparatus configured to:

14. 14. The apparatus of claim 13, wherein decreasing the cost value for the initial MV comprises decreasing the cost value for the initial MV by a first value determined using a predetermined process.

15. The apparatus of claim 14 , wherein the first value is determined as 1 / N of the cost value for the initial MV, where N is an integer value.

16. The apparatus of claim 14 , wherein the first value is determined as a constant value.

17. 15. The apparatus of claim 14, wherein the first value is determined according to coding information of the current block, the coding information including at least one or a combination of a coding block size, a motion vector magnitude, the cost value relative to the initial MV, and a relative position of a decoder-side motion vector correction (DMVR) process unit.

18. A computer program comprising instructions which, when executed by a processor, cause the processor to perform a method according to any one of claims 1 to 12.

19. A method of transmitting a bitstream to be decoded by a decoding device, the decoding device comprising one or more processing units configured to perform the method according to any one of claims 1 to 12.

Images