Multiple syntax elements for adaptive motion vector resolution

Abstract

A method for visual media processing includes: for a conversion between a current video block and a bitstream representation of the current video block, determining to represent motion information using one or more precisions, the motion information including motion vectors and / or motion vector predictors and / or motion vector differences used during the conversion in an affine inter-frame mode using adaptive motion vector resolution (AMVR) or a normal inter-frame mode; and performing the conversion based on the determination of one or more precisions, wherein multiple syntax elements are used in the bitstream representation to signal one or more precisions.

Description

[0001] Cross-reference to related applications

[0002] This application is the Chinese national phase application of International Patent Application No. PCT / CN2020 / 074135, filed on February 1, 2020, and promptly claims priority and benefit to International Patent Application No. PCT / CN2019 / 074433, filed on February 1, 2019. The entire disclosure of the aforementioned application is incorporated herein by reference as a part of this application. Technical Field

[0003] The patent document relates to video encoding and decoding technologies, devices, and systems. Background Technology

[0004] Despite advancements in video compression, digital video still accounts for the largest share of bandwidth usage on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth required for digital video usage is expected to continue to grow. Summary of the Invention

[0005] The disclosed technology relates to video coding techniques. Specifically, it relates to post-reconstruction filters, such as bilateral filters in image / video coding. It can be applied to existing video coding standards (e.g., HEVC) or pending standards (Multi-Functional Video Coding). It may also be applicable to future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for visual media processing. The method includes: for a conversion between a current video block and a bitstream representation of the current video block, determining to represent motion information using one or more precisions, the motion information including motion vectors and / or motion vector predictors and / or motion vector differences used during the conversion in an affine inter-frame mode using adaptive motion vector resolution (AMVR) or a normal inter-frame mode; and performing the conversion based on the determination of the one or more precisions, wherein multiple syntax elements are used in the bitstream representation to signal the one or more precisions.

[0007] In one representative aspect, the disclosed technology can be used to provide a method for visual media processing. The method includes: for a conversion between a current video block and a bitstream representation of the current video block, determining to represent motion information using one or more precisions, the motion information including motion vectors and / or motion vector predictors and / or motion vector differences used during the conversion in an intra-block copy (IBC) mode using adaptive motion vector resolution (AMVR); performing the conversion based on the determination of the one or more precisions, wherein multiple syntax elements are used in the bitstream representation to signal the one or more precisions; and wherein the conversion is performed based on block vectors derived from an image of the current video block in the IBC mode.

[0008] In another representative aspect, the above methods are embodied in processor-executable code and stored in a computer-readable program medium.

[0009] In another representative aspect, a device configured or operable to perform the methods described above is disclosed. This device may include a processor programmed to implement the methods.

[0010] In another representative aspect, the video decoder device can implement one or more of the methods described herein.

[0011] In another representative aspect, the video encoder device can implement one or more of the methods described herein.

[0012] The above and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, description and claims. Attached Figure Description

[0013] Figure 1 An example of constructing a Merge candidate list is shown.

[0014] Figure 2 An example of a candidate location for the airspace is shown.

[0015] Figure 3 An example of a candidate pair that has undergone a redundancy check of the spatial merge candidate is shown.

[0016] Figure 4A and 4B An example of the position of a second prediction unit (PU) based on the size and shape of the current block is shown.

[0017] Figure 5 An example of motion vector scaling for temporal Merge candidates is shown.

[0018] Figure 6 An example of candidate locations for temporal Merge candidates is shown.

[0019] Figure 7 An example of generating bidirectional prediction Merge candidates using a combination is shown.

[0020] Figure 8 An example of constructing motion vector prediction candidates is shown.

[0021] Figure 9 An example of motion vector scaling for spatial motion vector candidates is shown.

[0022] Figure 10 An example of using the Optional Temporal Motion Vector Prediction (ATMVP) algorithm for motion prediction of a coding unit (CU) is shown.

[0023] Figure 11 An example of a coding unit (CU) with sub-blocks and neighboring blocks is shown, used by the spatial-temporal motion vector prediction (STMVP) algorithm.

[0024] Figure 12A and 12B An example snapshot of a sub-block is shown when the Overlapping Block Motion Compensation (OBMC) algorithm is used.

[0025] Figure 13 An example of neighboring samples used to derive the parameters of the Local Illumination Compensation (LIC) algorithm is shown.

[0026] Figure 14 An example of a simplified affine motion model is shown.

[0027] Figure 15 An example of the affine motion vector field (MVF) for each sub-block is shown.

[0028] Figure 16 An example of motion vector prediction (MVP) for the AF_INTER affine motion pattern is shown.

[0029] Figure 17A and 17B Examples of affine models with 4 and 6 parameters are shown respectively.

[0030] Figure 18A and 18B Example candidates for the AF_MERGE affine motion mode are shown.

[0031] Figure 19 An example of bilateral matching in the Pattern Matching Motion Vector Derivation (PMMVD) mode is shown, which is a special Merge mode based on the Frame Rate Upconversion (FRUC) algorithm.

[0032] Figure 20 An example of template matching in the FRUC algorithm is shown.

[0033] Figure 21 An example of one-sided motion estimation in the FRUC algorithm is shown.

[0034] Figure 22 An example of the optical flow trajectory used by the bidirectional optical flow (BDOF) algorithm is shown.

[0035] Figure 23A and 23B An example snapshot is shown using the Bidirectional Optical Flow (BDOF) algorithm without block expansion.

[0036] Figure 24 An example of a decoder-side motion vector refinement (DMVR) algorithm based on bilateral template matching is shown.

[0037] Figure 25 A flowchart of an example method for video encoding is shown.

[0038] Figure 26 This is a block diagram of an example hardware platform used to implement the visual media decoding or visual media encoding techniques described herein.

[0039] Figure 27 An example of a symmetric pattern is shown.

[0040] Figure 28 This is a block diagram of an example video processing system that can implement the disclosed technology.

[0041] Figure 29 A flowchart of an example method for video processing is shown.

[0042] Figure 30 A flowchart of an example method for video processing is shown. Detailed Implementation

[0043] Due to the increasing demand for higher resolution video, video coding methods and technologies are ubiquitous in modern technology. Video codecs typically consist of electronic circuitry or software that compresses or decompresses digital video and are constantly being improved to provide higher coding efficiency. Video codecs convert uncompressed video into a compressed format and vice versa. There are complex relationships between video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, and end-to-end latency. Compression formats typically conform to standard video compression specifications, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the Universal Video Coding Standard to be Completed, or other current and / or future video coding standards.

[0044] Embodiments of the disclosed techniques can be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used in this document to improve readability of the description and do not in any way limit the discussion or embodiments (and / or implementations) to the relevant sections only.

[0045] 1. Example of inter-frame prediction in HEVC / H.265

[0046] Over the years, video coding standards have improved significantly and now offer, in part, high coding efficiency and support for higher resolutions. Latest standards such as HEVC and H.265 are based on a hybrid video coding architecture, which utilizes temporal prediction plus transform coding.

[0047] 1.1. Examples of Predictive Patterns

[0048] Each inter-frame prediction PU (prediction unit) has motion parameters for one or two lists of reference images. In some embodiments, the motion parameters include motion vectors and reference image indices. In other embodiments, inter_pred_idc can also be used to signal the use of one of the two lists of reference images. In yet another embodiment, the motion vectors can be explicitly encoded as increments relative to the predictor.

[0049] When a CU is encoded using the skip mode, a PU is associated with a CU, and there are no significant residual coefficients, no encoded motion vector increments, or reference picture indices. A Merge mode is specified to obtain the motion parameters of the current PU from neighboring PUs, including spatial and temporal candidates. The Merge mode can be applied to any PU for inter-frame prediction, not just the skip mode. An alternative to the Merge mode is explicit transmission of motion parameters, where, for each PU, the motion vectors, the corresponding reference picture index for each reference picture list, and the reference picture list usage are explicitly communicated via signaling.

[0050] A PU is generated from a sample block when the signaling indicates that one of two reference image lists will be used. This is called "uni-prediction". Uni-prediction can be used for both P-strips and B-strips.

[0051] When the signaling indicates that two lists of reference images will be used, a PU is generated from two sample blocks. This is called "bi-prediction". Bi-prediction is only applicable to B-strips.

[0052] 1.1.1.1 Constructing candidate implementations for the Merge pattern

[0053] When predicting the PU using the Merge pattern, the indices pointing to entries in the Merge candidate list are parsed from the bitstream and used to retrieve motion information. The construction of this list can be summarized in the following steps:

[0054] Step 1: Derivation of the original candidates

[0055] Step 1.1: Spatial Candidate Derivation

[0056] Step 1.2: Redundancy check of airspace candidates

[0057] Step 1.3: Time-domain candidate derivation

[0058] Step 2: Insert additional candidates

[0059] Step 2.1: Create bidirectional prediction candidates

[0060] Step 2.2: Insert zero-motion candidates

[0061] Figure 1 An example of constructing a Merge candidate list based on the sequence of steps summarized above is shown. For spatial Merge candidate derivation, up to four Merge candidates are selected from candidates located at five distinct positions. For temporal Merge candidate derivation, up to one Merge candidate is selected from two candidates. Since a constant number of candidates is assumed for each PU at the decoder, additional candidates are generated when the number of candidates does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the stripe header. Because the number of candidates is constant, truncated unary binarization (TU) is used to encode the index of the best Merge candidate. If the CU size is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of a 2N×2N prediction unit.

[0062] 1.1.2 Constructing Spatial Merge Candidates

[0063] In the derivation of the spatial Merge candidate, located in Figure 2 Up to four merge candidates are selected from the candidates for the described positions. The derivation order is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, or A0 is unavailable (e.g., because it belongs to another stripe or block) or if it is intra-coded. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thus improving coding efficiency.

[0064] To reduce computational complexity, not all possible candidate pairs were considered in the redundancy check mentioned above. Instead, only... Figure 3 Pairs connected by arrows are added to the list only if the corresponding candidate used for redundancy checking has different motion information. Another source of duplicate motion information is a "second PU" associated with a segmentation different from 2N×2N. As an example, Figure 4A and 4B The second PU is depicted for the N×2N and 2N×N cases, respectively. When the current PU is segmented into N×2N, the candidate at position A1 is not considered for list construction. In some embodiments, adding this candidate may result in two prediction units with the same motion information, which is redundant for a coding unit with only one PU. Similarly, when the current PU is segmented into 2N×N, position B1 is not considered.

[0065] 1.1.3 Constructing Temporal Merge Candidates

[0066] In this step, only one candidate is added to the list. Specifically, in the derivation of the temporal merge candidate, the scaled motion vector is derived based on the co-localized PU, which belongs to the image within a given list of reference images that has the smallest point of interest (POC) difference with the current image. The list of reference images to be used for deriving the co-localized PU is explicitly notified by signaling in the strip header.

[0067] Figure 5 An example of the derivation for scaling motion vectors for temporal merge candidates (e.g., dashed lines) is shown, which is scaled from the motion vectors of co-localized PUs using POC distances tb and td, where tb is defined as the POC difference between the current image and the reference image of the current image, and td is defined as the POC difference between the reference image of the co-localized image and the co-localized image. The reference image index of the temporal merge candidate is set to zero. For the B-strip, two motion vectors are obtained, one for reference image list 0 and the other for reference image list 1, and these two motion vectors are combined to obtain bidirectional predicted merge candidates.

[0068] In the common localization PU(Y) belonging to the reference frame, the temporal candidate position is selected between candidate C0 and C1, such as... Figure 6 As shown. If the PU at position C0 is unavailable, intra-coded, or outside the current CTU, then position C1 is used. Otherwise, position C0 is used for the derivation of the temporal merge candidate.

[0069] 1.1.4 Constructing Additional Types for Merge Candidates

[0070] In addition to spatiotemporal merge candidates, two additional types of merge candidates exist: combined bidirectional predictive merge candidates and zero merge candidates. Combined bidirectional predictive merge candidates are generated by utilizing spatiotemporal merge candidates. These combined bidirectional predictive merge candidates are only used for B-strips. Combined bidirectional predictive candidates are generated by combining the motion parameters of the first reference image list of the original candidate with the motion parameters of the second reference image list of another candidate. If these two tuples provide different motion hypotheses, they will form a new bidirectional predictive candidate.

[0071] Figure 7 An example of the process is shown, where two candidates in the original list (710 on the left) have mvL0 and refIdxL0 or mvL1 and refIdxL1, which are used to create a bidirectional prediction of the Merge candidate that is added to the final list (on the right).

[0072] Zero-motion candidates are inserted to populate the remaining entries in the Merge candidate list, thus achieving the MaxNumMergeCand capacity. These candidates have zero spatial displacements and a reference image index that starts at zero and increases whenever a new zero-motion candidate is added to the list. The number of reference frames used for these candidates is 1 and 2, for unidirectional and bidirectional prediction, respectively. In some embodiments, redundancy checks are not performed on these candidates.

[0073] 1.1.5 Example of motion estimation region for parallel processing

[0074] To accelerate encoding processing, motion estimation can be performed in parallel, thereby simultaneously deriving the motion vectors of all prediction units within a given region. Deriving merge candidates from spatial neighborhoods can interfere with parallel processing because a prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is complete. To mitigate the trade-off between encoding efficiency and processing latency, a Motion Estimation Region (MER) can be defined. The size of the MER is signaled in the Picture Parameter Set (PPS) using the syntax element "log2_parallel_merge_level_minus2". When defining the MER, merge candidates falling into the same region are marked as unavailable and therefore not considered in the list construction.

[0075] 1.2 Examples of Advanced Motion Vector Prediction (AMVP)

[0076] AMVP utilizes the spatiotemporal correlation of motion vectors with neighboring PUs for explicit transmission of motion parameters. It constructs a motion vector candidate list by first checking the availability above and to the left of the temporally nearest PU locations, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and send the corresponding index indicating the selected candidate. Similar to Merge index signaling, a truncated unary is used to encode the index of the best motion vector candidate. The maximum value to be encoded in this case is 2 (see [link to documentation]). Figure 8 The following sections provide details of the derivation process for the motion vector prediction candidates.

[0077] 1.2.1 Example of constructing motion vector prediction candidates

[0078] Figure 8 The derivation process of motion vector prediction candidates is summarized, and it can be implemented by taking the index as input for each list of reference images.

[0079] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the derivation of spatial motion vector candidates, the final result is based on the previously established... Figure 2 The motion vector derivation for each PU at the five different locations shown in the figure derives two motion vector candidates.

[0080] For temporal motion vector candidate derivation, a motion vector candidate is selected from two candidates, derived based on the locations of two distinct co-localizations. After generating the first list of spatiotemporal candidates, duplicate motion vector candidates are removed from the list. If the number of potential candidates is greater than two, motion vector candidates with a reference image index greater than 1 from the associated reference image list are removed from the list. If the number of spatiotemporal motion vector candidates is less than two, additional zero motion vector candidates are added to the list.

[0081] 1.2.2 Constructing Candidate Spatial Motion Vectors

[0082] In the derivation of the spatial motion vector candidates, at most two candidates are considered from the five potential candidates, which are located as previously stated. Figure 2 The derivation in the PU at the positions shown is the same as the position of the motion merge. The derivation order to the left of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order to the upper side of the current PU is defined as B0, B1, B2, and scaled B0, scaled B1, scaled B2. Therefore, for each side, there are four cases that can be used as motion vector candidates, two of which do not require spatial scaling, and two of which do. The four different cases are summarized as follows:

[0083] No spatial scaling

[0084] (1) The same list of reference images, and the same index of reference images (the same POC).

[0085] (2) Different lists of reference images, but the same reference image (same POC).

[0086] Spatial scaling

[0087] (3) Same list of reference images, but different reference images (different POCs)

[0088] (4) A list of different reference images, and different reference images (different POCs).

[0089] First, check for the case without spatial scaling, then check for spatial scaling. Spatial scaling is considered when the POC differs between the reference image of the neighboring PU and the reference image of the current PU, regardless of the reference image list. If all PUs of the left candidate are unavailable or intra-coded, scaling of the above motion vectors is allowed to aid in the parallel derivation of the left and top MV candidates. Otherwise, spatial scaling of the above motion vectors is not allowed.

[0090] like Figure 9 As shown in the example, for spatial scaling, the motion vectors of neighboring PUs are scaled in a similar manner to temporal scaling. One difference is that a list of reference images and the index of the current PU are given as input; the actual scaling process is the same as the temporal scaling process.

[0091] 1.2.3 Constructing candidate time-domain motion vectors

[0092] Apart from the derivation of the reference image index, all the procedures for deriving the temporal Merge candidate are the same as those for deriving the spatial motion vector candidate (e.g., ...). Figure 6 (As shown in the example). In some embodiments, the reference image index is signaled to the decoder.

[0093] 2. Examples of inter-frame prediction methods in the Joint Exploration Model (JEM)

[0094] In some embodiments, reference software called Joint Exploration Mode (JEM) is used to explore future video coding techniques. In JEM, sub-block-based predictions are employed across several coding tools, such as affine prediction, optional temporal motion vector prediction (ATMVP), spatiotemporal motion vector prediction (STMVP), bidirectional optical flow (BDOF or BIO), frame rate upconversion (FRUC), locally adaptive motion vector resolution (LAMVR), overlapping block motion compensation (OBMC), local illumination compensation (LIC), and decoder-side motion vector refinement (DMVR).

[0095] 2.1 Example of motion vector prediction based on sub-CU

[0096] In a JEM with a quadtree plus binary tree (QTBT), each CU can have at most one set of motion parameters for each prediction direction. In some embodiments, two sub-CU level motion vector prediction methods are considered in the encoder by dividing the large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU. The Alternative Temporal Motion Vector Prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple blocks smaller than the current CU in the juxtaposed reference image. In the Spatial-Temporal Motion Vector Prediction (STMVP) method, the motion vectors of the sub-CUs are recursively derived using temporal motion vector prediction values ​​and spatially neighboring motion vectors. In some embodiments, motion compression of the reference frame may be disabled to preserve a more accurate motion field for sub-CU motion prediction.

[0097] 2.1.1 Example of Optional Temporal Motion Vector Prediction (ATMVP)

[0098] In the ATMVP method, the temporal motion vector prediction (TMVP) method is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU.

[0099] Figure 10 An example of the ATMVP motion prediction process for CU 1000 is shown. The ATMVP 1000 method predicts the motion vectors of sub-CUs 1001 within CU 1000 in two steps. The first step is to identify the corresponding block 1051 in a reference image 1050 using temporal vectors. The reference image 1050 is also referred to as the motion source image. The second step is to divide the current CU 1000 into sub-CUs 1001 and obtain the motion vectors from the blocks corresponding to each sub-CU, as well as the reference index for each sub-CU.

[0100] In the first step, reference image 1050 and corresponding block are determined using motion information from spatially neighboring blocks of the current CU 1000. To avoid repeated scanning of neighboring blocks, the first Merge candidate from the Merge candidate list of the current CU 1000 is used. The first available motion vector and its associated reference index are set to the index of the temporal vector and the motion source image. In this way, corresponding blocks can be identified more accurately than TMVP, where the corresponding block (sometimes called the juxtaposed block) is always located in the lower right or center position relative to the current CU.

[0101] In the second step, the corresponding block of sub-CU 1051 is identified by adding a temporal vector to the coordinates of the current CU, using the temporal vector in the motion source image 1050. For each sub-CU, the motion information of its corresponding block (e.g., the smallest motion grid covering the center sample point) is used to derive the sub-CU's motion information. After identifying the motion information of the corresponding N×N block, it is converted into the reference index and motion vector of the current sub-CU in the same manner as the TMVP of HEVC, where motion scaling and other processes are also applied. For example, the decoder checks whether a low-latency condition is met (e.g., the POC of all reference images of the current image is less than the POC of the current image) and may use the motion vector MV. x (For example, the motion vector corresponding to the reference image list X) to predict the motion vector MV of each sub-CU. y (For example, where X equals 0 or 1 and Y equals 1-X).

[0102] 2.1.2 Example of Space-Time Motion Vector Prediction (STMVP)

[0103] In the STMVP method, the motion vector of the sub-CU is recursively derived according to the raster scan order. Figure 11 An example of a CU with four sub-blocks and neighboring blocks is shown. Consider an 8×8 CU1100 containing four 4×4 sub-CUs A(1101), B(1102), C(1103), and D(1104). The neighboring 4×4 blocks in the current frame are labeled a(1111), b(1112), c(1113), and d(1114).

[0104] Motion derivation of sub-CU A begins by identifying its two spatial neighbors. The first neighbor is the N×N block above sub-CU A1101 (block c 1103). If block c (1113) is unavailable or intra-coded, the other N×N blocks above sub-CU A (1101) are checked (starting from block c 1113, from left to right). The second neighbor is the block to the left of sub-CU A1101 (block b 1112). If block b (1112) is unavailable or intra-coded, the other blocks to the left of sub-CU A1101 are checked (starting from block b 1112, from top to bottom). Motion information obtained from neighboring blocks in each list is scaled to the first reference frame of the given list. Next, the temporal motion vector prediction (TMVP) of sub-block A1101 is derived by following the same procedure as the TMVP derivation specified in HEVC. Motion information of the juxtaposed block at D 1104 is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors are averaged separately for each reference list. The averaged motion vector is then designated as the motion vector for the current sub-CU.

[0105] 2.1.3 Example of Sub-CU Motion Prediction Mode Signaling

[0106] In some embodiments, a sub-CU mode is enabled as an additional Merge candidate, and no additional syntax elements are required to signal this mode. Two additional Merge candidates are added to the Merge candidate list of each CU to represent the ATMVP mode and the STMVP mode. In some embodiments, up to seven Merge candidates can be used if the sequence parameter set indicates that ATMVP and STMVP are enabled. The encoding logic for the additional Merge candidates is the same as that for the Merge candidates in HM, meaning that for each CU in a P or B stripe, the two additional Merge candidates may require two additional RD checks. In some embodiments, such as JEM, all binary bits (bins) of the Merge index are context-encoded using CABAC (Context-Based Adaptive Binary Arithmetic Encoding). In other embodiments, such as HEVC, only the first binary bit is context-encoded, while the remaining binary bits are context-bypass encoded.

[0107] 2.2 Adaptive Motion Vector Difference Resolution

[0108] In some embodiments, when the use_integer_mv_flag in the stripe header is equal to 0, the motion vector difference (MVD) between the PU's motion vector and the predicted motion vector is signaled in units of quarter-luminance samples. In JEM, Locally Adaptive Motion Vector Resolution (LAMVR) is introduced. In JEM, MVD can be encoded in units of quarter-luminance samples, integer luminance samples, or four luminance samples. The MVD resolution is controlled at the coding unit (CU) level, and the MVD resolution flag is conditionally signaled for each CU having at least one non-zero MVD component.

[0109] For a CU with at least one non-zero MVD component, signaling notifies a first flag to indicate whether quarter-luminance sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter-luminance sample MV precision is not used, signaling notifies another flag to indicate whether integer luminance sample MV precision or four-luminance sample MV precision is used.

[0110] When the first MVD resolution flag of the CU is zero or not encoded for the CU (meaning all MVDs in the CU are zero), a quarter-luminance sample MV resolution is used for the CU. When the CU uses integer luminance sample MV precision or four-luminance sample MV precision, the MVPs in the CU's AMVP candidate list are rounded to the corresponding precision.

[0111] In the encoder, CU-level RD checks are used to determine which MVD resolution to use for the CU. That is, for each MVD resolution, three CU-level RD checks are performed. To speed up the encoder, the following encoding scheme is applied in JEM.

[0112] --During the RD check of a CU with a normal quarter-luminance sample MVD resolution, the motion information of the current CU (integer luminance sample precision) is stored. The stored motion information (after rounding) is used as the starting point for further small-range motion vector refinement for the same CU with integer luminance sample and 4-luminance sample MVD resolutions during the RD check, so that the time-consuming motion estimation process is not repeated three times.

[0113] --Conditionally invoke the RD check for a CU with a 4-luminance sample MVD resolution. For a CU, skip the RD check for a 4-luminance sample MVD resolution if the RD cost integer luminance sample MVD resolution is much greater than a quarter luminance sample MVD resolution.

[0114] 2.3 Examples of Higher Motion Vector Storage Accuracy

[0115] In HEVC, motion vector precision is one-quarter of a pixel (one-quarter of a luma sample and one-eighth of a chroma sample for a 4:2:0 video). In JEM, the precision of internal motion vector storage and merge candidates is increased to 1 / 16 pixel. This higher motion vector precision (1 / 16 pixel) is used for motion-compensated inter-frame prediction in CUs encoded in skip / merge mode. For CUs encoded using normal AMVP mode, integer pixel or one-quarter pixel motion is used.

[0116] The SHVC upsampling interpolation filter, with the same filter length and normalization factor as the HEVC motion compensation interpolation filter, is used as the motion compensation interpolation filter for additional fractional pixel locations. In JEM, the chroma component motion vector accuracy is 1 / 32 sample, and the additional interpolation filter for the 1 / 32 pixel fractional location is derived by averaging the filters for two adjacent 1 / 16 pixel fractional locations.

[0117] 2.4 Example of Overlapping Block Motion Compensation (OBMC)

[0118] In JEM, OBMC can be enabled and disabled using CU-level syntax. When using OBMC in JEM, OBMC is performed on all Motion Compensation (MC) block boundaries except for the right and bottom boundaries of the CU. Furthermore, it is applied to both luma and chroma components. In JEM, MC blocks correspond to coded blocks. When encoding a CU using sub-CU modes (including sub-CUE, affine, and FRUC modes), each sub-block of the CU is an MC block. To handle CU boundaries uniformly, OBMC is performed at the sub-block level for all MC block boundaries, where the sub-block size is set to equal to 4×4, such as... Figure 12A and 12B As shown.

[0119] Figure 12A The sub-block at the CU / PU boundary is shown; the shaded sub-block is where the OBMC is applied. Similarly, Figure 12B The sub-blocks in the ATMVP pattern are shown.

[0120] When OBMC is applied to the current sub-block, in addition to the current motion vector, the motion vectors of the four connected neighboring sub-blocks (if available and different from the current motion vector) are also used to derive the prediction block for the current sub-block. These multiple prediction blocks based on multiple motion vectors are combined to generate the final prediction signal for the current sub-block.

[0121] The prediction block based on the motion vectors of neighboring sub-blocks is represented as P. N , where N indicates the indices of the neighboring upper, lower, left, and right sub-blocks, and the predicted block based on the motion vector of the current sub-block is represented as P. C When P N When the motion information is based on neighboring sub-blocks that contain the same motion information as the current sub-block, it does not start from P. N Execute OBMC. Otherwise, execute each P. N Samples added to P C In the same point in the middle, P is about to be N Add four rows / columns to P C The weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for P. N And the weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are applied to the PC. An exception is small MC blocks (i.e., when the height or width of the coded block is equal to 4 or the CU is encoded using a sub-CU mode), for which only the P is applied. N Add two rows / columns to P C In this case, the weighting factors {1 / 4, 1 / 8} are used for P. N And the weighting factors {3 / 4, 7 / 8} are used for P. CFor P generated based on the motion vectors of vertical (horizontal) neighboring sub-blocks N , will P N Samples in the same row (column) are added to P with the same weighting factor. C .

[0122] In JEM, for CUs with a size of 256 lumen samples or less, signaling informs a CU-level flag to indicate whether OBMC should be applied to the current CU. For CUs with a size exceeding 256 lumen samples or not encoded using AMVP mode, OBMC is applied by default. At the encoder, when OBMC is applied to the CU, its effects are considered during the motion estimation phase. The predicted signal formed by OBMC using motion information from the upper and left neighboring blocks is used to compensate for the upper and left boundaries of the original signal of the current CU, and then normal motion estimation processing is applied.

[0123] 2.5 Example of Local Illumination Compensation (LIC)

[0124] Illumination compensation (LIC) is based on a linear pattern for illumination variations, using a scaling factor a and an offset b. It is adaptively enabled or disabled for each inter-frame mode coding unit (CU).

[0125] When LIC is applied to CU, the least squares error method is used to derive parameters a and b by using the neighboring samples of the current CU and their corresponding reference samples. Figure 13 An example of neighboring samples used to derive the parameters of the IC algorithm is shown. More specifically, as... Figure 13 As shown, neighboring samples from the subsampled (2:1 subsampled) CU and corresponding samples from the reference image (identified by motion information from the current CU or subCU) are used. IC parameters are derived and applied to each prediction direction.

[0126] When encoding a CU in Merge mode, the LIC flag is copied from the neighboring block in a manner similar to motion information copying in Merge mode; otherwise, the CU signaling notifies the LIC flag to indicate whether LIC should be applied.

[0127] When LIC is enabled for an image, additional CU-level RD checks are required to determine whether LIC should be applied to the CU. When LIC is enabled for the CU, Mean-Removed Sum Of Absolute Difference (MR-SAD) and Mean-Removed Sum Of Absolute Hadamard-Transformed Difference (MR-SATD) are used instead of SAD and SATD for integer pixel motion search and fractional pixel motion search, respectively.

[0128] To reduce coding complexity, the following coding scheme is applied in JEM.

[0129] When there is no significant lighting change between the current image and its reference images, LIC is disabled for the entire image. To identify this situation, histograms of the current image and each reference image of the current image are calculated at the encoder. If the histogram difference between the current image and each reference image of the current image is less than a given threshold, LIC is disabled for the current image; otherwise, LIC is enabled for the current image.

[0130] 2.6 Examples of Affine Motion Compensation Prediction

[0131] In HEVC, only the translational motion mode is applied to motion compensation prediction (MCP). However, the camera and object may exhibit various motions, such as zooming in / out, rotation, perspective motion, and / or other irregular motions. On the other hand, JEM applies a simplified affine transformation motion compensation prediction. Figure 14 An example of the affine motion field of block 1400 described by two control point motion vectors V0 and V1 is shown. The motion vector field (MVF) of block 1400 is described by the following equation:

[0132]

[0133] like Figure 14 As shown, (v 0x ,v 0y (v) is the motion vector of the top-left control point. 1x ,v 1y () is the motion vector of the upper right control point. To further simplify motion compensation prediction, sub-block-based affine transformation prediction can be applied. The sub-block size M×N is derived as follows:

[0134]

[0135] Here, MvPre is the fractional precision of the motion vector (e.g., 1 / 16 in JEM), (v 2x ,v 2y ) is the motion vector of the lower left control point calculated according to Equation 1. If necessary, M and N can be adjusted downwards to make them the divisors of w and h, respectively.

[0136] Figure 15 An example of the affine MVF for each sub-block of block 1500 is shown. To derive the motion vector for each M×N sub-block, the motion vector of the center sample point of each sub-block is calculated according to Equation 1 and rounded to the fractional precision of the motion vector (e.g., 1 / 16 in JEM). A motion-compensated interpolation filter is then applied to generate a prediction for each sub-block using the derived motion vector. After MCP, the high-precision motion vector for each sub-block is rounded and saved with the same precision as the normal motion vector.

[0137] 2.6.1 Implementation Examples of AF_INTER Mode

[0138] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. AF_INTER mode can be applied to CUs with both width and height greater than 8. The affine flag at the CU level is signaled in the bitstream to indicate whether AF_INTER mode is used. In AF_INTER mode, a motion vector pair {(v0,v1)|v0={v...} is constructed using neighboring blocks. A ,v B ,v c},v1={v D ,v E The candidate list of}}.

[0139] Figure 16 An example of motion vector prediction (MVP) for block 1600 in AF_INTER mode is shown. Figure 16As shown, v0 is selected from the motion vectors of sub-blocks A, B, or C. Motion vectors from neighboring blocks can be scaled according to a reference list. The scaling can also be based on the relationship between the Picture Order Count (POC) for the reference of the neighboring block, the POC for the reference of the current CU, and the POC of the current CU. The method for selecting v1 from neighboring sub-blocks D and E is similar. If the number of candidates in the candidate list is less than two, the list can be populated by motion vector pairs formed by copying each AMVP candidate. When the candidate list is greater than two, the candidates can first be sorted according to neighboring motion vectors (e.g., based on the similarity of two motion vectors in a candidate pair). In some embodiments, the first two candidates are retained. In some embodiments, a rate distortion (RD) cost check is used to determine which motion vector pair candidate is selected as the Control Point Motion Vector Prediction (CPMVP) for the current CU. The index indicating the position of the CPMVP in the candidate list can be signaled in the bitstream. After determining the CPMVP of the current affine CU, affine motion estimation is applied and the Control Point Motion Vector (CPMV) is found. Then, the difference between CPMV and CPMVP is communicated in the bitstream using signaling.

[0140] In AF_INTER mode, when using the 4 / 6 parameter affine mode, 2 / 3 control points are required. Therefore, these control points need to be encoded with 2 / 3 MVD, such as... Figure 17A and 17B As shown. In the existing implementation, the derivation of MV is as follows, for example, predicting mvd1 and mvd2 from mvd0.

[0141]

[0142]

[0143]

[0144] here, mvd i and MV i These are the predicted motion vector, motion vector difference, and motion vector for the top-left pixel (i=0), top-right pixel (i=1), or bottom-left pixel (i=2), respectively. Figure 18B As shown. In some embodiments, the sum of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) is equal to the sum of their two components. For example, newMV = mvA + mvB implies that the two components of newMV are set to (xA + xB) and (yA + yB), respectively.

[0145] 2.6.2. Example of Fast Affine ME Algorithm in AF_INTER Mode

[0146] In some embodiments of affine patterns, it is necessary to jointly determine the MV (Multiple Views) of two or three control points. Jointly searching multiple MVs directly is computationally complex. For example, a fast affine ME (Multiple Views) algorithm has been proposed and applied to VTM / BMS (Variable Memory Management System).

[0147] For example, the fast affine ME algorithm is described for a 4-parameter affine model, and the idea can be extended to a 6-parameter affine model.

[0148]

[0149]

[0150] Replace (a 1) with a', so that the motion vector is rewritten as:

[0151]

[0152] Assuming the motion vectors of the two control points (0, 0) and (0, w) are known, the affine parameters can be derived from equation (5) as follows:

[0153]

[0154] The motion vector can be rewritten in vector form as:

[0155]

[0156] Here, P = (x, y) is the pixel position.

[0157]

[0158]

[0159] In some embodiments, the MVD of AF_INTER can be iteratively derived on the encoder. The MVD... i (P) represents the MV derived for position P in the i-th iteration, dMV C i It is represented as MV in the i-th iteration. C The update increment. Then in the (i+1)th iteration,

[0160]

[0161] Pic ref Pic is shown as a reference image. cur This represents the current image, and Q = P + MV. i(P). If MSE is used as the matching criterion, then the function that needs to be minimized can be written as:

[0162]

[0163]

[0164] If we assume Small enough, then It can be approximately rewritten based on the first-order Taylor expansion as follows:

[0165]

[0166] here, If E is used i+1 (P)=Pic cur (P)-Pic ref (Q), then

[0167]

[0168] The term can be derived by setting the derivative of the error function to zero. Then it can be based on Calculate the delta (MV) values ​​for control points (0, 0) and (0, w).

[0169]

[0170]

[0171]

[0172]

[0173] In some embodiments, the MVD derivation process can be iterated n times, and the final MVD calculation can be as follows:

[0174]

[0175]

[0176]

[0177]

[0178] In the aforementioned implementation, predicting the increment MV of the control point (0, w) represented by mvd1 from the increment MV of the control point (0, 0) represented by mvd0 only results in It is encoded as mvd1.

[0179] 2.6.3 Implementation Examples of AF_MERGE Mode

[0180] When CU is applied in AF_MERGE mode, it obtains the first block encoded in affine mode from the valid neighboring reconstructed blocks. Figure 18A This shows an example of the current selection order of candidate blocks for the CU 1800. (Example:) Figure 18A As shown, the selection order can be from the left (1801), top (1802), top right (1803), bottom left (1804) to top left (1805) of the current CU 1800. Figure 18B Another example of a candidate block for the current CU 1800 in AF_MERGE mode is shown. If the adjacent lower-left block 1801 is encoded in affine mode, such as... Figure 18B As shown, the motion vectors v2, v3, and v4 of the upper left, upper right, and lower left corners of the CU containing block A are derived. Furthermore, the motion vector v0 of the upper left corner of the current CU 1800 is calculated based on v2, v3, and v4. The motion vector v1 of the upper right corner of the current CU can be calculated accordingly.

[0181] After calculating the CPMV v0 and v1 of the current CU according to the affine motion mode in equation (1), the MVF of the current CU can be generated. In order to identify whether the current CU is encoded in AF_MERGE mode, when at least one neighboring block is encoded in affine mode, the affine flag can be notified by signaling in the bit stream.

[0182] 2.7 Example of Motion Vector Derivation (PMMVD) for Pattern Matching

[0183] PMMVD mode is a special merge mode based on the Frame-Rate Up Conversion (FRUC) method. This mode uses the decoder to derive block motion information instead of sending signaling to notify the block of its motion.

[0184] When the Merge flag of the CU is true, the FRUC flag can be signaled to the CU. When the FRUC flag is false, the Merge index can be signaled and the regular Merge mode can be used. When the FRUC flag is true, additional FRUC mode flags can be signaled to indicate which method (e.g., bilateral matching or template matching) will be used to derive the block's motion information.

[0185] On the encoder side, the decision on whether to use the FRUC Merge mode for the CU is based on the RD cost selection made for normal Merge candidates. For example, multiple matching modes for the CU (e.g., bilateral matching and template matching) are examined using RD cost selection. The matching mode that results in the minimum cost is further compared with other CU modes. If the FRUC matching mode is the most efficient mode, the FRUC flag is set to true for the CU, and the relevant matching mode is used.

[0186] Typically, the motion derivation process in the FRUC Merge model has two steps: first, a CU-level motion search is performed, followed by sub-CU-level motion refinement. At the CU level, the original motion vector of the entire CU is derived based on bilateral matching or template matching. First, a candidate MV list is generated, and the candidate that causes the minimum matching cost is selected as the starting point for further CU-level refinement. Then, a local search based on bilateral matching or template matching is performed near the starting point. The MV result with the minimum matching cost is taken as the MV of the entire CU. Subsequently, using the derived CU motion vector as the starting point, the motion information is further refined at the sub-CU level.

[0187] For example, the following derivation process is performed for the motion information derivation of a W×H CU. In the first stage, the MV of the entire W×H CU is derived. In the second stage, the CU is further divided into M×M sub-CUs. The value of M is calculated as shown in (16), and D is a predefined partitioning depth, which is set to 3 by default in JEM. Then the MV of each sub-CU is derived.

[0188]

[0189] Figure 19 An example of bilateral matching used in the Frame Rate Upconversion (FRUC) method is shown. Bilateral matching is used to derive motion information of the current CU (1900) by finding the closest match between two blocks along the motion trajectory of the current CU (1900) in two different reference images (1910, 1911). Under the assumption of continuous motion trajectories, the motion vectors MV0 (1901) and MV1 (1902) pointing to the two reference blocks are proportional to the temporal distances between the current image and the two reference images (e.g., TD0 (1903) and TD1 (1904)). In some embodiments, bilateral matching becomes a mirror-based bidirectional MV when the current image 1900 is temporally between the two reference images (1910, 1911) and the temporal distances from the current image to the two reference images are equal.

[0190] Figure 20An example of template matching used in the Frame Rate Upconversion (FRUC) method is shown. Template matching is used to deduce motion information for the current CU 2000 by finding the closest match between a template in the current image (the top and / or left neighboring block of the current CU) and a block in the reference image 2010 (e.g., of the same size as the template). In addition to the FRUCMerge mode described above, template matching can also be applied to the AMVP mode. As in both JEM and HEVC, AMVP has two candidates. New candidates are deduced using the template matching method. If a newly deduced candidate by template matching differs from the first existing AMVP candidate, it is inserted at the beginning of the AMVP candidate list, and the list size is then set to 2 (e.g., by removing the second existing AMVP candidate). When applied to AMVP mode, only CU-level search is performed.

[0191] The MV candidates set at the CU level may include the following: (1) the original AMVP candidate if the current CU is in AMVP mode, (2) all Merge candidates, (3) several MVs in the interpolated MV field (described later), and the top and left adjacent motion vectors.

[0192] When using bilateral matching, each valid MV of the Merge candidate can be used as input to generate MV pairs assuming bilateral matching. For example, in reference list A, a valid MV of the Merge candidate is (MVa, refa). Then, in another reference list B, a reference image refb for its paired bilateral MV is found such that refa and refb are on different sides of the current image in the temporal domain. If such a refb is not available in reference list B, then refb is determined to be a different reference from refa, and its temporal distance to the current image is the minimum in list B. After determining refb, MVb is derived by scaling MVa based on the temporal distance between refa and refb in the current image.

[0193] In some embodiments, four MVs from the interpolated MV field can also be added to the CU-level candidate list. More specifically, the interpolated MVs at the current CU positions (0,0), (W / 2,0), (0,H / 2), and (W / 2,H / 2) are added. When FRUC is applied to the AMVP pattern, the original AMVP candidates are also added to the CU-level MV candidate set. In some embodiments, at the CU level, 15 MVs are added to the candidate list for AMVPCU and 13 MVs are added to the candidate list for MergeCU.

[0194] The MV candidates set at the sub-CU level include: (1) MVs determined from the CU level search, (2) neighboring MVs at the top, left, top-left, and top-right corners, (3) scaled versions of the juxtaposed MVs from the reference image, (4) one or more ATMVP candidates (up to four), and (5) one or more STMVP candidates (e.g., up to four). The scaled MVs from the reference image are derived as follows. The reference images in both lists are traversed. The MVs at the juxtaposed positions of the sub-CUs in the reference images are scaled to the reference of the starting CU level MV. The ATMVP and STMVP candidates can be limited to the first four. At the sub-CU level, one or more MVs (e.g., up to 17) are added to the candidate list.

[0195] Generation of interpolated MV fields. Before encoding the frames, an interpolated motion field is generated for the entire image based on a one-sided ME. The motion field can then be used later as a CU-level or sub-CU-level MV candidate.

[0196] In some embodiments, the motion field of each reference image in the two reference lists is traversed in a 4×4 block-level manner. Figure 21 An example of one-sided motion estimation (ME)2100 in the FRUC method is shown. For each 4×4 block, if the motion associated with the block passes through 4×4 blocks in the current image and the block is not assigned any interpolated motion, the motion of the reference block is scaled to the current image according to the temporal distances TD0 and TD1 (in the same way as the MV scaling of TMVP in HEVC), and the scaled motion is assigned to the block in the current frame. If no scaled MV is assigned to the 4×4 block, the block's motion is marked as unavailable in the interpolated motion field.

[0197] Interpolation and matching costs. When the motion vector points to the fractional sample point location, motion-compensated interpolation is required. To reduce complexity, bilinear interpolation can be used for bilateral matching and template matching instead of conventional 8-tap HEVC interpolation.

[0198] The calculation of matching cost differs slightly at different steps. When selecting candidates from the candidate set at the CU level, the matching cost can be the absolute sum difference (SAD) of bilateral matching or template matching. After determining the starting MV, the matching cost C of bilateral matching in the sub-CU level search is calculated as follows:

[0199]

[0200] Here, w is the weighting factor. In some embodiments, w can be set to 4. MV and MV s These indicate the current MV and the starting MV, respectively. SAD can still be used as the matching cost for template matching in sub-CU level searches.

[0201] In FRUC mode, the motion signature (MV) is derived solely using luma samples. The derived motion is then used for both luma and chroma predictions in the inter-frame prediction (MC). After determining the MV, the final MC is performed using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma.

[0202] MV refinement is a pattern-based MV search, categorized by bilateral matching cost or template matching cost. JEM supports two search modes—Unrestricted Center-Biased Diamond Search (UCBDS) and Adaptive Cross Search—performing MV refinement at the CU and sub-CU levels, respectively. For both CU and sub-CU level MV refinement, the MV is directly searched with a quarter-luminance sample MV precision, followed by an eighth-luminance sample MV refinement. The search range for MV refinement in the CU and sub-CU steps is set to equal 8 luminance samples.

[0203] In the bilateral matching Merge mode, bidirectional prediction is applied because the motion information of the CU is derived based on the nearest match between two blocks along the current CU's motion trajectory in two different reference images. In the template matching Merge mode, the encoder can select for the CU from unidirectional predictions in list 0, unidirectional predictions in list 1, or bidirectional predictions. The template matching cost can be selected based on the following:

[0204] If costBi <= factor * min(cost0, cost1)

[0205] Then use bidirectional prediction;

[0206] Otherwise, if cost0 <= cost1

[0207] Then use the one-way prediction from list 0;

[0208] otherwise,

[0209] Use the one-way prediction from List 1;

[0210] Here, cost0 is the SAD of template matching in list 0, cost1 is the SAD of template matching in list 1, and costBi is the SAD of bidirectional prediction template matching. For example, when the value of factor equals 1.25, this means that the selection process is biased towards bidirectional prediction. Inter-frame prediction direction selection can be applied to CU-level template matching processes.

[0211] 2.8 Examples of bidirectional optical flow (BIO)

[0212] The bidirectional optical flow (BIO) method is a sample-level motion refinement that is performed on top of block-by-block motion compensation for bidirectional prediction. In some embodiments, sample-level motion refinement does not use signaling.

[0213] Let I (k) The brightness value of reference k (k=0,1) after block motion compensation, and I (k) The horizontal and vertical components of the gradient. Assuming optical flow is effective, the motion vector field (v) x ,v y The following formula is given:

[0214]

[0215] Combining this optical flow equation with the Hermitian interpolation of the motion trajectory of each sample point yields a unique third-order polynomial, which ultimately matches the function value I. (k) and its derivative Both. The value of the third-order polynomial at t=0 is the BIO prediction:

[0216]

[0217] Figure 22 An example optical flow trajectory in the bidirectional optical flow (BIO) method is shown. Here, τ0 and τ1 represent the distances to the reference frame, such as... Figure 21 As shown. The distances t0 and τ1 are calculated based on the POC of Ref0 and Ref1: τ0 = POC(current) - POC(Ref0), τ1 = POC(Ref1) - POC(current). If both predictions originate from the same temporal direction (both from the past or both from the future), then the sign is different, i.e., τ0·τ1 < 0. In this case, BIO is applied only when the predictions do not originate from the same time (i.e., τ0 ≠ τ1), both reference regions have non-zero motion (MVx0, MVy0, MVx1, MVy1 ≠ 0), and the block motion vector is proportional to the temporal distance (MVx0 / MVx1 = MVy0 / MVy1 = -τ0 / τ1).

[0218] The motion vector field (v) is determined by minimizing the difference Δ between the values ​​at points A and B. x ,v y ). Figure 9 An example of the intersection of the motion trajectory and the reference frame plane is shown. The pattern uses only the first linear term of the local Taylor expansion of Δ:

[0219]

[0220] All values ​​in the above equation depend on the sample point location, denoted as (i′, j′). Assuming the motion is consistent in the local surrounding region, minimize Δ within a square window Ω of (2M+1)×(2M+1) centered on the current prediction point, where M equals 2:

[0221]

[0222] For this optimization problem, JEM uses a simplification method, first minimizing in the vertical direction and then minimizing in the horizontal direction. This results in the following:

[0223]

[0224]

[0225] in,

[0226]

[0227] To avoid division by zero or very small values, regularization parameters r and m can be introduced in equations 28 and 29.

[0228] r = 500·4 d-8 (31)

[0229] m = 700·4 d-8 (32)

[0230] Here, d is the bit depth of the video sample.

[0231] To ensure that BIO memory access remains the same as regular bidirectional predictive motion compensation, all predicted and gradient values ​​I are computed only for the position within the current block. (k) , Figure 23A An example of an access location outside block 2300 is shown. Figure 23A As shown, in equation (30), a (2M+1)×(2M+1) square window Ω centered on the current prediction point on the boundary of the prediction block needs to access locations outside the block. In JEM, the I outside the block... (k) , The value is set to be equal to the nearest available value within the block. For example, this could be implemented to fill region 2301, such as... Figure 23B As shown.

[0232] Using BIO, the motion field can be refined for each sample point. To reduce computational complexity, a block-based BIO design is used in JEM. Motion refinement can be calculated based on 4×4 blocks. In block-based BIO, the s in Equation 30 of all samples in a 4×4 block can be aggregated. n The value of s, then sn The aggregated values ​​are used to derive the BIO motion vector offset for a 4×4 block. More specifically, the following formula can be used for block-based BIO derivation:

[0233]

[0234] Here, b k Let represent the sample set belonging to the k-th 4×4 block of the prediction block. Then, in equations 28 and 29, s... n Replace with ((s) n,bk )>>4), to derive the associated motion vector offset.

[0235] In some scenarios, the MV regiment of BDOF may be unreliable due to noise or irregular motion. Therefore, in BIO, the size of the MV regiment is thresholded. The threshold is determined based on whether all reference images of the current image come from the same direction. For example, if all reference images of the current image come from the same direction, the threshold value is set to 12×2. 14-d Otherwise, set it to 12×2 13-d .

[0236] The gradient of the BIO can be simultaneously computed using motion-compensated interpolation that operates in accordance with the HEVC motion compensation process (e.g., 2D separable finite impulse response (FIR)). In some embodiments, the input to the 2D separable FIR is a reference frame sample that is identical to the motion compensation process and the fractional position (fracX, fracY) based on the fractional portion of the block motion vector. For the horizontal gradient... First, the vertically interpolated signal is used using BIOfilterS, corresponding to the fractional position fracY with a descaling offset of d-8. Then, a gradient filter BIOfilterG is applied in the horizontal direction, corresponding to the fractional position fracX with a descaling offset of 18-d. For the vertical gradient... First, a gradient filter is applied vertically using BIOfilterG, corresponding to the fractional position fracY with a descaling offset of d-8. Then, signal displacement is performed horizontally using BIOfilterS, corresponding to the fractional position fracX with a descaling offset of 18-d. The lengths of the interpolation filter BIOfilterG for gradient calculation and the interpolation filter BIOfilterS for signal displacement can be relatively short (e.g., 6 taps) to maintain reasonable complexity. Table 1 shows examples of filters that can be used for gradient calculation at different fractional positions of block motion vectors in BIO. Table 2 shows examples of interpolation filters that can be used for predictive signal generation in BIO.

[0237] Table 1: Exemplary filters for gradient computation in BIO

[0238] Fractional pixel position Gradient interpolation filter (BIOfilterG) 0 {8,-39,-3,46,-17,5} 1 / 16 {8,-32,-13,50,-18,5} 1 / 8 {7,-27,-20,54,-19,5} 3 / 16 {6,-21,-29,57,-18,5} 1 / 4 {4,-17,-36,60,-15,4} 5 / 16 {3,-9,-44,61,-15,4} 3 / 8 {1,-4,-48,61,-13,3} 7 / 16 {0,1,-54,60,-9,2} 1 / 2 {-1,4,-57,57,-4,1}

[0239] Table 2: Exemplary interpolation filters used for predictive signal generation in BIO

[0240] Fractional pixel position Interpolation filters for predicted signals (BIOfilters) 0 {0,0,64,0,0,0} 1 / 16 {1,-3,64,4,-2,0} 1 / 8 {1,-6,62,9,-3,1} 3 / 16 {2,-8,60,14,-5,1} 1 / 4 {2,-9,57,19,-7,2} 5 / 16 {3,-10,53,24,-8,2} 3 / 8 {3,-11,50,29,-9,2} 7 / 16 {3,-11,44,35,-10,3} 1 / 2 {3,-10,35,44,-11,3}

[0241] In JEM, BIO can be applied to all dual-prediction blocks when the two predictions come from different reference images. BIO can be disabled when Local Illumination Compensation (LIC) is enabled for a CU.

[0242] In some embodiments, OBMC is applied to blocks after the normal MC process. To reduce computational complexity, BIO may not be applied during OBMC. This means that BIO is only applied to the block's MC process when its own MV is used, and not during OBMC when the MV of an adjacent block is used.

[0243] 2.9 Example of Decoder-Side Motion Vector Refinement (DMVR)

[0244] In bidirectional prediction, for the prediction of a block region, two prediction blocks formed by the motion vectors (MV) of list0 and list1 are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of the bidirectional prediction are further refined through a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral templates and reconstructed samples in the reference image to obtain a refined MV without transmitting additional motion information.

[0245] In DMVR, the bilateral template is generated as a weighted combination (i.e., average) of the two prediction blocks from the original MV0 of list 0 and the MV1 of list 1, respectively, as follows: Figure 24 As shown. The template matching operation involves calculating a cost metric between the generated template and the sample region (around the original predicted block) in the reference image. For each of the two reference images, the MV that produces the minimum template cost is considered as the updated MV of that list to replace the original MV. In JEM, nine MV candidates are searched for each list. These nine MV candidates include the original MV and eight surrounding MVs that have a brightness sample offset from the original MV in the horizontal or vertical direction or in both directions. Finally, the two new MVs, i.e., as shown... Figure 24 MV0' and MV1' shown are used to generate the final bidirectional prediction results. The sum of absolute differences (SAD) is used as the cost metric.

[0246] DMVR is applied to the Merge pattern of bidirectional prediction, where one MV comes from a past reference image and the other from a future reference image, without transferring additional syntax elements. In JEM, DMVR is not applied when LIC, affine motion, FRUC, or subCU Merge candidates are enabled for a CU.

[0247] 2.10 Example of symmetric motion vector difference

[0248] Symmetric Motion Vector Difference (SMVD) can be used to encode MVD more effectively.

[0249] First, at the strip level, the variables BiDirPredFlag, RefIdxSymL0, and RefIdxSymL1 are derived as follows:

[0250] • Search for the closest preceding reference image in reference image list 0. If found, set RefIdxSymL0 to the reference index of the preceding image.

[0251] • Search for the closest backward reference image in reference image list 1. If found, set RefIdxSymL1 to the reference index of the backward image.

[0252] If both the forward and backward images are found, set BiDirPredFlag to 1.

[0253] Otherwise, the following applies:

[0254] • Search for the closest backward reference image in reference image list 0. If found, set RefIdxSymL0 to the reference index of the backward image.

[0255] • Search for the closest preceding reference image in reference image list 1. If found, set RefIdxSymL1 to the reference index of the preceding image.

[0256] If both the backward and forward images are found, set BiDirPredFlag to 1. Otherwise, set BiDirPredFlag to 0.

[0257] Secondly, at the CU level, if the CU's prediction direction is bidirectional and BiDirPredFlag equals 1, then the symmetric mode flag of the symmetric mode is explicitly indicated by signaling.

[0258] When this flag is true, only mvp_l0_flag, mvp_l1_flag, and MVD0 are explicitly notified via signaling. For list 0 and list 1, the reference indices are set to equal RefIdxSymL0 and RefIdxSymL1, respectively. Only MVD1 is set to equal –MVD0. The final motion vector is shown in the following formula.

[0259]

[0260] Figure 27 An example of a symmetric pattern is shown.

[0261] Table 3 shows the modifications to the coding unit syntax in bold and / or italics.

[0262] Table 3: Modifications in the coding unit syntax

[0263]

[0264]

[0265] 3. Context-Adaptive Binary Arithmetic Coding (CABAC)

[0266] 3.1 CABAC Design in HEVC

[0267] 3.1.1 Context Representation and Initialization Process in HEVC

[0268] In HEVC, for each context variable, two variables, pStateIdx and valMps, are initialized.

[0269] From the 8-bit table entry initValue, the two 4-bit variables slopeIdx and offsetIdx are derived as follows:

[0270] slopeIdx = initValue >> 4

[0271] offsetIdx = initValue & 15 (5)

[0272] The variables m and n used in the initialization of the context variables are derived from lopedIdx and offsetIdx as follows:

[0273] m = slopeIdx * 5 - 45

[0274] n=(offsetIdx<<3)-16 (6)

[0275] The two values ​​assigned to pStateIdx and valMps for initialization are derived from the brightness quantization parameters of the stripe, represented as SliceQpY. Given variables m and n, the initialization is specified as follows:

[0276] preCtxState=Clip3(1,126,((m*Clip3(0,51,SliceQp Y ))>>4)+n)

[0277] valMps=(preCtxState<=63)? 0:1 (7)

[0278] pStateIdx=valMps? (preCtxState-64):(63-preCtxState)

[0279] 3.1.2 State Transition Process in HEVC

[0280] The input to this process is the current pStateIdx, the decoded values ​​of the context variables binVal and valMps associated with ctxTable and ctxIdx.

[0281] The output of this process is the updated pStateIdx and valMps of the context variables associated with ctxIdx.

[0282] Depending on the decoded value binVal, the updates of the two variables pStateIdx and valMps associated with ctxIdx are derived in (37) as follows:

[0283]

[0284] 3.2 CABAC Design in VVC

[0285] The context-adaptive binary arithmetic encoder (BAC) in VVC has been modified, differing from HEVC in both the context update process and the arithmetic encoder.

[0286] This is a summary of the recently adopted proposal (JVET-M0473, CE test 5.1.13).

[0287] Table 4: Summary of CABAC modifications in VCC

[0288]

[0289]

[0290] 3.2.1 Context Initialization Process in VVC

[0291] In VVC, the two values ​​assigned to pStateIdx0 and pStateIdx1 for initialization are derived from SliceQpY. Given variables m and n, the initialization is specified as follows:

[0292] preCtxState=Clip3(0,127,((m*Clip3(0,51,SliceQp Y ))>>4)+n)

[0293] pStateIdx0=initStateIdxToState[preCtxState]>>4

[0294] pStateIdx1=initStateIdxToState[preCtxState] (9)

[0295] 3.2.2 State transition process in VVC

[0296] The input to this process is the current pStateIdx0 and pStateIdx1, as well as the decoded value binVal.

[0297] The output of this process is an update of the context variables pStateIdx0 and pStateIdx1 associated with ctxIdx.

[0298] The variables shift0 (corresponding to variable a in the summary of CABAC modifications in VVC Table 4) and shift1 (corresponding to variable b in the summary of CABAC modifications in VVC Table 4e) are derived from the shiftIdx values ​​associated with ctxTable and ctxInc.

[0299] shift0 = (shiftIdx >> 2) + 2

[0300] shift1=(shiftIdx&3)+3+shift0 (10)

[0301] Depending on the decoded value binVal, the update derivation for the two variables pStateIdx0 and pStateIdx1 associated with ctxIdx is as follows:

[0302] pStateIdx0=pStateIdx0-(pStateIdx0>>shift0)+(1023*binVal>>shift0)

[0303] pStateIdx1=pStateIdx1-(pStateIdx1>>shift1)+(16383*binVal>>shift1)(11)

[0304] 4. Disadvantages of existing implementation methods

[0305] In some existing implementations, when the MV / MV difference (MVD) can be selected from a set of multiple MV / MVD precisions of an affine coding block, it remains uncertain how to obtain a more accurate motion vector.

[0306] In other existing implementations, MV / MVD precision information also plays an important role in determining the overall coding gain of AMVR applied to affine modes, but achieving this goal remains uncertain.

[0307] 5. Example method for MV predictor using AMVR's affine mode

[0308] The embodiments of the currently disclosed technology overcome the shortcomings of existing implementations, thereby providing video coding with higher coding efficiency. Based on the disclosed technology, the derivation and signaling of a motion vector predictor using affine modes with Adaptive Motion Vector Resolution (AMVR) that can enhance existing and future video coding standards are illustrated in the examples described below for various implementations. The examples of the disclosed technology provided below illustrate general concepts and are not intended to be construed as limiting. In one example, the various features described in these examples can be combined unless explicitly stated otherwise.

[0309] In some embodiments, when applying AMVR, the following examples can be applied to either affine mode or normal mode. These examples assume that the precision Prec (i.e., the MV has a precision of 1 / (2^Prec)) is used to encode the MVD in AF_INTER mode or in normal inter-frame mode. The motion vector predictor (e.g., inherited from the neighboring block MV) and its precision are respectively determined by MVPred (MVPred X ,MVPred Y ) and PredPrec represent.

[0310] Improvements to AMVR's affine mode support

[0311] 1. The allowed MVD precision set may vary by image, by strip, or by block.

[0312] a. In one example, the allowed set of MVD precision can depend on encoding information such as block size, block shape, etc.

[0313] b. Allowed MV precision sets can be predefined, such as {1 / 16, 1 / 4, 1}.

[0314] c. The allowed MV precision can be indicated by signaling in SPS / PPS / VPS / sequence header / picture header / strip header / CTU group, etc.

[0315] d. The signaling for selecting the MV precision from the set of allowed MV precisions also depends on the number of MV precisions allowed for the block.

[0316] 2. Signal the syntax elements to the decoder to indicate the MVD precision used in the affine inter-frame mode.

[0317] a. In one example, only a single syntax element is used to indicate the MVD precision applied to affine mode and AMVR mode.

[0318] i. In one example, the same semantics are used, that is, for AMVR and affine modes, the same syntax element values ​​are mapped to the same MVD precision.

[0319] ii. Alternatively, the semantics of a single syntactic element differ for AMVR and affine modes. That is, for AMVR and affine modes, the same value of a syntactic element can be mapped to different MVD precisions.

[0320] b. In one example, when the affine mode uses the same MVD precision set as AMVR (e.g., the MVD precision set is {1, 1 / 4, 4} pixels), the MVD precision syntax element in AMVR is reused in the affine mode, i.e., only a single syntax element is used.

[0321] i. Alternatively, when encoding / decoding the syntax element in the CABAC encoder / decoder, the same or different context models can be used for AMVR and affine modes.

[0322] ii. Alternatively, the syntax element may have different semantics in AMVR and affine mode. For example, syntax elements equal to 0, 1, and 2 indicate 1 / 4 pixel, 1 pixel, and 4 pixel MV precision in AMVR, respectively, while in affine mode, syntax elements equal to 0, 1, and 2 indicate 1 / 4 pixel, 1 / 16 pixel, and 1 pixel MV precision.

[0323] c. In one example, when the affine mode uses the same number of MVD precisions as AMVR, but uses a different MVD precision set (e.g., AMVR's MVD precision set is {1, 1 / 4, 4} pixels, while the affine mode's MVD precision set is {1 / 16, 1 / 4, 1} pixels), then the MVD precision syntax element in AMVR is reused in the affine mode; that is, only a single syntax element is used.

[0324] i. Alternatively, when encoding / decoding the syntax element in the CABAC encoder / decoder, the same or different context modes can be used for AMVR and affine modes.

[0325] ii. Alternatively, this grammatical element may have different semantics in AMVR and affine modes.

[0326] d. In one example, affine mode uses less MVD precision than AMVR, reusing MVD precision syntax elements from AMVR in affine mode. However, only a subset of syntax element values ​​are valid for affine mode.

[0327] i. Alternatively, when encoding / decoding the syntax element in the CABAC encoder / decoder, the same or different context models can be used for AMVR and affine modes.

[0328] ii. Alternatively, this grammatical element may have different semantics in AMVR and affine modes.

[0329] e. In one example, affine mode uses more MVD precision than AMVR, reusing MVD precision syntax elements from AMVR in affine mode. However, such class syntax elements are extended to allow more values ​​in affine mode.

[0330] i. Alternatively, when encoding / decoding the syntax element in the CABAC encoder / decoder, the same or different context models can be used for AMVR and affine modes.

[0331] ii. Alternatively, this grammatical element may have different semantics in AMVR and affine modes.

[0332] f. In one example, a new syntax element is used to encode the MVD precision of the affine mode; that is, two different syntax elements are used to encode the MVD precision of both AMVR and the affine mode.

[0333] g. The syntax for signaling an indication of the MVD precision of an affine mode can be used if one or all of the following conditions are true:

[0334] i. The MVD of all control points is not zero.

[0335] ii. The MVD of at least one control point is not zero.

[0336] iii. The MVD of a control point (e.g., the first CPMV) is not zero.

[0337] In this case, when one or all of the above conditions are not met, there is no need to notify the MVD accuracy via signaling.

[0338] h. Syntax elements indicating MVD precision for affine or AMVR modes can be encoded using context, and the context depends on the encoding information.

[0339] i. In one example, when there is only a single syntactic element, the context can depend on whether the current block is encoded in affine mode.

[0340] i. In one example, the context may depend on the block size / block shape / MVD accuracy / temporal layer index / prediction direction of neighboring blocks, etc.

[0341] j. Signaling can be used in SPS / PPS / VPS / sequence header / image header / strip header / CTU group, etc., to indicate whether multiple MVD precisions are enabled in affine mode.

[0342] i. In one example, whether to use signaling to notify the user of enabling or disabling multiple MVD precisions for affine mode can depend on other syntax elements. For example, when affine mode is enabled, signaling is used to notify the user of enabling or disabling multiple MV and / or MVP and / or MVD precisions for affine mode; and when affine mode is disabled, no signaling is used to notify the user of this information and it is inferred as 0.

[0343] k. Alternatively, multiple syntax elements can be signaled in affine inter-frame mode to indicate the MV and / or MVP and / or MVD precision used (collectively referred to as "MVD precision" in the following discussion).

[0344] i. In one example, the syntax elements used to indicate the MVD precision used in affine inter-frame mode and normal inter-frame mode can be different.

[0345] 1. The number of syntax elements indicating MVD precision used in affine inter-frame mode and normal inter-frame mode can be different.

[0346] 2. The semantics of syntax elements indicating the MVD precision used in affine inter-frame mode and normal inter-frame mode can be different.

[0347] 3. The context model in the arithmetic coding that encodes a syntax element indicating the MVD precision used in affine inter-frame mode and normal inter-frame mode can be different.

[0348] 4. The methods for deriving the context model in arithmetic coding that encodes a syntax element indicating the MVD precision used in affine inter-frame mode and normal inter-frame mode can differ.

[0349] ii. In one example, the first syntax element (e.g., amvr_flag) can be signaled to indicate whether AMVR is applied in the affine-coded block.

[0350] 1. Conditionally notify the first syntax element using signaling.

[0351] a. In one example, when the current block is encoded in a certain mode (e.g., CPR / IBC mode), the signaling of the first syntax element (amvr_flag) is skipped.

[0352] b. In one example, when the MVD of all CPMVs (including the horizontal and vertical components) is zero, the signaling of the first syntax element (amvr_flag) is skipped.

[0353] c. In one example, when the MVD (including the horizontal and vertical components) of a selected CPMV are all zero, the signaling of the first syntax element (amvr_flag) is skipped.

[0354] i. In one example, the MVD of the selected CPMV is the MVD of the first CPMV to be encoded / decoded.

[0355] d. In one example, when the use of multiple MVD precision for an affine-coded block is false, the signaling of the first syntax element (amvr_flag) is skipped.

[0356] e. In one example, the first syntax element can be notified by signaling under the following conditions:

[0357] i. The use of multiple MVD precisions is true for the affine-coded block, and the current block is encoded in affine mode;

[0358] ii. Alternatively, the use of multiple MVD precisions is enabled for an affine-coded block, the current block is encoded in affine mode, and at least one component of the MVD of the CPMV is not equal to 0.

[0359] iii. Alternatively, the use of multiple MVD precisions is true for an affine-coded block, the current block is encoded in affine mode, and at least one component of the MVD of the selected CPMV is not equal to 0.

[0360] 1. In one example, the MVD of the selected CPMV is the MVD of the first CPMV to be encoded / decoded.

[0361] 2. When AMVR is not applied to an affine code block or the first syntax element does not exist, the default MV and / or MVD precision is used.

[0362] a. In one example, the default precision is 1 / 4 pixel.

[0363] b. Alternatively, the default precision can be set to the precision used in motion compensation of affine coded blocks.

[0364] 3. For example, if amvr_flag equals 0, the MVD precision of affine mode is 1 / 4 pixel; otherwise, the MVD precision of affine mode can be other values.

[0365] a. Alternatively, other MVD precisions can also be notified via signaling through a second syntax element.

[0366] iii. In one example, a signaling instruction can be used to inform a second syntax element (such as amvr_coarse_precision_flag) to indicate the MVD precision of the affine mode.

[0367] 1. In one example, whether to signal the second syntax element can depend on the first syntax element. For example, the second syntax element may only be signaled if the first syntax element is 1.

[0368] 2. In one example, if the second syntax element is 0, the MVD precision of the affine mode is 1 pixel; otherwise, the MVD precision of the affine mode is 1 / 16 pixel.

[0369] 3. In one example, if the second syntax element is 0, the MVD precision of the affine mode is 1 / 16 pixel; otherwise, the MVD precision of the affine mode is full pixels.

[0370] iv. In one example, the syntax element used to indicate the MVD precision used in affine inter-frame mode shares the same context model as the syntax element with the same name used to indicate the MVD precision used in normal inter-frame mode.

[0371] 1. Alternatively, the syntax element used to indicate the MVD precision used in affine inter-frame mode may use a different context model than the syntax element with the same name used to indicate the MVD precision used in normal inter-frame mode.

[0372] 3. Whether to apply AMVR to an affine block or how to apply it to an affine block can depend on the reference picture of the current block.

[0373] a. In one example, if the reference image is the current image, then AMVR is not applied, i.e.

[0374] Apply intra-block copying in the current block.

[0375] Fast AVMR algorithm for encoder in affine mode

[0376] For IMV = i, the RD cost (actual RD cost, or SATD / SSE / SAD cost plus a rough bit cost) for affine mode and AMVP mode is denoted as affineCosti and amvpCosti, where i = 0, 1, or 2. Here, IMV = 0 represents 1 / 4 pixel MV, and for AMVP mode, IMV = 1 represents an integer MV; for affine mode, IMV = 1 represents 1 / 16 pixel, and for AMVP mode, IMV = 2 represents 4 pixel MV; for affine mode, IMV = 2 represents an integer MV. The RD cost for Merge mode is denoted as mergeCost.

[0377] 4. It is recommended to disable AMVR for the affine mode of the current CU if the optimal mode of the parent CU is not AF_INTER or AF_MERGE mode.

[0378] a. Alternatively, if the optimal mode of the parent CU of the current CU is not AF_INTER mode, then AMVR is disabled for the affine mode of the current CU.

[0379] 5. It is recommended to disable AMVR for affine mode if affineCost0 > th1 * amvpCost0, where th1 is a positive threshold.

[0380] a. Alternatively, AMVR is disabled for affine mode if min(affineCost0, amvpCost0) > th2 * mergeCost, where th2 is a positive threshold.

[0381] b. Alternatively, if affineCost0 > th3 * affineCost1, then integer MV is disabled for affine mode, where th3 is a positive threshold.

[0382] 6. It is recommended to disable AMVR for AMVP mode if amvpCost0 > th4 * affineCost0, where th4 is a positive threshold.

[0383] a. Alternatively, if min(affineCost0, amvpCost0) > th5 * mergeCost, then AMVR is disabled for AMVP mode, where th5 is a positive threshold.

[0384] 7. It is suggested that the 4 / 6 parameter affine model obtained in one MV accuracy can be used as a candidate starting search point for other MV accuracies.

[0385] a. In one example, the 4 / 6 parameter affine model obtained at 1 / 16 MV can be used as a candidate starting search point for other MV accuracies.

[0386] b. In one example, the 4 / 6 parameter affine model obtained in 1 / 4MV can be used as a candidate starting search point for other MV accuracies.

[0387] 8. If the parent block of the current block does not select affine mode, then AMVR for affine mode is not checked at the encoder of the current block.

[0388] 9. The rate-distortion calculation of the MV precision of the affine coded block in the current stripe / slice / CTU line can be terminated in advance by utilizing the statistical information of the different MV precision usage of the affine coded block in the previously encoded frame / strip / slice / CTU line.

[0389] a. In one example, record the percentage of affine-coded blocks with a certain MV precision. If the percentage is too low, skip the check for the corresponding MV precision.

[0390] b. In one example, previously encoded frames with the same temporal layer are used to determine whether to skip a certain MV precision.

[0391] Context for encoding affine AMVR

[0392] 10. For each context used to encode affine AMVR codes, it is recommended to set a variable (represented by shiftIdx) to control the update rate of the two probabilities associated with that context.

[0393] a. In one example, a faster update speed is defined by (shiftIdx>>2)+2.

[0394] b. In one example, a slower update speed is defined by (shiftIdx&3)+3+shift0.

[0395] c. In one example, the consistent bitstream should follow the deduced rule that faster update rates should be in the range of [2, 5].

[0396] d. In one example, the consistent bitstream should follow the deduced faster update rate should be within [3,

[0397] Rules within the scope of 6].

[0398] 11. It is recommended that when encoding the AMVR mode of a block, context modeling should not allow affine AMVR mode information from adjacent blocks.

[0399] a. In one example, the AMVR mode index of neighboring blocks can be used, and the affine AMVR mode information of neighboring blocks can be excluded. An example is shown in Table 5 (including Tables 5-1 and 5-2), where (xNbL, yNbL) and (xNbA, yNbA) represent the left and upper neighboring blocks. In one example, the context index offset ctxInc = (condL && availableL) + (condA && availableA) + ctxSetIdx * 3.

[0400] Table 5-1 – ctxInc Specification Using Left-hand and Top-hand Syntax Elements

[0401]

[0402] Table 5-2 – ctxInc Specification Using Left-hand and Top-hand Syntax Elements

[0403]

[0404] b. Alternatively, affine AMVR mode information from neighboring blocks can be utilized, but this is done in conjunction with a function rather than directly. In one example, the function `func`, as described in Table 6-1, can return true when the `amvr_mode[xNbL][yNbL]` of the affine-coded neighboring block indicates a certain MV precision (such as 1 / 4 pixel MV precision). In another example, the function `func`, as described in Table 6-2, can return true when the `amvr_flag[xNbL][yNbL]` of the affine-coded neighboring block indicates a certain MV precision (such as 1 / 4 pixel MV precision).

[0405] Table 6-1 – ctxInc Specification Using Left-hand and Top-hand Syntax Elements

[0406]

[0407]

[0408] Table 6-2 – ctxInc Specification Using Top-Left Syntax Elements

[0409]

[0410] c. Alternatively, affine AMVR mode information from neighboring blocks can be used to encode the first syntax element (e.g., amvr_flag) of the AMVR mode (applied to normal inter-frame mode). Tables 6-3 and 6-4 provide some examples.

[0411] Table 6-3 – ctxInc Specification Using Left-hand and Top-hand Syntax Elements

[0412]

[0413] Table 6-4 – ctxInc Specification Using Left-hand and Top-hand Syntax Elements

[0414]

[0415] d. When AMVR mode information is represented by multiple syntax elements (e.g., the first and second syntax elements represented as amvr_flag and amvr_coarse_precision_flag), the above syntax amvr_mode can be replaced by any of the multiple syntax elements, and the above method can still be applied.

[0416] 12. It is recommended that when encoding affine AMVR modes, the AMVR mode information of neighboring blocks be used for context encoding.

[0417] a. In one example, the AMVR mode information of the neighboring block is used directly. An example is shown in Table 7. Alternatively, the context index offset ctxInc = (condL&&

[0418] availableL)+(condA&&availableA)+ctxSetIdx*3.

[0419] Table 7 – ctxInc Specification Using Left-hand and Top-hand Syntax Elements

[0420]

[0421] b. Alternatively, AMVR mode information from neighboring blocks cannot be used for context modeling.

[0422] An example is shown in Table 8.

[0423] Table 8 – ctxInc Specification Using Left-hand and Top-hand Syntax Elements

[0424]

[0425] c. Alternatively, AMVR mode information from neighboring blocks can be utilized, but this is done in conjunction with a function rather than directly. In one example, the function func, as described in Table 9, can return true when the amvr_mode[xNbL][yNbL] of a non-affine encoded neighboring block indicates a certain MV precision (such as 1 / 4 pixel MV precision).

[0426] Table 9 – ctxInc Specification Using Left-hand and Top-hand Syntax Elements

[0427]

[0428]

[0429] d. When the affine AMVR mode information is represented by multiple syntax elements (e.g., the first and second syntax elements denoted as amvr_flag, amvr_coarse_precision_flag), the above syntax amvr_mode can be replaced by any of the multiple syntax elements, and the above method can still be applied.

[0430] Fast algorithms for SMVD and affine SMVD

[0431] When checking the SMVD mode, it is assumed that the currently selected best mode is CurBestMode, and the AMVRMVD precision in AMVR is MvdPrec or the MVD precision in affine AMVR is MvdPrecAff.

[0432] 13. Depending on the currently selected best mode (i.e., CurBestMode) and the MVD accuracy in AMVR, the SMVD mode can be skipped.

[0433] a. In one example, if CurBestMode is Merge mode or / and UMVE mode, then SMVD mode need not be checked.

[0434] b. In one example, if CurBestMode is not encoded using SMVD mode, then checking SMVD mode is unnecessary.

[0435] c. In one example, if CurBestMode is affine mode, then SMVD mode need not be checked.

[0436] d. In one example, if CurBestMode is the sub-block Merge mode, then the SMVD mode need not be checked.

[0437] e. In one example, if CurBestMode is an affine SMVD mode, then checking the SMVD mode is unnecessary.

[0438] f. In one example, if CurBestMode is an affine Merge mode, then the SMVD mode need not be checked.

[0439] g. In one example, the above fast method (i.e., bullet points 13.a-13.f) may only be applied to certain MVD precisions.

[0440] i. In one example, the above fast method can only be applied if the MVD precision is greater than or equal to the precision (e.g., integer pixel precision).

[0441] ii. In one example, the above fast method can only be applied if the MVD precision is greater than the precision (e.g., integer pixel precision).

[0442] iii. In one example, the above fast method can only be applied if the MVD precision is less than or equal to the precision (e.g., integer pixel precision).

[0443] iv. In one example, the above fast method can only be applied if the MVD precision is less than the precision (e.g., integer pixel precision).

[0444] 14. Depending on the currently selected best mode (i.e., CurBestMode) and the MVD accuracy in affine AMVR, the affine SMVD mode can be skipped.

[0445] a. In one example, if CurBestMode is Merge mode or / and UMVE mode, then affine SMVD mode need not be checked.

[0446] b. In one example, if CurBestMode is not encoded using an affine SMVD mode, then the affine SMVD mode need not be checked.

[0447] c. In one example, if CurBestMode is a sub-block Merge mode, then the affine SMVD mode need not be checked.

[0448] d. In one example, if CurBestMode is SMVD mode, then affine SMVD mode need not be checked.

[0449] e. In one example, if CurBestMode is an affine Merge mode, then the affine SMVD mode need not be checked.

[0450] f. In one example, the above-described fast method (i.e., bullet points 14.a-14.e) may be applied only to certain MVD accuracies.

[0451] i. In one example, the above fast method can only be applied if the affine MVD precision is greater than or equal to the precision (e.g., integer pixel precision).

[0452] ii. In one example, the above fast method can only be applied if the affine MVD precision is greater than the precision (e.g., integer pixel precision).

[0453] iii. In one example, the above fast method can only be applied if the affine MVD precision is less than or equal to the precision (e.g., integer pixel precision).

[0454] iv. In one example, the above-mentioned fast method can be applied only when the affine MVD accuracy is less than the accuracy (e.g., integer pixel accuracy).

[0455] 15. The above-mentioned proposed method can be applied under certain conditions, such as block size, slice / picture / tile type, or motion information.

[0456] a. In one example, when the block size contains less than M * H samples, such as 16 or 32 or 64 luma samples, the proposed method is not allowed.

[0457] b. Alternatively, when the minimum size of the width or / and height of the block is less than or not greater than X, the proposed method is not allowed. In one example, X is set to 8.

[0458] c. Alternatively, when the minimum size of the width or / and height of the block is not less than X, the proposed method is not allowed. In one example, X is set to 8.

[0459] d. Alternatively, when the width of the block > th1 or >= th1 and / or the height of the block > th2 or >= th2, the proposed method is not allowed. In one example, th1 and / or th2 are set to 8.

[0460] e. Alternatively, when the width of the block < th1 or <= th1 and / or the height of the block < th2 or <= th2, the proposed method is not allowed. In one example, th1 and / or th2 are set to 8.

[0461] f. Alternatively, whether to enable or disable the above method and / or which method to apply can depend on the block size, video processing data unit (VPDU), picture type, low latency check flag, encoding information of the current block or previous encoded block (such as, for example, reference picture, uni - directional or bi - directional prediction).

[0462] 16. When it comes to whether to apply Intra Block Copy (IBC, also known as Current Picture Reference (CPR)), the AMVR method in affine mode can be performed in different ways.

[0463] a. In one example, if the block is encoded by IBC, the AMVR in affine mode cannot be used.

[0464] b. In one example, if the block is encoded by IBC, the AMVR in affine mode can be used, but the candidate MV / MVD / MVP accuracy can be different from that for affine - encoded blocks that are not IBC - encoded.

[0465] 17. All terms "slice" in this article can be replaced by "slice group" or "tile".

[0466] 18. In the VPS / SPS / PPS / strip header / fragment header, syntax elements equal to 1 (e.g., no_amvr_constraint_flag), syntax elements indicating whether AMVR is enabled (e.g., sps_amvr_enabled_flag), and syntax elements indicating whether affine AMVR is enabled (e.g., sps_affine_avmr_enabled_flag) should both be equal to 0 for bitstream consistency. Syntax elements equal to 0 (e.g., no_amvr_constraint_flag) impose no constraints.

[0467] 19. In VPS / SPS / PPS / strip header / group header or other video data units, signaling can be used to notify syntax elements (e.g., no_affine_amvr_constraint_flag).

[0468] a. In one example, `no_affine_amvr_constraint_flag` equal to 1 indicates whether affine AMVR is enabled. Syntax elements indicating whether affine AMVR is enabled (e.g., `sps_affine_avmr_enabled_flag`) should be equal to 0 as a bitstream consistency requirement. Syntax elements equal to 0 (e.g., `no_affine_avvr_constraint_flag`) impose no constraints.

[0469] 6. Example

[0470] The bold and italic sections below show the revised specifications.

[0471] 6.1 Example 1: Instructions for using affine AMVR mode

[0472] Signaling can be used to notify in SPS / PPS / VPS / APS / sequence headers / image headers / fragment group headers, etc. This section introduces signaling in SPS.

[0473] 6.1.1 SPS Syntax Table

[0474]

[0475] The alternative SPS syntax table is given below:

[0476]

[0477]

[0478] Semantics

[0479] A value of 1 for `sps_affine_amvr_enabled_flag` specifies that adaptive motion vector differential resolution is used for motion vector coding in affine inter-frame mode. A value of 0 for `amvr_enabled_flag` specifies that adaptive motion vector differential resolution is not used for motion vector coding in affine inter-frame mode.

[0480] 6.2 The process of parsing affine AMVR mode information

[0481] The syntax for affine AMVR mode information can reuse the syntax for AMVR mode information (applied to normal inter-frame mode). Alternatively, different syntax elements can be used.

[0482] Affine AMVR mode information can be conditionally communicated via signaling. The following different embodiments illustrate some examples of these conditions.

[0483] 6.2.1 Example #1: CU Syntax Table

[0484]

[0485]

[0486]

[0487]

[0488] 6.2.2 Example 2: Alternative CU syntax table design

[0489]

[0490]

[0491]

[0492] 6.2.3 Example 3: Design of the Third CU Syntax Table

[0493]

[0494]

[0495]

[0496]

[0497] 6.2.4 Example 4: Design of syntax tables using different syntaxes of AMVR and affine modes

[0498]

[0499]

[0500]

[0501] In one example, conditionA is defined as follows:

[0502] (sps_affine_amvr_enabled_flag&&inter_affine_flag==1&&

[0503] (MvdCpL0[x0][y0][0][0]!=0||MvdCpL0[x0][y0][0][1]!=0||

[0504] MvdCpL1[x0][y0][0][0]! =0||MvdCpL1[x0][y0][0][1]! =0||

[0505] MvdCpL0[x0][y0][1][0]! =0||MvdCpL0[x0][y0][1][1]! =0||

[0506] MvdCpL1[x0][y0][1][0]! =0||MvdCpL1[x0][y0][1][1]! =0||

[0507] MvdCpL0[x0][y0][2][0]! =0||MvdCpL0[x0][y0][2][1]! =0||

[0508] MvdCpL1[x0][y0][2][0]! =0||MvdCpL1[x0][y0][2][1]! =0))

[0509] Alternatively, conditionsA can be defined as follows:

[0510] (sps_affine_amvr_enabled_flag&&inter_affine_flag==1&&

[0511] (MvdCpL0[x0][y0][0][0]!=0||MvdCpL0[x0][y0][0][1]!=0||

[0512] MvdCpL1[x0][y0][0][0]! =0||MvdCpL1[x0][y0][0][1]! =0||)

[0513] Alternatively, conditionsA can be defined as follows:

[0514] (sps_affine_amvr_enabled_flag&&inter_affine_flag==1&&

[0515] (MvdCpLX[x0][y0][0][0]!=0||MvdCpLX[x0][y0][0][1]!=0)

[0516] Where X is 0 or 1.

[0517] Alternatively, conditionsA can be defined as follows:

[0518] (sps_affine_amvr_enabled_flag&&inter_affine_flag==1)

[0519] In one example, conditionB is defined as follows:

[0520] ! sps_cpr_enabled_flag||! (inter_pred_idc[x0][y0]==PRED_L0&&ref_idx_l0[x0][y0]==num_ref_idx_l0_active_minus1)

[0521] Alternatively, conditionB is defined as follows:

[0522] ! sps_cpr_enabled_flag||! (pred_mode[x0][y0]==CPR)

[0523] Alternatively, conditionB is defined as follows:

[0524] ! sps_ibc_enabled_flag||! (pred_mode[x0][y0]==IBC)

[0525] When AMVR or affine AMVR is encoded using different syntax elements, context modeling and / or context applied to the affine AMVR in the embodiment in 6.5 can be applied accordingly.

[0526] 6.2.5 Semantics

[0527] `amvr_flag[x0][y0]` specifies the resolution of the motion vector difference. Array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the coded block under consideration relative to the top-left luminance sample of the image. `amvr_flag[x0][y0]` equal to 0 specifies that the resolution of the motion vector difference is 1 / 4 of the luminance sample. `amvr_flag[x0][y0]` equal to 1 specifies that the resolution of the motion vector difference is also specified by `amvr_coarse_precisoin_flag[x0][y0]`.

[0528] When amvr_flag[x0][y0] does not exist, the following inference can be made:

[0529] – If sps_cpr_enabled_flag equals 1, then it is inferred that amvr_flag[x0][y0] equals 1.

[0530] Otherwise (sps_cpr_enabled_flag equals 0), infer that amvr_flag[x0][y0] equals 0.

[0531] The `amvr_coarse_precisoin_flag[x0][y0]` value of 1 indicates that when `inter_affine_flag` is 0, the resolution of the motion vector difference is four luminance samples, while when `inter_affine_flag` is 1, it is one luminance sample. The array indices `x0` and `y0` specify the position (x0, y0) of the top-left luminance sample of the coded block under consideration relative to the top-left luminance sample of the image.

[0532] If amvr_coarse_precisoin_flag[x0][y0] does not exist, it is inferred to be equal to 0.

[0533] If inter_affine_flag[x0][y0] equals 0, then set the variable MvShift to equal (amvr_flag[x0][y0]+amvr_coarse_precisoin_flag[x0][y0])<<1, and modify the variables MvdL0[x0][y0][0], MvdL0[x0][y0][1], MvdL1[x0][y0][0], and MvdL1[x0][y0][1] as follows:

[0534] MvdL0[x0][y0][0]=MvdL0[x0][y0][0]<<(MvShift+2) (7-70)

[0535] MvdL0[x0][y0][1]=MvdL0[x0][y0][1]<<(MvShift+2) (7-71)

[0536] MvdL1[x0][y0][0]=MvdL1[x0][y0][0]<<(MvShift+2) (7-72)

[0537] MvdL1[x0][y0][1]=MvdL1[x0][y0][1]<<(MvShift+2) (7-73)

[0538] If inter_affine_flag[x0][y0] equals 1, then set the variable MvShift to equal (amvr_coarse_precisoin_flag?(amvr_coarse_precisoin_flag<<1):(-(amvr_flag<<1))), and modify the variables MvdCpL0[x0][y][0][0], MvdCpL0[x0][y0][0][1], MvdCpL0[x0][y0][1][0], MvdCpL0[x0][y0][1][1], MvdCpL0[x0][y0][2][0], and MvdCpL0[x0][y0][2][1] as follows:

[0539] MvdCpL0[x0][y0][0][0]=MvdCpL0[x0][y0][0][0]<<(MvShift+2)

[0540] (7-73)

[0541] MvdCpL1[x0][y0][0][1]=MvdCpL1[x0][y0][0][1]<<(MvShift+2)

[0542] (7-67)

[0543] MvdCpL0[x0][y0][1][0]=MvdCpL0[x0][y0][1][0]<<(MvShift+2)

[0544] (7-66)

[0545] MvdCpL1[x0][y0][1][1]=MvdCpL1[x0][y0][1][1]<<(MvShift+2)

[0546] (7-67)

[0547] MvdCpL0[x0][y0][2][0]=MvdCpL0[x0][y0][2][0]<<(MvShift+2)

[0548] (7-66)

[0549] MvdCpL1[x0][y0][2][1]=MvdCpL1[x0][y0][2][1]<<(MvShift+2)

[0550] (7-67)

[0551] Alternatively, if inter_affine_flag[x0][y0] equals 1, then set the variable MvShift to equal (affine_amvr_coarse_precisoin_flag?(affine_amvr_coarse_precisoin_flag<<1):(-(affine_amvr_flag<<1))).

[0552] 6.3 Rounding process of motion vector

[0553] Modify the rounding process so that when a given rightShift value is equal to 0 (occurring at 1 / 16 pixel precision), the rounding offset is set to 0 instead of (1<<(rightShift-1)).

[0554] For example, the sub-clause for the rounding process of MV is modified as follows:

[0555] The input to this process is:

[0556] – Motion vector mvX,

[0557] – The right shift parameter used for rounding, rightShift

[0558] – The left-shift parameter used to improve resolution.

[0559] The output of this process is the rounded motion vector mvX.

[0560] For rounding mvX, the following applies:

[0561] offset=(rightShift==0)? 0:(1<<(rightShift-1)) (8-371)

[0562] mvX[0]=(mvX[0]>=0?(mvX[0]+offset)>>rightShift:-((-mvX[0]+offset)>>rightShift))< <leftShift (8-372)

[0563] mvX[1]=(mvX[1]>=0?(mvX[1]+offset)>>rightShift:-((-mvX[1]+offset)>>rightShift))< <leftShift (8-373)

[0564] 6.4 Decoding Process

[0565] The rounding process called during the derivation of the affine motion vector is performed with an input of (MvShift+2), instead of fixing the input to 2.

[0566] Derivation of the motion vector predictor for affine control points of brightness

[0567] The input to this process is:

[0568] – The brightness position (xCb, yCb) of the top-left sample of the current luminance coding block relative to the top-left luminance sample of the current image.

[0569] – Two variables, cbWidth and cbHeight, specify the width and height of the current luma-coded block.

[0570] – The reference index of the current coding unit refIdxLX, where X is 0 or 1.

[0571] – The number of control point motion vectors, numCpMv.

[0572] The output of this process is a luminance affine control point motion vector predictor mvpCpLX[cpIdx], where X is 0 or 1, and cpIdx = 0..numCpMv-1.

[0573] To derive the candidate list cpMvpListLX for the control point motion vector predictor, where X is 0 or 1, the following ordered steps apply:

[0574] Set the number of control point motion vector predictor candidates in the list numCpMvpCandLX to 0.

[0575] Set both availableFlagA and availableFlagB to FALSE.

[0576] …

[0577] The motion vector rounding procedure specified in Section 8.4.2.14 is invoked with mvX set to equal cpMvpLX[cpIdx], rightShift set to equal (MvShift+2), and leftShift set to equal (MvShift+2) as inputs, and the rounded cpMvpLX[cpIdx] (cpIdx = 0..numCpMv-1) as output.

[0578] …

[0579] Set the variable availableFlagA to TRUE.

[0580] The process of deriving the luminance affine control point motion vectors from neighboring blocks is invoked with the following inputs: the position of the luminance coded block (xCb, yCb), the width and height of the luminance coded block (cbWidth, cbHeight), the position of the neighboring luminance coded block (xNb, yNb), the width and height of the neighboring luminance coded block (nbW, nbH), and the number of control point motion vectors numCpMv. The process of deriving the luminance affine control point motion vectors from neighboring blocks is invoked with the following output: the candidate cpMvpLY[cpIdx] (cpIdx = 0..numCpMv-1) of the control point motion vector predictor.

[0581] The motion vector rounding procedure specified in Section 8.4.2.14 is invoked with mvX set to equal cpMvpLY[cpIdx], rightShift set to equal to (MvShift+2), and leftShift set to equal to (MvShift+2) as inputs, and the rounded cpMvpLY[cpIdx] (cpIdx = 0..numCpMv-1) as output.

[0582] …

[0583] The luminance coding block position (xCb, yCb), the luminance coding block width and height (cbWidth, cbHeight), the neighboring block luminance coding block position (xNb, yNb), the neighboring luminance coding block width and height (nbW, nbH), and the number of control point motion vectors numCpMv are taken as input, and the control point motion vector predictor candidate cpMvpLX[cpIdx] (cpIdx = 0..numCpMv-1) is taken as output to invoke the derivation process for the luminance affine control point motion vectors from the neighboring blocks specified in Section 8.4.4.5.

[0584] The motion vector rounding procedure specified in Section 8.4.2.14 is invoked with mvX set to equal cpMvpLX[cpIdx], rightShift set to equal (MvShift+2), and leftShift set to equal (MvShift+2) as inputs, and the rounded cpMvpLX[cpIdx] (cpIdx = 0..numCpMv-1) as output.

[0585] Perform the following assignment:

[0586] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[0] (8-618)

[0587] cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[1] (8-619)

[0588] cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[2] (8-620)

[0589] numCpMvpCandLX=numCpMvpCandLX+1 (8-621)

[0590] Otherwise, if PredFlagLY[xNbBk][yNbBk](Y=!X) equals 1 and DiffPicOrderCnt(RefPicListY[RefIdxLY[xNbBk][yNbBk]], RefPicListX[refIdxLX]) equals 0, then the following applies:

[0591] Set the variable availableFlagB to TRUE.

[0592] The process of deriving the luminance affine control point motion vectors from neighboring blocks is invoked using the following as inputs: the position of the luminance coded block (xCb, yCb), the width and height of the luminance coded block (cbWidth, cbHeight), the position of the neighboring block luminance coded block (xNb, yNb), the width and height of the neighboring luminance coded blocks (nbW, nbH), and the number of control point motion vectors numCpMv. The process of deriving the luminance affine control point motion vectors from neighboring blocks is invoked using the control point motion vector predictor candidate cpMvpLY[cpIdx] (cpIdx = 0..numCpMv-1) as output.

[0593] The motion vector rounding procedure specified in Section 8.4.2.14 is invoked with mvX set to equal cpMvpLY[cpIdx], rightShift set to equal to (MvShift+2), and leftShift set to equal to (MvShift+2) as inputs, and the rounded cpMvpLY[cpIdx] (cpIdx = 0..numCpMv-1) as output.

[0594] Perform the following assignment:

[0595] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLY[0] (8-622)

[0596] cpMvpListLX[numCpMvpCandLX][1]=cpMvpLY[1] (8-623)

[0597] cpMvpListLX[numCpMvpCandLX][2]=cpMvpLY[2] (8-624)

[0598] numCpMvpCandLX=numCpMvpCandLX+1 (8-625)

[0599] When numCpMvpCandLX is less than 2, the following applies:

[0600] The derivation process for constructing affine control point motion vector prediction candidates, as specified in Section 8.4.4.8, is invoked with the inputs of the luma coding block position (xCb, yCb), luma coding block width cbWidth, luma coding block height cbHeight, and the reference index of the current coding unit refIdxLX, and the outputs of the availability flag availableConsFlagLX, availability flag availableFlagLX[cpIdx], and cpMvpLX[cpIdx] (cpIdx = 0..numCpMv-1).

[0601] When availableConsFlagLX equals 1 and numCpMvpCandLX equals 0, the following assignment is performed:

[0602] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[0] (8-626)

[0603] cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[1] (8-627)

[0604] cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[2] (8-628)

[0605] numCpMvpCandLX=numCpMvpCandLX+1 (8-629)

[0606] The following applies to cpIdx = 0..numCpMv-1:

[0607] When numCpMvpCandLX is less than 2 and availableFlagLX[cpIdx] is equal to 1, the following assignment is performed:

[0608] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[cpIdx] (8-630)

[0609] cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[cpIdx] (8-631)

[0610] cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[cpIdx] (8-632)

[0611] numCpMvpCandLX=numCpMvpCandLX+1 (8-633)

[0612] When numCpMvpCandLX is less than 2, the following applies:

[0613] The derivation of the temporal luminance motion vector prediction specified in Section 8.4.2.11 is performed by taking the luminance block position (xCb, yCb), luminance block width cbWidth, luminance block height cbHeight, and refIdxLX as inputs, and taking the availability flag availableFlagLXCol and the temporal motion vector predictor mvLXCol as outputs.

[0614] When availableFlagLXCol equals 1, the following applies:

[0615] The motion vector rounding procedure specified in Section 8.4.2.14 is invoked with mvX set to equal to mvLXCol, rightShift set to equal to (MvShift+2), and leftShift set to equal to (MvShift+2) as inputs and the rounded mvLXCol as output.

[0616] Perform the following assignment:

[0617] cpMvpListLX[numCpMvpCandLX][0]=mvLXCol(8-634)

[0618] cpMvpListLX[numCpMvpCandLX][1]=mvLXCol(8-635)

[0619] cpMvpListLX[numCpMvpCandLX][2]=mvLXCol(8-636)

[0620] numCpMvpCandLX=numCpMvpCandLX+1 (8-637)

[0621] When numCpMvpCandLX is less than 2, repeat the following until numCpMvpCandLX equals 2, where mvZero[0] and mvZero[1] are both equal to 0:

[0622] cpMvpListLX[numCpMvpCandLX][0]=mvZero (8-638)

[0623] cpMvpListLX[numCpMvpCandLX][1]=mvZero (8-639)

[0624] cpMvpListLX[numCpMvpCandLX][2]=mvZero (8-640)

[0625] numCpMvpCandLX=numCpMvpCandLX+1 (8-641)

[0626] The affine control point motion vector predictor cpMvpLX (where X is 0 or 1) is derived as follows:

[0627] cpMvpLX=cpMvpListLX[mvp_lX_flag[xCb][yCb]] (8-642)

[0628] Derivation of the candidate affine control point motion vector prediction method

[0629] The input to this process is:

[0630] – Luminance position (xCb, yCb), specifies the top-left sample of the current luminance coding block relative to the top-left luminance sample of the current image.

[0631] – Two variables, cbWidth and cbHeight, specify the width and height of the current luma-coded block.

[0632] – The reference index for the current prediction unit to segment refIdxLX, where X is 0 or 1.

[0633] The output of this process is:

[0634] – The availability flag availableConsFlagLX for the constructed affine control point motion vector prediction candidates, where X is 0 or 1.

[0635] – Availability flag availableFlagLX[cpIdx], where cpIdx = 0..2 and X is 0 or 1.

[0636] – Construct affine control point motion vector prediction candidate cpMvLX[cpIdx], cpIdx=0..numCpMv-1 and X is 0 or 1.

[0637] The motion vector cpMvLX[0] and availability flag availableFlagLX[0] of the first (top left) control point are derived in the following ordered steps:

[0638] Set the sample point positions (xNbB2, yNbB2), (xNbB3, yNbB3) and (xNbA2, yNbA2) to be equal to (xCb-1, yCb-1), (xCb, yCb-1) and (xCb-1, yCb) respectively.

[0639] Set the availability flag availableFlagLX[0] to 0, and set both components of cpMvLX[0] to 0.

[0640] For (xNbTL, yNbTL) (TL is replaced by B2, B3, and A2), the following applies:

[0641] The process of invoking the availability derivation of the luminance block specified in the section is to take the luminance block position (xCb, yCb), luminance block width cbWidth, luminance block height cbHeight, and luminance position (xNbY, yNbY) set to be equal to (xNbTL, yNbTL) as input and assign the output to the luminance block availability flag availableTL.

[0642] The following applies when availableTL equals TRUE and availableFlagLX[0] equals 0:

[0643] If PredFlagLX[xNbTL][yNbTL] equals 1, and DiffPicOrderCnt(RefPicListX[RefIdxLX[xNbTL][yNbTL]], RefPicListX[refIdxLX]) equals 0, and the reference image for RefIdxLTL[xNbTL][yNbTL] is not the current image, then set availableFlagLX[0] to 1 and perform the following assignment:

[0644] cpMvLX[0]=MvLX[xNbTL][yNbTL] (8-643)

[0645] Otherwise, if PredFlagLY[xNbTL][yNbTL] (Y = !X) equals 1 and DiffPicOrderCnt(RefPicListY[RefIdxLY[xNbTL][yNbTL]], RefPicListX[refIdxLX]) equals 0, and the reference image corresponding to RefIdxLY[xNbTL][yNbTL] is not the current image, then set availableFlagLX[0] to equal 1 and perform the following assignment:

[0646] cpMvLX[0]=MvLY[xNbTL][yNbTL] (8-644)

[0647] When availableFlagLX[0] equals 1, the motion vector rounding procedure specified in Section 8.4.2.14 is invoked with mvX set to equal cpMvLX[0], rightShift set to equal (MvShift+2), and leftShift set to equal (MvShift+2) as inputs and cpMvLX[0] rounded as output.

[0648] The motion vector cpMvLX[1] and availability flag availableFlagLX[1] of the second (top right) control point are derived in the following ordered steps:

[0649] Set the sample point positions (xNbB1, yNbB1) and (xNbB0, yNbB0) to (xCb+cbWidth-1, yCb-1) and (xCb+cbWidth, yCb-1) respectively.

[0650] Set the availability flag availableFlagLX[1] to 0 and set both components of cpMvLX[1] to 0.

[0651] For (xNbTR, yNbTR) (TR is replaced by B1 and B0), the following applies:

[0652] The process of invoking the availability derivation of the coded block specified in Section 6.4.X is to take the luma coded block position (xCb, yCb), luma coded block width cbWidth, luma coded block height cbHeight, and luma position (xNbY, yNbY) set to be equal to (xNbTR, yNbTR) as input and assign the output to the coded block availability flag availableTR.

[0653] When availableTR equals TRUE and availableFlagLX[1] equals 0, the following applies:

[0654] If PredFlagLX[xNbTR][yNbTR] equals 1, and DiffPicOrderCnt(RefPicListX[RefIdxLX[xNbTR][yNbTR]], RefPicListX[refIdxLX]) equals 0, and the reference image corresponding to RefIdxLX[xNbTR][yNbTR] is not the current image, then set availableFlagLX[1] to equal 1 and perform the following assignment:

[0655] cpMvLX[1]=MvLX[xNbTR][yNbTR] (8-645)

[0656] Otherwise, when PredFlagLY[xNbTR][yNbTR](Y=!X) equals 1 and DiffPicOrderCnt(RefPicListY[RefIdxLY[xNbTR][yNbTR]], RefPicListX[refIdxLX]) equals 0, and the reference image corresponding to RefIdxLY[xNbTR][yNbTR] is not the current image, set availableFlagLX[1] to equal 1 and perform the following assignment:

[0657] cpMvLX[1]=MvLY[xNbTR][yNbTR] (8-646)

[0658] When availableFlagLX[1] equals 1, the motion vector rounding process specified in Section 8.4.2.14 is invoked with mvX set to equal cpMvLX[1], rightShift set to equal (MvShift+2), and leftShift set to equal (MvShift+2) as inputs and the rounded cpMvLX[1] as output.

[0659] The motion vector cpMvLX[2] and availability flag availableFlagLX[2] of the third (lower left) control point are derived in the following ordered steps:

[0660] Set the sample point positions (xNbA1, yNbA1) and (xNbA0, yNbA0) to be equal to (xCb-1, yCb+cbHeight-1) and (xCb-1, yCb+cbHeight), respectively.

[0661] Set the availability flag availableFlagLX[2] to 0, and set both components of cpMvLX[2] to 0.

[0662] For (xNbBL, yNbBL) (BL is replaced by A1 and A0), the following applies:

[0663] The availability derivation process for the coded block specified in Section 6.4.X is invoked by taking the luminance coded block position (xCb, yCb), luminance coded block width cbWidth, luminance coded block height cbHeight, and luminance position (xNbY, yNbY) set to (xNbBL, yNbBL) as inputs and assigning the output to the coded block availability flag availableBL.

[0664] When availableBL equals TRUE and availableFlagLX[2] equals 0, the following conditions apply:

[0665] If PredFlagLX[xNbBL][yNbBL] equals 1, and DiffPicOrderCnt(RefPicListX[RefIdxLX[xNbBL][yNbBL]], RefPicListX[refIdxLX]) equals 0, and the reference image corresponding to RefIdxLY[xNbBL] is not the current image, then set availableFlagLX[2] to equal 1 and perform the following assignment:

[0666] cpMvLX[2]=MvLX[xNbBL][yNbBL] (8-647)

[0667] Otherwise, when PredFlagLY[xNbBL][yNbBL](Y=!X) equals 1 and DiffPicOrderCnt(RefPicListY[RefIdxLY[xNbBL][yNbBL]]RefPicListX[refIdxLX]) equals 0, and the reference image corresponding to RefIdxLY[xNbBL][yNbBL] is not the current image, set availableFlagLX[2] to equal 1 and perform the following assignment:

[0668] cpMvLX[2]=MvLY[xNbBL][yNbBL] (8-648)

[0669] When availableFlagLX[2] equals 1, the mvX set to equal cpMvLX[2], the rightShift set to equal (MvShift+2), and the leftShift set to equal (MvShift+2) are used as inputs, and the rounded cpMvLX[2] is used as output to call the motion vector rounding process specified in 8.4.2.14.

[0670] 6.5 Context Modeling

[0671] Assign ctxInc to the syntax element using context-encoded binary bits:

[0672]

[0673] The ctxInc specification uses left-hand and top-hand syntax elements.

[0674] In one example, the context increment offset ctxInc = (condL&&availableL) + (condA&&availableA) + ctxSetIdx * 3.

[0675] Alternatively, ctxInc = ((condL&&availableL)||(condA&&availableA)) + ctxSetIdx * 3.

[0676] ctxInc=(condL&&availableL)+M*(condA&&availableA)+ctxSetIdx*3 (for example, M=2)

[0677] ctxInc=M*(condL&&availableL)+(condA&&availableA)+ctxSetIdx*3. (For example, M=2).

[0678]

[0679] The initValue value of ctxIdx used for mvr_flag:

[0680] Different contexts are used when the current block is affine or non-affine.

[0681]

[0682] The initValue value of ctxIdx used for amvr_coarse_precisoin_flag:

[0683] Different contexts are used when the current block is affine or non-affine.

[0684]

[0685] The above example can be incorporated into the context of the method described below (e.g., method 2500), which can be implemented at the video decoder or video encoder.

[0686] Figure 25 A flowchart of an exemplary method for video decoding is shown. Method 2500 includes, in step 2510, determining, for the affine coding of the current video block, whether to use (a) multiple motion vector difference (MVD) precision or (b) adaptive motion vector difference resolution (AMVR), and in step 2520, performing a conversion between the current video block and its bitstream representation based on the determination.

[0687] Method 2500 includes, at step 2520, performing a conversion between a bitstream representation and a current block based on a final motion vector, the current block being encoded using an affine inter-frame mode or a normal inter-frame mode that supports an adaptive motion vector resolution (AMVR) process. In some embodiments, the conversion generates the current block from the bitstream representation (e.g., it can be implemented in a video decoder). In other embodiments, the conversion generates a bitstream representation from the current block (e.g., it can be implemented in a video encoder).

[0688] In some embodiments, and in the context of Example 1, the precision of the final motion vector is the same as the precision of the motion vector stored in the current block. In one example, the precision of the final motion vector is 1 / 16 of a pixel. In another example, the precision of the final motion vector is 1 / 8 of a pixel.

[0689] In some embodiments, and in the context of Example 2, method 2500 further includes the step of bypassing the scaling operation of the MVP when the precision of the MVD is determined to be a fraction (e.g., greater than 1 pixel). In other embodiments, method 2500 further includes the step of scaling the MVP when the precision of the MVD is determined to be less than or equal to 1 pixel.

[0690] In some embodiments, and in the context of Example 5, the current block is encoded using an affine inter-frame mode, and the precision of the motion vector or the precision of the motion vector difference is indicated by a signaling notification syntax element. In one example, a value of 0, 1, or 2 for the signaling notification syntax element corresponds to a motion vector precision of 1 / 4 pixel, 1 / 16 pixel, and 1 pixel, respectively. In another example, a value of 0, 1, or 2 for the signaling notification syntax element corresponds to a motion vector precision of 1 / 4 pixel, 1 pixel, and 1 / 16 pixel, respectively. In yet another example, a value of 0, 1, or 2 for the signaling notification syntax element corresponds to a motion vector precision of 1 / 16 pixel, 1 / 4 pixel, and 1 pixel, respectively.

[0691] In some embodiments, and in the context of Example 6, the AMVR process is enabled based on signaling in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Video Parameter Set (VPS), Strip header, Slice header, Coding Tree Unit (CTU) group, Coding Unit (CU), Prediction Unit (PU), or Transform Unit (TU).

[0692] In some embodiments, and in the context of Example 7, a set of allowed values ​​for the precision of the motion vector difference or the precision of the motion vector is signaled in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Video Parameter Set (VPS), Slice Header, Slice Header, Code Tree Unit (CTU) Group, Code Unit (CU), Prediction Unit (PU), or Transform Unit (TU). In one example, the set of allowed values ​​is signaled for each Code Tree Unit (CTU) of the current block. In another example, the set of allowed values ​​is based on the codec mode or size of the current block. In yet another example, the set of allowed values ​​is based on the slice type, temporal layer index, or low-latency check flag. In yet another example, the set of allowed values ​​is based on the precision of the motion vector stored in the decoded picture buffer.

[0693] In some implementations of the above embodiments, the MVP can be based on the spatially or temporally adjacent blocks of the current block. In other implementations of the above embodiments, the MVP can be the default MVP.

[0694] 6. Example implementation of the publicly available technology

[0695] Figure 26This is a block diagram of a video processing apparatus 2600. Apparatus 2600 can be used to implement one or more methods described herein. Apparatus 2600 can be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Apparatus 2600 may include one or more processors 2602, one or more memories 3704, and video processing hardware 2606. Processor 2602 can be configured to implement one or more methods described herein (including, but not limited to, method 2800). Although some embodiments may operate without memory, memory(s) 2604 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 2606 can be used to implement some of the techniques described in this document in hardware circuitry.

[0696] In some embodiments, the video encoding method may use methods such as those described above. Figure 26 The device described is implemented on a hardware platform.

[0697] Various technologies and embodiments can be described using the following terms-based format.

[0698] 1. A method for video encoding, comprising:

[0699] For the affine coding of the current video block, determine whether to use (a) multiple motion vector difference (MVD) precision or (b) adaptive motion vector difference resolution (AMVR), and

[0700] Based on the given information, perform a conversion between the current video block and its bitstream representation.

[0701] 2. The method according to Clause 1, wherein the determination is made based on one or more syntax elements in the bitstream representation.

[0702] 3. The method according to Clause 2, wherein one or more syntax elements are selectively included in the bitstream representation based on enabling affine coding for the current video block.

[0703] 4. The method according to any one of clauses 1-3, wherein the MVD precision for affine inter-frame mode is different from the MVD precision for normal inter-frame mode coding.

[0704] 5. The method according to any one of clauses 2-4, wherein context-based encoding is used to encode one or more syntactic elements.

[0705] 6. The method according to Clause 5, wherein different context models are used for affine inter-frame coding and normal inter-frame mode coding of the current video block.

[0706] 7. The method according to Clause 1, wherein the determination is made based on a first syntax element, which is included in the bitstream representation to indicate the applicability of AMVR to the current affine-coded video block.

[0707] 8. The method according to Clause 1, wherein it is determined that AMVR is disabled during the conversion of the current video block, or wherein it is determined that a first syntax element is not present in the bitstream representation, and wherein the conversion uses the default motion vector or the default motion vector difference precision.

[0708] 9. The method according to Clause 8, wherein the default precision is 1 / 4 pixel.

[0709] 10. The method according to Clause 7, wherein the determination is made based on a second syntax element indicating the precision of the motion vector difference.

[0710] 11. The method according to Clause 7, wherein the first syntax element is excluded from the bitstream representation when the control point motion vector of the current video block is zero.

[0711] 12. The method according to any one of clauses 1 to 11, wherein, when intra-block copying is used during conversion, AMVR includes a first AMVR scheme, and when intra-block copying is not used during conversion, AMVR includes a different second AMVR scheme.

[0712] 13. The method according to any one of clauses 2 to 12, wherein syntactic elements are included at the strip level, picture level, or sequence level.

[0713] 14. The method according to any one of clauses 1 to 13, wherein the transformation includes generating pixel values ​​of the current video block from a bitstream representation, or generating a bitstream representation from the current video block.

[0714] 15. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of clauses 1 to 14.

[0715] 16. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing the method of any one of clauses 1 to 14.

[0716] Figure 27 An example of a symmetric pattern is shown.

[0717] Figure 28This is a block diagram illustrating an example video processing system 2800 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 2800. System 2800 may include an input 2802 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or in a compressed or encoded format. Input 2802 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, Passive Optical Networking (PON), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[0718] System 2800 may include an encoding component 2804 capable of implementing the various encoding or encoding methods described herein. Encoding component 2804 can reduce the average bit rate of the video from input 2802 to output to produce an encoded representation of the video. Therefore, the encoding technique is sometimes referred to as video compression or video transcoding. As indicated by component 2806, the output of encoding component 2804 can be stored or transmitted via a connected communication. The stored or transmitted bitstream (or encoded) representation of the video received at input 2802 can be used by component 2808 to generate pixel values ​​or displayable video that is sent to display interface 2810. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “encoding” operations or tools, it should be understood that encoding tools or operations are used at the encoder, and corresponding decoding tools or operations will be performed by the encoder to reverse the encoded results.

[0719] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include SATA (Serial Advanced Technology Accessory), PCI, IDE, etc. The technologies described herein can be embodied in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0720] Figure 29A flowchart of an example method for visual media processing is shown. The steps of this flowchart illustrate an implementation of the example embodiment discussed in Example 2k in Part 4 of this document. In step 2910, the process, for the conversion between a current video block and its bitstream representation, determines to represent motion information using one or more precisions, the motion information including motion vectors and / or motion vector predictors and / or motion vector differences used during the conversion in affine inter-frame mode or normal inter-frame mode using adaptive motion vector resolution (AMVR). In step 2920, the process performs the conversion based on the determination of the one or more precisions, wherein multiple syntax elements are used in the bitstream representation to signal the one or more precisions.

[0721] Figure 30 A flowchart of an example method for visual media processing is shown. The steps of this flowchart illustrate an implementation of the example embodiment discussed in Example 2kii1 in Part 4 of this document. In step 3010, the process makes a determination to represent motion information using one or more precisions for the conversion between the current video block and its bitstream representation. The motion information includes motion vectors and / or motion vector predictors and / or motion vector differences used during the conversion in Intra-Block Copy (IBC) mode using Adaptive Motion Vector Resolution (AMVR). In step 3020, the process performs the conversion based on the determination of the one or more precisions, wherein multiple syntax elements are used in the bitstream representation to signal the one or more precisions, and wherein the conversion is performed based on block vectors derived from the picture of the current video block in IBC mode.

[0722] Some examples are now presented in a terms-based format.

[0723] 1. A method for visual media processing, comprising:

[0724] For the conversion between the current video block and its bitstream representation, a determination is made to represent motion information using one or more precisions, including motion vectors and / or motion vector predictors and / or motion vector differences used during the conversion in affine inter-frame mode or normal inter-frame mode using adaptive motion vector resolution (AMVR); and

[0725] The conversion is performed based on a determination of one or more precisions, wherein multiple syntax elements are used in the bitstream representation to signal one or more precisions.

[0726] 2. The method according to Clause 1 also includes:

[0727] In bitstream representation, multiple syntax elements corresponding to multiple precisions are selectively notified by signaling.

[0728] 3. The method according to one or more of Clauses 1-2, wherein the plurality of syntax elements includes at least a first syntax element to indicate whether default motion precision is applied.

[0729] 4. According to one or more of the methods in Clauses 1-3, wherein the first syntax element is an amvr flag or a motion information precision flag.

[0730] 5. The method according to any one or more of clauses 1-4, wherein the motion information precision used for the affine inter-frame mode is different from the motion information precision used for the normal inter-frame mode.

[0731] 6. The method according to any one or more of the provisions 1-5, wherein at least one of the plurality of syntax elements is encoded using context-based encoding, the context-based encoding including the use of at least one context model.

[0732] 7. The method according to Clause 6, wherein the context model used for affine inter-frame encoding / decoding mode is different from the context model used for normal inter-frame mode.

[0733] 8. The method according to any one or more of Clauses 1-2, wherein the determination is based on a first syntax element included in the bitstream representation to indicate the applicability of AMVR to the current video block.

[0734] 9. The method according to any one or more of Clauses 1-2, wherein: when it is determined that the first syntax element does not exist in the bitstream representation, it is determined that AMVR is disabled during the conversion of the current video block, and wherein the conversion uses default motion information or default motion information precision.

[0735] 10. The method according to any one or more of Clauses 1-2, wherein, when it is determined that a first syntax element is not present in the bitstream representation, it is determined that multiple precisions are prohibited from being used to represent motion information.

[0736] 11. The method according to Clause 8, wherein the first syntax element is excluded from the bitstream representation when the motion vector difference (MVD) of all control points in the current video block is zero.

[0737] 12. The method according to Clause 8, wherein the first syntax element is excluded from the bitstream representation if the motion vector difference (MVD) at a selected control point of the current video block is zero.

[0738] 13. The method according to Clause 8, wherein, in the case of disabling AMVR for affine inter-frame mode, the first syntax element is excluded from the bitstream representation.

[0739] 14. The method according to any one or more of Clauses 11 to 13, wherein the affine inter-frame mode is used for the current video block.

[0740] 15. The method according to Clause 8, wherein, in the case of disabling AMVR for normal inter-frame mode and using normal inter-frame mode for the current video block, the first syntax element is excluded from the bitstream representation.

[0741] 16. The method according to Clause 8, wherein the default motion information precision is 1 / 4 luminance sample or 1 luminance sample or 1 / 16 luminance sample.

[0742] 17. The method according to Clause 8, wherein the default motion information precision is set to the value used during the motion compensation step.

[0743] 18. The method according to any one or more of Clauses 1-2, wherein when it is determined that the first syntax element is not present in the bitstream representation, it is determined that AMVR is enabled during the conversion of the current video block.

[0744] 19. A method for visual media processing, comprising:

[0745] For the conversion between the current video block and its bitstream representation, a determination is made to represent motion information using one or more precisions, including motion vectors and / or motion vector predictors and / or motion vector differences used during the conversion in Intra-Block Copy (IBC) mode using Adaptive Motion Vector Resolution (AMVR); and

[0746] The conversion is performed based on the determination of one or more precisions, wherein multiple syntax elements are used in the bitstream representation to signal one or more precisions;

[0747] The conversion is performed based on the block vector derived from the image of the current video block in IBC mode.

[0748] 20. The method of Clause 19, wherein the determination is based on a first syntax element included in the bitstream representation to indicate the applicability of AMVR to the current video block.

[0749] 21. The method according to one or more of Clauses 8 or 20, wherein the determination is based on a second syntactic element indicating one or more precisions for representing motion information.

[0750] 22. According to the method of Clause 21, where the motion information precision is an integer value when the second syntax element is equal to zero, otherwise the motion information precision is 4 luminance samples.

[0751] 23. According to the method of Clause 21, where the motion information precision is 4 luminance samples when the second syntax element is equal to zero, otherwise the motion information precision is an integer value.

[0752] 24. The method according to Clause 21, wherein the signaling of the second syntax element is based on the value of the first syntax element or whether the first syntax element is missing in the bitstream representation.

[0753] 25. According to the method of Clause 21, wherein when the second syntax element is equal to zero, the motion information precision is an integer value, otherwise the motion information precision is 1 / 16 of the luminance sample.

[0754] 26. According to the method of Clause 21, wherein when the second syntax element is equal to zero, the motion information precision is 1 / 16 of the luminance sample point, otherwise the motion information precision is an integer value.

[0755] 27. The method according to Clause 6, wherein the same context model is used for both affine inter-frame mode and normal inter-frame mode.

[0756] 28. The method according to one or more of Clauses 7 or 27, wherein the same syntax elements are used for affine inter-frame encoding / decoding mode and normal inter-frame mode.

[0757] 29. The method according to any one or more of Clauses 1 to 28, wherein, when an intra-block copy mode is used during conversion, AMVR includes a first AMVR scheme, and when an intra-block copy mode is not used during conversion, AMVR includes a different second AMVR scheme.

[0758] 30. The method according to any one or more of clauses 1 to 29, wherein multiple syntax elements are included at the strip level, picture level or sequence level.

[0759] 31. The method according to any one or more of clauses 1 to 30, wherein the transformation includes generating pixel values ​​of the current video block from a bitstream representation, or generating a bitstream representation from the current video block.

[0760] 32. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method according to any one of clauses 1 to 31.

[0761] 33. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method pursuant to any one of clauses 1 to 31.

[0762] 34. A video encoder apparatus, including a processor configured to implement any one or more of the methods in clauses 1-31.

[0763] 35. A video decoder apparatus, including a processor configured to implement any one or more of the methods in clauses 1-31.

[0764] As can be understood from the foregoing, specific embodiments of the disclosed technology have been described herein for illustrative purposes, but various modifications can be made without departing from the scope of the invention. Therefore, the disclosed technology is not limited except for the appended claims.

[0765] The implementation and functional operation of the subject matter described in this patent document can be implemented in various systems, digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations thereof. The subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more transient and non-transitory computer program instruction modules encoded on a computer-readable medium for operation by a data processing apparatus or for controlling the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of material affecting machine-readable propagation signals, or a combination thereof. The terms "data processing unit" and "data processing apparatus" include all means, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may include code that creates an operating environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof.

[0766] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language (including compiled or interpreted languages) and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed to execute on one or more computers located at a single site or distributed across multiple sites interconnected by a communication network.

[0767] The processing and logic flows described in this specification can be executed by one or more programmable processors that run one or more computer programs to perform functions by manipulating input data and generating output. The processing and logic flows can also be executed by dedicated logic circuitry, and the device can be implemented as dedicated logic circuitry, such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0768] For example, processors suitable for running computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store those instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to one or more mass storage devices to receive data from or transfer data to, or both. However, a computer does not necessarily need to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. Processors and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0769] Intended to include the instruction manual and accompanying Figure 1 The above is to be considered as exemplary only, where exemplary means example. As used herein, unless the context clearly indicates otherwise, the use of "or" is intended to include "and / or".

[0770] While this patent document contains numerous details, it should not be construed as limiting the scope of any invention or claim, but rather as a description of features specific to particular embodiments of a particular invention. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment may also be implemented individually in multiple embodiments, or in any suitable sub-combination. Furthermore, while features may be described as functioning in certain combinations, or even initially claimed in this way, in some cases one or more features may be removed from the claimed combination, and the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0771] Similarly, although operations are described in a specific order in the accompanying drawings, this should not be construed as meaning that these operations must be performed in the specific order or sequence shown, or that all of the operations shown must be performed, in order to obtain the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0772] Only some implementation methods and examples have been described. Other implementation methods, enhancements and variations can be made based on the content described and illustrated in this patent document.

Claims

1. A method for visual media processing, comprising: For the conversion between the current video block and the bitstream of the current video block, a determination is made to represent motion information using one or more precisions, the motion information including motion vectors and / or motion vector predictors and / or motion vector difference MVD used during the conversion in the affine inter-frame mode using adaptive motion vector resolution AMVR; as well as The conversion is performed based on the determination of the one or more precisions, wherein multiple syntax elements are used in the bitstream to signal the one or more precisions. The determination is based on a first syntax element included in the bitstream to indicate whether AMVR should be applied to the current video block. Specifically, when the use of multiple MVD precisions for an affine coding block is false, the signaling notification of the first syntax element is skipped; and when the use of multiple MVD precisions for an affine coding block is true and the current video block uses affine mode encoding and decoding, the signaling notification of the first syntax element is given. The determination is further based on a second syntax element that indicates the precision of the one or more motion information, the signaling of which is based on the value of the first syntax element or whether the first syntax element is missing from the bitstream.

2. The method according to claim 1, further comprising: In the bitstream, the plurality of syntax elements corresponding to the plurality of precisions are selectively notified by signaling.

3. The method according to claim 1 or 2, wherein, The plurality of syntax elements includes at least a first syntax element to indicate whether default motion precision is applied.

4. The method according to claim 1, wherein, The first syntax element is an amvr flag or a motion information precision flag.

5. The method according to claim 1, wherein, The motion information also includes motion vectors and / or motion vector predictors and / or motion vector differences used during transitions in the normal inter-frame mode using adaptive motion vector resolution AMVR, wherein the motion information accuracy for the affine inter-frame mode differs from the motion information accuracy for the normal inter-frame mode.

6. The method according to claim 5, wherein, At least one of the plurality of syntax elements is encoded using context-based encoding, which includes the use of at least one context model.

7. The method according to claim 6, wherein, The context model used for the affine inter-frame mode is different from the context model used for the normal inter-frame mode.

8. The method according to claim 1 or 2, wherein, When it is determined that the first syntax element does not exist in the bitstream, the determination disables AMVR during the conversion of the current video block, wherein the conversion uses default motion information or default motion information precision.

9. The method according to claim 1 or 2, wherein, When it is determined that the first syntax element does not exist in the bitstream, the determination prohibits the use of the plurality of precisions to represent the motion information.

10. The method according to claim 1, wherein, The first syntax element is excluded from the bitstream if the motion vector difference (MVD) of all control points in the current video block is zero.

11. The method according to claim 1, wherein, If the motion vector difference (MVD) at a selected control point of the current video block is zero, the first syntax element is excluded from the bitstream.

12. The method of claim 1, wherein the first syntax element is excluded from the bitstream when AMVR is disabled for the affine inter-frame mode.

13. The method according to any one of claims 10 to 12, wherein, The affine inter-frame pattern is used for the current video block.

14. The method according to claim 5, wherein, When AMVR is disabled for the normal inter-frame mode and the normal inter-frame mode is used for the current video block, the first syntax element is excluded from the bitstream.

15. The method according to claim 8, wherein, The default motion information precision is 1 / 4 luminance sample point, 1 luminance sample point, or 1 / 16 luminance sample point.

16. The method according to claim 8, wherein, Set the default motion information precision to the value used during the motion compensation step.

17. The method according to claim 1 or 2, wherein, When it is determined that the first syntax element is not present in the bitstream, the determination enables AMVR during the conversion of the current video block.

18. The method according to claim 1, wherein, When the second syntax element is equal to zero, the motion information precision is an integer value; otherwise, the motion information precision is 4 luminance samples.

19. The method according to claim 1, wherein, When the second syntax element is equal to zero, the motion information precision is 4 luminance samples; otherwise, the motion information precision is an integer value.

20. The method according to claim 1, wherein, When the second syntax element is equal to zero, the motion information precision is an integer value; otherwise, the motion information precision is 1 / 16 of a luminance sample.

21. The method according to claim 1, wherein, When the second syntax element is equal to zero, the motion information precision is 1 / 16 of the brightness sample point; otherwise, the motion information precision is an integer value.

22. The method according to claim 6, wherein, The context model used for the affine inter-frame mode is the same as the context model used for the normal inter-frame mode.

23. The method according to claim 7 or 22, wherein, The syntax elements used for the affine inter-frame mode are the same as those used for the normal inter-frame mode.

24. The method according to claim 1, wherein, When intra-block copy mode is used during the conversion, AMVR includes a first AMVR scheme, and when intra-block copy mode is not used during the conversion, AMVR includes a different second AMVR scheme.

25. The method according to claim 1, wherein, The multiple syntax elements are included at the strip level, image level, or sequence level.

26. The method according to claim 1, wherein, The conversion includes generating the pixel values ​​of the current video block from the bitstream, or generating the bitstream from the current video block.

27. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein, When the instruction is executed by the processor, the processor: For the conversion between the current video block and the bitstream of the current video block, a determination of motion information using one or more precisions is made, the motion information including motion vectors and / or motion vector predictors and / or motion vector difference MVD used during the conversion in the affine inter-frame mode using adaptive motion vector resolution AMVR; as well as The conversion is performed based on the determination of the one or more precisions, wherein multiple syntax elements are used in the bitstream to signal the one or more precisions. The determination is based on a first syntax element included in the bitstream to indicate whether AMVR should be applied to the current video block. Specifically, when the use of multiple MVD precisions for an affine coding block is false, the signaling notification of the first syntax element is skipped; and when the use of multiple MVD precisions for an affine coding block is true and the current video block uses affine mode encoding and decoding, the signaling notification of the first syntax element is given. The determination is further based on a second syntax element that indicates the precision of the one or more motion information, the signaling of which is based on the value of the first syntax element or whether the first syntax element is missing from the bitstream.

28. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to: For the conversion between the current video block and the bitstream of the current video block, a determination of motion information using one or more precisions is made, the motion information including motion vectors and / or motion vector predictors and / or motion vector difference (MVD) used during the conversion in affine inter-frame mode using adaptive motion vector resolution AMVR; and The conversion is performed based on the determination of the one or more precisions, wherein, Multiple syntax elements are used in the bitstream to signal one or more precisions. The determination is based on a first syntax element included in the bitstream to indicate whether AMVR should be applied to the current video block. Specifically, when the use of multiple MVD precisions for an affine coding block is false, the signaling notification of the first syntax element is skipped; and when the use of multiple MVD precisions for an affine coding block is true and the current video block uses affine mode encoding and decoding, the signaling notification of the first syntax element is given. The determination is further based on a second syntax element that indicates the precision of the one or more motion information, the signaling of which is based on the value of the first syntax element or whether the first syntax element is missing from the bitstream.

29. A method for storing a bit stream, comprising: The bitstream is generated by performing the visual media processing method according to any one of claims 1-26; as well as The bit stream is stored in a non-transitory computer-readable storage medium.

Images