Symmetric motion vector differential coding
The implementation of SMVD encoding techniques addresses inefficiencies in video encoding by generating additional motion candidates and enabling flexible encoding modes, reducing bandwidth and complexity in video processing.
Patent Information
- Application Number
- JP2023100212
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-07-05
- Filing Date
- 2023-06-19
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-04-28
AI Technical Summary
Current video encoding technologies face inefficiencies in symmetric motion vector differential encoding, particularly in the handling of reference image lists and motion vector differences, leading to increased bandwidth usage and computational complexity.
Implementing symmetric motion vector difference (SMVD) encoding techniques that generate additional motion candidates based on symmetrical displacements and allow for flexible encoding modes, such as disabling motion vector difference (MVD) for specific reference lists, deriving MVDs from one list to another based on POC distances, and enabling decoder-side motion vector refinement methods.
Enhances encoding efficiency by reducing bandwidth demands and computational complexity while maintaining video quality, aligning with future video encoding standards like Versatile Video Coding (VVC).
Smart Images

Figure 0007594047000022 
Figure 0007594047000023 
Figure 0007594047000024
Abstract
Description
[Technical field]
[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application is a divisional application of patent application No. 2021-562776, which is the national stage of international patent application No. PCT / CN2020 / 087551 filed on April 28, 2020, and claims priority to and the benefit of international patent applications No. PCT / CN2019 / 08468 filed on April 28, 2019, PCT / CN2019 / 087201 filed on May 16, 2019, and PCT / CN2019 / 094954 filed on July 5, 2019. The entire disclosures of International Patent Applications PCT / CN2019 / 084768, PCT / CN2019 / 087201, and PCT / CN2019 / 094954 are incorporated by reference as part of the disclosure of this application.
[0002] This patent document relates to video coding techniques, devices and systems. [Background technology]
[0003] Despite advances in video compression, digital video remains the largest bandwidth user on the Internet and other digital communications networks. The bandwidth demands for digital video use are expected to continue to grow as the number of connected user devices capable of receiving and displaying video increases. Summary of the Invention
[0004] This specification describes various embodiments and techniques for video encoding or decoding using motion vector differential encoding using symmetric encoding or decoding techniques.
[0005] In one exemplary aspect, a method of video processing is disclosed. The method includes determining to use a Symmetric Motion Vector Difference (SMVD) mode for converting between a current video block and a bitstream representation of the current video block, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference pictures from Reference List 0 and Reference List 1 of reference pictures, and performing the conversion using the SMVD mode. The bitstream representation indicates disabling the use of the Motion Vector Difference (MVD) mode for one of Reference List 0 or Reference List 1.
[0006] In another exemplary aspect, another method of video processing is disclosed. The method includes determining to use a Symmetric Motion Vector Difference (SMVD) mode for converting between a current video block and a bitstream representation of the current video block, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks from two target reference pictures from Reference List 0 and Reference List 1 of reference pictures, and performing the conversion using the SMVD mode. The bitstream representation includes motion vector difference (MVD) information for Reference List 1.
[0007] In another exemplary aspect, another method of video processing is disclosed. The method includes determining to use a Symmetric Motion Vector Difference (SMVD) mode for converting between a current video block and a bitstream representation of the current video block, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in images from Reference List 0 and Reference List 1 of reference pictures, and performing the conversion using the SMVD mode. SMVD uses a first step to parse each reference picture list and identify a target reference picture to generate the two additional motion candidates.
[0008] In another exemplary aspect, another method of video processing is disclosed, which includes determining to perform the conversion based on the determination based on a condition to use a Symmetric Motion Vector Difference (SMVD) mode in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference pictures from Reference List 0 and Reference List 1 of reference pictures, and a Decoder-Side Motion Vector Difference (DMVD) coding mode for conversion between a current video block and a bitstream representation of the current video block.
[0009] In another exemplary aspect, another method of video processing is disclosed, the method including determining, based on a condition, whether to use an all-zero motion vector difference for a Symmetric Motion Vector Difference (SMVD) mode in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference pictures from Reference List 0 and Reference List 1 of reference pictures and a Decoder-side Motion Vector Difference (DMVD) coding mode for converting between a current video block and a bitstream representation of the current video block, and performing the conversion based on the determination.
[0010] In another exemplary aspect, another method of video processing is disclosed, the method including: selecting a first reference image and a second reference image from a plurality of reference images associated with a current video block; and converting between the current video block and a bitstream representation of the current video block based on the selected first and second reference images, wherein a first distance between the first reference image and a current image including the current video block and a second distance between the second reference image and the current image are equal.
[0011] In another exemplary aspect, another method of video processing is disclosed that includes determining that a first distance between a first reference image and a current image including a current video block and a second distance between a second reference image and the current image are not equal, and disabling a symmetric motion vector difference (SMVD) mode for the current video block based on the determination and for conversion between the underlying video block and a bitstream representation of the current video block.
[0012] In another exemplary aspect, another method of video processing is disclosed that includes deriving, for a current video block, a first motion vector difference (MVD) of a first reference image list based on a second MVD of a second reference image list, and converting between the current video block and a bitstream representation of the current video block based on the derivation, where the first MVD has a limited range.
[0013] In one exemplary aspect, a method of video processing is disclosed that includes determining whether a Symmetric Motion Vector Difference (SMVD) mode is allowed for a conversion between a current image of a video block and a bitstream representation of a first block based on coding information of the block, and performing the conversion based on the above.
[0014] In one exemplary embodiment, a method of video processing is disclosed, the method including: determining whether a symmetric motion vector (SMV) mode is permissible for a conversion between a block of video and a bitstream representation of the first block according to the method of the preceding example; and performing the conversion based on the determination.
[0015] In one exemplary aspect, a method of video processing is disclosed, which includes determining whether a Symmetric Motion Vector Difference (SMVD) mode is permissible for a block for conversion between a block of a current image of a video and a bitstream representation of the block, where whether the SMVD mode is permissible depends on a derivation process of two target reference images of the current image, the process including a first step for searching a forward target reference image in the reference image list 0 and searching a backward target reference image in the reference image list 1, and a second step for searching a backward target reference image in the reference image list 0 and searching a forward target reference image in the reference image list 1, where outputs of the first step and the second step are independent of each other, and performing conversion based on the determination.
[0016] In one exemplary aspect, a method of video processing is disclosed, the method including: deriving a second motion vector difference (MVD) of one reference image list from a first MVD associated with another reference image list for conversion between a block of a current image of a video and a bitstream representation of the block, the MVD including both horizontal and vertical components, imposing range constraints on at least one of the first MVD and the second MVD, and performing the conversion based on at least one of the constrained first MVD and the constrained second MVD.
[0017] In yet another exemplary aspect, a video processing apparatus is disclosed, the apparatus including a processing apparatus configured to perform the method described above, the apparatus may further perform video encoding or video decoding.
[0018] In yet another exemplary aspect, a computer readable medium is disclosed having stored thereon code for implementing the above-described method on a processor.
[0019] These and other aspects are described herein. [Brief description of the drawings]
[0020] [Figure 1] 1 illustrates an example of a derivation process for building a merge candidate list. [Diagram 2] 1 shows examples of spatial merge candidate locations. [Diagram 3] 1 shows examples of candidate pairs considered for redundancy check of spatial merge candidates. [Figure 4A] The positions for the second PU of Nx2N and 2NxN divisions are shown. [Figure 4B] The positions for the second PU of Nx2N and 2NxN divisions are shown. [Diagram 5] FIG. 13 is an illustration of motion vector scaling for temporal merge candidates. [Figure 6] 1 shows examples of candidate positions for temporal merge candidates, C0 and C1. [Figure 7] 1 illustrates an example of a combined bi-predictive merge candidate. [Figure 8] 1 summarizes the process of deriving motion vector prediction candidates. [Figure 9] 1 illustrates a description of motion vector scaling for spatial motion vector candidates. [Figure 10] 1 shows an example of decoder-side motion vector refinement (DMVR) based on bilateral template matching. [Figure 11] An example of an MVD(0,1) mirrored between list 0 and list 1 in a DMVR is shown. [Figure 12] An example of a motion vector (MV) that can be checked in one iteration is shown below. [Figure 13] FIG. 13 is an exemplary diagram of a symmetric mode. [Figure 14] FIG. 1 is a block diagram illustrating an example of a hardware platform that may be used to implement the methods described in this patent specification. [Figure 15] 1 is a flowchart illustrating an example of a video processing method. [Figure 16] 1 is a flowchart illustrating an example of a video processing method. [Figure 17] 1 is a flowchart illustrating an example of a video processing method. [Figure 18] 1 is a flowchart illustrating an example of a video processing method. [Figure 19] 1 is a flowchart illustrating an example of a video processing method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0021] Section headings are used herein for ease of understanding and are not intended to limit the embodiments disclosed in one section to only that section. Additionally, although certain embodiments have been described with reference to Versatile Video Coding (VVC) or other specific video codecs, the disclosed techniques are applicable to other video coding techniques. Additionally, although some embodiments describe video coding steps in detail, it will be understood that the corresponding steps of decoding the encoding are performed by a decoder. Additionally, the term video processing includes video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compressed format to another compressed format or at another compressed bit rate.
[0022] 1. Summary of the Invention This specification relates to video coding technology. In particular, it relates to symmetric motion vector differential coding. It may be applied to existing video coding standards such as HEVC, or to finalize standards (Versatile Video Coding). The present invention is also applicable to future video coding standards or video codecs.
[0023] 2. Initial consultations Video coding standards have evolved mainly through the development of well-known ITU-T and ISO / IEC standards. ITU-T created H.261 and H.263, ISO / IEC created MPEG-1 and MPEG-4 Visual, and the two organizations jointly created H.262 / MPEG-2 Video, H.264 / MPEG-4 AVC (Advanced Video Coding), and H.265 / HEVC standards. Since H.262, video coding standards have been based on hybrid video coding structures where temporal prediction and transform coding are utilized. In order to explore future video coding technologies beyond HEVC, in 2015, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET). Since then, many new methods have been adopted by JVET and incorporated into reference software called JEM (Joint Exploration Mode). In April 2018, the Joint Video Expert Team (JVET) was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG), and is working on formulating the VVC standard with the goal of reducing the bitrate by 50% compared to HEVC.
[0024] 2.1 Inter Prediction in HEVC / H.265 Each inter-predicted PU has motion parameters for one or two reference picture lists. The motion parameters include a motion vector and a reference picture index. The use of one of the two reference picture lists may be signaled using inter_pred_idc. The motion vector may be explicitly coded as a delta to the predictor.
[0025] When a CU is coded in skip mode, one PU is associated with this CU, there are no significant residual coefficients, and there are no coded motion vector deltas or reference picture indices. A merge mode is specified, which obtains motion parameters for the current PU from neighboring PUs, including spatial and temporal candidates. The merge mode can be applied to any inter-predicted PU, not just for skip mode. As an alternative to the merge mode, there is an explicit transmission of motion parameters, where the motion vectors (more precisely, the motion vector differentials (MVDs) compared to the motion vector predictor), the corresponding reference picture indices of each reference picture list, and the usage status of the reference picture list are explicitly signaled to each PU. Such a mode is referred to as Advanced Motion Vector Prediction (AMVP) in this disclosure.
[0026] If the signaling indicates to use one of the two reference picture lists, the PU is generated from one block of samples. This is called "uni-prediction". Uni-prediction is available for both P and B slices.
[0027] If the signaling indicates to use both reference picture lists, the PU is generated from two blocks of samples. This is called "bi-prediction". Bi-prediction is only available for B slices.
[0028] The inter prediction modes defined in HEVC will be described in detail below. First, the merge mode will be described.
[0029] 2.1.1 Reference Image Buffer In HEVC, the term inter prediction is used to indicate a prediction derived from data elements (e.g., sample values or motion vectors) of a reference picture other than the current decoded picture. Similar to H.264 / AVC, an image can be predicted from multiple reference pictures. The reference pictures used for inter prediction are organized into one or more reference picture lists. A reference index identifies which reference picture in the list to use to generate the prediction signal.
[0030] One reference image list, List0, is used for P slices, and two reference image lists, List0 and List1, are used for B slices. Note that the reference images included in List0 / 1 may be a mixture of past and future images in terms of the shooting / display order.
[0031] 2.2.1 Merge Mode
[0032] 2.1.2.1 Deriving Merge Mode Candidates When predicting a PU using merge mode, an index pointing to an entry in the merge candidate list is parsed from the bitstream and used to look up the motion information. The construction of this list is specified in the HEVC standard and can be summarized based on the following sequence of steps: ● Step 1: Derive initial candidates 〇Step 1.1: Spatial candidate derivation Step 1.2: Check for redundancy of spatial candidates 〇Step 1.3: Temporal candidate derivation Step 2: Insert additional candidates Step 2.1: Creating bidirectional prediction candidates Step 2.2: Inserting zero motion candidates
[0033] These steps are also shown diagrammatically in Fig. 1. For spatial merge candidate derivation, select up to four merge candidates among candidates at five different positions. For temporal merge candidate derivation, select up to one merge candidate among two candidates. Since the decoder side assumes a constant number of candidates per PU, if the number of candidates obtained in step 1 does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the slice header, generate additional candidates. Since the number of candidates is constant, we use a truncated unary binarization (TU) to encode the index of the best merge candidate. If the size of a CU is equal to 8, all PUs of the current CU share one merge candidate list, which is the same as the merge candidate list of the 2N × 2N prediction units.
[0034] The operations associated with the above steps are described in detail below.
[0035] FIG. 1 illustrates an example of a derivation process for building a merge candidate list.
[0036] 2.1.2.2 Spatial candidate derivation In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates at the positions shown in FIG. 2. The order of derivation is A1, B1, B0, A0, B2. Position B2 is considered only if any of the PUs at positions A1, B1, B0, A0 are unavailable (e.g., because they belong to another slice or tile) or are intra-coded. After adding the candidate at position A1, the remaining candidates are added and subjected to a redundancy check, which ensures that candidates with the same motion information are removed from the list, improving coding efficiency. To reduce computational complexity, the aforementioned redundancy check does not consider all possible candidate pairs. Instead, only the pairs linked by arrows in FIG. 3 are considered, and a candidate is added to the list only if the corresponding candidate used for the redundancy check does not have the same motion information. Another source of duplicated motion information is a "second PU" associated with a partition different from 2N×2N. FIG. 4A and FIG. 4B depict the second PU for the cases of N×2N and 2N×N, respectively. When splitting the current PU into N×2N, the candidate at position A1 is not considered in the list construction. In fact, adding this candidate would lead to two prediction units with the same motion information, which is redundant for having only one PU in one coding unit. Similarly, when splitting the current PU into 2N×N, position B1 is not considered.
[0037] 2.1.2.3 Temporal candidate derivation In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, we derive a scaled motion vector based on the co-located PU belonging to the image with the smallest POC difference with the current image in a given reference image list. In the slice header, the reference image list used for the derivation of the co-located PU is explicitly signaled. As shown by the dotted line in Figure 5, the scaled motion vector of the temporal merge candidate is obtained, which is scaled from the motion vector of the co-located PU using the POC distances tb and td. tb is defined as the POC difference between the reference image of the current image and the current image, and td is defined as the POC difference between the reference image of the co-located PU and the co-located image. We set the reference image index of the temporal merge candidate equal to zero. The practical realization of this scaling process is described in the HEVC specification. For B slices, we take two motion vectors, one for reference image list 0 and the other for reference image list 1, and combine them to form a bi-predictive merge candidate.
[0038] For the co-located PU(Y) belonging to the reference frame, select the location of the temporal candidate between candidates C0 and C1 as shown in Figure 6. If the PU at location C0 is not available, is intra-coded, or is outside the current coding tree unit (CTU, also known as LCU, largest coding unit) row, location C1 is used. Otherwise, location C0 is used to derive the temporal merge candidate.
[0039] 2.1.2.4 Additional Candidate Inserts Besides the spatial-temporal merge candidates, there are two additional types of merge candidates: joint bi-predictive merge candidates and zero merge candidates. The spatial-temporal merge candidates are utilized to generate joint bi-predictive merge candidates. The joint bi-predictive merge candidates are used only for B slices. The joint bi-predictive candidate is generated by combining the first reference picture list motion parameters of the first candidate with the second reference picture list motion parameters of another candidate. If these two tuples provide different motion hypotheses, these tuples form a new bi-predictive candidate. As an example, FIG. 7 shows the case where two candidates with mvL0 and refIdxL0, or mvL1 and refIdxL1 in the original list (left side) are used to generate a joint bi-predictive merge candidate that is added to the final list (right side). There are various rules for the combinations considered to generate these additional merge candidates.
[0040] By inserting zero motion candidates and filling the remaining entries in the merge candidate list, the MaxNumMergeCand capacity is hit. These candidates have a spatial displacement of zero and a reference image index that starts at zero and increases each time a new zero motion candidate is added to the list. Finally, no redundancy check is performed on these candidates.
[0041] 2.1.3 AMVP AMVP exploits the spatial-temporal correlation between motion vectors and neighboring PUs and uses it for explicit transmission of motion parameters. For each reference picture list, we first build a motion vector candidate list by checking the availability of the left and upper temporal neighboring PU positions, removing redundant candidates, and adding zero vectors to keep the length of the candidate list constant. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the merge index signaling, the index of the best motion vector candidate is encoded using a shortened unary term. The maximum value to be coded in this case is 2 (see Figure 8). The following sections will explain the details of the motion vector prediction candidate derivation process.
[0042] 2.1.3.1 Derivation of AMVP Candidates
[0043] FIG. 8 summarizes the process of deriving motion vector predictor candidates.
[0044] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. To derive spatial motion vector candidates, as shown in Figure 2, we finally derive two motion vector candidates based on the motion vectors of each PU at five different positions.
[0045] To derive a temporal motion vector candidate, select one motion vector candidate from two candidates derived based on two different co-located positions. After creating an initial list of spatial-temporal candidates, remove duplicate motion vector candidates in the list. If the number of candidates is greater than two, remove from the list the motion vector candidates whose reference image index in the associated reference image list is greater than 1. If the number of spatial-temporal motion vector candidates is less than two, add an additional zero motion vector candidate to the list.
[0046] 2.1.3.2 Spatial Motion Vector Candidates In deriving spatial motion vector candidates, we consider up to two candidates that are in the same position as the motion merge out of five possible candidates derived from PUs located as shown in Figure 2. The order of derivation for the left side of the current PU is specified as A0, A1, scaled A0, scaled A1. The order of derivation for the top side of the current PU is specified as B0, B1, B2, scaled B0, scaled B1, scaled B2. So, for each edge, there are four possible cases for motion vector candidates: two cases that do not require spatial scaling and two cases that use spatial scaling. The four different cases can be summarized as follows: No spatial scaling -(1) Same reference image list and same reference image index (same POC) - (2) Different reference image lists, but the same reference image (same POC) Spatial scaling - (3) Same reference image list, but different reference images (different POC) -(4) Different reference image lists and different reference images (different POCs)
[0047] First check the non-spatial scaling case, then do spatial scaling. Regardless of the reference picture list, if the POC is different between the reference pictures of the neighboring PUs and the current PU, consider spatial scaling. If all PUs of the left candidate are not available or are intra-coded, scaling of the top motion vector helps the parallel derivation of the left and top MV candidates. Otherwise, spatial scaling is not allowed for the top motion vector.
[0048] In the spatial scaling process, the motion vectors of neighboring PUs are scaled in the same manner as in temporal scaling, as shown in Fig. 9. The main difference is that the reference image list and index of the current PU are given as input, and the actual scaling process is the same as in temporal scaling.
[0049] 2.1.3.3 Temporal Motion Vector Candidates Except for deriving the reference picture index, all the processes for deriving the temporal merge candidates are the same as those for deriving the spatial motion vector candidates (see FIG. 6). The reference picture index is signaled to the decoder.
[0050] 2.2 Inter Prediction Method in VVC For inter prediction fine-tuning, several new coding tools are adopted, such as Adaptive Motion Vector Difference Resolution (AMVR) for signaling MVD, Affine Prediction Mode, Triangular Prediction Mode (TPM), Advanced TMVP (ATMVP, also known as SbTMVP), Generalized Bidirectional Prediction (GBI), and Bidirectional Optical Flow (BIO).
[0051] 2.2.1 Coding block structure in VVC In VVC, a quadtree / binarytree / multipletree (QT / BT / TT) structure is adopted to divide an image into square or rectangular blocks.
[0052] Besides QT / BT / TT, a separate tree (also known as a dual coding tree) is also adopted in VVC for I-frames. Using separate trees, the coding block structure is signaled separately for luma and chroma components.
[0053] 2.2.2 Decoder-side Motion Vector Refinement (DMVR) In the bidirectional prediction operation, to predict one block region, the bidirectional prediction blocks constructed using the motion vectors (MVs) in list0 and the MVs in list1 are combined to form one prediction signal. In the decoder-side motion vector fine-tuning (DMVR) method, the two motion vectors of the bidirectional prediction are further fine-tuned.
[0054] 2.2.2.1 DMVR in JEM In the JEM design, the motion vectors are fine-tuned by a bidirectional template matching process. To obtain the fine-tuned MVs without transmitting additional motion information, we apply bilateral template matching in the decoder and perform a distortion-based search between the bilateral template and the reconstructed samples in the reference images. An example is shown in Fig. 10. As shown in Fig. 10, we generate a bilateral template as a weighted combination (i.e., average) of two prediction blocks from the first MV0 in list0 and the MV1 in list1, respectively. The template matching operation consists of calculating a cost measure between the generated template and a sample region (near the first prediction block) in the reference images. For each of the two reference images, the MV with the smallest template cost is considered as the updated MV of that list and replaces the original MV. In the JEM, we search for nine MV candidates for each list. The nine MV candidates include the original MV and eight surrounding MVs with one luma sample offset with respect to the original MV in either horizontal or vertical direction or both directions. Finally, we use two new MVs, namely MV0' and MV1' as shown in Fig. 10, to generate the final bidirectional prediction result. We use the sum of absolute differences (SAD) as the cost measure. Note that when calculating the cost of a predicted block generated by one surrounding MV, we use the rounded MV (integer pixels) instead of the actual MV to obtain the predicted block.
[0055] 2.2.2.2 DMVR in VVC For DMVR in VVC, assume that MVD mirroring is performed between list 0 and list 1 as shown in FIG. 11, and fine-tune the MV by performing bilateral matching. That is, find the best MVD among several MVD candidates. Let the MVs of two reference image lists be MVL0(L0X, L0Y) and MVL1(L1X, L1Y). Define the MVD denoted by (MvdX, MvdY) for list 0 that can minimize a cost function (e.g., SAD) as the best MVD. The SAD function is defined as the SAD between the reference block of list 0 derived by the motion vector (L0X+MvdX, L0Y+MvdY) in the reference image of list 0 and the reference block of list 1 derived by the motion vector (L1X-MvdX, L1Y-MvdY) in the reference image of list 1.
[0056] The motion vector refinement process may be iterated twice. In each iteration, up to six MVDs (integer pixel accuracy) may be checked in two steps, as shown in FIG. 12. In the first step, MVDs (0,0), (-1,0), (1,0), (0,-1), (0,1) are checked. In the second step, one of MVDs (-1,-1), (-1,1), (1,-1) or (1,-1) may be selected and checked further. Let function Sad(x,y) return the SAD value of MVD(x,y). The MVD, denoted by (MvdX,MvdY) checked in the second step is determined as follows: MvdX=-1; MvdY=-1; If (Sad(1,0) <Sad(-1,0)) MvdX=1; If (Sad(0,1) <Sad(0,-1)) MvdY=1;
[0057] In the first iteration, the starting point is the signaled MV, and in the second iteration, the starting point is the signaled MV plus the best MVD selected in the first iteration. DMVR is only applied when one reference picture is a previous picture and the other reference picture is a later picture, and the two reference pictures have the same picture order count distance from the current picture.
[0058] To further simplify the DMVR process, we proposed several changes to the JEM design. Specifically, the main features of the DMVR design adopted for VTM-4.0 (coming soon) are: ●If the (0,0) position SAD between list0 and list1 is less than a threshold, terminate early. ●If at a certain position, the SAD between list0 and list1 is 0, then terminate early. ● DMVR block size: W*H>=64&&H>=8, where W and H are the width and height of the block. For DMVRs where the size of the CU is greater than 16*16, divide the CU into multiple sub-blocks of 16x16. If only the width or height of the CU is greater than 16, the CU is only divided vertically or horizontally. ● Size of reference block is (W+7)*(H+7) (for luminance). 25-point SAD-based integer pixel search (i.e., (+-)2 fine-tuned search ranges, single stage) Bilinear Interpolation Based DMVR ● Sub-pixel refinement based on the “parameter error surface equation”. This step is performed only if, in the previous MV refinement iteration, the minimum SAD cost is not equal to zero and the best MVD is (0,0). ● Luma / chroma MC with reference block padding (if necessary). ● Fine-tuned MV used only for MC and TMVP.
[0059] 2.2.2.2.1 How to Use DMVR DMVR can be enabled if all of the following conditions are true: - The DMVR enabled flag in the SPS (i.e., sp_dmvr_enabled_flag) is 1. - The TPM flag, the inter-affine flag, the sub-block merge flag (either ATMVP or affine merge), and the MMVD flag are all 0. - The merge flag is 1. - The current block is bidirectionally predicted and the POC distance between the current picture and the reference pictures in list 1 is equal to the POC distance between the reference pictures in list 0 and the current picture. - The height of the current CU is 8 or more. - The number of luma samples (CU width * height) is 64 or more.
[0060] 2.2.2.2.2 Sub-pixel refinement based on the “parameter error surface equation” The method is summarized below. 1. Compute the parametric error surface fit only if the center location is the best cost location at a given iteration. 2. Using the center location cost and the costs at locations (-1,0), (0,-1), (1,0), and (0,1) from the center, fit the following 2-D parabolic error surface equation:
[0061]
number
[0062] where (x0,y0) corresponds to the position with the minimum cost and C corresponds to the minimum cost value. (x0,y0) is calculated by solving Equation 5 in five unknowns as follows:
[0063]
number
[0064] (x0,y0) can be calculated with any required sub-pixel accuracy by adjusting the precision with which the division is done (i.e. how many bits of the quotient are calculated). To get 1 / 16 pixel precision, the absolute value of the quotient only needs to be calculated with 4 bits, which is suitable for a fast shift-and-subtract based implementation requiring two divisions per CU. 3. Add the calculated (x0,y0) to the integer refinement MV to get the sub-pixel accurate refinement delta MV.
[0065] 2.2.3 Symmetric Motion Vector Difference For motion information coding in bidirectional prediction, symmetric motion vector difference (SMVD) is applied.
[0066] First, at the slice level, the variables RefIdxSymL0 and RefIdxSymL1 indicating the reference image indexes of list 0 / 1 used in SMVD mode are derived by the following steps, respectively: If at least one of the two variables is equal to -1, the SMVD mode shall be disabled.
[0067] 8.3.5 Decoding Process for Symmetric Motion Vector Difference Reference Indices In this process, when sym_mvd_flag is 1 in a certain coding unit, RefIdxSymL0 and RefIdxSymL1 are output, which specify the reference image indexes in list 0 and list 1 for taking the symmetric motion vector difference.
[0068] The variable RefIdxSymLX, where x is 0 and 1, is derived as follows:
[0069] Step #1: First loop through each reference image list - The variable currPic specifies the current image. - RefIdxSymL0 is set to -1. - For each index i, i=0..NumRefIdxActive[0], the following applies: - Set RefIdxSymL0 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[0][i])>0, - DiffPicOrderCnt(currPic,RefPicList[0][i]) <DiffPicOrderCnt(currPic,RefPicList[0][RefIdxSymL0])あるいはRefIdxSymL0は-1である. - RefIdxSymL1 is set to -1. - For each index i, i=0..NumRefIdxActive[1], the following applies: - Set RefIdxSymL1 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[1][i])<0, - DiffPicOrderCnt(currPic,RefPicList[1][i]) > DiffPicOrderCnt(currPic,RefPicList[1][RefIdxSymL1]) or RefIdxSymL1 is -1.
[0070] Step #2: Second loop through each reference image list - If RefIdxSymL1=-1 or RefIdxSymL1=-1, the following applies: - For each index i, i=0..NumRefIdxActive[0], the following applies: - Set RefIdxSymL0 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[0][i])<0, - DiffPicOrderCnt(currPic,RefPicList[0][i])>DiffPicOrderCnt(currPic,RefPicList[0][RefIdxSymL0]) or RefIdxSymL0 is -1. - For each index i, i=0..NumRefIdxActive[1], the following applies: - Set RefIdxSymL1 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[1][i])>0, DiffPicOrderCnt(currPic,RefPicList[1][i]) <DiffPicOrderCnt(currPic,RefPicList[1][RefIdxSymL1])あるいは、RefIdxSymL1は-1である。
[0071] Second, at the CU level, if the prediction direction of the CU is bi-predictive and BiDirPredFlag is equal to 1, we explicitly signal a symmetric mode flag indicating whether to use symmetric mode.
[0072] If this flag is true, then only mvp_l0_flag, mvp_l1_flag and MVD0 are explicitly signaled. The reference indices are set to RefIdxSymL0, RefIdxSymL1 for list 0 and list 1, respectively. MVD1 is simply set to -MVD0. The final motion vectors are given by the following equations:
[0073]
number
[0074] FIG. 13 is an exemplary diagram of the symmetric mode.
[0075] The relevant syntax is:
[0076] [Table 1] [Table 2]
[0077] 2.2.4 MVD Semantics
[0078] 7.4.8.8 Motion Vector Difference Semantics abs_mvd_greater0_flag[compIdx] specifies whether the absolute value of the motion vector component difference is greater than 0. abs_mvd_greater1_flag[compIdx] specifies whether the absolute value of the difference between the motion vector components is greater than one. If abs_mvd_greater1_flag[compIdx] is not present, it is inferred to be equal to 0. abs_mvd_minus2[compIdx] plus 2 specifies the absolute value of the motion vector component difference. If abs_mvd_minus2[compIdx] is not present, it is inferred to be equal to -1. mvd_sign_flag[compIdx] specifies the sign of the motion vector component difference as follows: - If mvd_sign_flag[compIdx]=0, then the difference of the corresponding motion vector components has a positive value. On the other hand (if mvd_sign_flag[compIdx]=1), the difference of the corresponding motion vector components has a negative value. If mvd_sign_flag[compIdx] is not present, it is inferred to be equal to 0.
[0079] For compIdx=0..1, the motion vector difference lMvd[compIdx] is derived as follows: lMvd[compIdx]=abs_mvd_greater0_flag[compIdx]*(abs_mvd_minus2[compIdx]+2)*(1-2*mvd_sign_flag[compIdx]) (7 126) The value of lMvd[compIdx] is in the range of -215 to 215-1.
[0080] Based on the values of MotionModelIdc[x0][y0], the motion vector difference is derived as follows: - if MotionModelIdc[x0][y0]=0, the variable MvdLX[x0][y0][compIdx]) (X=0 or 1) specifies the difference between the list X vector component to be used and its prediction. The array index x0,y0 specifies the position (x0,y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the image. Horizontal motion vector component differences are assigned compIdx=0 and vertical motion vector components are assigned compIdx=1. - if refList=0 then MvdL0[x0][y0][compIdx] is set to lMvd[compIdx], with compIdx=0..1 - On the other hand (when refList=1), MvdL1[x0][y0][compIdx] is set to lMvd[compIdx], with compIdx=0..1. - on the other hand (if MotionModelIdc[x0][y0] ≠ 0), the variables MvdCpLX[x0][y0][cpIdx][compIdx] (X=0 or 1) specify the difference between the list X vector component to be used and its prediction. The array indexes x0,y0 specify the position (x0,y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the image, and the array index cpIdx specifies the control point index. The difference in the horizontal motion vector component is assigned compIdx=0 and the difference in the vertical motion vector component is assigned compIdx=1. - if refList=0 then MvdCpL0[x0][y0][cpIdx][compIdx] is set to lMvd[compIdx], with compIdx=0..1 - On the other hand (when refList=1), MvdCpL1[x0][y0][cpIdx][compIdx] is set to lMvd[compIdx], with compIdx=0..1.
[0081] 3. Examples of problems to be solved by the disclosed embodiments The current SMVD may have the following problems: 1. If mvd_l1_zero_flag is true and inefficient, SMVD is not allowed. 2. In the current SMVD mode, the MVD of reference picture list 0 is always signaled and does not consider the bitrate cost of the side information. 3. The target reference image indices for lists 0 and 1 used in SMVD mode are derived at slice level using two steps. a. Find the closest reference picture from all forward reference pictures in list 0, and find the closest reference picture from all backward reference pictures in list 1. b. If one of them is not found, search in the opposite direction.
[0082] Note that the second step (ie, step b) needs to be invoked for all of the following cases: 1) List 1 has only forward reference images, and list 0 has only backward reference images. 2) Both List 0 and List 1 have only forward reference pictures. 3) Both List 0 and List 1 have only backward reference images. 4) List X has both forward and backward reference images, but list Y (Y=1-X) has only reference images that are not the same as those listed in bullet a.
[0083] For cases 2) and 3), it is not reasonable to activate SMVD since there is no symmetrical movement in the same direction (forward or backward) between the two reference points. 4. Moreover, this search method can find a reference image that is not closest to the current image, which may reduce the coding performance of SMVD. The current design has the limitation of finding forward from list 0, backward from list 1, and vice versa. For example, if list 0 and list 1 have both forward and backward reference images, but the closest reference image in list 0 is from backward, and the closest reference image in list 1 is from forward. The current design cannot select the closest one from backward in list 0. 5. If List 0 has both forward and backward reference images and List 1 has only forward reference images, in the current design, List 0 will identify two forward reference images, which contradicts the concept of SMVD design. 6. The target reference image may be found by invoking two loops for each reference image list, which may increase complexity. 4. Symmetric MVD is assumed in both DMVR and SMVD, but DMVR is disabled in SMVD mode. 5. In SMVD mode, all MVDs can be zero, just as in normal AMVR mode. 6. The MVD for list 1 is derived from the signaled MVD for list 0, i.e., the inverse of both the horizontal and vertical components of the MVD for list 0. The assumption is that the two reference images are at the same distance as the current image. However, that is not always true. 7. VVC is a function of MVD [-2 15 ,2 15 -1]. In SMVD, the MVD for list 1 is derived from the signaled MVD for list 0, which may exceed the constraints on the MVD.
[0084] 4. Exemplary Embodiments The following should be considered as examples to illustrate the general concept. These embodiments should not be construed in a narrow sense. Moreover, these inventions can be combined in any manner.
[0085] In this specification, decoder-side motion vector derivation (DMVD) includes methods such as DMVR, FRUC, etc., which perform motion estimation, and BIO, which performs sample-by-sample motion refinement, to derive or refine block / sub-block motion information.
[0086] The unequal weights applied to a predicted block may refer to weights used in a GBI process, a LIC process, a weighted prediction process, or other encoding / decoding process of an encoding tool when additional operations need to be applied to a predicted block instead of the average value of the two predicted blocks.
[0087] Let the reference images in list0 and list1 be Ref0 and Ref1, respectively, the POC distance between the current image and Ref0 be PocDist0 (e.g., (POC of the current image minus the POC of Ref0)), and the POC distance between Ref1 and the current image be PocDist1 (e.g., (POC of Ref1 minus the POC of the current image)). Let the width and height of the block be W and H, respectively. Let the function abs(x) return the absolute value of x.
[0088] An all-zero MVD represents the case where all signaled MVDs (including both x and y components) are zero.
[0089] For SMVD mode, let the two target reference images be RefIdxL0 and RefIdxL1, respectively. Two variables currPocDiffL0 and currPocDiffL1 are defined to represent the POC distance between the reference image and the current image. For example, currPocDiffL0=POC(current image)-POC(RefIdxL0), currPocDiffL1=POC(current image)-POC(RefIdxL1), Abs(x)=x>0?x:-x. If the MVD in a list X (X=0,1) is indicated as being forced to zero (e.g.,
[0090] [ka]
[0091] is 1), SMVD may be permitted. In one example, the MVD encoding for list 1 is uncoded and zeroed out (e.g.,
[0092] [ka]
[0093] is equal to 1), the SMVD mode may still be enabled for one block. i. In one example, the MVD for reference image list 1 is also set to 0 (e.g.,
[0094] [ka]
[0095] is equal to 1), may be derived from the associated MVD for reference picture list 0. b. In one example, in SMVD mode, MVD may not be signaled for list X and may be signaled for list Y=1-X, and the signaled MVD in list Y may be used to derive (e.g., mirror) the MVD in list X. For example, X may be equal to 0. c. Alternatively, if the MVD in list X is indicated as being forced to zero, then the SMVD may not be allowed. If i.SMVD is not permitted, all syntax elements related to SMVD may not be signaled, e.g., they may be implicitly derived. 2. In SMVD mode, it is proposed to code the MVD information of reference picture list 1. In this case, the MVD of reference picture list 0 is derived from the MVD of reference picture list 1. In one example, a reference list with associated MVDs to be signaled (or derived) may be signaled in the bitstream. i. In one example, the reference picture list indication that the associated MVD needs to be signaled may be signaled at the sequence / picture / slice / tile / picture unit level, e.g., in the SPS / PPS / slice header. b. In one example, the reference list with associated MVD to be signaled (or derived) may be derived according to the reference pictures in the reference picture list. i. In one example, a reference list associated with the MVD to be signaled (or derived) may be derived according to two reference images of the object. 1. Further, alternatively, if Abs(currPocDiffL0) is greater than Abs(currPocDiffL1), then the MVD of list 0 may be signaled. Otherwise, the MVD of list 1 may be signaled. 2. Further, alternatively, if Abs(currPocDiffL0) is greater than Abs(currPocDiffL1), then the MVD of list 1 may be signaled. Otherwise, the MVD of list 0 may be signaled. 3. Further, alternatively, if Abs(currPocDiffL0) is greater than or equal to Abs(currPocDiffL1), then the MVD of list 0 may be signaled. Otherwise, the MVD of list 1 may be signaled. 4. Further, alternatively, if Abs(currPocDiffL0) is greater than or equal to Abs(currPocDiffL1), then the MVD of list 1 may be signaled. Otherwise, the MVD of list 0 may be signaled. 5. Further, alternatively, if Abs(currPocDiffL0) is equal to Abs(currPocDiffL1), then the MVD of list 0 may be signaled. Otherwise, the MVD of list 1 may be signaled. 6. Further, alternatively, if Abs(currPocDiffL0) is equal to Abs(currPocDiffL1), then the MVD of list 1 may be signaled. Otherwise, the MVD of list 0 may be signaled. c. In one example, a reference list with an associated MVD to be signaled (or derived) may be changed from one video unit (e.g., slice / tile group / tile / brick / CU) to another video unit. i. In one example, the selection of a reference list with an associated MVD to be signaled may depend on the reference image information of neighboring blocks. ii. In one example, a reference image list with a small absolute POC distance between PocDist0 and PocDist1 (where the two reference images are the two used in SMVD) is selected. 1. In one example, if PocDist0 is less than PocDist1, then the MVD of list0 is signaled and the MVD of list1 is derived. 2. Alternatively, select the reference image list with the larger absolute POC distance between PocDist0 and PocDist1 (the two reference images are the two used in SMVD). 3. If PocDist0 and PocDist1 are equal, then list X (X is 0 or 1) may be selected. 3. It is proposed to allow SMVD using both target reference images from the same reference image list. In one example, one of the two target reference images is in front (ie, its POC is smaller than the current POC) and the other is in the back (ie, its POC is larger than the current POC). b. In one example, all reference images in list 0 and list 1 may be checked to find a first reference image whose POC is closest to the current POC among all POCs of reference images smaller than the current POC, and a second reference image whose POC is closest to the current POC among all POCs of reference images larger than the current POC. 4. It is proposed to skip the second step in the derivation of the target reference image for the SMVD mode if at least one target reference image is found from the first step. In one example, if one reference image is found in list 0 and no reference images are found in list 1, the second step in deriving the target reference image for SMVD mode is skipped. b. In one example, if no reference image is found in list 0 and one reference image is found in list 1, the second step in deriving the target reference image for SMVD mode is skipped. c. Additionally or alternatively, SMVD is disabled, i.e., all SMVD related syntax signaling is skipped. 5. It is proposed to separate the output of the first step (e.g., step #1 in section 2.2.3) from the output of the second step (e.g., step #1 in section 2.2.3). Only three cases are supported: when both target reference image indices are found either in the first step (search forward in list 0 for target reference images in list 0 and backward in list 1 for target reference images in list 1) or in the second step (search backward in list 0 for target reference images in list 0 and forward in list 1 for target reference images in list 1) or when SMVD is disabled (not found at all in the first or second step). a. In one example, before performing the second step in deriving the target reference image in SMVD mode, if at least one of the target reference images is not found, reset the target reference image index to be unavailable (e.g., reset RefIdxSymL0 and RefIdxSymL1 to -1). 6. If there are two target reference images found, it is proposed to add a constraint to select the one with the smaller and larger POC value compared to the current image. a. In one example, an additional check of POC distance is added after the target reference image derivation process, and the target reference image may be reset to be unavailable (eg, resetting RefIdxSymL0 and RefIdxSymL1 to -1). b. In one example, if (RefIdxSymL1 !=-1 && RefIdxSymL0 !=-1 && DiffPicOrderCnt(currPic, RefPicList[0][RefIdxSymL1]) * DiffPicOrderCnt(currPic, RefPicList[1][RefIdxSymL1]) > 0) is true, the target reference picture may be reset to be unavailable. 7. It is proposed to remove the restriction of finding a forward reference image (whose POC is smaller than the current POC) from list 0 and a backward reference image (whose POC is larger than the current POC) from list 1, and vice versa. a. In one example, the target reference images used in SMVD may be derived using two dependent checks, one loop for each reference image list. i. The first check is to find one reference image from list X (X is 0 or 1). ii. The second check is to find one reference image from list Y according to the output of the first check image and the reference images in list Y, where Y is 1-X. b. In one example, first, for a list X (X is 0 or 1), calculate the POC distance between each reference image and the current image, whether it is forward or backward according to a given order. i. In one example, the given order is ascending / descending order of reference image index. ii. In one example, the reference image with the smallest (or largest) POC distance (eg, Abs(POC(reference image)-POC(current image)) is selected as the target reference image of list X, denoted by RefInLX. iii. Further, alternatively, select one reference image from list Y (Y is 1-X) according to reference image RefInLX. 1. In one example, if RefInLX has a small POC value compared to the current image, then the target reference image for list Y shall have a large POC value compared to the current image. 2. In one example, if RefInLX has a large POC value compared to the current image, then the target reference image for list Y shall have a small POC value compared to the current image. 3. In one example, the target reference image for list Y is the one among all backward reference images (if RefInLX is forward) or forward reference images (if RefInLX is backward) in list Y that has the smallest POC distance compared to the current image. 8. Whether SMVD is allowed may depend on the coding information of the block. In one example, the coding information may include MVD precision information and weights of prediction samples in bidirectional prediction. b. In one example, if unequal weights are selected for bi-predictive blocks, SMVD may not be allowed. i. Alternatively, in SMVD mode, unequal weight mode may not be allowed. ii. If SMVD is not allowed, all syntax elements related to SMVD may not be signaled and / or implicitly derived. iii. If unequal weights are not allowed, then all relevant syntax elements regarding unequal weights may not be signaled and / or implicitly derived. iv. c. In one example, SMVD may not be permitted at a particular MVD / MV precision. i. For example, for N pixel MV / MVD precision, SMVD may not be allowed, e.g., N=1,4. Alternatively, in SMVD mode, AMVR may not be allowed, for example, in SMVD mode, only N pixel MV / MVD precision may be allowed, for example, N=1 / 4. iii. If SMVD is not allowed, all syntax elements related to SMVD may not be signaled and may be implicitly derived. iv. If AMVR is not allowed, all syntax elements related to AMVR may not be signaled and may be implicitly derived. 9. DMVR or / and other DMVD methods may be enabled in SMVD mode. In one example, in SMVD mode, if the MV / MVD precision is N pixels, DMVR and / or other DMVD methods may be used to fine-tune the MVD with mvDmvr, where mvdDmvr is M pixels precision. N, M=1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8, 16, etc. b. In one example, M may be less than or equal to N. c. In one example, the MVD may not be signaled in SMVD mode, and instead, the DMVR and / or other DMVD methods may be applied to generate the MVD. i. Additionally, alternatively, no AMVR information may be signaled and the MV / MVD accuracy may be derived to a predefined value (eg, MVD has a quarter pixel accuracy). d. In one example, an indication of whether DMVR or / and other DMVD methods are applied may be signaled in SMVD mode. i. If DMVR or / and other DMVD methods are applied, MVD may not be signaled. In one example, such an indication may be signaled for a particular MV / MVD precision, e.g., for 1 pixel and / or 4 pixel MV / MVD precision. iii. In one example, such an indication may be signaled only if PocDist0 is equal to PocDist1, Ref0 is the previous image, and Ref1 is the image after the current image in the display order. iv. In one example, such an indication may be signaled in display order only if PocDist0 is equal to PocDist1, Ref0 is a successor image, and Ref1 is a predecessor image of the current image. 10. Whether all-zero MVD is allowed may depend on the coding information of the block. For example, the coding information may include SMVD mode information, motion information of the block. In one example, in SMVD mode, all-zero MVD may not be allowed. i. For example, if the horizontal MVD is zero in SMVD mode, instead of directly encoding the vertical MVD, it may be decreased by one (e.g., if the vertical MVD is positive) or increased by one (e.g., if the vertical MVD is negative) before encoding, or may be increased by one (if the vertical MVD is positive) or decreased by one (if the vertical MVD is negative) after decoding. ii. For example, if the vertical MVD is zero in SMVD mode, instead of directly encoding the horizontal MVD, it may be decremented by one (e.g., if the horizontal MVD is positive) or incremented by one (e.g., if the horizontal MVD is negative) before encoding, and may be incremented by one (if the horizontal MVD is positive) or decremented by one (if the horizontal MVD is negative) after decoding. b. In one example, in non-SMVD mode, all-zero MVD may not be allowed. i. For example, in non-SMVD modes, if the reference image for the block is the same as the derived reference image used in the SMVD mode block, then all-zero MVD may not be allowed. 1. For example, in non-SMVD mode, if the first K-1 MVD components are zero, then there are K MVD components in total. Then, instead of directly encoding the Kth MVD component, it may be decremented by 1 (e.g., if it is positive) or incremented by 1 (e.g., if it is negative) before encoding, and incremented by 1 (e.g., if it is positive) or decremented by 1 (e.g., if it is negative) after decoding. 11. Instead of finding the closest reference image (forward or backward), it is proposed to give a higher priority to finding two target reference images that have the same distance from the current image. a. There are N sets of reference images in List 0 and List 1, (RefL0 i ,RefL1 i ), where i is [0,N-1], one of the N pairs may be selected as the two reference images to be used for SMVD. i. Additionally, alternatively, for one pair, one reference image is from list 0 and the other reference image is from list 1. ii. Additionally, alternatively, for one pair, one reference image is a forward reference image from list 0 and the other reference image is a backward reference image from list 1. iii. Additionally, alternatively, for one pair, one reference image is a backward reference image from list 0 and the other reference image is a forward reference image from list 1. iv. Additionally, alternatively, for a pair, one reference image is a backward reference image and the other reference image is a forward reference image, both from the same reference image list. b. In List 0 and List 1, there are N sets of reference images, (RefL0 i ,RefL1 i ), where i is [0,N-1], one of the N pairs with the smallest or largest absolute POC distance between it and the current image may be selected as the two-reference image of interest to be used for SMVD. c. If there is no such set in the reference image list, a target reference image may be selected according to its POC distance with the current image, for example using the current method or the method described above. 12. If the two target reference images are at unequal distances from the current image (i.e., currPocDiffL0!=currPocDiffL1), it is proposed to disable SMVD. a. Additionally or alternatively, syntax elements related to SMVD (e.g.,
[0096] [ka]
[0097] ) signaling may be skipped and it is assumed that SMVD is not used. 13. An encoding method for deriving an MVD of reference picture list X from a signaled MVD of reference picture list Y (Y is 1-X) according to POC values of two reference pictures and a current picture. Given that the signaled MVD of reference picture list Y is MvdLY with horizontal and vertical components denoted by MvdLY[0] and MvdLY[1], the MVD of reference picture list X denoted by (MvdLX[0], MvdLX[1]) may be derived in the following manner: In one example, MvdLX[0]=MvdLY[0]*(POC(targetRef in List X)-POC(current picture)) / (POC(targetRef in List Y)-POC(current picture)) b. In one example, MvdLX[1]=MvdLY[1]*(POC(targetRef in List X)-POC(current picture)) / (POC(targetRef in List Y)-POC(current picture)) c. In one example, the above method may be applied to SMVD mode, where instead of always using the opposite MVD of list 0 as the MVD of list 1, it is proposed to scale the MVD of list 0 according to the POC distance when deriving the MVD of list 1. d. Similarly, in SMVD, if MVD is signaled for list 1, then when deriving the MVD for list 0, it is suggested to scale the MVD for list 1 according to the POC distance. e. Alternatively, the above method is applicable to a symmetric MV coding mode, where the MV information of one reference picture list is not signaled and may be derived from the other reference picture list by scaling. 14. If the MVD of one reference picture list is derived from another associated with another reference picture list, the range restrictions of the derived MVD shall be met. In one example, the x and / or y components of a second MVD derived from a first MVD by the SMVD method must be within a predetermined range. b. In one example, the x and / or y components of the second MVD must be clipped or conditionally modified according to a predetermined range. i. For example, Mz = Clip3(-2 N ,2 N -1,Mz), where Mz represents the x or y component of the second MVD, and N is an integer such as 16, or 18. 1. For example, N may be 15 or 17. ii. For example, set Mz to Mz=Clip3(-2 N +1,2 N -1,Mz), where Mz represents the x or y component of the second MVD, and N is an integer such as 16, or 18. 1. For example, N may be 15 or 17. iii. For example, Mz is Mz=2 N Then, Mz=2 N where Mz represents the x or y component of the second MVD, and N is an integer such as 16, or 18. 1. For example, N may be 15 or 17. c. In one example, the x and / or y components of the second MVD should be within a predefined range in a conforming bitstream. i. In one example, the x and / or y components of the second MVD are within [-2 N ,2 N -1], where N is an integer such as 16, 18, etc. 1. For example, N may be 15 or 17. ii. In one example, the x and / or y components of the second MVD are within [-2 N +1,2 N -1], where N is an integer such as 16, 18, etc. 1. For example, N may be 15 or 17. d. In one example, the x and / or y components of the first MVD should be clipped or conditionally modified according to a first predetermined range such that the x and / or y components of a second MVD derived from the first MVD by the SMVD method are within a second predetermined range. i. For example, Mz = Clip3(-2 N +1,2 N -1,Mz), where Mz represents the x or y component of the first MVD, and N is an integer such as 16, or 18. 1. For example, N may be 15 or 17. ii. For example, Mz is Mz=-2 N Then, Mz=-2 N +1, where Mz represents the x or y component of the first MVD, and N is an integer such as 16, or 18. 1. For example, N may be 15 or 17. e. In one example, the x and / or y components of a first MVD should be within a first predetermined range in a conforming bitstream, such that the x and / or y components of a second MVD derived from the first MVD by the SMVD method are within a second predetermined range. i. In one example, the first predetermined range may be different from the predetermined range of the signaled MVD when the SMVD does not apply. ii. In one example, the x and / or y components of the first MVD are within [-2 N +1,2 N -1], where N is an integer such as 16, 18, etc. 1. For example, N may be 15 or 17. f. Alternatively, the signaled MVD may be denoted as [-2 N ,2 N -1], but [-2 N +1,2 N -1]. i. In one example, the MVD range for SMVD (or other tools that derive the MVD from a separate reference image list) may be set differently than the MVD for non-SMVD encoding tools. 1. For example, if the signaled MVD is [-2 N +SMVDFlag,2 N -1] and if SMVD is applied, SMVDFlag is 1; if SMVD is not applied, SMVDFlag is 0. g. In one example, clipping or constraints (eg, VVC or as described above) on the MVD may be performed before modifying the MVD according to the resolution of the MV. i. Alternatively, clipping or constraints (eg, VVC or as described above) may be performed after modifying the MVD according to the resolution of the MV. h. In one example, clipping or constraints in MVD (eg, in VVC or disclosed above) may depend on the resolution of the MV.
[0098] 5. Embodiment The bold face italics highlight the changes at the top of the VVC Working Draft, version 5. The deleted text includes
[0099] [ka]
[0100] It is marked with a .
[0101] 5.1 Example 1: How to use SMVD when mvd_l1_zero_flag is true
[0102] [Table 3] [Table 4] [Table 5] [Table 6]
[0103] 5.2 Selection of reference images used in SMVD in the second embodiment
[0104] 5.2.2 An example
[0105] This subsection describes one example that corresponds to bullet 4 of Chapter 4.
[0106] 8.3.5 Decoding Process for Symmetric Motion Vector Difference Reference Indices In this process, when sym_mvd_flag is 1 in a certain coding unit, RefIdxSymL0 and RefIdxSymL1 are output, which specify the reference image indexes in list 0 and list 1 for taking the symmetric motion vector difference. The variable RefIdxSymLX, where x is 0 and x is 1, is derived as follows: - The variable currPic specifies the current image. - RefIdxSymL0 is set to -1. For each index i, with i=0..NumRefIdxActive[0]-1, the following applies: - Set RefIdxSymL0 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[0][i])>0, - DiffPicOrderCnt(currPic,RefPicList[0][i]) <DiffPicOrderCnt(currPic,RefPicList[0][RefIdxSymL0])あるいはRefIdxSymL0は-1である。 - RefIdxSymL1 is set to -1. - For each index i, for i=0..NumRefIdxActive[1]-1, the following applies: - Set RefIdxSymL1 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[1][i])<0, - DiffPicOrderCnt(currPic, RefPicList[1][i]) > DiffPicOrderCnt(currPic, RefPicList[1][RefIdxSymL1]) or RefIdxSymL1 is -1. - RefIdxSymL0 is equal to -1 and
[0107]
Chemical formula
[0108] If RefIdxSymL1 is equal to -1, the following applies. - For each index i where i = 0..NumRefIdxActive[0] - 1, the following applies. - If all of the following conditions are true, set RefIdxSymL0 to i. - DiffPicOrderCnt(currPic, RefPicList[0][i]) < 0, - DiffPicOrderCnt(currPic, RefPicList[0][i]) > DiffPicOrderCnt(currPic, RefPicList[0][RefIdxSymL0]) or RefIdxSymL0 is equal to -1. - For each index i where i = 0..NumRefIdxActive[1] - 1, the following applies. - If all of the following conditions are true, set RefIdxSymL1 to i. - DiffPicOrderCnt(currPic, RefPicList[1][i]) > 0, - DiffPicOrderCnt(currPic, RefPicList[1][i]) < DiffPicOrderCnt(currPic, RefPicList[1][RefIdxSymL1]) or RefIdxSymL1 is equal to -1.
[0109] 5.2.2 An example This subsection gives one example that corresponds to bullet 5 of Chapter 4.
[0110] 8.3.5 Decoding Process for Symmetric Motion Vector Difference Reference Indices In this process, when sym_mvd_flag is 1 in a coding unit, RefIdxSymL0 and RefIdxSymL0 are output, which specify the reference image indexes in list 0 and list 1 for taking the symmetric motion vector difference. The variable RefIdxSymLX, where X is 0 and 1, is derived as follows: - The variable currPic specifies the current image. - RefIdxSymL0 is set to -1. For each index i, with i=0..NumRefIdxActive[0]-1, the following applies: - Set RefIdxSymL0 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[0][i])>0, - DiffPicOrderCnt(currPic,RefPicList[0][i]) <DiffPicOrderCnt(currPic,RefPicList[0][RefIdxSymL0])あるいはRefIdxSymL0は-1である。 - RefIdxSymL1 is set to -1. - For each index i, for i=0..NumRefIdxActive[1]-1, the following applies: - Set RefIdxSymL1 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[1][i])<0, - DiffPicOrderCnt(currPic,RefPicList[1][i]) > DiffPicOrderCnt(currPic,RefPicList[1][RefIdxSymL1]) or RefIdxSymL1 is -1. - If RefIdxSymL0 is -1 or RefIdxSymL1 is -1, the following applies.
[0111] [Chemical formula]
[0112] - For each index i in i = 0..NumRefIdxActive[0] - 1, the following applies. - If all of the following conditions are true, set RefIdxSymL0 to i. - DiffPicOrderCnt(currPic, RefPicList[0][i]) < 0, - DiffPicOrderCnt(currPic, RefPicList[0][i]) > DiffPicOrderCnt(currPic, RefPicList[0][RefIdxSymL0]) or RefIdxSymL0 is equal to -1. - For each index i in i = 0..NumRefIdxActive[1] - 1, the following applies. - If all of the following conditions are true, set RefIdxSymL1 to i. - DiffPicOrderCnt(currPic, RefPicList[1][i]) > 0, - DiffPicOrderCnt(currPic, RefPicList[1][i]) < DiffPicOrderCnt(currPic, RefPicList[1][RefIdxSymL1]) or RefIdxSymL1 is equal to -1.
[0113] 5.2.3 One example
[0114] In this subsection, one example corresponding to item 6 in the bullet points of Chapter 4 is described.
[0115] 8.3.5 Decoding process for symmetric motion vector difference reference index In this process, when sym_mvd_flag is 1 in a certain coding unit, RefIdxSymL0 and RefIdxSymL1 are output, which specify the reference image indexes in list 0 and list 1 for taking the symmetric motion vector difference. The variable RefIdxSymLX, where x is 0 and x is 1, is derived as follows: - The variable currPic specifies the current image. - RefIdxSymL0 is set to -1. - For each index i, i=0..NumRefIdxActive[0]-1, the following applies: - Set RefIdxSymL0 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[0][i])>0, - DiffPicOrderCnt(currPic,RefPicList[0][i]) <DiffPicOrderCnt(currPic,RefPicList[0][RefIdxSymL0])あるいはRefIdxSymL0は-1である。 - RefIdxSymL1 is set to -1. - For each index i, for i=0..NumRefIdxActive[1]-1, the following applies: - Set RefIdxSymL1 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[1][i])<0, - DiffPicOrderCnt(currPic,RefPicList[1][i]) > DiffPicOrderCnt(currPic,RefPicList[1][RefIdxSymL1]) or RefIdxSymL1 is -1. - if RefIdxSymL0 is -1 or RefIdxSymL1 is -1, the following applies: For each index i, with i=0..NumRefIdxActive[0]-1, the following applies: - Set RefIdxSymL0 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[0][i])<0, - DiffPicOrderCnt(currPic,RefPicList[0][i]) > DiffPicOrderCnt(currPic,RefPicList[0][RefIdxSymL0]) or RefIdxSymL0 is -1. For each index i, with i=0..NumRefIdxActive[1]-1, the following applies: - Set RefIdxSymL1 to i if all of the following conditions are true: - DiffPicOrderCnt(currPic,RefPicList[1][i])>0, - DiffPicOrderCnt(currPic,RefPicList[1][i]) <DiffPicOrderCnt(currPic,RefPicList[1][RefIdxSymL1])あるいはRefIdxSymL1は-1である。
[0116] [ka]
[0117] 5.3 Example #1 How to use SMVD when mvd_l1_zero_flag is true Changes to the Working Draft are in bold italics.
[0118] [ka]
[0119] specifies the sign of the motion vector component difference as follows: - If mvd_sign_flag[compIdx]=0, then the difference of the corresponding motion vector components has a positive value. Otherwise (mvd_sign_flag[compIdx] is equal to 1), the difference of the corresponding motion vector components has a negative value. If mvd_sign_flag[compIdx] is not present, it is inferred to be equal to 0. For compIdx=0..1, the motion vector difference lMvd[compIdx] is derived as follows: lMvd[compIdx]=abs_mvd_greater0_flag[compIdx]*(abs_mvd_minus2[compIdx]+2)*(1-2*mvd_sign_flag[compIdx]) (7-157) The value of lMvd[compIdx] is -2 15 From 2 15 It is assumed to be in the range of -1.
[0120] [ka]
[0121] FIG. 14 is a block diagram of a video processing device 1400. The device 1400 may be used to implement one or more of the methods described herein. The device 1400 may be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 1400 may include one or more processing units 1402, one or more memories 1404, and video processing hardware 1406. The one or more processors 1402 may be configured to implement one or more of the methods described herein. The memory(s) 1404 may be used to store data and codes used to implement the methods and techniques described herein. The video processing hardware 1406 may be used to implement the techniques described herein in a hardware circuit.
[0122] 15 is a flow chart illustrating an example method 1500 of video processing. Method 1500 includes determining (1502) to use a symmetric motion vector difference (SMVD) mode for converting between a current video block and a bitstream representation of the current video block, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference pictures from reference list 0 and reference list 1 of reference pictures. Method 1500 includes performing (1504) the conversion using the SMVD mode. The bitstream representation indicates disabling the use of motion vector difference (MVD) mode for one of reference list 0 or reference list 1.
[0123] The following exemplary list provides embodiments that can address the technical problems described herein, among others.
[0124] 1. A video processing method comprising: determining to use a symmetric motion vector difference (SMVD) mode for conversion between a current video block and a bitstream representation of the current video block, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference images from reference list 0 and reference list 1 of reference images; and performing the conversion using the SMVD mode, wherein the bitstream representation indicates disabling the use of motion vector difference (MVD) mode for one of reference list 0 or reference list 1.
[0125] 2. The method of Example 1, wherein the bitstream representation indicates that the MVD mode is not available for reference list 1, and the conversion includes deriving motion candidates corresponding to reference list 1 from motion candidates corresponding to reference list 0 using a relationship.
[0126] 3. The method of example 2, wherein the relationship specifies using the motion candidates corresponding to reference list 1 by mirroring the motion candidates corresponding to reference list 0.
[0127] Additional embodiments of Examples 1-3 are described in Section 1 of Chapter 4.
[0128] 4. A video processing method, comprising: determining to use a symmetric motion vector difference (SMVD) mode for conversion between a current video block and a bitstream representation of the current video block, the SMVD mode using two additional motion candidates generated for the conversion based on symmetric displacements of reference images from reference list 0 and reference list 1 of a reference block in an image from two target reference images; and performing the conversion using the SMVD mode, wherein the bitstream representation includes motion vector difference (MVD) information for reference list 1.
[0129] 5. The method of example 4, wherein the MVD information for the reference list 1 is a function of the video unit in which the current video block is located.
[0130] 6. The method of example 5, wherein the video unit is a slice or a tile group or a tile or a brick or a coding unit.
[0131] 7. The method of example 4, wherein the identity of reference list 1 depends on reference lists used for neighboring video blocks of the current video block.
[0132] 8. The method of example 4, wherein the reference list 1 is selected because the temporal distance to the current video block is less than the reference list 0.
[0133] 9. The method of example 4, wherein the reference list 1 is selected because its temporal distance to the current video block is greater than the reference list 0.
[0134] Additional embodiments of Examples 4-9 are described in Section 2 of Chapter 4.
[0135] 10. The method of any of Examples 1 to 9, wherein Reference List 0 and Reference List 1 are identical.
[0136] 11. The method of example 10, wherein one of Reference List 0 and Reference List 1 is used for forward prediction and the other is used for backward prediction.
[0137] Additional embodiments of Examples 10-11 are described in Section 3 of Chapter 4.
[0138] 12. A video processing method including: determining to use a symmetric motion vector difference (SMVD) mode for converting between a current video block and a bitstream representation of the current video block, the SMVD mode using two additional motion candidates generated for the conversion based on symmetric displacements of a reference block in an image from reference list 0 and a reference image from reference list 1; and performing the conversion using the SMVD mode, the SMVD generating the two additional motion candidates by parsing each reference image list using a first step to identify a target reference image.
[0139] 13. The method of Example 12, wherein the SMVD further optionally uses a second step after the first step to generate two additional motion candidates and identify a target reference image.
[0140] 14. The method of example 13, wherein the second step is selectable based on the number of reference images found in the first step.
[0141] Additional embodiments of Examples 12-14 are described in Section 4 of Chapter 4.
[0142] 15. A video processing method comprising: determining to use a symmetric motion vector difference (SMVD) mode for converting between a current video block and a bitstream representation of the current video block, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference pictures from reference list 0 and reference list 1 of reference pictures; and performing the conversion using the SMVD mode, in which the SMVD generates the two additional motion candidates such that both target reference picture indexes are found in either a first step including searching forward in list 0 for a target reference picture in list 0 and backward in list 1 for a target reference picture in list 1, or a second step including searching backward in list 0 for a target reference picture in list 0 and forward in list 1 for a target reference picture in list 1.
[0143] Additional embodiments of Example 15 are described in Chapter 4, Section 5.
[0144] 16. A video processing method comprising: determining to use a symmetric motion vector difference (SMVD) mode for converting between a current video block and a bitstream representation of the current video block, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference pictures from reference list 0 and reference list 1 of reference pictures; and performing the conversion using the SMVD mode, wherein a picture order count of one of the two target reference pictures is greater than a picture order count of the current video block, and a picture order count of another of the two target reference pictures is less than a picture order count of the current video block.
[0145] 17. The method of example 16, wherein the SMVD mode further includes verifying a picture order count of the current video block and a distance between the two target reference pictures.
[0146] Additional embodiments of Examples 16-17 are described in Section 6 of Chapter 4.
[0147] 18. A video processing method comprising: determining, based on a condition, whether to use a symmetric motion vector difference (SMVD) mode for conversion between a current video block and a bitstream representation of the current video block, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference pictures from reference list 0 and reference list 1 of reference pictures; and performing the conversion based on the determination.
[0148] 19. The method of example 18, wherein the condition includes encoding information of the current video block.
[0149] 20. The method of example 19, wherein the conditions include motion vector difference or prediction weights for bidirectional prediction of the current video block.
[0150] 21. The method of example 18, disabling SMVD mode if unequal weights are used for bi-prediction.
[0151] 22. The method of example 18, disabling SMVD mode on the condition that the motion vector precision is an integer value.
[0152] Additional embodiments of Examples 18-22 are described in Section 8 of Chapter 4.
[0153] 23. A video processing method comprising: determining, based on a condition, to use a symmetric motion vector difference (SMVD) mode in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference images from reference list 0 and reference list 1 of reference images, and a decoder-side motion vector difference (DMVD) coding mode for conversion between a current video block and a bitstream representation of the current video block; and performing the conversion based on the determining.
[0154] 24. The method of example 1, wherein the DMVD includes decoder-side motion vector refinement.
[0155] 25. A method according to any of Examples 23 to 24, wherein the conditions include using N pixel precision for calculating motion vectors and motion vector difference during the transformation, and the DMVD is fine-tuned with M pixel resolution, where N and M are integers or fractions.
[0156] 26. The method according to any of Examples 23 to 25, wherein N, M=1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8, or 16.
[0157] Additional embodiments of Examples 23-26 are described in Chapter 4, Section 9.
[0158] 27. A video processing method including: determining, based on a condition, whether to use an all-zero motion vector difference for a symmetric motion vector difference (SMVD) mode, in which two additional motion candidates are generated for the conversion based on symmetric displacements of reference blocks in two target reference pictures from reference list 0 and reference list 1 of reference pictures, and a decoder-side motion vector difference (DMVD) coding mode for conversion between a current video block and a bitstream representation of the current video block; and performing the conversion based on the determining.
[0159] 28. The method of example 27, wherein the condition prohibits using the all-zero motion vector differentials.
[0160] Additional embodiments of Examples 27-28 are described in Section 10 of Chapter 4.
[0161] 29. A video processing method comprising: selecting a first reference image and a second reference image from a plurality of reference images associated with a current video block; and converting between the current video block and a bitstream representation of the current video block based on the selected first and second reference images, wherein a first distance between the first reference image and a current image including the current video block and a second distance between the second reference image and the current image are equal.
[0162] 30. The method of example 29, wherein the first reference image is selected from list 0 and the second reference image is selected from list 1.
[0163] 31. The method of example 29, wherein the first reference image is a forward reference image from list 0 and the second reference image is a backward reference image from list 1.
[0164] 32. The method of example 29, wherein the first reference image is a backward reference image from list 0 and the second reference image is a forward reference image from list 1.
[0165] 33. The method of example 29, wherein the first reference image is a forward reference image, the second reference image is a backward reference image, and the first reference image and the second reference image are from the same list.
[0166] Additional embodiments of Examples 29-33 are described in Section 11 of Chapter 4.
[0167] 34. A video processing method comprising: determining that a first distance between a first reference image and a current image including a current video block and a second distance between a second reference image and the current image are not equal; and based on the determination, disabling symmetric motion vector difference (SMVD) mode for the current video block for conversion between the first reference image and a bitstream representation of the current video block.
[0168] 35. The method of example 34, wherein the bitstream representation excludes signaling of one or more SMVD-related syntax elements.
[0169] Additional embodiments of Examples 34-35 are described in Chapter 4, Section 12.
[0170] 36. A video processing method comprising: deriving a first motion vector difference (MVD) of a first reference image list for a current video block based on a second MVD of a second reference image list; and converting between the current video block and a bitstream representation of the current video block based on the derivation, wherein the range of the first MVD is limited.
[0171] 37. The method of example 36, wherein the deriving is based on a symmetric motion vector difference (SMVD) method.
[0172] 38. The range is [-2 N ,2 N -1], and N is a positive integer.
[0173] 39. The range is [-2 N +1,2 N -1], and N is a positive integer.
[0174] 40. The method of example 38 or 39, wherein N=16 or N=18.
[0175] Additional embodiments of Examples 36-40 are described in Chapter 4, Section 14.
[0176] 41. The method of any of examples 1-40, wherein the converting includes generating pixel values of the current video block from the bitstream representation.
[0177] 42. The method of any of examples 1-40, wherein the converting includes generating pixel values of the current video block from the bitstream representation.
[0178] 43. A video processing device comprising a processing device configured to implement one or more of Examples 1-42.
[0179] 44. A computer readable medium having stored thereon code, the code, when executed by a processor, causing the processor to implement a method according to one or more of Examples 1-42.
[0180] 16 is a flow chart illustrating an example of a method 1600 of video processing, which includes determining, at 1602, whether a Symmetric Motion Vector Difference (SMVD) mode is allowed for a conversion between a bitstream representation of a block of a current image of a video and a first block based on coding information of the block, and performing the conversion based thereon, at 1604.
[0181] In some examples, in SMVD mode, a motion vector difference (MVD) is signaled only for reference list Y where Y=1-X among two reference lists including reference list X and reference list Y, and is used to derive an MVD in reference list X where X=0 or 1.
[0182] In some examples, X=0.
[0183] In some examples, the encoding information includes a flag indicating whether the MVD in the reference list X is forced to zero, X=0 or 1.
[0184] In some instances, SMVD mode is allowed for this block even if the flag indicates that the MVD in reference list X is forced to zero.
[0185] In some examples, if the flag indicates that the MVD in Reference List 1 is forced to zero, then the MVD in Reference List 1 is forced to zero and SMVD mode is allowed for the block.
[0186] In some examples, if the flag indicates that the MVD in Reference List 1 is forced to zero, the MVD in Reference List 1 is derived from the associated MVD in Reference List 0 rather than being forced to zero, and SMVD mode is allowed for this block.
[0187] In some examples, if the flag indicates that the MVD in reference list X is forced to zero, then SMVD mode is not allowed for this block.
[0188] In some examples, if SMVD mode is not allowed, all syntax elements related to SMVD mode are implicitly derived instead of being signaled.
[0189] In some examples, the reference list Y for which the MVD needs to be signaled is signaled by an indication in the bitstream.
[0190] In some examples, the indication is signaled at at least one of the sequence, image, slice, tile, and video unit levels.
[0191] In some examples, this indication is signaled in at least one of the sequence parameter set (SPS) and picture parameter set (PPS), picture header, slice header, and tile group header.
[0192] In some examples, the reference list Y for which the MVD needs to be signaled is derived according to the reference pictures in the reference list.
[0193] In some examples, a reference list Y that needs to signal MVD is derived according to two reference images in two reference lists and corresponding two parameters Abs(currPocDiffL0) and Abs(currPocDiffL1), where Abs(currPocDiffL0) indicates the absolute value of the POC (Picture Order Count) distance from the current image to the reference image in reference list 0, and Abs(currPocDiffL1) indicates the absolute value of the POC distance from the current image to the reference image in reference list 1.
[0194] In some examples, if Abs(currPocDiffL0) is greater than Abs(currPocDiffL1), then the MVD for reference list 0 is signaled, otherwise the MVD for reference list 1 is signaled.
[0195] In some examples, if Abs(currPocDiffL0) is greater than Abs(currPocDiffL1), then the MVD for reference list 1 is signaled, otherwise the MVD for reference list 0 is signaled.
[0196] In some examples, if Abs(currPocDiffL0) is greater than or equal to Abs(currPocDiffL1), then the MVD for reference list 0 is signaled, otherwise the MVD for reference list 1 is signaled.
[0197] In some examples, if Abs(currPocDiffL0) is greater than or equal to Abs(currPocDiffL1), then the MVD for reference list 1 is signaled, otherwise the MVD for reference list 0 is signaled.
[0198] In some examples, if Abs(currPocDiffL0) is Abs(currPocDiffL1), then the MVD for reference list 0 is signaled, otherwise the MVD for reference list 1 is signaled.
[0199] In some examples, if Abs(currPocDiffL0) is Abs(currPocDiffL1), then the MVD for reference list 1 is signaled, otherwise the MVD for reference list 0 is signaled.
[0200] In some instances, the reference list Y for which the MVD needs to be signaled changes from one video unit to another.
[0201] In some examples, the video unit includes at least one of a slice, a tile group, a tile, a brick, and a coding unit (CU).
[0202] In some examples, the selection of the reference list Y depends on the reference image information of blocks neighboring said block.
[0203] In some examples, the selection of reference list Y depends on two reference images in two reference lists and two corresponding parameters PocDist0 and PocDist1, where PocDist0 is the absolute value of the picture order count (POC) distance from a reference image in reference list 0 to the current image, and PocDist1 is the absolute value of the POC distance from a reference image in reference list 1 to the current image.
[0204] In some examples, a reference list having a small absolute value of the POC distance between PocDist0 and PocDist1 is selected as reference list Y.
[0205] In some examples, a reference list having a large absolute value of the POC distance between PocDist0 and PocDist1 is selected as reference list Y.
[0206] In some examples, when PocDist0 and PocDist1 are equal, either reference list 0 or reference list 1 is selected as reference list Y.
[0207] In some examples, in SMVD mode, the two reference images are both selected from one and the same reference list.
[0208] In some examples, one of the two reference images is a forward reference image whose POC is smaller than the current POC, and the other is a backward reference image whose POC is larger than the current POC.
[0209] In some examples, all reference images in Reference List 0 and Reference List 1 are checked to find a first reference image whose POC is closest to the current POC among all reference images whose POC is smaller than the current POC, and a second reference image whose POC is closest to the current POC among all reference images whose POC is larger than the current POC.
[0210] In some examples, the SMVD mode includes a target reference image derivation process that includes a first step of searching for a forward target reference image in reference list 0, a second step of searching for a backward target reference image in reference list 1, and a second step of searching for the backward target reference image in reference list 0 and searching for the forward target reference image in reference list 1.
[0211] In some examples, if the first step finds at least one target reference image, the second step of the SMVD mode is skipped.
[0212] In some examples, if one target reference image is found in Reference List 0 and no target reference images are found in Reference List 1, the second step of the SMVD mode is skipped.
[0213] In some examples, if no target reference image is found in Reference List 0 and one target reference image is found in Reference List 1, the second step of the SMVD mode is skipped.
[0214] In some examples, if the first step does not find at least one target reference image, the SMVD mode is not allowed and signaling of all SMVD related syntax is skipped.
[0215] In some examples, the two target reference images found in the target reference image derivation process include a forward reference image with a smaller POC than the current image and a backward reference image with a larger POC than the current image.
[0216] In some examples, after the derivation process of the two target reference images, the POC of the two target reference images is checked, and if both of the two target reference images have a POC smaller than the current image or a POC larger than the current image, the target reference image is reset to be unavailable.
[0217] In some examples, the target reference image can be reset to be unavailable by resetting RefIdxSymL0 and RefIdxSymL1 to -1, where RefIdxSymL0 is the index of the target reference image in reference list 0 and RefIdxSymL1 is the index of the target reference image in reference list 0.
[0218] In some examples, if (RefIdxSymL1 !=-1 && RefIdxSymL0 !=-1 && DiffPicOrderCnt(currPic,RefPicList[0][RefIdxSymL1]) * DiffPicOrderCnt(currPic,RefPicList[1][RefIdxSymL1]) > 0) is true, the target reference image is reset to be unavailable and DiffPicOrderCnt(pic0,pic1) returns the POC distance from image pic0 to image pic1.
[0219] In some examples, the SMVD mode includes a target reference image derivation process to eliminate the limitation of finding a forward reference image from reference list 0 and finding a backward reference image from reference list 1, or finding a forward reference image from reference list 1 and finding a backward reference image from reference list 0.
[0220] In some examples, the target reference image used in the SMVD mode is derived from two dependent checks, including a first check and a second check, each of which is used for one of reference list 0 and reference list 1.
[0221] In some examples, the first check is to find one reference image in reference list X, where X is 0 or 1, and the second check is to find one reference image in reference list Y according to the output of reference list Y, the first check and the reference image in Y=1-X.
[0222] In some examples, the reference list X calculates the POC distance between each reference image and the current image according to a given order, regardless of whether it is forward or backward.
[0223] In some examples, the given order may be an ascending or descending order of reference image index.
[0224] In some examples, the reference image with the smallest POC distance is selected as the target reference image in reference list X, and the reference image is designated as RefInLX.
[0225] In some examples, the reference image with the largest POC distance is selected as the target reference image in reference list X, and the reference image is designated as RefInLX.
[0226] In some examples, select one reference image from reference list Y according to reference image RefInLX as the target reference image of reference list Y.
[0227] In some examples, if the POC distance of reference image RefInLX is small compared to the current image, the POC distance of the target reference image for reference list Y is large compared to the current image.
[0228] In some examples, if the POC distance of reference image RefInLX is large compared to the current image, the POC distance of the target reference image for reference list Y is small compared to the current image.
[0229] In some examples, when reference image RefInLX is a forward reference image, the target reference image for reference list Y is the one among all the backward reference images that has the smallest POC distance compared to the current image.
[0230] In some examples, when reference image RefInLX is a backward reference image, the target reference image for reference list Y is the one among all forward reference images that has the smallest POC distance compared to the current image.
[0231] In some examples, the coding information includes at least one of motion vector (MV) information, MVD precision information of the block, and weights of prediction samples in bidirectional prediction.
[0232] In some examples, if unequal weights are selected for bi-predictive blocks, the SMVD mode is not allowed.
[0233] In some examples, if SMVD mode is not allowed, all relevant syntax elements for SMVD mode are not signaled and / or are not implicitly derived.
[0234] In some instances, in SMVD mode, unequal weights are not allowed.
[0235] In some examples, if unequal weights are not allowed, then all syntax elements relating to unequal weights are not signaled and / or are not implicitly derived.
[0236] In some instances, the SMVD mode is not allowed for certain MVD accuracy and / or MV accuracy.
[0237] In some examples, SMVD mode is not allowed for N pixel MVD accuracy and / or MV accuracy, where N=1 or 4.
[0238] In some examples, in SMVD mode, only MVD accuracy and / or MV accuracy are allowed, and N=1 / 4.
[0239] In some examples, if SMVD mode is not allowed, all syntax elements related to SMVD mode are not signaled and are implicitly derived.
[0240] In some examples, in SMVD mode, adaptive motion vector difference resolution (AMVR) is not allowed.
[0241] In some examples, if AMVR is not enabled, all syntax elements related to AMVR are not signaled and are implicitly derived.
[0242] In some examples, in SMVD mode, decoder-side motion vector refinement (DMVR) or / and other decode-side motion vector derivation (DMVD) methods are enabled.
[0243] In some examples, in SMVD mode, when MV and / or MVD precision is N pixels, DMVR and / or other DMVD methods are used to fine-tune the MVD with parameters mvdDmvr, mvdDmvr with M pixel precision, where N, M = 1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8, or 16.
[0244] In some instances, M is less than or equal to N.
[0245] In some examples, in SMVD mode, the MVD is derived by the DMVR and / or other DMVD methods instead of being signaled.
[0246] In some examples, in SMVD mode, AMVR information is not signaled and the MV / MVD precision is derived to be a predefined value, where the predefined value is ¼ pixel precision.
[0247] In some examples, in SMVD mode, an indication of whether DMVR and / or other DMVD methods are applied is signaled.
[0248] In some instances, if DMVR or / and other DMVD methods are applied, MVD is not signaled.
[0249] In some examples, this indication is signaled for a particular MV and / or MVD accuracy.
[0250] In some examples, this indication is signaled for MV and / or MVD accuracy of 1 pixel or / and 4 pixels.
[0251] In some examples, this indication is signaled only when PocDist0 is equal to PocDist1, where PocDist0 is the POC distance between the current image and Ref0, PocDist1 is the POC distance between the current image and Ref1, where Ref0 is the previous image and Ref1 is the image that follows the current image in display order.
[0252] In some examples, this indication is signaled only when PocDist0 is equal to PocDist1, where PocDist0 is the POC distance between the current image and Ref0, PocDist1 is the POC distance between the current image and Ref1, Ref0 is the next image, and Ref1 is the image before the current image in display order.
[0253] In some examples, in an SMVD mode, whether all-zero MVD is allowed depends on the coding information of the block, which includes SMVD mode information and motion information of the block.
[0254] In some instances, in SMVD mode, all-zero MVD is not allowed.
[0255] In some examples, when the horizontal MVD is zero in SMVD mode, the vertical MVD is decreased by one if the vertical MVD is positive or increased by one if the vertical MVD is negative before encoding, and increased by one if the vertical MVD is positive or decreased by one if the vertical MVD is negative after decoding.
[0256] In some examples, when the vertical MVD is zero in SMVD mode, the horizontal MVD is decreased by one if the horizontal MVD is positive or increased by one if the horizontal MVD is negative before encoding, and increased by one if the horizontal MVD is positive or decreased by one if the horizontal MVD is negative after decoding.
[0257] In some examples, in SMVD mode, instead of finding the closest reference image, a preference is given to finding two target reference images that are equidistant to the current image.
[0258] In some examples, in Reference List 0 and Reference List 1, when i is [0, N-1], there are N sets of reference images represented by (RefL0i, RefL1i), and one set of the N sets is selected as the two reference images to be used in the SMVD mode, where N is an integer.
[0259] In some examples, for a set of reference images, one reference image is from reference list 0 and the other reference image is from reference list 1.
[0260] In some examples, for a set of reference pictures, one reference picture is a forward reference picture from reference list 0 and the other reference picture is a backward reference picture from reference list 1.
[0261] In some examples, for a set of reference pictures, one reference picture is a backward reference picture from reference list 0 and the other reference picture is a forward reference picture from reference list 1.
[0262] In some examples, for a set of reference pictures, one reference picture is a backward reference picture and the other reference picture is a forward reference picture, both from the same reference picture list.
[0263] In some examples, in Reference List 0 and Reference List 1, there are N sets of reference images represented by (RefL0i, RefL1i), where i is [0, N-1], and one set among the N sets having the minimum or maximum absolute value of POC distance to the current image is selected as the two reference images to be used in SMVD mode, where N is an integer.
[0264] In some examples, if there is no set of reference images equidistant to the current image in Reference List 0 and Reference List 1, a target reference image is selected based on its POC distance to the current image.
[0265] In some instances, if the two target reference images are not at equal distances from the current image, the SMVD mode is not allowed.
[0266] In some instances, signaling of SMVD related syntax elements, including sym_mvd_flag, is skipped and it is inferred that SMVD mode is not used.
[0267] In some examples, the MVD of reference image list X is derived from the signaled MVD of reference image list Y based on the POC distances between the two reference images and the current image.
[0268] In some examples, if the MVD signaled for reference list Y is MvdLY with its horizontal and vertical components represented by MvdLY[0] and MvdLY[1], then the MVD for reference list X represented by (MvdLX[0], MvdLX[1]) is derived as follows: MvdLX[0]=MvdLY[0]*(POC(targetRef in List X)-POC(current picture)) / (POC(targetRef in List Y)-POC(current picture)). MvdLX[1]=MvdLY[1]*(POC(targetRef in List X)-POC(current picture)) / (POC(targetRef in List Y)-POC(current picture)), Here, POC(targetRef in List X) is the POC of the reference picture in reference list X, POC(targetRef in List Y) is the POC of the reference picture in reference list Y, and POC(current picture) is the POC of the current picture.
[0269] In some examples, in SMVD mode, when deriving the MVD of reference list X, instead of using the inverse MVD of reference list Y as the MVD of reference list X, the MVD of reference list Y is scaled according to the POC distance, where X=1.
[0270] In some examples, in SMVD mode, when deriving the MVD of reference list X, instead of using the inverse MVD of reference list Y as the MVD of reference list X, the MVD of reference list Y is scaled according to the POC distance, where X=0.
[0271] 17 is a flow chart illustrating an example of a method 1700 of video processing. The method 1700 includes, at step 1702, determining whether a symmetric motion vector (SMV) mode is allowable for the block according to the method of the previous example for converting between a video block and a bitstream representation of a first block, and, at step 1704, performing the conversion based on the determination.
[0272] 18 is a flow chart showing an example of a method 1800 of video processing. In step 1802, the method 1800 includes determining whether a Symmetric Motion Vector Difference (SMVD) mode is acceptable for the block for conversion between the block of the image of the current video and a bitstream representation of the block, and whether the SMVD mode is acceptable for the block depends on a derivation process of two target reference images of the current image, which includes a first step of searching for a forward target reference image in reference image list 0 and searching for a backward target reference image in reference image list 1, and a second step of searching for a backward target reference image in reference image list 0 and searching for a forward target reference image in reference image list 1, and the outputs of the first step and the second step are independent of each other. In step 1804, the method includes performing the conversion based on the determination.
[0273] In some examples, whether the SMVD mode is allowed depends on whether both the target reference image is found in one of the first and second steps.
[0274] In some examples, if both target reference images are found in the first step, SMVD mode is allowed.
[0275] In some examples, if both target reference images are found in the second step, SMVD mode is allowed.
[0276] In some examples, if both target reference images are not found in the first step and both target reference images are not found in the second step, the SMVD mode is not allowed.
[0277] In some examples, if at least one target reference image of the two target reference images is not found in the first step, a second step is performed.
[0278] In some examples, before performing the second step, both of the target reference image indices are reset so that they are unavailable again.
[0279] In some examples, where the indices of two target reference images include RefIdxSymL0 and RefIdxSymL1 variables, RefIdxSymL0 represents the target reference image index in reference image list 0 and RefIdxSymL1 represents the target reference image index in reference image list 1.
[0280] In some examples, resetting both of the target reference image indices to unavailable includes resetting both RefIdxSymL0 and RefIdxSymL1 to −1.
[0281] In some examples, in SMVD mode, a motion vector difference (MVD) is signaled only for reference list Y of two reference lists including reference list X and reference list Y, where Y=1-X. The signaled MVD in reference list Y is used to derive the MVD in reference list X, where X=0 or 1.
[0282] 19 is a flow chart illustrating an example of a method 1900 of video processing. The method 1900 includes, at 1902, deriving a second motion vector difference (MVD) of one reference image list from a first MVD associated with another reference image list for conversion between a block of an image of a current video and a bitstream representation of the block, where the MVD includes both horizontal and vertical components, at 1904, applying range constraints to at least one of the first MVD and the second MVD, and at 1906, performing the conversion based on the determination.
[0283] In some examples, the first MVD is included in the bitstream representation of the block.
[0284] In some examples, the second MVD is derived from the first MVD in a symmetric motion vector difference (SMVD) mode.
[0285] In some examples, the horizontal and / or vertical components of the derived second MVD are limited to a predetermined range.
[0286] In some examples, the horizontal and / or vertical components of the derived second MVD are clipped or conditionally modified according to predetermined ranges.
[0287] In some examples, Mz is calculated as Mz=Clip3(-2 N ,2 N −1, Mz), where Mz represents the horizontal or vertical component of the derived second MVD, and N is an integer.
[0288]
number
[0289] In some examples, N=15, 16, 17 or 18.
[0290] In some examples, Mz is calculated as Mz=Clip3(-2 N +1,2 N −1,Mz), where Mz represents the horizontal or vertical component of the derived second MVD, and N is an integer.
[0291] In some examples, N=15, 16, 17 or 18.
[0292] In some cases, Mz=2 N If Mz is Mz=2 N -1, where Mz represents the horizontal or vertical component of the derived second MVD, and N is an integer.
[0293] In some examples, N=15, 16, 17 or 18.
[0294] In some examples, the horizontal and / or vertical components of the derived second MVD are within a predetermined range in the conforming bitstream.
[0295] In some examples, the horizontal and / or vertical components of the derived second MVD may be within [-2 N ,2 N -1].
[0296] In some examples, N=15, 16, 17 or 18.
[0297] In some examples, the horizontal and / or vertical components of the derived second MVD may be within [-2 N +1,2 N -1].
[0298] In some examples, N=15, 16, 17 or 18.
[0299] In some examples, the horizontal and / or vertical components of the first MVD are clipped or conditionally modified according to a first predetermined range such that the horizontal and / or vertical components of the derived second MVD are within a second predetermined range.
[0300] In some examples, Mz is calculated as Mz=Clip3(-2 N +1,2 N −1, Mz), where Mz represents the horizontal or vertical component of the first MVD, and N is an integer.
[0301]
number
[0302] In some examples, N=15, 16, 17 or 18.
[0303] In some cases, Mz=-2 N In the case of Mz=-2 N Modify by +1, where Mz represents the horizontal or vertical component of the first MVD, and N is an integer.
[0304] In some examples, N=15, 16, 17 or 18.
[0305] In some examples, the horizontal and / or vertical components of the first MVD are within a first predetermined range such that the horizontal and / or vertical components of the derived second MVD are within a second predetermined range in the conforming bitstream.
[0306] In some examples, the first predetermined range is different from the signaled MVD predetermined range in which the SMVD mode does not apply.
[0307] In some examples, the horizontal and / or vertical components of the first MVD may be within [-2 inclusive] in a conforming bitstream, where N is an integer. N +1,2 N-1].
[0308] In some examples, N=15, 16, 17 or 18.
[0309] In some examples, the horizontal and / or vertical components of the first MVD may be within [-2 inclusive] in a conforming bitstream, where N is an integer. N +1,2 N -1].
[0310] In some examples, the first MVD is a signaled MVD and the range of the signaled MVD is set differently for SMVD mode or other tools that derive the MVD from another reference image list and for non-SMVD encoding tools.
[0311] In some instances, the signaled MVD is [-2 N +SMVDFlag,2 N -1] and if SMVD mode applies, SMVDFlag is 1; if SMVD mode does not apply, SMVDFlag is 0.
[0312] In some examples, clipping or constraining is performed on the MVD before modifying it according to the precision of the MV associated with the MVD.
[0313] In some examples, clipping or constraints are performed on the MVD after modifying the MVD according to the precision of the MV associated with the MVD.
[0314] In some examples, the clipping or constraints on the MVD depend on the precision of the MV associated with the MVD.
[0315] In some examples, this conversion produces blocks of video from the bitstream representation.
[0316] In some examples, this conversion generates a bitstream representation from blocks of video.
[0317] In the list of examples herein, the term "convert" can refer to generating a bitstream representation for a current video block, or generating the current video block from a bitstream representation. The bitstream representation need not represent a contiguous group of bits, but may be divided into bits contained in a header field or codewords that represent encoded pixel value information.
[0318] In the above example, the applicability rules may be predefined and known to the encoder and decoder.
[0319] It will be appreciated that the disclosed techniques, as described herein, may be implemented in a video encoder or decoder to improve compression efficiency using techniques that include the use of various implementation rules for consideration regarding the use of differential encoding modes in intra-coding.
[0320] Implementations of the disclosed and other solutions, examples, embodiments, modules, and functional operations described herein, including the structures disclosed herein and their structural equivalents, may be implemented in digital electronic circuitry, or computer software, firmware, or hardware, or in one or more combinations thereof. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for implementation by or for controlling the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that provides a machine-readable propagated signal, or one or more combinations thereof. The term "data processing apparatus" includes all apparatus, devices, and machines for processing data, including, for example, a programmable processing apparatus, a computer, or multiple processing apparatuses, or computers. In addition to hardware, the apparatus may include code that creates an environment for the execution of the computer program, such as code that constitutes a processing apparatus firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. A propagated signal is an artificially generated signal, for example a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiving device.
[0321] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be recorded as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), may be stored in a single file dedicated to the program, or may be stored in multiple coordinating files (e.g., a file that stores one or more modules, subprograms, or code portions). A computer program can be deployed to run on one computer located at one site, or on multiple computers distributed across multiple sites and interconnected by a communication network.
[0322] The processes and logic flows described herein may be performed by one or more programmable processing devices executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and devices may be implemented as, special purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
[0323] Processors suitable for executing a computer program include, for example, both general purpose and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may include one or more mass storage devices, e.g., magnetic, magneto-optical, or optical disks, for storing data, or may be operatively coupled to receive data from or transfer data to these mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor storage devices such as EPROMs, EEPROMs, flash storage devices, magnetic disks, e.g., internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.
[0324] Although this patent specification contains many details, these should not be construed as limiting the scope of any subject matter or the scope of the claims, but rather as descriptions of features that may be specific to particular embodiments of a particular technology. Certain features described in this patent document in the context of separate embodiments may be implemented in combination in one example. Conversely, various features described in the context of one example may be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination and initially claimed as such, one or more features from a claimed combination may, in some cases, be extracted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.
[0325] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desired results, nor should the separation of various system modules in the examples described in the patent documents be understood as requiring such separation in all embodiments.
[0326] Only certain implementations and examples have been described; other embodiments, extensions and variations are possible based on the content described and illustrated in this patent document.
Claims
1. 1. A method for processing video data, comprising the steps of: For conversion between a block of a current image of a video and a bitstream of the video, determine that a first coding mode is applied to the block based on a derivation process of two target reference images of the current image; The derivation process includes: performing a first step of searching for a forward target reference image in a reference image list 0 and searching for a backward target reference image in a reference image list 1, where target reference image indexes in both the reference image list 0 and the reference image list 1 are initialized as unavailable; maintaining a target reference image index in one of the reference image list 0 and the reference image list 1 as unavailable because the forward target reference image or the backward target reference image is not found in one of the reference image list 0 and the reference image list 1 in the first step; resetting a target reference image index of the other of the reference image list 0 and the reference image list 1 to available because the forward target reference image or the backward target reference image was found in the other of the reference image list 0 and the reference image list 1 in the first step; resetting both indices of the target reference picture to unavailable before the second step is invoked because the target reference picture index in the one of the reference picture list 0 and the reference picture list 1 is unavailable; performing the second step of searching for a rear target reference image in the reference image list 0 and searching for a front target reference image in the reference image list 1; performing said conversion based on said determination; In the first coding mode, a syntax structure indicating a motion vector difference of a reference image list X is not present in the bitstream, and the motion vector difference of the reference image list X is derived based on the motion vector difference of a reference image list (1-X); X is equal to 0 or 1; method.
2. Resetting both of the target reference pictures to unavailable includes resetting both RefIdxSymL0 and RefIdxSymL1 to −1, where RefIdxSymL0 indicates an index of the target reference picture in the reference picture list 0 and RefIdxSymL1 indicates the target reference picture index in the reference picture list 1. The method of claim 1.
3. whether the first coding mode is acceptable or not is based on whether both of the target reference pictures are found in one of the first step and the second step; The method of claim 1.
4. If both of the target reference pictures are found in the first step, the first coding mode is enabled. The method according to claim 3.
5. If both of the target reference pictures are found in the second step, the first coding mode is enabled. The method according to claim 3.
6. if at least one of the two target reference pictures is not found in the first step and at least one of the two target reference pictures is not found in the second step, the first coding mode is not allowed. The method according to claim 3.
7. an image order of the forward target reference image is less than an image order of the current image, and an image order of the backward target reference image is greater than the image order of the current image; The method of claim 1.
8. the converting includes decoding blocks of the video from the bitstream. The method of claim 1.
9. the transforming includes encoding blocks of the video into the bitstream. The method of claim 1.
10. 1. An apparatus for processing video data comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to: For conversion between a block of a current image of a video and a bitstream of the video, determine that a first coding mode is applied to the block based on a derivation process of two target reference images of the current image; The derivation process includes: performing a first step of searching for a forward target reference image in a reference image list 0 and searching for a backward target reference image in a reference image list 1, where target reference image indexes in both the reference image list 0 and the reference image list 1 are initialized as unavailable; maintaining a target reference image index in one of the reference image list 0 and the reference image list 1 as unavailable because the forward target reference image or the backward target reference image is not found in one of the reference image list 0 and the reference image list 1 in the first step; resetting a target reference image index of the other of the reference image list 0 and the reference image list 1 to available because the forward target reference image or the backward target reference image was found in the other of the reference image list 0 and the reference image list 1 in the first step; resetting both indices of the target reference picture to unavailable before the second step is invoked because the target reference picture index in the one of the reference picture list 0 and the reference picture list 1 is unavailable; performing the second step of searching for a rear target reference image in the reference image list 0 and searching for a front target reference image in the reference image list 1; performing said conversion based on said determination; In the first coding mode, a syntax structure indicating a motion vector difference of a reference image list X is not present in the bitstream, and the motion vector difference of the reference image list X is derived based on the motion vector difference of a reference image list (1-X); X is equal to 0 or 1; Device.
11. A non-transitory computer readable storage medium storing instructions that cause a processor to: For conversion between a block of a current image of a video and a bitstream of the video, determine that a first coding mode is applied to the block based on a derivation process of two target reference images of the current image; The derivation process includes: performing a first step of searching for a forward target reference image in a reference image list 0 and searching for a backward target reference image in a reference image list 1, where target reference image indexes in both the reference image list 0 and the reference image list 1 are initialized as unavailable; maintaining a target reference image index in one of the reference image list 0 and the reference image list 1 as unavailable because the forward target reference image or the backward target reference image is not found in one of the reference image list 0 and the reference image list 1 in the first step; resetting a target reference image index of the other of the reference image list 0 and the reference image list 1 to available because the forward target reference image or the backward target reference image was found in the other of the reference image list 0 and the reference image list 1 in the first step; resetting both indices of the target reference picture to unavailable before the second step is invoked because the target reference picture index in the one of the reference picture list 0 and the reference picture list 1 is unavailable; performing the second step of searching for a rear target reference image in the reference image list 0 and searching for a front target reference image in the reference image list 1; performing said conversion based on said determination; In the first coding mode, a syntax structure indicating a motion vector difference of a reference image list X is not present in the bitstream, and the motion vector difference of the reference image list X is derived based on the motion vector difference of a reference image list (1-X); X is equal to 0 or 1; A computer-readable storage medium.
12. A method for storing a video bitstream, comprising the steps of: The method comprises: For a block of a current image of a video, determine that a first coding mode is applied to the block based on a derivation process of two target reference images of the current image; The derivation process includes: performing a first step of searching for a forward target reference image in a reference image list 0 and searching for a backward target reference image in a reference image list 1, where target reference image indexes in both the reference image list 0 and the reference image list 1 are initialized as unavailable; maintaining a target reference image index in one of the reference image list 0 and the reference image list 1 as unavailable because the forward target reference image or the backward target reference image is not found in one of the reference image list 0 and the reference image list 1 in the first step; resetting a target reference image index of the other of the reference image list 0 and the reference image list 1 to available because the forward target reference image or the backward target reference image was found in the other of the reference image list 0 and the reference image list 1 in the first step; resetting both indices of the target reference picture to unavailable before the second step is invoked because the target reference picture index in the one of the reference picture list 0 and the reference picture list 1 is unavailable; performing the second step of searching for a backward target reference image in the reference image list 0 and searching for a forward target reference image in the reference image list 1; generating the bitstream based on the determination; and storing the bitstream on a non-transitory computer readable recording medium; In the first coding mode, a syntax structure indicating a motion vector difference of a reference image list X is not present in the bitstream, and the motion vector difference of the reference image list X is derived based on the motion vector difference of a reference image list (1-X); X is equal to 0 or 1; method.
Citation Information
Patent Citations
JPP7303330B
