Method and device for pattern-based motion vector derivation for video coding

Abstract

The present invention discloses a video encoding method and apparatus based on bidirectional matching or template matching using decoder-derived motion information. According to one method, a decoder-side merge candidate for a current block using bidirectional matching, template matching, or both is derived. A merge candidate group including the decoder-side merge candidate is generated. A current merge index selected for the current block is signaled at the encoder side or decoded at the decoder side using one of at least two different codeword groups or one of at least two contexts encoded using a context basis, wherein the at least two different codeword groups or the at least two context-based encoded contexts are used to encode a merge index associated with a merge candidate in the merge candidate group.

Description

[0001] Description of the case

[0002] This application is a divisional application of the invention patent application with application number 201780018068.4, and the invention name is method and device for style-based motion vector derivation for video coding. Technical Field

[0003] The present invention relates to motion compensation for video coding using decoder-side derived motion information. More specifically, the present invention relates to improving the performance or reducing the complexity of pattern-based motion vector derivation. Background Art

[0004] In conventional video coding systems using motion-compensated inter-frame prediction (MCIP), motion information is typically sent from the encoder to the decoder so that the decoder can correctly perform MCIP. In such systems, motion information consumes some coding bits. To improve coding efficiency, a decoder-side motion vector derivation method was disclosed in VCEG-AZ07 (Jianle Chen et al., Further improvements to HMKTA-1.0, ITU-Telecommunications Standardization Sector, Study Group 16 Question 6, Video Coding Experts Group (VCEG), 52nd Meeting: June 19-26, 2015, Warsaw, Poland). According to VCEG-AZ07, the decoder-side motion vector derivation method uses two frame rate up-conversion (FRUC) modes. One of the FRUC modes is called B-slice bilateral matching, while the other is called template matching for P-slices or B-slices.

[0005] Figure 1An example of the FRUC bidirectional matching mode is shown, where the motion information of the current block 110 is derived based on two reference images. The motion information of the current block is derived by finding the best match between two blocks (120 and 130) in two different reference images (i.e., Ref0 and Ref1) along a motion trajectory 140 along the current block (Cur block). Under the assumption of continuous motion trajectories, the motion vector MV0 associated with Ref0 and the motion vector MV1 associated with Ref1 pointing to the two reference blocks should be proportional to the temporal distances (TD0 and TD1) between the current image (i.e., Cur pic) and the two reference images.

[0006] Figure 2 An example of FRUC template matching is shown. Neighboring areas (220a and 220b) of the current block 210 in the current image (i.e., Cur pic) are used as a template to match the corresponding templates (230a and 230b) in the reference image (i.e., Ref0). The best match between templates 220a / 220b and templates 230a / 230b determines a decoder derived motion vector 240. Although Ref0 is Figure 2 As shown, Ref1 can also be used as a reference image.

[0007] According to VCEG-AZ07, FRUC_mrg_flag is signaled when merge_flag or skip_flag is true. If FRUC_mrg_flag is 1, FRUC_merge_mode is signaled to indicate that bilateral matching merge mode or template matching merge mode is selected. If FRUC_mrg_flag is 0, this implies the use of regular merge mode, and in this case, a merge index is signaled. In video coding, to improve coding efficiency, the motion vector of a block can be predicted using motion vector prediction (MVP), which generates a candidate list. In merge mode, a merge candidate list can be used to encode a block. When merging a block, the motion information of the block (e.g., motion vector) can be represented by one of the candidate MVs in the merge MV list. Therefore, instead of directly transmitting the motion information of the block, a merge index is transmitted to the decoder. The decoder maintains the same merge table and uses the merge index to retrieve the merge candidate signaled by the merge index. Generally, the merge candidate list contains a small number of candidates and transmitting the merge index is more efficient than transmitting motion information. When a block is encoded in merge mode, its motion information is "merged" with the motion information of neighboring blocks by signaling a merge index, rather than actually transmitting the motion information. However, the prediction residuals are still transmitted. In the case where the prediction residual is zero or very small, the prediction residual is "skipped" (i.e., skip mode) and the block is encoded in skip mode using the merge index to identify the merge MV in the merge table.

[0008] While the term FRUC refers to Frame Rate Up-Conversion (FRUC), the technique is used by the decoder to derive one or more merged MV candidates without actually transmitting motion information. Therefore, FRUC is also referred to as decoder-derived motion information in this application. Because template matching is a pattern-based MV derivation technique, the template matching method for FRUC is also referred to as pattern-based MV derivation (PMVD) in this application.

[0009] At the decoder, a new temporal MVP derivation method, called temporally derived MVP, is derived by scanning all MVs in all reference frames. To derive the LIST_0 temporally derived MVP, for each LIST_0 MV in the LIST_0 reference frame, the MV is scaled to point to the current frame. The 4x4 block in the current frame pointed to by the scaled MV is the target current block. The MV is further scaled to point to the reference image with refIdx equal to 0 in LIST_0 for the target current block. The further scaled MV is stored in the LIST_0 MV field for the target current block. Figure 3A and Figure 3B An example of MVP derived in the time domain is shown for List_0 and List_1 respectively. Figure 3A and Figure 3B In the MVP, each small square block corresponds to a 4x4 block. The process of temporal derivation of MVP scans all MVs within the 4x4 blocks in all reference images to generate the temporal derivation of LIST_0 and LIST_1 MVP of the current frame. For example, Figure 3A In FIG, block 310, block 312 and block 314 correspond to a 4x4 block of the current image, a List_0 reference image with an index equal to 0 (i.e., refidx=0), and a List_0 reference image with an index equal to 1 (i.e., refidx=1), respectively. The motion vectors 320 and 330 within the two blocks of the List_0 reference image with an index equal to 1 are known. Then, the temporally derived MVPs 322 and 332 can be derived by scaling the motion vectors 320 and 330, respectively. The scaled MVP is then assigned to a corresponding block. Similarly, in Figure 3B In FIG, block 340, block 342, and block 344 correspond to a 4x4 block of the current image, a List_1 reference image with an index equal to 0 (i.e., refidx=0), and a List_1 reference image with an index equal to 1 (i.e., refidx=1), respectively. Motion vectors 350 and 360 within the two blocks of the List_0 reference image with an index equal to 1 are known. Then, temporally derived MVPs 352 and 362 can be derived by scaling motion vectors 350 and 360, respectively.

[0010] For the bidirectional matching merge mode and the template matching merge mode, two-stage matching is adopted. The first stage is PU-level matching, and the second stage is sub-PU-level matching. In the PU-level matching, multiple initial MVs are selected from LIST_0 and LIST_1 respectively. These MVs include MVs from merge candidates (i.e., traditional merge candidates, such as those specified in the HEVC standard) and MVs from time-domain derived MVPs. Two different staring MV groups are generated for the two tables. For each MV in a table, an MV pair is generated by synthesizing the MV and a mirrored MV, which is derived by scaling the MV to another table. For each MV pair, two reference blocks are compensated by using the MV pair. The sum of absolutely differences (SAD) of the two blocks is calculated. The MV pair with the smallest SAD is selected as the best MV pair.

[0011] After deriving the optimal MV for the PU, a diamond search is performed to refine the MV pair. The refinement accuracy is 1 / 8 pel. The search range for refinement is limited to ±1 pixel. The final MV pair is the MV pair derived at the PU level. Diamond search is a fast block-matching motion estimation algorithm well-known in the video coding community. Therefore, the details of the diamond search algorithm are not detailed here.

[0012] For the second stage sub-PU-level search, the current PU is split into sub-PUs. The depth of the sub-PU (e.g., 3) is signaled in the sequence parameter set (SPS). The minimum sub-PU size is a 4x4 block. For each sub-PU, multiple starting MVs are selected in LIST_0 and LIST_1, which include the PU-level derived MV, the zero MV, the HEVC collocated TMVP of the current sub-PU and the bottom-right block, the time-domain derived MVP of the current sub-PU, and the MVs of the left and above PUs / sub-PUs. The best M pair for the sub-PU is determined by using a similar mechanism as for the PU-level search. A diamond search is performed to refine the MV pair. Motion compensation is performed on the sub-PU to generate a predictor for the sub-PU.

[0013] For the template matching merge mode, the reconstructed pixels in the top 4 columns and the left 4 rows are used to form a template. Template matching is performed to find the best matching template and its corresponding MV. Template matching also uses two-stage matching. In PU-level matching, multiple starting MVs in LIST_0 and LIST_1 are selected respectively. These MVs include MVs from merge candidates (i.e., traditional merge candidates, such as those specified in the HEVC standard) and MVs from time-domain derived MVPs. Two different starting MVs are generated for the two tables. For each MV in a table, the SAD cost of the template with the MV is calculated. The MV with the minimum cost is the best MV. A diamond search is then performed to improve the MV. The improvement accuracy is 1 / 8-pel. The improvement search range is limited to ±1 pixel. The final MV is the PU-level derived MV. The MVs in LIST_0 and LIST_1 are generated independently.

[0014] For the second-stage sub-PU-level search, the current PU is split into sub-PUs. The depth of the sub-PU (e.g., 3) is signaled in the sequence parameter set (SPS). The minimum sub-PU size is a 4x4 block. For each sub-PU at the left or top PU boundary, multiple starting MVs are selected in LIST_0 and LIST_1, including the MVs of the PU-level derived MVs, the zero MV, the HEVC collocated TMVP of the current sub-PU and the bottom-right block, the time-domain derived MVP of the current sub-PU, and the MVs of the left and above PUs / sub-PUs. The best M pair for the sub-PU is determined using a similar mechanism as the PU-level search. A diamond search is performed to refine the MV pair. Motion compensation is performed on the sub-PU to generate a predictor for the sub-PU. Since these PUs are not at the left or top PU boundary, the second-stage sub-PU-level search is not performed, and the corresponding MVs are set equal to the MVs in the first stage.

[0015] In this decoder's MV derivation method, template matching is also used to generate the MVP for inter-mode coding. When a reference image is selected, template matching is performed to find the optimal template on the selected reference image. The corresponding MV is the derived MVP. This MVP is inserted into the first position of the AMVP. AMVP stands for Advanced MV Prediction, in which the current MV is predictively coded using a candidate list. The difference between the current MV and the MV candidate selected from the candidate list is encoded.

[0016] Bi-directional optical flow (BIO) was disclosed in JCTVC-C204 (Elena Alshina and Alexander Alshin, "Bi-directional optical flow," Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, Third Meeting: Guangzhou, China, October 7-15, 2010) and VECG-AZ05 (E. Alshina et al., "Known tools performance investigation for next generation video coding," ITU-T SG 16 Question 6, Video Coding Experts Group (VCEG), 52nd Meeting: June 19-26, 2015, Warsaw, Poland, Document: VCEG-AZ05). BIO uses optical flow and the steady motion assumption to achieve sample-level motion refinement. This is only used for true bidirectionally predicted blocks, which are predicted from two reference frames, one of which is the previous frame and the other is the subsequent frame. In VCEG-AZ05, BIO uses a 5x5 window to derive the motion improvement for each sample. Therefore, for an NxN block, the motion compensation results and corresponding gradient information of an (N+4)x(N+4) block are used to derive the sample-based motion improvement of the NxN block. According to VCEG-AZ05, a 6-tap gradient filter and a 6-tap interpolation filter are used to generate the gradient information for BIO. Therefore, the computational complexity of BIO is higher than that of traditional bidirectional prediction. To further improve the performance of BIO, the following method is proposed.

[0017] In a technical paper by Marpe et al. (D. Marpe, H. Schwarz, and T. Wiegand, "Context-Based Adaptive Binary Arithmetic Coding in the H.264 / AVC Video Compression Standard," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 620-636, July 2003), a multi-parameter probability update (context-adaptive binary coding) for HEVC CABAC was proposed. The parameter N = 1 / (1–α) is a measure of the number of previously coded bins and has a significant impact on the current update ("window size"). This value, in part, determines the average memory required by the system. Choosing the parameter that determines the model's sensitivity is a difficult and important issue. A sensitive system responds quickly to real-world changes. On the other hand, an insensitive model is insensitive to noise and random errors. Both parameters are useful but mutually restrictive. α can be varied during encoding by using different control signals. However, this method is very laborious and complicated. In this way, different values ​​of αi can be calculated at the same time:

[0018] pi_new = (1- αi) y+ α i pi_old. (1)

[0019] Weighted average is used to predict the probability of the next unit:

[0020] pnew = Σβi pi_new. (2)

[0021] In the above equation, βi is a weighting factor. In AVC CABAC, lookup tables (i.e., m_aucNextStateMPS and m_aucNextStateLPS) and an exponential mesh are used for probability updates. However, probability updates can also use a uniform mesh and an explicit calculation with a multiplication-free formula.

[0022] Assuming that the probability pi is represented by an integer from 0 to 2k, the probability is determined as follows:

[0023] pi=Pi / 2k.

[0024] If 'αi is the inverse of a quadratic number (i.e., αi = 1 / 2Mi), then we get the multiplication-free equation for the probability update:

[0025] Pi = (Y >> Mi) + P – (Pi >> Mi). (3)

[0026] In the above equation, “>>Mi” represents a right shift operation of Mi. The equation predicts the probability that the next unit will be “1”, where Y=2k if the last coding unit is “1” and Y=0 if the last coding unit is “0”.

[0027] To balance increased complexity with improved performance, a linear combination of probability estimates involving only two parameters is used:

[0028] P0 = (Y>>4) + P0 – (P0>>4) (4)

[0029] P1 = (Y>>7) + P1 – (P0>>7) (5)

[0030] P = (P0 + P1+1)>>1 (6)

[0031] For probability calculations in AVC CABAC, floating-point values ​​are always less than or equal to 1 / 2. If the probability exceeds this limit, the LPS (least probable symbol) is converted to the MPS (most probable symbol) to maintain the probability within the above interval. This concept has some significant advantages, such as reducing the size of the lookup table.

[0032] However, direct generalization of the above method for multi-parameter update models may encounter some difficulties. In practice, one probability estimate can exceed the limit while the other remains less than 1 / 2. Therefore, either MPS / LPS switching is required for each Pi or for some average value. In both cases, this introduces additional complexity without achieving a significant performance improvement. Therefore, this application proposes increasing the allowed probability level (in floating-point values) to a maximum of 1 and disabling MPS / LPS switching. This leads to the derivation of a lookup table (LUT) storing RangeOne or RangeZero. Summary of the Invention

[0033] In order to solve the above technical problems, the present application provides a new method and apparatus for video encoding using motion compensation.

[0034] In one method, a decoder-side merge candidate for a current block using bidirectional matching, template matching, or both is derived. A merge candidate set comprising the decoder-side merge candidate is generated. A selected current merge index for the current block is signaled at the encoder or decoded at the decoder using one of at least two different codeword sets or one of at least two contexts encoded using a context-based encoding, wherein the at least two different codeword sets or the at least two context-based encoded contexts are used to encode merge indices associated with merge candidates in the merge candidate set. A codeword set is selected based on the picture type of a picture containing the current block. The at least two different codeword sets belong to a code set comprising fixed-length codes, unary codes, and truncated unary codes. In one embodiment, a first context for a first merge index corresponding to a bidirectional matching or template matching merge mode is different from a second context for a second merge index corresponding to a normal merge mode. In another embodiment, a fixed-length code is used for the current block belonging to a low-delay B / P picture or a P / B picture whose reference frames have a picture sequence number less than that of the current picture.

[0035] The method and apparatus for video encoding using motion compensation provided by the present invention can reduce indexing costs, thereby improving encoding and decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Shows an example of motion compensation using the bidirectional matching technique, where the current block is predicted by two reference blocks along the motion track.

[0037] Figure 2 Shows an example of motion compensation using template matching techniques, where the template of the current block is matched with a reference template in a reference image.

[0038] Figure 3A An example of the temporal motion vector prediction (MVP) derivation process for the LIST_0 reference picture is shown.

[0039] Figure 3B An example showing the temporal motion vector prediction (MVP) derivation process for the LIST_1 reference picture.

[0040] Figure 4 A flowchart is shown of a video coding system using decoder-side derived motion information according to an embodiment of the present invention, where merge indices are signaled using different codewords.

[0041] Figure 5A flowchart of a video coding system using decoder-side derived motion information according to an embodiment of the present invention is shown, wherein a first-stage MV or a first-stage MV pair is used as the only initial MV or MV pair for the second-stage search or the center MV of the search window.

[0042] Figure 6 A flowchart of a video coding system using decoder-side derived motion information according to an embodiment of the present invention is shown, wherein after finding a reference template in a first reference table, the current template is modified for template search in another reference table.

[0043] Figure 7 A flowchart of a video coding system using decoder-side derived motion information according to an embodiment of the present invention is shown, wherein sub-PU search is disabled in template search.

[0044] Figure 8 An example of pixels within a current block and a reference block showing how differences are calculated between the current block and the reference block.

[0045] Figure 9 An example of calculating differences between pixels in a current block and a reference block according to an embodiment of the present invention is shown, wherein subblocks with virtual pixel values ​​are used to reduce the operation of calculating the differences.

[0046] Figure 10 A flowchart of a video encoding system using decoder-side derived motion information according to an embodiment of the present invention is shown, wherein a decoder-side MV or a decoder-side MV pair is derived according to a decoder-side MV derivation process, wherein the decoder-side MV derivation process uses a block difference calculation based on a reduced bit depth in an MV derivation associated with the decoder-side MV derivation process. DETAILED DESCRIPTION

[0047] The following description is the best way to implement the present invention. The description is for the purpose of illustrating the basic principles of the present invention and should not be interpreted as limiting. The scope of the present invention is best determined by reference to the appended claims.

[0048] In the present invention, multiple methods are disclosed to reduce bandwidth or complexity or improve coding efficiency for decoder-side motion vector derivation.

[0049] Signaling Merge Index with Different Codewords

[0050] In the bidirectional matching merge mode and the template matching merge mode, the LIST_0 and LIST_1 MVs in the merge candidate are used as the starting MVs. The best MV is undoubtedly derived by searching all these MVs. These merge modes will result in high memory bandwidth. Therefore, the present invention discloses a method to signal the merge index (merge index) of the bidirectional matching merge mode or the template matching merge mode. If the merge index is signaled, the best starting MV in LIST_0 and LIST_1 is known. Bidirectional matching or template matching only needs to perform an improved search around the signaled merge candidate. For bidirectional matching, if the merge match is a uni-directional MV, the corresponding MV in the other table can be generated by using a mirrored (scaled) MV.

[0051] In another embodiment, by using a predetermined MV generation method, the starting MVs and / or MV pairs in LIST_0 and LIST_1 are known. The best starting MVs or best MV pairs in LIST_0 and / or LIST_1 are explicitly signaled to reduce bandwidth requirements.

[0052] Although bidirectional matching and template matching have been commonly used in a two-stage manner, the method of signaling a combined index using different codewords according to the present invention is not limited to a two-stage manner.

[0053] In another embodiment, when a merge index is signaled, the selected MV can also be used to exclude or select some candidates in the first stage, i.e., during PU-level matching. For example, some MVs in the candidate list that are far from the selected MV can be excluded. Alternatively, N MVs in the candidate list that are closest to the selected MV but in different reference frames can be selected.

[0054] In the above method, the codeword of the merge index can be a fixed-length (FL) code, a unary code, or a truncated unary (TU) code. The context of the merge index for the bidirectional matching and template matching merge modes can be different from that of the normal merge mode. A different context model set can be used. The codeword can be slice-type dependent or signaled in the slice header. For example, the TU code can be used in random-access (RA) pictures or for B-pictures when the picture order count (POC) of the reference frame is not always smaller than that of the current picture. The FL code can be used for low-latency B / P pictures or for P-pictures or B-pictures where the POC of the reference frame is not always smaller than that of the current picture.

[0055] Figure 4A flowchart of a time-domain coding system using decoder-side derived information according to this embodiment is shown, wherein merge indices are signaled using different codewords. The steps shown in the following flowchart or any subsequent flowchart, as well as other flowcharts in this disclosure, can be implemented as program code executable on one or more processors (e.g., one or more CPUs) at the encoder and / or decoder. The steps shown in the flowchart can also be implemented in hardware, such as one or more electronic devices or processors configured to execute the steps of the flowchart. According to the present method, in step 410, input data related to a current block in a current image is received at the encoder, or a video stream containing coded data related to the current block in the current image is received at the decoder. In step 420, a decoder-side merge candidate is derived for the current block using bidirectional matching, template matching, or both. In step 430, a merge candidate set is generated that includes the decoder-side merge candidate. Methods for generating a merge candidate set are well known in the art. Typically, it includes motion information of spatially and / or temporally neighboring blocks as merge candidates. The decoder-side merge candidate derived according to this embodiment is included in the merge candidate set. At step 440, a current merge index of the selected current block is signaled at an encoder side or decoded at a decoder side using one of at least two different codeword groups or one of at least two contexts encoded using a context basis. The at least two contexts encoded using the at least two different codeword groups or context basis are used to encode a merge index associated with a merge candidate of the merge candidate group.

[0056] No MV Cost, Sub-block Refined from Merge Candidate

[0057] In the bidirectional matching merge mode and the template matching merge mode, an initial MV is first derived from neighboring blocks and the template co-located block. In the pattern-based MV search, the MV cost (i.e., the MV difference multiplied by lambda, λ) is added to the prediction distortion. The method of the present invention limits the searched MV to a region around the initial MV. The MV cost is typically used at the encoder to reduce the bit overhead of the MVD (MV difference), as signaling the MVD consumes coding bits. However, motion vector derivation at the decoder is a decoder-side process that does not require additional side information. Therefore, in one embodiment, the MV cost can be eliminated.

[0058] In the bidirectional matching merge mode and the template matching merge mode, a two-stage MV search is employed. The best MV in the first search stage (CU / PU-level stage) serves as one of the initial MVs in the second search stage. The search window in the second search stage is centered around the initial MV in the second search stage. However, this requires memory bandwidth. To further reduce bandwidth requirements, the present invention discloses a method that uses the initial MV in the first search stage as the center MV in the search window for the second subblock search. This allows the first search window to be reused in the second search stage, eliminating the need for additional bandwidth.

[0059] In VCEG-AZ07, for sub-PU MV search in template and bidirectional matching, the MVs to the left and above the current PU are used as initial search candidates. In one embodiment, to reduce memory bandwidth, the second-stage sub-PU search uses only the best MV from the first stage as the initial MV for the second stage.

[0060] In another embodiment, in combination with the above-mentioned method of sending merge index, a search window of an explicit sending merge index MV is used in the first-stage and second-stage searches.

[0061] Figure 5 A flowchart of a video coding system using decoder-derived motion information according to one embodiment of the present invention is shown, wherein a first-stage MV or first-stage MV pair is used as the sole initial MV or MV pair for a second-stage search or as the center MV of a search window. According to this method, in step 510, input data is received related to a current block in a current image, where each current block is divided into multiple subblocks. At the encoder, the input data may correspond to pixel data to be encoded and the input data may correspond to encoded data to be decoded at the decoder. In step 520, a first-stage MV or first-stage MV pair is derived based on one or more first-stage MV candidates using bidirectional matching, template matching, or both. In step 530, second-stage MVs are derived for multiple subblocks by deriving one or more second-stage MVs for each subblock using bidirectional matching, template matching, or both, where the first-stage MV or first-stage MV pair serves as the sole initial MV or MV pair for a second-stage search or serves as the center MV of a search window. In step 540, a final MV or final MVP is derived from a set of MV candidates or MVP candidates, including the second-stage MV. In step 550 , the current block or the current MV of the current block is encoded or decoded using the final MV or the final MVP at the encoder or decoder, respectively.

[0062] Disable Weighted Prediction for PMVD

[0063] In template matching merge mode and bidirectional matching merge mode, weighted prediction is disabled according to this method. If both LIST_0 and LIST_1 contain matching reference blocks, the weight is 1:1.

[0064] Matching Criterion

[0065] When the best or several best LIST_0 / LIST_1 templates are found, the templates in LIST_0 / LIST_1 can be used to search for templates in LIST_1 / LIST_0 (i.e., the templates in LIST_0 are used to search for templates in LIST_1, and vice versa). For example, the current template of List_0 can be modified to "2*(current template)-LIST_0 template", where the LIST_0 template corresponds to the best LIST_0 template. The new current template is used to search for the best template in LIST_1. The notation "2*(current template)-LIST_0 template" means a pixel-wise operation between the current template and the best template found in reference table 0 (i.e., LIST_0 template). Although traditional template matching wants to achieve the best match between the current template and the reference templates in reference table 0 and the best match between the current template and the reference templates in reference table 1, respectively. The modified current template of another reference table can help achieve the best match together. Repeated searches can be used. For example, after finding the best LIST_1 template, the current template can be modified to "2*(current template) - LIST_1 template". The modified new current template is used to search for the best template in LIST_0 again. The number of repetitions and the first target reference table should be defined in the standard.

[0066] The proposed matching criteria for LIST_1 can be slice-type dependent. For example, when the image sequence numbers of the reference frames are not all smaller than the current image, "2*(current template) - LIST_0 template" can be used for random-access (RA) pictures or B-pictures, and "current template" can be used for other types of pictures; and vice versa.

[0067] Figure 6A flow chart of a video encoding system using decoder-side derived motion information according to the present invention is shown, wherein after finding a reference template in a first reference table, a current template is modified for use in template searches in other reference tables. According to the method, in step 610, input data related to a current block or subblock in a current image is received. At the encoder side, the input data may correspond to pixel data to be encoded, and the input data may correspond to encoded data to be decoded at the decoder side. In step 620, one or more first-best templates of the current template of the current block or subblock are derived using template matching, which are pointed to by one or more first-best MVs in the first reference table, wherein the one or more first-best MVs are derived based on template matching. In step 630, after deriving the one or more first-best templates, a new current template is determined based on the current template, the one or more first-best templates, or both. In step 640, template matching is used to derive one or more second-best templates for a new current template of the current block or subblock, which are pointed to by one or more second-best MVs in a second reference table, wherein the one or more second-best MVs are derived based on template matching, and the first reference table and the second reference table belong to a group including Table 0 and Table 1, and the first reference table is different from the second reference table. In step 650, one or more final MVs or final MVPs are determined from a set of MV candidates or MVP candidates, wherein the MV candidates or MVP candidates include one or more best MVs associated with the one or more first-best MVs and the one or more second-best MVs. In step 660, the current block or subblock or the current MV of the current block or subblock is encoded or decoded using the final MV or final MVP at the encoder or decoder, respectively.

[0068] Disable Sub-PU-Level Search for templateMatching

[0069] According to the method of the present invention, sub-PU search is disabled for template matching merge mode. Sub-PU search is only used for bidirectional matching merge. For template matching merge mode, since the entire PU / CU can have the same MV, BIO can be used for coding blocks in template matching merge mode. As mentioned above, BIO is used for truly bidirectional predicted blocks to improve motion vectors.

[0070] Figure 7A flowchart of a video coding system using decoder-derived motion information according to the present method is shown, wherein template search for sub-PU search is disabled. According to the present method, in step 710, input data is received related to a current block or sub-block in a current image. At the encoder, the input data may correspond to pixel data to be encoded, and at the decoder, the input data may correspond to encoded data to be decoded. In step 720, a first-stage MV or a first-stage MV pair is derived based on one or more MV candidates using bidirectional matching or template matching. In step 730, a check is performed to determine whether bidirectional matching or template matching is used. If bidirectional matching is used, steps 740 and 750 are performed. If template matching is used, step 760 is performed. In step 740, one or more second-stage MVs are derived for each sub-block based on the first-stage MV or first-stage MV pair using bidirectional matching to generate second-stage MVs for a plurality of sub-blocks into which the current block is divided. In step 750, a final MV or final MVP is determined from a set of MV candidates or MVP candidates, including the second-stage MV. In step 760, a final MV or final MVP is determined from a set of MV candidates or MVP candidates including the first-stage MV. In step 770, after the final MV or final MVP is determined, the current block or the current MV of the current block is encoded or decoded using the final MV or final MVP at the encoder or decoder, respectively.

[0071] Reduce the Operations of Block Matching

[0072] For MV derivation at the decoder side, the SAD cost of a template with multiple MVs is calculated to find the best MV at the decoder side. In order to reduce the number of SAD calculation operations, a method for approximating the SAD between the current block 810 and the reference block 820 is disclosed. In the SAD calculation of traditional block matching, for example, Figure 8 As shown, the squared differences between corresponding pixel pairs of the current block (8x8 block) and the reference block (8x8 block) are calculated and summed as shown in Equation (1) to obtain the final sum of the squared differences, where Ci,j and Ri,j represent pixels in the current block 810 and the reference block 820, respectively, and the width is equal to N and the height is equal to M.

[0073]

[0074] For speedup, the current block and the reference block are divided into sub-blocks of size KxL, where K and L can be any integers. Figure 9As shown, both the current block 910 and the reference block 920 are 8x8 blocks divided into 2x2 subblocks. Each subblock is then treated as a virtual pixel and represented using a virtual pixel value. The virtual pixel value can be the sum of the pixels within the subblock, the average of the pixels within the subblock, the dominant pixel value within the subblock, a pixel within the subblock, a preset pixel value, or any other method of calculating a value using pixels within the subblock. The SAD can be calculated as the sum of the absolute differences between the virtual pixels of the current block and the reference block. Alternatively, the sum of the squared differences (SSD) can be calculated as the sum of the squared differences between the virtual pixels of the current block and the reference block. Therefore, the per-pixel SAD or SSD is approximated by the SAD or SSD of the virtual pixels, which requires fewer operations (e.g., fewer multiplications).

[0075] Furthermore, to maintain similar search results, the present method also discloses a refinement search stage after finding M best matches using the SAD or SSD of virtual pixels, where M can be any positive integer. For each of the M best candidates, a per-pixel SAD or SSD can be calculated to find the final best matching block.

[0076] To reduce the complexity of SAD and SSD calculations, the method of the present invention calculates the first K-bit MSB (or truncated L-bit LSB) data. For example, for a 10-bit video input, it can use the 8-bit MSB to calculate the distortion of the current block and the reference block.

[0077] Figure 10 A flow chart of a video coding system using decoder-side derived motion information according to the present method is shown. According to the present method, at step 1010, input data associated with a current block or subblock in a current image is received. At the encoder side, the input data may correspond to pixel data to be encoded, and the input data may correspond to encoded data to be decoded at the decoder side. At step 1020, a decoder-side MV or a decoder-side MV pair is derived according to the decoder-side MV derivation process using a block difference calculation based on a reduced bit depth in an MV search associated with the decoder-side MV derivation process. At step 1030, a final MV or a final MVP is determined from a set of MV candidates or MVP candidates comprising the decoder-side MV or decoder-side MV pair. At step 1040, the current block or the current MV of the current block is encoded or decoded using the final MV or final MVP at the encoder side or the decoder side, respectively.

[0078] Range Derivation for Multi-Parameter CABAC

[0079] In multi-parameter CABAC, the method of the present invention uses the LPS table to derive the RangeOne or RangeZero for each probability state. The average RangeOne or RangeZero can be derived by averaging the RangeOnes or RangeZeros. The encoded RangeOne (RangeOne for coding, ROFC) and the encoded RangeZero (RangeZero for coding, RZFC) can be derived using equation (8):

[0080] RangeZero_0=(MPS_0==1)? RLPS_0:(range–RLPS_0);

[0081] RangeZero_1=(MPS_1==1)? RLPS_1:(range–RLPS_1);

[0082] ROFC=(2*range–RangeZero_0–RangeZero_1)>>1; or

[0083] ROFC = (2*range – RangeZero_0 – RangeZero_1)>>1; (8)

[0084] In CABAC, the present invention utilizes a "stand-alone" context model for some syntaxes. The probability or probability state of a stand-alone context can differ from that of other contexts. For example, the probability or probability transformation of a "stand-alone" context model can use a different mathematical model. In one embodiment, a context model with a fixed probability can be used for the stand-alone context. In another embodiment, a context model with a fixed probability range can be used for the stand-alone context.

[0085] The above flowchart is intended to illustrate an example of video encoding according to the present invention. Those skilled in the art may modify each step, rearrange the steps, split a step, or combine steps to implement the present invention without departing from the spirit of the present invention. In this disclosure, specific syntax and semantics are used to illustrate examples for implementing embodiments of the present invention. Those skilled in the art may implement the present invention by replacing the syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

[0086] The description presented above is intended to enable one of ordinary skill in the art to implement the present invention in the context of the specific application and its requirements as provided. Various variations of the described embodiments will be apparent to those skilled in the art, and the general principles defined therein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but is intended to be consistent with the widest scope of the principles and novel features disclosed herein. In the above detailed description, various specific details are described to provide a thorough understanding of the present invention. Furthermore, it should be understood by those skilled in the art that the present invention is capable of being implemented.

[0087] The embodiments described herein may be implemented using various hardware, software code, or a combination of both. For example, an embodiment of the present invention may be software code integrated into one or more circuits of a video compression chip or into the video compression software that performs the aforementioned process. An embodiment of the present invention may also be software code to be executed on a digital signal processor (DSP) to perform the aforementioned process. The present invention may also involve various functions performed by computer processors, digital signal processors, microprocessors, field programmable gate arrays (FPGAs), and the like. These processors may be configured to perform specific tasks in accordance with the present invention by executing machine-readable software or firmware code that defines the specific methods implemented by the present invention. The software or firmware code may be developed using various programming languages ​​and in various formats or styles. The software code may also be compiled for various target platforms. However, different code formats, styles, and software code languages, as well as other configurations of code, to perform tasks in accordance with the present invention do not depart from the spirit and scope of the present invention.

[0088] The present invention may be embodied in other forms without departing from its spirit or essential characteristics. The embodiments described above are to be considered in their entirety and are intended to illustrate, not to limit, the present invention. All variations that come within the meaning and range of equivalents of the claims are intended to be within the scope of the claims.

Claims

1. A video encoding or decoding method using motion compensation, characterized in that: The method includes: Receiving input data related to a current block in a current picture at an encoder, or receiving a video stream including encoded data related to the current block in the current picture at a decoder; Derives a decoder-side merge candidate for the current block using bidirectional matching, template matching, or both; generating a merge candidate group including the decoder-side merge candidate; as well as A signal is sent at the encoder end indicating a current merge index selected for the current block, or the current merge index selected for the current block is decoded at the decoder end using one of at least two different codeword groups or one of at least two context-based encoded contexts, wherein the at least two different codeword groups or the at least two context-based encoded contexts are used to encode a merge index associated with a merge candidate of the merge candidate group.

2. The method according to claim 1, wherein A codeword group is selected according to a picture type of a picture including the current block.

3. The method according to claim 1, wherein The at least two different codeword groups belong to a code group, and the code group includes a fixed-length code, a unary code, and a truncated unary code.

4. The method according to claim 1, wherein The first context of the first merge index corresponding to the bidirectional matching or template matching merge mode is different from the second context of the second merge index corresponding to the normal merge mode.

5. The method according to claim 1, wherein The fixed length code is used for the current block belonging to a low-delay B / P picture or a P / B picture whose reference frames have a picture sequence number smaller than that of the current picture.

6. A video encoding and decoding device using motion compensation, characterized in that: The device comprises one or more electronic circuits or processors for: Receiving input data related to a current block in a current picture at an encoder, or receiving a video stream including encoded data related to the current block in the current picture at a decoder; Derives a decoder-side merge candidate for the current block using bidirectional matching, template matching, or both; generating a merge candidate group including the decoder-side merge candidate; as well as A signal is sent at the encoder end indicating a current merge index selected for the current block, or the current merge index selected for the current block is decoded at the decoder end using one of at least two different codeword groups or one of at least two context-based encoded contexts, wherein the at least two different codeword groups or the at least two context-based encoded contexts are used to encode a merge index associated with a merge candidate of the merge candidate group.

Images