Apparatus and method for refining conditional decoder side motion vectors in video coding

Abstract

This provides a method for predicting the current image block within the current picture of a video. [Solution] The method includes the steps of determining whether a first temporal distance (TD0, etc.) is equal to a second temporal distance (TD1, etc.), wherein the first temporal distance is expressed in terms of the difference between the picture order count value of the current picture and the picture order count value of a first reference image, and the second temporal distance is expressed in terms of the difference between the picture order count value of a second reference image and the picture order count value of the current picture, and not performing a motion vector refinement (DMVR) procedure when it is determined that the first temporal distance (TD0) is not equal to the second temporal distance (TD1). Thus, the DMVR procedure is limited to image blocks that have equidistant references.

Description

Technical Field

[0001] Cross - Reference to Related Applications This patent application claims the priority of Indian Provisional Patent Application No. IN201831034607, filed on September 13, 2018. The aforementioned patent application is hereby incorporated herein by reference in its entirety.

[0002] Embodiments of the present disclosure generally relate to techniques for encoding and decoding video data, and more particularly to decoder - side motion vector refinement (DMVR) in video coding.

Background Art

[0003] The amount of video data required to render even relatively short videos can be quite large, which can, as a result, become difficult when the data is streamed or transmitted in other ways across a communication network with limited bandwidth capacity. Thus, video data is generally compressed before being transmitted across modern telecommunications networks. The size of the video can also be a problem when the video is stored in a memory device since memory resources can be limited. Video compression devices often use software and / or hardware at the source to encode the video data before transmission or storage, thereby reducing the amount of data necessary to represent the digital video image. The compressed data is then received at the destination by a video decompression device that decodes the video data. Due to limited network resources and the ever - increasing demand for higher video quality, improved compression and decompression techniques that improve the compression ratio without sacrificing much or any of the image quality are desirable.

Summary of the Invention

Means for Solving the Problems

[0004] Embodiments of this application provide an apparatus and method for interpretation of the current image block within the current picture of a video, an encoder and decoder capable of conditionally performing decoder-side motion vector refinement (DMVR), and thus enabling improved coding efficiency.

[0005] Embodiments of the invention are defined by the features of the independent claims, and further advantageous implementations of the embodiments are defined by the features of the dependent claims.

[0006] Certain embodiments, along with other embodiments in the dependent claims, are outlined in the appended independent claims.

[0007] According to the first aspect, the disclosure relates to a method for interpretation (biprediction) of the current image block in the current picture of a video, wherein the method is A step of determining whether the current picture is temporally (with respect to time) between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and whether a first temporal distance (e.g., TD0) and a second temporal distance (e.g., TD1) are the same, wherein the first temporal distance (TD0) is between the current picture and the first reference picture (RefPic0), and the second temporal distance (TD1) is between the current picture and the second reference picture (RefPic1), and the first refined reference block The process includes the steps of performing a motion vector refinement (DMVR) procedure to obtain the position and the position of a second refined reference block, determining a predicted block (predicted sample value) for the current image block based on the position of the first refined reference block and the position of the second refined reference block, when it is determined that the current picture is temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and that the first temporal distance (TD0) and the second temporal distance (TD1) are the same.

[0008] It should be noted that the phrase "when it is determined that the current picture is temporally between a first reference picture (such as RefPic0) and a second reference picture (such as RefPic1), and that the first temporal distance (TD0) and the second temporal distance (TD1) are the same" should not be understood as "only when it is determined that the current picture is temporally between a first reference picture (such as RefPic0) and a second reference picture (such as RefPic1), and that the first temporal distance (TD0) and the second temporal distance (TD1) are the same." Other conditions may also be considered when deciding whether to perform the motion vector refinement (DMVR) procedure.

[0009] Regarding the "position of the first refined reference block and the position of the second refined reference block," the position can be an absolute position, which is the position within the reference picture, or a relative position, which is a position offset based on the position of the initial reference block.

[0010] It should be noted that DMVR applies to a merge mode of dual predictions using one motion vector (MV) from a past reference picture and another MV from a different future reference picture. The reference pictures may be two pictures that are in different temporal directions with respect to the current picture, which contains the current image block. This disclosure is not applicable to scenarios where both predictions originate from the same temporal direction (either both from the past or both from the future).

[0011] In a possible implementation of the method by any prior implementation of the first aspect, the step of determining whether the current picture is temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and whether a first temporal distance (e.g., TD0) and a second temporal distance (e.g., TD1) are the same, wherein the first temporal distance (TD0) is between the current picture and the first reference picture (RefPic0), and the second temporal distance (TD1) is between the current picture and the second reference picture (RefPic1), is: |TD0|==|TD1| and TD0*TD1<0 This includes being. For each merge candidate indicating bidirectional movement, TD0 and TD1 are calculated as the temporal distance from the current picture to the L0 and L1 reference pictures. TD0 and TD1 can be calculated using the Picture Order Count (POC). For example, TD0 = POCc - POC0 TD1 = POCc - POC1 Here, POCc, POC0, and POC1 represent the POC of the current picture, the POC of the first reference picture, and the POC of the second reference picture, respectively.

[0012] In any possible implementation of the method by any prior implementation of the first embodiment, or in the first embodiment itself, the method includes the steps of performing motion compensation using a first initial motion vector (MV0) and a second initial motion vector (MV1) when it is determined that a first temporal distance (TD0) and a second temporal distance (TD1) are different distances, or that the current picture is not temporally between a first reference picture (such as RefPic0) and a second reference picture (such as RefPic1); further including the step of performing motion compensation using a first initial motion vector (MV0) and a second initial motion vector (MV1) when, in one example, TD0 = POCc - POC0 and TD1 = POCc - POC1, |TD0| ≠ |TD1| or TD0 * TD1 >= 0. Alternatively, in another example, motion compensation can be performed using a first initial motion vector (MV0) and a second initial motion vector (MV1) when TD0 ≠ TD1, given that TD0 = POCc - POC0 and TD1 = POC1 - POCc.

[0013] In any possible implementation of the Method by any prior implementation of the First Embodiment, or in the First Embodiment itself, the Method further comprises the step of obtaining initial motion information for a current image block in a current picture, the initial motion information comprising a first initial motion vector, a first reference index, a second initial motion vector, and a second reference index, wherein the first reference index points to a first reference picture, and the second reference index points to a second reference picture.

[0014] In any possible implementation of the method by any prior implementation of the first embodiment, or in the first embodiment itself, the step of performing a motion vector refinement (DMVR) procedure is: A step of determining a first initial reference block in a first reference picture based on a first initial motion vector, The steps include determining a second initial reference block in a second reference picture based on a second initial motion vector, A step of generating a bilateral reference block based on a first initial reference block and a second initial reference block, wherein, for example, the bilateral reference block may be referred to as a bilateral template, and the template has the shape and size of an image prediction block, and The steps include: comparing the costs between a bilateral reference block and each of several first reference blocks in a first reference picture to determine the position of a first refined reference block or a first refined motion vector; The process includes the step of comparing the costs between a bilateral reference block and each of several second reference blocks in a second reference picture to determine the position of a second refined reference block or a second refined motion vector.

[0015] In any possible implementation of the method by any prior implementation of the first embodiment, or in the first embodiment itself, the step of performing a motion vector refinement (DMVR) procedure is: Steps include obtaining a template for the current image block based on a first initial reference block pointed to by a first initial motion vector in a first reference picture (e.g., RefPic0) and a second initial reference block pointed to by a second initial motion vector in a second reference picture (e.g., RefPic1), The method includes the step of determining a first refined motion vector and a second refined motion vector by template matching with the template in a first search space and a second search space, respectively, wherein the first search space includes a position given by a first initial motion vector and the second search space includes a position given by a second initial motion vector.

[0016] In any possible implementation of the method by any prior implementation of the first embodiment, or in the first embodiment itself, the step of performing a motion vector refinement (DMVR) procedure is: A step of determining the best matching block pair indicated by the best motion vector from a plurality of pairs of reference blocks based on a matching cost criterion (for example, based on the matching cost of each pair of reference blocks), wherein the pair of reference blocks includes a first reference block in a sample region determined based on a first initial motion vector of a first reference picture, and a second reference block in a sample region determined based on a second initial motion vector of a second reference picture. The best motion vector includes a step, which includes a first refined motion vector and a second refined motion vector.

[0017] In any possible implementation of the method by any prior implementation of the first embodiment, or in the first embodiment itself, the step of performing a motion vector refinement (DMVR) procedure to obtain the position of a first refined reference block and the position of a second refined reference block is: A step of determining the positions of N first reference blocks and N second reference blocks based on a first initial motion vector, a second initial motion vector, and the position of the current image block, wherein the N first reference blocks are contained within a first reference image, the N second reference blocks are contained within a second reference image, and N is an integer greater than 1. The step of determining the positions of pairs of reference blocks from the positions of M pairs of reference blocks based on a matching cost criterion, as the positions of a first refined reference block and the positions of a second refined reference block, wherein the position of each pair of reference blocks includes the position of the first reference block and the position of the second reference block, and for each pair of reference blocks, the first position offset (delta0x, delta0y) and the second position offset (delta1x, delta1y) are mirrored, the first position offset (delta0x, delta0y) represents the offset of the position of the first reference block relative to the position of the first initial reference block, and the second position offset (delta1x, delta1y) represents the offset of the position of the second reference block relative to the position of the second initial reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N.

[0018] In one example, the expression that the first position offset (delta0x, delta0y) and the second position offset (delta1x, delta1y) are mirrored can be understood as meaning that the direction of the first position offset is opposite to the direction of the second position offset, and the magnitude of the first position offset is the same as the magnitude of the second position offset.

[0019] In any possible implementation of the method by any prior implementation of the first embodiment, or in the first embodiment itself, the step of determining a predicted block of the current image block based on the position of a first refined reference block and the position of a second refined reference block is: Determining a prediction block based on a first refined reference block and a second refined reference block, wherein the first refined reference block is determined within a first reference picture based on the position of the first refined reference block, and the second refined reference block is determined within a second reference picture based on the position of the second refined reference block, or Determining a prediction block based on a first refined reference block and a second refined reference block, wherein the first refined reference block and the second refined reference block are determined by performing motion compensation using a first refined motion vector and a second refined motion vector.

[0020] In any preceding implementation of the first aspect or in the first aspect itself, the first reference picture is a reference image that temporally precedes the current picture, and the second reference picture is a reference image that is temporally preceded by the current picture, or the first reference picture is a reference image that is temporally preceded by the current picture, and the second reference picture is a reference image that temporally precedes the current picture, or The first reference picture is a past reference image and the second reference picture is a future reference image, or the first reference picture is a future reference image and the second reference picture is a past reference image. In other words, the picture before the current picture is the first reference picture and the picture after the current picture is the second reference picture, or the picture before the current picture is the second reference picture and the picture after the current picture is the first reference picture.

[0021] In any possible implementation of the method by any prior implementation of the first aspect, or in the first aspect itself, the first temporal distance (TD0) represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance (TD1) represents the POC distance between the current picture and the second reference picture, or The first temporal distance (TD0) is expressed as the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed as the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

[0022] In any possible implementation of the method by any prior implementation of the first aspect, or in the first aspect itself, the step of determining whether the current picture is temporally between a first reference picture (such as RefPic0) and a second reference picture (such as RefPic1) is: The process includes determining whether the picture order count (POCc) of the current picture is greater than the picture order count (POC0) of the first reference image and less than the picture order count (POC1) of the second reference image, or whether the picture order count (POCc) of the current picture is less than the picture order count (POC0) of the first reference image and greater than the picture order count (POC1) of the second reference image.

[0023] According to a second aspect, the disclosure relates to a method for interprediction (biprediction) of a current image block in a current picture of a video, the method comprising the steps of determining whether the current picture is temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and whether a first temporal distance (e.g., TD0) and a second temporal distance (e.g., TD1) are the same, wherein the first temporal distance (TD0) is the distance between the current picture and the first reference picture (RefPic0) The process includes the steps of: determining that the second temporal distance (TD1) is between the current picture and a second reference picture (RefPic1); and performing motion compensation using a first initial motion vector (MV0) and a second initial motion vector (MV1) when it is determined that the first temporal distance (TD0) and the second temporal distance (TD1) are different distances, or that the current picture is not temporally between a first reference picture (RefPic0, etc.) and a second reference picture (RefPic1, etc.).

[0024] In a possible implementation of the method by the second embodiment itself, the initial motion information of the current image block includes a first initial motion vector, a first reference index, a second initial motion vector, and a second reference index, wherein the first reference index indicates a first reference picture and the second reference index indicates a second reference picture.

[0025] In any possible implementation of the method by any prior implementation of the second aspect, or in the second aspect itself, the first temporal distance represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance represents the POC distance between the current picture and the second reference picture, or The first temporal distance (TD0) is expressed as the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed as the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

[0026] According to a third aspect, the disclosure relates to a method for interpretation (biprediction) of the current image block in the current picture of a video, wherein the method is The steps include determining whether a first temporal distance (such as TD0) is equal to a second temporal distance (such as TD1), wherein the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of a first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POC1) of a second reference image and the picture order count value (POCc) of the current picture; and, when it is determined that the first temporal distance (TD0) is equal to the second temporal distance (TD1), performing a motion vector refinement (DMVR) procedure to determine the predicted block of the current image block.

[0027] In a possible implementation of the method by its third aspect, the method further includes performing motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block for the current image block when it is determined that a first temporal distance is not equal to a second temporal distance.

[0028] According to a fourth aspect, the disclosure relates to a method for interpretation (biprediction) of the current image block in the current picture of a video, wherein the method is A step of determining whether a first temporal distance (such as TD0) is equal to a second temporal distance (such as TD1), wherein the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of a first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POC1) of a second reference image and the picture order count value (POCc) of the current picture. The procedure includes a step of not performing a motion vector refinement (DMVR) procedure (or disabling a motion vector refinement (DMVR) procedure) when it is determined that the first temporal distance (TD0) is not equal to the second temporal distance (TD1).

[0029] For each merge candidate indicating bidirectional merge, TD0 and TD1 can be calculated using the picture order count (POC). For example, TD0 = POCc - POC0 TD1 = POC1 - POCc Here, POCc, POC0, and POC1 represent the POC of the current picture, the POC of the first reference picture, and the POC of the second reference picture, respectively.

[0030] In any possible implementation of the method by any prior implementation of the second, third, or fourth aspect, or in the second, third, or fourth aspect itself, the first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or the first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or The first reference picture is a past reference image and the second reference picture is a future reference image, or the first reference picture is a future reference image and the second reference picture is a past reference image. In other words, the picture before the current picture is the first reference picture and the image after the current picture is the second reference picture, or the picture before the current picture is the second reference picture and the image after the current picture is the first reference picture.

[0031] According to the fifth aspect, the disclosure relates to a method for encoding video images, The steps include: performing an interpretation of the current image block in the current picture of the video according to the method described above to obtain the predicted block of the current image block; A bitstream comprising the steps of encoding the difference between a current image block and a predicted block (e.g., residual or residual block), and generating a bitstream containing the encoded difference and an index (such as a merge candidate index) for indicating initial motion information, wherein the initial motion information includes a first initial motion vector and a second initial motion vector.

[0032] According to the sixth aspect, the disclosure relates to a method for decoding a video image from a bitstream, A step of analyzing a bitstream for an index (such as a merge candidate index) to indicate initial motion information and an encoded difference (e.g., residual or residual block) between the current image block and a predicted block of the current image block, wherein the initial motion information includes a first initial motion vector and a second initial motion vector. The steps include: performing an interpretation of the current image block within the current picture of the video according to the method previously described above to obtain the predicted block of the current image block; This includes the step of reconstructing the current image block as the sum of the analyzed difference and the predicted block.

[0033] According to the seventh aspect, the disclosure relates to a method of encoding implemented by an encoding device, The steps include determining the value of a syntax element that indicates whether the above method is enabled or not, The process includes the step of generating a bitstream containing syntax elements.

[0034] In any possible implementation of the method by any prior implementation of the seventh aspect, or in the seventh aspect itself, syntax elements are signaled at one of the following levels: sequence parameter set (SPS) level, picture parameter set (PPS) level, slice header, coding tree unit (CTU) syntax, or coding unit (CU) syntax.

[0035] In any possible implementation of the method by any prior implementation of the seventh aspect, or in the seventh aspect itself, the syntax element includes a first flag (e.g., sps_conditional_dmvr_flag) and / or a second flag (e.g., pps_conditional_dmvr_flag), If the first flag (sps_conditional_dmvr_flag) is equal to 0, then the method previously described above is not performed on the image block of the sequence. If the first flag (sps_conditional_dmvr_flag) is equal to 1 and the second flag (pps_conditional_dmvr_flag) is equal to 0, then the method previously described above is not performed on the image blocks of the sequence's pictures, or If the first flag (sps_conditional_dmvr_flag) is equal to 1 and the second flag (pps_conditional_dmvr_flag) is equal to 1, the method previously described above is performed on the image blocks of the sequence's pictures.

[0036] According to the eighth aspect, the disclosure relates to a method of decoding implemented by a decoding device, The bitstream is analyzed for syntax elements that indicate whether the method described above is enabled or not, The method includes the step of adaptively enabling or disabling the decoder-side motion vector refinement (DMVR) procedure according to a syntax element indicating whether the above method is enabled or not.

[0037] In any possible implementation of the method by any prior implementation of the eighth aspect, or in the eighth aspect itself, syntax elements are obtained from one of the following: the bitstream sequence parameter set (SPS) level, the bitstream picture parameter set (PPS) level, a slice header, a coding tree unit (CTU) syntax, or a coding unit (CU) syntax.

[0038] In any possible implementation of the method by any prior implementation of the eighth aspect, or in the eighth aspect itself, the syntax element includes a first flag (sps_conditional_dmvr_flag) and a second flag (pps_conditional_dmvr_flag), If the first flag (sps_conditional_dmvr_flag) is equal to 0, then the method previously described above is not performed on the image block of the sequence. If the first flag (sps_conditional_dmvr_flag) is equal to 1 and the second flag (pps_conditional_dmvr_flag) is equal to 0, then the method previously described above is not performed on the image blocks of the sequence's pictures, or If the first flag (sps_conditional_dmvr_flag) is equal to 1 and the second flag (pps_conditional_dmvr_flag) is equal to 1, the method previously described above is performed on the image blocks of the sequence's pictures.

[0039] According to the ninth aspect, the disclosure relates to a coding device, Memory for storing instructions, A memory-coupled processor, the processor being configured to execute instructions stored in memory to cause the processor to perform the previously described method.

[0040] According to the tenth aspect, the disclosure relates to an apparatus for interpretation of the current image block in the current picture of a video, A decision unit configured to determine whether the current picture is temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and whether a first temporal distance (e.g., TD0) and a second temporal distance (e.g., TD1) are the same, wherein the first temporal distance (TD0) is between the current picture and the first reference picture (RefPic0), and the second temporal distance (TD1) is between the current picture and the second reference picture (RefPic1), The system includes an interprediction processing unit configured to determine a predicted block for the current image block based on the positions of the first and second refined reference blocks, when it is determined that the current picture is temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and that the first temporal distance (TD0) and the second temporal distance (TD1) are the same.

[0041] In any possible implementation of the method by any prior implementation of the 10th aspect, the decision unit is given by the following equation |TD0|==|TD1| and TD0*TD1<0 The system is configured to determine whether the current picture is temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and whether the first temporal distance (e.g., TD0) and the second temporal distance (e.g., TD1) are the same. For each merge candidate indicating bidirectional movement, TD0 and TD1 are calculated as the temporal distance from the current picture to the L0 and L1 reference pictures. TD0 and TD1 can be calculated using the Picture Order Count (POC). For example, TD0 = POCc - POC0 TD1 = POCc - POC1 Here, POCc, POC0, and POC1 represent the POC of the current picture, the POC of the first reference picture, and the POC of the second reference picture, respectively.

[0042] In a possible implementation of the method by its tenth aspect, the interpredictive processing unit is further configured to perform motion compensation using a first initial motion vector (MV0) and a second initial motion vector (MV1) when it is determined that a first temporal distance (TD0) and a second temporal distance (TD1) are different distances, or that the current picture is not temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1). In one example, the interpredictive processing unit is further configured to perform motion compensation using a first initial motion vector (MV0) and a second initial motion vector (MV1) when |TD0|≠|TD1| or TD0*TD1>=0, given that TD0=POCc-POC0 and TD1=POCc-POC1. Alternatively, in another example, the interpretation processing unit is further configured to perform motion compensation using a first initial motion vector (MV0) and a second initial motion vector (MV1) when TD0 ≠ TD1, in the cases TD0 = POCc - POC0 and TD1 = POC1 - POCc.

[0043] In any possible implementation of the method by any prior implementation of the tenth aspect, or in the tenth aspect itself, the interpredictive processing unit is: It is further configured to retrieve initial motion information for the current image block within the current picture, the initial motion information including a first initial motion vector, a first reference index, a second initial motion vector, and a second reference index, where the first reference index points to the first reference picture and the second reference index points to the second reference picture.

[0044] In any possible implementation of the method by any prior implementation of the 10th aspect, or in the 10th aspect itself, the interpredictive processing unit performs a motion vector refinement (DMVR) procedure to obtain the position of a first refined reference block and the position of a second refined reference block, specifically, Based on the first initial motion vector, determine the first initial reference block in the first reference picture. Based on the second initial motion vector, determine the second initial reference block in the second reference picture. A bilateral reference block is generated based on the first initial reference block and the second initial reference block. The cost between the bilateral reference block and each of the multiple first reference blocks in the first reference picture is compared to determine the position of the first refined reference block or the first refined motion vector. It is configured to determine the position of a second refined reference block or a second refined motion vector by comparing the costs between a bilateral reference block and each of several second reference blocks in a second reference picture.

[0045] In any possible implementation of the method by any prior implementation of the 10th aspect, or in the 10th aspect itself, the interpredictive processing unit performs a motion vector refinement (DMVR) procedure to obtain the position of a first refined reference block and the position of a second refined reference block, specifically, A template for the current image block is obtained based on a first initial reference block pointed to by a first initial motion vector in a first reference picture (e.g., RefPic0) and a second initial reference block pointed to by a second initial motion vector in a second reference picture (e.g., RefPic1). The system is configured to determine a first refined motion vector and a second refined motion vector by template matching with the template in the first and second search spaces, respectively, with the first search space being located at a position given by the first initial motion vector and the second search space being located at a position given by the second initial motion vector.

[0046] In any possible implementation of the method by any prior implementation of the 10th aspect, or in the 10th aspect itself, the interpredictive processing unit performs a motion vector refinement (DMVR) procedure to obtain the position of a first refined reference block and the position of a second refined reference block, specifically, The system is configured to determine the best matching block pair indicated by the best motion vector from a plurality of pairs of reference blocks based on a matching cost criterion (for example, based on the matching cost of each pair of reference blocks), wherein the pair of reference blocks includes a first reference block in a sample region determined based on a first initial motion vector of a first reference picture, and a second reference block in a sample region determined based on a second initial motion vector of a second reference picture. The best motion vector includes a first refined motion vector and a second refined motion vector.

[0047] In any possible implementation of the method by any prior implementation of the 10th aspect, or in the 10th aspect itself, the interpredictive processing unit performs a motion vector refinement (DMVR) procedure to obtain the position of a first refined reference block and the position of a second refined reference block, specifically, It is configured to determine the positions of N first reference blocks and N second reference blocks based on a first initial motion vector, a second initial motion vector, and the position of the current image block, wherein the N first reference blocks are contained within a first reference image and the N second reference blocks are contained within a second reference image, and N is an integer greater than 1. The interpretation processing unit is configured to determine the positions of pairs of reference blocks from the positions of M pairs of reference blocks based on a matching cost criterion, as the positions of a first refined reference block and a second refined reference block, where the position of each pair of reference blocks includes the position of the first reference block and the position of the second reference block, and for each pair of reference blocks, the first position offset (delta0x, delta0y) and the second position offset (delta1x, delta1y) are mirrored, where the first position offset (delta0x, delta0y) represents the offset of the position of the first reference block relative to the position of the first initial reference block, and the second position offset (delta1x, delta1y) represents the offset of the position of the second reference block relative to the position of the second initial reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N.

[0048] In any possible implementation of the method by any prior implementation of the 10th aspect, or in the 10th aspect itself, the interpredictive processing unit, in determining the predicted block of the current image block based on the position of the first refined reference block and the position of the second refined reference block, specifically, It is configured to determine a predicted block based on a first refined reference block and a second refined reference block, wherein the first refined reference block is determined in a first reference picture based on the position of the first refined reference block, and the second refined reference block is determined in a second reference picture based on the position of the second refined reference block, or The interpretation processing unit is specifically configured to determine a prediction block based on a first refined reference block and a second refined reference block, which are determined by performing motion compensation using the first refined motion vector and the second refined motion vector.

[0049] In any possible implementation of the method by any prior implementation of the 10th aspect, or in the 10th aspect itself, the first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or the first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or The first reference picture is a past reference image and the second reference picture is a future reference image, or the first reference picture is a future reference image and the second reference picture is a past reference image.

[0050] In any possible implementation of the method by any prior implementation of the 10th aspect, or in the 10th aspect itself, the first temporal distance (TD0) represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance (TD1) represents the POC distance between the current picture and the second reference picture, or The first temporal distance (TD0) is expressed as the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed as the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

[0051] In any possible implementation of the method by any prior implementation of the 10th aspect, or in the 10th aspect itself, the interpretation processing unit determines whether the current picture is temporally between a first reference picture (such as RefPic0) and a second reference picture (such as RefPic1), specifically: The system is configured to determine whether the picture order count value (POCc) of the current picture is greater than the picture order count value (POC0) of the first reference image and less than the picture order count value (POC1) of the second reference image, or whether the picture order count value (POCc) of the current picture is less than the picture order count value (POC0) of the first reference image and greater than the picture order count value (POC1) of the second reference image.

[0052] According to the eleventh aspect of the disclosure, the disclosure relates to an encoding device for encoding video images, the encoding device is A decision unit configured to determine whether the current picture is temporally between a first reference picture and a second reference picture, and whether the first temporal distance and the second temporal distance are the same, wherein the first temporal distance is between the current picture and the first reference picture, and the second temporal distance is between the current picture and the second reference picture. The system includes an interpredictive processing unit configured to perform motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block for the current image block when it is determined that the first temporal distance and the second temporal distance are different, or that the current picture is not temporally between the first reference picture and the second reference picture.

[0053] In a possible implementation of the method by the eleventh aspect itself, the initial motion information of the current image block includes a first initial motion vector, a first reference index, a second initial motion vector, and a second reference index, wherein the first reference index points to a first reference picture and the second reference index points to a second reference picture.

[0054] In any possible implementation of the method by any prior implementation of the 11th aspect, or in the 11th aspect itself, the first temporal distance represents the picture order count (POC) distance between the current picture and the first reference picture, the second temporal distance represents the POC distance between the current picture and the second reference picture, or the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

[0055] According to the twelfth aspect, the disclosure relates to an encoding device for encoding video images, the encoding device is The previously shown device for obtaining the predicted block of the current image block, An entropy coding unit configured to encode the difference (such as a residual) between a current image block and a predicted block of the current image block, and to generate a bitstream containing the encoded difference and an index (such as a merge candidate index) for indicating initial motion information, wherein the initial motion information includes a first initial motion vector and a second initial motion vector.

[0056] According to the 13th aspect of the disclosure, the disclosure relates to a decoding device for decoding a video image from a bitstream, the device is An entropy decoding unit configured to analyze from a bitstream an index for indicating initial motion information and an encoded difference (such as a residual) between the current image block and the predicted block of the current image block, wherein the initial motion information includes a first initial motion vector and a second initial motion vector, The previously shown device for obtaining the predicted block of the current image block, It includes an image reconstruction unit configured to reconstruct the current image block as the sum of the analyzed difference (such as residuals) and the predicted block.

[0057] According to the 14th aspect, the disclosure relates to an encoding device, It includes one or more processing circuits configured to determine the value of a syntax element indicating whether the previously described method is enabled or not, and to generate a bitstream containing the syntax element.

[0058] In possible implementations of the method by its 14th aspect, syntax elements are signaled at one of the following levels: sequence parameter set (SPS) level, picture parameter set (PPS) level, slice header, coding tree unit (CTU) syntax, or coding unit (CU) syntax.

[0059] In any possible implementation of the method by any prior implementation of the 14th aspect, or in the 14th aspect itself, the syntax element includes a first flag (sps_conditional_dmvr_flag) and a second flag (pps_conditional_dmvr_flag), If the first flag (sps_conditional_dmvr_flag) is equal to 0, the previously described method is not performed on the image block of the sequence. If the first flag (sps_conditional_dmvr_flag) is equal to 1 and the second flag (pps_conditional_dmvr_flag) is equal to 0, then the previously described method is not performed on the image blocks of the sequence's pictures, or If the first flag (sps_conditional_dmvr_flag) is equal to 1 and the second flag (pps_conditional_dmvr_flag) is equal to 1, then method 1, as previously described, is performed on the image block of the picture in the sequence.

[0060] According to the 15th aspect, the disclosure relates to a decoding device, From the bitstream, analyze the syntax elements to indicate whether the previously expressed method is enabled or not. Includes one or more processing circuits configured to adaptively enable or disable decoder-side motion vector refinement (DMVR) procedures according to syntax elements.

[0061] In possible implementations of the method by its 15th aspect, syntax elements are obtained from one of the following: the bitstream sequence parameter set (SPS) level, the bitstream picture parameter set (PPS) level, a slice header, a coding tree unit (CTU) syntax, or a coding unit (CU) syntax.

[0062] In any possible implementation of the method by any prior implementation of the 15th aspect, or in the 15th aspect itself, the syntax element includes a first flag (sps_conditional_dmvr_flag) and a second flag (pps_conditional_dmvr_flag), If the first flag (sps_conditional_dmvr_flag) is equal to 0, the previously described method is not performed on the image block of the sequence. If the first flag (sps_conditional_dmvr_flag) is equal to 1 and the second flag (pps_conditional_dmvr_flag) is equal to 0, then the previously described method is not performed on the image blocks of the sequence's pictures, or If the first flag (sps_conditional_dmvr_flag) is equal to 1 and the second flag (pps_conditional_dmvr_flag) is equal to 1, the previously described method is performed on the image blocks of the sequence's pictures.

[0063] According to the sixteenth aspect, the disclosure relates to a computer-readable medium for storing computer-readable instructions that, when executed in a processor, perform steps in the manner described above.

[0064] According to the 17th aspect, the disclosure relates to a method for predicting the current image block in the current picture of a video, the method being A step of determining whether at least one condition is met, the at least one condition being that a first temporal distance (e.g., TD0) is equal to a second temporal distance (e.g., TD1), where the first temporal distance is expressed in terms of the difference between the picture order count value of the current picture and the picture order count value of a first reference image, and the second temporal distance is expressed in terms of the difference between the picture order count value of a second reference image and the picture order count value of the current picture, The procedure includes the step of performing a decoder-side motion vector refinement (DMVR) procedure to determine the predicted block of the current image block when at least one condition is met.

[0065] According to the 18th aspect of the disclosure, the disclosure relates to a method for predicting the current image block in the current picture of a video, wherein the method is Steps include: performing a motion vector refinement (DMVR) procedure to obtain a first refined motion vector and a second refined motion vector for a subblock of the current image block when one or more conditions are met, wherein the first refined motion vector corresponds to a first reference picture and the second refined motion vector corresponds to a second reference picture; A step of determining a predicted block (such as a predicted sample value) for the current image block, wherein the predicted block for the current image block includes a predicted block for a subblock, and the predicted block for the subblock is determined at least in part on a first refined motion vector and a second refined motion vector, One or more conditions are at least, This includes the condition that a first temporal distance (e.g., TD0) is equal to a second temporal distance (e.g., TD1), where the first temporal distance (TD0) is expressed as the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed as the difference between the picture order count value (POCc) of the second reference picture and the picture order count value (POC1) of the current image.

[0066] Possible implementations of the method by any prior implementation of the 15th, 16th, 17th, and 18th aspects, or in the 15th, 16th, 17th, and 18th aspects themselves, the first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or the first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or The first reference picture is a past reference image and the second reference picture is a future reference image, or the first reference picture is a future reference image and the second reference picture is a past reference image.

[0067] Methods according to some aspects of the invention can be carried out by apparatus according to some aspects of the invention. Further features and implementations of methods according to some aspects of the invention arise directly from the functions of apparatus according to some aspects of the invention and their different implementations.

[0068] It should be noted that a coding device may be an encoding device or a decoding device.

[0069] In another aspect, the invention relates to an apparatus for decoding a video stream, comprising a processor and memory. The memory stores instructions that cause the processor to perform the previously described method.

[0070] In another aspect, the invention relates to an apparatus for encoding a video stream, comprising a processor and memory. The memory stores instructions that cause the processor to perform the previously described method.

[0071] In another embodiment, a computer-readable storage medium is proposed that stores instructions causing one or more processors to be configured to encode video data when executed. The instructions cause one or more processors to perform the method described above.

[0072] In another embodiment, a computer program product is provided which has program code for performing the previously described method when the computer program is running on a computer.

[0073] Details of one or more embodiments are described in the accompanying drawings and the following description. Other features, purposes, and advantages will become apparent from the description, drawings, and claims.

[0074] For the purpose of clarity, any one of the embodiments described above may be combined with any one or more of the other embodiments described above to create new embodiments within the scope of this disclosure.

[0075] These and other features will be more clearly understood from the following detailed description, which will be used in conjunction with the attached drawings and claims.

[0076] For a more complete understanding of this disclosure, similar reference numerals represent similar parts, and references to the following brief description are made herein by reference in conjunction with the accompanying drawings and detailed description. [Brief explanation of the drawing]

[0077] [Figure 1] This block diagram illustrates an example coding system that can utilize conditional decoder-side motion vector refinement techniques. [Figure 2]This block diagram illustrates an example of a video encoder that can implement conditional decoder-side motion vector refinement technology. [Figure 3] This block diagram illustrates an example of a video decoder capable of implementing conditional decoder-side motion vector refinement technology. [Figure 4] This is a schematic diagram of a coding device. [Figure 5] This is a block diagram illustrating an example of an encoding or decoding device. [Figure 6A] This is a schematic example of temporal distance for conditional decoder-side motion vector refinement in video coding. [Figure 6B] This is a schematic example of a decoder-side motion vector refinement (DMVR) procedure. [Figure 6C] Figure 6B is a flowchart illustrating an example of the decoder-side motion vector refinement (DMVR) procedure. [Figure 6D] This flowchart illustrates another example of the decoder-side motion vector refinement (DMVR) procedure. [Figure 7] This is a flowchart illustrating an example of an encoding method. [Figure 8] This is a flowchart illustrating an example of a decoding method. [Figure 9] This flowchart illustrates an example of a method for predicting the current image block within the current picture of a video. [Figure 10] This is a block diagram illustrating the structure of an example device for interpretation of the current image block within the current picture of a video. [Figure 11] This is a block diagram illustrating the structure of an example content supply system that provides content distribution services. [Figure 12] This is a block diagram showing the structure of an example terminal device. [Modes for carrying out the invention]

[0078] Firstly, while exemplary implementations of one or more embodiments are provided below, it should be understood that the disclosed systems and / or methods can be implemented using any number of techniques, whether currently known or existing. The disclosure should not be limited in any way to the exemplary implementations, drawings, and techniques illustrated below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the accompanying claims, along with the full range of equivalents.

[0079] Definitions of acronyms and vocabulary

[0080] DMVR Decoder Side Motion Vector Refinement

[0081] SAD (Sum of Absolute Differences)

[0082] MV (Motion Vector)

[0083] MCP (Motion Compensated Prediction)

[0084] HEVC (High Efficiency Video Coding)

[0085] Figure 1 is a block diagram illustrating an exemplary coding system 10 that may utilize bidirectional prediction technology. As shown in Figure 1, the coding system 10 includes a source device 12 that provides encoded video data to be later decoded by a destination device 14. In particular, the source device 12 may provide the video data to the destination device 14 via a computer-readable medium 16. The source device 12 and destination device 14 may include any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or the like. In some cases, the source device 12 and destination device 14 may be equipped for wireless communication.

[0086] The destination device 14 may receive encoded video data to be decoded via a computer-readable medium 16. The computer-readable medium 16 may include any type of medium or device that can move the encoded video data from the source device 12 to the destination device 14. In one example, the computer-readable medium 16 may include a communication medium that enables the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that can help facilitate communication from the source device 12 to the destination device 14.

[0087] In some examples, encoded data may be output to a storage device via the output interface 22. Similarly, encoded data may be accessed from the storage device via the input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, digital video disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, the storage device may correspond to a file server or another intermediate storage device capable of storing encoded video generated by the source device 12. The destination device 14 may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and transmitting that encoded video data to the destination device 14. Exemplary file servers include web servers (e.g., for websites), File Transfer Protocol (FTP) servers, network-attached storage (NAS) devices, or local disk drives. The destination device 14 may access the encoded video data through a data connection of any standard, including an internet connection. This may include wireless channels (e.g., Wi-Fi connection), wired connections (e.g., digital subscriber line, DSL, cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. Transmission of the encoded video data from the storage device may be streaming transmission, download transmission, or a combination thereof.

[0088] The technology of this disclosure is not necessarily limited to wireless applications or configurations. The technology may be applied to video coding supporting any of a variety of multimedia applications, such as terrestrial television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission such as dynamic adaptive streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, the coding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video phone.

[0089] In the example in Figure 1, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. According to this disclosure, the video encoder 20 of the source device 12 and / or the video decoder 30 of the destination device 14 may be configured to apply the technique for bidirectional prediction. In other examples, the source and destination devices may include other components or arrangements. For example, the source device 12 may receive video data from an external video source, such as an external camera. Similarly, the destination device 14 may interface with an external display device rather than including an integrated display device.

[0090] The coding system 10 illustrated in Figure 1 is merely an example. The technique for bidirectional prediction can be performed by any digital video coding and / or decoding device. While the technique of this disclosure is generally performed by a video coding device, the technique can also be performed by a video encoder / decoder, typically referred to as a “CODEC”. Furthermore, the technique of this disclosure can also be performed by a video preprocessor. The video encoder and / or decoder may be a graphics processing unit (GPU) or a similar device.

[0091] The source device 12 and destination device 14 are merely examples of coding devices in which the source device 12 generates coded video data for transmission to the destination device 14. In some examples, the source device 12 and destination device 14 may operate in a substantially symmetrical manner such that each of the source device 12 and destination device 14 includes video coding and decoding components. Thus, the coding system 10 may support one-way or two-way video transmission between video devices 12, 14 for, for example, video streaming, video playback, video broadcasting, or video phone calls.

[0092] The video source 18 of the source device 12 may include video capture devices such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, the video source 18 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video.

[0093] In some cases, when the video source 18 is a video camera, the source device 12 and the destination device 14 may form a so-called cameraphone or videophone. However, as mentioned above, the techniques described in this disclosure may be applicable to video coding in general and to wireless and / or wired applications. In each case, captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video information may then be output onto a computer-readable medium 16 via the output interface 22.

[0094] The computer-readable medium 16 may include temporary media such as wireless broadcasting or wired network transmission, or storage media (i.e., non-temporary storage media) such as hard disks, flash drives, compact discs, digital video discs, Blu-ray discs, or other computer-readable media. In some examples, a network server (not shown) may receive encoded video data from a source device 12 and provide the encoded video data to a destination device 14, for example, via network transmission. Similarly, a computing device in a media production facility, such as a disc compression molding machine, may receive encoded video data from a source device 12 and produce a disc containing the encoded video data. Thus, the computer-readable medium 16 may be understood to include one or more computer-readable media in various forms in various examples.

[0095] The input interface 28 of the destination device 14 receives information from the computer-readable medium 16. The information on the computer-readable medium 16 may include syntax information defined by the video encoder 20, which is also used by the video decoder 30, including syntax elements that describe the characteristics and / or processing of blocks and other coded units, such as groups of pictures (GOPs). The display device 32 displays the decoded video data to the user and may include any of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device.

[0096] The video encoder 20 and video decoder 30 may operate in accordance with a video coding standard, such as the High Efficiency Video Coding (HEVC) standard currently under development, and may conform to the HEVC Test Model (HM). Alternatively, the video encoder 20 and video decoder 30 may operate in accordance with other proprietary or industry standards, such as the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.264 standard, H.265 / HEVC, or extensions of such standards, which may be referred to as Motion Picture Expert Group (MPEG)-4, Part 10, Advanced Video Coding (AVC). However, the technology of this disclosure is not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. Although not shown in Figure 1, in some embodiments, the video encoder 20 and video decoder 30 may be integrated with the audio encoder and decoder, respectively, and may include a suitable multiplexer-demultiplexer (MUX-DEMUX) unit or other hardware and software for handling the encoding of both audio and video in a common data stream or separate data streams. Where applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).

[0097] The video encoder 20 and video decoder 30 may each be implemented as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), separate logic, software, hardware, firmware, or any combination thereof, among many other suitable encoder circuits. When the technology is partially implemented in software, the device may store instructions for the software in a suitable non-temporary computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the video encoder 20 and video decoder 30 may be comprised of one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (codec) within each device. A device comprising the video encoder 20 and / or video decoder 30 may include an integrated circuit, a microprocessor, and / or a wireless communication device such as a mobile phone.

[0098] Figure 2 is a block diagram illustrating an example of a video encoder 20 capable of implementing bidirectional prediction technology. The video encoder 20 can perform intra and intercoding of video blocks within a video slice. Intra coding relies on spatial prediction to reduce or eliminate spatial redundancy in video within a given video frame or picture. Intercoding relies on temporal prediction to reduce or eliminate temporal redundancy in video within adjacent frames or pictures of a video sequence. Intra mode (I mode) may refer to any of several space-based coding modes. Inter modes, such as unidirectional prediction (P mode) or bidirectional prediction (B mode), may refer to any of several time-based coding modes.

[0099] As shown in Figure 2, the video encoder 20 receives the current video block in the video frame to be encoded. In the example in Figure 2, the video encoder 20 includes a mode selection unit 40, a reference frame memory 64, an adder 50, a transformation unit 52, a quantization unit 54, and an entropy coding unit 56. The mode selection unit 40 then includes a motion compensation unit 44, a motion estimation unit 42, an intra-prediction unit 46, and a segmentation unit 48. For video block reconstruction, the video encoder 20 also includes an inverse quantization unit 58, an inverse transformation unit 60, and an adder 62. A deblocking filter (not shown in Figure 2) may also be included to filter block boundaries to remove block noise artifacts from the reconstructed video. If desired, the deblocking filter typically filters the output of the adder 62. In addition to the deblocking filter, additional filters (in-loop or after-loop) may also be used. Such filters are not shown for brevity, but if desired, the output of adder 50 may be filtered (as an in-loop filter).

[0100] During the encoding process, the video encoder 20 receives video frames or slices to be coded. A frame or slice may be divided into multiple video blocks. The motion estimation unit 42 and the motion compensation unit 44 perform inter-predictive coding of the received video blocks for one or more blocks within one or more reference frames to provide temporal predictions. The intra-predictive unit 46 may alternatively perform intra-predictive coding of the received video blocks for one or more neighboring blocks within the same frame or slice as the block to be coded to provide spatial predictions. The video encoder 20 may perform multiple coding passes, for example, to select an appropriate coding mode for each block of video data.

[0101] Furthermore, the partitioning unit 48 may partition blocks of video data into sub-blocks based on an evaluation of the previous partitioning method in the previous coding pass. For example, the partitioning unit 48 may first partition a frame or slice into large coding units (LCUs), and then, based on rate distortion analysis (e.g., rate distortion optimization), partition each LCU into sub-coding units (sub-CUs). The mode selection unit 40 may further create a quadtree data structure that shows the partitioning of LCUs into sub-CUs. A leaf node CU of the quadtree may include one or more prediction units (PUs) and one or more transformation units (TUs).

[0102] This disclosure uses the term “block” to refer to any CU, PU, ​​or TU in the context of HEVC, or a similar data structure in the context of other standards (e.g., macroblocks and their subblocks in H.264 / AVC). A CU includes a coding node, a PU, and a TU associated with the coding node. The size of a CU corresponds to the size of the coding node and its shape is square. The size of a CU may range from 8x8 pixels to the size of a tree block with a maximum of 64x64 pixels or greater. Each CU may contain one or more PUs and one or more TUs. Syntax data associated with a CU may, for example, describe the division of the CU into one or more PUs. The division mode may differ between the CU being skipped, directly mode coded, intra-predictive mode coded, or inter-predictive mode coded. A PU may be divided so that its shape is non-square. Syntax data associated with a CU may also describe the division of the CU into one or more TUs by, for example, a quadtree. In one embodiment, the CU, PU, ​​or TU can have a square or non-square shape (for example, a rectangle).

[0103] The mode selection unit 40 may, for example, select one of the coding modes, intra or intercoded, based on the result of an error, and provide the resulting intra or intercoded block to the adder 50 to generate residual block data, and to the adder 62 to reconstruct the encoded block for use as a reference frame. The mode selection unit 40 also provides syntax elements such as motion vectors, intra mode indicators, segmentation information, and other such syntax information to the entropy coding unit 56.

[0104] The motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are illustrated separately for conceptual purposes. The motion estimation performed by the motion estimation unit 42 is a process that generates motion vectors, which estimate the motion of a video block. The motion vectors may, for example, represent the displacement of the PU of a video block in the current video frame or picture relative to a predicted block in a reference frame (or other coded unit) relative to the current block coded in the current frame (or other coded unit). The predicted block is a block that is found to closely match the block to be coded, with respect to pixel differences, which may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics. In some examples, the video encoder 20 may calculate values ​​for pixel positions below an integer in the reference picture stored in the reference frame buffer 64. For example, the video encoder 20 may interpolate values ​​for quarter-pixel, eighth-pixel, or other fractional-pixel positions in the reference picture. Therefore, the motion estimation unit 42 may perform motion search for full pixel positions and fractional pixel positions and output motion vectors with fractional pixel accuracy.

[0105] The motion estimation unit 42 calculates a motion vector for the PU of a video block in an intercoded slice by comparing the PU's position with the predicted block's position in a reference picture. The reference picture may be selected from a first reference picture list (List 0) or a second reference picture list (List 1), each identifying one or more reference pictures stored in the reference frame memory 64. The motion estimation unit 42 transmits the calculated motion vector to the entropy coding unit 56 and the motion compensation unit 44.

[0106] Motion compensation performed by the motion compensation unit 44 may involve fetching or generating a predicted block based on the motion vector determined by the motion estimation unit 42. Again, the motion estimation unit 42 and the motion compensation unit 44 may be functionally integrated in some examples. Upon receiving the motion vector for the PU of the current video block, the motion compensation unit 44 may locate the predicted block pointed to by the motion vector in one of the reference picture lists. The adder 50 forms the residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block, which are coded, to form a pixel difference value, as discussed below. Generally, the motion estimation unit 42 performs motion estimation on the lumen component, and the motion compensation unit 44 uses the motion vector calculated based on the lumen component for both the chromen and lumen components. The mode selection unit 40 may also generate syntax elements associated with the video block and video slice for use by the video decoder 30 when decoding the video block of the video slice.

[0107] The intra-prediction unit 46 may intra-predict the current block as an alternative to the inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, as described above. In particular, the intra-prediction unit 46 may determine the intra-prediction mode to use to encode the current block. In some examples, the intra-prediction unit 46 may encode the current block using various intra-prediction modes, for example, between separate encoding passes, and the intra-prediction unit 46 (or, in some examples, the mode selection unit 40) may select an appropriate intra-prediction mode from the tested modes to use.

[0108] For example, the intra-prediction unit 46 may use rate distortion analysis to calculate rate distortion values ​​for various tested intra-prediction modes and select the intra-prediction mode with the best rate distortion characteristics among the tested modes. Rate distortion analysis generally determines the amount of distortion (or error) between the encoded block and the original unencoded block encoded to produce the encoded block, as well as the bit rate (i.e., the number of bits) used to produce the encoded block. The intra-prediction unit 46 may calculate a ratio from the distortion and rate for various encoded blocks to determine which intra-prediction mode exhibits the best rate distortion value for the block.

[0109] In addition, the intra-prediction unit 46 may be configured to encode depth blocks of the depth map using a depth modeling mode (DMM). The mode selection unit 40 may determine whether an available DMM mode produces better coding results than the intra-prediction mode and other DMM modes, for example, using rate-distortion optimization (RDO). Data for the texture image corresponding to the depth map may be stored in the reference frame memory 64. The motion estimation unit 42 and the motion compensation unit 44 may also be configured to interpret the depth blocks of the depth map.

[0110] After selecting an intra-prediction mode for a block (for example, one of the conventional intra-prediction mode or DMM mode), the intra-prediction unit 46 may provide the entropy coding unit 56 with information indicating the selected intra-prediction mode for the block. The entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. The video encoder 20 may include, within the transmitted bitstream configuration data, a definition encoding the context for various blocks to be used for each context, an indication of the most likely intra-prediction mode, an intra-prediction mode index table, and a modified intra-prediction mode index table, which may include a plurality of intra-prediction mode index tables and a plurality of modified intra-prediction mode index tables (also referred to as codeword mapping tables).

[0111] The video encoder 20 forms a residual video block by subtracting predicted data from the mode selection unit 40 from the original video block that is being coded. The adder 50 represents one or more components that perform this subtraction operation.

[0112] The transformation processing unit 52 applies a transformation, such as a discrete cosine transform (DCT) or a conceptually similar transformation, to the residual block to produce a video block containing residual transformation coefficient values. The transformation processing unit 52 may perform other transformations conceptually similar to the DCT. Wavelet transforms, integer transforms, subband transforms, or other types of transformations may also be used.

[0113] The transformation processing unit 52 applies a transformation to the residual block to create a block of residual transformation coefficients. The transformation may convert the residual information from the pixel value domain to a transformation domain such as the frequency domain. The transformation processing unit 52 may transmit the resulting transformation coefficients to the quantization unit 54. The quantization unit 54 quantizes the transformation coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 may then perform a scan of the matrix containing the quantized transformation coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0114] Following quantization, the entropy coding unit 56 entropy codes the quantized transformation coefficients. For example, the entropy coding unit 56 may perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding technique. In the case of context-based entropy coding, the context may be based on neighborhood blocks. Following entropy coding by the entropy coding unit 56, the encoded bitstream can be transmitted to another device (e.g., video decoder 30) or archived for later transmission or retrieval.

[0115] The inverse quantization unit 58 and the inverse transform unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block within the pixel region for later use, for example, as a reference block. The motion compensation unit 44 may compute a reference block by adding the residual block to a predicted block of one of the frames in the reference frame memory 64. The motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to compute pixel values ​​less than integers for use in motion estimation. The adder 62 adds the reconstructed residual block to the motion-compensated predicted block created by the motion compensation unit 44 to create a reconstructed video block for storage in the reference frame memory 64. The reconstructed video block can be used as a reference block by the motion estimation unit 42 and the motion compensation unit 44 to intercode blocks in subsequent video frames.

[0116] Other structural variations of the video encoder 20 can be used to encode video streams. For example, an unconverted encoder 20 can directly quantize the residual signal for a given block or frame without a conversion processing unit 52. In another implementation, the encoder 20 may have a quantization unit 54 and an inverse quantization unit 58 combined into a single unit.

[0117] Figure 3 is a block diagram illustrating an example of a video decoder 30 capable of realizing bidirectional prediction technology. In the example in Figure 3, the video decoder 30 includes an entropy decoding unit 70, a motion compensation unit 72, an intra-prediction unit 74, an inverse quantization unit 76, an inverse transform unit 78, a reference frame memory 82, and an adder 80. In some examples, the video decoder 30 may perform a decoding path that is generally the reverse of the encoding path described with respect to the video encoder 20 (Figure 2). The motion compensation unit 72 may generate prediction data based on motion vectors received from the entropy decoding unit 70, while the intra-prediction unit 74 may generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 70.

[0118] During the decoding process, the video decoder 30 receives an encoded video bitstream from the video encoder 20 representing the video blocks and associated syntax elements of the encoded video slices. The entropy decoding unit 70 of the video decoder 30 entropy-decodes the bitstream to generate quantized coefficients, motion vectors or intra-predictive mode indicators, and other syntax elements. The entropy decoding unit 70 transfers the motion vectors and other syntax elements to the motion compensation unit 72. The video decoder 30 may receive syntax elements at the video slice level and / or video block level.

[0119] When a video slice is encoded as an intra-coded (I) slice, the intra-prediction unit 74 may generate prediction data for the video block of the current video slice based on the signaled intra-prediction mode and data from blocks decoded before the current frame or picture. When a video frame is encoded as an inter-coded (i.e., B, P, or GPB) slice, the motion compensation unit 72 creates prediction blocks for the video block of the current video slice based on motion vectors and other syntax elements received from the entropy decoding unit 70. The prediction blocks may be created from one of the reference pictures in one of the reference picture lists. The video decoder 30 may construct the reference frame lists, List 0 and List 1, using default construction techniques based on the reference pictures stored in the reference frame memory 82.

[0120] The motion compensation unit 72 determines prediction information for the video block of the current video slice by analyzing motion vectors and other syntax elements, and uses the prediction information to create a prediction block for the current video block being decoded. For example, the motion compensation unit 72 uses some of the received syntax elements to determine the prediction mode used to encode the video block of the video slice (e.g., intra or interpredict), the interpredict slice type (e.g., B slice, P slice, or GPB slice), construction information for one or more of the reference picture lists for the slice, motion vectors for each intercoded video block of the slice, the interpredict status for each intercoded video block of the slice, and other information for decoding the video block in the current video slice.

[0121] The motion compensation unit 72 may also perform interpolation based on an interpolation filter. The motion compensation unit 72 may use an interpolation filter to compute interpolated values ​​for pixels below an integer in the reference block, as used by the video encoder 20 during the encoding of the video block. In this case, the motion compensation unit 72 may determine the interpolation filter to be used by the video encoder 20 from the received syntax elements and use the interpolation filter to create a prediction block.

[0122] Data for the texture image corresponding to the depth map can be stored in the reference frame memory 82. The motion compensation unit 72 can also be configured to interpret depth blocks of the depth map.

[0123] As will be understood by those skilled in the art, the coding system 10 in Figure 1 is suitable for implementing various video coding or compression techniques. Several video compression techniques, such as interpretation, intrapretation, and loop filtering, have been demonstrated to be effective. Accordingly, video compression techniques have been adopted in various video coding standards such as H.264 / AVC and H.265 / HEVC.

[0124] Various coding tools, such as Adaptive Motion Vector Prediction (AMVP) and Merge Mode (MERGE), are used to predict motion vectors (MVs), improving interpretation efficiency and therefore overall video compression efficiency.

[0125] The MV noted above is used in dual prediction. In dual prediction operation, two prediction blocks are formed. One prediction block is formed using the MV from list0 (referred to here as MV0). The other prediction block is formed using the MV from list1 (referred to here as MV1). The two prediction blocks are then combined (e.g., averaged) to form a single prediction signal (e.g., a prediction block or predictor block).

[0126] Other variations of the video decoder 30 can be used to decode a compressed bitstream. For example, the decoder 30 can produce an output video stream without a loop filtering unit. For example, an unconverted decoder 30 can directly dequantize the residual signal for a given block or frame without an inverse conversion processing unit 78. In another implementation, the video decoder 30 may have an inverse quantization unit 76 and an inverse conversion processing unit 78 combined into a single unit.

[0127] Figure 5 is a schematic diagram of a coding device 500 according to an embodiment of the disclosure. The coding device 500 is suitable for implementing the disclosed embodiment as described herein. In embodiments, the coding device 500 may be a decoder, such as the video decoder 30 in Figure 1, or an encoder, such as the video encoder 20 in Figure 1. In embodiments, the coding device 500 may be one or more components of the video decoder 30 in Figure 1 or the video encoder 20 in Figure 1 as described above.

[0128] The coding device 500 includes an inlet port 510 and a receiver unit (Rx) 520 for receiving data, a processor, logic unit, or central processing unit (CPU) 530 for processing data, a transmitter unit (Tx) 540 and an exit port 550 for transmitting data, and a memory 560 for storing data. The coding device 500 may also include optical-electrical (OE) components and electrical-optical (EO) components coupled to the inlet port 510, receiver unit 520, transmitter unit 540, and exit port 550 for the exit or input of optical or electrical signals.

[0129] The processor 530 is implemented by hardware and software. The processor 530 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), FPGAs, ASICs, and DSPs. The processor 530 communicates with the inlet port 510, the receiver unit 520, the transmitter unit 540, the exit port 550, and the memory 560. The processor 530 includes a coding module 570. The coding module 570 implements the disclosed embodiments described above. For example, the coding module 570 implements, processes, prepares, or provides various coding operations. Thus, the inclusion of the coding module 570 provides a substantial improvement to the functionality of the coding device 500 and affects the conversion of the coding device 500 to different states. Alternatively, the coding module 570 is implemented as instructions stored in the memory 560 and executed by the processor 530.

[0130] Memory 560 includes one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store the program when such a program is selected for execution, and to store instructions and data read during the execution of the program. Memory 560 may be volatile and / or non-volatile, and may be read-only memory (ROM), random-access memory (RAM), tri-level associative memory (TCAM), and / or static random-access memory (SRAM).

[0131] Figure 4 is a simplified block diagram of apparatus 1000, which can be used as either or both of the source device 12 and destination device 14 from Figure 1A, according to an exemplary embodiment. Apparatus 1000 can realize the technology of this application. Apparatus 1000 can take the form of a computing system comprising multiple computing devices, or it can take the form of a single computing device, such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, and the like.

[0132] The processor 1002 within the device 1000 can be a central processing unit. Alternatively, the processor 1002 can be any other type of device or multiple devices capable of manipulating or processing information that currently exists or will be developed in the future. The disclosed implementation can be carried out using a single processor as represented, for example, processor 1002, but advantages in speed and efficiency can be achieved using one or more processors.

[0133] The memory 1004 within the device 1000 can be a read-only memory (ROM) device or random-access memory (RAM) in one implementation. Any other suitable type of storage device can be used as memory 1004. Memory 1004 may contain code and data 1006 accessed by the processor 1002 using the bus 1012. Memory 1004 may further contain an operating system 1008 and an application program 1010, the application program 1010 containing at least one program that enables the processor 1002 to perform the method described herein. For example, the application program 1010 may contain applications 1 through N, which further contain video coding applications that perform the method described herein. The device 1000 may also contain additional memory in the form of secondary storage 1014, which may be, for example, a memory card used with a mobile computing device. Since video communication sessions can contain a considerable amount of information, all or part of it can be stored in secondary storage 1014 and loaded into memory 1004 when needed for processing.

[0134] The device 1000 may also include one or more output devices, such as a display 1018. In one example, the display 1018 may be a touch-sensitive display combining a display with a touch-sensing element capable of sensing touch input. The display 1018 can be coupled to the processor 1002 via a bus 1012. Other output devices that enable a user to program or otherwise use the device 1000 may be provided in addition to, or as alternatives to, the display 1018. When the output device is a display, or includes a display, the display can be implemented in a variety of ways, including liquid crystal displays (LCDs), cathode ray tube (CRT) displays, plasma displays, or light-emitting diode (LED) displays such as organic LED (OLED) displays.

[0135] The device 1000 may also include, or be capable of communicating with, any other image sensing devices 1020, which are currently existing or to be developed in the future, that are capable of sensing images, such as a camera or an image of the user operating the device 1000. The image sensing devices 1020 can be positioned so that they are directed toward the user operating the device 1000. In one example, the position and optical axis of the image sensing device 1020 can be configured such that its field of view is directly adjacent to the display 1018 and includes the area in which the display 1018 is visible.

[0136] The device 1000 may also include, or communicate with, a sound sensing device 1022, such as a microphone, or any other sound sensing device that is currently existing or may be developed in the future, capable of sensing sounds near the device 1000. The sound sensing device 1022 may be positioned to face the user operating the device 1000 and may be configured to receive sounds made by the user while the user is operating the device 1000, such as speech or other utterances.

[0137] Figure 4 depicts the processor 1002 and memory 1004 of device 1000 as integrated into a single unit, but other configurations are possible. The operation of processor 1002 can be distributed across multiple machines (each machine having one or more processors), which can be coupled directly or across local area or other networks. Memory 1004 can be distributed across multiple machines, such as network-based memory or memory in multiple machines running the operation of device 1000. Although depicted as a single bus here, the bus 1012 of device 1000 can consist of multiple buses. Furthermore, secondary storage 1014 can be directly coupled to other components of device 1000 or accessed via a network, and can include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Thus, device 1000 can be implemented in a wide variety of configurations.

[0138] In video compression, interpretation is the process of using reconstructed samples of a previously decoded reference picture by specifying motion vectors for the current block. These motion vectors can be coded as prediction residuals by using spatial or temporal motion vector predictors. Motion vectors can be subpixel precision. Interpolation filters are applied to derive subpixel precision pixel values ​​in the reference frame from the reconstructed integer position values. Bipretation refers to the process in which the prediction for the current block is derived as a weighted combination of two prediction blocks derived using two motion vectors from two reference picture regions. In this case, in addition to the motion vectors, the reference indices to the reference picture from which the two prediction blocks are derived must also be coded. The motion vector for the current block can also be derived through a merge process in which the spatial neighbor motion vectors and reference indices are inherited without coding any motion vector residuals. In addition to spatial neighbors, the motion vectors of the previously coded reference frame are also preserved and used as a temporal merge option, along with appropriate scaling of the motion vectors to account for the distance to the reference frame relative to the distance to the reference frame for the current block.

[0139] Several methods have been proposed to perform decoder-side motion vector refinement or derivation so that motion vector residual coding bits can be further reduced.

[0140] In a class of methods called bilateral matching (BM), the motion information of the current CU is derived by finding the closest match between two blocks along the motion trajectory of the current CU in two different reference pictures. This is shown in Figure 6A. Under the assumption of a continuous motion trajectory, the motion vectors MV0 and MV1 pointing to the two reference blocks are assumed to be proportional to the temporal distance between the current picture and the two reference pictures, i.e., TD0 and TD1.

[0141] In bilateral matching merge mode, biprediction is always applied because CU motion information is derived based on the closest match between two blocks along the current CU motion trajectory in two different reference pictures.

[0142] Explicit merge modes to indicate template matching merge or bilateral matching merge can be signaled to distinguish these modes from the default merge mode, which does not require any decoder-side motion vector derivation.

[0143] In some examples, temporal distance is ignored, and bilateral matching is performed using motion vectors with equal magnitude and opposite signs in past and future reference frames.

[0144] In some examples, the merge index is not signaled, while in others, an explicit merge index is signaled to simplify the complexity of decoders performing multiple motion compensations.

[0145] Figure 6B is a schematic illustration of an example of the DMVR method 400. In this example, the DMVR method 400 begins with the current block 402 in the current picture 404. In this example, the current block 402 may be square or non-square in shape. The current picture 404 may also be referred to as the current region, image, tile, etc. As shown in Figure 6B, MV0 points to the first reference block (also referring to the first initial reference block) 406 in the first reference picture 408, and MV1 points to the second reference block (also referring to the second initial reference block) 410 in the second reference picture 412. In this example, the first reference block 406 is ahead of the current block 402 in terms of time, sequence, decoding order, or some other parameter. In this example, the second reference block 410 is ahead of the current block 402 in terms of time, sequence, decoding order, or some other parameter. The first reference block 406 and the second reference block 410 may be referred to here as the initial reference blocks.

[0146] The first reference block 406 and the second reference block 410 are combined to form a bilateral template block 414. In one example, the first reference block 406 and the second reference block 410 are averaged together to generate the bilateral template block 414. In one example, the bilateral template block 414 is generated as a weighted combination of the first reference block 406 and the second reference block 410.

[0147] Once the bilateral template block 414 is generated, a template matching operation is performed. This involves calculating a first cost between the bilateral template block 414 and each candidate reference block in the sample region around the first reference block 406, and a second cost between the bilateral template block 414 and each candidate reference block in the sample region around the second reference block 410. In one example, the potential reference block that yields the corresponding lowest cost (e.g., minimum template cost) determines which reference block in each sample region will act as the refined reference block (also known as the revised reference block). In one example, the first and second costs are determined using SAD. Other cost measures may be used in actual applications.

[0148] In the example in Figure 6B, the first refined reference block 416 in the first reference picture 408 pointed to by MV0' results in the lowest first cost, and the second refined reference block 418 in the second reference picture 412 pointed to by MV1' results in the lowest second cost. In one example, the first refined reference block 416 and the second refined reference block 418 replace the first reference block 406 and the second reference block 410, respectively.

[0149] Subsequently, a prediction block is generated using the first refined reference block 416 and the second refined reference block 418. The prediction block 420 may be referred to as a predictor block or final dual prediction result. Once generated, the prediction block 420 may be used to generate an image for display on the display of an electronic device (e.g., a smartphone, tablet device, laptop computer, etc.).

[0150] The DMVR method 400 can be applied to a dual-prediction merge mode using one MV from a past reference picture and another from a future reference picture without the need to transmit additional syntax elements. The DMVR method 400 is not applied when local illumination compensation (LIC), affine motion, frame rate upconversion (FRUC), or CU merge candidates are enabled for CU in the Joint Exploration Model (JEM) reference software.

[0151] The JEM reference software searches for nine MV candidates (which will point to nine reference block candidates) for each reference block in a reference picture (for example, for each list, e.g., list 0 or list 1). The nine MV candidates include the original MV (i.e., initial MV, e.g., initial MV0 or initial MV1) that points to the reference block (i.e., the initial reference block, e.g., reference block 406 or reference block 410), and eight MVs that point to the reference block around the reference block that has one luma sample offset relative to the original MV in either the horizontal, vertical, or both directions. However, using MV candidates with one luma sample offset relative to the original MV may not provide the best MV candidate.

[0152] In the exemplary method, a decoder-side motion vector refinement (DMVR) method based on bilateral template matching is provided, in which a bilaterally averaged template is first created using reference blocks in L0 and L1 references obtained from explicitly signaled merge candidate indices, and bilateral matching is performed on this template. This is illustrated in Figures 6B and 6C. If there is any movement between the initial reference block (406, 410) referenced by the initial motion vector and the reference block (416, 418) referenced by the latest best motion vector, the template is updated. Also, in some examples, refinement is performed on one reference, and the motion vector in the other reference is obtained by mirroring this refined motion vector. Refinement alternates between the two references until either the center position has the smallest matching error or the maximum number of iterations is reached.

[0153] Figure 6C is a flowchart illustrating an example of the coding method 600. In one example, the coding method 600 is implemented in a decoder, such as the video decoder 30 in Figure 1. The coding method 600 can be implemented, for example, when a bitstream received from an encoder, such as the video encoder 20 in Figure 1, is to be decoded to generate an image to be displayed on the display of an electronic device. The coding method 600 may also be implemented in an encoder, such as the video encoder 20 in Figure 1. The coding method 600 will be described with reference to the elements identified in Figure 6B.

[0154] In block 602, the first reference block (e.g., reference block 406) within the first reference picture (e.g., reference picture 408) is determined based on the first motion vector (e.g., MV0) corresponding to the current block (e.g., current block 402) within the current picture (e.g., current picture 404).

[0155] In block 604, the second reference block (e.g., reference block 410) within the second reference picture (e.g., reference picture 412) is determined based on a second motion vector (e.g., MV1) corresponding to the current block (e.g., current block 402) within the current picture (e.g., current picture 604).

[0156] In block 606, a bilateral reference block (for example, bilateral reference block 414) is generated based on the first reference block and the second reference block. In one example, the bilateral reference block is obtained using a weighted average of the first and second reference blocks.

[0157] In block 608, a cost comparison is performed between the bilateral reference block and each of several candidate first reference blocks within the first reference picture. The candidate first reference blocks may be, for example, various reference blocks surrounding the first reference block 406 within the first reference picture 408. The cost comparison is used to determine the first refined motion vector (e.g., MV0'). In one example, the candidate first reference block is determined based on a step size selected from several available step sizes (e.g., 1 / 8, 1 / 4, 1 / 2, 1, etc.).

[0158] In block 610, a cost comparison is performed between the bilateral reference block and each of several candidate second reference blocks within the second reference picture. The candidate second reference blocks may be, for example, various reference blocks surrounding the second reference block 410 within the second reference picture 412. The cost comparison is used to determine the second refined motion vector (e.g., MV1'). In one example, the candidate second reference block is determined based on a step size selected from several available step sizes (e.g., 1 / 8, 1 / 4, 1 / 2, 1, etc.).

[0159] In block 612, a first refined reference block in the first reference picture (e.g., refined reference block 416) is selected based on the first refined motion vector, and a second refined reference block in the second reference picture (e.g., refined reference block 418) is selected based on the second refined motion vector.

[0160] In block 614, the prediction block (for example, prediction block 420) is determined based on the first refined reference block and the second refined reference block.

[0161] In block 616, the image generated using the prediction block is displayed on the display of an electronic device.

[0162] In some examples of refinement methods, CU-level refinement is performed first. Then, multiple candidate evaluations at the sub-CU (i.e., sub-block) level are performed using the refined CU-level MV as multiple candidates. Optionally, each sub-CU can perform its own refinement with respect to the best matching candidate. In another example, CU-level refinement is not performed, and each sub-CU can perform its own refinement.

[0163] Some cost functions use the motion vector refined distance as a bias term.

[0164] Assuming an implicit decoder-side derivation or refinement process, the encoder must perform these steps in exactly the same manner as the decoder so that the encoder-side reconstruction matches the decoder-side reconstruction.

[0165] Typically, only lumar samples are used during the decoder-side motion vector refinement or derivation process. However, in some cases, chrominance is also motion-compensated using the final refined motion vector (appropriately scaled to account for any chromar downsampling), which is then used for lumar motion compensation.

[0166] Another technique for refining motion vectors on the decoder side is called the Bidirectional Optical Flow (BIO) technique. In this method, motion-compensated interpolation is performed for a given coding unit using a prescriptive motion compensation method that uses samples from two reference frames indicated by the reference index and motion vector associated with the coding unit. In addition, horizontal and vertical slopes at sub-pixel precision positions are evaluated from the reference samples used for motion compensation, or using the motion-compensated samples themselves. The coding unit is divided into sub-blocks of uniform size, and the sub-block size can be 1x1 pixels, 2x2 pixels, 4x4 pixels, etc. The sample value of the sub-block in the current frame is pred BIO An equation based on optical flow for various values ​​associated with the reference frame to generate an estimate of is provided below in equation (1). In equation (1), (vx, vy) represents the flow of a subblock from reference frame L0 to the current frame and then to reference frame L1. Gx0 and Gy0 represent the horizontal and vertical gradients in L0, respectively. Gx1 and Gy1 represent the horizontal and vertical gradients in L1, respectively. I0 and I1 represent the intensity values ​​of the two reference patches in L0 and L1, respectively. τ1 and τ0 represent the distance of the current frame to reference frames L0 and L1, respectively. τ0 = POC(current) - POC(Ref0), τ1 = POC(Ref1) - POC(current). Nod BIO=1 / 2(I0+I1+vx / 2(τ1Gx1-τ0Gx0)+vy / 2*(τ1Gy1-τ0Gy0)) (1)

[0167] Using the individual flow formulas for each time interval, the difference between the predictor using L0 samples (i.e., the predicted sample value of the subblock) and the predictor using L1 samples is: Δ=(I0-I1)+vx(τ1Gx1+τ0Gx0)+vy(τ1Gy1+τ0Gy0) It is possible to write it as follows.

[0168] By minimizing the difference Δ, estimates of vx and vy can be obtained. For example, by taking the partial derivative of the square of the difference Δ with respect to vx and vy and setting that partial derivative to zero, an equation with vx and vy as unknowns can be obtained for the samples within the subblock and for the samples adjacent to the subblock. This set of overly constrained equations can be solved by least squares to obtain estimates of vx and vy. Using equation Eqn.(1) mentioned above, the calculated estimates of vx and vy, and the gradients Gx0 and Gy0, a correction term is added to the usual bipredictive motion compensation. In some ways, τ0 and τ1 are assumed to be equal in these equations.

[0169] As explained above, the decoder-side motion vector derivation / refinement method based on bilateral matching computes a difference motion vector (i.e., a change to the initial motion vectors MV0 and MV1) around the merge mode motion vector (i.e., the merge candidate containing the initial motion vectors MV0 and MV1) in two references (used for biprediction). The computed difference motion vector depends on the temporal distances TD0 and TD1 from the current picture to the two references. However, because the ratio between TD1 and TD0 means that an integer distance movement in one reference may be a non-integer distance movement in the other reference, evaluating the refinement requires evaluating sub-pixel-precision positions with different phases when compared to the phases of the merge mode motion vectors that constitute the starting position for the refinement (for example, if TD1 and TD0 are not equal, and L0 pic is at a distance of 2 and L1 is at a distance of 1, then a refinement movement of 1 pixel in L0 leads to a movement of 0.5 pixels in L1). This thus leads to the high computational complexity of the refinement procedure. To simplify the elaboration procedure, the elaboration procedure may ignore TD0 and TD1 and use differential motion vectors that are equal in magnitude and opposite in direction in the two references, such that an integer distance movement in one reference remains as an integer distance movement in the other reference. Multiple iterations of elaboration are applied until either a predefined maximum iteration count is reached or the center position of a given iteration is found to be the position with the lowest cost. This method also works well when hierarchical B-pictures are used in diadic configurations, such as those used in common test conditions employed in multipurpose video coding. In such diadic configurations, the number of pictures in the next temporal layer is twice as many as the number of pictures in the current temporal layer. Thus, a B-picture has one reference from the past and one reference from the future that are equidistant from the current picture.

[0170] However, due to occlusion in one of the references, the candidate merge motion vector may be applicable to one-way predictions, or the candidate merge motion vector may refer to two reference frames that are at unequal distances to the current picture. Also, commercial encoders tend to adapt picture types based on temporal correlations, so strict dyadic hierarchical B-pictures may not be used. For example, some encoders use two non-referenced pictures between every two referenced pictures. Some other encoders have variable distances between pictures belonging to the same temporal layer due to the characteristics of the underlying motion. In such cases, the use of refinement based on equally opposite differential motion vectors fails to produce a large coding gain and can affect the coding gain whenever an explicit flag is not used to indicate the need for refinement. However, signaling a flag for all CUs to indicate the use of refinement offsets some of the coding gain that refinement brings.

[0171] Therefore, whenever refinement based on bilateral matching using equally opposite difference motion vectors is used, for example, it is necessary to selectively or adaptively provide the ability to restrict or enable decoder-side refinement to coding units with equidistant references.

[0172] Furthermore, whenever the BIO method assumes that τ0 and τ1 are equal, it is necessary to provide the ability to restrict or enable the BIO method to be applicable only when the temporal distances are truly equal.

[0173] This disclosure addresses the above problem by providing a method for selectively restricting or enabling a dual-predictive merge-mode coding unit, coded within at least one access unit in a group of frames, to use decoder-side motion vector refinement based on the temporal distance of the coding unit to its two references. Such restriction can be achieved by setting a flag at the sequence parameter setting level on the coding side to allow decoder-side MV refinement only when the temporal distance of the coding unit to its two references is substantially equal. This method is used on both the coding and decoding sides when decoder-side MV refinement is enabled.

[0174] Detailed example of the presented method

[0175] Embodiments of the present disclosure provide a method for restricting decoder-side motion vector derivation / refinement to coding units having equidistant references only. The embodiments may be used when decoder-side MV refinement is used. For example, the method may be used when decoder-side MV refinement based on bilateral matching is used with differential motion vectors that are equal in magnitude and opposite in sign in two references used for bilateral prediction (whether or not a bilateral template is used). Such a method ignores the temporal distance from the current picture to the reference frame, as can be done by performing integer distance refinement starting from the center of a subpixel-precision merged motion vector using interpolation of samples at the phase of a single subpixel in each of the references, as previously described. The merged motion vector obtained from the merge candidate list can be at any subpixel position. For example, the subpixel precision may be 1 / 16. The method may also be used when decoder-side motion vector refinement based on bilateral optical flow (BDOF) is independent of temporal distance or assumes that they are equal.

[0176] In this embodiment, decoder-side MV refinement is used for a given coding unit or coding block only when the temporal distances from the coding block to the reference picture used for biprediction are substantially equal in magnitude and opposite in sign or direction. Figure 7 is a flowchart illustrating an embodiment of the coding method 700. In this embodiment, the coding method 700 is implemented in an encoder such as the video encoder 20 in Figure 1. In block 701, once a merge candidate list containing one or more merge candidates is constructed on the coding or decoding side according to a normative process, the merge motion vectors and their reference indices become available. In block 703, for each merge candidate indicating biprediction, two reference frames, referred to here as L0 and L1 references, corresponding to the two reference indices required for biprediction, are identified, and their temporal distances to the current picture, i.e., TD0 and TD1, are obtained. In block 705, it is determined whether the temporal distances to the reference picture used for biprediction are substantially equal in magnitude and opposite in sign. In block 707, if the temporal distances are substantially equal in magnitude and opposite in sign, the encoder or decoder performs a normative decoder-side MV refinement process on the merge candidate. The refined motion vector is used to perform motion-compensated biprediction using reference frames L0 and L1. In block 705, if the temporal distances to the reference pictures used for biprediction are determined to be substantially equal in magnitude and opposite in sign, then in block 709, decoder-side MV refinement is skipped and the merge motion vector is used for biprediction. In block 711, if the best merge mode candidate has a lower cost than any other mode evaluated, the encoder signals the coding unit to merge with a merge flag of 1. In addition, the encoder may also explicitly signal the merge candidate index for the winning merge candidate in the merge list. In some cases, this can be implicitly derived on the decoding side.

[0177] Figure 8 is a flowchart illustrating an embodiment of the decoding method 800. In this embodiment, the decoding method 800 is implemented in a decoder such as the video decoder 30 in Figure 1. In block 801, for the coding unit, a merge flag is analyzed from the received bitstream, and if the merge flag is set to true (e.g., 1), the merge candidate is obtained either by decoding an explicitly signaled merge index or by implicitly deriving it on the decoding side. In block 803, a normative merge list construction process is used to arrive at the motion vector and reference index associated with the merge index. In block 805, if the merge candidate indicates a biprediction, two reference frames corresponding to the two reference indices are identified, and their temporal distances to the current picture, i.e., TD0 and TD1, are obtained. In block 807, it is determined whether the temporal distances to the reference pictures used for the biprediction are substantially equal in magnitude and opposite in sign. In block 809, if the temporal distances are substantially equal in magnitude and opposite in sign, the decoder performs a normative decoder-side MV refinement process on the merge candidate. In particular, if the adoption of decoder-side motion vector refinement is enabled at the sequence level, decoder-side MV refinement is performed only when TD0 and TD1 are substantially equal in magnitude and opposite in sign. The refined motion vectors are used to perform motion-compensated biprediction using reference frames L0 and L1. Otherwise, in block 811, decoder-side MV refinement is skipped, and the motion vectors for the merge index are used to perform motion-compensated biprediction. In block 813, the coding block is reconstructed based on motion-compensated data, e.g., the sum of the analyzed residual data and the predicted data.

[0178] In another embodiment, the same checks on TD0 and TD1 may also be used in a bidirectional optical flow (BIO) based method (TD0 corresponding to τ0 and TD1 corresponding to -τ1 as described above) to enable a BIO-based method to a given coding unit only when TD0 and TD1 are substantially equal and have opposite signs.

[0179] Figure 9 is a flowchart of a method for interpretation (biprediction) of the current image block within the current picture of a video. The method begins in step 901.

[0180] In step 903, it is determined whether the current picture is temporally between the first reference picture (e.g., RefPic0) and the second reference picture (e.g., RefPic1), and whether the first temporal distance (e.g., TD0) and the second temporal distance (e.g., TD1) are the same. The first temporal distance (TD0) is between the current picture and the first reference picture (RefPic0), and the second temporal distance (TD1) is between the current picture and the second reference picture (RefPic1).

[0181] In step 905, when it is determined that the current picture is temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and that the first temporal distance (TD0) and the second temporal distance (TD1) are the same, a motion vector refinement (DMVR) procedure is performed to determine the predicted block of the current image block. The motion vector refinement (DMVR) procedure can be performed with respect to blocks 602 to 610 in Figure 6C or blocks 632 to 640 in Figure 6D as described above. Other methods of performing motion vector refinement can also be used.

[0182] In step 907, if it is determined that the first temporal distance (TD0) and the second temporal distance (TD1) are different distances, or that the current picture is not temporally between the first reference picture (RefPic0, etc.) and the second reference picture (RefPic1, etc.), motion compensation is performed to obtain a predicted block of the current image block using the first initial motion vector (MV0) and the second initial motion vector (MV1).

[0183] Figure 10 is a block diagram illustrating the structure of an example apparatus for interpretation of the current image block within the current picture of a video. The apparatus is A determination unit 101 configured to determine whether the current picture is temporally between a first reference picture (such as RefPic0) and a second reference picture (such as RefPic1), and whether a first temporal distance (such as TD0) and a second temporal distance (such as TD1) are the same, wherein the first temporal distance (TD0) is between the current picture and the first reference picture (RefPic0), and the second temporal distance (TD1) is between the current picture and the second reference picture (RefPic1), The system may include an interpredictive processing unit 103 configured to perform a motion vector refinement (DMVR) procedure to determine a predicted block for the current image block when it is determined that the current picture is temporally between a first reference picture (e.g., RefPic0) and a second reference picture (e.g., RefPic1), and the first temporal distance (TD0) and the second temporal distance (TD1) are the same.

[0184] In some embodiments, instead of always applying the aforementioned time-distance-based checks to conditionally perform decoder-side motion vector refinement at the coding unit or coding block level, it is possible to conditionally perform the checks only when a specific flag is signaled at the sequence parameter set level and / or picture level.

[0185] In one embodiment, flags such as sps_conditional_dmvr_flag are signaled at the sequence parameter set level whenever decoder-side motion vector refinement is enabled at the sequence parameter set level. When this flag is set to 0, decoder-side MV refinement can be performed at all access units independently of the temporal distance from the current picture to the reference frame. When this flag is set to 1, additional flags such as pps_conditional_dmvr_flag are signaled at the picture parameter set level. When pps_conditional_dmvr_flag is set to 0, decoder-side MV refinement can be performed independently of the temporal distance from the current picture to the reference frame. When pps_conditional_dmvr_flag is set to 1, decoder-side MV refinement can be performed only when the temporal distances from the current picture to the reference frame for a given CU are substantially equal in magnitude and opposite in sign.

[0186] The encoder can set sps_conditional_dmvr_flag to zero when a regular dyadic hierarchical B-picture group of pictures (GOP) structure is used, the maximum number of reference frames within a B-picture is set to 2, and the reference picture selection always selects a reference frame that is equal in time to the current picture and falls on the opposite side of the current picture. An example of a dyadic GOP structure in display order is I00B14B23B34B42B54B63B74B81B94B103B114B122B134B143B154P160, where the subscript indicates the time layer and the number following the picture type indicates the frame count in display order.

[0187] The encoder may set sps_conditional_dmvr_flag to 1 if (a) a normal diadic hierarchical B-picture GOP structure is used, but the maximum number of reference frames for a B-picture is set to more than 2, or (b) there is a possibility that the selection of a reference picture will be one that does not have a substantially equal temporal distance to the current picture or will not be on the opposite side of the current picture in display order, or (c) a non-diadic hierarchical B-picture or a non-diadic single B-picture layer GOP structure is used. An example of a non-diadic GOP structure is I00B11B21P30B41B51P60, which has only one layer of B-pictures. An example of an adaptive hierarchical B-picture is I00B12B22B31B42P50, in which the spacing between two pictures at the same temporal layer level is adaptively determined based on content characteristics.

[0188] Alternatively, sps_conditional_dmvr_flag can be manually configured by the user as an encoder parameter based on the conditions described above.

[0189] When sps_conditional_dmvr_flag is set to 1, the encoder can set pps_conditional_dmvr_flag to zero for frames where the maximum number of reference frames is set to 2, and those reference frames are substantially equal in temporal distance from the current picture and appear on the opposite side of the display order from the current picture.

[0190] When sps_conditional_dmvr_flag is set to 1, the encoder may set pps_conditional_dmvr_flag to 1 in frames where the maximum number of reference frames is set to a value greater than 2, or where the two reference frames used for biprediction for CU do not need to be substantially equal in time distance from the current picture, or where both reference pictures do not appear on opposite sides of the current picture in display order. In the example where the coding / decoding sequence is I00P10, P60, B41, B22, B32, B52 and the display order is I00P10B22B32B41B52P60, picture B22 may have I00P10, P60, B41 as its reference pictures. Of these reference pictures, reference pictures I0 and B4 are at equal distance and in opposite directions. Therefore, when I0 and B4 are used as references for B2, the temporal distances are equal and opposite, but this is not the case when P1 and B4 are used as references. When pps_conditional_dmvr_flag is set to 1, coding units in B2 that have I0 and B4 as references use decoder-side MV refinement, but coding units in B2 that have P1 and B4 as references cannot use decoder-side MV refinement (depending on a predetermined ratio threshold, e.g., the ratio of the distance between the current picture and the first reference (refL0) to the distance between the second reference (refL1) ​​and the current picture).

[0191] Since the motion of an object is not necessarily linear from L0 to the current picture and from the current picture to L1, the assumption of equally opposite motion can sometimes hold true even when the reference picture is not at substantially equal temporal distances. In one embodiment, the encoder can perform encoding for frames with pps_conditional_dmvr_flag set to 0, and another encoding for the same frames with pps_conditional_dmvr_flag set to 1, and select the setting that provides a lower rate-distortion optimization cost. The rate-distortion optimization cost is calculated as the sum of the distortion measure of the reconstructed frame with respect to the source frame and the consumed bits multiplied by a suitable Lagrangian multiplier that depends on the average quantization parameter used for the frame.

[0192] In other embodiments, the rate distortion optimization cost can be accumulated for coding units whose time distances are substantially unequal, both with and without decoder-side MV refinement, and the flag can be set to 1 for subsequent pictures if the absence of refinement results in a lower accumulated cost than the presence of refinement.

[0193] It is also possible to disable decoder-side motion vector refinement itself at the sequence parameter set (SPS) level or picture parameter set level when substantially equal temporal distances to a reference frame are not possible for any coding unit based on the GOP structure determined by the encoder. Conditional flags at the SPS level, if present, are signaled only when decoder-side motion vector refinement is enabled at the SPS level. Conditional flags at the PPS level, if present, are signaled only when decoder-side motion vector refinement is enabled (explicitly or implicitly) at the PPS level. Any alternative methods for signaling decoder-side MV refinement at the SPS / PPS level, the ability to signal unconditional refinement with respect to temporal distance to reference, and the ability to signal conditional refinement with respect to temporal distance to reference are anticipated by this invention. For example, instead of two flags, it is possible to encode a syntax element that takes one of three possible values ​​(e.g., 0, 1, and 2) by concatenating two flags together.

[0194] Figure 6D is a flowchart illustrating another example of performing a decoder-side motion vector refinement (DMVR) procedure or process. In embodiments, the process is implemented in a decoder, such as the video decoder 30 in Figure 1. The process can be implemented when a bitstream received from an encoder, such as the video encoder 20 in Figure 1, is to be decoded to generate an image on the display of an electronic device. The process is also implemented in an encoder, such as the video encoder 20 in Figure 1. The process will be described with reference to the elements identified in Figure 6B.

[0195] In block 632, the position of the first initial reference block (e.g., reference block 406) within the first reference picture (e.g., reference picture 408) is determined based on the first motion vector (e.g., MV0) corresponding to the current block (e.g., current block 402) within the current picture (e.g., current picture 404).

[0196] In block 634, the position of the second initial reference block (e.g., reference block 410) within the second reference picture (e.g., reference picture 412) is determined based on a second motion vector (e.g., MV1) corresponding to the current block (e.g., current block 402) within the current picture (e.g., current picture 404).

[0197] In block 636, the positions of multiple first reference blocks (for example, N-1 first reference blocks) within the first reference picture are determined.

[0198] In block 638, the positions of multiple second reference blocks (for example, N-1 second reference blocks) within the second reference picture are determined. In blocks 636 and 638, the position of each pair of reference blocks includes the position of the first reference block and the position of the second reference block, and for each pair of reference blocks, the first position offset (delta0x, delta0y) and the second position offset (delta1x, delta1y) are mirrored (i.e., equal in magnitude and opposite in sign), the first position offset (delta0x, delta0y) represents the offset of the position of the first reference block relative to the position of the first initial reference block, and the second position offset (delta1x, delta1y) represents the offset of the position of the second reference block relative to the position of the second initial reference block. In particular, for each pair of first and second reference blocks, the direction of the first offset is opposite to that of the second offset, the magnitude of the first offset is the same as that of the second offset, and the first and second offsets are associated with the respective first and second reference blocks of the pair.

[0199] In block 640, cost comparisons are performed between each pair of first and second reference blocks among the multiple first reference blocks in the first reference picture and the multiple second reference blocks in the second reference picture. Cost comparisons between first and second initial reference blocks may also be performed. The first reference blocks may be, for example, the various reference blocks surrounding the first initial reference block 406 in the first reference picture 408. The cost comparisons are used to determine the first refined motion vector (e.g., MV0') and the second refined motion vector (e.g., MV1'). The second reference block may be, for example, various reference blocks surrounding the second initial reference block 410 within the second reference picture 412. Alternatively, the positions of the pairs of reference blocks from the positions of the N pairs of reference blocks are determined as the positions of the first refined reference block and the second refined reference block based on a matching cost criterion. It can be understood that the N pairs of reference blocks may include the pairs of the first and second initial reference blocks.

[0200] In block 642, a first refined reference block in the first reference picture (e.g., refined reference block 416) is selected based on the first refined motion vector, and a second refined reference block in the second reference picture (e.g., refined reference block 418) is selected based on the second refined motion vector. Alternatively, the first refined reference block is determined in the first reference picture based on the position of the first refined reference block, and the second refined reference block is determined in the second reference picture based on the position of the second refined reference block.

[0201] In block 644, the prediction block (for example, prediction block 420) is determined based on the first refined reference block and the second refined reference block.

[0202] In block 646, the image generated using the prediction block is displayed on the display of an electronic device.

[0203] Those skilled in the art will recognize that many solutions can be applied to perform decoder-side motion vector refinement (DMVR) procedures, and that the present invention is not limited to the processes illustrated in the preceding text.

[0204] Based on the foregoing, this disclosure makes it possible to conditionally restrict (e.g., enable or disable) decoder-side motion vector refinement based on the temporal distance to two references used by each CU, and thus improves coding efficiency by not applying refinement when there is no prospect that the assumptions underlying the equally opposite difference motion vectors that refinement assumes are true.

[0205] Based on the foregoing, the Disclosure also provides granularity for unconditionally disabling refinement for all access units, conditionally disabling refinement for certain access units, enabling unconditional refinement at the access unit level, or enabling conditional refinement in an access unit based on the temporal distance from the current picture to the reference used by the coding unit within that access unit.

[0206] Based on the foregoing, this disclosure also offers the advantage of disabling the refinement performed on the decoder side for merge indices that are unlikely to improve compression gain.

[0207] Furthermore, based on the above, this disclosure may have the advantage of limiting refinement to only two equidistant references, resulting in less cache contamination from other references on the decoder side.

[0208] Based on the foregoing, this disclosure enables the normative invalidation of decoder-side motion vector refinement based on bilateral matching at the CU level whenever the temporal distances to two references are substantially equal in magnitude and not opposite in direction. In particular, this invalidation applies when the refinement process does not use temporal distance to scale the differential motion vector.

[0209] Based on the foregoing, the Disclosure adds flags at the sequence and picture parameter set levels so that the encoder has the option to signal appropriate flag values ​​based on factors such as the observed GOP structure and coding gain, enabling a temporal distance-based check at the CU level to be performed only when indicated by these flags.

[0210] The following describes the encoding and decoding methods as shown in the embodiments described above, and the applications of systems using them.

[0211] Figure 11 is a block diagram representing a content supply system 3100 for realizing a content distribution service. The content supply system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 over a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 may include, but is not limited to, Wi-Fi, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination of these, or similar.

[0212] The capture device 3102 may generate data and encode the data using an encoding method as represented in the embodiments described above. Alternatively, the capture device 3102 may deliver the data to a streaming server (not shown in the figure), which encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, a smartphone or tablet, a computer or laptop, a video conferencing system, a PDA, an in-vehicle device, or any combination thereof, or similar. For example, the capture device 3102 may include a source device 12 as described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform video encoding. When the data includes audio (i.e., speech), the audio encoder included in the capture device 3102 may actually perform audio encoding. In some practical scenarios, the capture device 3102 delivers the encoded video and audio data by multiplexing them together. In other practical scenarios, for example in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and encoded video data separately to the terminal device 3106.

[0213] In the content supply system 3100, the terminal device 310 receives and plays back encoded data. The terminal device 3106 can be a device with data receiving and retrieval capabilities, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination thereof, or a similar device capable of decoding the encoded data described above. For example, the terminal device 3106 may include the destination device 14 as described above. When the encoded data includes video, the video decoder 30 included in the terminal device is prioritized to perform video decoding. When the encoded data includes audio, the audio decoder included in the terminal device is prioritized to perform audio decoding.

[0214] For terminal devices having a display, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device can supply the decoded data to its display. For terminal devices not equipped with a display, such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, an external display 3126 is connected to it to receive and display the decoded data.

[0215] When each device in this system performs encoding or decoding, a picture encoding device or picture decoding device, as described in the embodiments described above, can be used.

[0216] Figure 12 shows the structure of an example terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, the protocol progression unit 3202 analyzes the transmission protocol of the stream. The protocol may include, but is not limited to, Real-Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real-Time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination of these, or similar.

[0217] After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As described above, in some practical scenarios, for example in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder 3206 and audio decoder 3208 without passing through the demultiplexing unit 3204.

[0218] Through demultiplexing, a video elementary stream (ES), an audio ES, and optionally a subtitle are generated. A video decoder 3206, including a video decoder 30 as described in the above embodiments, decodes the video ES by the decoding method represented in the above embodiments to generate video frames and supplies this data to the synchronization unit 3212. An audio decoder 3208 decodes the audio ES to generate audio frames and supplies this data to the synchronization unit 3212. Alternatively, the video frames may be stored in a buffer (not shown in Figure Y) before supplying them to the synchronization unit 3212. Similarly, the audio frames may be stored in a buffer (not shown in Figure Y) before supplying them to the synchronization unit 3212.

[0219] The synchronization unit 3212 synchronizes video and audio frames and supplies video / audio to the video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video and audio information. The information may be encoded within the syntax using timestamps for the presentation of encoded audio and visual data, and timestamps for the delivery of the data stream itself.

[0220] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles, synchronizes them with the video and audio frames, and supplies the video / audio / subtitle to the video / audio / subtitle display 3216.

[0221] The present invention is not limited to the system described above, and either the picture encoding device or the picture decoding device in the above-described embodiment can be incorporated into other systems, such as a car system.

[0222] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or codes in a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including, for example, any medium that facilitates the transfer of computer programs from one place to another according to a communication protocol. In this way, the computer-readable medium may generally correspond to (1) a non-transient tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementation of the technology described in this disclosure. A computer program product may include a computer-readable medium.

[0223] For example, but not limited to, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other media accessible by a computer that can be used to store desired program code in the form of instructions or data structures. Also, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but rather are directed toward non-temporary, tangible storage media. The terms "disk" and "disc" used herein include compact discs (CDs), laserdiscs (registered trademarks), optical discs, digital multipurpose discs (DVDs), floppy disks, and Blu-ray discs. A "disk" typically reproduces data magnetically, while a "disc" reproduces data optically using a laser. Any combination of the above should also be included within the scope of computer-readable media.

[0224] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated circuits or separate logic circuits. Thus, the term “processor” as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some embodiments, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, these techniques can be fully implemented in one or more circuits or logic elements.

[0225] The technology of this disclosure can be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to highlight the functional aspects of devices configured to perform the disclosed technology, but do not necessarily require implementation by different hardware units. Rather, as described above, various units may be combined within a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with suitable software and / or firmware.

[0226] While several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods can be embodied in many other specific forms without departing from the spirit or scope of this disclosure. These examples should be considered illustrative and non-limiting, and the invention should not be limited to the details given herein. For example, various elements or components may be combined or integrated within another system, or certain features may be omitted or not realized.

[0227] In addition, the technologies, systems, subsystems, and methods described and illustrated individually or separately in various embodiments may be combined with or integrated with other systems, modules, technologies, or methods without departing from the scope of this disclosure. Other items described or discussed as being combined, directly combined, or communicating with one another may be indirectly combined or communicated through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Other examples of modifications, substitutions, and alterations can be seen by those skilled in the art and may be made without departing from the spirit and scope of what is disclosed herein.

[0228] While several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods can be embodied in many other specific forms without departing from the spirit or scope of this disclosure. These examples should be considered illustrative and non-limiting, and the invention should not be limited to the details given herein. For example, various elements or components may be combined or integrated within another system, or certain features may be omitted or not realized.

[0229] In addition, the technologies, systems, subsystems, and methods described and illustrated individually or separately in various embodiments may be combined with or integrated with other systems, modules, technologies, or methods without departing from the scope of this disclosure. Other items described or discussed as being combined, directly combined, or communicating with one another may be indirectly combined or communicated through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Other examples of modifications, substitutions, and alterations can be seen by those skilled in the art and may be made without departing from the spirit and scope of what is disclosed herein. [Explanation of Symbols]

[0230] 10 Coding Systems 12. Source device 13 Communication Channels 14 Destination device 16 Computer-readable media 18 Video Sources 20 Video Encoders 22 Output Interfaces 28 Input Interfaces 30 video decoders 32 Display Devices 40 Mode Selection Unit 42 Motion Estimation Unit 44 Motion compensation unit 46 Intra Prediction Units 48 division units 50 Adder 52 Conversion Processing Unit 54 Quantization Units 56 Entropy Coding Unit 58 Inverse Quantization Unit 60 Reverse Conversion Unit 62 Adder 64 Reference frame memory 70 Entropy Decoding Unit 72 Motion Compensation Unit 74 Intra Prediction Units 76 Inverse Quantization Unit 78 Reverse Conversion Unit 80 Adder 82 Reference frame memory 101 Decision Unit 310 Terminal devices 400 DMVR method 402 Current Block 404 Current Picture 406 First reference block 408 First reference picture 410 Second reference block 412 Second reference picture 414 Bilateral Template Blocks 416 First refined reference block 418 Second refined reference block 420 Prediction block 500 Coding device 510 Inlet port 520 Receiver unit (Rx) 530 Processor 540 Transmitter unit (Tx) 550 Outlet port 560 Memory 570 Coding module 600 Coding method 700 Encoding method 800 Decoding method 1000 Device 1002 Processor 1004 Memory 1006 Code and data 1008 Operating system 1010 Application program 1012 Bus 1014 Secondary storage device 1018 Display 1020 Image sensing device<� 1022 Sound sensing device 3100 Content supply system 3102 Capture device 3104 Communication link 3106 Terminal device 3108 Smartphone or pad 3110 Computer or laptop 3112 Network video recorder (NVR) / Digital video recorder (DVR) 3114 TV 3116 Set-top box (STB) 3118 Video conferencing system 3120 Video surveillance system 3122 Personal digital assistant (PDA) 3124 In-vehicle devices 3126 Display 3202 Protocol Progress Unit 3204 Reverse Multiplexing Unit 3206 Video Decoder 3208 Audio Decoder 3210 Subtitle Decoder 3212 Synchronization Unit 3214 Video / Audio Display 3216 Video / Audio / Subtitle Display

Claims

1. A method for predicting the current image block within the current picture of a video, A step of determining whether the current picture is temporally between a first reference picture and a second reference picture, and whether the first temporal distance and the second temporal distance are the same, wherein the first temporal distance is between the current picture and the first reference picture, and the second temporal distance is between the current picture and the second reference picture. A method comprising the steps of: performing a motion vector refinement (DMVR) procedure to determine a predicted block of the current image block when it is determined that the current picture is temporally between the first reference picture and the second reference picture and that the first temporal distance and the second temporal distance are the same.

2. The method according to claim 1, further comprising the step of performing motion compensation using a first initial motion vector and a second initial motion vector to determine the predicted block of the current image block when it is determined that the first temporal distance and the second temporal distance are different distances, or that the current picture is not temporally between the first reference picture and the second reference picture.

3. The step of performing a motion vector refinement (DMVR) procedure is: A step of determining a best matching block pair indicated by the best motion vector from a plurality of pairs of reference blocks, wherein each of the plurality of pairs of reference blocks includes a first reference block located in the first reference picture and determined based on a first initial motion vector, and a second reference block located in the second reference picture and determined based on a second initial motion vector, The method according to claim 1, comprising the step that the best motion vector includes a first refined motion vector and a second refined motion vector.

4. The step of performing a motion vector refinement (DMVR) procedure is: A step of determining the positions of pairs of reference blocks from the positions of M pairs of reference blocks based on a matching cost criterion, as the positions of a first refined reference block and the positions of a second refined reference block, wherein the positions of the M pairs of reference blocks are determined based on a first initial motion vector, a second initial motion vector, and the position of the current image block, and the position of each pair of reference blocks includes the position of the first reference block and the position of the second reference block, the first reference block is contained within the first reference image, the second reference block is contained within the second reference image, for each pair of reference blocks, the first position offset (delta0x, delta0y) and the second position offset (delta1x, delta1y) are equal in magnitude and opposite in sign, the first position offset (delta0x, delta0y) represents the offset of the position of the first reference block relative to the position of the first initial reference block, and the second position offset (delta1x, The method according to claim 1, comprising a step, wherein delta(y) represents the offset of the position of the second reference block relative to the position of the second initial reference block, and M is an integer of 1 or more.

5. The method according to any one of claims 2 to 4, wherein the initial motion information of the current image block includes the first initial motion vector, the first reference index, the second initial motion vector, and the second reference index, the first reference index indicating the first reference picture, and the second reference index indicating the second reference picture.

6. The first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or The first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or The method according to any one of claims 1 to 5, wherein the first reference picture is a past reference image and the second reference picture is a future reference image, or the first reference picture is a future reference image and the second reference picture is a past reference image.

7. The first temporal distance represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance represents the POC distance between the current picture and the second reference picture, or The method according to any one of claims 1 to 6, wherein the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

8. The step of determining whether the current picture is temporally between the first reference picture and the second reference picture, and whether the first temporal distance and the second temporal distance are the same, The first temporal distance and the second temporal distance are determined by a step of determining whether they are equal in magnitude and opposite in sign, wherein the first temporal distance is expressed with respect to the difference between the picture order count value of the current picture and the picture order count value of the first reference image, and the second temporal distance is expressed with respect to the difference between the picture order count value of the current picture and the picture order count value of the second reference image, The step of performing a motion vector refinement (DMVR) procedure is: The method according to any one of claims 1 to 7, further comprising the step of performing a motion vector refinement (DMVR) procedure to determine a predicted block of the current image block when the first temporal distance and the second temporal distance are determined to be equal in magnitude and opposite in sign.

9. A method for predicting the current image block within the current picture of a video, A step of determining whether the current picture is temporally between a first reference picture and a second reference picture, and whether the first temporal distance and the second temporal distance are the same, wherein the first temporal distance is between the current picture and the first reference picture, and the second temporal distance is between the current picture and the second reference picture. A method comprising the steps of performing motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block of the current image block when it is determined that the first temporal distance and the second temporal distance are different distances, or that the current picture is not temporally between the first reference picture and the second reference picture.

10. The method according to claim 9, wherein the initial motion information of the current image block includes the first initial motion vector, the first reference index, the second initial motion vector, and the second reference index, the first reference index indicating the first reference picture, and the second reference index indicating the second reference picture.

11. The first temporal distance represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance represents the POC distance between the current picture and the second reference picture, or The method according to claim 9 or 10, wherein the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

12. A method for predicting the current image block within the current picture of a video, A step of determining whether a first temporal distance is equal to a second temporal distance, wherein the first temporal distance is expressed in terms of the difference between the picture order count value of the current picture and the picture order count value of a first reference image, and the second temporal distance is expressed in terms of the difference between the picture order count value of the second reference image and the picture order count value of the current picture. A method comprising the step of performing a motion vector refinement (DMVR) procedure to determine a predicted block of the current image block when the first temporal distance is determined to be equal to the second temporal distance.

13. The method according to claim 12, further comprising the step of determining a predicted block of the current image block by performing motion compensation using a first initial motion vector and a second initial motion vector when it is determined that the first temporal distance is not equal to the second temporal distance.

14. The step of performing a motion vector refinement (DMVR) procedure is: A step of determining a pair of best-matching blocks pointed to by the best motion vector from a plurality of pairs of reference blocks, wherein the pair of reference blocks includes a first reference block determined based on a first initial motion vector of the first reference picture and a second reference block determined based on a second initial motion vector of the second reference picture. The method according to claim 12, comprising the step, wherein the best motion vector includes a first refined motion vector and a second refined motion vector.

15. The method according to claim 13 or 14, wherein the initial motion information of the current image block includes the first initial motion vector, a first reference index, the second initial motion vector, and the second reference index, the first reference index indicating the first reference picture, and the second reference index indicating the second reference picture.

16. A method for predicting the current image block within the current picture of a video, A step of determining whether a first temporal distance is equal to a second temporal distance, wherein the first temporal distance is expressed in terms of the difference between the picture order count value of the current picture and the picture order count value of a first reference image, and the second temporal distance is expressed in terms of the difference between the picture order count value of the second reference image and the picture order count value of the current picture. A method comprising the step of not performing a motion vector refinement (DMVR) procedure on the current image block when it is determined that the first temporal distance is not equal to the second temporal distance.

17. A method for encoding video images, The steps of obtaining a predicted block of the current image block by performing an interpretation of the current image block in the current picture of the video according to the method of any one of claims 1 to 15, A method comprising the steps of generating a bitstream including the residual between the current image block and the predicted block, and an index for indicating initial motion information.

18. A method for decoding a video image from a bitstream, The steps include analyzing the bitstream to obtain an index for indicating initial motion information, and the residual between the current image block and the predicted block of the current image block, A step of obtaining a predicted block of the current image block by performing an interpretation of the current image block in the current picture of a video according to the method of any one of claims 1 to 15, A method comprising the step of reconstructing the current image block based on the residual and the predicted block.

19. A coding method implemented by a coding device, A step of determining the value of a syntax element indicating whether or not the method according to any one of claims 1, 3 to 8, and 12 to 15 is effective, A method comprising the step of generating a bitstream containing the aforementioned syntax elements.

20. The method according to claim 19, wherein the syntax element is signaled at one of the following levels: sequence parameter set (SPS) level, picture parameter set (PPS) level, slice header, coding tree unit (CTU) syntax, or coding unit (CU) syntax.

21. A decoding method implemented by a decoding device, A step of analyzing a bitstream for syntax elements indicating whether or not the method according to any one of claims 1, 3 to 8, and 12 to 15 is effective, A method comprising the step of adaptively enabling or disabling a decoder-side motion vector refinement (DMVR) procedure according to the aforementioned syntax elements.

22. The method according to claim 21, wherein the syntax element is obtained from one of the following: the sequence parameter set (SPS) level of the bitstream, the picture parameter set (PPS) level of the bitstream, a slice header, a coding tree unit (CTU) syntax, or a coding unit (CU) syntax.

23. A coding device, Memory for storing instructions, A processor coupled to the memory, wherein the processor executes instructions stored in the memory. The processor is instructed to determine whether the current picture is temporally between a first reference picture and a second reference picture, and whether the first temporal distance and the second temporal distance are the same, wherein the first temporal distance (TD0) is between the current picture and the first reference picture (RefPic0), and the second temporal distance (TD1) is between the current picture and the second reference picture (RefPic1). A coding device including a processor configured to cause the processor to perform a motion vector refinement (DMVR) procedure to determine a predicted block of the current image block in the current picture when it is determined that the current picture is temporally between the first reference picture and the second reference picture, and the first temporal distance and the second temporal distance are the same.

24. The coding device according to claim 23, wherein the processor is further configured to perform motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block of the current image block when it is determined that the first temporal distance and the second temporal distance are different, or that the current picture is not temporally between the first reference picture and the second reference picture.

25. With regard to performing the motion vector refinement (DMVR) procedure, the processor: It is configured to determine the best matching block pair indicated by the best motion vector from a plurality of pairs of reference blocks, wherein the pair of reference blocks includes a first reference block located in the first reference picture and determined based on a first initial motion vector, and a second reference block located in the second reference picture and determined based on a second initial motion vector. The device according to claim 23, wherein the best motion vector includes a first refined motion vector and a second refined motion vector.

26. Regarding the execution of the motion vector refinement (DMVR) procedure, the processor specifically: The device according to claim 23, configured to determine the positions of a pair of reference blocks from the positions of M pairs of reference blocks based on a matching cost criterion, the positions of the M pairs of reference blocks being determined based on the first initial motion vector, the second initial motion vector, and the position of the current image block, the position of each pair of reference blocks including the position of the first reference block and the position of the second reference block, the first reference block being contained within the first reference image, the second reference block being contained within the second reference image, and for each pair of reference blocks, the first position offset and the second position offset are equal in magnitude and opposite in sign, the first position offset represents the offset of the position of the first reference block relative to the position of the first initial reference block, and the second position offset represents the offset of the position of the second reference block relative to the position of the second initial reference block, where M is an integer of 1 or more.

27. The device according to any one of claims 24 to 26, wherein the processor is further configured to acquire initial motion information of the current image block in the current picture, the initial motion information comprising a first initial motion vector, a first reference index, a second initial motion vector, and a second reference index, the first reference index indicating the first reference picture, and the second reference index indicating the second reference picture.

28. The first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or the first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or The device according to any one of claims 23 to 27, wherein the first reference picture is a past reference image and the second reference picture is a future reference image, or the first reference picture is a future reference image and the second reference picture is a past reference image.

29. The first temporal distance represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance represents the POC distance between the current picture and the second reference picture, or The device according to any one of claims 23 to 28, wherein the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

30. The processor is configured to determine whether the first temporal distance and the second temporal distance are equal in magnitude and opposite in sign, wherein the first temporal distance is expressed with respect to the difference between the picture sequence count value of the current picture and the picture sequence count value of the first reference image, and the second temporal distance is expressed with respect to the difference between the picture sequence count value of the second reference image and the picture sequence count value of the current picture. The device according to any one of claims 23 to 29, wherein the processor is configured to perform a motion vector refinement (DMVR) procedure to determine a predicted block for the current image block when it is determined that the first temporal distance and the second temporal distance are equal in magnitude and opposite in sign.

31. A coding device, Memory for storing instructions, A processor coupled to the memory, wherein the processor executes instructions stored in the memory. The processor is made to determine whether the current picture is temporally between a first reference picture and a second reference picture, and whether the first temporal distance and the second temporal distance are the same, wherein the first temporal distance is between the current picture and the first reference picture, and the second temporal distance is between the current picture and the second reference picture. A coding device including a processor configured to cause the processor to perform motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block of the current image block in the current picture when it is determined that the first temporal distance and the second temporal distance are different distances, or that the current picture is not temporally between the first reference picture and the second reference picture.

32. The coding device according to claim 31, wherein the processor is further configured to acquire initial motion information of the current image block in the current picture, the initial motion information comprising a first initial motion vector, a first reference index, a second initial motion vector, and a second reference index, the first reference index indicating the first reference picture, and the second reference index indicating the second reference picture.

33. The first temporal distance represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance represents the POC distance between the current picture and the second reference picture, or The coding device according to claim 31 or 32, wherein the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

34. A coding device, Memory for storing instructions, A processor coupled to the memory, wherein the processor executes instructions stored in the memory. The processor is made to determine whether the first temporal distance is equal to the second temporal distance, the first temporal distance being expressed in terms of the difference between the picture order count value of the current picture and the picture order count value of the first reference image, and the second temporal distance being expressed in terms of the difference between the picture order count value of the second reference image and the picture order count value of the current picture. A coding device including a processor configured to cause the processor to perform a motion vector refinement (DMVR) procedure to determine a predicted block for the current image block when the first temporal distance is determined to be equal to the second temporal distance.

35. The aforementioned processor, The device according to claim 34, further configured to perform motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block of the current image block when it is determined that the first temporal distance is not equal to the second temporal distance.

36. With regard to performing the motion vector refinement (DMVR) procedure, the processor: It is configured to determine the best matching block pair indicated by the best motion vector from a plurality of pairs of reference blocks, wherein the pair of reference blocks includes a first reference block determined based on a first initial motion vector of the first reference picture, and a second reference block determined based on a second initial motion vector of the second reference picture. The device according to claim 34, wherein the best motion vector includes a first refined motion vector and a second refined motion vector.

37. The aforementioned processor, The device according to claim 35 or 36, further configured to obtain initial motion information of the current image block in the current picture, wherein the initial motion information includes the first initial motion vector, a first reference index, the second initial motion vector, and the second reference index, the first reference index indicating the first reference picture, and the second reference index indicating the second reference picture.

38. A coding device, Memory for storing instructions, A processor coupled to the memory, wherein the processor executes instructions stored in the memory. The processor is made to determine whether the first temporal distance is equal to the second temporal distance, the first temporal distance being expressed in terms of the difference between the picture order count value of the current picture and the picture order count value of the first reference image, and the second temporal distance being expressed in terms of the difference between the picture order count value of the second reference image and the picture order count value of the current picture. A coding device including a processor configured to prevent the processor from performing a motion vector refinement (DMVR) procedure on the current image block when it is determined that the first temporal distance is not equal to the second temporal distance.

39. An apparatus for interpretation of the current image block within the current picture of a video, A determination unit configured to determine whether the current picture is temporally between a first reference picture and a second reference picture, and whether the first temporal distance and the second temporal distance are the same, wherein the first temporal distance (TD0) is between the current picture and the first reference picture (RefPic0), and the second temporal distance (TD1) is between the current picture and the second reference picture (RefPic1), Apparatus including an interpredictive processing unit configured to perform a motion vector refinement (DMVR) procedure to determine a predicted block of the current image block when the current picture is temporally between the first reference picture and the second reference picture and it is determined that the first temporal distance and the second temporal distance are the same.

40. The apparatus according to claim 39, wherein the interpretation processing unit is further configured to perform motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block of the current image block when it is determined that the first temporal distance and the second temporal distance are different, or that the current picture is not temporally between the first reference picture and the second reference picture.

41. Regarding the execution of the motion vector refinement (DMVR) procedure, the interpretation processing unit specifically performs the following: It is configured to determine the best matching block pair indicated by the best motion vector from a plurality of pairs of reference blocks, wherein the pair of reference blocks includes a first reference block located in the first reference picture and determined based on a first initial motion vector, and a second reference block located in the second reference picture and determined based on a second initial motion vector. The apparatus according to claim 39, wherein the best motion vector includes a first refined motion vector and a second refined motion vector.

42. Regarding the execution of the motion vector refinement (DMVR) procedure, the interpretation processing unit specifically performs the following: The apparatus according to claim 39, configured to determine the positions of a pair of reference blocks from the positions of M pairs of reference blocks based on a matching cost criterion, the positions of the M pairs of reference blocks being determined based on a first initial motion vector, a second initial motion vector, and the position of the current image block, the position of each pair of reference blocks including the position of the first reference block and the position of the second reference block, the first reference block being contained within the first reference image, the second reference block being contained within the second reference image, and for each pair of reference blocks, the first position offset and the second position offset are equal in magnitude and opposite in sign, the first position offset represents the offset of the position of the first reference block relative to the position of the first initial reference block, and the second position offset represents the offset of the position of the second reference block relative to the position of the second initial reference block, where M is an integer of 1 or more.

43. The apparatus according to any one of claims 40 to 42, wherein the interpretation processing unit is further configured to acquire initial motion information of the current image block in the current picture, the initial motion information comprising a first initial motion vector, a first reference index, a second initial motion vector, and a second reference index, the first reference index indicating the first reference picture, and the second reference index indicating the second reference picture.

44. The first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or the first reference picture is a reference image that precedes the current picture in time, and the second reference picture is a reference image that precedes the current picture in time, or The apparatus according to any one of claims 39 to 43, wherein the first reference picture is a past reference image and the second reference picture is a future reference image, or the first reference picture is a future reference image and the second reference picture is a past reference image.

45. The first temporal distance represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance represents the POC distance between the current picture and the second reference picture, or The apparatus according to any one of claims 39 to 44, wherein the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

46. The determination unit is configured to determine whether the first temporal distance and the second temporal distance are equal in magnitude and opposite in sign, wherein the first temporal distance is expressed in terms of the difference between the picture sequence count value of the current picture and the picture sequence count value of the first reference image, and the second temporal distance is expressed in terms of the difference between the picture sequence count value of the current picture and the picture sequence count value of the second reference image. The apparatus according to any one of claims 39 to 45, wherein the interpretation processing unit is configured to perform a motion vector refinement (DMVR) procedure to determine a predicted block for the current image block when it is determined that the first temporal distance and the second temporal distance are equal in magnitude and opposite in sign.

47. An apparatus for interpretation of the current image block within the current picture of a video, A determination unit configured to determine whether the current picture is temporally between a first reference picture and a second reference picture, and whether a first temporal distance and a second temporal distance are the same, wherein the first temporal distance is between the current picture and the first reference picture, and the second temporal distance is between the current picture and the second reference picture. Apparatus including an interprediction processing unit configured to perform motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block of the current image block when it is determined that the first temporal distance and the second temporal distance are different distances, or that the current picture is not temporally between the first reference picture and the second reference picture.

48. The apparatus according to claim 47, wherein the initial motion information of the current image block includes the first initial motion vector, the first reference index, the second initial motion vector, and the second reference index, wherein the first reference index indicates the first reference picture and the second reference index indicates the second reference picture.

49. The first temporal distance represents the picture order count (POC) distance between the current picture and the first reference picture, and the second temporal distance represents the POC distance between the current picture and the second reference picture, or The apparatus according to claim 47 or 48, wherein the first temporal distance (TD0) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC0) of the first reference image, and the second temporal distance (TD1) is expressed in terms of the difference between the picture order count value (POCc) of the current picture and the picture order count value (POC1) of the second reference image.

50. An apparatus for interpretation of the current image block within the current picture of a video, A determination unit configured to determine whether a first temporal distance is equal to a second temporal distance, wherein the first temporal distance is expressed in terms of the difference between the picture order count value of the current picture and the picture order count value of a first reference image, and the second temporal distance is expressed in terms of the difference between the picture order count value of the second reference image and the picture order count value of the current picture, Apparatus including an interpredictive processing unit configured to perform a motion vector refinement (DMVR) procedure to determine a predicted block for the current image block when the first temporal distance is determined to be equal to the second temporal distance.

51. The apparatus according to claim 50, wherein the interpretation processing unit is further configured to perform motion compensation using a first initial motion vector and a second initial motion vector to determine a predicted block of the current image block when it is determined that the first temporal distance is not equal to the second temporal distance.

52. Apparatus according to claim 50, wherein, with respect to performing a motion vector refinement (DMVR) procedure, the interpredictive processing unit is configured to determine a pair of best-matching blocks from a plurality of pairs of reference blocks, the pair of reference blocks comprising a first reference block of the first reference picture, determined based on a first initial motion vector, and a second reference block of the second reference picture, determined based on a second initial motion vector, the best motion vector comprising a first refined motion vector and a second refined motion vector.

53. The apparatus according to claim 51 or 52, wherein the interpretation processing unit is further configured to acquire initial motion information of the current image block in the current picture, the initial motion information comprising the first initial motion vector, a first reference index, the second initial motion vector, and the second reference index, the first reference index indicating the first reference picture, and the second reference index indicating the second reference picture.

54. An apparatus for interpretation of the current image block within the current picture of a video, A determination unit configured to determine whether a first temporal distance is equal to a second temporal distance, wherein the first temporal distance is expressed in terms of the difference between the picture order count value of the current picture and the picture order count value of a first reference image, and the second temporal distance is expressed in terms of the difference between the picture order count value of the second reference image and the picture order count value of the current picture, Apparatus comprising an interpredictive processing unit configured not to perform a motion vector refinement (DMVR) procedure on the current image block when it is determined that the first temporal distance is not equal to the second temporal distance.

55. An encoding device for encoding video images, An apparatus according to any one of claims 39 to 53 for obtaining a predicted block of the current image block, An encoding apparatus including an entropy coding unit configured to generate a bitstream including the residual between the current image block and the predicted block, and an index for indicating initial motion information.

56. A decoding device for decoding video images from a bitstream, An entropy decoding unit configured to analyze the bitstream, an index for indicating initial motion information, and the residual between the current image block and the predicted block of the current image block, An apparatus according to any one of claims 39 to 53 for obtaining the predicted block of the current image block, A decoding apparatus comprising an image reconstruction unit configured to reconstruct the current image block based on the residual and the predicted block.

57. An encoding device, Determining the value of a syntax element for enabling or disabling the method according to any one of claims 1, 3 to 8, and 12 to 15, Generate a bitstream containing the aforementioned syntax elements. An encoding device comprising one or more processing circuits configured as such.

58. The encoding device according to claim 57, wherein the syntax element is signaled at one of the following levels: sequence parameter set (SPS) level, picture parameter set (PPS) level, slice header, coding tree unit (CTU) syntax, or coding unit (CU) syntax.

59. A decoding device, Analyze the bitstream for syntax elements to indicate whether or not the method described in any one of claims 1, 3 to 8, and 12 to 15 is effective, Adaptively enable or disable the decoder-side motion vector refinement (DMVR) procedure according to the aforementioned syntax elements. A decoding device comprising one or more processing circuits configured as follows.

60. The decoding device according to claim 59, wherein the syntax element is obtained from one of the following: the sequence parameter set (SPS) level of the bitstream, the picture parameter set (PPS) level of the bitstream, a slice header, a coding tree unit (CTU) syntax, or a coding unit (CU) syntax.

61. A computer-readable medium for storing computer-readable instructions that, when executed in a processor, perform the steps described in any one of claims 1 to 22.

Images