Image prediction method and device, and codec

Abstract

The invention provides an image prediction method and device, and a codec. The method comprises the following steps: acquiring initial motion information of a current image block; when an early termination condition is satisfied, determining the positions of an initial forward reference block and an initial backward reference block of the current image block as the positions of a target forward reference block and a target backward reference block of the current image block, the positions of the initial forward reference block and the initial backward reference block of the current image block are obtained based on the initial motion information of the current image block; and obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block of the current image block. According to the method, the image prediction accuracy is improved, and meanwhile, the image prediction complexity can be reduced.

Description

[0001] This application is a divisional application. The application number of the original application is 201880084937.8, and the original application date is December 27, 2018. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of video coding and decoding technology, and in particular to an image prediction method, device and codec. Background Art

[0003] Through video compression technology, such as those described in MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Coding (AVC), ITU-TH.265 High Efficiency Video Coding (HEVC) standards and extensions of the standards, digital video information can be efficiently transmitted and received between devices. Typically, the images of a video sequence are divided into image blocks for encoding or decoding.

[0004] In video compression technology, in order to reduce or remove redundant information in video sequences, spatial prediction (intra prediction) and / or temporal prediction (inter prediction) based on image blocks are introduced. Among them, inter prediction modes may include but are not limited to: merge mode (Merge Mode) and non-merge mode (such as advanced motion vector prediction mode (AMVP mode)), and all of them use the method of multi-motion information competition for inter prediction.

[0005] In the inter-frame prediction process, a candidate motion information list (referred to as candidate list) including multiple sets of motion information (also called multiple candidate motion information) is introduced. For example, the encoder can use a set of motion information selected from the candidate list as or predict the motion information (such as motion vector) of the current image block to be encoded, and then obtain the reference image block (i.e., reference sample) of the current image block to be encoded. Correspondingly, the decoder can decode the indication information from the bitstream to obtain a set of motion information. Since the encoding overhead of motion information (i.e., the bit overhead occupied by the bitstream) is limited in the inter-frame prediction process, the accuracy of the motion information is affected to a certain extent, and thus the accuracy of image prediction is affected.

[0006] In order to improve the accuracy of image prediction, the existing decoder-side motion vector refinement (DMVR) technology can be used to correct the motion information. However, when using the DMVR scheme for image prediction, not only the template matching block must be calculated, but also the template matching block must be used to perform a search and matching process in the forward reference image and the backward reference image, respectively, resulting in a high search complexity. Therefore, how to reduce the complexity of image prediction while improving the accuracy of image prediction is a problem that needs to be solved. Summary of the invention

[0007] The embodiments of the present application provide an image prediction method, an apparatus, and a corresponding encoder and decoder, which can improve the accuracy of image prediction while reducing the complexity of image prediction to a certain extent, thereby improving encoding and decoding performance.

[0008] In a first aspect, an embodiment of the present application provides an image prediction method, the method comprising: obtaining initial motion information of a current image block; determining positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, the N forward reference blocks being located in a forward reference image, the N backward reference blocks being located in a backward reference image, and N being an integer greater than 1; determining, based on a matching cost criterion, from the positions of M pairs of reference blocks, a position of a pair of reference blocks as a position of a target forward reference block and a position of a target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block. The method comprises the following steps: a first position offset and a position of a backward reference block, and for each pair of reference block positions, a first position offset and a second position offset are in a mirror relationship, the first position offset represents a position offset of the position of the forward reference block relative to the position of the initial forward reference block, the second position offset represents a position offset of the position of the backward reference block relative to the position of the initial backward reference block, M is an integer greater than or equal to 1, and M is less than or equal to N; based on the pixel value (sample) of the target forward reference block and the pixel value (sample) of the target backward reference block, a predicted value of the pixel value of the current image block is obtained.

[0009] It is particularly important to explain that, in the embodiment of the present application, the positions of the N forward reference blocks include the position of an initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, so that the position offset of the initial forward reference block relative to the position of the initial forward reference block is 0, and when the position offset of the initial backward reference block relative to the position of the initial backward reference block is 0, 0 offset and 0 offset also satisfy a mirror relationship.

[0010] It can be seen that in the embodiment of the present application, the positions of the N forward reference blocks in the forward reference image and the positions of the N backward reference blocks in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the forward reference block relative to the position of the initial forward reference block is mirrored to the second position offset of the backward reference block relative to the position of the initial backward reference block. On this basis, a pair of reference block positions (for example, with the smallest matching cost) is determined from the N pairs of reference block positions as the position of the target forward reference block (i.e., the best forward reference block / forward prediction block) and the position of the target backward reference block (i.e., the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, thereby simplifying the image prediction process, thereby improving the accuracy of image prediction and reducing the complexity of image prediction.

[0011] In addition, it should be understood that the current image block (referred to as the current block) here can be understood as the image block currently being processed. For example, in the encoding process, it refers to the image block currently being encoded (encoding block); in the decoding process, it refers to the image block currently being decoded (decoding block).

[0012] In addition, it should be understood that the reference block here refers to the block that provides a reference signal for the current block. During the search process, it is necessary to traverse multiple reference blocks to find the best reference block. The reference block located in the forward reference image is called the forward reference block; the reference block located in the backward reference image is called the backward reference block.

[0013] In addition, it should be understood that a block that provides prediction for a current block is called a prediction block. For example, after traversing multiple reference blocks, the best reference block is found. This best reference block will provide prediction for the current block, and this block can be called a prediction block. The pixel value or sample value or sample signal in the prediction block is called a prediction signal.

[0014] In addition, it should be understood that the matching cost criterion here can be understood as a criterion for considering the matching cost between a pair of forward reference blocks and backward reference blocks, wherein the matching cost can be understood as the difference value between the two blocks, which can be regarded as the accumulation of the difference values ​​of the pixels at corresponding positions in the two blocks. The difference calculation method is generally based on the SAD (sum of absolute difference) criterion, or other criteria, such as SATD (Sum of Absolute Transform Difference), MR-SAD (mean-removed sum of absolute difference), SSD (sum of squared differences), etc.

[0015] In addition, it should be noted that the initial motion information of the current image block in the embodiment of the present application may include a motion vector MV and reference image indication information. Of course, the initial motion information may also include one or both of them. For example, when the codec end jointly agrees on the reference image, the initial motion information may only include the motion vector MV. The reference image indication information is used to indicate which one or which reconstructed images are used as reference images for the current block, and the motion vector indicates the position offset of the reference block position relative to the current block position in the reference image used, generally including a horizontal component offset and a vertical component offset. For example, (x, y) is used to represent MV, x represents the position offset in the horizontal direction, and y represents the position offset in the vertical direction. By adding the position of the current block to MV, the position of its reference block in the reference image can be obtained. The reference image indication information may include a reference image list and / or a reference image index corresponding to the reference image list. The reference image index is used to identify the reference image corresponding to the used motion vector in the specified reference image list (RefPicList0 or RefPicList1). An image may be referred to as a frame, and a reference image may be referred to as a reference frame.

[0016] The initial motion information of the current image block in the embodiment of the present application is initial bidirectional prediction motion information, that is, it includes motion information for forward and backward prediction directions. Here, the forward and backward prediction directions are two prediction directions of the bidirectional prediction mode. It can be understood that "forward" and "backward" correspond to the reference image list 0 (RefPicList0) and reference image list 1 (RefPicList1) of the current image, respectively.

[0017] In addition, it should be noted that the position of the initial forward reference block in the embodiment of the present application refers to the position of the reference block in the forward reference image obtained by adding the position of the current block to the initial motion MV offset; the position of the initial backward reference block in the embodiment of the present application refers to the position of the reference block in the backward reference image obtained by adding the position of the current block to the initial motion MV offset.

[0018] It should be understood that the executor of the method of the embodiment of the present application may be an image prediction device, such as a video encoder or a video decoder or an electronic device with video encoding and decoding functions, specifically, for example, an inter-frame prediction unit in a video encoder, or a motion compensation unit in a video decoder.

[0019] In combination with the first aspect, in certain implementations of the first aspect, the first position offset and the second position offset are in a mirror relationship, which can be understood as the first position offset and the second position offset being the same, for example, the direction of the first position offset (also called the vector direction) is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset.

[0020] In one example, the first position offset includes a first horizontal component offset and a first vertical component offset, and the second position offset includes a second horizontal component offset and a second vertical component offset, wherein the direction of the first horizontal component offset is opposite to the direction of the second horizontal component offset, and the amplitude of the first horizontal component offset is the same as the amplitude of the second horizontal component offset; the direction of the first vertical component offset is opposite to the direction of the second vertical component offset, and the amplitude of the first vertical component offset is the same as the amplitude of the second vertical component offset.

[0021] In another example, the first position offset and the second position offset are both zero.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: obtaining updated motion information of the current image block, the updated motion information including an updated forward motion vector and an updated backward motion vector, wherein the updated forward motion vector points to the position of the target forward reference block, and the updated backward motion vector points to the position of the target backward reference block.

[0023] In different examples, the updated motion information of the current image block is obtained based on the position of the target forward reference block, the position of the target backward reference block and the position of the current image block, or is obtained based on the first position offset and the second position offset corresponding to the determined pair of reference block positions.

[0024] It can be seen that the embodiment of the present application can obtain the corrected motion information of the current image block and improve the accuracy of the motion information of the current image block, which will also be beneficial to the prediction of other image blocks, such as improving the prediction accuracy of the motion information of other image blocks.

[0025] In combination with the first aspect, in some implementations of the first aspect, the positions of the N forward reference blocks include the position of an initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset relative to the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance; or,

[0026] The positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset by an integer pixel distance or a fractional pixel distance relative to the position of the initial backward reference block.

[0027] It should be noted that the positions of the N pairs of reference blocks include: the positions of pairs of initial forward reference blocks and initial backward reference blocks, and the positions of pairs of candidate forward reference blocks and candidate backward reference blocks, wherein the position offset of the candidate forward reference blocks in the forward reference image relative to the position of the initial forward reference block is mirrored to the position offset of the candidate backward reference blocks in the backward reference image relative to the position of the initial backward reference block.

[0028] In combination with the first aspect, in some implementations of the first aspect, the initial motion information includes forward prediction motion information and backward prediction motion information;

[0029] The determining the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block comprises:

[0030] Determining positions of N forward reference blocks in a forward reference image according to the forward prediction motion information and the position of the current image block, wherein the positions of the N forward reference blocks include a position of an initial forward reference block and positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset relative to the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance;

[0031] The positions of N backward reference blocks are determined in the backward reference image according to the backward prediction motion information and the position of the current image block, the positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset relative to the position of the initial backward reference block by an integer pixel distance or a fractional pixel distance.

[0032] In combination with the first aspect, in some implementations of the first aspect, the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction;

[0033] The determining the positions of N forward reference blocks and N backward reference blocks according to the initial motion information and the position of the current image block comprises:

[0034] Determining the position of an initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index according to the first motion vector and the position of the current image block; determining the positions of (N-1) candidate forward reference blocks in the forward reference image using the position of the initial forward reference block as a first search starting point, wherein the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks;

[0035] Determine the position of an initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index according to the second motion vector and the position of the current image block; and determine the positions of (N-1) candidate backward reference blocks in the backward reference image using the position of the initial backward reference block as a second search starting point, wherein the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0036] In combination with the first aspect, in some implementations of the first aspect, determining, according to a matching cost criterion, a pair of reference block positions from the M pairs of reference block positions as a position of a target forward reference block and a position of a target backward reference block of a current image block includes:

[0037] From the M pairs of reference block positions, determine the positions of a pair of reference blocks with the smallest matching error as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or

[0038] From the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks whose matching errors are less than or equal to a matching error threshold as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, where M is less than or equal to N.

[0039] In one example, the matching cost criterion is a criterion for minimizing the matching cost. For example, for the positions of M pairs of reference blocks, the difference between the pixel value of the forward reference block and the pixel value of the backward reference block in each pair of reference blocks is calculated; from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks with the smallest pixel value difference are determined as the position of the forward target reference block and the position of the backward target reference block of the current image block.

[0040] In another example, the matching cost criterion is a matching cost and an early termination criterion. For example, for the position of the nth pair of reference blocks (a forward reference block and a backward reference block), the difference between the pixel value of the forward reference block and the pixel value of the backward reference block is calculated, where n is an integer greater than or equal to 1 and less than or equal to N; when the pixel value difference is less than or equal to the matching error threshold, the position of the nth pair of reference blocks (a forward reference block and a backward reference block) is determined as the position of the forward target reference block and the position of the backward target reference block of the current image block.

[0041] In combination with the first aspect, in some implementations of the first aspect, the method is used to encode the current image block, and obtaining the initial motion information of the current image block includes: obtaining the initial motion information from a candidate motion information list of the current image block;

[0042] Alternatively, the method is used to decode the current image block, and before obtaining the initial motion information of the current image block, the method further includes: obtaining indication information from a code stream of the current image block, wherein the indication information is used to indicate the initial motion information of the current image block.

[0043] It can be seen that the image prediction method of the embodiment of the present application is not only applicable to the merge prediction mode (Merge) and / or the advanced motion vector prediction mode (AMVP), but can also be applied to other modes that use the motion information of the spatial reference block, the temporal reference block and / or the inter-view reference block to predict the motion information of the current image block, thereby improving the encoding and decoding performance.

[0044] A second aspect of the present application provides an image prediction method, comprising: obtaining initial motion information of a current image block;

[0045] Based on the initial motion information and the position of the current image block, the positions of N forward reference blocks and the positions of N backward reference blocks are determined, the N forward reference blocks are located in the forward reference image, the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1; based on the matching cost criterion, a pair of reference block positions are determined from the M pairs of reference block positions as the position of the target forward reference block and the position of the target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, a first position offset and a second position offset have a proportional relationship based on a time domain distance, the first position offset represents a position offset of the forward reference block relative to the initial forward reference block position; the second position offset represents a position offset of the backward reference block relative to the initial backward reference block position, and M is an integer greater than or equal to 1, and M is less than or equal to N; based on the pixel value of the target forward reference block and the pixel value of the target backward reference block, a predicted value of the pixel value of the current image block is obtained.

[0046] It is particularly important to explain that, in the embodiment of the present application, when the position offset of the initial forward reference block relative to the position of the initial forward reference block is 0, and the position offset of the initial backward reference block relative to the position of the initial backward reference block is 0, the 0 offset and the 0 offset also satisfy a mirror relationship or a proportional relationship based on the time domain distance. To describe it from another angle, for each pair of reference block positions in the (N-1) pairs of reference block positions, the first position offset and the second position offset have a proportional relationship / mirror relationship based on the time domain distance. The (N-1) pairs of reference block positions here do not include the position of the initial forward reference block and the position of the initial backward reference block.

[0047] It can be seen that in the embodiment of the present application, the positions of the N forward reference blocks in the forward reference image and the positions of the N backward reference blocks in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the forward reference block relative to the position of the initial forward reference block and the second position offset of the backward reference block relative to the position of the initial backward reference block have a proportional relationship based on the time domain distance (also referred to as a mirror relationship based on the time domain distance). On this basis, a pair of reference block positions (for example, with the minimum matching cost) is determined from the N pairs of reference block positions as the position of the target forward reference block (that is, the best forward reference block / forward prediction block) and the position of the target backward reference block (that is, the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, thereby simplifying the image prediction process, thereby improving the accuracy of image prediction and reducing the complexity of image prediction.

[0048] In combination with the second aspect, in some implementations of the second aspect, for each pair of reference blocks, the first position offset and the second position offset have a proportional relationship based on a time domain distance, including:

[0049] For each pair of reference blocks, the proportional relationship between the first position offset and the second position offset is determined based on the proportional relationship between the first time domain distance and the second time domain distance, wherein the first time domain distance represents the time domain distance between the current image to which the current image block belongs and the forward reference image; and the second time domain distance represents the time domain distance between the current image and the backward reference image.

[0050] In combination with the second aspect, in some implementations of the second aspect, the first position offset and the second position offset have a proportional relationship based on a time domain distance, including:

[0051] If the first time domain distance is the same as the second time domain distance, the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset; or,

[0052] If the first time domain distance is different from the second time domain distance, the direction of the first position offset is opposite to the direction of the second position offset, and the proportional relationship between the amplitude of the first position offset and the amplitude of the second position offset is based on the proportional relationship between the first time domain distance and the second time domain distance;

[0053] The first time domain distance represents the time domain distance between the current image to which the current image block belongs and the forward reference image; the second time domain distance represents the time domain distance between the current image and the backward reference image.

[0054] In combination with the second aspect, in certain implementations of the second aspect, the method also includes: obtaining updated motion information of the current image block, the updated motion information including an updated forward motion vector and an updated backward motion vector, wherein the updated forward motion vector points to the position of the target forward reference block, and the updated backward motion vector points to the position of the target backward reference block.

[0055] It can be seen that the embodiment of the present application can obtain the corrected motion information of the current image block and improve the accuracy of the motion information of the current image block, which will also be beneficial to the prediction of other image blocks, for example, improving the prediction accuracy of the motion information of other image blocks.

[0056] In combination with the second aspect, in some implementations of the second aspect, the positions of the N forward reference blocks include the position of an initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset relative to the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance; or,

[0057] The positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset by an integer pixel distance or a fractional pixel distance relative to the position of the initial backward reference block.

[0058] In combination with the second aspect, in certain implementations of the second aspect, the positions of the N pairs of reference blocks include: the positions of pairs of initial forward reference blocks and initial backward reference blocks, and the positions of pairs of candidate forward reference blocks and candidate backward reference blocks, wherein the position offset of the candidate forward reference blocks in the forward reference image relative to the position of the initial forward reference block has a proportional relationship based on time domain distance with the position offset of the candidate backward reference blocks in the backward reference image relative to the position of the initial backward reference block.

[0059] In conjunction with the second aspect, in some implementations of the second aspect, the initial motion information includes forward prediction motion information and backward prediction motion information;

[0060] The determining the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block comprises:

[0061] Determining positions of N forward reference blocks in a forward reference image according to the forward prediction motion information and the position of the current image block, wherein the positions of the N forward reference blocks include a position of an initial forward reference block and positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset relative to the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance;

[0062] The positions of N backward reference blocks are determined in the backward reference image according to the backward prediction motion information and the position of the current image block, the positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset relative to the position of the initial backward reference block by an integer pixel distance or a fractional pixel distance.

[0063] In combination with the second aspect, in some implementations of the second aspect, the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction;

[0064] The determining the positions of N forward reference blocks and N backward reference blocks according to the initial motion information and the position of the current image block comprises:

[0065] Determining the position of an initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index according to the first motion vector and the position of the current image block; determining the positions of (N-1) candidate forward reference blocks in the forward reference image using the position of the initial forward reference block as a first search starting point, wherein the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks;

[0066] Determine the position of an initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index according to the second motion vector and the position of the current image block; and determine the positions of (N-1) candidate backward reference blocks in the backward reference image using the position of the initial backward reference block as a second search starting point, wherein the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0067] In combination with the second aspect, in some implementations of the second aspect, determining, according to a matching cost criterion, a pair of reference block positions from the M pairs of reference block positions as a position of a target forward reference block and a position of a target backward reference block of a current image block includes:

[0068] From the M pairs of reference block positions, determine the positions of a pair of reference blocks with the smallest matching error as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or

[0069] From the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks whose matching errors are less than or equal to a matching error threshold as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, where M is less than or equal to N.

[0070] In one example, the matching cost criterion is a criterion for minimizing the matching cost. For example, for the positions of M pairs of reference blocks, the difference between the pixel value of the forward reference block and the pixel value of the backward reference block in each pair of reference blocks is calculated; from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks with the smallest pixel value difference are determined as the position of the forward target reference block and the position of the backward target reference block of the current image block.

[0071] In another example, the matching cost criterion is a matching cost and an early termination criterion. For example, for the position of the nth pair of reference blocks (a forward reference block and a backward reference block), the difference between the pixel value of the forward reference block and the pixel value of the backward reference block is calculated, where n is an integer greater than or equal to 1 and less than or equal to N; when the pixel value difference is less than or equal to the matching error threshold, the position of the nth pair of reference blocks (a forward reference block and a backward reference block) is determined as the position of the forward target reference block and the position of the backward target reference block of the current image block.

[0072] In combination with the second aspect, in some implementations of the second aspect, the method is used to encode the current image block, and obtaining the initial motion information of the current image block includes: obtaining the initial motion information from a candidate motion information list of the current image block;

[0073] Alternatively, the method is used to decode the current image block, and before obtaining the initial motion information of the current image block, the method further includes: obtaining indication information from a code stream of the current image block, wherein the indication information is used to indicate the initial motion information of the current image block.

[0074] A third aspect of the present application provides an image prediction method, comprising: obtaining i-th round motion information of a current image block;

[0075] Determine the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block, wherein the N forward reference blocks are located in the forward reference image, and the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1; determine the positions of a pair of reference blocks from the positions of the M pairs of reference blocks as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block based on a matching cost criterion, wherein the positions of each pair of reference blocks include a position of a forward reference block and a position of a backward reference block, and for each pair of reference block positions, the first position deviation is The first position offset is in a mirror relationship with the second position offset, the first position offset represents the position offset of the forward reference block relative to the position of the i-1th round target forward reference block, the second position offset represents the position offset of the backward reference block relative to the position of the i-1th round target backward reference block, the M is an integer greater than or equal to 1, and the M is less than or equal to N; according to the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block, the predicted value of the pixel value of the current image block is obtained, wherein j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0076] It is particularly important to note that in an embodiment of the present application, when the position offset of the initial forward reference block relative to the position of the initial forward reference block is 0, and the position offset of the initial backward reference block relative to the position of the initial backward reference block is 0, 0 offset and 0 offset also satisfy a mirror relationship.

[0077] It can be seen that in the embodiment of the present application, the positions of the N forward reference blocks in the forward reference image and the positions of the N backward reference blocks in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the forward reference block relative to the position of the initial forward reference block is mirrored to the second position offset of the backward reference block relative to the position of the initial backward reference block. On this basis, a pair of reference block positions (for example, with the smallest matching cost) is determined from the N pairs of reference block positions as the position of the target forward reference block (i.e., the best forward reference block / forward prediction block) and the position of the target backward reference block (i.e., the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, simplifies the image prediction process, thereby improving the accuracy of image prediction while reducing the complexity of image prediction. In addition, the embodiment of the present application can further improve the accuracy of the corrected motion vector MV through an iterative method, thereby further improving the encoding and decoding performance.

[0078] In combination with the third aspect, in certain implementations of the third aspect, if i=1, then the i-th round motion information is the initial motion information of the current image block; accordingly, the positions of the N forward reference blocks include the position of an initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset from the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance; or, the positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset from the position of the initial backward reference block by an integer pixel distance or a fractional pixel distance.

[0079] If i>1, the i-th round motion information includes: a forward motion vector pointing to the position of the i-1-th round target forward reference block and a backward motion vector pointing to the position of the i-1-th round target backward reference block; accordingly, the positions of the N forward reference blocks include the position of an i-1-th round target forward reference block and the positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset from the position of the i-1-th round target forward reference block by an integer pixel distance or a fractional pixel distance; or, the positions of the N backward reference blocks include the position of an i-1-th round target backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset from the position of the i-1-th round target backward reference block by an integer pixel distance or a fractional pixel distance.

[0080] It should be noted that, if the method is used to encode the current image block, the initial motion information of the current image block is obtained by the following method: determining the initial motion information from a candidate motion information list of the current image block; or, if the method is used to decode the current image block, the initial motion information of the current image block is obtained by the following method: obtaining indication information from the code stream of the current image block, wherein the indication information is used to indicate the initial motion information of the current image block.

[0081] In combination with the third aspect, in some implementations of the third aspect, obtaining a predicted value of a pixel value of the image block according to a pixel value of the j-th round target forward reference block and a pixel value of the j-th round target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1, includes:

[0082] When the iteration termination condition is met, the predicted value of the pixel value of the image block is obtained according to the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0083] In combination with the third aspect, in certain implementations of the third aspect, the first position offset is in a mirror image relationship with the second position offset, including: the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset.

[0084] In combination with the third aspect, in some implementations of the third aspect, the i-th round motion information includes a forward motion vector and a forward reference image index, and a backward motion vector and a backward reference image index;

[0085] The determining the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block comprises:

[0086] According to the forward motion vector and the position of the current image block, the position of the i-1th round target forward reference block of the current image block is determined in the forward reference image corresponding to the forward reference image index; the position of the i-1th round target forward reference block is used as the i-1th round target forward reference block. f Searching for a starting point, determining positions of (N-1) candidate forward reference blocks in the forward reference image, wherein the positions of the N forward reference blocks include a position of an i-1th round target forward reference block and the positions of the (N-1) candidate forward reference blocks;

[0087] According to the backward motion vector and the position of the current image block, the position of the i-1th round target backward reference block of the current image block is determined in the backward reference image corresponding to the backward reference image index; the position of the i-1th round target backward reference block is used as the i-1th round target backward reference block. b A search starting point is determined to determine the positions of (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of an i-1th round target backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0088] In combination with the third aspect, in some implementations of the third aspect, determining, according to a matching cost criterion, a pair of reference block positions from the M pairs of reference block positions as a position of an i-th round target forward reference block and a position of an i-th round target backward reference block of the current image block includes:

[0089] From the positions of the M pairs of reference blocks, determine the positions of a pair of reference blocks with the smallest matching error as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block; or

[0090] From the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks whose matching errors are less than or equal to the matching error threshold as the position of the i-th round target forward reference block of the current image block and the position of the i-th round target backward reference block of the current image block, where M is less than or equal to N.

[0091] A fourth aspect of the present application provides an image prediction method, comprising: obtaining i-th round motion information of a current image block;

[0092] Determine the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block, wherein the N forward reference blocks are located in the forward reference image, and the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1; based on the matching cost criterion, determine the positions of a pair of reference blocks from the positions of the M pairs of reference blocks as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block, wherein the positions of each pair of reference blocks include the position of a forward reference block and the position of a backward reference block, and for each pair of reference block positions, the first position offset and the second position offset have a time domain-based The first position offset represents the position offset of the forward reference block in the forward reference image relative to the position of the i-1th round target forward reference block; the second position offset represents the position offset of the backward reference block in the backward reference image relative to the position of the i-1th round target backward reference block, M is an integer greater than or equal to 1, and M is less than or equal to N; according to the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block, the predicted value of the pixel value of the current image block is obtained, wherein j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0093] It is particularly important to explain that, in the embodiment of the present application, when the position offset of the initial forward reference block relative to the position of the initial forward reference block is 0, and the position offset of the initial backward reference block relative to the position of the initial backward reference block is 0, the 0 offset and the 0 offset also satisfy a mirror relationship or a proportional relationship based on the time domain distance. To describe it from another angle, for each pair of reference block positions in the (N-1) pairs of reference block positions, the first position offset and the second position offset have a proportional relationship / mirror relationship based on the time domain distance. The (N-1) pairs of reference block positions here do not include the position of the initial forward reference block and the position of the initial backward reference block.

[0094] It can be seen that in the embodiment of the present application, the positions of the N forward reference blocks in the forward reference image and the positions of the N backward reference blocks in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the forward reference block relative to the position of the initial forward reference block and the second position offset of the backward reference block relative to the position of the initial backward reference block have a proportional relationship based on the time domain distance. On this basis, a pair of reference block positions (for example, with the smallest matching cost) are determined from the N pairs of reference block positions as the position of the target forward reference block (that is, the best forward reference block / forward prediction block) and the position of the target backward reference block (that is, the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, simplifies the image prediction process, thereby improving the accuracy of image prediction while reducing the complexity of image prediction. In addition, the embodiment of the present application can further improve the accuracy of the corrected motion vector MV through an iterative method, thereby further improving the encoding and decoding performance.

[0095] In combination with the fourth aspect, in certain implementations of the fourth aspect, if i=1, the i-th round motion information is the initial motion information of the current image block; if i>1, the i-th round motion information includes: a forward motion vector pointing to the position of the i-1-th round target forward reference block and a backward motion vector pointing to the position of the i-1-th round target backward reference block.

[0096] In combination with the fourth aspect, in some implementations of the fourth aspect, obtaining a predicted value of the pixel value of the image block according to the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1, includes:

[0097] When the iteration termination condition is met, the predicted value of the pixel value of the image block is obtained according to the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0098] In combination with the fourth aspect, in some implementations of the fourth aspect, the first position offset and the second position offset have a proportional relationship based on a time domain distance, including:

[0099] If the first time domain distance is the same as the second time domain distance, the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset; or,

[0100] If the first time domain distance is different from the second time domain distance, the direction of the first position offset is opposite to the direction of the second position offset, and the proportional relationship between the amplitude of the first position offset and the amplitude of the second position offset is based on the proportional relationship between the first time domain distance and the second time domain distance;

[0101] The first time domain distance represents the time domain distance between the current image to which the current image block belongs and the forward reference image; the second time domain distance represents the time domain distance between the current image and the backward reference image.

[0102] In combination with the fourth aspect, in certain implementations of the fourth aspect, the i-th round motion information includes a forward motion vector and a forward reference image index, and a backward motion vector and a backward reference image index;

[0103] The determining the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block comprises:

[0104] According to the forward motion vector and the position of the current image block, the position of the i-1th round target forward reference block of the current image block is determined in the forward reference image corresponding to the forward reference image index; the position of the i-1th round target forward reference block is used as the i-1th round target forward reference block. f Searching for a starting point, determining positions of (N-1) candidate forward reference blocks in the forward reference image, wherein the positions of the N forward reference blocks include a position of an i-1th round target forward reference block and the positions of the (N-1) candidate forward reference blocks;

[0105] According to the backward motion vector and the position of the current image block, the position of the i-1th round target backward reference block of the current image block is determined in the backward reference image corresponding to the backward reference image index; the position of the i-1th round target backward reference block is used as the i-1th round target backward reference block. b The search starting point determines the positions of (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of an i-1th round target backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0106] In combination with the fourth aspect, in some implementations of the fourth aspect, determining, according to a matching cost criterion, a pair of reference block positions from the positions of the M pairs of reference blocks as a position of an i-th round target forward reference block and a position of an i-th round target backward reference block of the current image block includes:

[0107] From the positions of the M pairs of reference blocks, determine the positions of a pair of reference blocks with the smallest matching error as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block; or

[0108] From the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks whose matching errors are less than or equal to the matching error threshold as the position of the i-th round target forward reference block of the current image block and the position of the i-th round target backward reference block of the current image block, where M is less than or equal to N.

[0109] The fifth aspect of the present application provides an image prediction device, comprising a plurality of functional units for implementing any one of the methods of the first aspect. For example, the image prediction device may include: a first acquisition unit, for acquiring initial motion information of a current image block; a first search unit, for determining positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, wherein the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1; based on a matching cost criterion, determining a pair of reference block positions from the positions of the M pairs of reference blocks as the position of a target forward reference block and the position of a target backward reference block of the current image block, wherein each pair of reference block positions includes a The invention relates to a forward reference block and a backward reference block, and for each pair of reference block positions, a first position offset and a second position offset are in a mirror relationship, the first position offset represents a position offset of the forward reference block relative to the position of an initial forward reference block, the second position offset represents a position offset of the backward reference block relative to the position of the initial backward reference block, M is an integer greater than or equal to 1, and M is less than or equal to N; a first prediction unit, used to obtain a predicted value of a pixel value of the current image block based on a pixel value of the target forward reference block and a pixel value of the target backward reference block.

[0110] In different application scenarios, the image prediction device is applied to, for example, a video encoding device (video encoder) or a video decoding device (video decoder).

[0111] The sixth aspect of the present application provides an image prediction device, comprising a plurality of functional units for implementing any one of the methods of the second aspect. For example, the image prediction device may include: a second acquisition unit, for acquiring initial motion information of the current image block; a second search unit, for determining the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, wherein the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1; based on a matching cost criterion, determining a pair of reference block positions from the positions of the M pairs of reference blocks as the position of the target forward reference block and the position of the target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block. The invention relates to a method for generating a prediction unit for a current image block, wherein the prediction unit comprises a first prediction unit and a second prediction unit for obtaining a predicted value of a pixel value of the current image block based on a pixel value of the target forward reference block and a pixel value of the target backward reference block, and for each pair of reference block positions, a first position offset and a second position offset have a proportional relationship based on a time domain distance, the first position offset represents a position offset of the position of the forward reference block relative to the position of the initial forward reference block; the second position offset represents a position offset of the position of the backward reference block relative to the position of the initial backward reference block, wherein M is an integer greater than or equal to 1, and M is less than or equal to N; and a second prediction unit is used to obtain a predicted value of a pixel value of the current image block based on a pixel value of the target forward reference block and a pixel value of the target backward reference block.

[0112] In different application scenarios, the image prediction device is applied to, for example, a video encoding device (video encoder) or a video decoding device (video decoder).

[0113] The seventh aspect of the present application provides an image prediction device, comprising a plurality of functional units for implementing any one of the methods of the third aspect. For example, the image prediction device may include: a third acquisition unit, for acquiring the i-th round motion information of the current image block; a third search unit, for determining the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block, wherein the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1; based on a matching cost criterion, determining a pair of reference block positions from the positions of the M pairs of reference blocks as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block position and a backward reference block position. , and for each pair of reference blocks, the first position offset is in a mirror relationship with the second position offset, the first position offset represents the position offset of the forward reference block relative to the position of the i-1th round target forward reference block, the second position offset represents the position offset of the backward reference block relative to the position of the i-1th round target backward reference block, M is an integer greater than or equal to 1, and M is less than or equal to N; a third prediction unit, used to obtain a predicted value of the pixel value of the current image block according to the pixel value of the jth round target forward reference block and the pixel value of the jth round target backward reference block, wherein j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0114] In different application scenarios, the image prediction device is applied to, for example, a video encoding device (video encoder) or a video decoding device (video decoder).

[0115] The eighth aspect of the present application provides an image prediction device, comprising a plurality of functional units for implementing any one of the methods of the fourth aspect. For example, the image prediction device may include: a fourth acquisition unit, for acquiring the i-th round motion information of the current image block; a fourth search unit, for determining the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block, wherein the N forward reference blocks are located in the forward reference image, and the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1; based on the matching cost criterion, a pair of reference block positions are determined from the positions of the M pairs of reference blocks as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, The first position offset and the second position offset have a proportional relationship based on the time domain distance, the first position offset represents the position offset of the forward reference block in the forward reference image relative to the position of the i-1th round target forward reference block; the second position offset represents the position offset of the backward reference block in the backward reference image relative to the position of the i-1th round target backward reference block, M is an integer greater than or equal to 1, and M is less than or equal to N; a fourth prediction unit is used to obtain a predicted value of the pixel value of the current image block according to the pixel value of the jth round target forward reference block and the pixel value of the jth round target backward reference block, wherein j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0116] In different application scenarios, the image prediction device is applied to, for example, a video encoding device (video encoder) or a video decoding device (video decoder).

[0117] The ninth aspect of the present application provides an image prediction device, which includes: a processor and a memory coupled to the processor; the processor is used to execute the method in various implementation methods of the first aspect or the second aspect or the third aspect or the fourth aspect or the aforementioned aspects.

[0118] The tenth aspect of the present application provides a video encoder, which is used to encode an image block, comprising: an inter-frame prediction module, wherein the inter-frame prediction module includes the image prediction device as described in the fifth aspect or the sixth aspect or the seventh aspect or the eighth aspect, wherein the inter-frame prediction module is used to predict a predicted value of a pixel value of the image block; an entropy coding module, which is used to encode indication information into a bitstream, wherein the indication information is used to indicate initial motion information of the image block; and a reconstruction module, which is used to reconstruct the image block based on the predicted value of the pixel value of the image block.

[0119] The eleventh aspect of the present application provides a video decoder, which is used to decode an image block from a bit stream, including: an entropy decoding module, used to decode indication information from the bit stream, wherein the indication information is used to indicate initial motion information of a currently decoded image block; an inter-frame prediction module, including the image prediction device as described in the fifth aspect or the sixth aspect or the seventh aspect or the eighth aspect, wherein the inter-frame prediction module is used to predict a predicted value of a pixel value of the image block; and a reconstruction module, used to reconstruct the image block based on the predicted value of the pixel value of the image block.

[0120] The twelfth aspect of the present application provides a video encoding device, including a non-volatile storage medium and a processor, wherein the non-volatile storage medium stores an executable program, the processor is coupled to the non-volatile storage medium, and executes the executable program to implement the methods in the first, second, third or fourth aspect or its various implementation methods.

[0121] The thirteenth aspect of the present application provides a video decoding device, including a non-volatile storage medium and a processor, wherein the non-volatile storage medium stores an executable program, the processor is coupled to the non-volatile storage medium, and executes the executable program to implement the methods in the first, second, third or fourth aspect or its various implementation methods.

[0122] The fourteenth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the methods in the above-mentioned first, second, third or fourth aspects or their various implementation methods.

[0123] The fifteenth aspect of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first, second, third or fourth aspect or its various implementations.

[0124] The sixteenth aspect of the present application provides an electronic device, comprising the video encoder described in the tenth aspect, or the video decoder described in the eleventh aspect, or the image prediction device described in the fifth, sixth, seventh or eighth aspect.

[0125] It should be understood that the beneficial effects achieved by various aspects and corresponding feasible design methods are similar and will not be elaborated on again. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] Figure 1 A schematic block diagram of a video encoding and decoding system in an embodiment of the present application;

[0127] Figure 2A is a schematic block diagram of a video encoder in an embodiment of the present application;

[0128] Figure 2B is a schematic block diagram of a video decoder in an embodiment of the present application;

[0129] Figure 3 is a schematic flow chart of an image prediction method according to an embodiment of the present application;

[0130] Figure 4 It is a schematic diagram of the encoder obtaining initial motion information in the merge mode of inter-frame prediction;

[0131] Figure 5 It is a schematic diagram of obtaining initial motion information at the decoding end in the merge mode of inter-frame prediction;

[0132] Figure 6 is a schematic diagram of the initial reference block of the current image block;

[0133] Figure 7 is a schematic diagram of pixels at integer pixel positions and pixels at sub-pixel positions;

[0134] Figure 8 It is a schematic diagram of the search starting point;

[0135] Fig. 9 is a schematic block diagram of a first position offset and a second position offset in a mirror image relationship in an embodiment of the present application;

[0136] Fig.10 is a schematic flow chart of another image prediction method according to an embodiment of the present application;

[0137] Fig.11 is a schematic flow chart of another image prediction method according to an embodiment of the present application;

[0138] Fig.12 is a schematic flow chart of another image prediction method according to an embodiment of the present application;

[0139] Fig.13 is a schematic block diagram of a proportional relationship between a first position offset and a second position offset based on a time domain distance in an embodiment of the present application;

[0140] Fig.14 is a schematic flow chart of another image prediction method 1400 according to an embodiment of the present application;

[0141] Fig.15 is a schematic flow chart of another image prediction method according to an embodiment of the present application;

[0142] Fig.16 is a schematic flow chart of another image prediction method 1600 according to an embodiment of the present application;

[0143] Fig.17is a schematic flow chart of another image prediction method according to an embodiment of the present application;

[0144] Fig.18 is a schematic block diagram of an image prediction device according to an embodiment of the present application;

[0145] Fig.19 is a schematic block diagram of another image prediction device according to an embodiment of the present application;

[0146] Fig. 20 is a schematic block diagram of another image prediction device according to an embodiment of the present application;

[0147] Fig.21 is a schematic block diagram of another image prediction device according to an embodiment of the present application;

[0148] Fig. 22 It is a schematic block diagram of an encoding device or a decoding device according to an embodiment of the present application. DETAILED DESCRIPTION

[0149] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0150] Figure 1 The video encoder 20 and the video decoder 30 in the system are used to predict the predicted values ​​of the pixel values ​​of the image block according to the various image prediction method examples proposed in the present application, and to correct the motion information of the current encoded or decoded image block, such as the motion vector, so as to further improve the encoding and decoding performance. Figure 1 As shown, the system includes a source device 12 that generates encoded video data to be decoded at a later time by the destination device 14, and a destination device 14. Source device 12 and destination device 14 may comprise any of a wide range of devices, including desktop computers, notebook computers, tablet computers, set-top boxes, telephone handsets such as so-called “smart” phones, so-called “smart” touchpads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or the like.

[0151] Destination device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of media or device capable of moving the encoded video data from source device 12 to destination device 14. In one possible embodiment, link 16 may include a communication medium that enables source device 12 to transmit the encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and transmitted to destination device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network (e.g., a local area network, a wide area network, or the global network of the Internet). The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from source device 12 to destination device 14.

[0152] Alternatively, the encoded data may be output from the output interface 22 to the storage device 24. Similarly, the encoded data may be accessed from the storage device 24 by the input interface. The storage device 24 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another possible implementation, the storage device 24 may correspond to a file server or another intermediate storage device that can hold the encoded video generated by the source device 12. The destination device 14 may access the stored video data from the storage device 24 via streaming or downloading. The file server may be any type of server capable of storing encoded video data and transmitting this encoded video data to the destination device 14. A feasible implementation file server includes a website server, a file transfer protocol server, a network attached storage device, or a local disk drive. The destination device 14 may access the encoded video data via any standard data connection including an Internet connection. This data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a cable modem, etc.), or a combination of both that is suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from storage device 24 may be a streaming transmission, a download transmission, or a combination of both.

[0153] The techniques of the present application are not necessarily limited to wireless applications or settings. The techniques may be applied to video decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasts, cable television transmissions, satellite television transmissions, streaming video transmissions (e.g., via the Internet), encoding digital video for storage on a data storage medium, decoding digital video stored on a data storage medium, or other applications. In some possible implementations, the system may be configured to support one-way or two-way video transmissions to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0154] exist Figure 1 In a possible implementation, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. In some applications, the output interface 22 may include a modulator / demodulator (modem) and / or a transmitter. In the source device 12, the video source 18 may include a source such as a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of these sources. As a possible implementation, if the video source 18 is a camera, the source device 12 and the destination device 14 may form a so-called camera phone or video phone. The technology described in the present application may be exemplarily applicable to video decoding, and may be applicable to wireless and / or wired applications.

[0155] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 24 for later access by destination device 14 or other devices for decoding and / or playback.

[0156] Destination device 14 includes input interface 28, video decoder 30, and display device 32. In some applications, input interface 28 may include a receiver and / or a modem. Input interface 28 of destination device 14 receives encoded video data via link 16. The encoded video data communicated via link 16 or provided on storage device 24 may include a variety of syntax elements generated by video encoder 20 for use by a video decoder of video decoder 30 to decode the video data. These syntax elements may be included with the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0157] Display device 32 may be integrated with destination device 14 or external to destination device 14. In some possible implementations, destination device 14 may include an integrated display device and also be configured to interface with an external display device. In other possible implementations, destination device 14 may be a display device. In general, display device 32 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display, a plasma display, an organic light emitting diode display, or another type of display device.

[0158] The video encoder 20 and the video decoder 30 may operate according to, for example, the next generation video codec compression standard (H.266) currently under development and may comply with the H.266 Test Model (JEM). Alternatively, the video encoder 20 and the video decoder 30 may operate according to, for example, the ITU-T H.265 standard, also known as the High Efficiency Video Decoding standard, or other proprietary or industrial standards such as the ITU-T H.264 standard, or extensions of these standards, the ITU-T H.264 standard being alternatively referred to as MPEG-4 Part 10, also known as advanced video coding (AVC). However, the technology of the present application is not limited to any particular decoding standard. Other possible implementations of video compression standards include MPEG-2 and ITU-T H.263.

[0159] Although not in Figure 1 20 and video decoder 30 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams. If applicable, in some possible implementations, the MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).

[0160] Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are partially implemented in software, a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of the present application. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0161] The present application may exemplarily refer to the video encoder 20 "signaling" specific information to another device, such as the video decoder 30. However, it should be understood that the video encoder 20 may signal information by associating specific syntax elements with various encoded portions of the video data. That is, the video encoder 20 may "signal" data by storing specific syntax elements to header information of various encoded portions of the video data. In some applications, these syntax elements may be encoded and stored (e.g., stored to the storage system 34 or file server 36) before being received and decoded by the video decoder 30. Therefore, the term "signaling" may exemplarily refer to the communication of syntax or other data used to decode the compressed video data, regardless of whether such communication occurs in real time or near real time or occurs within a time span, such as when the syntax elements are stored to the media at the time of encoding, and the syntax elements can then be retrieved by the decoding device at any time after being stored to such media.

[0162] JCT-VC developed the H.265 (HEVC) standard. HEVC standardization is based on an evolutionary model of a video decoding device called the HEVC Test Model (HM). The latest standard document for H.265 can be obtained from http: / / www.itu.int / rec / T-REC-H.265, and the latest version of the standard document is H.265 (12 / 16), which is incorporated herein by reference in its entirety. HM assumes that the video decoding device has several additional capabilities relative to the existing algorithm of ITU-TH.264 / AVC. For example, H.264 provides 9 intra-frame prediction coding modes, while HM can provide up to 35 intra-frame prediction coding modes.

[0163] JVET is committed to the development of the H.266 standard. The process of H.266 standardization is based on an evolutionary model of a video decoding device called the H.266 test model. The algorithm description of H.266 is available from http: / / phenix.int-evry.fr / jvet, where the latest algorithm description is contained in JVET-F1001-v2, which is incorporated herein by reference in its entirety. At the same time, the reference software of the JEM test model can be obtained from https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / , which is also incorporated herein by reference in its entirety.

[0164] In general, the working model of HM describes that a video frame or image can be divided into a sequence of tree blocks or largest coding units (LCUs) containing both luminance and chrominance samples, and LCUs are also called CTUs. Tree blocks have a similar purpose to macroblocks of the H.264 standard. A slice contains several consecutive tree blocks in decoding order. A video frame or image can be partitioned into one or more slices. Each tree block can be split into coding units according to a quadtree. For example, a tree block that is the root node of a quadtree can be split into four child nodes, and each child node can be a mother node and split into another four child nodes. The final indivisible child nodes that are leaf nodes of the quadtree include decoding nodes, such as decoded image blocks. Syntax data associated with the decoded code stream can define the maximum number of times a tree block can be split, and can also define the minimum size of a decoding node.

[0165] The coding unit includes a decoding node and a prediction unit (PU) and a transform unit (TU) associated with the decoding node. The size of the CU corresponds to the size of the decoding node and must be square in shape. The size of the CU can range from 8×8 pixels to a maximum tree block size of 64×64 pixels or larger. Each CU may contain one or more PUs and one or more TUs. For example, the syntax data associated with the CU may describe the situation in which the CU is divided into one or more PUs. The partitioning mode may be different between the situations in which the CU is skipped or encoded in direct mode, intra-frame prediction mode, or inter-frame prediction mode. The PU may be partitioned into a non-square shape. For example, the syntax data associated with the CU may also describe the situation in which the CU is partitioned into one or more TUs according to a quadtree. The shape of the TU may be square or non-square.

[0166] The HEVC standard allows transformation based on TUs, which may be different for different CUs. TUs are usually sized based on the size of PUs within a given CU defined for a partitioned LCU, but this may not always be the case. The size of a TU is usually the same as or smaller than a PU. In some possible implementations, a quadtree structure called a "residual quadtree" (RQT) may be used to subdivide the residual samples corresponding to a CU into smaller units. The leaf nodes of the RQT may be referred to as TUs. The pixel difference values ​​associated with the TU may be transformed to produce transform coefficients, which may be quantized.

[0167] In general, TU uses a transform and quantization process. A given CU with one or more PUs may also include one or more TUs. After prediction, the video encoder 20 may calculate a residual value corresponding to the PU. The residual value includes a pixel difference value, which can be transformed into a transform coefficient, quantized and scanned using a TU to produce a serialized transform coefficient for entropy decoding. This application generally uses the term "image block" to refer to the decoding node of a CU. In some specific applications, this application may also use the term "image block" to refer to a tree block including a decoding node as well as a PU and a TU, such as an LCU or a CU. The following embodiments of the present application will introduce in detail various method examples described in the adaptive inverse quantization method in video encoding or decoding to perform an inverse quantization process of the transform coefficients corresponding to the current image block (ie, the current transform block) to improve encoding and decoding performance.

[0168] A video sequence typically includes a series of video frames or images. A group of pictures (GOP) exemplarily includes a series of one or more video images. A GOP may include syntax data in the header information of the GOP, in the header information of one or more of the images, or elsewhere, and the syntax data describes the number of images included in the GOP. Each slice of the image may include slice syntax data describing the encoding mode of the corresponding image. The video encoder 20 typically operates on image blocks within individual video slices to encode video data. Image blocks may correspond to decoding nodes within a CU. Image blocks may have fixed or variable sizes and may differ in size according to a specified decoding standard.

[0169] As a possible implementation, HM supports prediction for various PU sizes. Assuming that the size of a particular CU is 2N×2N, HM supports intra-prediction for PU sizes of 2N×2N or N×N, and inter-prediction for symmetric PU sizes of 2N×2N, 2N×N, N×2N or N×N. HM also supports asymmetric partitioning for inter-prediction for PU sizes of 2N×nU, 2N×nD, nL×2N and nR×2N. In asymmetric partitioning, one direction of the CU is not partitioned, while the other direction is partitioned into 25% and 75%. The portion of the CU corresponding to the 25% segment is indicated by an indication of "n" followed by "Up", "Down", "Left" or "Right". Thus, for example, "2N×nU" refers to a horizontally partitioned 2N×2NCU with a 2N×0.5NPU at the top and a 2N×1.5NPU at the bottom.

[0170] In this application, "N×M" and "N times M" are used interchangeably to refer to the pixel size of an image block in terms of horizontal and vertical dimensions, for example, 16×8 pixels or 16 times 8 pixels. In general, a 16×8 block will have 16 pixels in the horizontal direction, i.e., the width of the image block is 16 pixels, and 8 pixels in the vertical direction, i.e., the height of the image block is 8 pixels.

[0171] After intra-predictive or inter-predictive decoding using a PU of a CU, video encoder 20 may calculate residual data for the TUs of the CU. A PU may include pixel data in a spatial domain (also referred to as a pixel domain), and a TU may include coefficients in a transform domain after a transform (e.g., a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform) is applied to the residual video data. The residual data may correspond to pixel differences between pixels of an unencoded image and prediction values ​​corresponding to the PU. Video encoder 20 may form a TU including the residual data for the CU, and then transform the TU to produce transform coefficients for the CU.

[0172] The following embodiments of the present application will introduce in detail various method examples of the inter-frame prediction process in video encoding or decoding to obtain the sampling values ​​of the sampling points of the best forward reference block of the current image block and the sampling values ​​of the sampling points of the best backward reference block of the current image block, and then predict the sampling values ​​of the sampling points of the current image block. An image block refers to a two-dimensional array of sampling points, which can be a square array or a rectangular array. For example, a 4x4 image block can be regarded as a square sampling point array consisting of 4x4 and a total of 16 sampling points. The signal within the image block refers to the sampling value of the sampling point within the image block. In addition, the sampling point can also be called a pixel point or a pixel, which will be used without distinction in the present invention document. Correspondingly, the value of the sampling point can also be called a pixel value, which will be used without distinction in this application. The image can also be represented as a two-dimensional array of sampling points, marked in a similar way to the image block.

[0173] After any transforms to produce transform coefficients, video encoder 20 may perform quantization of the transform coefficients. Quantization illustratively refers to the process of quantizing the coefficients to possibly reduce the amount of data used to represent the coefficients, thereby providing further compression. The quantization process may reduce the bit depth associated with some or all of the coefficients. For example, an n-bit value may be truncated to an m-bit value during quantization, where n is greater than m.

[0174] The JEM model further improves the coding structure of video images. Specifically, a block coding structure called "quadtree combined with binary tree" (QTBT) is introduced. The QTBT structure abandons the concepts of CU, PU, ​​TU, etc. in HEVC, and supports more flexible CU division shapes. A CU can be square or rectangular. A CTU is first divided into a quadtree, and the leaf nodes of the quadtree are further divided into a binary tree. At the same time, there are two division modes in the binary tree division, symmetrical horizontal division and symmetrical vertical division. The leaf node of the binary tree is called a CU. The CU of JEM cannot be further divided during the prediction and transformation process, that is, the CU, PU, ​​and TU of JEM have the same block size. In the current JEM, the maximum size of a CTU is 256×256 brightness pixels.

[0175] In some possible embodiments, video encoder 20 may utilize a predefined scan order to scan the quantized transform coefficients to produce a serialized vector that may be entropy encoded. In other possible embodiments, video encoder 20 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, video encoder 20 may entropy decode the one-dimensional vector according to context adaptive variable length decoding (CAVLC), context adaptive binary arithmetic decoding (CABAC), syntax-based context adaptive binary arithmetic decoding (SBAC), probability interval partitioning entropy (PIPE) decoding, or other entropy decoding methods. Video encoder 20 may also entropy encode syntax elements associated with the encoded video data for use by video decoder 30 in decoding the video data.

[0176] Figure 2A FIG. 2 is a schematic block diagram of a video encoder 20 in an embodiment of the present application. Figure 3 , the video encoder 20 may perform an image prediction process, and in particular, the motion compensation unit 44 in the video encoder 20 may perform an image prediction process.

[0177] like Figure 2A As shown, the video encoder 20 may include: a prediction module 41, a summer 50, a transform module 52, a quantization module 54, and an entropy coding module 56. In an example, the prediction module 41 may include a motion estimation unit 42, a motion compensation unit 44, and an intra-frame prediction unit 46. The embodiment of the present application does not limit the internal structure of the prediction module 41. Optionally, for a video encoder with a hybrid architecture, the video encoder 20 may also include an inverse quantization module 58, an inverse transform module 60, and a summer 62.

[0178] exist Figure 2AIn a feasible implementation manner, the video encoder 20 may further include a segmentation unit (not shown) and a reference image memory 64. It should be understood that the segmentation unit and the reference image memory 64 may also be arranged outside the video encoder 20.

[0179] In another possible implementation, the video encoder 20 may further include a filter (not shown) to filter the block boundaries to remove blockiness artifacts from the reconstructed video. If necessary, the filter will generally filter the output of the summer 62.

[0180] like Figure 2A As shown, video encoder 20 receives video data, and a segmentation unit segments the data into tiles. This segmentation may also include segmentation into stripes, tiles or other larger units, such as tile segmentation according to a quadtree structure of LCUs and CUs. Generally speaking, a stripe may be divided into a plurality of tiles.

[0181] The prediction module 41 is used to generate a prediction block for the current coded image block. The prediction module 41 may select one of multiple possible decoding modes for the current image block based on the coding quality and cost calculation result (e.g., rate-distortion cost, RDcost), such as one of multiple intra-frame decoding modes or one of multiple inter-frame decoding modes. The prediction module 41 may provide the resulting intra-frame decoded or inter-frame decoded block to the summer 50 to generate residual block data and provide the resulting intra-frame decoded or inter-frame decoded block to the summer 62 to reconstruct the encoded block for use as a reference image.

[0182] Motion estimation unit 42 and motion compensation unit 44 within prediction module 41 perform inter-predictive decoding of a current image block relative to one or more prediction blocks in one or more reference images to provide temporal compression. Motion estimation unit 42 is used to determine an inter-prediction mode for a video slice according to a predetermined pattern for a video sequence. The predetermined pattern may specify a video slice in a sequence as a P slice, a B slice, or a GPB slice. Motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes. Motion estimation performed by motion estimation unit 42 is a process of generating a motion vector for an estimated image block. For example, a motion vector may indicate the displacement of a PU of an image block within a current video frame or image relative to a prediction block within a reference image.

[0183] A prediction block is a block that is found to be a close match to a PU of the image block to be decoded based on pixel difference, which may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference metrics. In some possible embodiments, video encoder 20 may calculate values ​​for sub-integer pixel positions of reference images stored in reference image memory 64.

[0184] Motion estimation unit 42 calculates a motion vector for a PU of an image block in an inter-decoded slice by comparing the position of the PU to the position of a prediction block of a reference picture. The reference picture may be selected from a first reference picture list (List 0) or a second reference picture list (List 1), each of which identifies one or more reference pictures stored in reference picture memory 64. Motion estimation unit 42 sends the calculated motion vector to entropy encoding module 56 and motion compensation unit 44.

[0185] Motion compensation performed by motion compensation unit 44 may involve extracting or generating a prediction block based on a motion vector determined by motion estimation, possibly performing interpolation to sub-pixel accuracy. Upon receiving the motion vector for the PU of the current image block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference image lists. Video encoder 20 forms a residual image block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current image block being decoded, thereby forming pixel difference values. The pixel difference values ​​form residual data for the block and may include both luminance and chrominance difference components. Summer 50 represents one or more components that perform this subtraction operation. Motion compensation unit 44 may also generate syntax elements associated with image blocks and video slices for use by video decoder 30 in decoding image blocks of video slices. This will be discussed below in conjunction with Figure 3 , Figure 10-12 , Figure 14-17 The image prediction process of the embodiment of the present application is introduced in detail and will not be repeated here.

[0186] Intra-prediction unit 46 within prediction module 41 may perform intra-predictive decoding of a current image block relative to one or more neighboring blocks in the same image or slice as the current block to be decoded to provide spatial compression. Thus, as an alternative to inter-prediction performed by motion estimation unit 42 and motion compensation unit 44 (as described above), intra-prediction unit 46 may intra-predict the current block. Specifically, intra-prediction unit 46 may determine an intra-prediction mode to encode the current block. In some possible implementations, intra-prediction unit 46 may, for example, use a variety of intra-prediction modes to encode the current block during a separate encoding pass, and intra-prediction unit 46 (or in some possible implementations, mode selection unit 40) may select an appropriate intra-prediction mode to use from the tested modes.

[0187] After the prediction module 41 generates a prediction block for the current image block via inter-frame prediction or intra-frame prediction, the video encoder 20 forms a residual image block by subtracting the prediction block from the current image block. The residual video data in the residual block may be included in one or more TUs and applied to the transform module 52. The transform module 52 is used to transform the residual between the original block of the current coded image block and the prediction block of the current image block. The transform module 52 transforms the residual data into residual transform coefficients using, for example, a discrete cosine transform (DCT) or a conceptually similar transform (e.g., a discrete sine transform DST). The transform module 52 can convert the residual video data from a pixel domain to a transform domain (e.g., a frequency domain).

[0188] The transform module 52 may send the resulting transform coefficients to the quantization module 54. The quantization module 54 quantizes the transform coefficients to further reduce the bit rate. In some possible implementations, the quantization module 54 may then perform a scan of the matrix containing the quantized transform coefficients. Alternatively, the entropy encoding module 56 may perform the scan.

[0189] After quantization, entropy encoding module 56 may entropy encode the quantized transform coefficients. For example, entropy encoding module 56 may perform context adaptive variable length decoding (CAVLC), context adaptive binary arithmetic decoding (CABAC), syntax-based context adaptive binary arithmetic decoding (SBAC), probability interval partitioning entropy (PIPE) decoding, or another entropy encoding method or technique. Entropy encoding module 56 may also entropy encode the motion vectors and other syntax elements for the current video slice being encoded. After entropy encoding by entropy encoding module 56, the encoded codestream may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30.

[0190] Inverse quantization module 58 and inverse transform module 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain for later use as a reference block of a reference image. Summer 62 adds the reconstructed residual block to the prediction block generated by prediction module 41 to produce a reconstructed block as a reference block for storage in reference image memory 64. These reference blocks may be used as reference blocks by motion estimation unit 42 and motion compensation unit 44 to inter-predict blocks in subsequent video frames or images.

[0191] It should be understood that other structural changes of the video encoder 20 can be used to encode the video stream. For example, for some image blocks or image frames, the video encoder 20 can directly quantize the residual signal without processing by the transform module 52, and accordingly, it does not need to be processed by the inverse transform module 58; or, for some image blocks or image frames, the video encoder 20 does not generate residual data, and accordingly, it does not need to be processed by the transform module 52, the quantization module 54, the inverse quantization module 58 and the inverse transform module 60; or, the video encoder 20 can directly store the reconstructed image block as a reference block without being processed by the filter unit; or, the quantization module 54 and the inverse quantization module 58 in the video encoder 20 can be combined together; or, the transform module 52 and the inverse transform module 60 in the video encoder 20 can be combined together; or, the summer 50 and the summer 62 can be combined together.

[0192] Figure 2B FIG. 1 is a schematic block diagram of a video decoder 30 in an embodiment of the present application. Figure 3 , Figure 10-12 , Figure 14-17 , the video decoder 30 can perform the image prediction process, especially the motion compensation unit 82 in the video decoder 30 can perform the image prediction process.

[0193] like Figure 2B As shown, the video decoder 30 may include an entropy decoding module 80, a prediction processing module 81, an inverse quantization module 86, an inverse transformation module 88, and a reconstruction module 90. In one example, the prediction module 81 may include a motion compensation unit 82 and an intra-frame prediction unit 84, which is not limited in the present embodiment.

[0194] In a feasible embodiment, the video decoder 30 may further include a reference image memory 92. It should be understood that the reference image memory 92 may also be set outside the video decoder 30. In some feasible embodiments, the video decoder 30 may perform the operation related to the image from Figure 2A The decoding process is an exemplary reciprocal of the encoding process described for video encoder 20 .

[0195] During the decoding process, the video decoder 30 receives an encoded video code stream representing image blocks of an encoded video slice and associated syntax elements from the video encoder 20. The video decoder 30 may receive syntax elements at the video slice level and / or the image block level. The entropy decoding module 80 of the video decoder 30 entropy decodes the bitstream / code stream to generate quantized coefficients and some syntax elements. The entropy decoding module 80 forwards the syntax elements to the prediction processing module 81. In the present application, in one example, the syntax elements here may include inter-frame prediction data related to the current image block, and the inter-frame prediction data may include an index identifier block_based_index to indicate which motion information (also called the initial motion information of the current image block) is used for the current image block; optionally, it may also include a switch flag block_based_enable_flag to indicate whether to use the current image block. Figure 3 or 14 to perform image prediction (in other words, to indicate whether to perform inter-frame prediction under the MVD mirror constraint proposed in this application for the current image block), or whether to perform inter-frame prediction on the current image block Fig.12 Or 16 to perform image prediction (in other words, to indicate whether to perform inter-frame prediction on the current image block based on the proportional relationship based on the time domain distance proposed in the present application).

[0196] When the video slice is decoded as an intra-frame decoded (I) slice, the intra-frame prediction unit 84 of the prediction processing module 81 may generate a prediction block of the image block of the current video slice based on the signaled intra-frame prediction mode and data from the previously decoded block of the current frame or image. When the video slice is decoded as an inter-frame decoded (i.e., B or P) slice, the motion compensation unit 82 of the prediction processing module 81 may determine the inter-frame prediction mode for decoding the current image block of the current video slice based on the syntax elements received from the entropy decoding module 82, and decode the current image block (e.g., perform inter-frame prediction) based on the determined inter-frame prediction mode. Specifically, the motion compensation unit 82 may determine which image prediction method is used to predict the current image block of the current video slice, for example, the syntax element indicates that the image prediction method based on the MVD mirror constraint condition is used to predict the current image block, predict or correct the motion information of the current image block of the current video slice, and thereby obtain or generate the prediction block of the current image block using the predicted motion information of the current image block through the motion compensation process. The motion information here may include reference image information and motion vectors, wherein the reference image information may include but is not limited to unidirectional / bidirectional prediction information, reference image list number and reference image index corresponding to the reference image list. For inter-frame prediction, a prediction block may be generated from one of the reference images in one of the reference image lists. The video decoder 30 may construct reference image lists, i.e., list 0 and list 1, based on the reference images stored in the reference image memory 92. The reference frame index of the current image may be included in one or more of the reference frame list 0 and list 1. In some instances, the video encoder 20 may send a signal to indicate which new image prediction method is to be adopted.

[0197] In this embodiment, the prediction processing module 81 is used to generate a prediction block of the current decoded image block; specifically, when the video slice is decoded as an intra-frame decoded (I) slice, the intra-frame prediction unit 84 of the prediction module 81 can generate a prediction block of the image block of the current video slice based on the intra-frame prediction mode sent by the signal and the data from the previously decoded image block of the current frame or image. When the video image is decoded as an inter-frame decoded (e.g., B, P or GPB) slice, the motion compensation unit 82 of the prediction module 81 generates a prediction block of the image block of the current video image based on the motion vector and other syntax elements received from the entropy coding unit 80.

[0198] Inverse quantization module 86 inversely quantizes, i.e., dequantizes, the quantized transform coefficients provided in the bitstream and decoded by entropy decoding module 80. The inverse quantization process may include using quantization parameters calculated by video encoder 20 for each image block in a video slice to determine the degree of quantization that should be applied and likewise the degree of inverse quantization that should be applied. Inverse transform module 88 applies an inverse transform to the transform coefficients, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to produce a residual block in the pixel domain.

[0199] After the motion compensation unit 82 generates a prediction block for the current image block, the video decoder 30 obtains a reconstructed block, i.e., a decoded image block, by summing the residual block from the inverse transform module 88 with the corresponding prediction block generated by the motion compensation unit 82. The summer 90 represents a component that performs this summing operation. If necessary, a loop filter (in the decoding loop or after the decoding loop) may also be used to smooth pixel transitions or otherwise improve video quality. The filter unit (not shown) may represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Furthermore, the decoded image blocks in a given frame or image may also be stored in a decoded image buffer 92, which stores reference images for subsequent motion compensation. The decoded image buffer 92 may be part of a memory, which may also store decoded video for later display on a display device (e.g. Figure 1 The memory may be presented on a display device 32) or may be separate from such memory.

[0200] It should be understood that other structural changes of the video decoder 30 can be used to decode the encoded video bitstream. For example, the video decoder 30 can generate an output video stream without processing by the filter unit; or, for some image blocks or image frames, the entropy decoding module 80 of the video decoder 30 does not decode the quantized coefficients, and accordingly does not need to be processed by the inverse quantization module 86 and the inverse transformation module 88. For example, the inverse quantization module 86 and the inverse transformation module 88 in the video decoder 30 can be combined together.

[0201] Figure 3 It is a schematic flow chart of the image prediction method of an embodiment of the present application. Figure 3 The method shown can be executed by a video coding and decoding apparatus, a video codec, a video coding and decoding system, and other devices with video coding and decoding functions. Figure 3 The method shown can occur in both the encoding process and the decoding process. More specifically, Figure 3The method shown may occur in an inter-frame prediction process during encoding and decoding. The process 300 may be performed by the video encoder 20 or the video decoder 30, and specifically, may be performed by a motion compensation unit of the video encoder 20 or the video decoder 30. Assuming that a video data stream having a plurality of video frames is using a video encoder or a video decoder, the process 300 including the following steps is performed to predict a predicted value of a pixel value of a current image block of a current video frame;

[0202] Figure 3 The method shown includes steps 301 to 304, which are described in detail below.

[0203] 301. Obtain initial motion information of the current image block.

[0204] The image block here can be an image block in the image to be processed, or a sub-image in the image to be processed. In addition, the image block here can be an image block to be encoded in the encoding process, or an image block to be decoded in the decoding process.

[0205] In addition, the above-mentioned initial motion information may include indication information of the prediction direction (usually bidirectional prediction), a motion vector pointing to a reference image block (usually a motion vector of an adjacent block) and image information where the reference image block is located (usually understood as reference image information), wherein the motion vector includes a forward motion vector and a backward motion vector, and the reference image information includes reference frame index information of a forward prediction reference image block and a backward prediction reference image block.

[0206] When obtaining the initial motion information of the image block, multiple methods may be used. For example, the following method 1 and method 2 may be used to obtain the initial motion information of the image block.

[0207] Method 1:

[0208] See also Figure 4 and Figure 5 In the merge mode of inter-frame prediction, a candidate motion information list is constructed based on the motion information of the neighboring blocks of the current image block, and a candidate motion information is selected from the candidate motion information list as the initial motion information of the current image block. The candidate motion information list includes motion vectors, reference frame index information, etc. For example, the motion information of the neighboring block A0 is selected (see Figure 5 Specifically, the forward motion vector of A0 is used as the forward prediction motion vector of the current block, and the backward motion vector of A0 is used as the backward prediction motion vector of the current block.

[0209] Method 2:

[0210] In the non-merge mode of inter-frame prediction, a motion vector prediction value list is constructed based on the motion information of the adjacent blocks of the current image block, and a certain motion vector is selected from the motion vector prediction value list as the motion vector prediction value of the current image block. In this case, the motion vector of the current image block can be the motion vector value of the adjacent blocks, or it can be the sum of the motion vector difference of the selected adjacent blocks and the motion vector of the current image block, where the motion vector difference is the difference between the motion vector obtained by motion estimation of the current image block and the motion vector of the selected adjacent block. For example, the motion vectors corresponding to indexes 1 and 2 in the motion vector prediction value list are selected as the forward motion vector and backward motion vector of the current image block.

[0211] It should be understood that the above-mentioned method 1 and method 2 are only two specific methods of obtaining the initial motion information of the image block. The present application does not limit the method of obtaining the motion information of the prediction block. Any method that can obtain the initial motion information of the image block is within the protection scope of the present application.

[0212] 302. Determine the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, wherein the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1.

[0213] See also Figure 6 The current image to which the current image block involved in the embodiment of the present application belongs has two reference images, one before and one after, namely a forward reference image and a backward reference image.

[0214] In one example, the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction;

[0215] Accordingly, step 302 may include:

[0216] Determine the position of an initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index according to the first motion vector and the position of the current image block; use the position of the initial forward reference block as a first search starting point (with Figure 8 , determining positions of (N-1) candidate forward reference blocks in the forward reference image;

[0217] The position of an initial backward reference block of the current image block is determined in the backward reference image corresponding to the second reference image index according to the second motion vector and the position of the current image block; and the positions of (N-1) candidate backward reference blocks are determined in the backward reference image using the position of the initial backward reference block as a second search starting point.

[0218] In one example, see Figure 7 , the positions of the N forward reference blocks include the position of an initial forward reference block (indicated by (0,0)) and the positions of (N-1) candidate forward reference blocks (indicated by (0,-1)(-1,-1), (-1,1), (1,-1) and (1,1), etc.), and the position of each candidate forward reference block is offset from the position of the initial forward reference block by an integer pixel distance (such as Figure 8 As shown) or fractional pixel distance, where N=9; or, the positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset from the position of the initial backward reference block by an integer pixel distance or a fractional pixel distance, where N=9.

[0219] See also Figure 8 In the process of motion estimation or motion compensation, the accuracy of MV can be fractional pixel accuracy (for example, 1 / 2 pixel accuracy, or 1 / 4 pixel accuracy). If there are only integer pixel values ​​in the image, and the accuracy of the current MV is fractional pixel accuracy, it is necessary to use the interpolation filter to interpolate the pixel values ​​at the integer pixel positions of the reference image to obtain the pixel values ​​at the fractional pixel positions as the value of the prediction block of the current block. The specific interpolation operation process is related to the interpolation filter used. Generally speaking, the pixel values ​​of the integer pixels around the reference pixel can be linearly weighted to obtain the value of the reference pixel. Commonly used interpolation filters include 4-tap, 6-tap, 8-tap, etc.

[0220] like Figure 7 As shown in the figure, Ai,j is the pixel point at the integer pixel position, and its bit width is bitDepth. a0,0,b0,0,c0,0,d0,0,h0,0,n0,0e0,0,i0,0,p0,0,f0,0,j0,0,q0,0,g0,0,k0,0, and r0,0 are the pixel points at the sub-pixel position. If an 8-tap interpolation filter is used, a0,0 can be calculated by the following formula:

[0221] a0,0=(C0*A -3,0 +C1*A -2,0 +C2*A -1,0 +C3*A 0,0 +C4*A 1,0 +C5*A 2,0 +C6*A 3,0 +C7*A 4,0 )>>shift1

[0222] In the above formula, C k,k=0,1,…,7 are the coefficients of the interpolation filter. If the sum of the coefficients of the interpolation filter is 2 to the power of N, then the gain of the interpolation filter is N. For example, N is 6, which means that the gain of the interpolation filter is 6 bits. shift1 is the right shift number. Shift1 can be set to bitDepth-8, where bitDepth is the target bit width. Thus, the pixel value bit width of the prediction block finally obtained according to the above formula is bitDepth+6-shift1=14 bits.

[0223] 303. Based on a matching cost criterion, determine a pair of reference block positions from M pairs of reference block positions as the position of a target forward reference block and the position of a target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, a first position offset is in a mirror relationship with a second position offset, the first position offset represents a position offset of the forward reference block relative to an initial forward reference block position, the second position offset represents a position offset of the backward reference block relative to an initial backward reference block position, and M is an integer greater than or equal to 1, and M is less than or equal to N.

[0224] See also Fig. 9 , the position of the candidate forward reference block 904 in the forward reference image Ref0 is offset by MVD0(delta0x, delta0y) relative to the position of the initial forward reference block 902 (i.e., the forward search base point). The position of the candidate backward reference block 905 in the backward reference image Ref1 is offset by MVD1(delta1x, delta1y) relative to the position of the initial backward reference block 903 (i.e., the backward search base point).

[0225] MVD0=-MVD1; that is:

[0226] delta0x = -delta1x;

[0227] delta0y=-delta1y;

[0228] In various examples, step 303 may include:

[0229] From the positions of M pairs of reference blocks (a forward reference block and a backward reference block), determine the positions of a pair of reference blocks with the smallest matching error as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or from the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks with a matching error less than or equal to a matching error threshold as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, where M is less than or equal to N. In addition, when comparing the difference between the pixel values ​​of the forward reference block and the pixel values ​​of the backward reference block, the sum of absolute differences (SAD), the sum of absolute transformation differences (SATD), or the sum of absolute square differences can be used for measurement.

[0230] 304. Obtain a predicted value of a pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block.

[0231] In one example, in step 304, pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block are weighted to obtain predicted values ​​of pixel values ​​of the current image block.

[0232] Optionally, as an embodiment, Figure 3 The method shown also includes: obtaining updated motion information of the current image block, the updated motion information including an updated forward motion vector and an updated backward motion vector, wherein the updated forward motion vector points to the position of the target forward reference block, and the updated backward motion vector points to the position of the target backward reference block. The updated motion information of the current image block may be obtained based on the position of the target forward reference block, the position of the target backward reference block, and the position of the current image block, or may be obtained based on the first position offset and the second position offset corresponding to the determined pair of reference block positions.

[0233] By updating the motion vector of the image block, other image blocks can be effectively predicted based on the image block when performing the next image prediction.

[0234] It can be seen that in the embodiment of the present application, the positions of the N forward reference blocks in the forward reference image and the positions of the N backward reference blocks in the backward reference image form the positions of N pairs of reference blocks, and the first position offset of the position of the forward reference block relative to the position of the initial forward reference block is mirrored with the second position offset of the position of the backward reference block relative to the position of the initial backward reference block. On this basis, the positions of a pair of reference blocks (for example, with the minimum matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the best forward reference block / forward prediction block) and the position of the target backward reference block (i.e., the best backward reference block / backward prediction block) of the current image block, thereby obtaining the predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, simplifies the image prediction process, thereby improving the accuracy of image prediction and reducing the complexity of image prediction.

[0235] Combine the following Fig.10 The image prediction method of the embodiment of the present application is described in detail.

[0236] Fig.10 It is a schematic flow chart of the image prediction method of an embodiment of the present application. Fig.10 The method shown can be executed by a video coding and decoding apparatus, a video codec, a video coding and decoding system, and other devices with video coding and decoding functions. Fig.10 The method shown can occur in both the encoding process and the decoding process. More specifically, Fig.10 The method shown may occur in the inter-frame prediction process during encoding and decoding.

[0237] Fig.10 The method shown includes steps 1001 to 1007, and steps 1001 to 1007 are described in detail below.

[0238] 1001. Obtain initial motion information of the current block.

[0239] For example, for an image block whose inter-frame prediction / coding mode is merge, a set of motion information is obtained from the mergecandidate list according to the merge index, and this motion information is the initial motion information of the current block. For example, for an image block whose inter-frame prediction / coding mode is AMVP, the MVP is obtained from the MVP candidate list according to the AMVP index, and the MVP is summed with the MVD contained in the bitstream to obtain the MV of the current block. The initial motion information includes reference image indication information and motion vectors, and the forward reference image and the backward reference image are determined by the reference image indication information. The position of the forward reference block and the position of the backward reference block are determined by the motion vector.

[0240] 1002. Determine the position of a starting forward reference block of the current image block in the forward reference image, where the position of the starting forward reference block is a search starting point (also called a search base point) in the forward reference image;

[0241] Specifically, according to the forward MV and the position information of the current block, the search base point in the forward reference image (hereinafter referred to as the first search base point) is obtained. For example, the forward MV information is (MV0x, MV0y). The position information of the current block is (B0x, B0y). Then the first search base point of the forward reference image is (MV0x+B0x, MV0y+B0y).

[0242] 1003. Determine the position of a starting backward reference block of the current image block in the backward reference image, where the position of the starting backward reference block is a search starting point in the backward reference image;

[0243] Specifically, according to the backward MV and the position information of the current block, the search base point in the backward reference image (hereinafter referred to as the second search base point) is obtained. For example, the backward MV is (MV1x, MV1y). The position information of the current block is (B0x, B0y). Then the second search base point of the backward reference image is (MV1x+B0x, MV1y+B0y).

[0244] 1004. Under the MVD mirror constraint condition, determine the positions of a pair of reference blocks (i.e., a forward reference block and a backward reference block) that best match, and obtain an optimal forward motion vector and an optimal backward motion vector;

[0245] The MVD mirror constraint here can be interpreted as the block position in the forward reference image is offset by MVD0 (delta0x, delta0y) relative to the forward search base point. The block position in the backward reference image is offset by MVD1 (delta1x, delta1y) relative to the backward search base point. The following relationship is satisfied:

[0246] MVD0=-MVD1; that is:

[0247] delta0x = -delta1x;

[0248] delta0y=-delta1y;

[0249] See also Figure 7 In the forward reference image, a motion search with an integer pixel step is performed starting from the search base point (indicated by (0,0)). An integer pixel step means that the position of the candidate reference block is offset by an integer pixel distance relative to the position of the search base point. It should be noted that, regardless of whether the search base point is an integer pixel point (the starting point can be an integer pixel, or a sub-pixel, such as: 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), an integer pixel step motion search can be performed first to obtain the position of the forward reference block of the current image block. It should be understood that when searching with an integer pixel step, the search starting point can be either an integer pixel or a sub-pixel, for example, an integer pixel, 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, and 1 / 16 pixel, etc.

[0250] like Figure 7 As shown, with the point (0,0) as the search base point, 8 search points with integer pixel steps around the search base point are searched to obtain the position of the corresponding candidate reference block. Figure 7 Eight candidate reference blocks are shown; if the position of the forward candidate reference block in the forward reference image is offset by (-1, -1) relative to the position of the forward search base point, then the position of the corresponding backward candidate reference block in the backward reference image is offset by (1, 1) relative to the position of the backward search base point. The positions of the paired forward candidate reference blocks and the backward candidate reference blocks are obtained accordingly. For the positions of the obtained pair of reference blocks, the matching cost between the corresponding two candidate reference blocks is calculated. The forward reference block and the backward reference block with the smallest matching cost are taken as the optimal forward reference block and the optimal backward reference block, and the optimal forward motion vector and the optimal backward motion vector are obtained.

[0251] 1005-1006. Use the optimal forward motion vector obtained in step 1004 to perform motion compensation process to obtain the pixel value of the optimal forward reference block; use the optimal backward motion vector obtained in step 1004 to perform motion compensation process to obtain the pixel value of the optimal backward reference block.

[0252] 1007. Weight the obtained pixel value of the optimal forward reference block and the pixel value of the optimal backward reference block to obtain a predicted value of the pixel value of the current image block.

[0253] Specifically, for example, the predicted value of the pixel value of the current image block can be obtained according to formula (2).

[0254] predSamples'[x][y] = (predSamplesL0'[x][y] + predSamplesL1'[x][y] +1) >> 1 (2)

[0255] Among them, predSamplesL0' is the optimal forward reference block, predSamplesL1' is the optimal backward reference block, predSamples' is the prediction block of the current image block, predSamplesL0'[x][y] is the pixel value of the optimal forward reference block at the pixel point (x, y), predSamplesL1'[x][y] is the pixel value of the optimal backward reference block at the pixel point (x, y), and predSamples'[x][y] is the pixel value of the final prediction block at the pixel point (x, y).

[0256] It should be noted that in the embodiment of the present application, it is not limited to which search method is used, and any search method may be used. For each forward candidate block obtained by the search, the difference between the forward candidate block and the corresponding backward candidate block in step 4 is calculated, and the forward candidate block with the smallest SAD and its corresponding forward motion vector and the backward candidate block and its corresponding backward motion vector are selected as the optimal forward prediction block and the corresponding optimal forward motion vector and the optimal backward prediction block and the corresponding optimal backward motion vector. Alternatively, for each backward candidate block obtained by the search, the difference between the backward candidate block and the corresponding forward candidate block in step 4 is calculated, and the backward candidate block with the smallest SAD and its corresponding backward motion vector and the forward candidate block and its corresponding forward motion vector are selected as the optimal backward reference block and the corresponding optimal backward motion vector and the optimal forward reference block and the corresponding optimal forward motion vector.

[0257] It should be noted that in step 1004, only an example of an integer pixel step search method is given. In fact, in addition to performing an integer pixel step search, a fractional pixel step search can also be used. For example, after performing an integer pixel step search in step 1004, a fractional pixel step search is performed. Alternatively, a fractional pixel step search is performed directly. The specific search method is not limited here.

[0258] It should be noted that, in the embodiments of the present application, the method for calculating the matching cost is not limited, for example, the SAD criterion may be used, the MR-SAD criterion may be used, or other criteria may be used. In addition, when calculating the matching cost, only the luminance component may be used for calculation, or both the luminance and chrominance components may be used for calculation.

[0259] It should be noted that, during the search process, if the matching cost is 0 or reaches a preset threshold, the traversal operation or the search operation can be terminated in advance. Here, the early termination condition of the search method is not limited.

[0260] It should be understood that there is no limitation on the order of step 1005 and step 1005 can be performed simultaneously or one after another.

[0261] It can be seen that compared with the existing method that needs to first calculate the template matching block and use the template matching block for forward search and backward search respectively, in the embodiment of the present application, in the process of finding the matching block, the candidate blocks in the forward reference image and the candidate blocks in the backward reference image are used to directly calculate the matching cost, and the two blocks with the smallest matching cost are determined, which simplifies the image prediction process and reduces the complexity while improving the image prediction accuracy.

[0262] Fig.11 It is a schematic flow chart of the image prediction method of an embodiment of the present application. Fig.11 The method shown can be executed by a video coding and decoding apparatus, a video codec, a video coding and decoding system, and other devices with video coding and decoding functions. Fig.11 The method shown includes steps 1101 to 1105, wherein steps 1101 to 1103 and 1105 refer to Fig.10 The description of steps 1001 to 1003 and 1007 will not be repeated here.

[0263] Compared with the embodiment of the present application Fig.10 The difference of the embodiment shown is that, during the search process, the pixel values ​​of the current optimal forward reference block and the optimal backward reference block are retained and updated. After the search is completed, the pixel values ​​of the current optimal forward reference block and the optimal backward reference block can be used to calculate the predicted value of the pixel value of the current image block.

[0264] For example, it is necessary to traverse the positions of N pairs of reference blocks. Costi is the i-th matching cost, and MinCost represents the current minimum matching cost. Bfi and Bbi are the pixel values ​​of the forward reference block and the pixel values ​​of the backward reference block obtained for the i-th time, respectively. BestBf and BestBb are the values ​​of the current optimal forward reference block, respectively. CalCost(M,N) represents the matching cost of block M and block N.

[0265] When the search starts (i=1), MinCost=Cost0=CalCost(Bf0,Bb0), BestBf=Bf0, BestBb=Bb0;

[0266] During subsequent traversals of other pairs of reference blocks, update at all times. For example, during the i-th (i > 1) search, if Costi < MinCost, then BestBf = Bfi and BestBb = Bbi; otherwise, do not update.

[0267] At the end of the search, use BestBf and BestBb to obtain the predicted value of the pixel value of the current block.

[0268] Fig.12 It is a schematic flowchart of the image prediction method according to an embodiment of the present application. Fig.12 The method shown can be executed by a video codec device, a video codec, a video codec system, and other devices with video codec capabilities. Fig.12 The method shown can occur both during the encoding process and during the decoding process. More specifically, Figure 3 The method shown can occur during the inter-frame prediction process during encoding and decoding. Process 1200 can be executed by video encoder 20 or video decoder 30. Specifically, it can be executed by the motion compensation unit of video encoder 20 or video decoder 30. Assume that a video data stream with multiple video frames is using a video encoder or a video decoder, and execute process 1200 including the following steps to predict the predicted value of the pixel value of the current image block of the current video frame;

[0269] Fig.12 The method shown includes steps 1201 to 1204, where the descriptions of steps 1201, 1202, and 1204 can be referred to Figure 3 in the descriptions of steps 301, 302, and 304 in, and will not be elaborated here.

[0270] The difference between the embodiment of the present application and Figure 3 the embodiment shown is that in step 1203, based on the matching cost criterion, determine the positions of a pair of reference blocks from the positions of M pairs of reference blocks as the positions of the target forward reference block and the target backward reference block of the current image block, where the position of each pair of reference blocks includes the position of a forward reference block and the position of a backward reference block, and for the position of each pair of reference blocks, the first position offset and the second position offset have a proportional relationship based on the temporal distance. The first position offset represents the position offset of the position of the forward reference block relative to the position of the initial forward reference block; the second position offset represents the position offset of the position of the backward reference block relative to the position of the initial backward reference block. M is an integer greater than or equal to 1 and less than or equal to N;

[0271] Refer to together Fig.13, the position of the candidate forward reference block 1304 in the forward reference image Ref0 is offset by MVD0(delta0x, delta0y) relative to the position of the initial forward reference block 1302 (i.e., the forward search base point). The position of the candidate backward reference block 1305 in the backward reference image Ref1 is offset by MVD1(delta1x, delta1y) relative to the position of the initial backward reference block 1303 (i.e., the backward search base point).

[0272] During the search process, the position offset of the two matching blocks satisfies the mirror relationship, and the mirror relationship needs to consider the time domain interval. Here TC, T0, and T1 represent the time of the current frame, the time of the forward reference image, and the time of the backward reference image respectively. TD0 and TD1 represent the time interval between the two moments.

[0273] TD0=TC-T0

[0274] TD1=TC-T1

[0275] In the specific encoding and decoding process, TD0 and TD1 can be calculated using the picture order count (POC). For example:

[0276] TD0=POCc-POC0

[0277] TD1=POCc-POC1

[0278] Here, POCc, POC0, and POC1 represent the POC of the current image, the POC of the forward reference image, and the POC of the backward reference image, respectively. TD0 represents the picture order count (POC) distance between the current image and the forward reference image; TD1 represents the POC distance between the current image and the backward reference image.

[0279] delta0=(delta0x,delta0y)

[0280] delta1=(delta1x,delta1y)

[0281] The mirror relationship considering the time domain interval is described as follows:

[0282] delta0x=(TD0 / TD1)*delta1x;

[0283] delta0y=(TD0 / TD1)*delta1y;

[0284] or

[0285] delta0x / delta1x=(TD0 / TD1);

[0286] delta0y / delta1y=(TD0 / TD1);

[0287] It can be seen that in the embodiment of the present application, the positions of the N forward reference blocks in the forward reference image and the positions of the N backward reference blocks in the backward reference image form the positions of N pairs of reference blocks, and the first position offset of the position of the forward reference block relative to the position of the initial forward reference block and the second position offset of the position of the backward reference block relative to the position of the initial backward reference block have a proportional relationship based on the time domain distance. On this basis, the positions of a pair of reference blocks (for example, with the minimum matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (that is, the best forward reference block / forward prediction block) and the position of the target backward reference block (that is, the best backward reference block / backward prediction block) of the current image block, so as to obtain the predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, simplifies the image prediction process, thereby improving the accuracy of image prediction and reducing the complexity of image prediction.

[0288] In the above-described embodiment, the search process is performed once. In addition, multiple rounds of search can also be performed through an iterative method. Specifically, after obtaining the forward reference block and the backward reference block in each round of search, one or more rounds of search can be performed based on the current corrected MV.

[0289] Combine the following Fig.14 The process of the image prediction method of the embodiment of the present application is described in detail. Figure 3 Similar to the method shown, Fig.14 The method shown may also be executed by a video coding and decoding apparatus, a video codec, a video coding and decoding system, and other devices having video coding and decoding functions. Fig.14 The method shown can occur in the encoding process or in the decoding process. Specifically, Fig.14 The method shown can occur in the inter-frame prediction process during encoding or decoding.

[0290] Fig.14 The method shown specifically includes steps 1401 to 1404, as follows:

[0291] 1401. Obtain the i-th round of motion information of the current image block;

[0292] The image block here can be an image block in the image to be processed, or a sub-image in the image to be processed. In addition, the image block here can be an image block to be encoded in the encoding process, or an image block to be decoded in the decoding process.

[0293] If i=1, the i-th round motion information is the initial motion information of the current image block;

[0294] If i>1, the i-th round motion information includes: a forward motion vector pointing to the position of the i-1-th round target forward reference block and a backward motion vector pointing to the position of the i-1-th round target backward reference block.

[0295] In addition, the above-mentioned initial motion information may include indication information of the prediction direction (usually bidirectional prediction), a motion vector pointing to a reference image block (usually a motion vector of an adjacent block) and image information where the reference image block is located (usually understood as reference image information), wherein the motion vector includes a forward motion vector and a backward motion vector, and the reference image information includes reference frame index information of a forward prediction reference image block and a backward prediction reference image block.

[0296] When obtaining the initial motion information of the image block, multiple methods may be used. For example, the following method 1 and method 2 may be used to obtain the initial motion information of the image block.

[0297] Method 1:

[0298] See also Figure 4 and Figure 5 In the merge mode of inter-frame prediction, a candidate motion information list is constructed based on the motion information of the neighboring blocks of the current image block, and a candidate motion information is selected from the candidate motion information list as the initial motion information of the current image block. The candidate motion information list includes motion vectors, reference frame index information, etc. For example, the motion information of the neighboring block A0 is selected (see Figure 5 Specifically, the forward motion vector of A0 is used as the forward prediction motion vector of the current block, and the backward motion vector of A0 is used as the backward prediction motion vector of the current block.

[0299] Method 2:

[0300] In the non-merge mode of inter-frame prediction, a motion vector prediction value list is constructed based on the motion information of the adjacent blocks of the current image block, and a certain motion vector is selected from the motion vector prediction value list as the motion vector prediction value of the current image block. In this case, the motion vector of the current image block can be the motion vector value of the adjacent blocks, or it can be the sum of the motion vector difference of the selected adjacent blocks and the motion vector of the current image block, where the motion vector difference is the difference between the motion vector obtained by motion estimation of the current image block and the motion vector of the selected adjacent block. For example, the motion vectors corresponding to indexes 1 and 2 in the motion vector prediction value list are selected as the forward motion vector and backward motion vector of the current image block.

[0301] It should be understood that the above-mentioned method 1 and method 2 are only two specific methods of obtaining the initial motion information of the image block. The present application does not limit the method of obtaining the motion information of the prediction block. Any method that can obtain the initial motion information of the image block is within the protection scope of the present application.

[0302] 1402. Determine the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block, wherein the N forward reference blocks are located in the forward reference image, and the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1.

[0303] In one example, the i-th round motion information includes a forward motion vector and a forward reference image index, and a backward motion vector and a backward reference image index;

[0304] Accordingly, step 1402 may include:

[0305] According to the forward motion vector and the position of the current image block, the position of the i-1th round target forward reference block of the current image block is determined in the forward reference image corresponding to the forward reference image index; the position of the i-1th round target forward reference block is used as the i-1th round target forward reference block. f Searching for a starting point, determining positions of (N-1) candidate forward reference blocks in the forward reference image;

[0306] According to the backward motion vector and the position of the current image block, the position of the i-1th round target backward reference block of the current image block is determined in the backward reference image corresponding to the backward reference image index; the position of the i-1th round target backward reference block is used as the i-1th round target backward reference block. b The search starting point determines the positions of (N-1) candidate backward reference blocks in the backward reference image.

[0307] In one example, see Figure 7, the positions of the N forward reference blocks include the position of the i-1th round target forward reference block (indicated by (0,0)) and the positions of the (N-1)th candidate forward reference blocks (indicated by (0,-1)(-1,-1), (-1,1), (1,-1) and (1,1) etc.), and the position of each candidate forward reference block is offset from the position of the i-1th round target forward reference block by an integer pixel distance (such as Figure 8 As shown) or fractional pixel distance, where N=9; or, the positions of the N backward reference blocks include the position of the i-1th round target backward reference block and the positions of the (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset from the position of the i-1th round target backward reference block by an integer pixel distance or a fractional pixel distance, where N=9.

[0308] 1403. Based on a matching cost criterion, determine a pair of reference block positions from M pairs of reference block positions as the position of a target forward reference block and the position of a target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, a first position offset is in a mirror relationship with a second position offset, the first position offset represents a position offset of the forward reference block relative to the position of the target forward reference block of the i-1th round, the second position offset represents a position offset of the backward reference block relative to the position of the target backward reference block of the i-1th round, and M is an integer greater than or equal to 1, and M is less than or equal to N.

[0309] The first position offset is in a mirror image relationship with the second position offset, which can be understood as: the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset.

[0310] See also Fig. 9 , the position of the candidate forward reference block 904 in the forward reference image Ref0 is offset by MVD0(delta0x, delta0y) relative to the position of the i-1th round target forward reference block 902 (i.e., the forward search base point). The position of the candidate backward reference block 905 in the backward reference image Ref1 is offset by MVD1(delta1x, delta1y) relative to the position of the i-1th round target backward reference block 903 (i.e., the backward search base point).

[0311] MVD0=-MVD1; that is:

[0312] delta0x = -delta1x;

[0313] delta0y=-delta1y;

[0314] In various examples, step 1403 may include:

[0315] From the positions of M pairs of reference blocks (a forward reference block and a backward reference block), the positions of a pair of reference blocks with the smallest matching error are determined as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block; or, from the positions of M pairs of reference blocks, the positions of a pair of reference blocks with a matching error less than or equal to a matching error threshold are determined as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block, where M is less than or equal to N. In addition, when comparing the difference between the pixel values ​​of the forward reference block and the pixel values ​​of the backward reference block, the sum of absolute differences (SAD), the sum of absolute transformation differences (SATD), or the sum of absolute square differences can be used for measurement.

[0316] 1404. Obtain a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block.

[0317] In one example, in step 404, the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block are weighted to obtain the predicted value of the pixel value of the current image block. In addition, in the present application, the predicted value of the pixel value of the current image block can also be obtained according to other methods, which is not limited in the present application.

[0318] By updating the motion vector of the image block, for example, the initial motion information is updated to the second round of motion information, wherein the second round of motion information includes: a forward motion vector pointing to the position of the first round target forward reference block and a backward motion vector pointing to the position of the first round target backward reference block, ..., it is possible to effectively predict other image blocks based on this image block when performing the next image prediction.

[0319] It can be seen that in the embodiment of the present application, the positions of the N forward reference blocks in the forward reference image and the positions of the N backward reference blocks in the backward reference image form the positions of N pairs of reference blocks, and the first position offset of the position of the forward reference block relative to the position of the initial forward reference block is mirrored with the second position offset of the position of the backward reference block relative to the position of the initial backward reference block. On this basis, the positions of a pair of reference blocks (for example, with the minimum matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the best forward reference block / forward prediction block) and the position of the target backward reference block (i.e., the best backward reference block / backward prediction block) of the current image block, thereby obtaining the predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, simplifies the image prediction process, thereby improving the accuracy of image prediction and reducing the complexity of image prediction. In addition, increasing the number of iterations can further improve the accuracy of the modified MV, thereby further improving the encoding and decoding performance.

[0320] Combine the following Fig.15 The process of the image prediction method of the embodiment of the present application is introduced in detail. Fig.15 The method shown may also be executed by a video coding and decoding apparatus, a video codec, a video coding and decoding system, and other devices having video coding and decoding functions. Fig.15 The method shown can occur in the encoding process or in the decoding process. Specifically, Fig.15 The method shown can occur in the inter-frame prediction process during encoding or decoding.

[0321] Fig.15 The method shown specifically includes steps 1501 to 1508, and steps 1501 to 1508 are described in detail below.

[0322] 1501. Obtain initial motion information of the current image block;

[0323] For example, if it is the first round of search, the initial motion information of the current block is used. For example, for an image block whose coding mode is merge, motion information is obtained from the merge candidate list according to the merge index, and this motion information is the initial motion information of the current block. For example, for an image block whose coding mode is AMVP, MVP is obtained from the MVPcandidate list according to the AMVP index, and the MVP is summed with the MVD contained in the bitstream to obtain the MV of the current block. If it is not the first round of search, the MV information updated in the previous round is used. Motion information includes reference image indication information and motion vector information. The forward reference image and the backward reference image are determined by the reference image indication information. The position of the forward reference block and the position of the backward reference block are determined by the motion vector information.

[0324] 1502. Determine a search base point in a forward reference image;

[0325] According to the forward MV information and the position information of the current block, the search base point in the forward reference image is determined. Fig.10 The process of the embodiment of or 11 is similar. For example, the forward MV information is (MV0x, MV0y), the position information of the current block is (B0x, B0y), and the search base point in the forward reference image is (MV0x+B0x, MV0y+B0y).

[0326] 1503. Determine the search base point in the backward reference image

[0327] According to the backward MV information and the position information of the current block, the search base point in the backward reference image is determined. Fig.10 The process of the embodiment of or 11 is similar. For example, the backward MV information is (MV1x, MV1y), and the position information of the current block is (B0x, B0y), then the search base point of the backward reference image is (MV1x+B0x, MV1y+B0y).

[0328] 1504. Determine the positions of a pair of reference blocks (i.e., a forward reference block and a backward reference block) that best match the image under the MVD mirror constraint in the forward reference image and the backward reference image, and obtain a corrected forward motion vector and a corrected backward motion vector of the current image block;

[0329] The specific search steps are Fig.10 The process of the embodiment of or 11 is similar and will not be repeated here.

[0330] 1505. Determine whether the iteration termination condition is met. If not, execute steps 1502 and 1503. Otherwise, execute steps 1506 and 1507.

[0331] The design of the termination condition of the iterative search is not limited here, for example, the set number of iterations L can be traversed, or other iterative termination conditions can be reached. For example: if after the current iterative operation, MVD0 is close to or equal to 0 and MVD1 is close to or equal to 0, for example, MVD0 = (0, 0), MVD1 = (0, 0), then the iterative operation can be terminated.

[0332] Wherein, L is a preset value, and L is an integer greater than 1. L can be a value preset before predicting the image, L can also be set according to the accuracy of image prediction and the complexity of searching the prediction block, L can also be set according to historical experience, or L can also be determined according to the verification of the results in the intermediate search process.

[0333] For example, in the present embodiment, a total of two searches are performed with an integer pixel step size, wherein, in the first search, the position of the initial forward reference block can be used as a search base point to determine the positions of (N-1) candidate forward reference blocks in the forward reference image (also referred to as the forward reference area); and the position of the initial backward reference block is used as a search base point to determine the positions of (N-1) candidate backward reference blocks in the backward reference image (also referred to as the forward reference area), and for one or more pairs of reference block positions among the N pairs of reference block positions, the matching costs of the corresponding two reference blocks are calculated, for example, the matching cost of the initial forward reference block and the initial backward reference block is calculated, and the matching cost of a candidate forward reference block and a candidate backward reference block that satisfy the MVD mirror constraint is calculated, thereby obtaining the positions of the first round target forward reference block and the first round target backward reference block of the first search, and then obtaining updated motion information, which includes: a forward motion vector indicating that the position of the current image block points to the position of the first round target forward reference block and a backward motion vector indicating that the position of the current image block points to the position of the first round target backward reference block. It should be understood that the updated motion information is the same as the reference frame index and other information in the initial motion information. Next, a second search is performed, using the position of the first-round target forward reference block as the search base point to determine the positions of (N-1) candidate forward reference blocks in the forward reference image (also called the forward reference area); and using the position of the first-round target backward reference block as the search base point to determine the positions of (N-1) candidate backward reference blocks in the backward reference image (also called the forward reference area), and for one or more pairs of reference block positions among the N pairs of reference block positions, the matching costs of the corresponding two reference blocks are calculated, for example, the matching cost of the first-round target forward reference block and the first-round target backward reference block is calculated, and the matching cost of a candidate forward reference block and a candidate backward reference block that satisfy the MVD mirror constraint is calculated, thereby obtaining the positions of the second-round target forward reference block and the second-round target backward reference block of the second search, and then obtaining updated motion information, which includes: a forward motion vector indicating that the position of the current image block points to the position of the second-round target forward reference block and a backward motion vector indicating that the position of the current image block points to the position of the second-round target backward reference block. It should be understood that the updated motion information is the same as other information such as the reference frame index in the initial motion information. When the preset number of iterations L=2, the second round target forward reference block and the second round target backward reference block in the second search process here are the final target forward reference block and target backward reference block (also called the optimal forward reference block and the optimal backward reference block).

[0334] 1506 to 1507, use the optimal forward motion vector obtained in step 1504 to perform a motion compensation process to obtain the pixel value of the optimal forward reference block; use the optimal backward motion vector obtained in step 1504 to perform a motion compensation process to obtain the pixel value of the optimal backward reference block.

[0335] 1508. Obtain a predicted value of the pixel value of the current image block according to the pixel value of the optimal forward reference block and the pixel value of the optimal backward reference block obtained in steps 1506 and 1507.

[0336] In step 1504, when searching in the forward reference image or the backward reference image, the search may be performed with an integer pixel step (or referred to as motion search) to obtain the position of at least one forward reference block and the position of at least one backward reference block. When searching with an integer pixel step, the search starting point may be either an integer pixel or a fractional pixel, for example, an integer pixel, 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, 1 / 16 pixel, etc.

[0337] In addition, when searching for the position of at least one forward reference block and the position of at least one backward reference block in step 1504, the search may be performed directly with a sub-pixel step size, or both an integer pixel step size search and a sub-pixel step size search may be performed. The present application does not limit the search method.

[0338] In step 1504, for each pair of reference block addresses, when calculating the difference between the pixel value of the corresponding forward reference block and the pixel value of the corresponding backward reference block, SAD, SATD or the sum of absolute square differences can be used to measure the difference between the pixel value of each forward reference block and the pixel value of the corresponding backward reference block. However, the present application is not limited thereto.

[0339] When determining the predicted value of the pixel value of the current image block based on the optimal forward reference block and the optimal backward reference block, the pixel value of the optimal forward reference block and the pixel value of the optimal backward reference block obtained in step 1506 and step 1507 can be weighted, and the pixel value obtained after the weighted processing can be used as the predicted value of the pixel value of the current image block.

[0340] Specifically, the predicted value of the pixel value of the current image block can be obtained according to formula (8).

[0341] predSamples'[x][y] = (predSamplesL0'[x][y] + predSamplesL1'[x][y] +1) >> 1 (8)

[0342] Among them, predSamplesL0’[x][y] is the pixel value of the optimal forward reference block at pixel point (x, y), predSamplesL1’[x][y] is the pixel value of the optimal backward reference block at pixel point (x, y), and predSamples’[x][y] is the pixel prediction value of the current image block at pixel point (x, y).

[0343] Refer to together Fig.11 In the iterative search process of the embodiment of the present application, the pixel values of the current optimal forward reference block and the optimal backward reference block can also be retained and updated. After the search is completed, the pixel values of the current optimal forward reference block and the optimal backward reference block are directly used to calculate the predicted value of the pixel value of the current image block. In this implementation manner, steps 1506 and 1507 are optional steps.

[0344] For example, it is necessary to traverse the positions of N pairs of reference blocks. Costi is the matching cost of the i-th time, and MinCost represents the current minimum matching cost value. Bfi and Bbi are the pixel values of the forward reference block and the backward reference block obtained at the i-th time respectively. BestBf and BestBb are the pixel values of the current optimal forward reference block respectively. CalCost(M, N) represents the matching cost between block M and block N.

[0345] When the search starts (i = 1), MinCost = Cost0 = CalCost(Bf0, Bb0), BestBf = Bf0, BestBb = Bb0;

[0346] When traversing other pairs of reference blocks subsequently, it is updated at all times. For example, when performing the i-th (i > 1) search, if Costi < MinCost, then BestBf = Bfi, BestBb = Bbi; otherwise, it is not updated.

[0347] When the search ends, the predicted value of the pixel value of the current block is obtained using BestBf and BestBb.

[0348] Above Fig.12 In the embodiment shown above, the search process is performed once. In addition, the search can also be performed in multiple rounds by an iterative method. Specifically, after obtaining the forward reference block and the backward reference block in each round of search, one or more rounds of search can be performed according to the currently corrected MV.

[0349] The following combines Fig.16 The process of the image prediction method 1600 of the embodiment of the present application will be introduced in detail. Fig.16 The method shown can also be executed by a video codec device, a video codec, a video codec system, and other devices with video codec functions. Fig.16 The method shown can occur in the encoding process or in the decoding process. Specifically, Fig.16 The method shown can occur in the inter-frame prediction process during encoding or decoding.

[0350] Fig.16 The method 1600 shown includes steps 1601 to 1604, wherein steps 1601, 1602 and 1604 refer to Fig.14 The description of steps 1401, 1402 and 1404 will not be repeated here.

[0351] Compared with the embodiment of the present application Fig.14 The difference between the illustrated embodiments is that, step 1603, based on the matching cost criterion, a pair of reference block positions are determined from the M pairs of reference block positions as the position of the target forward reference block and the position of the target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, a first position offset and a second position offset have a proportional relationship based on a time domain distance, the first position offset represents a position offset of the forward reference block relative to the initial forward reference block position; the second position offset represents a position offset of the backward reference block relative to the initial backward reference block position, and M is an integer greater than or equal to 1, and M is less than or equal to N;

[0352] See also Fig.13 , the position of the candidate forward reference block 1304 in the forward reference image Ref0 is offset by MVD0(delta0x, delta0y) relative to the position of the initial forward reference block 1302 (i.e., the forward search base point). The position of the candidate backward reference block 1305 in the backward reference image Ref1 is offset by MVD1(delta1x, delta1y) relative to the position of the initial backward reference block 1303 (i.e., the backward search base point).

[0353] During the search process, the position offset of the two matching blocks satisfies the mirror relationship, and the mirror relationship needs to consider the time domain interval. Here TC, T0, and T1 represent the time of the current frame, the time of the forward reference image, and the time of the backward reference image respectively. TD0 and TD1 represent the time interval between the two moments.

[0354] TD0=TC-T0

[0355] TD1=TC-T1

[0356] In the specific encoding and decoding process, TD0 and TD1 can be calculated using the picture order count (POC). For example:

[0357] TD0=POCc-POC0

[0358] TD1=POCc-POC1

[0359] Here, POCc, POC0, and POC1 represent the POC of the current image, the POC of the forward reference image, and the POC of the backward reference image, respectively. TD0 represents the picture order count (POC) distance between the current image and the forward reference image; TD1 represents the POC distance between the current image and the backward reference image.

[0360] delta0=(delta0x,delta0y)

[0361] delta1=(delta1x,delta1y)

[0362] The mirror relationship considering the time domain interval is described as follows:

[0363] delta0x=(TD0 / TD1)*delta1x;

[0364] delta0y=(TD0 / TD1)*delta1y;

[0365] or

[0366] delta0x / delta1x=(TD0 / TD1);

[0367] delta0y / delta1y=(TD0 / TD1);

[0368] In various examples, step 1603 may include:

[0369] From the positions of M pairs of reference blocks (a forward reference block and a backward reference block), the positions of a pair of reference blocks with the smallest matching error are determined as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block; or, from the positions of M pairs of reference blocks, the positions of a pair of reference blocks with a matching error less than or equal to a matching error threshold are determined as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block, where M is less than or equal to N. In addition, when comparing the difference between the pixel values ​​of the forward reference block and the pixel values ​​of the backward reference block, the sum of absolute differences (SAD), the sum of absolute transformation differences (SATD), or the sum of absolute square differences can be used for measurement.

[0370] It can be seen that in the embodiment of the present application, the positions of the N forward reference blocks in the forward reference image and the positions of the N backward reference blocks in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the forward reference block relative to the position of the initial forward reference block and the second position offset of the backward reference block relative to the position of the initial backward reference block have a proportional relationship based on the time domain distance. On this basis, a pair of reference block positions (for example, with the smallest matching cost) are determined from the N pairs of reference block positions as the position of the target forward reference block (that is, the best forward reference block / forward prediction block) and the position of the target backward reference block (that is, the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, simplifies the image prediction process, thereby improving the accuracy of image prediction while reducing the complexity of image prediction. In addition, increasing the number of iterations can further improve the accuracy of the modified MV, thereby further improving the encoding and decoding performance.

[0371] Combine the following Fig.17 The process of the image prediction method of the embodiment of the present application is introduced in detail. Fig.17 The method shown may also be executed by a video coding and decoding apparatus, a video codec, a video coding and decoding system, and other devices having video coding and decoding functions. Fig.17 The method shown can occur in the encoding process or in the decoding process. Specifically, Fig.17 The method shown can occur in the inter-frame prediction process during encoding or decoding.

[0372] Fig.17 The method shown includes steps 1701 to 1708, wherein steps 1701 to 1703, 1705 to 1708 refer to Fig.15 The descriptions of steps 1501 to 1503 and 1505 to 1508 are not repeated here.

[0373] Compared with the embodiment of the present application Fig.15 The embodiment shown differs in that

[0374] 1704. Under the mirror constraint condition of MVD based on time domain distance, determine the positions of a pair of reference blocks (i.e., a forward reference block and a backward reference block) that best match, and obtain a corrected forward motion vector and a corrected backward motion vector of the current image block;

[0375] The mirror constraint of the MVD based on the time domain distance here can be interpreted as that the position offset MVD0 (delta0x, delta0y) of the block position in the forward reference image relative to the forward search base point and the position offset MVD1 (delta1x, delta1y) of the block position in the backward reference image relative to the backward search base point satisfy the following relationship:

[0376] The position offset of the two matching blocks satisfies the mirror relationship based on the time domain distance. Here TC, T0, T1 represent the time of the current image, the time of the forward reference image, and the time of the backward reference image respectively. TD0, TD1 represent the time interval between the two moments.

[0377] TD0=TC-T0

[0378] TD1=TC-T1

[0379] In the specific encoding and decoding process, TD0 and TD1 can be calculated using the picture order count (POC). For example:

[0380] TD0=POCc-POC0

[0381] TD1=POCc-POC1

[0382] Here, POCc, POC0, and POC1 represent the POC of the current image, the POC of the forward reference image, and the POC of the backward reference image, respectively. TD0 represents the picture order count (POC) distance between the current image and the forward reference image; TD1 represents the POC distance between the current image and the backward reference image.

[0383] delta0=(delta0x,delta0y)

[0384] delta1=(delta1x,delta1y)

[0385] The mirror relationship considering the time domain distance (also called time domain interval) is described as follows:

[0386] delta0x=(TD0 / TD1)*delta1x;

[0387] delta0y=(TD0 / TD1)*delta1y;

[0388] or

[0389] delta0x / delta1x=(TD0 / TD1);

[0390] delta0y / delta1y=(TD0 / TD1).

[0391] The specific search steps are Fig.10 The process of the embodiment of or 11 is similar and will not be repeated here.

[0392] It should be understood that in the embodiment of the present application, the mirror relationship either considers the time domain interval or does not consider the time domain interval. In actual use, it is possible to adaptively select whether the mirror relationship considers the time domain interval when performing motion vector correction on the current frame or current block.

[0393] For example, indication information may be added to sequence level header information (SPS), picture level header information (PPS), slice header, or block stream information to indicate whether the mirror relationship used by the current sequence, or the current picture, or the current slice, or the current block considers the time interval.

[0394] Alternatively, the current block adaptively determines whether the mirror relationship used by the current block takes the time interval into consideration according to the POC of the forward reference image and the POC of the backward reference image.

[0395] For example: if |POCc-POC0|-|POCc-POC1|>T, the mirror relationship used needs to consider the interval, otherwise the time interval is not considered, where T is a preset threshold value. For example, T=2, or T=3. The specific T value is not limited here.

[0396] For another example: If the ratio of the larger value of |POCc-POC0| and |POCc-POC1| to the smaller value of |POCc-POC0| and |POCc-POC1| is greater than the threshold value R,

[0397] Right now:

[0398] (Max(|POCc-POC0|,|POCc-POC1|) / Min(|POCc-POC0|,|POCc-POC1|))>R.

[0399] Here Max(A,B) represents the larger value of A and B, and Min(A,B) represents the smaller value of A and B.

[0400] The mirror relationship used needs to consider the interval, otherwise the time interval is not considered, where R is a preset threshold value, for example, R = 2, or R = 3. The specific R value is not limited here.

[0401] It should be understood that the image prediction method of the embodiment of the present application can be specifically performed by a motion compensation module in an encoder (e.g., encoder 20) or a decoder (e.g., decoder 30). In addition, the image prediction method of the embodiment of the present application can be implemented in any electronic device or device that needs to encode and / or decode video images.

[0402] Combine the following Figures 18 to 21 The image prediction device according to the embodiment of the present application is described in detail.

[0403] Fig.18 It is a schematic block diagram of an image prediction device according to an embodiment of the present application. It should be noted that the prediction device 1800 is applicable to both inter-frame prediction of decoded video images and inter-frame prediction of encoded video images. It should be understood that the prediction device 1800 here may correspond to Figure 2A The motion compensation unit 44 in, or may correspond to Figure 2B The motion compensation unit 82 in the prediction device 1800 may include:

[0404] The first acquisition unit 1801 is used to acquire initial motion information of the current image block;

[0405] A first search unit 1802 is configured to determine positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, wherein the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1; based on a matching cost criterion, determine a pair of reference block positions from the positions of the M pairs of reference blocks as the position of a target forward reference block and the position of a target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, a first position offset is in a mirror relationship with a second position offset, wherein the first position offset represents a position offset of the forward reference block relative to the initial forward reference block position, and the second position offset represents a position offset of the backward reference block relative to the initial backward reference block position, and M is an integer greater than or equal to 1, and M is less than or equal to N;

[0406] The first prediction unit 1803 is used to obtain a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block.

[0407] Among them, the first position offset is in a mirror relationship with the second position offset, which can be understood as the first position offset is the same as the second position offset, for example, the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset.

[0408] Preferably, in the device 1800 of the embodiment of the present application, the first prediction unit 1803 is also used to obtain updated motion information of the current image block, and the updated motion information includes an updated forward motion vector and an updated backward motion vector, wherein the updated forward motion vector points to the position of the target forward reference block, and the updated backward motion vector points to the position of the target backward reference block.

[0409] It can be seen that the motion vector of the image block is updated, so that when the next image prediction is performed, other image blocks can be effectively predicted based on the image block.

[0410] In the apparatus 1800 of the embodiment of the present application, the positions of the N forward reference blocks include the position of an initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset relative to the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance; or,

[0411] The positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset by an integer pixel distance or a fractional pixel distance relative to the position of the initial backward reference block.

[0412] In the device 1800 of the embodiment of the present application, the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction;

[0413] In the aspect of determining the positions of N forward reference blocks and the positions of N backward reference blocks according to the initial motion information and the position of the current image block, the first search unit is specifically configured to:

[0414] Determining the position of an initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index according to the first motion vector and the position of the current image block; determining the positions of (N-1) candidate forward reference blocks in the forward reference image using the position of the initial forward reference block as a first search starting point, wherein the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks;

[0415] Determine the position of an initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index according to the second motion vector and the position of the current image block; and determine the positions of (N-1) candidate backward reference blocks in the backward reference image using the position of the initial backward reference block as a second search starting point, wherein the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0416] In the device 1800 of the embodiment of the present application, in the aspect of determining, based on the matching cost criterion, a pair of reference block positions from the M pairs of reference block positions as the position of the target forward reference block and the position of the target backward reference block of the current image block, the first search unit 1802 is specifically configured to:

[0417] From the M pairs of reference block positions, determine the positions of a pair of reference blocks with the smallest matching error as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or

[0418] From the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks whose matching errors are less than or equal to a matching error threshold as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, where M is less than or equal to N.

[0419] It should be understood that the above-mentioned device 1800 can perform the above-mentioned Figure 3 , Fig.10 and Fig.11 In the method shown, the device 1800 may be a video encoding device, a video decoding device, a video encoding and decoding system or other equipment with video encoding and decoding functions. The device 1800 may be used for image prediction in the encoding process or in the decoding process.

[0420] For more details, please refer to the introduction of image prediction methods in this article. For the sake of brevity, we will not go into details here.

[0421] It can be seen that in the prediction device of the embodiment of the present application, the positions of the N forward reference blocks located in the forward reference image and the positions of the N backward reference blocks located in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the position of the forward reference block relative to the position of the initial forward reference block is mirrored to the second position offset of the position of the backward reference block relative to the position of the initial backward reference block. On this basis, a pair of reference block positions (for example, with the smallest matching cost) is determined from the N pairs of reference block positions as the position of the target forward reference block (that is, the best forward reference block / forward prediction block) and the position of the target backward reference block (that is, the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, thereby simplifying the image prediction process, thereby improving the accuracy of image prediction and reducing the complexity of image prediction.

[0422] Fig.19 It is a schematic block diagram of another image prediction device according to an embodiment of the present application. It should be noted that the prediction device 1900 is applicable to both inter-frame prediction of decoded video images and inter-frame prediction of encoded video images. It should be understood that the prediction device 1900 here can correspond to Figure 2A The motion compensation unit 44 in, or may correspond to Figure 2B The motion compensation unit 82 in the prediction device 1900 may include:

[0423] The second acquisition unit 1901 is used to acquire initial motion information of the current image block;

[0424] A second search unit 1902 is configured to determine positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, wherein the N forward reference blocks are located in the forward reference image, and the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1; based on a matching cost criterion, determine a pair of reference block positions from the positions of the M pairs of reference blocks as the position of a target forward reference block and the position of a target backward reference block of the current image block, wherein each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, a first position offset and a second position offset have a proportional relationship based on a time domain distance, wherein the first position offset represents a position offset of the forward reference block relative to the initial forward reference block position; and the second position offset represents a position offset of the backward reference block relative to the initial backward reference block position, and M is an integer greater than or equal to 1, and M is less than or equal to N;

[0425] The second prediction unit 1903 is used to obtain a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block.

[0426] For each pair of reference blocks, the first position offset and the second position offset have a proportional relationship based on the time domain distance, which can be understood as:

[0427] For each pair of reference blocks, the proportional relationship between the first position offset and the second position offset is determined based on the proportional relationship between the first time domain distance and the second time domain distance, wherein the first time domain distance represents the time domain distance between the current image to which the current image block belongs and the forward reference image; and the second time domain distance represents the time domain distance between the current image and the backward reference image.

[0428] As an implementation manner, the first position offset and the second position offset have a proportional relationship based on a time domain distance, which may include:

[0429] If the first time domain distance is the same as the second time domain distance, the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset; or,

[0430] If the first time domain distance is different from the second time domain distance, the direction of the first position offset is opposite to the direction of the second position offset, and the proportional relationship between the amplitude of the first position offset and the amplitude of the second position offset is based on the proportional relationship between the first time domain distance and the second time domain distance;

[0431] The first time domain distance represents the time domain distance between the current image to which the current image block belongs and the forward reference image; the second time domain distance represents the time domain distance between the current image and the backward reference image.

[0432] Preferably, in the device of this embodiment, the second prediction unit 1903 is also used to obtain updated motion information of the current image block, and the updated motion information includes an updated forward motion vector and an updated backward motion vector, wherein the updated forward motion vector points to the position of the target forward reference block, and the updated backward motion vector points to the position of the target backward reference block.

[0433] It can be seen that the embodiment of the present application can obtain the corrected motion information of the current image block and improve the accuracy of the motion information of the current image block, which will also be beneficial to the prediction of other image blocks, for example, improving the prediction accuracy of the motion information of other image blocks.

[0434] As an implementation manner, the positions of the N forward reference blocks include the position of an initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset relative to the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance; or,

[0435] The positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset by an integer pixel distance or a fractional pixel distance relative to the position of the initial backward reference block.

[0436] As an implementation manner, the initial motion information includes forward prediction motion information and backward prediction motion information;

[0437] In the aspect of determining the positions of N forward reference blocks and the positions of N backward reference blocks based on the initial motion information and the position of the current image block, the second searching unit 1902 is specifically configured to:

[0438] Determining positions of N forward reference blocks in a forward reference image according to the forward prediction motion information and the position of the current image block, wherein the positions of the N forward reference blocks include a position of an initial forward reference block and positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset relative to the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance;

[0439] The positions of N backward reference blocks are determined in the backward reference image according to the backward prediction motion information and the position of the current image block, the positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset relative to the position of the initial backward reference block by an integer pixel distance or a fractional pixel distance.

[0440] As another implementation manner, the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction;

[0441] In the aspect of determining the positions of N forward reference blocks and the positions of N backward reference blocks according to the initial motion information and the position of the current image block, the second search unit is specifically configured to:

[0442] Determining the position of an initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index according to the first motion vector and the position of the current image block; determining the positions of (N-1) candidate forward reference blocks in the forward reference image using the position of the initial forward reference block as a first search starting point, wherein the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks;

[0443] Determine the position of an initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index according to the second motion vector and the position of the current image block; and determine the positions of (N-1) candidate backward reference blocks in the backward reference image using the position of the initial backward reference block as a second search starting point, wherein the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0444] As an implementation manner, in the aspect of determining, according to the matching cost criterion, from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks as the position of the target forward reference block and the position of the target backward reference block of the current image block, the second searching unit 1902 is specifically configured to:

[0445] From the M pairs of reference block positions, determine the positions of a pair of reference blocks with the smallest matching error as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or

[0446] From the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks whose matching errors are less than or equal to a matching error threshold as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, where M is less than or equal to N.

[0447] In one example, the matching cost criterion is a criterion for minimizing the matching cost. For example, for the positions of M pairs of reference blocks, the difference between the pixel value of the forward reference block and the pixel value of the backward reference block in each pair of reference blocks is calculated; from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks with the smallest pixel value difference are determined as the position of the forward target reference block and the position of the backward target reference block of the current image block.

[0448] In another example, the matching cost criterion is a matching cost and an early termination criterion. For example, for the position of the nth pair of reference blocks (a forward reference block and a backward reference block), the difference between the pixel value of the forward reference block and the pixel value of the backward reference block is calculated, where n is an integer greater than or equal to 1 and less than or equal to N; when the pixel value difference is less than or equal to the matching error threshold, the position of the nth pair of reference blocks (a forward reference block and a backward reference block) is determined as the position of the forward target reference block and the position of the backward target reference block of the current image block.

[0449] As an implementation, the second acquisition unit 1901 is used to acquire the initial motion information from the candidate motion information list of the current image block, or acquire the initial motion information according to the indication information, wherein the indication information is used to indicate the initial motion information of the current image block. It should be understood that the initial motion information is relative to the corrected motion information.

[0450] It should be understood that the above-mentioned device 1900 can perform the above-mentioned Fig.12 In the method shown, the apparatus 1900 may be a video encoding apparatus, a video decoding apparatus, a video encoding and decoding system or other equipment with video encoding and decoding functions. The apparatus 1900 may be used for image prediction in the encoding process or in the decoding process.

[0451] For more details, please refer to the introduction of image prediction methods in this article. For the sake of brevity, we will not go into details here.

[0452] It can be seen that in the prediction device of the embodiment of the present application, the positions of the N forward reference blocks located in the forward reference image and the positions of the N backward reference blocks located in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the position of the forward reference block relative to the position of the initial forward reference block and the second position offset of the position of the backward reference block relative to the position of the initial backward reference block have a proportional relationship based on the time domain distance. On this basis, a pair of reference block positions (for example, with the minimum matching cost) is determined from the N pairs of reference block positions as the position of the target forward reference block (that is, the best forward reference block / forward prediction block) and the position of the target backward reference block (that is, the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, thereby simplifying the image prediction process, thereby improving the accuracy of image prediction and reducing the complexity of image prediction.

[0453] Fig. 20 It is a schematic block diagram of another image prediction device according to an embodiment of the present application. It should be noted that the prediction device 2000 is applicable to both inter-frame prediction of decoded video images and inter-frame prediction of encoded video images. It should be understood that the prediction device 2000 here can correspond to Figure 2A The motion compensation unit 44 in, or may correspond to Figure 2B The motion compensation unit 82 in the prediction device 2000 may include:

[0454] The third acquisition unit 2001 is used to acquire the i-th round of motion information of the current image block;

[0455] The third search unit 2002 is used to determine the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block, wherein the N forward reference blocks are located in the forward reference image, and the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1; based on the matching cost criterion, determine the positions of a pair of reference blocks from the positions of M pairs of reference blocks as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block, wherein the positions of each pair of reference blocks include the position of a forward reference block and the position of a backward reference block, and for each pair of reference block positions, a first position offset is in a mirror relationship with a second position offset, wherein the first position offset represents the position offset of the forward reference block relative to the position of the i-1th round target forward reference block, and the second position offset represents the position offset of the backward reference block relative to the position of the i-1th round target backward reference block, and M is an integer greater than or equal to 1, and M is less than or equal to N;

[0456] The third prediction unit 2003 is used to obtain the predicted value of the pixel value of the current image block based on the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0457] It should be noted that, if i=1, the i-th round motion information is the initial motion information of the current image block; accordingly, the positions of the N forward reference blocks include the position of an initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset from the position of the initial forward reference block by an integer pixel distance or a fractional pixel distance; or, the positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset from the position of the initial backward reference block by an integer pixel distance or a fractional pixel distance.

[0458] If i>1, the i-th round motion information includes: a forward motion vector pointing to the position of the i-1-th round target forward reference block and a backward motion vector pointing to the position of the i-1-th round target backward reference block; accordingly, the positions of the N forward reference blocks include the position of an i-1-th round target forward reference block and the positions of (N-1) candidate forward reference blocks, and the position of each candidate forward reference block is offset from the position of the i-1-th round target forward reference block by an integer pixel distance or a fractional pixel distance; or, the positions of the N backward reference blocks include the position of an i-1-th round target backward reference block and the positions of (N-1) candidate backward reference blocks, and the position of each candidate backward reference block is offset from the position of the i-1-th round target backward reference block by an integer pixel distance or a fractional pixel distance.

[0459] As an embodiment of the present application, the third prediction unit 2003 is specifically used to obtain the predicted value of the pixel value of the image block according to the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block when the iteration termination condition is met, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1. The description of the iteration termination condition refers to other embodiments and will not be repeated here.

[0460] In the device of the embodiment of the present application, the first position offset and the second position offset are in a mirror image relationship, which can be understood as: the first position offset amount is the same as the second position offset amount, for example, the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset.

[0461] In one implementation, the i-th round motion information includes a forward motion vector and a forward reference image index, and a backward motion vector and a backward reference image index;

[0462] In the aspect of determining the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block, the third search unit 2002 is specifically configured to:

[0463] According to the forward motion vector and the position of the current image block, the position of the i-1th round target forward reference block of the current image block is determined in the forward reference image corresponding to the forward reference image index; the position of the i-1th round target forward reference block is used as the i-1th round target forward reference block. f Searching for a starting point, determining positions of (N-1) candidate forward reference blocks in the forward reference image, wherein the positions of the N forward reference blocks include a position of an i-1th round target forward reference block and the positions of the (N-1) candidate forward reference blocks;

[0464] According to the backward motion vector and the position of the current image block, the position of the i-1th round target backward reference block of the current image block is determined in the backward reference image corresponding to the backward reference image index; the position of the i-1th round target backward reference block is used as the i-1th round target backward reference block. b The search starting point determines the positions of (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of an i-1th round target backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0465] In one implementation, in the aspect of determining, according to the matching cost criterion, from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks as the positions of the i-th round target forward reference blocks and the i-th round target backward reference blocks of the current image block, the third search unit 2002 is specifically configured to:

[0466] From the positions of the M pairs of reference blocks, determine the positions of a pair of reference blocks with the smallest matching error as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block; or

[0467] From the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks whose matching errors are less than or equal to the matching error threshold as the position of the i-th round target forward reference block of the current image block and the position of the i-th round target backward reference block of the current image block, where M is less than or equal to N.

[0468] It should be understood that the above-mentioned device 2000 can perform the above-mentioned Fig.14 and Fig.15 In the method shown, the device 2000 may be a video encoding device, a video decoding device, a video encoding and decoding system or other equipment with video encoding and decoding functions. The device 2000 may be used for image prediction in the encoding process or in the decoding process.

[0469] For more details, please refer to the introduction of image prediction methods in this article. For the sake of brevity, we will not go into details here.

[0470] It can be seen that in the prediction device of the embodiment of the present application, the positions of the N forward reference blocks located in the forward reference image and the positions of the N backward reference blocks located in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the position of the forward reference block relative to the position of the initial forward reference block is mirrored to the second position offset of the position of the backward reference block relative to the position of the initial backward reference block. On this basis, a pair of reference block positions (for example, with the smallest matching cost) is determined from the N pairs of reference block positions as the position of the target forward reference block (i.e., the best forward reference block / forward prediction block) and the position of the target backward reference block (i.e., the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, simplifies the image prediction process, thereby improving the accuracy of image prediction while reducing the complexity of image prediction. In addition, increasing the number of iterations can further improve the accuracy of the modified MV, thereby further improving the encoding and decoding performance.

[0471] Fig.21 It is a schematic block diagram of another image prediction device according to an embodiment of the present application. It should be noted that the prediction device 2100 is applicable to both inter-frame prediction of decoded video images and inter-frame prediction of encoded video images. It should be understood that the prediction device 2100 here can correspond to Figure 2A The motion compensation unit 44 in, or may correspond to Figure 2B The motion compensation unit 82 in the prediction device 2100 may include:

[0472] The fourth acquisition unit 2101 is used to acquire the i-th round of motion information of the current image block;

[0473] The fourth search unit 2102 is used to determine the positions of N forward reference blocks and N backward reference blocks according to the i-th round motion information and the position of the current image block, wherein the N forward reference blocks are located in the forward reference image, and the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1; based on the matching cost criterion, determine the positions of a pair of reference blocks from the positions of M pairs of reference blocks as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block, wherein the positions of each pair of reference blocks include the position of a forward reference block and the position of a backward reference block, and for each pair of reference block positions, a first position offset and a second position offset have a proportional relationship based on a time domain distance, wherein the first position offset represents the position offset of the position of the forward reference block in the forward reference image relative to the position of the i-1th round target forward reference block; and the second position offset represents the position offset of the position of the backward reference block in the backward reference image relative to the position of the i-1th round target backward reference block, and M is an integer greater than or equal to 1, and M is less than or equal to N;

[0474] The fourth prediction unit 2103 is used to obtain the predicted value of the pixel value of the current image block based on the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0475] In the iterative search process, if i=1, the i-th round motion information is the initial motion information of the current image block;

[0476] If i>1, the i-th round motion information includes: a forward motion vector pointing to the position of the i-1-th round target forward reference block and a backward motion vector pointing to the position of the i-1-th round target backward reference block.

[0477] As an implementation method, the fourth prediction unit 2103 is specifically used to: when the iteration termination condition is met, obtain the predicted value of the pixel value of the image block according to the pixel value of the j-th round target forward reference block and the pixel value of the j-th round target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0478] In the device of this embodiment, the first position offset and the second position offset have a proportional relationship based on the time domain distance, which can be understood as:

[0479] If the first time domain distance is the same as the second time domain distance, the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude of the first position offset is the same as the amplitude of the second position offset; or,

[0480] If the first time domain distance is different from the second time domain distance, the direction of the first position offset is opposite to the direction of the second position offset, and the proportional relationship between the amplitude of the first position offset and the amplitude of the second position offset is based on the proportional relationship between the first time domain distance and the second time domain distance;

[0481] The first time domain distance represents the time domain distance between the current image to which the current image block belongs and the forward reference image; the second time domain distance represents the time domain distance between the current image and the backward reference image.

[0482] As an implementation manner, the i-th round motion information includes a forward motion vector and a forward reference image index, and a backward motion vector and a backward reference image index; accordingly, in the aspect of determining the positions of N forward reference blocks and the positions of N backward reference blocks according to the i-th round motion information and the position of the current image block, the fourth search unit 2102 is specifically used to:

[0483] According to the forward motion vector and the position of the current image block, the position of the i-1th round target forward reference block of the current image block is determined in the forward reference image corresponding to the forward reference image index; the position of the i-1th round target forward reference block is used as the i-1th round target forward reference block. f Searching for a starting point, determining positions of (N-1) candidate forward reference blocks in the forward reference image, wherein the positions of the N forward reference blocks include a position of an i-1th round target forward reference block and the positions of the (N-1) candidate forward reference blocks;

[0484] According to the backward motion vector and the position of the current image block, the position of the i-1th round target backward reference block of the current image block is determined in the backward reference image corresponding to the backward reference image index; the position of the i-1th round target backward reference block is used as the i-1th round target backward reference block. b The search starting point determines the positions of (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of an i-1th round target backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0485] As an implementation manner, in the aspect of determining, according to the matching cost criterion, from the positions of M pairs of reference blocks, the positions of a pair of reference blocks as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block, the fourth search unit 2102 is specifically used to:

[0486] From the positions of the M pairs of reference blocks, determine the positions of a pair of reference blocks with the smallest matching error as the position of the i-th round target forward reference block and the position of the i-th round target backward reference block of the current image block; or

[0487] From the positions of M pairs of reference blocks, determine the positions of a pair of reference blocks whose matching errors are less than or equal to the matching error threshold as the position of the i-th round target forward reference block of the current image block and the position of the i-th round target backward reference block of the current image block, where M is less than or equal to N.

[0488] It should be understood that the above-mentioned device 2100 can perform the above-mentioned Fig.16 The method shown in FIG. 17, the apparatus 2100 may be a video encoding apparatus, a video decoding apparatus, a video encoding and decoding system or other equipment with video encoding and decoding functions. The apparatus 2100 may be used for image prediction in the encoding process or in the decoding process.

[0489] For more details, please refer to the introduction of image prediction methods in this article. For the sake of brevity, we will not go into details here.

[0490] It can be seen that in the prediction device of the embodiment of the present application, the positions of the N forward reference blocks located in the forward reference image and the positions of the N backward reference blocks located in the backward reference image form N pairs of reference block positions. For each pair of reference block positions in the N pairs of reference block positions, the first position offset of the position of the forward reference block relative to the position of the initial forward reference block and the second position offset of the position of the backward reference block relative to the position of the initial backward reference block have a proportional relationship based on the time domain distance. On this basis, a pair of reference block positions (for example, with the smallest matching cost) is determined from the N pairs of reference block positions as the position of the target forward reference block (that is, the best forward reference block / forward prediction block) and the position of the target backward reference block (that is, the best backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method of the embodiment of the present application avoids the calculation process of pre-calculating the template matching block and avoids the process of using the template matching block to perform forward search matching and backward search matching respectively, simplifies the image prediction process, thereby improving the accuracy of image prediction while reducing the complexity of image prediction. In addition, increasing the number of iterations can further improve the accuracy of the modified MV, thereby further improving the encoding and decoding performance.

[0491] Fig. 22A schematic block diagram of an implementation method of a video encoding device or video decoding device (referred to as a decoding device 2200) of an embodiment of the present application. Among them, the decoding device 2200 may include a processor 2210, a memory 2230 and a bus system 2250. Among them, the processor and the memory are connected via a bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory. The memory of the encoding device stores program code, and the processor can call the program code stored in the memory to execute various video encoding or decoding methods described in this application, especially video encoding or decoding methods in various inter-frame prediction modes or intra-frame prediction modes, and methods for predicting motion information in various inter-frame or intra-frame prediction modes. To avoid repetition, it is not described in detail here.

[0492] In the embodiment of the present application, the processor 2210 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0493] The memory 2230 may include a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may also be used as the memory 2230. The memory 2230 may include code and data 2231 accessed by the processor 2210 using the bus 2250. The memory 2230 may further include an operating system 2233 and an application 2235, which includes at least one program that allows the processor 2210 to perform the video encoding or decoding method described in the present application (especially the image prediction method described in the present application). For example, the application 2235 may include applications 1 to N, which further include a video encoding or decoding application (referred to as a video decoding application) that executes the video encoding or decoding method described in the present application.

[0494] The bus system 2250 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as the bus system 2250 in the figure.

[0495] Optionally, the decoding device 2200 may further include one or more output devices, such as a display 2270. In one example, the display 2270 may be a touch-sensitive display or a touch display screen that combines a display with a touch-sensitive unit operable to sense touch input. The display 2270 may be connected to the processor 2210 via the bus 2250.

[0496] It should be noted that the explanations and definitions of the same steps or terms are also applicable to different embodiments. For the sake of brevity, repeated descriptions are appropriately omitted herein.

[0497] Those skilled in the art will appreciate that the functions described in conjunction with the various illustrative logic blocks, modules, and algorithm steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or includes any communication media that facilitates the transfer of computer programs from one place to another (e.g., according to a communication protocol). In this manner, computer-readable media may generally correspond to (1) non-temporary tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product may include computer-readable media.

[0498] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals or other temporary media, but are actually directed to non-temporary tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser optical disks, optical optical disks, digital versatile disks (DVDs), and Blu-ray disks, where disks typically reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0499] The corresponding functions may be performed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. In addition, in some aspects, the functions described by the various illustrative logic blocks, modules, and steps described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Moreover, the techniques may be fully implemented in one or more circuits or logic elements. In one example, the various illustrative logic blocks, units, and modules in the video encoder 20 and the video decoder 30 may be understood as corresponding circuit devices or logic elements.

[0500] The techniques of the present application may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this application to emphasize functional aspects of devices for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units may be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by interoperating hardware units (including one or more processors as described above).

[0501] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image prediction method, characterized in that: include: Obtaining initial motion information of the current image block; When the early termination condition is met, determining the positions of the initial forward reference block and the initial backward reference block of the current image block as the positions of the target forward reference block and the target backward reference block of the current image block, wherein the positions of the initial forward reference block and the backward reference block of the current image block are obtained based on initial motion information of the current image block; Based on the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block of the current image block, a predicted value of the pixel value of the current image block is obtained.

2. The method according to claim 1, characterized in that: The early termination condition includes that a difference between a pixel value of the initial forward reference block and a pixel value of the initial backward reference block is less than a matching error threshold.

3. The method according to claim 1 or 2, characterized in that: The pixel value of the target forward reference block is determined according to the position of the target forward reference block, or the pixel value of the target backward reference block is determined according to the position of the target backward reference block.

4. The method according to claim 1 or 2, characterized in that: The initial motion information includes a first motion vector and a first reference image index corresponding to the first list (L0), and a second motion vector and a second reference image index corresponding to the second list (L1); The position of the initial forward reference image in the forward reference image corresponding to the first reference image index is obtained based on the first motion vector and the position of the current image block; The position of the initial backward reference block in the backward reference image corresponding to the second reference image index is obtained based on the second motion vector and the position of the current image block.

5. The method according to claim 1 or 2, characterized in that: The obtaining of the initial motion information of the current image block comprises: obtaining the initial motion information from a candidate motion information list of the current image block; The method further includes: encoding indication information into a bitstream, wherein the indication information indicates the initial motion information in the candidate motion information list of the current image block.

6. The method according to claim 1 or 2, characterized in that: Before obtaining the initial motion information of the current image block, the method further includes: obtaining indication information from a code stream of the current image block, wherein the indication information indicates the initial motion information of the current image block.

7. The method according to claim 1 or 2, characterized in that: The method further comprises: Obtaining initial motion information of another image block; When the early termination condition is not satisfied, based on a matching cost criterion, determining the position of a target forward reference block and a target backward reference block of the other image block from positions of N forward reference blocks and N backward reference blocks, wherein the positions of the N forward reference blocks and the N backward reference blocks are obtained based on the initial motion information; the N forward reference blocks include an initial forward reference block, the N backward reference blocks include an initial backward reference block, and N is an integer greater than 1; The position of the target forward reference block is associated with (delta0x′, delta0y′), and (delta0x′, delta0y′) represents the offset of the position of the target forward reference block relative to the position of the initial forward reference block. The position of the target backward reference block is associated with (delta1x′, delta1y′), wherein (delta1x′, delta1y′) represents an offset of the position of the target backward reference block relative to the position of the initial backward reference block, Among them, delta0x′=-delta1x′, delta0y′=-delta1y′; Based on the pixel values ​​of the target forward reference block of the another image block and the pixel values ​​of the target backward reference block of the another image block, a predicted value of the pixel value of the another image block is obtained.

8. An image prediction device, characterized in that: include: a memory including instructions; one or more processors in communication with the memory, wherein the one or more processors are configured to execute the instructions to: Obtaining initial motion information of the current image block; When the early termination condition is met, determining the positions of the initial forward reference block and the backward reference block of the current image block as the positions of the target forward reference block and the target backward reference block of the current image block, wherein the positions of the initial forward reference block and the backward reference block of the current image block are obtained based on initial motion information of the current image block; Based on the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block of the current image block, a predicted value of the pixel value of the current image block is obtained.

9. The device according to claim 8, characterized in that The one or more processors execute the instructions to: When the difference between the pixel values ​​of the initial forward reference block and the pixel values ​​of the initial backward reference block is less than a matching error threshold, the positions of the initial forward reference block and the initial backward reference block of the current image block are determined as the positions of the target forward reference block and the target backward reference block of the current image block.

10. The device according to claim 8 or 9, characterized in that The pixel value of the target forward reference block is determined according to the position of the target forward reference block, or the pixel value of the target backward reference block is determined according to the position of the target backward reference block.

11. The device according to claim 8 or 9, characterized in that The initial motion information includes a first motion vector and a first reference image index corresponding to the first list (L0), and a second motion vector and a second reference image index corresponding to the second list (L1); The position of the initial forward reference block in the forward reference image corresponding to the first reference image index is obtained based on the first motion vector and the position of the current image block; The position of the initial backward reference block in the backward reference image corresponding to the second reference image index is obtained based on the second motion vector and the position of the current image block.

12. The device according to claim 8 or 9, characterized in that The device is an encoding device for encoding the current image block; and the one or more processors execute the instructions to: Obtaining the initial motion information from a candidate motion information list of the current image block; The one or more processors further execute the instructions to: The indication information is encoded into a bitstream, where the indication information indicates the initial motion information in the candidate motion information list of the current image block.

13. The device according to claim 8 or 9, characterized in that The device is a decoding device for decoding the current image block; the one or more processors execute the instructions to: obtain indication information from the code stream of the current image block, where the indication information indicates initial motion information of the current image block.

14. The device according to claim 8 or 9, characterized in that The one or more processors further execute the instructions to: Obtaining initial motion information of another image block; When the early termination condition is not satisfied, based on a matching cost criterion, determining the position of a target forward reference block and a target backward reference block of the other image block from positions of N forward reference blocks and N backward reference blocks, wherein the positions of the N forward reference blocks and the N backward reference blocks are obtained based on the initial motion information; the N forward reference blocks include an initial forward reference block, the N backward reference blocks include an initial backward reference block, and N is an integer greater than 1; wherein the position of the target forward reference block is associated with (delta0x′, delta0y′) representing an offset of the position of the target forward reference block relative to the position of the initial forward reference block, The position of the target backward reference block is associated with (delta1x′, delta1y′), wherein delta1x′ represents the offset of the position of the target backward reference block relative to the position of the initial backward reference block. Among them, delta0x′=-delta1x′, delta0y′=-delta1y′; Based on the pixel values ​​of the target forward reference block of the another image block and the pixel values ​​of the target backward reference block of the another image block, a predicted value of the pixel value of the another image block is obtained.

15. A non-transitory computer-readable medium carrying program code, characterized in that: When the program code is executed by a computer device, the computer device performs the method, comprising: Obtaining initial motion information of the current image block; When the early termination condition is met, determining the positions of the initial forward reference block and the initial backward reference block of the current image block as the positions of the target forward reference block and the target backward reference block of the current image block, wherein the positions of the initial forward reference block and the backward reference block of the current image block are obtained based on initial motion information of the current image block; Based on the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block of the current image block, a predicted value of the pixel value of the current image block is obtained.

16. The non-transitory computer readable medium of claim 15, wherein: The early termination conditions are met including: A difference between a pixel value of the initial forward reference block and a pixel value of the initial backward reference block is smaller than a matching error threshold.

17. The non-transitory computer readable medium according to claim 15 or 16, wherein: The pixel value of the target forward reference block is determined based on the position of the target forward reference block, or the pixel value of the target backward reference block is determined based on the position of the target backward reference block.

18. The non-transitory computer readable medium according to claim 15 or 16, wherein: The initial motion information includes a first motion vector and a first reference image index corresponding to the first list (L0), and a second motion vector and a second reference image index corresponding to the second list (L1); The position of the initial forward reference image in the forward reference image corresponding to the first reference image index is obtained based on the first motion vector and the position of the current image block; The position of the initial backward reference block in the backward reference image corresponding to the second reference image index is obtained based on the second motion vector and the position of the current image block.

19. The non-transitory computer readable medium according to claim 15 or 16, wherein: The obtaining of the initial motion information of the current image block comprises: obtaining the initial motion information from a candidate motion information list of the current image block; the method further comprises: encoding indication information into a bitstream, wherein the indication information indicates the initial motion information in the candidate motion information list of the current image block.

20. The non-transitory computer readable medium according to claim 15 or 16, wherein: Before obtaining the initial motion information of the current image block, the method further includes: Indication information is obtained from a code stream of the current image block, where the indication information indicates initial motion information of the current image block.

21. The non-transitory computer readable medium according to claim 15 or 16, wherein: The method further comprises: Obtaining initial motion information of another image block; When the early termination condition is not satisfied, based on a matching cost criterion, determining the position of a target forward reference block and a target backward reference block of the other image block from positions of N forward reference blocks and N backward reference blocks, wherein the positions of the N forward reference blocks and the N backward reference blocks are obtained based on the initial motion information; the N forward reference blocks include an initial forward reference block, the N backward reference blocks include an initial backward reference block, and N is an integer greater than 1; wherein the position of the target forward reference block is associated with (delta0x′, delta0y′) representing an offset of the position of the target forward reference block relative to the position of the initial forward reference block, The position of the target backward reference block is associated with (delta1x′, delta1y′), wherein delta1x′ represents the offset of the position of the target backward reference block relative to the position of the initial backward reference block. Among them, delta0x′=-delta1x′, delta0y′=-delta1y′; Based on the pixel values ​​of the target forward reference block of the another image block and the pixel values ​​of the target backward reference block of the another image block, a predicted value of the pixel value of the another image block is obtained.

22. A non-transitory computer-readable medium for storing data associated with a video signal, characterized in that: include: a bitstream of the video signal stored in the non-transitory computer-readable medium, wherein the bitstream is generated according to the method, the method comprising: Obtaining initial motion information from a candidate motion information list of the current image block; When the early termination condition is met, determining the positions of the initial forward reference block and the initial backward reference block of the current image block as the positions of the target forward reference block and the target backward reference block of the current image block, wherein the positions of the initial forward reference block and the backward reference block of the current image block are obtained based on initial motion information of the current image block; Obtaining a predicted value of a pixel value of the current image block based on a pixel value of the target forward reference block and a pixel value of a target backward reference block of the current image block; The indication information is encoded into the bitstream, where the indication information indicates the initial motion information in the candidate motion information list of the current image block.

Images