Image prediction method and device, and codec

Abstract

To provide: a method and device for reducing image prediction complexity; and a codec.SOLUTION: An image prediction method includes: acquiring initial motion information of a present image block; determining positions of N forward reference blocks and positions of N backward reference blocks on the basis of the initial motion information and the position of the present image block; and determining that the positions of a pair of reference blocks are a position of a target forward reference block of the present image block and a position of a target backward reference block of the present image block from positions of M pairs of reference blocks on the basis of matching cost reference. The positions of each pair of reference blocks include the position of the forward reference block and the position of the backward reference block, and first and second position offsets are in a mirror image relation. Also, the method includes acquiring a prediction value of a pixel value of the present image block on the basis of a pixel value of the target forward reference block and a pixel value of the target backward reference block.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present application relates to the field of video coding technology, and in particular to a method and apparatus for image prediction, and a codec. [Background technology]

[0002] By using video compression techniques such as MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Coding (advanced video coding, AVC), ITU-TH.265 High Efficiency Video Coding (high efficiency video coding, HEVC), and extensions to these standards, digital video information can be efficiently transmitted and received between devices. Generally, images of a video sequence are divided into image blocks for encoding or decoding.

[0003] In video compression technology, image block-based spatial prediction (intra prediction) and / or temporal prediction (inter prediction) are introduced to reduce or remove redundant information in a video sequence. Inter prediction modes may include, but are not limited to, merge mode, non-merge mode (e.g., advanced motion vector prediction mode (AMVP mode)), and the like, and all inter predictions are performed by using a multi-motion information contention method.

[0004] In the inter-frame prediction process, a candidate motion information list (abbreviated as candidate list) is introduced, which includes multiple groups of motion information (also referred to as multiple candidate motion information). For example, an encoder may use a group of motion information selected from the candidate list as the motion information (e.g., motion vector) of the current image block to be coded or to predict it to obtain a reference image block (i.e., reference sample) of the current image block to be coded. Correspondingly, a decoder may decode a bitstream to obtain indication information and obtain a group of motion information. Since the overhead of encoding motion information (i.e., the overhead of bits of the occupied bitstream) is limited in the inter-frame prediction process, this affects the accuracy of the motion information to a certain extent, and further affects the image prediction accuracy.

[0005] To improve the accuracy of image prediction, existing decoder-side motion vector refinement (DMVR) technology can be used to refine motion information. However, when the DMVR solution is used to perform image prediction, a template matching block needs to be calculated, and the template matching block needs to be used to perform a search matching process separately in the forward reference image and the backward reference image, resulting in a relatively high search complexity. Therefore, how to reduce the complexity during image prediction while improving the accuracy of image prediction is a problem that needs to be solved. Summary of the Invention

[0006] SUMMARY OF THE INVENTION The embodiments of the present application provide a method and apparatus for image prediction, and a corresponding encoder and decoder, to improve image prediction accuracy, reduce image prediction complexity to a certain extent, and further improve coding performance. [Means for solving the problem]

[0007] According to a first aspect, an embodiment of the present application provides an image prediction method, the method includes: obtaining initial motion information of a current image block; determining positions of N forward reference blocks and N backward reference blocks according to the initial motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; and determining from the M pairs of reference block positions according to a matching cost criterion that the positions of a pair of reference blocks are the position of a target forward reference block of the current image block and the position of a target backward reference block of the current image block, where the positions of each pair of reference blocks are: The method includes determining a position of a forward reference block and a position of a backward reference block, where for each pair of reference block positions, the first position offset and the second position offset are in a mirror image relationship, the first position offset represents an offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; and obtaining a predicted value of a pixel value of the current image block based on a pixel value (sample) of the target forward reference block and a pixel value (sample) of the target backward reference block.

[0008] It is particularly noted that in this embodiment of the present application, the positions of the N forward reference blocks include the position of one initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks. Therefore, the offset of the position of the initial forward reference block relative to the position of the initial forward reference block is 0, and the offset of the position of the initial backward reference block relative to the position of the initial backward reference block is 0. The offset 0 and the offset 0 also satisfy the condition of mirror image relationship.

[0009] It can be seen that in this embodiment of the present application, the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a mirror image relationship between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward predicted block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward predicted block) of the current image block, thereby obtaining predicted values ​​of pixel values ​​of the current image block according to the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating template matching blocks and the process of performing forward search matching and backward search matching by using template matching blocks, and simplifies the image prediction process. This improves image prediction accuracy and reduces image prediction complexity.

[0010] Furthermore, it should be understood that a current image block (referred to as a current block) in this specification can be understood as an image block currently being processed. For example, in an encoding process, the current image block is an encoding block. In a decoding process, the current image block is a decoding block.

[0011] Furthermore, it should be understood that a reference block in this specification is a block that provides a reference signal for a current block. In a search process, multiple reference blocks need to be traversed to find an optimal reference block. A reference block that is located in a forward reference image is referred to as a forward reference block. A reference block that is located in a backward reference image is referred to as a backward reference block.

[0012] Furthermore, it should be understood that a block that provides a prediction for a current block is referred to as a prediction block. For example, after traversing multiple reference blocks, a best reference block is found. The best reference block provides a prediction for the current block and is referred to as a prediction block. A pixel value, a sampling value, or a sampling signal in the prediction block is referred to as a prediction signal.

[0013] Furthermore, it should be understood that the matching cost criterion in this specification is understood as a criterion for considering the matching cost between a pair of forward reference blocks and backward reference blocks. The matching cost may be understood as the difference between two blocks, and may be considered as the cumulative difference of samples at corresponding positions in the two blocks. The difference is usually calculated based on SAD (sum of absolute difference) criterion or another criterion, for example, SATD (sum of absolute transform difference), MR-SAD (mean-removed sum of absolute difference), or SSD (sum of squared differences).

[0014] Furthermore, it should be noted that the initial motion information of the current image block in this embodiment of the present application may include a motion vector MV and reference image indication information. Indeed, the initial motion information may alternatively include either a motion vector or reference image indication information, or both a motion vector and reference image indication information. For example, when the encoder side and the decoder side are in agreement on the reference image, the initial motion information may include only a motion vector MV. The reference image indication information is used to indicate which one or more reconstructed images are used as reference images for the current block. A motion vector indicates the offset of the position of a reference block in the used reference image relative to the position of the current block, and generally includes a horizontal component offset and a vertical component offset. For example, (x, y) is used to represent the MV, where x represents the horizontal position offset and y represents the vertical position offset. The position of the reference block of the current block in the reference image can be obtained by adding the MV to the position of the current block. The reference image indication information may include a reference image list and / or a reference image index corresponding to the reference image list. The reference picture index is used to identify the reference picture in the specified reference picture list (RefPicList0 or RefPicList1) that corresponds to the used motion vector. A picture is sometimes called a frame, and a reference picture is sometimes called a reference frame.

[0015] In this embodiment of the present application, the initial motion information of the current image block is initial bidirectional prediction motion information, that is, it includes motion information used in the forward prediction direction and motion information used in the backward prediction direction. In this specification, the forward and backward prediction directions are two prediction directions of the bidirectional prediction mode. "Forward" and "backward" can be understood to correspond to the reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1) of the current image, respectively.

[0016] Furthermore, it should be noted that the position of the initial forward reference block in this embodiment of the present application is the position of the reference block in the forward reference image and is the position obtained by adding the position of the current block to the offset represented by the initial MV. The position of the initial backward reference block in this embodiment of the present application is the position of the reference block in the backward reference image and is the position obtained by adding the position of the current block to the offset represented by the initial MV.

[0017] It should be understood that the method in this embodiment of the present application can be performed by an image prediction device. For example, the method can be performed by a video encoder, a video decoder, or an electronic device having video encoding functionality. For example, the method can be particularly performed by an inter-frame prediction unit in a video encoder or a motion compensation unit in a video decoder.

[0018] Regarding the first aspect, in some implementations of the first aspect, the first position offset and the second position offset being in a mirror image relationship may be understood as the first position offset value being the same as the second position offset value, e.g., the direction (also referred to as the vector direction) of the first position offset is opposite to the direction of the second position offset, and the amplitude value of the first position offset is the same as the amplitude value of the second position offset.

[0019] In one example, the first position offset includes a first horizontal component offset and a first vertical component offset, and the second position offset includes a second horizontal component offset and a second vertical component offset. The direction of the first horizontal component offset is opposite to the direction of the second horizontal component offset, and the amplitude value of the first horizontal component offset is the same as the amplitude value of the second horizontal component offset. The direction of the first vertical component offset is opposite to the direction of the second vertical component offset, and the amplitude value of the first vertical component offset is the same as the amplitude value of the second vertical component offset.

[0020] In another example, the first position offset and the second position offset are both zero.

[0021] With regard to the first aspect, in some implementation forms of the first aspect, the method includes obtaining updated motion information for a current image block, wherein the updated motion information further includes an updated forward motion vector and an updated backward motion vector, wherein the updated forward motion vector points to the position of a target forward reference block and the updated backward motion vector points to the position of the target backward reference block.

[0022] In different examples, the updated motion information of the current image block is obtained based on the position of the target forward reference block, the position of the target backward reference block, and the position of the current image block, or based on a first position offset and a second position offset corresponding to the determined positions of the pair of reference blocks.

[0023] It can be seen that refined motion information of the current image block can be obtained in this embodiment of the present application, which improves the accuracy of the motion information of the current image block and also facilitates the prediction of another image block, for example, improves the prediction accuracy of the motion information of another image block.

[0024] Regarding the first aspect, in some implementation forms of the first aspect, the positions of the N forward reference blocks include the position of one initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; or The positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0025] Note that the positions of the N pairs of reference blocks include the positions of the paired initial forward reference block and initial backward reference block, and the positions of the paired candidate forward reference block and candidate backward reference block. The offset of the position of the candidate forward reference block relative to the position of the initial forward reference block in the forward reference image is a mirror image of the offset of the position of the candidate backward reference block relative to the position of the initial backward reference block in the backward reference image.

[0026] Regarding the first aspect, in some implementation forms of the first aspect, the initial motion information includes forward prediction motion information and backward prediction motion information; Determining the positions of the N forward reference blocks and the N backward reference blocks based on the initial motion information and the position of the current image block includes: determining positions of N forward reference blocks in a forward reference image based on the forward prediction motion information and the position of the current image block, where the positions of the N forward reference blocks include a position of an initial forward reference block and positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; Determining positions of N backward reference blocks in a backward reference image based on backward prediction motion information and the position of the current image block, wherein the positions of the N backward reference blocks include a position of an initial backward reference block and positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0027] Regarding the first aspect, in some implementation forms of the first aspect, the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction; Determining the positions of the N forward reference blocks and the N backward reference blocks based on the initial motion information and the position of the current image block includes: Based on the first motion vector and the position of the current image block, use the position of the initial forward reference block as a first search starting point to determine the position of an initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index, and determine the positions of (N-1) candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks; Based on the second motion vector and the position of the current image block, using the position of the initial backward reference block as a second search starting point, determine the position of an initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index, and determine the positions of (N-1) candidate backward reference blocks in the backward reference image, where the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0028] Regarding the first aspect, in some implementation forms of the first aspect, determining, from the M pairs of reference block locations based on a matching cost criterion, that the pair of reference block locations are a target forward reference block location of the current image block and a target backward reference block location of the current image block, includes: determining, from the M pairs of reference block positions, a pair of reference block positions with the smallest matching error as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or determining, from the M pairs of reference block positions, that a pair of reference block positions whose matching error is less than or equal to a matching error threshold is the position of a target forward reference block of the current image block and the position of a target backward reference block of the current image block, where M is less than or equal to N;

[0029] In one example, the matching cost criterion is a matching cost minimization criterion. For example, for M pairs of reference block positions, the difference between the pixel value of the forward reference block and the pixel value of the backward reference block is calculated for each pair of reference blocks, and from the M pairs of reference block positions, the positions of the pair of reference blocks whose pixel values ​​have the smallest difference pixel value are determined as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block.

[0030] In another example, the matching cost criterion is a matching cost minimization and early termination criterion. For example, for the position of an n-th pair of reference blocks (one forward reference block and one backward reference block), a difference between a pixel value of the forward reference block and a pixel value of the backward reference block is calculated, where n is an integer greater than or equal to 1 and less than or equal to N, and when the pixel value difference is less than or equal to a matching error threshold, the position of the n-th pair of reference blocks (one forward reference block and one backward reference block) is determined as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block.

[0031] Regarding the first aspect, in some implementation forms of the first aspect, the method is used for encoding a current image block, and obtaining initial motion information for the current image block includes obtaining initial motion information from a candidate motion information list for the current image block; or The method is used to decode a current image block, and before obtaining initial motion information of the current image block, the method further includes obtaining indication information from the bitstream of the current image block, where the indication information is used to indicate the initial motion information of the current image block.

[0032] It can be seen that the image prediction method in this embodiment of the present application is not only applicable to merge prediction mode and / or advanced motion vector prediction (AMVP) mode, but also to other modes in which spatial reference blocks, temporal reference blocks, and / or inter-view reference blocks are used to predict the motion information of the current image block, which improves the coding information.

[0033] A second aspect of the present application provides an image prediction method, the method comprising: obtaining initial motion information of a current image block; The method includes: determining positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, where the N forward reference blocks are located in the forward reference image and the N backward reference blocks are located in the backward reference image, where N is an integer greater than 1; determining from the M pairs of reference block positions based on a matching cost criterion that the positions of a pair of reference blocks are the position of a target forward reference block of the current image block and the position of a target backward reference block of the current image block, where the positions of each pair of reference blocks include the position of the forward reference block and the position of the backward reference block, and for each pair of reference block positions, the first position offset and the second position offset have a proportional relationship based on a time-domain distance, where the first position offset represents the offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents the offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; and obtaining a predicted value of a pixel value of the current image block based on the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block.

[0034] It should be particularly noted that in this embodiment of the present application, the offset of the position of the initial forward reference block relative to the position of the initial forward reference block is 0, and the offset of the position of the initial backward reference block relative to the position of the initial backward reference block is 0. Offset 0 and offset 0 also satisfy the condition of a mirror image relationship or the condition of a proportional relationship based on the time domain distance. In other words, in the positions of the (N-1) pairs of reference blocks, for the positions of each pair of reference blocks, the first position offset and the second position offset are in a proportional relationship based on the time domain distance or a mirror image relationship. In this specification, the positions of the (N-1) pairs of reference blocks do not include the position of the initial forward reference block or the position of the initial backward reference block.

[0035] It can be seen that in this embodiment of the present application, the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image constitute N pairs of reference block positions.For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a proportional relationship based on time domain (also referred to as a mirror relationship based on time domain distance) between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block.Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward predicted block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward predicted block) of the current image block, thereby obtaining the predicted value of the pixel value of the current image block according to the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating template matching blocks and the process of performing forward search matching and backward search matching by using template matching blocks, and simplifies the image prediction process, which improves image prediction accuracy and reduces image prediction complexity.

[0036] Regarding the second aspect, in some implementation forms of the second aspect, for each pair of reference blocks, the first position offset and the second position offset are in a proportional relationship based on the time domain distance, For each pair of reference blocks, a proportional relationship between the first position offset and the second position offset is determined based on a proportional relationship between the first time-domain distance and the second time-domain distance, including: the first time-domain distance representing the time-domain distance between a current image to which the current image block belongs and a forward reference image; and the second time-domain distance representing the time-domain distance between the current image and a backward reference image.

[0037] Regarding the second aspect, in some implementation forms of the second aspect, the first position offset and the second position offset are in a proportional relationship based on the time domain distance, if the first time-domain distance is the same as the second time-domain distance, then the direction of the first position offset is opposite to the direction of the second position offset and the amplitude value of the first position offset is the same as the amplitude value of the second position offset; or if the first time-domain distance is different from the second time-domain distance, a direction of the first position offset is opposite to a direction of the second position offset, and the proportional relationship between the amplitude value of the first position offset and the amplitude value of the second position offset is based on the proportional relationship between the first time-domain distance and the second time-domain distance; The first time-domain distance represents the time-domain distance between the current image to which the current image block belongs and the forward reference image, and the second time-domain distance represents the time-domain distance between the current image and the backward reference image.

[0038] With regard to the second aspect, in some implementation forms of the second aspect, the method further includes obtaining updated motion information for the current image block, wherein the updated motion information includes an updated forward motion vector and an updated backward motion vector, wherein the updated forward motion vector points to the position of the target forward reference block and the updated backward motion vector points to the position of the target backward reference block.

[0039] It can be seen that refined motion information of the current image block can be obtained in this embodiment of the present application, which improves the accuracy of the motion information of the current image block and also facilitates the prediction of another image block, for example, improves the prediction accuracy of the motion information of another image block.

[0040] Regarding the second aspect, in some implementation forms of the second aspect, the positions of the N forward reference blocks include the position of one initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; or The positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0041] Regarding the second aspect, in some implementation forms of the second aspect, the positions of the N pairs of reference blocks include positions of paired initial forward reference blocks and initial backward reference blocks, and positions of paired candidate forward reference blocks and candidate backward reference blocks. A proportional relationship based on a time domain distance exists between the offset of the position of the candidate forward reference block relative to the position of the initial forward reference block in the forward reference image and the offset of the position of the candidate backward reference block relative to the position of the initial backward reference block in the backward reference image.

[0042] Regarding the second aspect, in some implementation forms of the second aspect, the initial motion information includes forward prediction motion information and backward prediction motion information; Determining the positions of the N forward reference blocks and the N backward reference blocks based on the initial motion information and the position of the current image block includes: determining positions of N forward reference blocks in a forward reference image based on the forward prediction motion information and the position of the current image block, where the positions of the N forward reference blocks include a position of an initial forward reference block and positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; Determining positions of N backward reference blocks in a backward reference image based on backward prediction motion information and the position of the current image block, wherein the positions of the N backward reference blocks include a position of an initial backward reference block and positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0043] Regarding the second aspect, in some implementation forms of the second aspect, the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction; Determining the positions of the N forward reference blocks and the N backward reference blocks based on the initial motion information and the position of the current image block includes: Based on the first motion vector and the position of the current image block, use the position of the initial forward reference block as a first search starting point to determine the position of an initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index, and determine the positions of (N-1) candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks; Based on the second motion vector and the position of the current image block, using the position of the initial backward reference block as a second search starting point, determine the position of an initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index, and determine the positions of (N-1) candidate backward reference blocks in the backward reference image, where the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0044] Regarding the second aspect, in some implementation forms of the second aspect, determining from the M pairs of reference block locations based on a matching cost criterion that the pair of reference block locations are a target forward reference block location of the current image block and a target backward reference block location of the current image block includes: determining, from the M pairs of reference block positions, a pair of reference block positions with the smallest matching error as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or determining, from the M pairs of reference block positions, that a pair of reference block positions whose matching error is less than or equal to a matching error threshold is the position of a target forward reference block of the current image block and the position of a target backward reference block of the current image block, where M is less than or equal to N;

[0045] In one example, the matching cost criterion is a matching cost minimization criterion. For example, for M pairs of reference block positions, the difference between the pixel value of the forward reference block and the pixel value of the backward reference block is calculated for each pair of reference blocks, and from the M pairs of reference block positions, the positions of the pair of reference blocks whose pixel values ​​have the smallest difference pixel value are determined as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block.

[0046] In another example, the matching cost criterion is a matching cost minimization and early termination criterion. For example, for the position of an n-th pair of reference blocks (one forward reference block and one backward reference block), a difference between a pixel value of the forward reference block and a pixel value of the backward reference block is calculated, where n is an integer greater than or equal to 1 and less than or equal to N, and when the pixel value difference is less than or equal to a matching error threshold, the position of the n-th pair of reference blocks (one forward reference block and one backward reference block) is determined as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block.

[0047] Regarding the second aspect, in some implementation forms of the second aspect, the method is used for encoding a current image block, and obtaining initial motion information for the current image block includes obtaining initial motion information from a candidate motion information list for the current image block; or The method is used to decode a current image block, and before obtaining initial motion information of the current image block, the method further includes obtaining indication information from the bitstream of the current image block, where the indication information is used to indicate the initial motion information of the current image block.

[0048] A third aspect of the present application provides an image prediction method, the method including: obtaining an ith motion information of a current image block; determining positions of N forward reference blocks and N backward reference blocks according to the i-th motion information and the position of the current image block, where the N forward reference blocks are located in the forward reference image and the N backward reference blocks are located in the backward reference image, where N is an integer greater than 1; and determining from the M pairs of reference block positions according to a matching cost criterion that a pair of reference block positions is a position of the i-th target forward reference block of the current image block and a position of the i-th target backward reference block of the current image block, where the positions of each pair of reference blocks include a position of the forward reference block and a position of the backward reference block, and determining a position of the jth target forward reference block, the first position offset and the second position offset being in a mirror image relationship, the first position offset representing an offset of the position of the forward reference block relative to the position of the (i-1)th target forward reference block, and the second position offset representing an offset of the position of the backward reference block relative to the position of the (i-1)th target backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; and obtaining predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the jth target forward reference block and pixel values ​​of the jth target backward reference block, where j is greater than or equal to i and both i and j are integers greater than or equal to 1.

[0049] It is particularly noted that in this embodiment of the present application, the offset of the position of the initial forward reference block relative to the position of the initial forward reference block is 0, and the offset of the position of the initial backward reference block relative to the position of the initial backward reference block is 0. Offset 0 and offset 0 also satisfy the condition of mirror image relationship.

[0050] It can be seen that in this embodiment of the present application, the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a mirror image relationship between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward predicted block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward predicted block) of the current image block, thereby obtaining predicted values ​​of pixel values ​​of the current image block according to the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating template matching blocks and the process of performing forward search matching and backward search matching by using template matching blocks, and simplifies the image prediction process. This improves image prediction accuracy and reduces image prediction complexity. Furthermore, in this embodiment of the present application, the accuracy of refining motion vector MV can be further improved by using iterative method, which can further improve coding performance.

[0051] Regarding the third aspect, in some implementation forms of the third aspect, if i=1, the i-th motion information is the initial motion information of the current image block, and correspondingly, the positions of the N forward reference blocks include the position of one initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block from the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance, or the positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block from the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0052] If i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block, and correspondingly, the positions of the N forward reference blocks include the position of one (i-1)th target forward reference block and the positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block from the position of the (i-1)th target forward reference block is an integer pixel distance or a fractional pixel distance, or the positions of the N backward reference blocks include the position of one (i-1)th target backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block from the position of the (i-1)th target backward reference block is an integer pixel distance or a fractional pixel distance.

[0053] It should be noted that when the method is used to encode a current image block, the initial motion information of the current image block is obtained by using a method for determining initial motion information from a list of candidate motion information for the current image block, or when the method is used to decode the current image block, the initial motion information of the current image block is obtained by using a method for obtaining indication information from a bitstream of the current image block, where the indication information is used to indicate the initial motion information of the current image block.

[0054] Regarding the third aspect, in some implementation forms of the third aspect, obtaining a predicted value of a pixel value of an image block based on a pixel value of a j-th target forward reference block and a pixel value of a j-th target backward reference block, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1, the obtaining may include: When the iteration termination condition is met, obtaining a predicted value of a pixel value of the image block based on the pixel value of the jth target forward reference block and the pixel value of the jth target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0055] With respect to the third aspect, in some implementation forms of the third aspect, the first position offset and the second position offset being in a mirror image relationship includes the direction of the first position offset being opposite to the direction of the second position offset, and the amplitude value of the first position offset being the same as the amplitude value of the second position offset.

[0056] Regarding the third aspect, in some implementation forms of the third aspect, the i-th motion information includes a forward motion vector, a forward reference image index, a backward motion vector, and a backward reference image index; Determining the positions of the N forward reference blocks and the N backward reference blocks based on the i-th motion information and the position of the current image block includes: Based on the forward motion vector and the position of the current image block, the position of the (i-1)th target forward reference block is calculated as f determining the position of the (i-1)th target forward reference block of the current image block in the forward reference image corresponding to the forward reference image index as a search starting point of the (i-1)th forward reference block, and determining the positions of the (N-1)th candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the (i-1)th target forward reference block and the positions of the (N-1)th candidate forward reference blocks; Based on the backward motion vector and the position of the current image block, the position of the (i-1)th target backward reference block is calculated as b as a search starting point to determine the position of the (i-1)th target backward reference block of the current image block in the backward reference image corresponding to the backward reference image index, and determine the positions of (N-1) candidate backward reference blocks in the backward reference image, where the positions of the N backward reference blocks include the position of the (i-1)th target backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0057] Regarding the third aspect, in some implementation forms of the third aspect, determining from the M pairs of reference block positions based on a matching cost criterion that the pair of reference block positions is the position of the ith target forward reference block of the current image block and the position of the ith target backward reference block of the current image block includes: determining, from the M pairs of reference block positions, a pair of reference block positions with the smallest matching error is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block; or The method includes determining, from the M pairs of reference block positions, that a pair of reference block positions whose matching error is less than or equal to a matching error threshold is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, where M is less than or equal to N.

[0058] A fourth aspect of the present application provides an image prediction method, the method including: obtaining an ith motion information of a current image block; determining positions of N forward reference blocks and N backward reference blocks based on the i-th motion information and the position of the current image block, where the N forward reference blocks are located in the forward reference image and the N backward reference blocks are located in the backward reference image, where N is an integer greater than 1; and determining from the M pairs of reference block positions based on a matching cost criterion that a pair of reference block positions is a position of the i-th target forward reference block of the current image block and a position of the i-th target backward reference block of the current image block, where the positions of each pair of reference blocks include a position of the forward reference block and a position of the backward reference block; The method includes determining a position offset and a second position offset that are proportional to each other based on a time-domain distance, where the first position offset represents an offset of the position of the forward reference block relative to the position of the (i-1)th target forward reference block in the forward reference image, and the second position offset represents an offset of the position of the backward reference block relative to the position of the (i-1)th target backward reference block in the backward reference image, where M is an integer greater than or equal to 1 and M is less than or equal to N; and obtaining a predicted value of a pixel value of the current image block based on the pixel value of the jth target forward reference block and the pixel value of the jth target backward reference block, where j is greater than or equal to i and both i and j are integers greater than or equal to 1.

[0059] It is particularly noted that in this embodiment of the present application, the offset of the position of the initial forward reference block relative to the position of the initial forward reference block is 0, and the offset of the position of the initial backward reference block relative to the position of the initial backward reference block is 0. The offset 0 and the offset 0 also satisfy the condition of a mirror image relationship or a condition of a proportional relationship based on the time domain distance. In other words, in the positions of the (N-1) pairs of reference blocks, for the positions of each pair of reference blocks, the first position offset and the second position offset are in a proportional relationship based on the time domain distance or a mirror image relationship. In this specification, the positions of the (N-1) pairs of reference blocks do not include the position of the initial forward reference block or the position of the initial backward reference block.

[0060] It can be seen that in this embodiment of the present application, the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a proportional relationship based on time domain distance between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward predicted block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward predicted block) of the current image block, thereby obtaining predicted values ​​of pixel values ​​of the current image block according to the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating template matching blocks and the process of performing forward search matching and backward search matching by using template matching blocks, and simplifies the image prediction process. This improves image prediction accuracy and reduces image prediction complexity. Furthermore, in this embodiment of the present application, the accuracy of refining motion vector MV can be further improved by using iterative method, which can further improve coding performance.

[0061] Regarding the fourth aspect, in some implementation forms of the fourth aspect, if i=1, the i-th motion information is the initial motion information of the current image block, or if i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block.

[0062] Regarding the fourth aspect, in some implementation forms of the fourth aspect, obtaining a predicted value of a pixel value of an image block based on a pixel value of a j-th target forward reference block and a pixel value of a j-th target backward reference block, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1, the obtaining includes: When the iteration termination condition is met, obtaining a predicted value of a pixel value of the image block based on the pixel value of the jth target forward reference block and the pixel value of the jth target backward reference block, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1.

[0063] Regarding the fourth aspect, in some implementation forms of the fourth aspect, the first position offset and the second position offset are in a proportional relationship based on the time domain distance, if the first time-domain distance is the same as the second time-domain distance, then the direction of the first position offset is opposite to the direction of the second position offset and the amplitude value of the first position offset is the same as the amplitude value of the second position offset; or if the first time-domain distance is different from the second time-domain distance, a direction of the first position offset is opposite to a direction of the second position offset, and the proportional relationship between the amplitude value of the first position offset and the amplitude value of the second position offset is based on the proportional relationship between the first time-domain distance and the second time-domain distance; The first time-domain distance represents the time-domain distance between the current image to which the current image block belongs and the forward reference image, and the second time-domain distance represents the time-domain distance between the current image and the backward reference image.

[0064] Regarding the fourth aspect, in some implementation forms of the fourth aspect, the i-th motion information includes a forward motion vector, a forward reference image index, a backward motion vector, and a backward reference image index; Determining the positions of the N forward reference blocks and the N backward reference blocks based on the i-th motion information and the position of the current image block includes: Based on the forward motion vector and the position of the current image block, the position of the (i-1)th target forward reference block is calculated as f determining the position of the (i-1)th target forward reference block of the current image block in the forward reference image corresponding to the forward reference image index as a search starting point of the (i-1)th forward reference block, and determining the positions of the (N-1)th candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the (i-1)th target forward reference block and the positions of the (N-1)th candidate forward reference blocks; Based on the backward motion vector and the position of the current image block, the position of the (i-1)th target backward reference block is calculated as b as a search starting point to determine the position of the (i-1)th target backward reference block of the current image block in the backward reference image corresponding to the backward reference image index, and determine the positions of (N-1) candidate backward reference blocks in the backward reference image, where the positions of the N backward reference blocks include the position of the (i-1)th target backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0065] Regarding the fourth aspect, in some implementation forms of the fourth aspect, determining from the M pairs of reference block positions based on a matching cost criterion that the pair of reference block positions is the position of the ith target forward reference block of the current image block and the position of the ith target backward reference block of the current image block includes: determining, from the M pairs of reference block positions, a pair of reference block positions with the smallest matching error is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block; or The method includes determining, from the M pairs of reference block positions, that a pair of reference block positions whose matching error is less than or equal to a matching error threshold is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, where M is less than or equal to N.

[0066] A fifth aspect of the present application provides an image prediction device, including several functional units configured to implement any of the methods in the first aspect. For example, the image prediction device includes: a first acquisition unit configured to acquire initial motion information for a current image block; a first search unit configured to determine positions of N forward reference blocks and N backward reference blocks based on the initial motion information and a position of the current image block, the N forward reference blocks being located in a forward reference image and the N backward reference blocks being located in a backward reference image, where N is an integer greater than 1; and determining, from the positions of M pairs of reference blocks based on a matching cost criterion, the positions of a pair of reference blocks as the position of a target forward reference block of the current image block and the position of a target backward reference block of the current image block, and a first search unit configured to: obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block; and a first prediction unit configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block, wherein for each pair of reference block positions, the first position offset and the second position offset are mirror image relationships, the first position offset represents an offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the initial backward reference block, and M is an integer greater than or equal to 1 and M is less than or equal to N.

[0067] In different application scenarios, the image prediction device is applied in, for example, a video encoding device (video encoder) or a video decoding device (video decoder).

[0068] A sixth aspect of the present application provides an image prediction device, including several functional units configured to implement any of the methods in the second aspect. For example, the image prediction device includes a second acquisition unit configured to acquire initial motion information of a current image block; and a second search unit configured to determine positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; and to determine, from the positions of M pairs of reference blocks based on a matching cost criterion, the positions of a pair of reference blocks as the position of a target forward reference block of the current image block and the position of a target backward reference block of the current image block, where the positions of each pair of reference blocks are determined as the position of a target forward reference block of the current image block and the position of a target backward reference block of the current image block. and a second prediction unit configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block, wherein for each pair of reference block positions, the first position offset and the second position offset are in a proportional relationship based on the time-domain distance, the first position offset represents an offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N.

[0069] In different application scenarios, the image prediction device is applied in, for example, a video encoding device (video encoder) or a video decoding device (video decoder).

[0070] A seventh aspect of the present application provides an image prediction device, including several functional units configured to implement any of the methods in the third aspect. For example, the image prediction device includes: a third acquisition unit configured to acquire i-th motion information of a current image block; and a third search unit configured to determine positions of N forward reference blocks and N backward reference blocks based on the i-th motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; and determine, from the M pairs of reference block positions based on a matching cost criterion, the positions of a pair of reference blocks as the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, where the positions of each pair of reference blocks are the positions of the forward reference block and the position of the backward reference block. and a third search unit configured to: obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the j-th target forward reference block and pixel values ​​of the j-th target backward reference block, wherein j is greater than or equal to i, and both i and j are integers greater than or equal to 1; and a third prediction unit configured to: obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the j-th target forward reference block and pixel values ​​of the j-th target backward reference block, wherein j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

[0071] In different application scenarios, the image prediction device is applied in, for example, a video encoding device (video encoder) or a video decoding device (video decoder).

[0072] An eighth aspect of the present application provides an image prediction device including several functional units configured to implement any of the methods in the fourth aspect. For example, the image prediction device includes a fourth acquisition unit configured to acquire i-th motion information of a current image block; and a fourth search unit configured to determine positions of N forward reference blocks and N backward reference blocks based on the i-th motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; and determine, from the M pairs of reference block positions based on a matching cost criterion, the positions of a pair of reference blocks as the position of an i-th target forward reference block of the current image block and the position of an i-th target backward reference block of the current image block, where the positions of each pair of reference blocks include the position of the forward reference block and the position of the backward reference block, and and a fourth prediction unit configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the j-th target forward reference block and pixel values ​​of the j-th target backward reference block, wherein j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

[0073] In different application scenarios, the image prediction device is applied in, for example, a video encoding device (video encoder) or a video decoding device (video decoder).

[0074] A ninth aspect of the present application provides an image prediction device, the device comprising: a processor; and a memory coupled to the processor, the processor configured to execute a method according to the first aspect, the second aspect, the third aspect, the fourth aspect, or an implementation of the aforementioned aspects.

[0075] A tenth aspect of the present application provides a video encoder configured to encode an image block, the video encoder comprising: an inter-frame prediction module, the inter-frame prediction module comprising the image prediction device according to the fifth, sixth, seventh, or eighth aspect, configured to obtain predicted values ​​of pixel values ​​of the image block through prediction; an entropy coding module configured to code indication information into a bitstream, the indication information being used to indicate initial motion information of the image block; and a reconstruction module configured to reconstruct the image block based on the predicted values ​​of pixel values ​​of the image block.

[0076] An eleventh aspect of the present application provides a video decoder configured to decode a bitstream to obtain an image block, the video decoder comprising: an entropy decoding module configured to decode the bitstream to obtain indication information, where the indication information is used to indicate initial motion information of the image block currently obtained through decoding; an inter-frame prediction module, the inter-frame prediction module comprising the image prediction device according to the fifth, sixth, seventh, or eighth aspect, configured to obtain predicted values ​​of pixel values ​​of the image block through prediction; and a reconstruction module configured to reconstruct the image block based on the predicted values ​​of pixel values ​​of the image block.

[0077] A twelfth aspect of the present application provides a video encoding device, comprising: a non-volatile storage medium and a processor, the non-volatile storage medium storing an executable program, the processor and the non-volatile storage medium being coupled to each other, and the processor executing the executable program implementing the first, second, third, or fourth aspect, or an implementation form of the first, second, third, or fourth aspect.

[0078] A thirteenth aspect of the present application provides a video decoding device, comprising: a non-volatile storage medium and a processor. The non-volatile storage medium stores an executable program. The processor and the non-volatile storage medium are coupled to each other, and the processor executes the executable program that implements the first, second, third, or fourth aspect, or an implementation form of the first, second, third, or fourth aspect.

[0079] A fourteenth aspect of the present application provides a computer-readable storage medium storing instructions that, when executed on a computer, can cause the computer to perform a method according to the first, second, third, or fourth aspect, or an implementation of the first, second, third, or fourth aspect.

[0080] A fifteenth aspect of the present application provides a computer program product comprising instructions that, when executed on a computer, enable the computer to perform a method of the first, second, third, or fourth aspect, or an implementation of the first, second, third, or fourth aspect.

[0081] A sixteenth aspect of the present application provides an electronic device comprising the video encoder of the tenth aspect, the video decoder of the eleventh aspect, or the image prediction device of the fifth, sixth, seventh, or eighth aspect.

[0082] It should be understood that the beneficial effects brought about by these aspects and corresponding implementable design schemes are similar and therefore will not be repeated. [Brief explanation of the drawings]

[0083] [Figure 1] 1 is a schematic block diagram of a video encoding system according to an embodiment of the present application; [Figure 2A] 1 is a schematic block diagram of a video encoder according to an embodiment of the present application; [Figure 2B] FIG. 2 is a schematic block diagram of a video decoder according to an embodiment of the present application; [Figure 3] 1 is a schematic flowchart of an image prediction method according to an embodiment of the present application; [Figure 4] FIG. 1 is a schematic diagram illustrating obtaining initial motion information at the encoder side in a merge mode of inter-frame prediction; [Figure 5] FIG. 1 is a schematic diagram showing obtaining initial motion information at the decoder side in a merge mode of inter-frame prediction; [Figure 6] FIG. 2 is a schematic diagram of an initial reference block for a current image block; [Figure 7] FIG. 2 is a schematic diagram of pixels at integer pixel locations and pixels at fractional pixel locations. [Figure 8] FIG. 1 is a schematic diagram of a search starting point. [Figure 9] FIG. 2 is a schematic block diagram of a first position offset and a second position offset in a mirror image relationship according to an embodiment of the present application; [Figure 10] 4 is a schematic flowchart of another image prediction method according to an embodiment of the present application; [Figure 11] 4 is a schematic flowchart of another image prediction method according to an embodiment of the present application; [Figure 12] 4 is a schematic flowchart of another image prediction method according to an embodiment of the present application; [Figure 13] FIG. 10 is a schematic block diagram of a first position offset and a second position offset in a proportional relationship based on time domain distance according to an embodiment of the present application; [Figure 14] 14 is a schematic flowchart of another image prediction method 1400 according to an embodiment of the present application. [Figure 15] 4 is a schematic flowchart of another image prediction method according to an embodiment of the present application; [Figure 16] 16 is a schematic flowchart of another image prediction method 1600 according to an embodiment of the present application. [Figure 17] 4 is a schematic flowchart of another image prediction method according to an embodiment of the present application; [Figure 18] 1 is a schematic block diagram of an image prediction device according to an embodiment of the present application; [Figure 19] FIG. 2 is a schematic block diagram of another image prediction device according to an embodiment of the present application; [Figure 20] FIG. 2 is a schematic block diagram of another image prediction device according to an embodiment of the present application; [Figure 21] FIG. 2 is a schematic block diagram of another image prediction device according to an embodiment of the present application; [Figure 22] 1 is a schematic block diagram of an encoding or decoding device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0084] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application.

[0085] 1 is a schematic block diagram of a video encoding system according to one embodiment of the present application. In the system, a video encoder 20 and a video decoder 30 are configured to predict pixel value predictions for image blocks based on various exemplary image prediction methods provided herein and refine motion information, such as motion vectors, for currently encoded or decoded image blocks to further improve encoding performance. As shown in FIG. 1, the system includes a source device 12 and a destination device 14. The source device 12 generates encoded video data to be subsequently decoded by the destination device 14. The source device 12 and the destination device 14 may include any one of a wide range of devices, including desktop computers, notebook computers, tablet computers, set-top boxes, telephone handsets such as "smart" phones, "smart" touchpads, televisions, cameras, display devices, digital media players, video game consoles, video streaming transmission devices, or the like.

[0086] The destination device 14 may receive the encoded video data to be decoded by using the link 16. The link 16 may include any type of medium or device capable of moving the encoded video data from the source device 12 to the destination device 14. In one possible implementation, the link 16 may include a communication medium capable of enabling the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and the modulated video data may be transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as the radio frequency spectrum or one or more physical transmission paths. The communication medium may be part of a packet-based network (e.g., a local area network, a wide area network, or the global network of the Internet). The communication medium may include a router, a switch, a base station, or any other device configured to facilitate communication from the source device 12 to the destination device 14.

[0087] Alternatively, the encoded data may be output from the output interface 22 to the storage device 24. Similarly, the encoded data may be accessed from the storage device 24 through the input interface. The storage device 24 may include any of a number of distributed or local data storage media, such as a hard disk drive, a Blu-ray disc, a DVD, a CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium used to store encoded video data. In another possible implementation, the storage device 24 may correspond to a file server or another intermediate storage device capable of storing the encoded video generated by the source device 12. The destination device 14 may access the stored video data from the storage device 24 through a streaming transmission or download function. The file server may be any type of server capable of storing the encoded video data and transmitting the encoded video data to the destination device 14. In one possible implementation, the file server includes a web server, a file transfer protocol server, a network-attached storage device, or a local disk drive. The destination device 14 may access the encoded video data through any standard data connection, including an Internet connection. The data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a cable modem), or a combination thereof applicable to access encoded video data stored on a file server. The transmission of the encoded video data from storage device 24 may be a streaming transmission, a download transmission, or a combination thereof.

[0088] The technology in this application is not necessarily limited to wireless applications or settings. The technology can be applied to video decoding to support any one of a number of multimedia applications, such as television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., over the Internet), digital video encoding for storage on a data storage medium, decoding of digital video stored on a data storage medium, or another application. In some possible implementations, the system can be configured to support one-way or two-way video transmission to support applications such as streaming video transmission, video playback, video broadcasting, and / or video telephony.

[0089] In one possible implementation of FIG. 1 , source device 12 includes video source 18, video encoder 20, and output interface 22. In some applications, output interface 22 may include a modulator / demodulator (modem) and / or a transmitter. In source device 12, video source 18 may include, for example, as a source, a video capture device (e.g., a video camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination thereof. In one possible implementation, if video source 18 is a video camera, source device 12 and destination device 14 may constitute a camera phone or a video phone. For example, the techniques described in this application may be applied to video decoding and may be applied to wireless and / or wired applications.

[0090] Video encoder 20 may encode video that is captured, pre-captured, or generated by a computer. The encoded video data may be transmitted directly to destination device 14 through output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 24 for subsequent access by destination device 14 or another device for decoding and / or playback.

[0091] Destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. In some applications, input interface 28 may include a receiver and / or a modem. Input interface 28 of destination device 14 receives encoded video data using link 16. The encoded video data transmitted or provided to storage device 24 using link 16 may include multiple syntax elements generated by video encoder 20 and used by a video decoder of video decoder 30 to decode the video data. These syntax elements may be included in the encoded video data transmitted over a communication medium, stored on a storage medium, or stored in a file server.

[0092] The display device 32 may be integrated with the destination device 14 or disposed external to the destination device 14. In some possible implementations, the destination device 14 may have an integrated display device and may also be configured to connect to an interface for an external display device. In other possible implementations, the destination device 14 may be a display device. In general, the display device 32 displays the decoded video data to a user and may include any one of a number of display devices, such as a liquid crystal display, a plasma display, an organic light-emitting diode display, or another type of display device.

[0093] Video encoder 20 and video decoder 30 may operate, for example, in accordance with the Next Generation Video Coding and Compression Standard (H.266), currently under development, and may conform to the H.266 Test Model (JEM). Alternatively, video encoder 20 and video decoder 30 may operate, for example, in accordance with other proprietary or industry standards or extensions of the ITU-TH.265 standard or the ITU-TH.264 standard. The ITU-TH.265 standard is also referred to as the High Efficiency Video Decoding Standard, and the ITU-TH.264 standard is alternatively referred to as MPEG-4 Part 10 or advanced video coding (AVC). However, the technology of the present application is not limited to any particular decoding standard. Other possible implementations of video compression standards include MPEG-2 and ITU-TH.263.

[0094] 1, in some aspects, video encoder 20 and video decoder 30 may be integrated with audio encoders and decoders, respectively, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or separate data streams. Where applicable, in some possible implementations, the MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol or other protocols, such as the User Datagram Protocol (UDP).

[0095] Each of the video encoder 20 and the video decoder 30 may be implemented as any of a number of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When these techniques are implemented partially as software, the device may store instructions for the software on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to implement the techniques of the present application. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and any of the one or more encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) within the corresponding device.

[0096] The present application may relate to another device, for example, video encoder 20 communicating certain information as a signal to, for example, video decoder 30. However, it should be understood that video encoder 20 may associate certain syntax elements with encoded portions of video data and communicate the information in the signal. That is, video encoder 20 may store certain syntax elements in header information for the encoded portions of video data and communicate the data in the signal. In some applications, these syntax elements may be encoded and stored (e.g., stored in storage system 34 or file server 36) before being received and decoded by video decoder 30. Thus, the term “signal” may refer to, for example, the transmission of syntax or other data used to decode compressed video data, regardless of whether the transmission occurs in real time, near real time, or within a period of time. For example, the transmission may occur when the syntax elements are stored on a medium during encoding, and the syntax elements may then be retrieved by a decoding device at any time after they are stored on the medium.

[0097] The JCT-VC developed the H.265 (HEVC) standard. The HEVC standardization is based on an evolutionary model for video decoding equipment, referred to as the HEVC Test Model (HM). The latest H.265 standard document is available at http: / / www.itu.int / rec / T-REC-H.265. The latest version of the standard document is H.265(12 / 16), which is incorporated herein by reference in its entirety. The HM assumes that video decoding equipment will have several additional features compared to the existing algorithms of ITU-TH.264 / AVC. For example, while H.264 provides nine intraframe prediction and coding modes, the HM can provide up to 35 intraframe prediction and coding modes.

[0098] JVET is working on the development of the H.266 standard. The H.266 standardization process is based on an evolutionary model of a video decoder, referred to as the H.266 Test Model. The H.266 algorithm description is available at http: / / phenix.int-evry.fr / jvet, and the latest algorithm description is contained in JVET-F1001-v2. This algorithm description is incorporated herein by reference in its entirety. Additionally, reference software for the JEM Test Model is available at https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / , which is also incorporated herein by reference in its entirety.

[0099] Generally, as described in the HM working model, a video frame or image may be divided into a series of treeblocks or largest coding units (LCUs), each containing both luma and chroma samples. LCUs are also referred to as CTUs. A treeblock has a function similar to a macroblock in the H.264 standard. A slice includes several consecutive treeblocks in decoding order. A video frame or image may be partitioned into one or more slices. Each treeblock may be divided into coding units based on a quadtree. For example, a treeblock serving as the root node of the quadtree may be divided into four child nodes, each of which may serve as a parent node and be divided into four other child nodes. The last non-divisible child node serving as a leaf node of the quadtree contains a decoding node, e.g., a decoded image block. The maximum number of times a treeblock can be divided and the minimum size of a decoding node may be defined in syntax data associated with the decoded bitstream.

[0100] A coding unit includes a decoding node, a prediction unit (PU), and a transform unit (TU) associated with the decoding node. A CU must have a size corresponding to the size of the decoding node and be square in shape. The size of a CU may range from 8x8 pixels to at most 64x64 pixels, or a larger treeblock size. Each CU may include one or more PUs and one or more TUs. For example, syntax data associated with a CU may describe partitioning of a CU into one or more PUs. The partitioning pattern may change when the CU is coded in skip or direct mode, in intra-frame prediction mode, or in inter-frame prediction mode. PUs obtained through partitioning may have a non-square shape. For example, syntax data associated with a CU may also describe partitioning of a CU into one or more TUs based on a quadtree. A TU may have a square or non-square shape.

[0101] The HEVC standard allows TU-based transforms, and TUs may be different for different CUs. TU size is typically set based on the size of the PU within a given CU defined for a partitioned LCU. However, this is not always the case. TU size is generally the same as or smaller than the PU size. In some possible implementations, a quadtree structure called a "residual quadtree" (RQT) may be used to divide the residual samples corresponding to a CU into smaller units. Leaf nodes of the RQT may be referred to as TUs. Pixel differences associated with the TUs are transformed to generate transform coefficients, and the transform coefficients may be quantized.

[0102] Generally, a transform and quantization process is used for a TU. A given CU having one or more PUs may also include one or more TUs. After prediction, video encoder 20 may calculate residual values ​​corresponding to the PU. The residual values ​​may include pixel differences, which may be transformed into transform coefficients, which are quantized and subjected to TU scanning to generate serialized transform coefficients for entropy decoding. In this application, the term "image block" is generally used to refer to a decoding node of a CU. In some specific applications, the term "image block" may also be used to refer to a tree block including a decoding node, a PU, and a TU, e.g., an LCU or a CU. In this embodiment of the application, an inverse quantization process of transform coefficients corresponding to a current image block (i.e., a current transform block) is performed to improve coding performance. Various example methods described in the adaptive inverse quantization method in video encoding or decoding are described in detail below.

[0103] A video sequence generally includes a series of video frames or images. For example, a group of pictures (GOP) includes a series of video images, one video image, or multiple video images. A GOP may include syntax data in the header information of the GOP, in the header information of one or more of the images, or elsewhere, describing the number of images included in the GOP. Each slice of an image may include slice syntax data describing the coding mode of the corresponding image. Video encoder 20 typically performs operations on image blocks in several video slices to encode video data. An image block may correspond to a decoding node within a CU. The size of an image block may be fixed or variable and may vary depending on a specified decoding standard.

[0104] In one possible implementation, the HM supports prediction for various PU sizes. Assuming that the size of a given CU is 2N×2N, the HM supports intra-frame prediction for PU sizes of 2N×2N or N×N, and inter-frame prediction for symmetric PU sizes of 2N×2N, 2N×N, N×2N, or N×N. The HM also supports asymmetric partitioning of inter-frame prediction for PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N. In asymmetric partitioning, the CU is not partitioned in one direction, but is partitioned into two parts in the other direction, one part occupying 25% of the CU and the other part occupying 75% of the CU. The part occupying 25% of the CU is indicated by an indicator including "n" followed by "Up," "Down," "Left," or "Right." So, for example, "2NxnU" refers to a 2Nx2N CU partitioned horizontally, with a 2Nx0.5N PU above and a 2Nx1.5N PU below.

[0105] In this application, "NxM" and "N times M" may be used interchangeably to refer to the pixel size of an image block in the vertical and horizontal dimensions, e.g., 16x8 pixels or 16 times 8 pixels. Generally, a 16x8 block has 16 pixels horizontally and 8 pixels vertically. In other words, the width of the image block is 16 pixels and the height of the image block is 8 pixels.

[0106] After intra-frame or inter-frame predictive decoding of the PUs in a CU, video encoder 20 may calculate residual data for the TUs in the CU. The PUs may include pixel data in the spatial domain (also referred to as the pixel domain). The TUs may include coefficients in the transform domain after a transform (e.g., a discrete cosine transform (DCT), an integer transform, a wavelet transform, or other conceptually similar transform) is performed on the residual video data. The residual data may correspond to differences between pixel values ​​of the uncoded image and predicted pixel values ​​corresponding to the PU. Video encoder 20 may generate TUs including the residual data of the CU and then transform the TUs to generate CU transform coefficients.

[0107] In this embodiment of the present application, various exemplary methods of an inter-frame prediction process in video encoding or decoding are described in detail below to obtain the sampling values ​​of the sampling points of the optimal forward reference block of the current image block and the sampling values ​​of the sampling points of the optimal backward reference block of the current image block, and further predict the sampling values ​​of the sampling points of the current image block. An image block is a two-dimensional sampling point array, which may be a square array or a rectangular array. For example, a 4x4 image block can be considered as a square sampling point array formed by a total of 4x4=16 sampling points. The signals in the image block are the sampling values ​​of the sampling points in the image block. Furthermore, sampling points may be referred to as samples or pixels, which shall be used interchangeably in this specification of the present invention. Correspondingly, the values ​​of sampling points may be referred to as pixel values, which shall be used interchangeably in this application. An image may be represented as a two-dimensional sampling point array, which is represented by using a method similar to that used for image blocks.

[0108] After performing the transform to generate transform coefficients, video encoder 20 may quantize the transform coefficients. Quantization, e.g., the process of quantizing the coefficients, reduces the amount of data used to represent the coefficients and implements further compression. The quantization process may reduce the bit depth associated with some or all of the coefficients. For example, during quantization, an n-bit value may be reduced to an m-bit value, where n is greater than m.

[0109] The JEM model further improves the video image coding structure. In particular, a block coding structure called "quadtree plus binary tree" (QTBT) is introduced. Without using concepts like CU, PU, ​​and TU in HEVC, the QTBT structure supports more flexible CU partitioning shapes. A CU can be square or rectangular. Quadtree partitioning is first performed on CTUs, and binary tree partitioning is further performed on the leaf nodes of the quadtree. Furthermore, there are two binary tree partitioning modes: symmetric horizontal partitioning and symmetric vertical partitioning. The leaf nodes of the binary tree are called CUs. CUs in JEM cannot be further partitioned during prediction and transform. In other words, CUs, PUs, and TUs in JEM have the same block size. In the existing JEM, the maximum CTU size is 256 × 256 luma pixels.

[0110] In some possible implementations, video encoder 20 may scan the quantized transform coefficients in a predefined scan order to generate a serialized vector that can be entropy coded. In other possible implementations, video encoder 20 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, video encoder 20 may perform entropy decoding on the one-dimensional vector by using context-based adaptive variable-length coding (CAVLC), context-based adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) decoding, or another entropy decoding method. Video encoder 20 may further perform entropy coding on syntax elements associated with the coded video data to decode the video data at video decoder 30.

[0111] 2A is a schematic block diagram of a video encoder 20 according to one embodiment of the present application. Referring also to FIG. 3, the video encoder 20 may perform an image prediction process, and in particular, a motion compensation unit 44 within the video encoder 20 may perform the image prediction process.

[0112] 2A , video encoder 20 may include a prediction module 41, an adder 50, a transform module 52, a quantization module 54, and an entropy coding module 56. In one example, prediction module 41 may include a motion estimation unit 42, a motion compensation unit 44, and an intra-frame prediction unit 46. The internal structure of prediction module 41 is not limited to this embodiment of the present application. Optionally, for a video encoder with a hybrid architecture, video encoder 20 may further include an inverse quantization module 58, an inverse transform module 60, and an adder 62.

[0113] 2A, video encoder 20 may further include a partitioning unit (not shown) and reference image memory 64. It should be understood that partitioning unit and reference image memory 64 may alternatively be disposed external to video encoder 20.

[0114] In another possible implementation, video encoder 20 may further include a filter (not shown) that filters block boundaries to remove block effect artifacts from the reconstructed video. When necessary, the filter typically performs filtering on the output of adder 62.

[0115] 2A, video encoder 20 receives video data, and a partitioning unit partitions the data into image blocks. Such partitioning may further include partitioning into slices, image blocks, or other larger units, for example, image block partitioning based on a quadtree structure of LCUs and CUs. Generally, a slice may be divided into multiple image blocks.

[0116] Prediction module 41 is configured to generate a prediction block of a current coded image block. Prediction module 41 may select one of multiple possible decoding modes for the current image block, for example, one of multiple intra-frame decoding modes or one of multiple inter-frame decoding modes, based on the quality of the coding and a cost calculation result (e.g., rate-distortion cost, RDcost). Prediction module 41 may provide the intra-frame decoded or inter-frame decoded block to adder 50 to generate residual block data, provide the intra-frame decoded or inter-frame decoded block to adder 62 to reconstruct the coded block, and use the reconstructed block as a reference image.

[0117] The motion estimation unit 42 and motion compensation unit 44 in the prediction module 41 perform inter-frame predictive decoding on a current image block with respect to one or more predictive blocks in one or more reference images to provide temporal compression. The motion estimation unit 42 is configured to determine an inter-frame prediction mode for a video slice based on a preset mode of the video sequence. In the preset mode, the video slices in the sequence may be designated as P slices, B slices, or GPB slices. The motion estimation unit 42 and the motion compensation unit 44 may be closely integrated, but are described separately for conceptual purposes. Motion estimation performed by the motion estimation unit 42 is a process of generating motion vectors to estimate image blocks. For example, the motion vector may indicate the displacement of a PU of an image block in a current video frame or image relative to a predictive block in a reference image.

[0118] The prediction block is a block within a PU that is known to exactly match the image block to be decoded based on pixel differences, which may be determined based on sum of absolute differences (SAD), sum of squared differences (SSD), or another difference metric. In some possible implementations, video encoder 20 may calculate values ​​for sub-integer pixel locations of reference images stored in reference image memory 64.

[0119] By comparing the position of the PU with the position of the prediction block of the reference image, the motion estimation unit 42 calculates the motion vector of the PU of the image block in the inter-frame decoded slice. The reference image may be selected from a first reference image list (List 0) or a second reference image list (List 1). Each entry in the list identifies one or more reference images stored in the reference image memory 64. The motion estimation unit 42 sends the calculated motion vector to the entropy coding unit 56 and the motion compensation unit 44.

[0120] The motion compensation performed by the motion compensation unit 44 may include removing or generating a predictive block based on a motion vector determined through motion estimation, and sub-pixel level interpolation may be performed. After receiving the motion vector of the PU of the current image block, the motion compensation unit 44 may locate the predictive block pointed to by the motion vector in one of the reference image lists. The video encoder 20 subtracts pixel values ​​of the predictive block from pixel values ​​of the current image block being decoded to obtain a residual image block and obtain pixel differences. The pixel differences form residual data of the block and may include a luma difference component and a chroma difference component. The adder 50 is one or more components that perform the subtraction operation. The motion compensation unit 44 may further generate syntax elements associated with the image block and the video slice, so that the video decoder 30 can decode the image block of the video slice. Next, the image prediction process in the embodiment of the present application will be described in detail with reference to Figures 3, 10 to 12, and 14 to 17. Details will not be described here.

[0121] Intra prediction unit 46 in prediction module 41 may perform intra predictive decoding on a current image block relative to one or more neighboring blocks in an image or slice that are the same as the current block to be decoded to provide spatial compression. Thus, instead of inter prediction (as described above) being performed by motion estimation unit 42 and motion compensation unit 44, intra prediction unit 46 may perform intra prediction on the current block. Specifically, intra prediction unit 46 may determine an intra prediction mode for encoding the current block. In some possible implementations, intra prediction unit 46 may use different intra prediction modes to encode the current block during (for example) separate encoding traversals, and intra prediction unit 46 (or in some possible implementations, mode selection unit 40) may select an appropriate intra prediction mode from the test modes.

[0122] After prediction module 41 generates a predictive block for a current image block by performing inter-frame prediction or intra-frame prediction, video encoder 20 generates a residual image block by subtracting the predictive block from the current image block. The residual video data in the residual block may be included in one or more TUs and applied to transform module 52. Transform module 52 is configured to transform the residual between the original block of the current coded image block and the predictive block of the current image block. Transform module 52 converts the residual data into residual transform coefficients, for example, by performing a discrete cosine transform (DCT) or a conceptually similar transform (e.g., a discrete sine transform (DST)). Transform module 52 may transform the residual video data from pixel domain data to domain (e.g., frequency domain) data.

[0123] The transform module 52 may send the obtained transform coefficients to the quantization module 54, which quantizes the transform coefficients to further reduce the bit rate. In some possible implementations, the quantization module 54 may subsequently scan the matrix containing the quantized transform coefficients. Alternatively, the entropy coding module 56 may perform the scan.

[0124] After quantization, entropy coding module 56 may perform entropy coding on the quantized transform coefficients. For example, entropy coding module 56 may perform context-based adaptive variable length decoding (CAVLC), context-based adaptive binary arithmetic decoding (CABAC), syntax-based context-adaptive binary arithmetic decoding (SBAC), probability interval partitioning entropy (PIPE) decoding, or another entropy coding method or technique. Entropy coding module 56 may also perform entropy coding on motion vectors and other syntax elements of the current video slice being coded. After entropy coding module 56 performs entropy coding, the coded bitstream may be transmitted to video decoder 30 or stored by video decoder 30 for subsequent transmission or search.

[0125] Inverse quantization module 58 and inverse transform module 60 perform inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain as a reference block in a reference image. Adder 62 adds the reconstructed residual block to the prediction block generated by prediction module 41 to generate a reconstructed block, which is used as a reference block and stored in reference image memory 64. The reference block may be used by motion estimation unit 42 and motion compensation unit 44 as a reference block to perform inter-frame prediction on blocks in subsequent video frames or images.

[0126] It should be understood that other structural variations of the video encoder 20 can also be used to encode the video stream. For example, for some image blocks or image frames, the residual signal may be directly quantized by the video encoder 20 without being processed by the transform module 52, and correspondingly, the residual signal does not need to be processed by the inverse transform module 58. Alternatively, for some image blocks or image frames, the video encoder 20 does not generate residual data, and correspondingly, no processing needs to be performed by the transform module 52, the quantization module 54, the inverse quantization module 58, and the inverse transform module 60. Alternatively, the reconstructed image block may be directly stored as a reference block by the video encoder 20 without being processed by a filter unit. Alternatively, the quantization module 54 and the inverse quantization module 58 in the video encoder 20 may be integrated. Alternatively, the transform module 52 and the inverse transform module 60 in the video encoder 20 may be integrated. Alternatively, the adder 50 and the adder 62 may be integrated.

[0127] 2B is a schematic block diagram of a video decoder 30 according to one embodiment of the present application. Also referring to FIG. 3, FIG. 10 to FIG. 12, and FIG. 14 to FIG. 17, the video decoder 30 may perform an image prediction process, and in particular, a motion compensation unit 82 within the video decoder 30 may perform the image prediction process.

[0128] 2B, video decoder 30 may include an entropy decoding module 80, a prediction processing module 81, an inverse quantization module 86, an inverse transform module 88, and a reconstruction module 90. In one example, prediction module 81 may include a motion compensation unit 82 and an intra-frame prediction unit 84, which is not limited in this embodiment of the present application.

[0129] In one possible implementation, video decoder 30 may further include reference image memory 92. It should be understood that reference image memory 92 may alternatively be disposed external to video decoder 30. In some possible implementations, video decoder 30 may perform an exemplary decoding process that is the reverse of the encoding process described in video encoder 20 in FIG.

[0130] During decoding, video decoder 30 receives from video encoder 20 an encoded video bitstream representing image blocks of encoded video slices and associated syntax elements. Video decoder 30 may receive syntax elements at the video slice level and / or the image block level. An entropy decoding module 80 of video decoder 30 performs entropy decoding on the bitstream to generate quantized coefficients and some syntax elements. Entropy decoding module 80 forwards the syntax elements to a prediction module 81. In the present application, in one example, the syntax elements herein may include inter-frame prediction data related to a current image block, where the inter-frame prediction data may include an index identifier block_based_index, which may indicate which motion information (also referred to as initial motion information of the current image block) is used by the current image block. Optionally, the inter-frame prediction data may further include a switch flag block_based_enable_flag, which indicates whether to perform image prediction on the current image block by using Figure 3 or Figure 14 (in other words, whether to perform inter-frame prediction on the current image block by using the MVD mirror constraint proposed in the present application), or whether to perform image prediction on the current image block by using Figure 12 or Figure 16 (in other words, whether to perform inter-frame prediction on the current image block by using the proportionality relationship proposed in the present application based on the time-domain distance).

[0131] When a video slice is decoded into an intra-frame decoded (I) slice, the intra-frame prediction unit 84 of the prediction module 81 may generate a prediction block for an image block of the current video slice based on an intra-frame prediction mode signaled by transmitting signals and data of previously decoded blocks from the current frame or image. When a video slice is decoded into an inter-frame decoded (i.e., B or P) slice, the motion compensation unit 82 of the prediction module 81 may determine an inter-frame prediction mode to be used to decode the current image block of the current video slice based on a syntax element received from the entropy decoding module 82, and decode the current image block (e.g., perform inter-frame prediction on the current image block) based on the determined inter-frame prediction mode. In particular, the motion compensation unit 82 may determine which prediction method is used to predict the current image block of the current video slice; for example, the syntax element indicates that an image prediction method based on an MVD mirror constraint should be used to predict the current image block. The motion information of the current image block of the current video slice is predicted or refined, thereby generating a predictive block for the current image block by using the predicted motion information of the current image block through a motion compensation process. The motion information herein may include reference image information and a motion vector. The reference image information may include, but is not limited to, unidirectional / bidirectional prediction information, a reference image list number, and a reference image index corresponding to the reference image list. For inter-frame prediction, the predictive block may be generated from one of the reference images in one of the reference image lists. The video decoder 30 may construct the reference image lists, i.e., List 0 and List 1, based on the reference images stored in the reference image memory 92. The reference frame index of the current image may be included in one or both of Reference Frame List 0 and Reference Frame List 1. In some cases, the video encoder 20 may send a signal to indicate which new image prediction method is used.

[0132] In this embodiment, prediction module 81 is configured to generate a predictive block of a currently coded image block. In particular, when a video slice is decoded into an intra-frame decoded (I) slice, intra-frame prediction unit 84 of prediction module 81 may generate a predictive block of an image block of the current video slice based on the transmitted signaled intra-frame prediction mode and data of previously decoded image blocks from the current frame or image. When a video image is decoded into an inter-frame decoded (e.g., B, P, or GPB) slice, motion compensation unit 82 of prediction module 81 generates a predictive block of an image block of the current video image based on the motion vectors and other syntax elements received from entropy coding unit 80.

[0133] Inverse quantization module 86 performs inverse quantization, i.e., dequantizes, on the quantized transform coefficients provided in the bitstream obtained by entropy decoding module 80 through decoding. The inverse quantization process may involve determining the degree of quantization to be applied by using quantization parameters calculated by video encoder 20 for each image block in a video slice, and similarly determining the degree of inverse quantization to be applied. Inverse transform module 88 performs an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, on the transform coefficients to produce pixel domain residual blocks.

[0134] After motion compensation unit 82 generates a predictive block for the current image block, video decoder 30 adds the residual block from inverse transform module 88 and the corresponding predictive block generated by motion compensation unit 82 to obtain a reconstructed block, i.e., a decoded image block. Adder 90 represents a component that performs the addition operation. When necessary, a loop filter (in the decoding loop or after the decoding loop) may be further used to smooth pixel transformations or improve video quality in another manner. The filter unit (not shown) may represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Furthermore, decoded image blocks in a given frame or image may be further stored in a decoded image buffer 92, which stores reference images used for subsequent motion compensation. Decoded image buffer 92 may be part of a memory and may further store decoded video for subsequent display on a display device (e.g., display device 32 of FIG. 1). Alternatively, the decoded image buffer 9 may be separate from such memory.

[0135] It should be understood that other structural variations of video decoder 30 can be used to decode the encoded video bitstream. For example, video decoder 30 may generate an output video stream without processing by a filter unit. Alternatively, for some image blocks or image frames, entropy decoding module 80 of video decoder 30 does not obtain quantized coefficients through decoding, and correspondingly, processing by inverse quantization module 86 and inverse transformer 88 is not required. For example, inverse quantization module 86 and inverse transform module 88 in video decoder 30 may be integrated.

[0136] FIG. 3 is a schematic flowchart of an image prediction method according to an embodiment of the present application. The method illustrated in FIG. 3 may be performed by a video encoding device, a video codec, a video encoding system, or another device having video encoding functionality. The method illustrated in FIG. 3 may be used in an encoding process or a decoding process. More specifically, the method illustrated in FIG. 3 may be used in an inter-frame prediction process during encoding or decoding. Process 300 may be performed by the video encoder 20 or the video decoder 30, and in particular, may be performed by a motion compensation unit of the video encoder 20 or the video decoder 30. For a video data stream having multiple video frames, the video encoder or the video decoder is used to predict predicted values ​​of pixel values ​​of a current image block of a current video frame, including the following steps:

[0137] The method shown in FIG. 3 includes steps 301 to 304, which are described in detail below.

[0138] 301: Obtain initial motion information for the current image block.

[0139] An image block herein may be an image block within an image to be processed or a sub-image within an image to be processed. Furthermore, an image block herein may be an image block to be coded in an encoding process or an image block to be decoded in a decoding process.

[0140] Furthermore, the initial motion information may include indication information of a prediction direction (usually bidirectional prediction), a motion vector pointing to a reference image block (usually a motion vector of an adjacent block), and information of an image in which the reference image block is located (usually understood as reference image information). The motion vector includes a forward motion vector and a backward motion vector, and the reference image information includes reference frame index information of a forward prediction reference image block and a backward prediction reference image block.

[0141] The initial motion information of the image block can be obtained in several ways, for example, the initial motion information of the image block can be obtained in the following ways 1 and 2:

[0142] Method 1: 4 and 5, in the merge mode of inter-frame prediction, a candidate motion information list is constructed based on the motion information of neighboring blocks of a current image block, and one candidate motion information is selected from the candidate motion information list as the initial motion information of the current image block. The candidate motion information list includes a motion vector, reference frame index information, and the like. For example, the motion information of neighboring block A0 (referring to the candidate motion information with index 0 in FIG. 5) is selected as the initial motion information of the current image block. In particular, the forward motion vector of A0 is used as the forward predicted motion vector of the current block, and the backward motion vector of A0 is used as the backward predicted motion vector of the current block.

[0143] Method 2: In the non-merge mode of inter-frame prediction, a motion vector predictor list is constructed based on motion information of neighboring blocks of a current image block, and a motion vector is selected from the motion vector predictor list as the motion vector predictor of the current image block. In this case, the motion vector of the current image block may be the sum of the motion vector values ​​of the neighboring blocks or the difference between the motion vector of the selected neighboring blocks and the motion vector of the current image block. The motion vector difference is the difference between the motion vector obtained by performing motion estimation on the current image block and the motion vector of the selected neighboring block. For example, the motion vectors corresponding to indexes 1 and 2 in the motion vector predictor list are selected as the forward and backward motion vectors of the current image block.

[0144] It should be understood that the above-mentioned Scheme 1 and Scheme 2 are merely two specific schemes for obtaining initial motion information of an image block. In this application, the scheme for obtaining motion information of a prediction block is not limited, and any scheme that can obtain initial motion information of an image block falls within the protection scope of this application.

[0145] 302: Determine the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and position of the current image block, where the N forward reference blocks are arranged in the forward reference image and the N backward reference blocks are arranged in the backward reference image, and N is an integer greater than 1.

[0146] Referring to FIG. 6, the current image to which the current image block in this embodiment of the present application belongs has two reference images, namely a forward reference image and a backward reference image.

[0147] In one example, the initial motion information includes a first motion vector and a first reference picture index in a forward prediction direction, and a second motion vector and a second reference picture index in a backward prediction direction.

[0148] Correspondingly, step 302 is According to the first motion vector and the position of the current image block, determine the position of the initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index, using the position of the initial forward reference block as a first search starting point (shown as (0,0) in FIG. 8), and determine the positions of (N-1) candidate forward reference blocks in the forward reference image; Based on the second motion vector and position of the current image block, using the position of the initial backward reference block as a second search starting point, determine the position of the initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index, and determine the positions of (N-1) candidate backward reference blocks in the backward reference image.

[0149] In one example, referring to Figure 7, the positions of the N forward reference blocks include the position of an initial forward reference block (denoted as (0,0)) and the positions of (N-1) candidate forward reference blocks (denoted as (0,-1), (-1,-1), (-1,1), (1,-1), (1,1), and the like), where the offset of the position of each candidate forward reference block from the position of the initial forward reference block is an integer pixel distance (as shown in Figure 8) or a fractional pixel distance, where N = 9; or the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of (N-1) candidate backward reference blocks, where the offset of the position of each candidate backward reference block from the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance, where N = 9.

[0150] Referring to Figure 8, in the motion estimation or motion compensation process, the MV precision may be fractional pixel precision (e.g., 1 / 2 pixel precision or 1 / 4 pixel precision). If an image only has pixel values ​​of integer pixels and the current MV precision is fractional pixel precision, interpolation needs to be performed by using an interpolation filter to obtain pixel values ​​at fractional pixel positions and by using pixel values ​​at integer pixel positions of a reference image, thereby obtaining pixel values ​​at fractional pixel positions, and the obtained pixel values ​​are used as values ​​of the predicted block of the current block. A specific interpolation process is related to the interpolation filter used. Generally, pixel values ​​of integer samples around a reference sample may be linearly weighted to obtain the value of the reference sample. Common interpolation filters include 4-tap, 6-tap, and 8-tap interpolation filters, and the like.

[0151] As shown in Figure 7, Ai,j is a sample at an integer pixel position and its bit width is bitDepth. a0,0, b0,0, c0,0, d0,0, h0,0, n0,0, e0,0, i0,0, p0,0, f0,0, j0,0, q0,0, g0,0, k0,0, and r0,0 are samples at fractional pixel positions. If an 8-tap interpolation filter is used, a0,0 can be obtained by calculating using the following formula: a0,0=(C0*A -3,0 +C1*A -2,0 +C2*A -1,0 +C3*A 0,0 +C4*A 1,0 +C5*A 2,0 +C6*A 3,0 +C7*A 4,0 )>> shift1

[0152] In the above formula, C k is the coefficient of the interpolation filter, and k=0, 1, ..., 7. If the sum of the coefficients of the interpolation filter is 2 to the Nth power, the gain of the interpolation filter is N. For example, N=6 indicates that the gain of the interpolation filter is 6 bits. shift1 is the number of bits of right shift, and shift1 may be set to bitDepth-8, where bitDepth is the target bitwidth. Thus, based on the above formula, the finally obtained bitwidth of the pixel value of the prediction block is bitDepth+6-shift1=14 bits.

[0153] 303: From the M pairs of reference block positions based on the matching cost criterion, determine a pair of reference block positions to be the position of a target forward reference block of a current image block and the position of a target backward reference block of the current image block, where the positions of each pair of reference blocks include the position of the forward reference block and the position of the backward reference block, and for each pair of reference block positions, the first position offset and the second position offset are in a mirror image relationship, the first position offset represents the offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents the offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N.

[0154] 9, the offset of the position of the candidate forward reference block 904 in the forward reference image Ref0 relative to the position of the initial forward reference block 902 (i.e., the forward search base point) is MVD0(delta0x, delta0y). The offset of the position of the candidate backward reference block 905 in the backward reference image Ref1 relative to the position of the initial backward reference block 903 (i.e., the backward search base point) is MVD1(delta1x, delta1y).

[0155] MVD0=-MVD1, specifically, delta0x=-delta1x, and delta0y=-delta1y.

[0156] In a different example, step 303 includes: This may include determining, from the positions of M pairs of reference blocks (one forward reference block and one backward reference block), the positions of the pair of reference blocks with the smallest matching error are the positions of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, or determining, from the positions of M pairs of reference blocks, the positions of the pair of reference blocks with a matching error equal to or less than a matching error threshold are the positions of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, where M is less than or equal to N. Furthermore, the difference between pixel values ​​of the forward reference block and pixel values ​​of the backward reference block may be measured by using sum of absolute differences (SAD), sum of absolute transformation differences (SATD), sum of absolute squared differences, or the like.

[0157] 304: Obtain a predicted value of a pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block.

[0158] In one example, in step 304, a weighting process is performed on the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block to obtain a predicted value of the pixel value of the current image block.

[0159] Optionally, in one embodiment, the method shown in Figure 3 further includes obtaining updated motion information for the current image block, where the updated motion information includes an updated forward motion vector and an updated backward motion vector, where the updated forward motion vector points to the position of the target forward reference block and the updated backward motion vector points to the position of the target backward reference block. The updated motion information for the current image block can be obtained based on the position of the target forward reference block, the position of the target backward reference block, and the position of the current image block, or is obtained based on a first position offset and a second position offset corresponding to the positions of the determined pair of reference blocks.

[0160] The motion vector of the image block is updated. In this way, another image block can be effectively predicted based on the image block during next image prediction.

[0161] It can be seen that in this embodiment of the present application, the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. There is a mirror image relationship between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward prediction block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating a template matching block and the process of performing forward search matching and backward search matching by using the template matching block, and simplifies the image prediction process. This improves image prediction accuracy and reduces image prediction complexity.

[0162] Next, the image prediction method in the embodiment of the present application will be described in detail with reference to FIG.

[0163] 10 is a schematic flowchart of an image prediction method according to an embodiment of the present application. The method shown in FIG. 10 can be performed by a video encoding device, a video codec, a video encoding system, or another device having video encoding functionality. The method shown in FIG. 10 can be used in an encoding process or a decoding process. More specifically, the method shown in FIG. 10 can be used in an inter-frame prediction process during encoding or decoding.

[0164] The method shown in FIG. 10 includes steps 1001 to 1007, which are described in detail below.

[0165] 1001: Get the initial motion information of the current block.

[0166] For example, for an image block whose inter-frame prediction / coding mode is merge, a group of motion information is obtained from a merge candidate list based on a merge index, and the motion information is the initial motion information of the current block. For example, for an image block whose inter-frame prediction / coding mode is AMVP, an MVP is obtained from an MVP candidate list based on an AMVP mode index, and the MV of the current block is obtained by obtaining the sum of the MVP and MVD included in the bitstream. The initial motion information includes reference image indication information and a motion vector. The forward reference image and the backward reference image are determined by using the reference image indication information. The positions of the forward reference block and the backward reference block are determined by using the motion vector.

[0167] 1002: Determine the position of the starting forward reference block of the current image block in the forward reference image, where the position of the starting forward reference block is the search starting point (also called the search base point) in the forward reference image.

[0168] In particular, a search base point in the forward reference image (hereinafter referred to as the first search base point) is obtained based on the forward MV and position information of the current block. For example, the forward MV information is (MV0x, MV0y). The position information of the current block is (B0x, B0y). The first search base point in the forward reference image is (MV0x+B0x, MV0y+B0y).

[0169] 1003: Determine the position of a starting backward reference block of the current image block in the backward reference image, where the position of the starting backward reference block is the search starting point in the backward reference image.

[0170] In particular, the search base point in the backward reference image (hereinafter referred to as the second search base point) is obtained based on the backward MV and position information of the current block. For example, the backward MV is (MV1x, MV1y). The position information of the current block is (B0x, B0y). The second search base point in the backward reference image is (MV1x+B0x, MV1y+B0y).

[0171] 1004: Based on the MVD mirror image constraint, determine the location of a pair of best-matched reference blocks (ie, one forward reference block and one backward reference block), and obtain an optimal forward motion vector and an optimal backward motion vector.

[0172] The MVD mirror constraint in this specification can be described as follows: The offset of the block position in the forward reference image relative to the forward search base point is MVD0(delta0x, delta0y). The offset of the block position in the backward reference image relative to the backward search base point is MVD1(delta1x, delta1y). The following relationships are satisfied: MVD0=-MVD1, specifically, delta0x=-delta1x, and delta0y=-delta1y.

[0173] 7, in the forward reference image, an integer-pixel step motion search is performed by using a search origin (denoted by (0,0)) as a starting point. The integer-pixel step means that the offset of the position of the candidate reference block relative to the search origin is an integer-pixel distance. It should be understood that regardless of whether the search origin is an integer sample (whether the starting point is an integer pixel or a sub-pixel, for example, 1 / 2, 1 / 4, 1 / 8, or 1 / 16), the integer-pixel step motion search may be performed first to obtain the position of the forward reference block of the current image block. It should be noted that when the search is performed by using integer-pixel step, the search starting point may be an integer pixel or a fractional pixel, for example, an integer pixel, 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, or 1 / 16 pixel.

[0174] As shown in FIG. 7, the point (0,0) is used as the search base point, and eight search points of integer pixel steps around the search base point are searched to obtain the position of the corresponding candidate reference block. FIG. 7 shows eight candidate reference blocks. If the offset of the position of the forward candidate reference block in the forward reference image relative to the position of the forward search base point is (-1,-1), the offset of the position of the corresponding backward candidate reference block in the backward reference image relative to the position of the backward search base point is (1,1). Thus, the positions of the paired forward candidate reference block and backward candidate reference block are obtained. For the obtained positions of the paired reference blocks, the matching cost between the two corresponding candidate reference blocks is calculated. The forward reference block and backward reference block with the smallest matching cost are selected as the optimal forward reference block and optimal backward reference block, and the optimal forward motion vector and optimal backward motion vector are obtained.

[0175] 1005 and 1006: Perform a motion compensation process by using the optimal forward motion vector obtained in step 1004 to obtain pixel values ​​of the optimal forward reference block, and perform a motion compensation process by using the optimal backward motion vector obtained in step 1004 to obtain pixel values ​​of the optimal backward reference block.

[0176] 1007: Perform a weighting process on the obtained pixel values ​​of the optimal forward reference block and the obtained pixel values ​​of the optimal backward reference block to obtain a predicted value of the pixel value of the current image block.

[0177] In particular, the predicted values ​​of the pixel values ​​of the current image block may be obtained based on the following equation (2): predSamples'[x][y]=(predSamplesL0'[x][y]+predSamplesL1'[x][y]+1)>>1 (2)

[0178] In the above formula, predSamplesL0' is the optimal forward reference block, predSamplesL1' is the optimal backward reference block, predSamples' is the predicted block of the current image block, predSamplesL0'[x][y] is the pixel value of the optimal forward reference block at sample (x, y), predSamplesL1'[x][y] is the pixel value of the optimal backward reference block at sample (x, y), and predSamples'[x][y] is the final pixel value of the predicted block at sample (x, y).

[0179] It should be noted that in this embodiment of the present application, the search method to be used is not limited, and any search method may be used.For each forward candidate block obtained through search, the difference between the forward candidate block and the corresponding backward candidate block in step 4 is calculated, and the forward candidate block and the backward candidate block with the smallest SAD, the forward motion vector corresponding to the forward candidate block, and the backward motion vector corresponding to the backward candidate block are selected as the optimal forward predicted block, the optimal backward predicted block, the optimal forward motion vector corresponding to the optimal forward predicted block, and the optimal backward motion vector corresponding to the optimal backward predicted block, respectively.Alternatively, for each backward candidate block obtained through search, the difference between the backward candidate block and the corresponding forward candidate block in step 4 is calculated, and the backward candidate block and the forward candidate block with the smallest SAD, the backward motion vector corresponding to the backward candidate block, and the forward motion vector corresponding to the forward candidate block are selected as the optimal backward reference block, the optimal forward reference block, the optimal backward motion vector corresponding to the optimal backward reference block, and the optimal forward motion vector corresponding to the optimal forward reference block, respectively.

[0180] It should be noted that only one example of a search method based on integer pixel steps is presented in step 1004. In fact, in addition to the search using integer pixel steps, the search using fractional pixel steps can also be used. For example, in step 1004, the search using integer pixel steps is performed followed by the search using fractional pixel steps. Alternatively, the search using fractional pixel steps is performed directly. The specific search method is not limited herein.

[0181] It should be noted that in this embodiment of the present application, the method for calculating the matching cost is not limited. For example, the SAD criterion, the MR-SAD criterion, or another criterion may also be used. Furthermore, the matching cost may be calculated by using only the luma component or by using both the luma component and the chroma component.

[0182] It should be noted that in the search process, if the matching cost is 0 or reaches a preset threshold, the traversal or search operation can be terminated in advance. The early termination conditions of the search method are not limited herein.

[0183] It should be understood that step 1005 and the sequence of steps 1005 are not limited and they may be performed simultaneously or sequentially.

[0184] It can be seen that in existing methods, a template matching block needs to be calculated first, and forward search and backward search are carried out separately by using the template matching block, but in this embodiment of the present application, in the process of searching for a matching block, the matching cost is calculated directly by using a candidate block in a forward reference image and a candidate block in a backward reference image, thereby determining the two blocks with the smallest matching cost.This simplifies the image prediction process, improves the image prediction accuracy, and reduces the complexity.

[0185] Figure 11 is a schematic flowchart of an image prediction method according to an embodiment of the present application. The method shown in Figure 11 can be performed by a video encoding device, a video codec, a video encoding system, or another device with video encoding capabilities. The method shown in Figure 11 includes steps 1101 to 1105. For steps 1101 to 1103 and step 1105, please refer to the descriptions of steps 1001 to 1003 and step 1007 in Figure 10. The details will not be described again here.

[0186] The difference between this embodiment of the present application and the embodiment shown in FIG. 10 is that the pixel values of the current optimal forward and backward reference blocks are retained and updated in the search process. After the search is completed, the predicted value of the pixel value of the current image block can be calculated by using the pixel values of the current optimal forward and backward reference blocks.

[0187] For example, the positions of N pairs of reference blocks need to be traversed. Costi is the i-th matching cost, and MinCost indicates the current minimum matching cost. Bfi is the pixel value of the forward reference block, Bbi is the pixel value of the backward reference block, and the pixel values are obtained at the i-th time. BestBf is the value of the current optimal forward reference block, and BestBb is the value of the current optimal backward reference block. CalCost(M, N) represents the matching cost between block M and block N.

[0188] When the search starts (i = 1), MinCost = Cost0 = CalCost(Bf0, Bb0), BestBf = Bf0, and BestBb = Bb0.

[0189] When other pairs of reference blocks are subsequently traversed, the update is executed in real time. For example, when the i-th (i>1) search is executed, if Costi < MinCost, then BestBf = Bfi and BestBb = Bbi; otherwise, the update is not executed.

[0190] When the search ends, BestBf and BestBb are used to obtain the predicted value of the pixel value of the current block.

[0191] FIG. 12 is a schematic flowchart of an image prediction method according to an embodiment of the present application. The method shown in FIG. 12 may be performed by a video encoding device, a video codec, a video encoding system, or another device having video encoding functionality. The method shown in FIG. 12 may be used in an encoding process or a decoding process. More specifically, the method shown in FIG. 3 may be used in an inter-frame prediction process during encoding or decoding. Process 1200 may be performed by the video encoder 20 or the video decoder 30, and in particular, may be performed by a motion compensation unit of the video encoder 20 or the video decoder 30. For a video data stream having multiple video frames, the video encoder or the video decoder may perform process 1200 to obtain predicted values ​​of pixel values ​​of a current image block of a current video frame, including the following steps:

[0192] The method shown in Fig. 12 includes steps 1201 to 1204. For steps 1201, 1202, and 1204, please refer to the descriptions of steps 301, 302, and 304 in Fig. 3. The details will not be described again here.

[0193] The difference between this embodiment of the present application and the embodiment shown in FIG. 3 is as follows: in step 1203, from the positions of M pairs of reference blocks based on a matching cost criterion, the positions of a pair of reference blocks are determined as the position of a target forward reference block of a current image block and the position of a target backward reference block of the current image block, the positions of each pair of reference blocks include the position of the forward reference block and the position of the backward reference block, for each pair of reference block positions, the first position offset and the second position offset have a proportional relationship based on the time-domain distance, the first position offset represents the offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents the offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N.

[0194] 13, the offset of the position of the candidate forward reference block 1304 in the forward reference image Ref0 relative to the position of the initial forward reference block 1302 (i.e., the forward search base point) is MVD0(delta0x, delta0y). The offset of the position of the candidate backward reference block 1305 in the backward reference image Ref1 relative to the position of the initial backward reference block 1303 (i.e., the backward search base point) is MVD1(delta1x, delta1y).

[0195] In the search process, the position offset of the two matching blocks must satisfy the condition of a mirror image relationship, and the time domain interval must be considered in the mirror image relationship. In this specification, TC, T0, and T1 represent the time point of the current frame, the time point of the forward reference image, and the time point of the backward reference image, respectively. TD0 and TD1 represent the time interval between the two time points.

[0196] TD0=TC-T0, and TD1=TC-T1.

[0197] In a particular encoding process, TD0 and TD1 may be calculated by using a picture order count (POC), for example: TD0 = POCc-POC0 and TD1 = POCc-POC1.

[0198] In this specification, POCc, POC0, and POC1 represent the POC of the current image, the POC of the forward reference image, and the POC of the backward reference image, respectively. TD0 represents the picture order count (POC) distance between the current image and the forward reference image, and TD1 represents the POC distance between the current image and the backward reference image.

[0199] delta0=(delta0x, delta0y), and delta1=(delta1x, delta1y).

[0200] The mirror image relationship considering the time domain interval is described as follows: delta0x=(TD0 / TD1)*delta1x, and delta0y=(TD0 / TD1)*delta1y, or delta0x / delta1x=(TD0 / TD1), and delta0y / delta1y=(TD0 / TD1).

[0201] It can be seen that in this embodiment of the present application, the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. There is a proportional relationship based on the time domain distance between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, a pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward prediction block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward prediction block) of the current image block, thereby obtaining predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the target forward reference block and the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating template matching blocks and the process of performing forward search matching and backward search matching by using template matching blocks, simplifying the image prediction process. This improves image prediction accuracy and reduces image prediction complexity.

[0202] In the above embodiment, the search process is performed once. Furthermore, the search can be performed multiple times by using an iterative method. In particular, after the forward reference block and the backward reference block are obtained in each search, the search can be performed one or multiple times based on the current refined MV.

[0203] A process of an image prediction method in an embodiment of the present application is described in detail below with reference to Figure 14. Similar to the method shown in Figure 3, the method shown in Figure 14 can also be performed by a video encoding device, a video codec, a video encoding system, or another device with video encoding capabilities. The method shown in Figure 14 can be used in an encoding process or a decoding process. In particular, the method shown in Figure 14 can be used in an inter-frame prediction process during encoding or decoding.

[0204] The method shown in FIG. 14 specifically includes the following steps 1401 to 1404:

[0205] 1401: Obtain the i-th motion information of the current image block.

[0206] An image block herein may be an image block within an image to be processed or a sub-image within an image to be processed. Furthermore, an image block herein may be an image block to be coded in an encoding process or an image block to be decoded in a decoding process.

[0207] If i=1, the i-th motion information is the initial motion information of the current image block.

[0208] If i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block.

[0209] Furthermore, the initial motion information may include indication information of a prediction direction (usually bidirectional prediction), a motion vector pointing to a reference image block (usually a motion vector of an adjacent block), and information of an image in which the reference image block is located (usually understood as reference image information). The motion vector includes a forward motion vector and a backward motion vector, and the reference image information includes reference frame index information of a forward prediction reference image block and a backward prediction reference image block.

[0210] The initial motion information of the image block can be obtained in several ways, for example, the initial motion information of the image block can be obtained in the following ways 1 and 2:

[0211] Method 1: 4 and 5, in the merge mode of inter-frame prediction, a candidate motion information list is constructed based on the motion information of neighboring blocks of a current image block, and one candidate motion information is selected from the candidate motion information list as the initial motion information of the current image block. The candidate motion information list includes a motion vector, reference frame index information, and the like. For example, the motion information of neighboring block A0 (referring to the candidate motion information with index 0 in FIG. 5) is selected as the initial motion information of the current image block. In particular, the forward motion vector of A0 is used as the forward predicted motion vector of the current block, and the backward motion vector of A0 is used as the backward predicted motion vector of the current block.

[0212] Method 2: In the non-merge mode of inter-frame prediction, a motion vector predictor list is constructed based on motion information of neighboring blocks of a current image block, and a motion vector is selected from the motion vector predictor list as the motion vector predictor of the current image block. In this case, the motion vector of the current image block may be the sum of the motion vector values ​​of the neighboring blocks or the difference between the motion vector of the selected neighboring blocks and the motion vector of the current image block. The motion vector difference is the difference between the motion vector obtained by performing motion estimation on the current image block and the motion vector of the selected neighboring block. For example, the motion vectors corresponding to indexes 1 and 2 in the motion vector predictor list are selected as the forward and backward motion vectors of the current image block.

[0213] It should be understood that the above-mentioned Scheme 1 and Scheme 2 are merely two specific schemes for obtaining initial motion information of an image block. In this application, the scheme for obtaining motion information of a prediction block is not limited, and any scheme that can obtain initial motion information of an image block falls within the protection scope of this application.

[0214] 1402: Determine the positions of N forward reference blocks and N backward reference blocks based on the i-th motion information and the position of the current image block, where the N forward reference blocks are located in the forward reference image and the N backward reference blocks are located in the backward reference image, and N is an integer greater than 1.

[0215] In one example, the i-th motion information includes a forward motion vector, a forward reference image index, a backward motion vector, and a backward reference image index.

[0216] Correspondingly, step 1402 is Based on the forward motion vector and the position of the current image block, the position of the (i-1)th target forward reference block is calculated as fDetermine the location of the (i-1)th target forward reference block of the current image block in the forward reference image corresponding to the forward reference image index, using the (i-1)th target forward reference block as the search starting point, and determine the locations of the (N-1) candidate forward reference blocks in the forward reference image; Based on the backward motion vector and the position of the current image block, the position of the (i-1)th target backward reference block is calculated as b as a search starting point to determine the location of the (i-1)th target backward reference block of the current image block in the backward reference image corresponding to the backward reference image index, and to determine the locations of the (N-1)th candidate backward reference blocks in the backward reference image.

[0217] In one example, referring to FIG. 7, the positions of the N forward reference blocks include the position of the i-th target forward reference block (denoted as (0,0)) and the positions of (N-1) candidate forward reference blocks (denoted as (0,-1), (-1,-1), (-1,1), (1,-1), (1,1), and the like), where the offset of the position of each candidate forward reference block relative to the position of the i-th target forward reference block is an integer pixel distance (as shown in FIG. 8) or a fractional pixel distance, where N=9; or the positions of the N backward reference blocks include the position of the i-th target backward reference block and the positions of (N-1) candidate backward reference blocks, where the offset of the position of each candidate backward reference block relative to the position of the i-th target backward reference block is an integer pixel distance or a fractional pixel distance, where N=9.

[0218] 1403: Determine from M pairs of reference block positions based on a matching cost criterion that a pair of reference block positions is the position of a target forward reference block of a current image block and the position of a target backward reference block of the current image block, where each pair of reference block positions includes the position of the forward reference block and the position of the backward reference block, and for each pair of reference block positions, the first position offset and the second position offset are in a mirror image relationship, the first position offset represents the offset of the position of the forward reference block relative to the position of the (i-1)th target forward reference block, and the second position offset represents the offset of the position of the backward reference block relative to the position of the (i-1)th target backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N.

[0219] The first position offset and the second position offset being in a mirror image relationship can be understood as follows: the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude value of the first position offset is the same as the amplitude value of the second position offset.

[0220] 9, the offset of the position of the candidate forward reference block 904 in the forward reference image Ref0 relative to the position of the (i-1)th target forward reference block 902 (i.e., the forward search base point) is MVD0(delta0x, delta0y). The offset of the position of the candidate backward reference block 905 in the backward reference image Ref1 relative to the position of the (i-1)th target backward reference block 903 (i.e., the backward search base point) is MVD1(delta1x, delta1y).

[0221] MVD0=-MVD1, specifically, delta0x=-delta1x, and delta0y=-delta1y.

[0222] In a different example, step 1403 includes: It may include determining, from the positions of M pairs of reference blocks (one forward reference block and one backward reference block), the positions of a pair of reference blocks with the smallest matching error are the positions of the ith target forward reference block of the current image block and the ith target backward reference block of the current image block, or determining, from the positions of M pairs of reference blocks, the positions of a pair of reference blocks with a matching error equal to or less than a matching error threshold are the positions of the ith target forward reference block of the current image block and the ith target backward reference block of the current image block, where M is less than or equal to N. Furthermore, the difference between pixel values ​​of the forward reference block and pixel values ​​of the backward reference block may be measured by using sum of absolute differences (SAD), sum of absolute transformation differences (SATD), sum of absolute squared differences, or the like.

[0223] 1404: Obtain a predicted value of a pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block.

[0224] In one example, in step 404, a weighting process is performed on the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block, thereby obtaining a predicted value of the pixel value of the current image block. Furthermore, in the present application, the predicted value of the pixel value of the current image block can alternatively be obtained by using another method, which is not limited in the present application.

[0225] The motion vector of the image block is updated. For example, the initial motion information is updated to the second motion information, and the second motion information includes a forward motion vector pointing to the position of the first target forward reference block and a backward motion vector pointing to the first target backward reference block. In this way, another image block can be effectively predicted based on the image block during the next image prediction.

[0226] It can be seen that in this embodiment of the present application, the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. There is a mirror image relationship between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward prediction block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block based on the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating a template matching block and the process of performing forward search matching and backward search matching by using the template matching block, and simplifies the image prediction process. This improves image prediction accuracy and reduces image prediction complexity. Furthermore, the accuracy of refining the MV can be further improved by increasing the number of iterations, which can further improve the coding performance.

[0227] A process of an image prediction method in an embodiment of the present application is described in detail below with reference to Figure 15. The method shown in Figure 15 can also be performed by a video encoding device, a video codec, a video encoding system, or another device having video encoding functionality. The method shown in Figure 15 can be used in an encoding process or a decoding process. In particular, the method shown in Figure 15 can be used in an inter-frame prediction process during encoding or decoding.

[0228] The method illustrated in FIG. 15 particularly includes steps 1501 through 1508, which are described in detail below.

[0229] 1501: Get initial motion information for the current image block.

[0230] For example, the initial motion information of the current block is used in the first search. For example, for an image block whose coding mode is merge, the motion information is obtained from the merge candidate list based on the index of the merge mode, and the motion information is the initial motion information of the current block. For example, for an image block whose coding mode is AMVP, the MVP is obtained from the MVP candidate list based on the index of the AMVP mode, and the MV of the current block is obtained by obtaining the sum of the MVP and MVD included in the bitstream. For searches other than the first search, the MV information updated in the previous search is used. The motion information includes reference picture indication information and motion vector information. The forward reference picture and the backward reference picture are determined by using the reference picture indication information. The positions of the forward reference block and the backward reference block are determined by using the motion vector information.

[0231] 1502: A search base point in the forward reference image is determined.

[0232] The search base point in the forward reference image is determined based on the forward MV information and position information of the current block. The specific process is similar to the process in the embodiment of Figure 10 or Figure 11. For example, if the forward MV information is (MV0x, MV0y) and the position information of the current block is (B0x, B0y), the search base point in the forward reference image is (MV0x+B0x, MV0y+B0y).

[0233] 1503: A search base point in the backward reference image is determined.

[0234] The search base point in the backward reference image is determined based on the backward MV information and position information of the current block. The specific process is similar to the process in the embodiment of Figure 10 or Figure 11. For example, if the backward MV information is (MV1x, MV1y) and the position information of the current block is (B0x, B0y), the search base point in the backward reference image is (MV1x+B0x, MV1y+B0y).

[0235] 1504: In the forward reference image and the backward reference image, the positions of a pair of best-matched reference blocks (i.e., one forward reference block and one backward reference block) are determined based on the MVD mirror image constraint, and a refined forward motion vector and a refined backward motion vector of the current image block are obtained.

[0236] The specific search process is similar to the process in the embodiment of Figure 10 or Figure 11. The details will not be described again here.

[0237] 1505: Determine whether the iteration termination condition is met, and if the iteration termination condition is not met, execute steps 1502 and 1503. If the iteration termination condition is met, execute steps 1506 and 1507.

[0238] The design of the termination condition of the iterative search is not limited herein. For example, the traversal may be performed based on a specified L number of iterations, or another iteration termination condition is met. For example, after the result of the current iteration operation is obtained, if MVD0 is close to or equal to 0 and MVD1 is close to or equal to 0, for example, if MVD0=(0,0) and MVD1=(0,0), the iteration operation may be terminated.

[0239] L is a preset value and is an integer greater than 1. L may be a value that is preset before an image is predicted, or the value of L may be set based on the accuracy of image prediction and the complexity of searching for a predicted block, or L may be set based on past experience, or L may be determined based on verification of the results of intermediate search processes.

[0240] For example, in this embodiment, by using integer pixel steps, the search is performed twice in total. During the first search, the position of the initial forward reference block may be used as a search base point, and the positions of (N-1) candidate forward reference blocks are determined in the forward reference image (also referred to as the forward reference area). The position of the initial backward reference block is used as a search base point, and the positions of (N-1) candidate backward reference blocks are determined in the backward reference image (also referred to as the forward reference area). For one or more pairs of reference block positions in the N pairs of reference blocks, the matching cost of two corresponding reference blocks is calculated, for example, the matching cost of the initial forward reference block and the initial backward reference block is calculated, and the matching cost of the candidate forward reference block and the candidate backward reference block that satisfy the MVD mirror image constraint is calculated. In this way, the positions of the first target forward reference block and the first target backward reference block in the first search are obtained, and updated motion information is further obtained. The updated motion information includes a forward motion vector indicating that the position of the current block points to the position of the first target forward reference block, and a backward motion vector indicating that the position of the current image block points to the position of the first target backward reference block. It should be understood that the updated motion information and the initial motion information include the same reference frame index and the like. Next, a second search is performed. The position of the first target forward reference block is used as a search base point to determine the positions of (N-1) candidate forward reference blocks in a forward reference image (also referred to as a forward reference area). The position of the first target backward reference block is used as a search base point to determine the positions of (N-1) candidate backward reference blocks in a backward reference image (also referred to as a forward reference area). For one or more pairs of reference block positions in the N pairs of reference block positions, the matching cost of two corresponding reference blocks is calculated, for example, the matching cost of the first target forward reference block and the first target backward reference block is calculated, and the matching cost of the candidate forward reference block and the candidate backward reference block that satisfy the MVD mirror image constraint is calculated.In this way, the positions of the second target forward reference block and the second target backward reference block in the second search are obtained, and updated motion information is further obtained. The updated motion information includes a forward motion vector indicating that the position of the current image block points to the position of the second target forward reference block, and a backward motion vector indicating that the position of the current image block points to the position of the second target backward reference block. It should be understood that the updated motion information and the initial motion information also include other similar information, such as a reference frame index. When the preset number of iterations L is 2, in the second search process herein, the second target forward reference block and the second target backward reference block are the finally obtained target forward reference block and target backward reference block (also referred to as the optimal forward reference block and the optimal backward reference block).

[0241] 1506 and 1507: Perform a motion compensation process by using the optimal forward motion vector obtained in step 1504 to obtain pixel values ​​of the optimal forward reference block, and perform a motion compensation process by using the optimal backward motion vector obtained in step 1504 to obtain pixel values ​​of the optimal backward reference block.

[0242] 1508: Obtain a predicted value of the pixel value of the current image block based on the pixel value of the optimal forward reference block and the pixel value of the optimal backward reference block obtained in steps 1506 and 1507.

[0243] In step 1504, a search (also referred to as a motion search) within the forward reference image or the backward reference image may be performed by using integer pixel steps, thereby obtaining the position of at least one forward reference block and the position of at least one backward reference block. When the search is performed by using integer pixel steps, the search starting point may be an integer pixel or a fractional pixel, for example, an integer pixel, 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, or 1 / 16 pixel.

[0244] Furthermore, in step 1504, the fractional pixel step can also be directly used to search the position of at least one forward reference block and the position of at least one backward reference block, or both the search by using the integer pixel step and the search by using the fractional pixel step are performed. The search method is not limited in this application.

[0245] In step 1504, when the difference between the pixel value of the forward reference block and the pixel value of the corresponding backward reference block is calculated for each pair of reference block addresses, the difference between the pixel value of each forward reference block and the pixel value of the corresponding backward reference block may be measured by using SAD, SATD, sum of absolute squared differences, or the like. However, the present application is not limited thereto.

[0246] When the predicted values ​​of pixel values ​​of the current image block are determined based on the optimal forward prediction block and the optimal backward prediction block, a weighting process may be performed on the pixel values ​​of the optimal forward reference block and the optimal backward reference block obtained in steps 1506 and 1507, and the pixel values ​​obtained after the weighting process are used as the predicted values ​​of pixel values ​​of the current image block.

[0247] In particular, the predicted values ​​of the pixel values ​​of the current image block may be obtained based on the following equation (8): predSamples'[x][y]=(predSamplesL0'[x][y]+predSamplesL1'[x][y]+1)>>1 (8)

[0248] In the above formula, predSamplesL0'[x][y] is the pixel value of the optimal forward reference block at sample (x, y), predSamplesL1'[x][y] is the pixel value of the optimal backward reference block at sample (x, y), and predSamples'[x][y] is the pixel prediction value of the current image block at sample (x, y).

[0249] 11, in this embodiment of the present application, the pixel values ​​of the current optimal forward reference block and the current optimal backward reference block can be further maintained and updated in the iterative search process.After the search is completed, the predicted value of the pixel value of the current image block is directly calculated by using the pixel values ​​of the current optimal forward and backward reference blocks.In this implementation, steps 1506 and 1507 are optional steps.

[0250] For example, N pairs of reference block locations need to be traversed. Costi is the i-th matching cost, and MinCost indicates the current minimum matching cost. Bfi is the pixel value of the forward reference block, and Bbi is the pixel value of the backward reference block, and the pixel value is obtained at the i-th time. BestBf is the pixel value of the current best forward reference block, and BestBb is the pixel value of the current best backward reference block. CalCost(M, N) represents the matching cost of block M and block N.

[0251] When the search begins (i=1), MinCost = Cost0 = CalCost(Bf0, Bb0), BestBf = Bf0, and BestBb = Bb0.

[0252] When the other pair of reference blocks are traversed thereafter, the update is performed in real time. For example, when the i-th (i > 1) search is performed, if Costi < MinCost, then BestBf = Bfi and BestBb = Bbi, and if not, the update is not performed.

[0253] When the search is finished, BestBf and BestBb are used to obtain the predicted values of the pixel values of the current block.

[0254] In the foregoing embodiment shown in FIG. 12, the search process is performed once. Further, the search can be performed multiple times by using an iterative method. In particular, after the forward reference block and the backward reference block are obtained in each round of the search, the search can be performed one or more times based on the current refined MV.

[0255] A process of an image prediction method 1600 in an embodiment of the present application is described in detail below while referring to FIG. 16. The method shown in FIG. 16 can also be executed by a video encoding device, a video codec, a video encoding system, or another device having a video encoding function. The method shown in FIG. 16 can be used in an encoding process or a decoding process. In particular, the method shown in FIG. 16 can be used in an inter-frame prediction process during encoding or decoding.

[0256] The method 1600 shown in FIG. 16 includes steps 160 | to step 1604. For steps 1601, step 1602, and step 1604, refer to the descriptions of steps 1401, step 1402, and step 1404 in FIG. 14. Details will not be described again here.

[0257] The difference between this embodiment of the present application and the embodiment shown in FIG. 14 is as follows: in step 1603, from the positions of M pairs of reference blocks based on a matching cost criterion, the positions of a pair of reference blocks are determined as the position of a target forward reference block of a current image block and the position of a target backward reference block of the current image block, the positions of each pair of reference blocks include the position of the forward reference block and the position of the backward reference block, for each pair of reference block positions, the first position offset and the second position offset have a proportional relationship based on the time-domain distance, the first position offset represents the offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents the offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N.

[0258] 13, the offset of the position of the candidate forward reference block 1304 in the forward reference image Ref0 relative to the position of the initial forward reference block 1302 (i.e., the forward search base point) is MVD0(delta0x, delta0y). The offset of the position of the candidate backward reference block 1305 in the backward reference image Ref1 relative to the position of the initial backward reference block 1303 (i.e., the backward search base point) is MVD1(delta1x, delta1y).

[0259] In the search process, the position offset of the two matching blocks must satisfy the condition of a mirror image relationship, and the time domain interval must be considered in the mirror image relationship. In this specification, TC, T0, and T1 represent the time point of the current frame, the time point of the forward reference image, and the time point of the backward reference image, respectively. TD0 and TD1 represent the time interval between the two time points.

[0260] TD0=TC-T0, and TD1=TC-T1.

[0261] In a particular encoding process, TD0 and TD1 may be calculated by using a picture order count (POC), for example: TD0 = POCc-POC0 and TD1 = POCc-POC1.

[0262] In this specification, POCc, POC0, and POC1 represent the POC of the current image, the POC of the forward reference image, and the POC of the backward reference image, respectively. TD0 represents the picture order count (POC) distance between the current image and the forward reference image, and TD1 represents the POC distance between the current image and the backward reference image.

[0263] delta0=(delta0x, delta0y), and delta1=(delta1x, delta1y).

[0264] The mirror image relationship considering the time domain interval is described as follows: delta0x=(TD0 / TD1)*delta1x, and delta0y=(TD0 / TD1)*delta1y, or delta0x / delta1x=(TD0 / TD1), and delta0y / delta1y=(TD0 / TD1).

[0265] In a different example, step 1603 includes: It may include determining, from the positions of M pairs of reference blocks (one forward reference block and one backward reference block), the positions of a pair of reference blocks with the smallest matching error are the positions of the ith target forward reference block of the current image block and the ith target backward reference block of the current image block, or determining, from the positions of M pairs of reference blocks, the positions of a pair of reference blocks with a matching error equal to or less than a matching error threshold are the positions of the ith target forward reference block of the current image block and the ith target backward reference block of the current image block, where M is less than or equal to N. Furthermore, the difference between pixel values ​​of the forward reference block and pixel values ​​of the backward reference block may be measured by using sum of absolute differences (SAD), sum of absolute transformation differences (SATD), sum of absolute squared differences, or the like.

[0266] It can be seen that in this embodiment of the present application, the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a proportional relationship based on time domain distance between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward predicted block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward predicted block) of the current image block, thereby obtaining predicted values ​​of pixel values ​​of the current image block according to the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating template matching blocks and the process of performing forward search matching and backward search matching by using template matching blocks, and simplifies the image prediction process. This improves the image prediction accuracy and reduces the image prediction complexity. Furthermore, the accuracy of refining the MV can be further improved by increasing the number of iterations, which can further improve the coding performance.

[0267] A process of an image prediction method in an embodiment of the present application is described in detail below with reference to Figure 17. The method shown in Figure 17 can also be performed by a video encoding device, a video codec, a video encoding system, or another device having video encoding functionality. The method shown in Figure 17 can be used in an encoding process or a decoding process. In particular, the method shown in Figure 17 can be used in an inter-frame prediction process during encoding or decoding.

[0268] The method shown in Fig. 17 includes steps 1701 to 1708. For steps 1701 to 1703 and steps 1705 to 1708, please refer to the descriptions of steps 1501 to 1503 and steps 1505 to 1508 in Fig. 15. The details will not be described again here.

[0269] The differences between this embodiment of the present application and the embodiment shown in FIG. 15 are as follows.

[0270] 1704: Based on the MVD mirror image constraint that considers the time domain distance, determine the location of a pair of best-matched reference blocks (i.e., one forward reference block and one backward reference block), and obtain a refined forward motion vector and a refined backward motion vector of the current image block.

[0271] The mirror image constraint in which MVD is based on the time-domain distance may be described herein as follows: The position offset MVD0(delta0x, delta0y) of the block position in the forward reference image relative to the forward search base point and the position offset MVD1(delta1x, delta1y) of the block position in the backward reference image relative to the backward search base point satisfy the following relationship:

[0272] The position offset of the two matching blocks satisfies the mirror image relationship condition based on the time domain distance. In this specification, TC, T0, and T1 represent the time point of the current frame, the time point of the forward reference image, and the time point of the backward reference image, respectively. TD0 and TD1 indicate the time interval between the two time points.

[0273] TD0=TC-T0, and TD1=TC-T1.

[0274] In a particular encoding process, TD0 and TD1 may be calculated by using a picture order count (POC), for example: TD0 = POCc-POC0 and TD1 = POCc-POC1.

[0275] In this specification, POCc, POC0, and POC1 represent the POC of the current image, the POC of the forward reference image, and the POC of the backward reference image, respectively. TD0 represents the picture order count (POC) distance between the current image and the forward reference image, and TD1 represents the POC distance between the current image and the backward reference image.

[0276] delta0=(delta0x, delta0y), and delta1=(delta1x, delta1y).

[0277] The mirror image relationship considering the time domain distance (also called time domain interval) can be written as follows: delta0x=(TD0 / TD1)*delta1x, and delta0y=(TD0 / TD1)*delta1y, or delta0x / delta1x=(TD0 / TD1), and delta0y / delta1y=(TD0 / TD1).

[0278] The specific search process is similar to the process in the embodiment of Figure 10 or Figure 11. The details will not be described again here.

[0279] It should be understood that in this embodiment of the present application, the time domain interval is either considered in a mirror image relationship or not considered. In actual use, whether the time domain interval is considered in a mirror image relationship when motion vector refinement is performed on the current frame or the current block can be adaptively selected.

[0280] For example, indication information may be added to sequence level header information (SPS), picture level header information (PPS), slice header, or block bitstream information that indicates whether a time interval is taken into account in the mirror relationship used for the current sequence, current picture, current slice, or current block.

[0281] Alternatively, based on the POC of the forward reference picture and the POC of the backward reference picture, the current block adaptively determines whether the time interval is considered in the mirror image relationship used for the current block.

[0282] For example, if |POCc-POC0|-|POCc-POC1|>T, the interval needs to be considered for the mirror relationship to be used; otherwise, the time interval is not considered for the mirror relationship to be used. T is a preset threshold value in this specification. For example, T=2 or T=3. The specific value of T is not limited in this specification.

[0283] In another example, it is assumed that the ratio of the larger of |POCc-POC0| and |POCc-POC1| to the smaller of |POCc-POC0| and |POCc-POC1| is greater than a threshold R, i.e., (Max(|POCc-POC0|,|POCc-POC1|) / Min(|POCc-POC0|,|POCc-POC1|))>R is.

[0284] Max(A,B) indicates the larger of A and B, and Min(A,B) indicates the smaller of A and B.

[0285] In this case, the interval needs to be considered for the mirror image relationship to be used. If the ratio of the larger value of |POCc-POC0| and |POCc-POC1| to the smaller value of |POCc-POC0| and |POCc-POC1| is equal to or less than a threshold R, the time interval is not considered for the mirror image relationship to be used. R is a preset threshold in this specification. For example, R=2 or R=3. The specific value of R is not limited in this specification.

[0286] It should be understood that the image prediction method in this embodiment of the present application may be particularly performed by a motion compensation module in an encoder (e.g., encoder 20) or a decoder (e.g., decoder 30). Furthermore, the image prediction method in this embodiment of the present application may be performed in any electronic device or apparatus that needs to encode and / or decode video images.

[0287] Next, the image prediction device according to the embodiment of the present application will be described in detail with reference to FIGS.

[0288] 18 is a schematic block diagram of an image prediction device according to an embodiment of the present application. It should be noted that the prediction device 1800 is applicable to both inter-frame prediction for decoding video images and inter-frame prediction for encoding video images. It should be understood that the prediction device 1800 herein may correspond to the motion compensation unit 44 in FIG. 2A or the motion compensation unit 82 in FIG. 2B. The prediction device 1800 a first obtaining unit 1801 configured to obtain initial motion information of a current image block; a first searching unit 1802 configured to: determine positions of N forward reference blocks and N backward reference blocks based on initial motion information and a position of a current image block, where the N forward reference blocks are located in a forward reference image and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; determine from M pairs of reference block positions based on a matching cost criterion that a pair of reference block positions is a position of a target forward reference block of the current image block and a position of a target backward reference block of the current image block, where each pair of reference block positions includes a position of a forward reference block and a position of a backward reference block, where for each pair of reference block positions, the first position offset and the second position offset are in a mirror image relationship, where the first position offset represents an offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; and a first prediction unit 1803 configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block.

[0289] The first position offset and the second position offset being mirror images may be understood as the first position offset value being the same as the second position cf, e.g., the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude value of the first position offset is the same as the amplitude value of the second position offset.

[0290] Preferably, in the device 1800 in this embodiment of the present application, the first prediction unit 1803 is further configured to obtain updated motion information of the current image block, where the updated motion information includes an updated forward motion vector and an updated backward motion vector, where the updated forward motion vector points to the position of the target forward reference block, and the updated backward motion vector points to the position of the target backward reference block.

[0291] It can be seen that the motion vector of the image block has been updated. In this way, another image block can be effectively predicted based on the image block during the next image prediction.

[0292] In the device 1800 in this embodiment of the present application, the positions of the N forward reference blocks include the position of one initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; or The positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0293] In the device 1800 in this embodiment of the present application, the initial motion information includes a first motion vector and a first reference image index in the forward prediction direction, and a second motion vector and a second reference image index in the backward prediction direction.

[0294] In an aspect of determining the positions of the N forward reference blocks and the N backward reference blocks based on the initial motion information and the position of the current image block, the first search unit: According to the first motion vector and the position of the current image block, determine a position of an initial forward reference block of the current image block in the forward reference image corresponding to a first reference image index, and use the position of the initial forward reference block as a first search starting point to determine positions of (N-1) candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks; It is particularly configured to determine, based on the second motion vector and the position of the current image block, the position of an initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index, and use the position of the initial backward reference block as a second search starting point to determine the positions of (N-1) candidate backward reference blocks in the backward reference image, where the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0295] In the device 1800 in this embodiment of the present application, in an aspect of determining, from the M pairs of reference block positions based on a matching cost criterion, the pair of reference block positions are the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, the first searching unit 1802: From the M pairs of reference block positions, determine that the pair of reference block positions with the smallest matching error is the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or It is particularly configured to determine, from the M pairs of reference block positions, that the pair of reference block positions whose matching error is equal to or less than a matching error threshold are the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, where M is equal to or less than N.

[0296] 3, 10, and 11, and it should be understood that the apparatus 1800 may be, in particular, a video encoding apparatus, a video decoding apparatus, a video encoding system, or another device having video encoding functionality. The apparatus 1800 may be configured not only to perform image prediction in the encoding process, but also to perform image prediction in the decoding process.

[0297] For more details, please refer to the description of the image prediction method herein, and for the sake of brevity, the details will not be described again here.

[0298] The prediction device in this embodiment of the present application can find that the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a mirror image relationship between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward prediction block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block according to the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating a template matching block and the process of performing forward search matching and backward search matching by using the template matching block, thereby simplifying the image prediction process. This improves image prediction accuracy and reduces image prediction complexity.

[0299] 19 is a schematic block diagram of another image prediction device according to an embodiment of the present application. It should be noted that the prediction device 1900 is applicable to both inter-frame prediction for decoding video images and inter-frame prediction for encoding video images. It should be understood that the prediction device 1900 herein may correspond to the motion compensation unit 44 in FIG. 2A or the motion compensation unit 82 in FIG. 2B. The prediction device 1900 a second obtaining unit 1901 configured to obtain initial motion information of a current image block; a second search unit 1902 configured to: determine positions of N forward reference blocks and N backward reference blocks according to the initial motion information and the position of the current image block, where the N forward reference blocks are located in the forward reference image and the N backward reference blocks are located in the backward reference image, where N is an integer greater than 1; determine from M pairs of reference block positions according to a matching cost criterion that a pair of reference block positions is a position of a target forward reference block of the current image block and a position of a target backward reference block of the current image block, where each pair of reference block positions includes a position of the forward reference block and a position of the backward reference block, where for each pair of reference block positions, a first position offset and a second position offset have a proportional relationship based on a time-domain distance, where the first position offset represents an offset of the position of the forward reference block relative to the position of the initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; and a second prediction unit 1903 configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block.

[0300] For each pair of reference blocks, the first position offset and the second position offset are in a proportional relationship based on the time domain distance. It can be understood that for each pair of reference blocks, the proportional relationship between the first position offset and the second position offset is determined based on the proportional relationship between the first time-domain distance and the second time-domain distance, where the first time-domain distance represents the time-domain distance between the current image to which the current image block belongs and the forward reference image, and the second time-domain distance represents the time-domain distance between the current image and the backward reference image.

[0301] In one implementation, the first position offset and the second position offset are proportionally related based on the time domain distance by: if the first time-domain distance is the same as the second time-domain distance, then the direction of the first position offset is opposite to the direction of the second position offset and the amplitude value of the first position offset is the same as the amplitude value of the second position offset; or If the first time domain distance is different from the second time domain distance, a direction of the first position offset is opposite to a direction of the second position offset, and a proportional relationship between the amplitude value of the first position offset and the amplitude value of the second position offset is based on a proportional relationship between the first time domain distance and the second time domain distance.

[0302] The first time-domain distance represents the time-domain distance between the current image to which the current image block belongs and the forward reference image, and the second time-domain distance represents the time-domain distance between the current image and the backward reference image.

[0303] Optimally, in the device in this embodiment, the second prediction unit 1903 is further configured to obtain updated motion information of the current image block, where the updated motion information includes an updated forward motion vector and an updated backward motion vector, where the updated forward motion vector points to the position of the target forward reference block and the updated backward motion vector points to the position of the target backward reference block.

[0304] It can be seen that refined motion information of the current image block can be obtained in this embodiment of the present application, which improves the accuracy of the motion information of the current image block and also facilitates the prediction of another image block, for example, improves the prediction accuracy of the motion information of another image block.

[0305] In one implementation, the positions of the N forward reference blocks include a position of one initial forward reference block and positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; or The positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0306] In one implementation, the initial motion information includes forward predicted motion information and backward predicted motion information; In an aspect of determining the positions of the N forward reference blocks and the N backward reference blocks based on the initial motion information and the position of the current image block, the second search unit 1902: Determine positions of N forward reference blocks in a forward reference image according to the forward prediction motion information and the position of the current image block, where the positions of the N forward reference blocks include a position of an initial forward reference block and positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; The method is particularly configured to determine the positions of N backward reference blocks in a backward reference image based on the backward prediction motion information and the position of the current image block, wherein the positions of the N backward reference blocks include the position of an initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0307] In another implementation, the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction; In an aspect of determining the positions of the N forward reference blocks and the N backward reference blocks based on the initial motion information and the position of the current image block, the second search unit: According to the first motion vector and the position of the current image block, determine a position of an initial forward reference block of the current image block in the forward reference image corresponding to a first reference image index, and use the position of the initial forward reference block as a first search starting point to determine positions of (N-1) candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks; It is particularly configured to determine, based on the second motion vector and the position of the current image block, the position of an initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index, and use the position of the initial backward reference block as a second search starting point to determine the positions of (N-1) candidate backward reference blocks in the backward reference image, where the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0308] In one implementation, in an aspect of determining, from the M pairs of reference block locations based on a matching cost criterion, that the pair of reference block locations is the location of the target forward reference block of the current image block and the location of the target backward reference block of the current image block, the second searching unit 1902: From the M pairs of reference block positions, determine that the pair of reference block positions with the smallest matching error is the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or It is particularly configured to determine, from the M pairs of reference block positions, that the pair of reference block positions whose matching error is equal to or less than a matching error threshold are the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, where M is equal to or less than N.

[0309] In one example, the matching cost criterion is a matching cost minimization criterion. For example, for M pairs of reference block positions, the difference between the pixel value of the forward reference block and the pixel value of the backward reference block is calculated for each pair of reference blocks, and from the M pairs of reference block positions, the positions of the pair of reference blocks whose pixel values ​​have the smallest difference pixel value are determined as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block.

[0310] In another example, the matching cost criterion is a matching cost minimization and early termination criterion. For example, for the position of an n-th pair of reference blocks (one forward reference block and one backward reference block), a difference between a pixel value of the forward reference block and a pixel value of the backward reference block is calculated, where n is an integer greater than or equal to 1 and less than or equal to N, and when the pixel value difference is less than or equal to a matching error threshold, the position of the n-th pair of reference blocks (one forward reference block and one backward reference block) is determined as the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block.

[0311] In one implementation, the second obtaining unit 1901 is configured to obtain initial motion information from the candidate motion information list of the current image block, or obtain initial motion information based on indication information, and the indication information is used to indicate the initial motion information of the current image block. It should be understood that the initial motion information is related to the refined motion information.

[0312] 12, and it should be understood that the apparatus 1900 may be a video encoding apparatus, a video decoding apparatus, a video encoding system, or another device having video encoding functionality. The apparatus 1900 may be configured to perform image prediction in the encoding process, as well as to perform image prediction in the decoding process.

[0313] For more details, please refer to the description of the image prediction method herein, and for the sake of brevity, the details will not be described again here.

[0314] The prediction device in this embodiment of the present application can find that the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a proportional relationship based on the time domain distance between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward predicted block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward predicted block) of the current image block, thereby obtaining predicted values ​​of pixel values ​​of the current image block according to the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating template matching blocks and the process of performing forward search matching and backward search matching by using template matching blocks, thereby simplifying the image prediction process. This improves image prediction accuracy and reduces image prediction complexity.

[0315] 20 is a schematic block diagram of another image prediction device according to an embodiment of the present application. It should be noted that the prediction device 2000 is applicable to both inter-frame prediction for decoding video images and inter-frame prediction for encoding video images. It should be understood that the prediction device 2000 herein may correspond to the motion compensation unit 44 in FIG. 2A or the motion compensation unit 82 in FIG. 2B. The prediction device 2000 a third acquisition unit 2001 configured to acquire the i-th motion information of the current image block; a third search unit 2002, for determining N forward reference block positions and N backward reference block positions according to the i-th motion information and the position of a current image block, where the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; determining from the M pairs of reference block positions according to a matching cost criterion that the pair of reference block positions is the i-th target forward reference block position of the current image block and the i-th target backward reference block position of the current image block; a third searching unit 2002 configured to: the paired reference block positions include a forward reference block position and a backward reference block position, and for each paired reference block position, the first position offset and the second position offset are in a mirror image relationship, the first position offset represents an offset of the position of the forward reference block relative to the position of the (i-1)th target forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the (i-1)th target backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; and a third prediction unit 2003 configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the jth target forward reference block and pixel values ​​of the jth target backward reference block, where j is greater than or equal to i and both i and j are integers greater than or equal to 1.

[0316] It should be noted that if i=1, the i-th motion information is the initial motion information of the current image block, and correspondingly, the positions of the N forward reference blocks include the position of one initial forward reference block and the positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance, or the positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

[0317] If i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block, and correspondingly, the positions of the N forward reference blocks include the position of one (i-1)th target forward reference block and the positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block from the position of the (i-1)th target forward reference block is an integer pixel distance or a fractional pixel distance, or the positions of the N backward reference blocks include the position of one (i-1)th target backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block from the position of the (i-1)th target backward reference block is an integer pixel distance or a fractional pixel distance.

[0318] In this embodiment of the present application, the third prediction unit 2003 is specifically configured to: obtain a predicted value of the pixel value of the image block according to the pixel value of the jth target forward reference block and the pixel value of the jth target backward reference block when the iteration termination condition is met, where j is greater than or equal to i, and i and j are both integers greater than or equal to 1. For the description of the iteration termination condition, please refer to other embodiments. The details will not be described again here.

[0319] In the device of this embodiment of the present application, the first position offset and the second position offset being mirror images can be understood as the first position offset value being the same as the second position offset value, e.g., the direction of the first position offset is opposite to the direction of the second position offset, and the amplitude value of the first position offset is the same as the amplitude value of the second position offset.

[0320] In one implementation, the i-th motion information includes a forward motion vector, a forward reference image index, a backward motion vector, and a backward reference image index; In an aspect of determining the positions of the N forward reference blocks and the N backward reference blocks based on the i-th motion information and the position of the current image block, the third search unit 2002: According to the forward motion vector and the position of the current image block, determine the position of the (i-1)th target forward reference block of the current image block in the forward reference image corresponding to the forward reference image index, and set the position of the (i-1)th target forward reference block as the i-th f as a search starting point to determine the positions of (N-1) candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the (i-1)th target forward reference block and the positions of the (N-1) candidate forward reference blocks; According to the backward motion vector and the position of the current image block, determine the position of the (i-1)th target backward reference block of the current image block in the backward reference image corresponding to the backward reference image index, and set the position of the (i-1)th target backward reference block as the i-th b is used as a search starting point to determine the positions of (N-1) candidate backward reference blocks in the backward reference image, and the positions of the N backward reference blocks include the position of the (i-1)th target backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0321] In one implementation, in an aspect in which the pair of reference block positions is determined to be the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block from the M pairs of reference block positions based on a matching cost criterion, the third searching unit 2002: From the M pairs of reference block positions, determine that the pair of reference block positions with the smallest matching error is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block; or It is particularly configured to determine, from the M pairs of reference block positions, that the pair of reference block positions whose matching error is equal to or less than the matching error threshold are the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, where M is equal to or less than N.

[0322] 14 and 15, and it should be understood that the apparatus 2000 may be, in particular, a video encoding apparatus, a video decoding apparatus, a video encoding system, or another device having video encoding functionality. The apparatus 2000 may be configured not only to perform image prediction in the encoding process, but also to perform image prediction in the decoding process.

[0323] For more details, please refer to the description of the image prediction method herein, and for the sake of brevity, the details will not be described again here.

[0324] The prediction device in this embodiment of the present application can find that the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a mirror image relationship between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward prediction block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward prediction block) of the current image block, thereby obtaining a predicted value of the pixel value of the current image block according to the pixel value of the target forward reference block and the pixel value of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating a template matching block and the process of performing forward search matching and backward search matching by using the template matching block, and simplifies the image prediction process. This improves the image prediction accuracy and reduces the image prediction complexity. Furthermore, the accuracy of refining the MV can be further improved by increasing the number of iterations, which can further improve the coding performance.

[0325] 21 is a schematic block diagram of another image prediction device according to an embodiment of the present application. It should be noted that the prediction device 2100 is applicable to both inter-frame prediction for decoding video images and inter-frame prediction for encoding video images. It should be understood that the prediction device 2100 herein may correspond to the motion compensation unit 44 in FIG. 2A or the motion compensation unit 82 in FIG. 2B. The prediction device 2100 a fourth obtaining unit 2101 configured to obtain the i-th motion information of the current image block; a fourth searching unit 2102, for determining N positions of forward reference blocks and N positions of backward reference blocks according to the i-th motion information and the position of a current image block, where the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; determining from the M pairs of reference block positions according to a matching cost criterion that the positions of a pair of reference blocks are the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, and the positions of each pair of reference blocks are , a position of a forward reference block and a position of a backward reference block, and for each pair of reference block positions, the first position offset and the second position offset are in a proportional relationship based on a time-domain distance, the first position offset represents an offset of the position of the forward reference block relative to the position of the (i-1)th target forward reference block in the forward reference image, and the second position offset represents an offset of the position of the backward reference block relative to the position of the (i-1)th target backward reference block in the backward reference image, where M is an integer greater than or equal to 1 and M is less than or equal to N; and a fourth prediction unit 2103 configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the jth target forward reference block and pixel values ​​of the jth target backward reference block, where j is greater than or equal to i and both i and j are integers greater than or equal to 1.

[0326] In the iterative search process, if i=1, the ith motion information is the initial motion information of the current image block.

[0327] If i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block.

[0328] In one implementation, the fourth prediction unit 2103 is specifically configured to obtain a predicted value of a pixel value of an image block based on a pixel value of the jth target forward reference block and a pixel value of the jth target backward reference block when an iteration termination condition is met, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

[0329] In the device of this embodiment, the first position offset and the second position offset are in a proportional relationship based on the time domain distance, if the first time-domain distance is the same as the second time-domain distance, then the direction of the first position offset is opposite to the direction of the second position offset and the amplitude value of the first position offset is the same as the amplitude value of the second position offset; or If the first time-domain distance is different from the second time-domain distance, it can be understood that the direction of the first position offset is opposite to the direction of the second position offset, and the proportional relationship between the amplitude value of the first position offset and the amplitude value of the second position offset is based on the proportional relationship between the first time-domain distance and the second time-domain distance.

[0330] The first time-domain distance represents the time-domain distance between the current image to which the current image block belongs and the forward reference image, and the second time-domain distance represents the time-domain distance between the current image and the backward reference image.

[0331] In one implementation, the i-th motion information includes a forward motion vector, a forward reference image index, a backward motion vector, and a backward reference image index, and correspondingly, in an aspect of determining the positions of the N forward reference blocks and the N backward reference blocks based on the i-th motion information and the position of the current image block, the fourth search unit 2102: According to the forward motion vector and the position of the current image block, determine the position of the (i-1)th target forward reference block of the current image block in the forward reference image corresponding to the forward reference image index, and set the position of the (i-1)th target forward reference block as the i-th fas a search starting point to determine the positions of (N-1) candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the (i-1)th target forward reference block and the positions of the (N-1) candidate forward reference blocks; According to the backward motion vector and the position of the current image block, determine the position of the (i-1)th target backward reference block of the current image block in the backward reference image corresponding to the backward reference image index, and set the position of the (i-1)th target backward reference block as the i-th b is used as a search starting point to determine the positions of (N-1) candidate backward reference blocks in the backward reference image, and the positions of the N backward reference blocks include the position of the (i-1)th target backward reference block and the positions of the (N-1) candidate backward reference blocks.

[0332] In one implementation, in an aspect of determining, from the M pairs of reference block positions based on a matching cost criterion, the pair of reference block positions is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, the fourth searching unit 2102: From the M pairs of reference block positions, determine that the pair of reference block positions with the smallest matching error is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block; or It is particularly configured to determine, from the M pairs of reference block positions, that the pair of reference block positions whose matching error is equal to or less than the matching error threshold are the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, where M is equal to or less than N.

[0333] It should be understood that the apparatus 2100 may perform the method shown in Figure 16 or 17, and the apparatus 2100 may be a video encoding apparatus, a video decoding apparatus, a video encoding system, or another device having video encoding functionality. The apparatus 2100 may be configured to perform image prediction in the encoding process as well as to perform image prediction in the decoding process.

[0334] For more details, please refer to the description of the image prediction method herein, and for the sake of brevity, the details will not be described again here.

[0335] The prediction device in this embodiment of the present application can find that the positions of N forward reference blocks in the forward reference image and the positions of N backward reference blocks in the backward reference image form N pairs of reference block positions. For the positions of each pair of reference blocks in the N pairs of reference block positions, there is a proportional relationship based on the time domain distance between the first position offset of the forward reference block relative to the initial forward reference block and the second position offset of the backward reference block relative to the initial backward reference block. Based on this, the positions of a pair of reference blocks (for example, the pair of reference blocks with the smallest matching cost) are determined from the positions of the N pairs of reference blocks as the position of the target forward reference block (i.e., the optimal forward reference block / forward predicted block) of the current image block and the position of the target backward reference block (i.e., the optimal backward reference block / backward predicted block) of the current image block, thereby obtaining predicted values ​​of pixel values ​​of the current image block according to the pixel values ​​of the target forward reference block and the pixel values ​​of the target backward reference block. Compared with the prior art, the method in this embodiment of the present application avoids the process of pre-calculating template matching blocks and the process of performing forward search matching and backward search matching by using template matching blocks, thereby simplifying the image prediction process. This improves the image prediction accuracy and reduces the image prediction complexity. Furthermore, the accuracy of refining the MV can be further improved by increasing the number of iterations, which can further improve the coding performance.

[0336] 22 is a schematic block diagram of an implementation of a video encoding device or a video decoding device (decoding device 2200 for short) according to an embodiment of the present application. The decoding device 2200 may include a processor 2210, a memory 2230, and a bus system 2250. The processor and the memory are connected by using the bus system. The memory is configured to store instructions. The processor is configured to execute the instructions stored in the memory. The memory of the encoding device stores program code. The processor may invoke the program code stored in the memory to perform the video encoding or decoding methods described in the present application, in particular, the video encoding or decoding methods of various inter-frame prediction modes or intra-frame prediction modes, and the motion information prediction methods of various inter-frame prediction modes or intra-frame prediction modes. The details will not be described again in this specification to avoid repetition.

[0337] In this embodiment of the application, the processor 2210 may be a Central Processing Unit (abbreviated "CPU"), or the processor 2210 may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor or any conventional processor or the like.

[0338] The memory 2230 may include a read-only memory (ROM) device or a random access memory (RAM). Any other suitable type of storage device may also be used as the memory 2230. The memory 2230 may include code and data 2231 accessed by the processor 2210 by using the bus 2250. The memory 2230 may further include an operating system 2233 and application programs 2235. The application programs 2235 include at least one program that enables the processor 2210 to perform the video encoding or decoding methods described herein (in particular, the image prediction methods described herein). For example, the application programs 2235 may include applications 1 through N, and further include a video encoding or decoding application (video decoding application for short) that performs the video encoding or decoding methods described herein.

[0339] In addition to a data bus, the bus system 2250 may further comprise a power bus, a control bus, a status signal bus, and the like. However, for clarity of explanation, the various types of buses in the figures are marked as bus system 2250.

[0340] Optionally, decoding device 2200 may further include one or more output devices, such as a display 2270. In one example, display 2270 may be a touch display or touch screen that combines a display with a touch unit that operatively senses touch input. Display 2270 may be connected to processor 2210 using bus 2250.

[0341] It should be noted that the same steps or the same terminology descriptions and limitations may also be applicable to different embodiments, and for the sake of brevity, repeated descriptions will be omitted where appropriate in this specification.

[0342] Those skilled in the art will understand that the functions described with reference to the various illustrative logical blocks, modules, and algorithm steps disclosed and described herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented by software, the functions described with reference to the illustrative logical blocks, modules, and steps can be stored on or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media, such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another (e.g., according to a communication protocol). In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media or (2) communication media, such as a signal or carrier wave. Data storage media may be any medium available for retrieving instructions, code, and / or data structures that can be accessed by one or more computers or one or more processors to implement the techniques described in this application. A computer program product may include computer-readable media.

[0343] For example, such computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other compact disk storage, magnetic disk storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Additionally, any connection may be referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or another remote source over coaxial cable, fiber optic cable, twisted pair wire, digital subscriber line (DSL), or wireless technologies such as infrared, radio waves, and microwaves, the coaxial cable, fiber optic cable, twisted pair wire, DSL, or wireless technologies such as infrared, radio waves, and microwaves are included within the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, and in fact refer to non-transitory, tangible storage media. As used herein, "Disk" and "Disc" include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the foregoing should also be included within the scope of computer-readable media.

[0344] The corresponding functions may be performed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated circuits or discrete logic circuits. Accordingly, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functions described with reference to the exemplary logical blocks, modules, and steps described herein may be provided in dedicated hardware and / or software modules configured for encoding, or may be incorporated into a combined codec. Furthermore, the techniques may be implemented entirely in one or more circuits or logic elements. In one example, the various exemplary logical blocks, units, and modules in video encoder 20 and video decoder 30 can be understood as corresponding circuit devices or logic elements.

[0345] The technology in this application may be implemented in a variety of apparatuses or devices, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this application to emphasize functional aspects of apparatuses configured to perform the disclosed technology, but are not necessarily implemented by different hardware units. Indeed, as described above, the various units may be integrated into a codec hardware unit or provided by interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

[0346] The above description is merely an example of a specific implementation of the present application and is not intended to limit the protection scope of the present application. Any modifications or alternatives that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present application shall fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims. [Explanation of symbols]

[0347] 12 Source Device 14 Destination device 16 Links 18 Video Sources 20 Video Encoder 22 Output Interface 28 Input Interface 30 Video Decoder 32 Display device 41 Prediction Module 42 Motion Estimation Unit 44 Motion Compensation Unit 46 Intra-frame prediction units 50 Adder 52 Conversion Module 54 Quantization Module 56 Entropy Coding Module 58 Inverse Quantization Module 60 Inverse Conversion Module 62 Adder 64 reference image memory 80 Entropy Decoding Module 81 Prediction Processing Module 82 Motion Compensation Unit 84 intra-frame prediction units 86 Inverse Quantization Module 88 Inverse Transformation Module 90 Reconstruction Module, Adder 92 Reference image memory, decoded image buffer 902 Initial lookahead block 903 Initial Backreference Block 904 candidate forward reference blocks 905 candidate backreference blocks 1302 Initial lookahead block 1303 Initial Backreference Block 1304 candidate forward reference blocks 1305 Candidate Backreference Block 1800 Prediction Device 1801 First Acquisition Unit 1802 First Exploration Unit 1803 First Prediction Unit 1900 Prediction Device 1901 Second Acquisition Unit 1902 Second Exploration Unit 1903 Second Prediction Unit 2000 Prediction Device 2001 Third Acquisition Unit 2002 Third Exploration Unit 2003 Third Prediction Unit 2100 Prediction Device 2101 4th Acquisition Unit 2102 Fourth Exploration Unit 2103 Fourth Prediction Unit 2200 Decryption Device 2210 processor 2230 memory 2231 Code and Data 2233 Operating System 2235 Application Program 2250 Bus 2270 display

Claims

1. 1. A method for predicting an image, comprising: obtaining initial motion information for a current image block; determining positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1; determining, based on a matching cost criterion, from M pairs of reference block positions, a pair of reference block positions as a target forward reference block position of the current image block and a target backward reference block position of the current image block, where each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, a first position offset and a second position offset are in a mirror image relationship, where the first position offset represents an offset of the position of the forward reference block relative to a position of an initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to a position of an initial backward reference block, where M is an integer greater than or equal to 1 and less than or equal to N; and obtaining a predicted value of a pixel value of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block.

2. The first position offset and the second position offset are in a mirror image relationship.

2. The method of claim 1, wherein a direction of the first position offset is opposite to a direction of the second position offset, and an amplitude value of the first position offset is the same as an amplitude value of the second position offset.

3. The method of claim 1 or 2, further comprising the step of obtaining updated motion information for the current image block, the updated motion information including an updated forward motion vector and an updated backward motion vector, the updated forward motion vector pointing to the position of the target forward reference block, and the updated backward motion vector pointing to the position of the target backward reference block.

4. the positions of the N forward reference blocks include a position of one initial forward reference block and positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; or 4. The method of claim 1, wherein the positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

5. the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction; determining the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block; determining the position of the initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index, based on the first motion vector and the position of the current image block, using the position of the initial forward reference block as a first search starting point, and determining the positions of the (N-1) candidate forward reference blocks in the forward reference image, wherein the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks; 5. A method according to claim 1, further comprising: determining, based on the second motion vector and the position of the current image block, the position of the initial backward reference block in the backward reference image corresponding to the second reference image index, using the position of the initial backward reference block as a second search starting point; and determining the positions of the (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

6. The step of determining, from M pairs of reference block locations based on a matching cost criterion, a pair of reference block locations is a target forward reference block location of the current image block and a target backward reference block location of the current image block, includes: determining, from the positions of the M pairs of reference blocks, the pair of positions of reference blocks with the smallest matching error is the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or 6. The method of claim 1, further comprising: determining, from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks whose matching error is less than or equal to a matching error threshold to be the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, wherein M is less than or equal to N.

7. The method is used for encoding the current image block, and the step of obtaining initial motion information for the current image block includes obtaining the initial motion information from a list of candidate motion information for the current image block; or 7. The method according to claim 1, wherein the method is used to decode the current image block, and before the step of obtaining initial motion information of the current image block, the method further comprises a step of obtaining indication information from a bitstream of the current image block, the indication information being used to indicate the initial motion information of the current image block.

8. 1. A method for predicting an image, comprising: obtaining initial motion information for a current image block; determining positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1; determining, based on a matching cost criterion, from M pairs of reference block positions, a pair of reference block positions as a target forward reference block position of the current image block and a target backward reference block position of the current image block, where each pair of reference block positions includes a forward reference block position and a backward reference block position, and for each pair of reference block positions, a first position offset and a second position offset have a proportional relationship based on a time-domain distance, where the first position offset represents an offset of the position of the forward reference block relative to a position of an initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to a position of an initial backward reference block, where M is an integer greater than or equal to 1 and less than or equal to N; and obtaining a predicted value of a pixel value of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block.

9. The fact that the first position offset and the second position offset are in a proportional relationship based on the time domain distance is if the first time domain distance is the same as the second time domain distance, then the direction of the first position offset is opposite to the direction of the second position offset and the amplitude value of the first position offset is the same as the amplitude value of the second position offset; or if a first time domain distance is different from a second time domain distance, a direction of the first position offset is opposite to a direction of the second position offset, and a proportional relationship between an amplitude value of the first position offset and an amplitude value of the second position offset is based on a proportional relationship between the first time domain distance and the second time domain distance; 9. The method of claim 8, wherein the first time domain distance represents a time domain distance between a current image to which the current image block belongs and the forward reference image, and the second time domain distance represents a time domain distance between the current image and the backward reference image.

10. The method comprises: The method of claim 8 or 9, further comprising the step of obtaining updated motion information for the current image block, the updated motion information including an updated forward motion vector and an updated backward motion vector, the updated forward motion vector pointing to the position of the target forward reference block, and the updated backward motion vector pointing to the position of the target backward reference block.

11. the positions of the N forward reference blocks include a position of one initial forward reference block and positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; or 11. A method according to any one of claims 8 to 10, wherein the positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

12. the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction; determining the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block; determining the position of the initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index, based on the first motion vector and the position of the current image block, using the position of the initial forward reference block as a first search starting point, and determining the positions of the (N-1) candidate forward reference blocks in the forward reference image, wherein the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks; 12. A method according to claim 8, further comprising: determining, based on the second motion vector and the position of the current image block, the position of the initial backward reference block as a second search starting point to determine the position of the initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index; and determining the positions of the (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

13. determining, from M pairs of reference block locations based on a matching cost criterion, a pair of reference block locations as a target forward reference block location of the current image block and a target backward reference block location of the current image block; determining, from the positions of the M pairs of reference blocks, the pair of positions of reference blocks with the smallest matching error is the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or 13. The method of claim 8, further comprising: determining, from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks whose matching error is less than or equal to a matching error threshold to be the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, wherein M is less than or equal to N.

14. The method is used for encoding the current image block, and the step of obtaining initial motion information for the current image block includes obtaining the initial motion information from a list of candidate motion information for the current image block; or 14. The method according to claim 8, wherein the method is used to decode the current image block, and before the step of obtaining initial motion information of the current image block, the method further comprises a step of obtaining indication information from a bitstream of the current image block, the indication information being used to indicate the initial motion information of the current image block.

15. 1. A method for predicting an image, comprising: obtaining the i-th motion information of the current image block; determining positions of N forward reference blocks and N backward reference blocks based on the i-th motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1; determining, from M pairs of reference block positions based on a matching cost criterion, a pair of reference block positions that is a position of an i-th target forward reference block of the current image block and a position of an i-th target backward reference block of the current image block, wherein the positions of each pair of reference blocks include a position of a forward reference block and a position of a backward reference block, and for the positions of each pair of reference blocks, a first position offset and a second position offset are in a mirror image relationship, the first position offset represents an offset of the position of the forward reference block relative to the position of the (i-1)th target forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the (i-1)th target backward reference block, where M is an integer greater than or equal to 1 and less than or equal to N; and obtaining a predicted value of a pixel value of the current image block based on pixel values ​​of the j-th target forward reference block and pixel values ​​of the j-th target backward reference block, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

16. If i=1, the i-th motion information is the initial motion information of the current image block; or 16. The method of claim 15, wherein if i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block.

17. The step of obtaining a predicted value of a pixel value of the image block based on a pixel value of a j-th target forward reference block and a pixel value of the j-th target backward reference block, where j is equal to or greater than i, and both i and j are integers equal to or greater than 1, includes:

17. The method of claim 15 or 16, comprising: when an iteration termination condition is met, obtaining the predicted value of the pixel value of the image block based on the pixel value of the jth target forward reference block and the pixel value of the jth target backward reference block, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

18. The first position offset and the second position offset are in a mirror image relationship.

18. The method of claim 15, wherein a direction of the first position offset is opposite to a direction of the second position offset, and an amplitude value of the first position offset is the same as an amplitude value of the second position offset.

19. the i-th motion information includes a forward motion vector, a forward reference image index, a backward motion vector, and a backward reference image index; determining the positions of N forward reference blocks and N backward reference blocks based on the i-th motion information and the position of the current image block, Based on the forward motion vector and the position of the current image block, the position of the (i-1)th target forward reference block is calculated as f determining the position of the (i-1)th target forward reference block of the current image block in the forward reference image corresponding to the forward reference image index using the (i-1)th target forward reference block as a search starting point, and determining the positions of (N-1) candidate forward reference blocks in the forward reference image, wherein the positions of the N forward reference blocks include the position of the (i-1)th target forward reference block and the positions of the (N-1) candidate forward reference blocks; Based on the backward motion vector and the position of the current image block, the position of the (i-1)th target backward reference block is calculated as b 19. A method according to any one of claims 15 to 18, comprising: determining the position of the (i-1)th target backward reference block of the current image block in the backward reference image corresponding to the backward reference image index using the backward reference image index as a search starting point; and determining positions of (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of the (i-1)th target backward reference block and the positions of the (N-1) candidate backward reference blocks.

20. The step of determining, from M pairs of reference block positions based on a matching cost criterion, a pair of reference block positions is a position of the i-th target forward reference block of the current image block and a position of the i-th target backward reference block of the current image block, comprises: determining, from the positions of the M pairs of reference blocks, the pair of positions of reference blocks with the smallest matching error is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block; or 20. A method according to any one of claims 15 to 19, comprising the step of determining, from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks whose matching error is less than or equal to a matching error threshold to be the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, wherein M is less than or equal to N.

21. 1. A method for predicting an image, comprising: obtaining the i-th motion information of the current image block; determining positions of N forward reference blocks and N backward reference blocks based on the i-th motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, and N is an integer greater than 1; determining, from M pairs of reference block positions based on a matching cost criterion, a pair of reference block positions that is a position of an i-th target forward reference block of the current image block and a position of an i-th target backward reference block of the current image block, wherein the positions of each pair of reference blocks include a position of a forward reference block and a position of a backward reference block, and for the positions of each pair of reference blocks, a first position offset and a second position offset have a proportional relationship based on a time-domain distance, the first position offset represents an offset of the position of the forward reference block relative to a position of the (i-1)-th target forward reference block in the forward reference image, and the second position offset represents an offset of the position of the backward reference block relative to a position of the (i-1)-th target backward reference block in the backward reference image, where M is an integer greater than or equal to 1 and M is less than or equal to N; and obtaining a predicted value of a pixel value of the current image block based on pixel values ​​of the j-th target forward reference block and pixel values ​​of the j-th target backward reference block, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

22. If i=1, the i-th motion information is the initial motion information of the current image block; or 22. The method of claim 21, wherein if i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block.

23. The step of obtaining a predicted value of a pixel value of the image block based on a pixel value of the j-th target forward reference block and a pixel value of the j-th target backward reference block, where j is equal to or greater than i, and both i and j are integers equal to or greater than 1, includes:

23. The method of claim 21 or 22, comprising: when an iteration termination condition is met, obtaining the predicted value of the pixel value of the image block based on the pixel value of the jth target forward reference block and the pixel value of the jth target backward reference block, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

24. The fact that the first position offset and the second position offset are in a proportional relationship based on the time domain distance is if a first time-domain distance is the same as a second time-domain distance, then a direction of the first position offset is opposite to a direction of the second position offset and an amplitude value of the first position offset is the same as an amplitude value of the second position offset; or if a first time domain distance is different from a second time domain distance, a direction of the first position offset is opposite to a direction of the second position offset, and a proportional relationship between an amplitude value of the first position offset and an amplitude value of the second position offset is based on a proportional relationship between the first time domain distance and the second time domain distance; 24. The method of claim 21, wherein the first time domain distance represents a time domain distance between a current image to which the current image block belongs and the forward reference image, and the second time domain distance represents a time domain distance between the current image and the backward reference image.

25. An image prediction device, comprising: a first obtaining unit configured to obtain initial motion information of a current image block; a first search unit configured to: determine positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, wherein the N forward reference blocks are located in a forward reference image and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; determine from M pairs of reference block positions based on a matching cost criterion that a pair of reference block positions is a position of a target forward reference block of the current image block and a position of a target backward reference block of the current image block, wherein the positions of each pair of reference blocks include a position of a forward reference block and a position of a backward reference block, and for the positions of each pair of reference blocks, a first position offset and a second position offset are in a mirror image relationship, wherein the first position offset represents an offset of the position of the forward reference block relative to the position of an initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; a first prediction unit configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block.

26. 26. The apparatus of claim 25, wherein the first positional offset and the second positional offset being mirror images includes a direction of the first positional offset being opposite to a direction of the second positional offset and an amplitude value of the first positional offset being the same as an amplitude value of the second positional offset.

27. 27. The apparatus of claim 25 or 26, wherein the first prediction unit is further configured to obtain updated motion information for the current image block, the updated motion information including an updated forward motion vector and an updated backward motion vector, the updated forward motion vector pointing to the position of the target forward reference block, and the updated backward motion vector pointing to the position of the target backward reference block.

28. the positions of the N forward reference blocks include a position of one initial forward reference block and positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; or 28. The apparatus of claim 25, wherein the positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

29. the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction; In an aspect of determining the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, the first search unit: Determine the position of the initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index based on the first motion vector and the position of the current image block, and use the position of the initial forward reference block as a first search starting point to determine the positions of the (N-1) candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks; 29. The device of claim 25, wherein the device is specifically configured to: determine, based on the second motion vector and the position of the current image block, the position of the initial backward reference block of the current image block in the backward reference image corresponding to the second reference image index; use the position of the initial backward reference block as a second search starting point; and determine the positions of the (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

30. In an aspect of determining, from M pairs of reference block locations based on a matching cost criterion, a pair of reference block locations is a target forward reference block location of the current image block and a target backward reference block location of the current image block, the first searching unit: From the positions of the M pairs of reference blocks, determine that the pair of reference block positions with the smallest matching error is the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or 30. The apparatus of claim 25, wherein the apparatus is specifically configured to: determine, from the positions of the M pairs of reference blocks, a pair of positions of reference blocks whose matching error is less than or equal to a matching error threshold to be the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, wherein M is less than or equal to N.

31. An image prediction device, comprising: a second acquisition unit configured to acquire initial motion information of the current image block; a second search unit configured to: determine positions of N forward reference blocks and N backward reference blocks according to the initial motion information and the position of the current image block, the N forward reference blocks being located in a forward reference image, and the N backward reference blocks being located in a backward reference image, where N is an integer greater than 1; determine from M pairs of reference block positions based on a matching cost criterion that a pair of reference block positions is a position of a target forward reference block of the current image block and a position of a target backward reference block of the current image block, where the positions of each pair of reference blocks include a position of a forward reference block and a position of a backward reference block, and for the positions of each pair of reference blocks, a first position offset and a second position offset have a proportional relationship based on a time-domain distance, the first position offset represents an offset of the position of the forward reference block relative to a position of an initial forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to a position of the initial backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; and a second prediction unit configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the target forward reference block and pixel values ​​of the target backward reference block.

32. The fact that the first position offset and the second position offset are in a proportional relationship based on the time domain distance is if a first time-domain distance is the same as a second time-domain distance, then a direction of the first position offset is opposite to a direction of the second position offset and an amplitude value of the first position offset is the same as an amplitude value of the second position offset; or if a first time domain distance is different from a second time domain distance, a direction of the first position offset is opposite to a direction of the second position offset, and a proportional relationship between an amplitude value of the first position offset and an amplitude value of the second position offset is based on a proportional relationship between the first time domain distance and the second time domain distance; 32. The apparatus of claim 31, wherein the first time-domain distance represents a time-domain distance between a current image to which the current image block belongs and the forward reference image, and the second time-domain distance represents a time-domain distance between the current image and the backward reference image.

33. 33. The apparatus of claim 31 or 32, wherein the second prediction unit is further configured to obtain updated motion information for the current image block, the updated motion information including an updated forward motion vector and an updated backward motion vector, the updated forward motion vector pointing to the position of the target forward reference block, and the updated backward motion vector pointing to the position of the target backward reference block.

34. the positions of the N forward reference blocks include a position of one initial forward reference block and positions of (N-1) candidate forward reference blocks, and the offset of the position of each candidate forward reference block relative to the position of the initial forward reference block is an integer pixel distance or a fractional pixel distance; or 34. The apparatus of claim 31, wherein the positions of the N backward reference blocks include the position of one initial backward reference block and the positions of (N-1) candidate backward reference blocks, and the offset of the position of each candidate backward reference block relative to the position of the initial backward reference block is an integer pixel distance or a fractional pixel distance.

35. the initial motion information includes a first motion vector and a first reference image index in a forward prediction direction, and a second motion vector and a second reference image index in a backward prediction direction; In an aspect of determining the positions of N forward reference blocks and N backward reference blocks based on the initial motion information and the position of the current image block, the second search unit: Determine the position of the initial forward reference block of the current image block in the forward reference image corresponding to the first reference image index based on the first motion vector and the position of the current image block, and use the position of the initial forward reference block as a first search starting point to determine the positions of the (N-1) candidate forward reference blocks in the forward reference image, where the positions of the N forward reference blocks include the position of the initial forward reference block and the positions of the (N-1) candidate forward reference blocks; 35. The device of claim 31, further comprising: a first motion vector for determining a position of the current image block in the backward reference image corresponding to the second reference image index based on the second motion vector and the position of the current image block; a second motion vector for determining a position of the initial backward reference block in the backward reference image corresponding to the second reference image index; a second motion vector for determining a position of the (N-1) candidate backward reference blocks in the backward reference image using the position of the initial backward reference block as a second search starting point; and a second motion vector for determining a position of the (N-1) candidate backward reference blocks in the backward reference image, wherein the positions of the N backward reference blocks include the position of the initial backward reference block and the positions of the (N-1) candidate backward reference blocks.

36. In an aspect of determining, from M pairs of reference block locations based on a matching cost criterion, a pair of reference block locations is a target forward reference block location of the current image block and a target backward reference block location of the current image block, the second searching unit: From the positions of the M pairs of reference blocks, determine that the pair of reference block positions with the smallest matching error is the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block; or 36. The apparatus of claim 31, wherein the apparatus is specifically configured to: determine, from the positions of the M pairs of reference blocks, a pair of positions of reference blocks whose matching error is less than or equal to a matching error threshold to be the position of the target forward reference block of the current image block and the position of the target backward reference block of the current image block, wherein M is less than or equal to N.

37. An image prediction device, comprising: a third acquisition unit configured to acquire the i-th motion information of the current image block; a third search unit configured to: determine positions of N forward reference blocks and N backward reference blocks according to the i-th motion information and the position of the current image block, the N forward reference blocks being located in a forward reference image, and the N backward reference blocks being located in a backward reference image, where N is an integer greater than 1; determine from M pairs of reference block positions according to a matching cost criterion that a pair of reference block positions is a position of a target forward reference block of the current image block and a position of a target backward reference block of the current image block, where each pair of reference block positions includes a position of a forward reference block and a position of a backward reference block, and for each pair of reference block positions, a first position offset and a second position offset are in a mirror image relationship, the first position offset represents an offset of the position of the forward reference block relative to the position of the (i-1)th target forward reference block, and the second position offset represents an offset of the position of the backward reference block relative to the position of the (i-1)th target backward reference block, where M is an integer greater than or equal to 1 and M is less than or equal to N; and a third prediction unit configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the jth target forward reference block and pixel values ​​of the jth target backward reference block, where j is greater than or equal to i and both i and j are integers greater than or equal to 1.

38. If i=1, the i-th motion information is the initial motion information of the current image block; or 38. The apparatus of claim 37, wherein if i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block.

39. 39. The apparatus of claim 37 or 38, wherein the third prediction unit is particularly configured to, when an iteration termination condition is met, obtain the predicted value of the pixel value of the image block based on the pixel value of the jth target forward reference block and the pixel value of the jth target backward reference block, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

40. The first position offset and the second position offset are in a mirror image relationship.

40. The apparatus of claim 37, wherein a direction of the first position offset is opposite to a direction of the second position offset, and an amplitude value of the first position offset is the same as an amplitude value of the second position offset.

41. the i-th motion information includes a forward motion vector, a forward reference image index, a backward motion vector, and a backward reference image index; In an aspect of determining the positions of the N forward reference blocks and the N backward reference blocks based on the i-th motion information and the position of the current image block, the third search unit: According to the forward motion vector and the position of the current image block, determine a position of the (i-1)th target forward reference block of the current image block in the forward reference image corresponding to the forward reference image index, and set the position of the (i-1)th target forward reference block as the i-th f as a search starting point for determining positions of (N-1) candidate forward reference blocks in a forward reference image, wherein the positions of the N forward reference blocks include the position of the (i-1)th target forward reference block and the positions of the (N-1) candidate forward reference blocks; Determine the position of the (i-1)th target backward reference block of the current image block in the backward reference image corresponding to the backward reference image index based on the backward motion vector and the position of the current image block, and set the position of the (i-1)th target backward reference block as the i-th b 41. The apparatus of claim 37, further comprising: a backward reference image processing unit configured to: use a backward reference image as a search starting point for determining the positions of (N-1) candidate backward reference blocks in the backward reference image; and determine the positions of the N backward reference blocks, the positions including the position of the (i-1)th target backward reference block and the positions of the (N-1) candidate backward reference blocks.

42. In an aspect of determining, from M pairs of reference block positions based on a matching cost criterion, a pair of reference block positions is a position of the i-th target forward reference block of the current image block and a position of the i-th target backward reference block of the current image block, the third searching unit: From the positions of the M pairs of reference blocks, determine that the pair of reference block positions with the smallest matching error is the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block; or 42. The apparatus of claim 37, wherein the apparatus is specifically configured to: determine, from the positions of the M pairs of reference blocks, the positions of a pair of reference blocks whose matching error is less than or equal to a matching error threshold are the position of the i-th target forward reference block of the current image block and the position of the i-th target backward reference block of the current image block, wherein M is less than or equal to N.

43. An image prediction device, comprising: a fourth acquisition unit configured to acquire the i-th motion information of the current image block; a fourth search unit, which determines positions of N forward reference blocks and N backward reference blocks according to the i-th motion information and the position of the current image block, where the N forward reference blocks are located in a forward reference image, and the N backward reference blocks are located in a backward reference image, where N is an integer greater than 1; and determines from the M pairs of reference block positions according to a matching cost criterion that the positions of a pair of reference blocks are the position of a target forward reference block of the current image block and the position of a target backward reference block of the current image block, and the positions of each pair of reference blocks are the position of a forward reference block; a fourth searching unit configured to: determine a position of a forward reference block and a position of a backward reference block; and for the positions of each pair of reference blocks, a first position offset and a second position offset are in a proportional relationship based on a time-domain distance, the first position offset represents an offset of the position of the forward reference block relative to a position of the (i-1)th target forward reference block in the forward reference image, and the second position offset represents an offset of the position of the backward reference block relative to a position of the (i-1)th target backward reference block in the backward reference image, where M is an integer greater than or equal to 1 and M is less than or equal to N; and a fourth prediction unit configured to obtain predicted values ​​of pixel values ​​of the current image block based on pixel values ​​of the jth target forward reference block and pixel values ​​of the jth target backward reference block, where j is greater than or equal to i and both i and j are integers greater than or equal to 1.

44. If i=1, the i-th motion information is the initial motion information of the current image block; or 44. The apparatus of claim 43, wherein if i>1, the i-th motion information includes a forward motion vector pointing to the position of the (i-1)th target forward reference block and a backward motion vector pointing to the position of the (i-1)th target backward reference block.

45. 45. The apparatus of claim 43 or 44, wherein the fourth prediction unit is specifically configured to, when an iteration termination condition is met, obtain the predicted value of the pixel value of the image block based on the pixel value of the jth target forward reference block and the pixel value of the jth target backward reference block, where j is greater than or equal to i, and both i and j are integers greater than or equal to 1.

46. The fact that the first position offset and the second position offset are in a proportional relationship based on the time domain distance is if a first time-domain distance is the same as a second time-domain distance, then a direction of the first position offset is opposite to a direction of the second position offset and an amplitude value of the first position offset is the same as an amplitude value of the second position offset; or if a first time domain distance is different from a second time domain distance, a direction of the first position offset is opposite to a direction of the second position offset, and a proportional relationship between an amplitude value of the first position offset and an amplitude value of the second position offset is based on a proportional relationship between the first time domain distance and the second time domain distance; 46. ​​The apparatus of claim 43, wherein the first time domain distance represents a time domain distance between a current image to which the current image block belongs and the forward reference image, and the second time domain distance represents a time domain distance between the current image and the backward reference image.

47. 1. A video encoder configured to encode an image block, comprising:

47. The image prediction device according to any one of claims 25 to 46, comprising an inter-frame prediction module configured to obtain predicted values ​​of pixel values ​​of the image block through prediction; and an entropy coding module configured to code indication information into a bitstream, the indication information being used to indicate initial motion information of the image block; and a reconstruction module configured to reconstruct the image block based on the predictions of the pixel values ​​of the image block.

48. 1. A video decoder configured to decode a bitstream to obtain an image block, comprising: an entropy decoding module configured to decode the bitstream to obtain indication information, the indication information being used to indicate initial motion information of a current image block obtained through decoding; and 47. The image prediction device according to any one of claims 25 to 46, comprising an inter-frame prediction module configured to obtain predicted values ​​of pixel values ​​of the image block through prediction; and a reconstruction module configured to reconstruct the image block based on the predictions of the pixel values ​​of the image block.

Images