Inter prediction method and device

Abstract

To provide an inter prediction method and device.SOLUTION: An embodiment of the present application relates to a field of video coding technology, and discloses an inter prediction method and device for solving a problem in a prior art that the prediction sample obtained in an inter prediction mode is spatially discontinuous, prediction efficiency is affected, and prediction residual energy is relatively high. A specific solution includes analyzing a bitstream to obtain the motion information of a processing object picture block, performing motion compensation on the processing object picture block according to the motion information to obtain a prediction block of the processing object picture block, where the prediction block of the processing object picture block includes a prediction value of a target sample, and performing weighted calculation on one or more reconstructed values of one or more reference samples and the prediction value of the target sample to update the prediction value of the target sample, where the reference sample has a preset spatial position relationship with the target sample.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The embodiments of the present application relate to the field of video coding technology, and in particular to an inter prediction method and apparatus. [Background technology]

[0002] Digital video technologies may be widely used in various digital video devices. Digital video devices may implement video coding techniques, such as those described in standards defined in MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG4 Part 10 Advanced Video Coding (AVC), ITU-T H.265 (also referred to as High Efficiency Video Coding, HEVC), and extensions of these standards. Digital video devices may implement these video coding techniques to more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0003] At present, inter-prediction and intra-prediction techniques are mainly used in video coding to eliminate the temporal and spatial redundancy of video. Inter-prediction techniques only consider the temporal correlation between the same objects in adjacent frames of a picture, but do not consider the spatial correlation. As a result, the prediction samples obtained in existing inter-prediction modes are spatially non-contiguous. Moreover, the prediction efficiency is affected and the prediction residual energy is relatively large. Summary of the Invention [Means for solving the problem]

[0004] The embodiments of the present application provide an inter-prediction method and apparatus for performing spatial filtering on an inter-coded predictive block to improve coding efficiency.

[0005] To achieve this goal, the following technical solutions are used in the embodiments of this application.

[0006] According to a first aspect of an embodiment of the present application, an inter prediction method is provided, which includes the steps of: analyzing a bitstream to obtain motion information of a current picture block; performing motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, where the prediction block of the current picture block includes a prediction value of a target sample; and performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the prediction value of the target sample to update the prediction value of the target sample, where the reference sample has a preset spatial position relationship with the target sample.

[0007] Based on this solution, a spatial filtering operation is performed on the prediction value of a target sample by using neighboring reconstructed samples to improve the compression coding efficiency.

[0008] Referring to the first aspect, in one possible implementation, the one or more reference samples include a reconstructed sample having the same horizontal coordinate as the target sample and a predetermined vertical coordinate difference from the target sample, or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference from the target sample.

[0009] According to this solution, a filtering process is performed on a target sample by using a reference sample that has a preset spatial relationship with the target sample, and the coding efficiency is improved compared with the conventional technology.

[0010] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0011]

number

[0012] Here, the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are preset constants, and M1 and M2 are preset positive integers.

[0013] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0014] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, w1+w2=R1, w3+w4=R2, or w5+w6+w7=R3, where R1, R2, and R3 are each a power of 2, and n is a non-negative integer.

[0015] Based on this solution, the coding efficiency can be further improved.

[0016] It should be understood that R1, R2, and R3 are each the nth power of 2. R1, R2, and R3 can be the same or different, this is not limited. For example, R1, R2, and R3 can all be 8, or R1, R2, and R3 can be 2, 4, and 16, respectively.

[0017] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0018]

number

[0019] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and rec on(xP, yN-M4) represents the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0020] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0021] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, w1+w2+w3=S1, w4+w5+w6=S2, or w7+w8+w9+w10+w11=S3, where S1, S2, and S3 are each a power of 2, and n is a non-negative integer.

[0022] Based on this solution, the coding efficiency can be further improved.

[0023] It should be understood that S1, S2, and S3 are each the nth power of 2. S1, S2, and S3 can be the same or different, this is not limited. For example, S1, S2, and S3 can all be 8, or S1, S2, and S3 can be 2, 4, and 16, respectively.

[0024] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0025]

number

[0026] Here, the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are preset constants, and M1 and M2 are preset positive integers.

[0027] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0028] Referring to the first embodiment and possible implementations described above, in another possible implementation, w1+w2+w3=R, where R is the nth power of 2 and n is a non-negative integer.

[0029] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0030]

number

[0031] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, w1, w2, w3, w4, and w5 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0032] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0033] Referring to the first aspect and possible implementations described above, in another possible implementation, w1+w2+w3+w4+w5=S, where S is the nth power of 2, and n is a non-negative integer.

[0034] Based on this solution, the coding efficiency can be further improved.

[0035] With reference to the first aspect and the possible implementation forms described above, in another possible implementation form, the one or more reference samples include one or more of the following samples: a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the top of the picture block to be processed, a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left of the picture block to be processed, a reconstructed sample at the top right of the picture block to be processed, a reconstructed sample at the bottom left of the picture block to be processed, or a reconstructed sample at the top left of the picture block to be processed.

[0036] According to this solution, a filtering process is performed on a target sample by using a reference sample that has a preset spatial relationship with the target sample, and the coding efficiency is improved compared with the conventional technology.

[0037] Referring to the first aspect and the possible implementations described above, in another possible implementation, the step of updating the predicted value of the target sample includes performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is expressed by the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) Obtained according to, predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), where the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0038] In an implementable implementation of the first aspect, the predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the picture block to be processed are (0,0), and predP(xP,yP) is updated according to where p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1), respectively; w1, w2, and w3 are preset constants; and nTbW and nTbH represent the width and height of the picture block to be processed.

[0039] In a possible implementation of the first aspect, the predicted value of the target sample is calculated using the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) It is updated according to: predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), where the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0040] Based on this solution, the updated predicted value of the target sample can be obtained through filtering processing.

[0041] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of updating the predicted value of the target sample includes performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed values of the reference samples to obtain the updated predicted value of the target sample. The updated predicted value of the target sample is obtained according to the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) and predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+1)>>1, predV(xP,yP)=((nTbH-1-(yP-yN))*recon(xP,yN-1)+(yP-yN+1)*recon(xN-1,yN+nTbH)+(nTbH>>1))>>Log2(nTbH), predH(xP,yP)=((nTbW-1-(xP-xN))*recon(xN-1,yP)+(xP-xN+1)*recon(xN+nTbW,yN-1)+(nTbW>>1))>>Log2(nTbW) where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yP), and recon(xN+nTbW, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yN+nTbH), (xN-1, yP), and (xN+nTbW, yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

[0042] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0043] Referring to the first embodiment and possible implementations described above, in another possible implementation, the sum of w1 and w2 is the nth power of 2, where n is a non-negative integer.

[0044] Based on this solution, the coding efficiency can be further improved.

[0045] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0046]

number

[0047] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the processing target picture block are (xN, , yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0048] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0049] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0050]

number

[0051] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the picture block to be processed are (xN,yN). where predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-1, yP) and recon(xP, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-1, yP) and (xP, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents the clipping operation.

[0052] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0053] With reference to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample includes, when a reconstructed value of a reference sample is unavailable, determining the availability of samples adjacent to the upper and left of the picture block to be processed in a predetermined order until a predetermined amount of available reference samples is obtained, and performing a weighted calculation on the reconstructed values ​​of the available reference samples and the predicted value of the target sample.

[0054] Based on this solution, when the reconstructed value of a reference sample is unavailable, the reference samples located to the left and above the current picture block and whose reconstructed values ​​are available can be searched in a preset order. In this way, the predicted value of the target sample can be updated by using the reconstructed value of the available reference sample.

[0055] With reference to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of determining the availability of samples adjacent to the upper and left sides of the picture block to be processed in a predetermined order until a predetermined amount of available reference samples is obtained includes a step of obtaining available reference samples in the order from coordinates (xN-1, yN+nTbH-1) to coordinates (xN-1, yN-1) and from coordinates (xN, yN-1) to coordinates (xN+nTbW-1, yN-1) in a predetermined order.

[0056] Based on this solution, reconstructed values ​​of the available reference samples can be obtained.

[0057] Referring to the first embodiment and the possible implementation described above, in another possible implementation, when there is at least one available reference sample among all reference samples, if the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is unavailable, the available sample is searched in a pre-set order from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), and from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). When the available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of the sample (x, y). If the reconstructed value of reference sample (x,y) is unavailable in the set containing reference sample (xN-1,yN+nTbH-M), then the reconstructed value of reference sample (x,y) is set to the reconstructed value of sample (x,y+1), where M is greater than or equal to 2 and less than or equal to nTbH+1. If the reconstructed value of reference sample (x,y) is unavailable in the set containing reference sample (xN+N,yN-1), then the reconstructed value of reference sample (x,y) is set to the reconstructed value of reference sample (x-1,y), where N is greater than or equal to 0 and less than or equal to nTbW-1.

[0058] Based on this solution, reconstructed values ​​of the available reference samples can be obtained.

[0059] Referring to the first aspect and possible implementations described above, in another possible implementation, if the reconstructed value of the reference sample (xN-1, yN+nTbH-M) is unavailable, available reference samples may be searched in a pre-set order starting from coordinate (xN-1, yN+nTbH-M), where M is 1 or more and nTbH+1 or less. If the available reference sample is B, the reconstructed value of the reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of the reference sample B. If the reconstructed value of the reference sample at coordinate (xN+N, yN-1) is unavailable, available reference samples may be searched in a pre-set order starting from coordinate (xN+N, yN-1), where N is 0 or more and nTbW-1 or less. If the available reference sample is C, the reconstructed value of the reference sample (xN+N, yN-1) may be set to the reconstructed value of the reference sample C.

[0060] Based on this solution, reconstructed values ​​of the available reference samples can be obtained.

[0061] With reference to the first aspect and the possible implementations described above, in another possible implementation, if the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a pre-set order from coordinates (xN-1, yN+nTbH-1) to coordinates (xN-1, yN-1) and from coordinates (xN, yN-1) to coordinates (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of sample (x, y). If the reconstructed value of the reference sample (xN-1, yN+nTbH-M) is unavailable, the available reference samples may be searched in a reverse order from the pre-set order starting from coordinate (xN-1, yN+nTbH-M), where M is 1 or more and nTbH+1 or less. If the available reference sample is C, the reconstructed value of the reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of the reference sample C. If the reconstructed value of the reference sample at coordinate (xN+N, yN-1) is unavailable, the available reference samples may be searched in a reverse order from the pre-set order starting from coordinate (xN+N, yN-1), where N is 0 or more and nTbW-1 or less. If the available reference sample is D, the reconstructed value of the reference sample (xN+N, yN-1) may be set to the reconstructed value of the reference sample D.

[0062] Based on this solution, reconstructed values ​​of the available reference samples can be obtained.

[0063] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, if it is determined that all adjacent samples to the upper and left of the picture block to be processed are unavailable, the reconstructed value of the reference sample is set to 1<<(bitDepth-1), where bitDepth represents the bit depth of the sample value of the reference sample.

[0064] Based on this solution, the reconstructed value of the reference sample may be set based on the bit depth when both the reconstructed value of the reference sample and the reconstructed value of the new reference sample are unavailable.

[0065] With reference to the first aspect and the possible implementation forms described above, in another possible implementation form, before performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, the method includes the steps of: when the reference sample is located above the picture block to be processed, performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of the neighboring sample to the left and the neighboring sample to the right of the reference sample; when the reference sample is located to the left of the picture block to be processed, performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of the neighboring sample above and the neighboring sample below the reference sample; and updating the reconstructed value of the reference sample by using the result of the weighted calculation.

[0066] According to this solution, before the filtering operation is performed on the target sample, the filtering operation is performed on the reconstructed values ​​of the reference samples, thus further improving the coding efficiency and reducing the prediction residual.

[0067] With reference to the first aspect and the above-mentioned possible implementation forms, in another possible implementation form, before performing motion compensation on the picture block to be processed based on the motion information, the method further includes a step of first updating the motion information by using a first pre-set algorithm, and correspondingly, the step of performing motion compensation on the picture block to be processed based on the motion information includes a step of first performing motion compensation on the picture block to be processed based on the updated motion information.

[0068] According to this solution, the motion information is updated before the motion compensation is performed on the current block, and the motion compensation is performed based on the updated motion information. In this way, the prediction residual can be reduced.

[0069] With reference to the first aspect and the above-mentioned possible implementation forms, in another possible implementation form, after the predictive block of the picture block to be processed is obtained, the method further includes a step of pre-updating the predictive block by using a second pre-set algorithm, and correspondingly, the step of performing a weighted calculation on one or more reconstructed values ​​of the one or more reference samples and the predicted value of the target sample includes a step of performing a weighted calculation on one or more reconstructed values ​​of the one or more reference samples and the pre-updated predicted value of the target sample.

[0070] According to this solution, the prediction block of the current block is pre-updated, and a weighted calculation is performed based on the pre-updated prediction value and the reconstructed value of the reference sample. In this way, the prediction residual can be reduced.

[0071] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, after performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, the method further includes a step of updating the predicted value of the target sample by using a second pre-set algorithm.

[0072] Based on this solution, the predicted value of the target sample after spatial filtering can be updated by using a preset algorithm, thus reducing the prediction residual.

[0073] With reference to the first aspect and the possible implementation described above, in another possible implementation, before performing weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, the method further includes: analyzing the bitstream to obtain a prediction mode corresponding to the picture block to be processed, and determining that the prediction mode is a merge mode and / or an inter-evolutionary motion vector prediction (inter AMVP) mode. It can be understood that the inter-evolutionary motion vector prediction (inter AMVP) mode can also be referred to as an inter-motion vector prediction (inter MVP) mode.

[0074] According to this solution, the prediction mode corresponding to the current picture block may be determined before the filtering process.

[0075] With reference to the first aspect and the possible implementation forms described above, in another possible implementation form, before performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, the method further includes a step of analyzing the bitstream to obtain update decision indication information for the picture block to be processed, and a step of determining that the update decision indication information is used to indicate updating the predicted block of the picture block to be processed.

[0076] Based on this solution, the update decision indication information of the current picture block can be obtained by analyzing the bitstream, and it is determined that the prediction block of the current picture block should be updated.

[0077] With reference to the first aspect and the possible implementation forms described above, in another possible implementation form, before performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, the method further includes a step of obtaining pre-set update decision indication information of the picture block to be processed, and a step of determining that the update decision indication information is used to indicate updating the predicted block of the picture block to be processed.

[0078] According to this solution, update decision indication information of a current picture block can be obtained, and it is determined that a prediction block of the current picture block should be updated based on the update decision indication information.

[0079] According to a second aspect of an embodiment of the present application, an inter prediction apparatus is provided, which includes: an analysis module configured to analyze a bitstream to obtain motion information of a picture block to be processed; a compensation module configured to perform motion compensation on the picture block to be processed based on the motion information to obtain a prediction block of the picture block to be processed, where the prediction block of the picture block to be processed includes a prediction value of a target sample; and a calculation module configured to perform a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the prediction value of the target sample to update the prediction value of the target sample, where the reference sample has a predetermined spatial positional relationship with the target sample.

[0080] With reference to the second aspect and the possible implementation forms described above, in another possible implementation form, the one or more reference samples include a reconstructed sample having the same horizontal coordinate as the target sample and a predetermined vertical coordinate difference from the target sample, or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference from the target sample.

[0081] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is particularly configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0082]

number

[0083] Here, the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are preset constants, and M1 and M2 are preset positive integers.

[0084] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, w1+w2=R1, w3+w4=R2, or w5+w6+w7=R3, where R1, R2, and R3 are each a power of 2, and n is a non-negative integer.

[0085] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0086]

number

[0087] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and rec on(xP, yN-M4) represents the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0088] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, w1+w2+w3=S1, w4+w5+w6=S2, or w7+w8+w9+w10+w11=S3, where S1, S2, and S3 are each a power of 2, and n is a non-negative integer.

[0089] With reference to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured, in particular, to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0090]

number

[0091] Here, the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are preset constants, and M1 and M2 are preset positive integers.

[0092] Referring to the second aspect and possible implementations described above, in another possible implementation, w1+w2+w3=R, where R is the nth power of 2, and n is a non-negative integer.

[0093] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0094]

number

[0095] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, w1, w2, w3, w4, and w5 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0096] Referring to the second aspect and possible implementations described above, in another possible implementation, w1+w2+w3+w4+w5=S, where S is the nth power of 2, and n is a non-negative integer.

[0097] With reference to the second aspect and the possible implementation forms described above, in another possible implementation form, the one or more reference samples include one or more of the following samples: a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the top of the picture block to be processed, a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left of the picture block to be processed, a reconstructed sample at the top right of the picture block to be processed, a reconstructed sample at the bottom left of the picture block to be processed, or a reconstructed sample at the top left of the picture block to be processed.

[0098] Referring to the second aspect and the possible implementation described above, in another possible implementation, the calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is expressed by the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) Obtained according to , predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values of the reference samples at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1) respectively, w1 and w2 are pre-set constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0099] In an implementable implementation of the second aspect, the predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the picture block to be processed are (0,0), and predP(xP,yP) is updated according to where p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1), respectively; w1, w2, and w3 are preset constants; and nTbW and nTbH represent the width and height of the picture block to be processed.

[0100] In a possible implementation of the second aspect, the predicted value of the target sample is calculated according to the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Updated according to, predV(xP, yP) = ((nTbH - 1 - yP) * p(xP, -1) + (yP + 1) * p(-1, nTbH)) << Log2(nTbW), predH(xP, yP) = ((nTbW - 1 - xP) * p(-1, yP) + (xP + 1) * p(nTbW, -1)) << Log2(nTbH), where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (0, 0), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1) respectively, w1 and w2 are pre-set constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0101] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to perform a weighted calculation based on, in particular, the predicted value of the target sample before the update and the reconstructed values of the reference samples to obtain the updated predicted value of the target sample, and the updated predicted value of the target sample is given by the following formula predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2) / 2)) / (w1 + w2) is obtained according to predP1(xP, yP) = (predV(xP, yP) + predH(xP, yP) + 1) >> 1, predV(xP, yP) = ((nTbH - 1 - (yP - yN)) * recon(xP, yN - 1) + (yP - yN + 1) * recon(xN - 1, yN + nTbH) + (nTbH >> 1)) >> Log2(nTbH), predH(xP,yP)=((nTbW-1-(xP-xN))*recon(xN-1,yP)+(xP-xN+1)*recon(xN+nTbW,yN-1)+(nTbW>>1))>>Log2(nTbW) where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yP), and recon(xN+nTbW, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yN+nTbH), (xN-1, yP), and (xN+nTbW, yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

[0102] Referring to the second embodiment and possible implementations described above, in another possible implementation, the sum of w1 and w2 is the nth power of 2, where n is a non-negative integer.

[0103] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0104]

number

[0105] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the processing target picture block are (xN, , yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0106] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following equation:

[0107]

number

[0108] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the picture block to be processed are (xN,yN). where predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-1, yP) and recon(xP, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-1, yP) and (xP, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents the clipping operation.

[0109] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to determine the availability of adjacent samples above and to the left of the picture block to be processed, in a predetermined order, when the reconstructed value of the reference sample is unavailable, until a predetermined amount of available reference samples is obtained, and perform a weighted calculation on the reconstructed values ​​of the available reference samples and the predicted value of the target sample.

[0110] With reference to the second aspect and the possible implementation forms mentioned above, in another possible implementation form, the calculation module is particularly configured to obtain reconstructed values ​​of the available reference samples in the order from coordinates (xN-1,yN+nTbH-1) to coordinates (xN-1,yN-1) and from coordinates (xN,yN-1) to coordinates (xN+nTbW-1,yN-1).

[0111] Referring to the second aspect and the possible implementation described above, in another possible implementation, when there is at least one available reference sample among all reference samples, if the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is unavailable, the available sample is searched in a pre-set order from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), and from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). When the available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of the sample (x, y). If the reconstructed value of reference sample (x,y) is unavailable in the set containing reference sample (xN-1,yN+nTbH-M), then the reconstructed value of reference sample (x,y) is set to the reconstructed value of sample (x,y+1), where M is greater than or equal to 2 and less than or equal to nTbH+1. If the reconstructed value of reference sample (x,y) is unavailable in the set containing reference sample (xN+N,yN-1), then the reconstructed value of reference sample (x,y) is set to the reconstructed value of reference sample (x-1,y), where N is greater than or equal to 0 and less than or equal to nTbW-1.

[0112] With reference to the second aspect and the possible implementation described above, in another possible implementation, the calculation module is particularly configured to search for available reference samples in a pre-set order starting from coordinates (xN-1, yN+nTbH-M), where M is 1 or more and nTbH+1 or less, if the reconstructed value of the reference sample (xN-1, yN+nTbH-M) is unavailable. If the available reference sample is B, the reconstructed value of the reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of the reference sample B. If the reconstructed value of the reference sample at coordinates (xN+N, yN-1) is unavailable, the calculation module may search for available reference samples in a pre-set order starting from coordinates (xN+N, yN-1), where N is 0 or more and nTbW-1 or less. If the available reference sample is C, the reconstructed value of the reference sample (xN+N, yN-1) may be set to the reconstructed value of the reference sample C.

[0113] Referring to the second aspect and the possible implementation described above, in another possible implementation, if the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a pre-set order from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1) and from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of sample (x, y). If the reconstructed value of the reference sample (xN-1, yN+nTbH-M) is unavailable, the available reference samples may be searched in a reverse order from the pre-set order starting from coordinate (xN-1, yN+nTbH-M), where M is 1 or more and nTbH+1 or less. If the available reference sample is C, the reconstructed value of the reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of the reference sample C. If the reconstructed value of the reference sample at coordinate (xN+N, yN-1) is unavailable, the available reference samples may be searched in a reverse order from the pre-set order starting from coordinate (xN+N, yN-1), where N is 0 or more and nTbW-1 or less. If the available reference sample is D, the reconstructed value of the reference sample (xN+N, yN-1) may be set to the reconstructed value of the reference sample D.

[0114] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, if it is determined that all adjacent samples to the upper and left of the picture block to be processed are unavailable, the reconstructed value of the reference sample is set to 1<<(bitDepth-1), where bitDepth represents the bit depth of the sample value of the reference sample.

[0115] With reference to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to perform a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of the left and right neighboring samples of the reference sample when the reference sample is located above the picture block to be processed, and to perform a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of the upper and lower neighboring samples of the reference sample when the reference sample is located to the left of the picture block to be processed, and to update the reconstructed value of the reference sample by using a result of the weighted calculation.

[0116] With reference to the second aspect and the above-mentioned possible implementation forms, in another possible implementation form, the calculation module is further configured to initially update the motion information by using a first pre-set algorithm, and correspondingly, the compensation module is particularly configured to perform motion compensation on the picture block to be processed based on the initially updated motion information.

[0117] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to pre-update the prediction block by using a second pre-set algorithm, and correspondingly, the calculation module is particularly configured to perform a weighted calculation on one or more reconstructed values ​​of the one or more reference samples and the pre-updated predicted value of the target sample.

[0118] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to update the predicted value of the target sample by using a second pre-set algorithm.

[0119] With reference to the second aspect and the possible implementation described above, in another possible implementation, the analysis module is further configured to analyze the bitstream to obtain a prediction mode corresponding to the picture block to be processed, and determine that the prediction mode is a merge mode and / or an inter-evolutionary motion vector prediction (inter AMVP) mode. It can be understood that the inter-evolutionary motion vector prediction (inter AMVP) mode can also be referred to as an inter-motion vector prediction (inter MVP) mode.

[0120] With reference to the second aspect and the possible implementation forms described above, in another possible implementation form, the analysis module is further configured to analyze the bitstream to obtain update decision indication information for the picture block to be processed, and determine that the update decision indication information is used to indicate updating a prediction block of the picture block to be processed.

[0121] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to obtain pre-set update decision indication information of the picture block to be processed, and determine that the update decision indication information is used to indicate updating the prediction block of the picture block to be processed.

[0122] According to a third aspect of the present application, there is provided a motion information prediction device, comprising a processor and a memory coupled to the processor, the processor configured to perform a method according to the first aspect.

[0123] According to a fourth aspect of the present application, there is provided a computer readable storage medium storing instructions which, when executed on a computer, enable the computer to carry out a method according to the first aspect.

[0124] According to a fifth aspect of the present application there is provided a computer program product comprising instructions which, when executed on a computer, enable the computer to carry out a method according to the first aspect.

[0125] It should be understood that the technical solutions of the second to fifth aspects of the present application are consistent with the technical solutions of the first aspect of the present application. The beneficial effects achieved in all aspects and corresponding possible implementations of all aspects are similar. Therefore, details will not be described again. [Brief description of the drawings]

[0126] [Figure 1] 1 is a block diagram of an example video coding system that may be configured for use in embodiments of the present application. [Diagram 2] FIG. 2 is a system block diagram of an example video encoder that may be configured for use in embodiments of the present application. [Diagram 3] FIG. 2 is a system block diagram of an example video decoder that may be configured for use in embodiments of the present application. [Figure 4] FIG. 2 is a block diagram of an example inter-prediction module that may be configured for use in embodiments of the present application. [Diagram 5] 1 is a flowchart of an example implementation of a merge prediction mode. [Figure 6] 11 is a flowchart of an example implementation of an evolved motion vector prediction mode. [Figure 7] 4 is a flowchart of an example implementation of motion compensation performed by a video decoder that may be configured for use in embodiments of the present application. [Figure 8] 1 is a schematic diagram of an example of a coding unit and neighboring picture blocks associated with the coding unit. [Figure 9] 1 is a flowchart of an example implementation of building a motion vector predictor candidate list. [Figure 10]FIG. 13 is a schematic diagram of an example implementation of adding a combined motion vector candidate to a merge mode predicted motion vector candidate list. [Figure 11] FIG. 13 is a schematic diagram of an example implementation of adding a scaled motion vector candidate to a merge mode predicted motion vector candidate list. [Figure 12] 1 is a schematic diagram of an example implementation of adding a zero motion vector to a merge mode predicted motion vector candidate list. [Figure 13] 1 is a schematic flowchart of an inter-prediction method according to an embodiment of the present application; [Figure 14] FIG. 1 is a schematic diagram of an application of an inter-prediction method according to an embodiment of the present application. [Figure 15] FIG. 2 is a schematic diagram of the application of an inter-prediction method according to an embodiment of the present application. [Figure 16] 3 is a schematic diagram of the application of an inter-prediction method according to an embodiment of the present application; [Figure 17] 4 is a schematic diagram of the application of an inter-prediction method according to an embodiment of the present application; [Figure 18] 1 is a schematic block diagram of an inter prediction device according to an embodiment of the present application; [Figure 19] 2 is a schematic block diagram of another inter prediction device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0127] The following clearly and completely describes the technical solutions of the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application.

[0128] FIG. 1 is a block diagram of an example of a video coding system according to an embodiment of the present application. As described herein, the term "video coder" generally refers to both a video encoder and a video decoder. In the present application, the term "video coding" or "coding" generally refers to video encoding or video decoding. The video encoder 100 and the video decoder 200 in the video coding system are configured to predict motion information, e.g., motion vectors, of a current coded picture block or a sub-block of the current coded picture block according to various example methods described based on any one of a number of new inter-prediction modes provided in the present application, so that the predicted motion vector is maximally close to the motion vector obtained by using a motion estimation method, and no motion vector difference needs to be transmitted during coding. Moreover, this improves coding performance.

[0129] As shown in FIG. 1, the video coding system includes a source device 10 and a destination device 20. The source device 10 generates encoded video data. Thus, the source device 10 may be referred to as a video encoding device. The destination device 20 may decode the encoded video data generated by the source device 10. Thus, the destination device 20 may be referred to as a video decoding device. In various implementation solutions, the source device 10, the destination device 20, or both, the source device 10 and the destination device 20 may include one or more processors and a memory coupled to the one or more processors. As described herein, the memory may include, but is not limited to, a RAM, a ROM, an EEPROM, a flash memory, or any other medium that may be used to store required program code in the form of instructions or data structures and that may be accessed by a computer.

[0130] The source device 10 and destination device 20 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handheld telephone sets such as “smart” phones, television sets, cameras, display devices, digital media players, video game consoles, in-vehicle computers, and the like.

[0131] The destination device 20 may receive the encoded video data from the source device 10 via the link 30. The link 30 may include one or more media or devices capable of transferring the encoded video data from the source device 10 to the destination device 20. In an example, the link 30 may include one or more communication media that enable the source device 10 to transmit the encoded video data directly to the destination device 20 in real time. In this example, the source device 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the destination device 20. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may constitute part of a packet-based network, which may be, for example, a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include a router, a switch, a base station, or another device that facilitates communication from the source device 10 to the destination device 20.

[0132] In another example, the encoded data may be output to storage device 40 through output interface 140. Similarly, the encoded data may be accessed from storage device 40 through input interface 240. Storage device 40 may include any one of a number of distributed or locally accessible data storage media, such as hard disk drives, Blu-ray discs, digital video discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium configured to store encoded video data.

[0133] In another example, the storage device 40 may correspond to a file server or another intermediate storage device capable of storing the encoded video generated by the source device 10. The destination device 20 may access the stored video data from the storage device 40 through streaming transmission or download. The file server may be any type of server capable of storing the encoded video data and transmitting the encoded video data to the destination device 20. In one example, the file server includes a network server (e.g., used for a website), a file transfer protocol (FTP) server, a network-attached storage (NAS) device, or a local disk drive. The destination device 20 may access the encoded video data through any standard data connection (including an Internet connection). The standard data connection may include a wireless channel (e.g., a wireless-fidelity (Wi-Fi) connection), a wired connection (e.g., a digital subscriber line (DSL) or cable modem), or a combination of wireless and wired channels suitable for accessing the encoded video data stored in the file server. The encoded video data may be transmitted from storage device 40 through a streaming transmission, a download transmission, or a combination thereof.

[0134] The motion vector prediction techniques of the present application are applicable to video coding to support multiple multimedia applications, such as terrestrial television broadcast, cable television transmission, satellite television transmission, video streaming transmission (e.g., over the Internet), encoding video data stored on a data storage medium, decoding video data stored on a data storage medium, or another application. In some examples, the video coding system may be configured to support unidirectional or bidirectional video transmission to support applications such as video streaming transmission, video playback, video broadcast, and / or video telephony.

[0135] The video coding system illustrated in FIG. 1 is merely an example, and the techniques of the present application are applicable to video coding settings (e.g., video encoding or video decoding) that do not necessarily include data communication between an encoding device and a decoding device. In another example, data is retrieved from a local memory, streamed over a network, etc. A video encoding device can encode data and store the data in memory, and / or a video decoding device can retrieve data from memory and decode data. In many examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data and store the data in memory and / or retrieve data from memory and decode data.

[0136] 1, source device 10 includes video source 120, video encoder 100, and output interface 140. In some examples, output interface 140 may include a modulator / demodulator (modem) and / or a transmitter. Video source 120 may include a video capture device (e.g., a video camera), a video archive containing previously captured video data, a video feed-in interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of the aforementioned video data sources.

[0137] Video encoder 100 may encode video data from video source 120. In some examples, source device 10 transmits the encoded video data directly to destination device 20 through output interface 140. In other examples, the encoded video data may be further stored in storage device 40 so that destination device 20 subsequently accesses the encoded video data for decoding and / or playback.

[0138] In the example of FIG. 1, destination device 20 includes an input interface 240, a video decoder 200, and a display device 220. In some examples, input interface 240 includes a receiver and / or a modem. Input interface 240 may receive encoded video data via link 30 and / or from storage device 40. Display device 220 may be integrated with destination device 20 or may be located external to destination device 20. In general, display device 220 displays the decoded video data. Display device 220 may include multiple types of display devices, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0139] 1, the video encoder 100 and the video decoder 200 may be integrated with an audio encoder and an audio decoder, respectively, and may include appropriate multiplexer-demultiplexer units or other hardware and software to encode both audio and video in the same or separate data streams. In some examples, where applicable, the demultiplexer (MUX-DEMUX) units may conform to the International Telecommunication Union (ITU) H.223 multiplexer protocol, or another protocol such as the user datagram protocol (UDP).

[0140] The video encoder 100 and the video decoder 200 may each be implemented as any one of a number of circuits, for example, one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuits, hardware, or any combination thereof. If the present application is implemented in part using software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium and use one or more processors to execute instructions in hardware to implement the techniques of the present application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered as one or more processors. The video encoder 100 and the video decoder 200 may each be included in one or more encoders or decoders. Any of the encoders or decoders may be integrated as part of a composite encoder / decoder (codec) in the corresponding device.

[0141] In this application, video encoder 100 may be generally referred to as a device that "signals" or "transmits" some information to another device, such as video decoder 200. The terms "signaling" or "transmitting" may generally refer to the transmission of syntax elements and / or other data used to decode compressed video data. The transmission may be performed in real time or near real time. Alternatively, the communication may be performed after a period of time, for example, when the syntax elements in the encoded bitstream are stored to a computer-readable storage medium during encoding. A decoding device may then retrieve the syntax elements at any time after the syntax elements are stored to the medium.

[0142] JCT-VC developed the H.265 high efficiency video coding (HEVC) standard. The HEVC standardization is based on an evolved model of a video decoder, which is called the HEVC test model (HM). The latest H.265 standard document is available at http: / / www.itu.int / rec / T-REC-H.265. The latest version of the standard document is H.265(12 / 16), which is incorporated herein by reference in its entirety. In the HM, it is assumed that the video decoder has some additional capabilities compared to the existing algorithms in ITU-TH.264 / AVC. For example, H.264 provides 9 intra-prediction coding modes, while the HM can provide up to 35 intra-prediction coding modes.

[0143] JVET is tasked with developing the H.266 standard. The H.266 standardization process is based on an evolved model of a video decoding device, which is called the H.266 Test Model. The H.266 algorithm is available at http: / / phenix.int-evry.fr / jvet, and the latest algorithm description is included in JVET-F1001-v2. The algorithm description document is incorporated herein by reference in its entirety. In addition, reference software for the JEM Test Model is available at http: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / , which is also incorporated herein by reference in its entirety.

[0144] In general, as described in the HM working model, a video frame or picture may be divided into a sequence of treeblocks or largest coding units (LCUs), which contain both luma and chroma samples. LCUs are also called coding tree units (CTUs). A treeblock has a function similar to that of a macroblock in the H.264 standard. A slice includes several consecutive treeblocks in decoding order. A video frame or picture may be partitioned into one or more slices. Each treeblock may be divided into coding units based on a quadtree. For example, a treeblock, which is a root node of the quadtree, may be divided into four child nodes, each of which may also be a parent node and may be further divided into four other child nodes. The last undividable child node, which is a leaf node of the quadtree, includes a decoding node, e.g., a decoded video block. The maximum number of times a treeblock may be divided and the minimum size of a decoding node may be defined in syntax data associated with the decoded bitstream.

[0145] A coding unit includes a decoding node, a prediction unit (PU), and a transform unit (TU) associated with the decoding node. The size of a CU corresponds to the size of a decoding node, and the shape of a CU should be square. The size of a CU may range from 8×8 pixels up to 64×64 pixels, or may be a larger treeblock size. Each CU may include one or more PUs and one or more TUs. For example, syntax data associated with a CU may describe a partition of one CU into one or more PUs. The partition mode may change when the CU is coded based on a skip mode or a direct mode, when coded based on an intra prediction mode, or when coded based on an inter prediction mode. The PU obtained through the partition may be a shape other than a square. For example, syntax data associated with a CU may alternatively describe a partition of one CU into one or more TUs based on a quadtree. The TUs may be square or non-square in shape.

[0146] The HEVC standard allows for TU-based transforms. TUs may be different for different CUs. The size of a TU is usually set based on the size of the PUs in a given CU defined for partitioned LCUs. However, this is not always the case. The size of a TU is usually equal to or less than the size of a PU. In some possible implementations, a quadtree structure called a "residual quadtree" (RQT) may be used to partition residual samples corresponding to a CU into smaller units. Leaf nodes of the RQT may be called TUs. Pixel differences associated with the TUs may be transformed to generate transform coefficients, and the transform coefficients may be quantized.

[0147] In general, a PU includes data related to a prediction process. For example, when a PU is coded based on an intra prediction mode, the PU may include data describing the intra prediction mode corresponding to the PU. In another possible implementation, when a PU is coded based on an inter prediction mode, the PU may include data defining a motion vector of the PU. For example, the data defining a motion vector of the PU may describe a horizontal component of the motion vector, a vertical component of the motion vector, a resolution of the motion vector (e.g., 1 / 4 sample precision or 1 / 8 sample precision), a reference picture to which the motion vector points, and / or a reference picture list of the motion vector (e.g., list 0, list 1, or list C).

[0148] In general, a transform and quantization process is used for the TU. A given CU, which includes one or more PUs, may also include one or more TUs. After prediction, the video encoder 100 may calculate a residual value corresponding to the PU. The residual value includes pixel differences. The pixel differences may be transformed into transform coefficients, which are quantized and undergo TU traversal to generate serialized transform coefficients for entropy decoding. In this application, the term "video block" is generally used to indicate a decoding node of a CU. In some specific applications, in this application, the term "video block" may also be used to indicate a tree block, such as an LCU or a CU, which includes a decoding node, a PU, and a TU.

[0149] A video sequence typically includes a series of video frames or pictures. For example, a group of pictures (GOP) includes a series of video pictures, or one or more video pictures. The GOP may include syntax data in the header information of the GOP, in one or more header information of the pictures, or elsewhere, which describes the amount of pictures included in the GOP. Each slice of a picture may include slice syntax data that describes a coding mode for the corresponding picture. Video encoder 100 typically performs operations on video blocks in video slices to encode video data. The video blocks may correspond to decoding nodes in a CU. The size of the video blocks may be constant or variable and may change with a specified decoding standard.

[0150] In a possible implementation, the HM supports prediction for PUs with various sizes. Assuming that the size of a given CU is 2N×2N, the HM supports intra prediction for PUs with sizes of 2N×2N or N×N, and inter prediction for symmetric PUs with sizes of 2N×2N, 2N×N, N×2N, or N×N. The HM also supports asymmetric prediction for inter prediction for PU sizes of 2N×nU, 2N×nD, nL×2N, or nR×2N. In asymmetric partitioning, the CU is not partitioned in one direction, but is partitioned into two parts in the other direction, one part occupies 25% of the CU, and the other part occupies 75% of the CU. The part that occupies 25% of the CU is indicated by an indicator that includes "U (Up)", "D (Down)", "L (Left)", or "R (Right)" after "n". So, for example, "2N×nU" refers to a horizontally partitioned 2N×2N CU, with 2N×0.5N PUs on top and 2N×1.5N PUs on the bottom.

[0151] In this specification, "NxN" and "N by N" are used interchangeably to indicate the pixel size of a video block in vertical and horizontal dimensions, e.g., 16x16 pixels or 16 by 16 pixels. In general, a 16x16 block has 16 pixels in the vertical direction (y=16) and 16 pixels in the horizontal direction (x=16). Similarly, an NxN block has N pixels in the vertical direction and N pixels in the horizontal direction, where N is a non-negative integer value. The pixels in a block may be arranged in rows and columns. In addition, the amount of pixels in the horizontal direction and the amount of pixels in the vertical direction may not necessarily be the same in a block. For example, a block may include NxM pixels, where M is not necessarily equal to N.

[0152] After intra-prediction or inter-prediction decoding is performed on the PUs of a CU, video encoder 100 may calculate residual data of the TUs in the CU. The PUs may include pixel data in the spatial domain (also referred to as the pixel domain). The TUs may include coefficients in the transform domain after a transform (e.g., a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform) is applied to the residual video data. The residual data may correspond to pixel differences between pixels of the uncoded picture and predicted values ​​corresponding to the PU. Video encoder 100 may generate TUs including the residual data of the CU and then transform the TUs to generate transform coefficients of the CU.

[0153] After performing any transforms to generate transform coefficients, video encoder 100 may quantize the transform coefficients. Quantization refers to the process of quantifying coefficients, e.g., to reduce the amount of data used to represent the coefficients and further implement compression. The quantization process can reduce the bit depth associated with some or all of the coefficients. For example, during quantization, an n-bit value may be reduced to an m-bit value through rounding, where n is greater than m.

[0154] The JEM model further improves the video picture coding structure. Specifically, a block coding structure called the "quadtree plus binary tree" (QTBT) structure is introduced. Without using concepts such as CU, PU, ​​and TU in HEVC, the QTBT structure supports more flexible CU partition shapes. The CUs may be square or rectangular in shape. Quadtree partitioning is first performed on the CTU, and binary tree partitioning is further performed on the leaf nodes of the quadtree. In addition, binary tree partitioning has two partitioning modes: symmetric horizontal partitioning and symmetric vertical partitioning. The leaf nodes of the binary tree are called CUs. The CUs in the JEM model cannot be further partitioned during prediction and transformation. In other words, the CUs, PUs, and TUs in the JEM model have the same block size. In the existing JEM model, the maximum CTU size is 256x256 luma pixels.

[0155] In some possible implementations, the video encoder 100 may scan the quantized transform coefficients in a predetermined scan order to generate a serialized vector that can be entropy coded. In other possible implementations, the video encoder 100 may perform adaptive scanning. After scanning the quantized transform coefficients to generate a one-dimensional vector, the video encoder 100 may perform entropy coding on the one-dimensional vector by using a context-based adaptive variable-length code (CAVLC), a context-based adaptive binary arithmetic coding (CABAC), a syntax-based adaptive binary arithmetic coding (SBAC), a probability interval partitioning entropy (PIPE) coding, or another entropy coding method. The video encoder 100 may further perform entropy coding on syntax elements associated with the coded video data for the video decoder 200 to decode the video data.

[0156] To perform CABAC, video encoder 100 may assign a context in a context model to a symbol to be transmitted. The context may relate to whether the adjacent values ​​of the symbol are non-zero. To perform CAVLC, video encoder 100 may select a variable-length code for the symbol to be transmitted. Codewords in variable-length coding (VLC) may be constructed such that shorter codes correspond to more probable symbols and longer codes correspond to less probable symbols. In this way, bit rates can be reduced using VLC compared to using equal-length codewords for all symbols to be transmitted. Probabilities in CABAC may be determined based on the contexts assigned to the symbols.

[0157] In this embodiment of the present application, the video encoder may perform inter prediction to reduce temporal redundancy between pictures. As described above, a CU may have one or more prediction units PUs depending on different video compression coding standards. In other words, multiple PUs may belong to one CU, or a PU and a CU may have the same size. In this specification, when a CU and a PU have the same size, the partition mode corresponding to a CU is, for illustration purposes, that the CU is not partitioned, or the CU is partitioned into one PU, and the PU is used uniformly. When a video encoder performs inter prediction, the video encoder may signal motion information of a PU to a video decoder. For example, the motion information of a PU may include a reference picture index, a motion vector, and a prediction direction indicator. The motion vector may indicate the offset between a picture block (also referred to as a video block, pixel block, pixel set, etc.) of a PU and a reference block of the PU. The reference block of a PU may be part of a reference picture similar to the picture block of the PU. The reference block may be located in a reference picture indicated by a reference picture index and a prediction direction indicator.

[0158] In order to reduce the amount of coding bits required to represent the motion information of a PU, a video encoder may generate a predicted motion vector (Motion Vector, MV) candidate list for each PU based on a merge prediction mode or an evolved motion vector prediction mode. Each predicted motion vector candidate in the predicted motion vector candidate list for a PU may indicate motion information. The motion information indicated by some predicted motion vector candidates in the predicted motion vector candidate list may be based on the motion information of other PUs. If a predicted motion vector candidate indicates motion information of one of a designated spatial predicted motion vector candidate at a position or a designated temporal predicted motion vector candidate at a position, the predicted motion vector candidate may be referred to as an "original" predicted motion vector candidate in this application. For example, in a merge mode, also referred to herein as a merge prediction mode, there may be five original spatial predicted motion vector candidates at a position and one original temporal predicted motion vector candidate at a position. In some examples, a video encoder may generate an additional predicted motion vector candidate by combining some motion vectors from different original predicted motion vector candidates, modifying the original predicted motion vector candidate, or inserting only a zero motion vector as a predicted motion vector candidate. The additional motion vector predictor candidate may be considered as an original motion vector predictor candidate and may be referred to as an artificially generated motion vector predictor candidate in this application.

[0159] The technology of the present application generally includes a technique for generating a predictive motion vector candidate list on a video encoder and a technique for generating the same predictive motion vector candidate list on a video decoder. The video encoder and the video decoder may generate the same predictive motion vector candidate list by implementing the same technique for constructing a predictive motion vector candidate list. For example, the video encoder and the video decoder may construct a list with the same amount of predictive motion vector candidates (e.g., five predictive motion vector candidates). The video encoder and the video decoder may first consider spatial predictive motion vector candidates (e.g., adjacent blocks in the same picture), then consider temporal predictive motion vector candidates (e.g., predictive motion vector candidates in different pictures), and finally consider artificially generated predictive motion vector candidates until a required amount of predictive motion vector candidates are added to the list. According to the technology of the present application, during the construction of the predictive motion vector candidate list, a pruning operation may be performed on some types of predictive motion vector candidates to remove repetitive predictive motion vector candidates from the predictive motion vector candidate list, while a pruning operation may not be performed on other types of predictive motion vector candidates to reduce the complexity of the decoder. For example, a pruning operation may be performed on the set of spatial motion vector predictor candidates and temporal motion vector predictor candidates to remove motion vector predictor candidates with the same motion information from the motion vector predictor candidate list. However, an artificially generated motion vector predictor candidate may be added to the motion vector predictor candidate list without being pruned.

[0160] After generating the predicted motion vector candidate for the PU of the CU, the video encoder may select a predicted motion vector candidate from the predicted motion vector candidate list and output a predicted motion vector candidate index in the bitstream. The selected predicted motion vector candidate may be a predicted motion vector candidate for generating a motion vector that best matches a predicted value of the target PU being decoded. The predicted motion vector candidate index may indicate a position of the selected predicted motion vector candidate in the predicted motion vector candidate list. The video encoder may further generate a predicted picture block of the PU based on a reference block indicated by the motion information of the PU. The motion information of the PU may be determined based on the motion information indicated by the selected predicted motion vector candidate. For example, in a merge mode, the motion information of the PU may be the same as the motion information indicated by the selected predicted motion vector candidate. In an AMVP mode, the motion information of the PU may be determined based on a motion vector differential for the PU and the motion information indicated by the selected predicted motion vector candidate. The video encoder may generate one or more residual picture blocks of the CU based on the predicted picture block of the PU of the CU and the original picture block of the CU. The video encoder may then encode the one or more residual picture blocks and output the one or more residual picture blocks in a bitstream.

[0161] The bitstream may include data identifying a selected predictive motion vector candidate in the predictive motion vector candidate list for the PU. The video decoder may determine motion information of the PU based on the motion information indicated by the selected predictive motion vector candidate in the predictive motion vector candidate list for the PU. The video decoder may identify one or more reference blocks of the PU based on the motion information of the PU. After identifying the one or more reference blocks of the PU, the video decoder may generate a predictive picture block of the PU based on the one or more reference blocks of the PU. The video decoder may reconstruct a picture block of the CU based on the predictive picture block of the PU of the CU and one or more residual picture blocks of the CU.

[0162] For ease of explanation, in this application, a location or a picture block may be described as a location or a picture block having various spatial relationships with a CU or a PU. This description may be described as a location or a picture block having various spatial relationships with a picture block associated with a CU or a PU. In addition, in this application, a PU currently being decoded by a video decoder may be referred to as a current PU, and may also be referred to as a currently processed picture block. In this application, a CU currently being decoded by a video decoder may be referred to as a current CU. In this application, a picture currently being decoded by a video decoder may be referred to as a current picture. This application is also applicable when a PU and a CU have the same size, or when a PU is a CU. For the purpose of explanation, a PU is used uniformly.

[0163] As briefly described above, video encoder 100 may generate predicted picture blocks and motion information for a PU of a CU through inter prediction. In many examples, the motion information of a given PU may be the same as or similar to the motion information of one or more neighboring PUs (i.e., PUs whose picture blocks are spatially or temporally adjacent to the picture blocks of the given PU). Because neighboring PUs often have similar motion information, video encoder 100 may encode the motion information of the given PU based on the motion information of the neighboring PUs. Encoding the motion information of a given PU based on the motion information of the neighboring PUs can reduce the amount of coding bits in a bitstream required to indicate the motion information of the given PU.

[0164] Video encoder 100 may encode the motion information of a given PU based on the motion information of a neighboring PU in various manners. For example, video encoder 100 may indicate that the motion information of a given PU is the same as that of a neighboring PU. In the present application, merge mode may be used to indicate that the motion information of a given PU is the same as or can be derived from that of a neighboring PU. In another possible implementation, video encoder 100 may calculate a motion vector difference (Motion Vector Difference, MVD) for a given PU. The MVD indicates the difference between the motion vector of the given PU and the motion vector of a neighboring PU. Video encoder 100 may include the MVD in the motion information of the given PU instead of the motion vector of the given PU. In the bitstream, the amount of coding bits required to represent the MVD is less than the amount of coding bits required to represent the motion vector of the given PU. In the present application, evolved motion vector prediction mode may be used to indicate that the motion information of a given PU is signaled to the decoder side by using the MVD and an index value used to identify a motion vector candidate.

[0165] To signal the motion information of a given PU to the decoder side based on the merge mode or the AMVP mode, video encoder 100 may generate a predictive motion vector candidate list for the given PU. The predictive motion vector candidate list may include one or more predictive motion vector candidates. Each of the predictive motion vector candidates in the predictive motion vector candidate list for the given PU may indicate motion information. The motion information indicated by each predictive motion vector candidate may include a motion vector, a reference picture index, and a prediction direction indicator. The predictive motion vector candidates in the predictive motion vector candidate list may include an "original" predictive motion vector candidate. Each of the predictive motion vector candidates indicates motion information in one of the specified predictive motion vector candidates at a position in a PU different from the given PU.

[0166] After generating the motion vector predictor candidate list for the PU, video encoder 100 may select one motion vector predictor candidate from the motion vector predictor candidate list to be used for the PU. For example, the video encoder may compare each motion vector predictor candidate with the PU being decoded and may select a motion vector predictor candidate with a required rate-distortion cost. Video encoder 100 may output a motion vector predictor candidate index for the PU. The motion vector predictor candidate index may identify a position of the selected motion vector predictor candidate in the motion vector predictor candidate list.

[0167] In addition, video encoder 100 may generate a predicted picture block of the PU based on the reference block indicated by the motion information of the PU. The motion information of the PU may be determined based on the motion information indicated by a selected predictive motion vector candidate in a predictive motion vector candidate list for the PU. For example, in a merge mode, the motion information of the PU may be the same as the motion information indicated by the selected predictive motion vector candidate. In an AMVP mode, the motion information of the PU may be determined based on a motion vector differential for the PU and the motion information indicated by the selected predictive motion vector candidate. As described above, video encoder 100 may process the predicted picture block of the PU.

[0168] When the video decoder 200 receives the bitstream, the video decoder 200 may generate a predictor motion vector candidate list for each PU of the CU. The predictor motion vector candidate list generated by the video decoder 200 for the PU may be the same as the predictor motion vector candidate list generated by the video encoder 100 for the PU. The syntax element obtained by parsing the bitstream may indicate the position of the selected predictor motion vector candidate in the predictor motion vector candidate list for the PU. After generating the predictor motion vector candidate list for the PU, the video decoder 200 may generate a predictive picture block of the PU based on one or more reference blocks indicated by the motion information of the PU. The video decoder 200 may determine the motion information of the PU based on the motion information indicated by the selected predictor motion vector candidate in the predictor motion vector candidate list for the PU. The video decoder 200 may reconstruct a picture block of the CU based on the predictive picture block of the PU and the residual picture block of the CU.

[0169] In one possible implementation form, at the decoder side, constructing a predicted motion vector candidate list and parsing the bitstream to obtain the position of a selected predicted motion vector candidate in the predicted motion vector candidate list are independent of each other and can be performed in any order or in parallel.

[0170] In another possible implementation, at the decoder side, the position of the selected motion vector predictor candidate in the motion vector predictor candidate list is first obtained by analyzing the bitstream, and then the motion vector predictor candidate list is constructed based on the position obtained through analysis.In this implementation, it is not necessary to construct all motion vector predictor candidate lists, and specifically, as long as the motion vector predictor candidate at the position obtained through analysis can be determined, only the motion vector predictor candidate list at the position needs to be constructed.For example, when the motion vector predictor candidate to be selected is the motion vector predictor candidate whose index is 3 in the motion vector predictor candidate list, it is obtained by analyzing the bitstream, only the motion vector predictor candidate list needs to be constructed including the motion vector predictor candidate whose index is 0 to the motion vector predictor candidate whose index is 3, and the motion vector predictor candidate whose index is 3 can be determined.This can reduce complexity and improve decoding efficiency.

[0171] 2 is a block diagram of an example of a video encoder 100 according to an embodiment of the present application. The video encoder 100 is configured to output video to a post-processing entity 41. The post-processing entity 41 represents an example of a video entity that can process encoded video data from the video encoder 100. For example, the video entity is a media aware network element (MANE) or a splicing device / editing device. In some cases, the post-processing entity 41 may be an example of a network entity. In some video coding systems, the post-processing entity 41 and the video encoder 100 may be components of separate devices. In other cases, the functions of the post-processing entity 41 may be performed by the same device that includes the video encoder 100. In one example, the post-processing entity 41 is an example of the storage device 40 of FIG. 1.

[0172] In the example of FIG. 2, the video encoder 100 includes a prediction processing unit 108, a filter unit 106, a decoded picture buffer (DPB) 107, an adder 112, a transformer 101, a quantizer 102, and an entropy encoder 103. The prediction processing unit 108 includes an inter predictor 110 and an intra predictor 109. To reconstruct picture blocks, the video encoder 100 further includes an inverse quantizer 104, an inverse transformer 105, and an adder 111. The filter unit 106 is intended to represent one or more loop filters, for example, a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although the filter unit 106 is illustrated as an in-loop filter in FIG. 2, in another implementation, the filter unit 106 may be implemented as a post-loop filter. In an example, the video encoder 100 may further include a video data memory and a partition unit (not shown in the figure).

[0173] The video data memory may store video data to be encoded by the components of the video encoder 100. The video data stored in the video data may be obtained from the video source 120. The DPB 107 may be a reference picture memory that stores reference video data used by the video encoder 100 to encode the video data in an intra-coding mode or an inter-coding mode. The video data memory and the DPB 107 may include any one of a number of types of memory devices, for example, dynamic random access memory (DRAM), including synchronous dynamic random access memory (SDRAM), magnetic random access memory (MRAM), resistive random access memory (RRAM), or another type of memory device. The video data memory and the DPB 107 may be provided by the same memory device or separate memory devices. In various examples, the video data memory may be integrated on a chip with other components of the video encoder 100 or may be disposed outside the chip relative to those components.

[0174] As shown in FIG. 2, the video encoder 100 receives video data and stores the video data in a video data memory. The partition unit partitions the video data into several picture blocks and may further partition the picture blocks into smaller blocks, for example, based on a quadtree structure or a binary tree structure. The partitioning may further include partitioning into slices, tiles, or other larger units. The video encoder 100 is typically a component that encodes picture blocks in a video slice to be encoded. A slice may be partitioned into a number of picture blocks (and into a picture block set called a tile). The prediction processing unit 108 may select one of a number of possible coding modes to be used for the current picture block, for example, one of a number of intra-coding modes or one of a number of inter-coding modes. Prediction processing unit 108 may provide the obtained intra-coded or inter-coded blocks for adder 112 to generate residual blocks, and may provide the obtained intra-coded or inter-coded blocks for adder 111 to reconstruct encoded blocks used as reference pictures.

[0175] An intra predictor 109 in prediction processing unit 108 may perform intra predictive coding on the current block to be coded relative to one or more neighboring blocks in the same frame or slice as the current picture block to remove spatial redundancy. An inter predictor 110 in prediction processing unit 108 may perform inter predictive coding on the current picture block relative to one or more predictive blocks in one or more reference pictures to remove temporal redundancy.

[0176] Specifically, the inter predictor 110 may be configured to determine an inter prediction mode used to encode a current picture block. For example, the inter predictor 110 may calculate rate-distortion values ​​for various inter prediction modes in an inter prediction mode candidate set through rate-distortion analysis, and select an inter prediction mode with optimal rate-distortion characteristics from the inter prediction mode candidate set. The rate-distortion analysis is typically used to determine the amount of distortion (or error) between an encoded block and an original uncoded block from which the encoded block is generated through encoding, and the bit rate (i.e., the amount of bits) used to generate the encoded block. For example, the inter predictor 110 may determine that the inter prediction mode in the inter prediction mode candidate set used to encode the current picture block that has the smallest bit rate-distortion cost is the inter prediction mode used to perform inter prediction on the current picture block.

[0177] The inter predictor 110 is configured to predict motion information (e.g., motion vectors) of one or more sub-blocks of the current picture block based on the determined inter prediction mode, and obtain or generate a predictive block of the current picture block based on the motion information (e.g., motion vectors) of the one or more sub-blocks of the current picture block. The inter predictor 110 may locate the predictive block indicated by the motion vector in one of the reference picture lists. The inter predictor 110 may further generate syntax elements associated with the picture block and the video slice, so that the video decoder 200 uses the syntax elements to decode the picture block in the video slice. Alternatively, in an example, the inter predictor 110 performs a motion compensation process based on the motion information of each sub-block to generate a predictive block of each sub-block to obtain a predictive block of the current picture block. It should be understood that the inter predictor 110 of this specification performs a motion estimation process and a motion compensation process.

[0178] Specifically, after selecting an inter prediction mode for the current picture block, the inter predictor 110 may provide information indicating the inter prediction mode selected for the current picture block to the entropy encoder 103, so that the entropy encoder 103 encodes the information indicating the selected inter prediction mode.

[0179] The intra predictor 109 may perform intra prediction on the current picture block. It is clear that the intra predictor 109 may determine the intra prediction mode used to encode the current block. For example, the intra predictor 109 may calculate rate-distortion values ​​for various test intra prediction modes through rate-distortion analysis, and select the intra prediction mode with the optimal rate-distortion characteristics from the test modes. In any case, after selecting the intra prediction mode for the picture block, the intra predictor 109 may provide information indicating the intra prediction mode selected for the current picture block to the entropy encoder 103, so that the entropy encoder 103 encodes the information indicating the selected intra prediction mode.

[0180] After prediction processing unit 108 generates a predictive block of a current picture block through inter prediction and intra prediction, video encoder 100 obtains a residual picture block by subtracting the predictive block from the current picture block to be encoded. Adder 112 represents one or more components that perform a subtraction operation. The residual video data in the residual block may be included in one or more TUs and is applied to transformer 101. Transformer 101 converts the residual video data into residual transform coefficients through a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform. Transformer 101 may convert the residual video data from a pixel value domain to a transform domain, for example, a frequency domain.

[0181] The transformer 101 may send the obtained transform coefficients to the quantizer 102. The quantizer 102 quantizes the transform coefficients to further reduce the bit rate. In some examples, the quantizer 102 may further scan a matrix including the quantized transform coefficients. Alternatively, the entropy encoder 103 may perform the scan.

[0182] After quantization, the entropy encoder 103 performs entropy coding on the quantized transform coefficients. For example, the entropy encoder 103 may perform context-adaptive variable length coding (CALVC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioned entropy (PIPE) coding, or another entropy coding method or technique. After performing the entropy coding, the entropy encoder 103 may transmit the encoded bitstream to the video decoder 200 or archive the encoded bitstream for later transmission or retrieval by the video decoder 200. The entropy encoder 103 may further perform entropy coding on syntax elements of the current picture block to be coded.

[0183] The inverse quantizer 104 and the inverse transformer 105 perform inverse quantization and inverse transformation, respectively, to reconstruct the residual block in the pixel domain, e.g., for later use as a reference block of a reference picture. The adder 111 adds the reconstructed residual block to a predictive block generated by the inter predictor 110 or the intra predictor 109 to generate a reconstructed picture block. A filter unit 106 can be applied to the reconstructed picture block to reduce distortion, e.g., block artifacts. The reconstructed picture block can then be used as a reference block and stored in the decoded picture buffer 107, and used by the inter predictor 110 as a reference block to perform inter prediction on blocks in a subsequent video frame or picture.

[0184] It should be understood that other structural variations of the video encoder 100 may be used to encode the video stream. For example, for some picture blocks or picture frames, the video encoder 100 may directly quantize the residual signal, and no processing by the transformer 101 and correspondingly no processing by the inverse transformer 105 are required. Alternatively, for some picture blocks or picture frames, the video encoder 100 does not generate residual data, and correspondingly no processing by the transformer 101, the quantizer 102, the inverse quantizer 104, and the inverse transformer 105 are required. Alternatively, the video encoder 100 may directly store the reconstructed picture block as a reference block, and no processing by the filter unit 106 is required. Alternatively, the quantizer 102 and the inverse quantizer 104 in the video encoder 100 may be combined.

[0185] 3 is a block diagram of an example of a video decoder 200 according to an embodiment of the present application. In the example of FIG. 3, the video decoder 200 includes an entropy decoder 203, a prediction processing unit 208, an inverse quantizer 204, an inverse transformer 205, an adder 211, a filter unit 206, and a DPB 207. The prediction processing unit 208 may include an inter predictor 210 and an intra predictor 209. In some examples, the video decoder 200 may perform a decoding process that is generally the inverse of the encoding process described with reference to the video encoder 100 of FIG. 2.

[0186] During decoding, the video decoder 200 receives from the video encoder 100 an encoded video bitstream representing picture blocks in an encoded video slice and associated syntax elements. The video decoder 200 may receive video data from a network entity 42 and may optionally further store the video data in a video data memory (not shown in the figure). The video data memory may store video data, e.g., an encoded video bitstream to be decoded by the components of the video decoder 200. The video data stored in the video data memory may be obtained, e.g., from a local video source, such as the storage device 40, or from a camera through a wired or wireless network communication of the video data, or by accessing a physical data storage medium. The video data memory may be used as a decoded picture buffer (CPB) for storing the encoded video data from the encoded video bitstream. Thus, although the video data memory is not shown in FIG. 3, the video data memory and the DPB 207 may be the same memory or may be memories arranged separately. The video data memory and DPB 207 may each include any one of a number of types of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetic RAM (MRAM), resistive RAM (RRAM), or another type of memory device. In various examples, the video data memory may be integrated on a chip with other components of the video decoder 200 or may be disposed off chip relative to those components.

[0187] Network entity 42 may be, for example, a server, a MANE, a video editor / splitter, or another device configured to implement one or more of the techniques described above. Network entity 42 may or may not include a video encoder, such as video encoder 100. Before network entity 42 transmits the encoded video bitstream to video decoder 200, network entity 42 may implement some of the techniques described in this application. In some video decoding systems, network entity 42 and video decoder 200 may be components of separate devices. In other cases, the functions of network entity 42 may be performed by the same device that includes video decoder 200. In some cases, network entity 42 may be an example of storage device 40 of FIG. 1.

[0188] The entropy decoder 203 of the video decoder 200 performs entropy decoding on the bitstream to generate quantized coefficients and some syntax elements. The entropy decoder 203 forwards the syntax elements to the prediction processing unit 208. The video decoder 200 may receive the syntax elements at the video slice level and / or the picture block level.

[0189] When a video slice is decoded into an intra-decoded (I) slice, an intra predictor 209 in prediction processing unit 208 may generate a predictive block of a picture block in the current video slice based on the signaled intra prediction mode and data of previously decoded blocks of the current frame or picture. When a video slice is decoded into an inter-decoded (i.e., B or P) slice, an inter predictor 210 in prediction processing unit 208 may determine an inter prediction mode used to decode a current picture block in the current video slice based on syntax elements received from entropy decoder 203, and decode (e.g., perform inter prediction on) the current picture block based on the determined inter prediction mode. Specifically, inter predictor 210 may determine whether a new inter prediction mode is used to predict a current picture block in the current video slice. If the syntax element indicates that a new inter prediction mode is used to predict the current picture block, the motion information of the current picture block or the motion information of the sub-block of the current picture block in the current video slice is predicted based on the new inter prediction mode (e.g., the new inter prediction mode specified by using the syntax element or the default new inter prediction mode), and the predictive block of the current picture block or the sub-block of the current picture block is obtained or generated by using the predicted motion information of the current picture block or the sub-block of the current picture block in the motion compensation process. The motion information herein may include reference picture information and motion vectors. The reference picture information may include, but is not limited to, uni-prediction / bi-prediction information, a reference picture list number, and a reference picture index corresponding to the reference picture list. In inter prediction, the predictive block may be generated from one of the reference pictures in one of the reference picture lists.The video decoder 200 may construct reference picture lists, i.e., List 0 and List 1, based on the reference pictures stored in the DPB 207. The reference frame index of the current picture may be included in one or both of Reference Frame List 0 and Reference Frame List 1. In some examples, the video encoder 100 may signal whether a new inter prediction mode is used to decode a particular syntax element of a particular block, or signal whether and which new inter prediction mode is used to decode a particular syntax element of a particular block. It should be understood that the inter predictor 210 herein performs a motion compensation process.

[0190] The inverse quantizer 204 performs inverse quantization, i.e., inverse quantizes, the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 203. The inverse quantization process may involve determining the degree of quantization to be applied by using a quantization parameter calculated by the video encoder 100 for each picture block in a video slice, and in the same manner, determining the degree of inverse quantization to be applied. The inverse transformer 205 performs an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, on the transform coefficients to generate residual blocks in the pixel domain.

[0191] After the inter predictor 210 generates a predictive block for the current picture block or a sub-block of the current picture block, the video decoder 200 adds the residual block from the inverse transformer 205 and the corresponding predictive block generated by the inter predictor 210 to obtain a reconstructed block, i.e., a decoded picture block. The adder 211 represents a component that performs an addition operation. When necessary, a loop filter (either in the decoding loop or after the decoding loop) may be further used to smooth pixel transitions or otherwise improve video quality. The filter unit 206 may represent one or more loop filters, e.g., a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although the filter unit 206 is illustrated as an in-loop filter in FIG. 2, in another implementation, the filter unit 206 may be implemented as a post-loop filter. In an example, the filter unit 206 can be applied to the reconstructed block to reduce blockiness, and the result is output as a decoded video stream. In addition, decoded picture blocks in a given frame or picture may be further stored in the DPB 207, which stores reference pictures used for subsequent motion compensation. The DPB 207 may be part of a memory and may further store decoded video for later presentation on a display device (e.g., the display device 220 of FIG. 1). Alternatively, the DPB 207 may be separate from such memory.

[0192] It should be understood that other structural variations of the video decoder 200 may be used to decode the encoded video bitstream. For example, the video decoder 200 may generate an output video stream, and no processing by the filter unit 206 is required. Alternatively, for some picture blocks or picture frames, the entropy decoder 203 in the video decoder 200 does not obtain quantized coefficients through decoding, and correspondingly, no processing by the inverse quantizer 204 and the inverse transformer 205 is required.

[0193] As described above, the technology of the present application relates to, for example, inter-decoding. It should be understood that the technology of the present application may be performed by any one of the video codecs described in the present application, and the video decoder includes (for example) the video encoder 100 and the video decoder 200 shown and described in FIG. 1 to FIG. 3. Specifically, in one possible implementation, the inter predictor 110 described in FIG. 2 may perform certain techniques described below when inter prediction is performed during encoding of a block of video data. In another possible implementation, the inter predictor 210 described in FIG. 3 may perform certain techniques described below when inter prediction is performed during decoding of a block of video data. Thus, reference to a "video encoder" or a "video decoder" may include the video encoder 100, the video decoder 200, or another video encoding or decoding unit.

[0194] It should be understood that in the encoder 100 and the decoder 200 of the present application, the processing result of a step may be further processed and then output to the next step. For example, after a step such as interpolation filtering, motion vector derivation, or loop filtering, further operations such as clipping or shifting are performed on the processing result of the corresponding step.

[0195] For example, the value of a motion vector may be constrained to be within a particular bit depth range. Assuming the allowed bit depth of a motion vector is bitDepth, the value of the motion vector ranges from -2^(bitDepth-1) to 2^(bitDepth-1)-1, where the symbol "^" represents exponentiation. If bitDepth is 16, the value ranges from -32768 to 32767. If bitDepth is 18, the value ranges from -131072 to 131071. The value of the motion vector may be constrained in one of two ways:

[0196] Method 1: The overflow most significant bits of the motion vectors are removed. ux=(vx+2 bitDepth )%2 bitDepth vx = (ux ≧ 2 bitDepth-1 )?(ux-2 bitDepth ):ux uy=(vy+2 bitDepth )%2 bitDepth vy=(uy≧2 bitDepth-1 )?(uy-2 bitDepth ):uy

[0197] For example, suppose the value of vx is -32769, and 32767 is derived according to the formula above. The values ​​are stored in the computer as two's complement representation, and the two's complement representation of -32769 is 1, 0111, 1111, 1111, 1111 (17 bits), and the process performed by the computer due to overflow is to discard the most significant bits. Thus, the values ​​of vx are 0111, 1111, 1111, 1111, or 32767. This value is consistent with the result derived through processing according to the formula.

[0198] Method 2: Clipping is performed on the motion vectors, using the following formula: vx=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vx) vy=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vy)

[0199] In the above formula, Clip3 is defined as clipping the value of z to the range [x,y].

[0200]

number

[0201] 4 is a schematic block diagram of an inter prediction module 121 according to an embodiment of the present application. For example, the inter prediction module 121 may include a motion estimation unit and a motion compensation unit. The relationship between PUs and CUs varies with video compression coding standards. The inter prediction module 121 may partition the current CU into PUs based on multiple partition modes. For example, the inter prediction module 121 may partition the current CU into PUs based on 2N×2N, 2N×N, N×2N, and N×N partition modes. In another embodiment, the current CU is the current PU. This is not limited.

[0202] The inter prediction module 121 may perform integer motion estimation (IME) and then fractional motion estimation (FME) for each PU. When the inter prediction module 121 performs IME on a PU, the inter prediction module 121 may search one or more reference pictures for a reference block of the PU. After finding a reference block of the PU, the inter prediction module 121 may generate a motion vector that indicates a spatial offset between the PU and the reference block of the PU in integer precision. When the inter prediction module 121 performs FME on a PU, the inter prediction module 121 may improve the motion vector generated by performing IME on the PU. The motion vector generated by performing FME on the PU may have sub-integer precision (e.g., 1 / 2 sample precision or 1 / 4 sample precision). After generating a motion vector for the PU, the inter prediction module 121 may generate a predicted picture block of the PU by using the motion vector of the PU.

[0203] In some possible implementations in which the inter prediction module 121 signals the motion information of the PU to the decoder side based on the AMVP mode, the inter prediction module 121 may generate a predicted motion vector candidate list for the PU. The predicted motion vector candidate list may include one or more original predicted motion vector candidates and one or more additional predicted motion vector candidates derived from the one or more original predicted motion vector candidates. After generating the predicted motion vector candidate list for the PU, the inter prediction module 121 may select a predicted motion vector candidate from the predicted motion vector candidate list and generate a motion vector differential (MVD) for the PU. The MVD for the PU may indicate the difference between the motion vector indicated by the selected predicted motion vector candidate and the motion vector generated for the PU through the IME and FME. In these possible implementations, the inter prediction module 121 may output a predicted motion vector candidate index used to locate the selected predicted motion vector candidate in the predicted motion vector candidate list. The inter prediction module 121 may further output the MVD for the PU. The following describes in detail a possible implementation of the Advanced Motion Vector Prediction (AMVP) mode of FIG. 6 in this embodiment of the present application.

[0204] In addition to performing IME and FME on a PU to generate motion information of the PU, inter prediction module 121 may further perform a merge operation on each PU. When inter prediction module 121 performs a merge operation on a PU, inter prediction module 121 may generate a predictive motion vector candidate list for the PU. The predictive motion vector candidate list for the PU may include one or more original predictive motion vector candidates and one or more additional predictive motion vector candidates derived from the one or more original predictive motion vector candidates. The original predictive motion vector candidates in the predictive motion vector candidate list may include one or more spatial predictive motion vector candidates and temporal predictive motion vector candidates. The spatial predictive motion vector candidates may indicate motion information of another PU of the current picture. The temporal predictive motion vector candidates may be based on motion information of a corresponding PU of a picture different from the current picture. The temporal predictive motion vector candidates may also be referred to as temporal motion vector prediction (TMVP).

[0205] After generating the predicted motion vector candidate list, the inter prediction module 121 may select one predicted motion vector candidate from the predicted motion vector candidate list. Then, the inter prediction module 121 may generate a predicted picture block of the PU based on the reference block indicated by the motion information of the PU. In the merge mode, the motion information of the PU may be the same as the motion information indicated by the selected predicted motion vector candidate. Figure 5 described below is a flowchart of an example of the merge mode.

[0206] After generating the predicted picture block of the PU through IME and FME and generating the predicted picture block of the PU through a merge operation, the inter prediction module 121 may select the predicted picture block generated through the FME operation or the predicted picture block generated through the merge operation. In some possible implementations, the inter prediction module 121 may select the predicted picture block of the PU by analyzing the rate-distortion cost of the predicted picture block generated through the FME operation and the predicted picture block generated through the merge operation.

[0207] After the inter prediction module 121 selects the predicted picture blocks of the PU generated by partitioning the current CU based on each partition mode (in some implementations, after the coding tree unit CTU is partitioned into CUs, the CUs are not further partitioned into smaller PUs, in which case the PUs are equivalent to CUs), the inter prediction module 121 may select a partition mode for the current CU. In some implementations, the inter prediction module 121 may select the partition mode for the current CU by analyzing the rate-distortion cost of the selected predicted picture blocks of the PU generated by partitioning the current CU based on each partition mode. The inter prediction module 121 may output the predicted picture blocks associated with the PUs belonging to the selected partition mode to the residual generation module 102. The inter prediction module 121 may output a syntax element indicating motion information of the PUs belonging to the selected partition mode to the entropy coding module.

[0208] In the schematic diagram of FIG. 4, inter-prediction module 121 includes IME modules 180A to 180N (collectively referred to as "IME modules 180"), FME modules 182A to 182N (collectively referred to as "FME modules 182"), merge modules 184A to 184N (collectively referred to as "merge modules 184"), PU mode decision-making modules 186A to 186N (collectively referred to as "PU mode decision-making modules 186"), and CU mode decision-making module 188 (which may further perform the CTU to CU mode decision-making process).

[0209] The IME module 180, the FME module 182, and the merge module 184 may perform an IME operation, an FME operation, and a merge operation, respectively, on the PUs of the current CU. In the schematic diagram of FIG. 4, the inter prediction module 121 is illustrated as including a separate IME module 180, a separate FME module 182, and a separate merge module 184 for each PU in each partitioning mode for the CU. In another possible implementation, the inter prediction module 121 does not include a separate IME module 180, a separate FME module 182, or a separate merge module 184 for each PU in each partitioning mode for the CU.

[0210] 4, IME module 180A, FME module 182A, and merge module 184A may perform IME, FME, and merge operations, respectively, on PUs generated by partitioning CUs based on a 2N×2N partitioning mode. PU mode decision-making module 186A may select one of the predictive picture blocks generated by IME module 180A, FME module 182A, and merge module 184A.

[0211] IME module 180B, FME module 182B, and merge module 184B may perform IME, FME, and merge operations, respectively, on the left PU generated by partitioning the CU based on the N×2N partition mode. PU mode decision-making module 186B may select one of the predicted picture blocks generated by IME module 180B, FME module 182B, and merge module 184B.

[0212] The IME module 180C, the FME module 182C, and the merge module 184C may perform an IME operation, an FME operation, and a merge operation, respectively, on the right PU generated by partitioning the CU based on the N×2N partition mode. The PU mode decision-making module 186C may select one of the predicted picture blocks generated by the IME module 180C, the FME module 182C, and the merge module 184C.

[0213] The IME module 180N, the FME module 182N, and the merge module 184N may perform an IME operation, an FME operation, and a merge operation, respectively, on the bottom-right PU generated by partitioning the CU based on the N×N partition mode. The PU mode decision-making module 186N may select one of the predicted picture blocks generated by the IME module 180N, the FME module 182N, and the merge module 184N.

[0214] The PU mode decision module 186 may select a predicted picture block by analyzing the rate-distortion costs of multiple possible predicted picture blocks, and select a predicted picture block that results in an optimal rate-distortion cost in a given decoding scenario. For example, in an application with limited bandwidth, the PU mode decision module 186 may prefer to select a predicted picture block with an increased compression ratio, and in another application, the PU mode decision module 186 may prefer to select a predicted picture block with an improved quality of the reconstructed video. After the PU mode decision module 186 selects a predicted picture block for a PU in the current CU, the CU mode decision module 188 selects a partition mode for the current CU, and outputs a predicted picture block and motion information of the PU that belongs to the selected partition mode.

[0215] 5 is a flowchart of an implementation of a merge mode according to an embodiment of the present application. A video encoder (e.g., video encoder 100) may perform a merge operation 201. The merge operation 201 may include: S202: generating a candidate list for a current prediction unit; S204: generating a predicted video block associated with a candidate in the candidate list; S206: selecting a candidate from the candidate list; and S208: outputting the candidate. The candidate is a motion vector candidate or a motion information candidate.

[0216] In another possible implementation, the video encoder may perform a merging operation different from the merging operation 201. For example, in another possible implementation, the video encoder may perform a merging operation, in which the video encoder performs more or fewer steps than the steps of the merging operation 201, or steps different from the steps of the merging operation 201. In another possible implementation, the video encoder may perform the steps of the merging operation 201 in a different order or in parallel. The encoder may further perform the merging operation 201 for a PU that is encoded based on a skip mode.

[0217] After the video encoder starts the merging operation 201, the video encoder may generate a motion vector predictor candidate list for the current PU (S202). The video encoder may generate the motion vector predictor candidate list for the current PU in various manners. For example, the video encoder may generate the motion vector predictor candidate list for the current PU by using one of the example techniques described below with reference to Figures 8 to 12.

[0218] As described above, the predictor motion vector candidate list for the current PU may include a temporal predictor motion vector candidate. The temporal predictor motion vector candidate may indicate motion information of a corresponding temporal (co-located) PU. A co-located PU may be spatially located at the same position as the current PU in a picture frame, but in a reference picture other than the current picture. In this application, a reference picture including a corresponding temporal PU may be referred to as an associated reference picture. In this application, a reference picture index of an associated reference picture may be referred to as an associated reference picture index. As described above, the current picture may be associated with one or more reference picture lists (e.g., list 0 and list 1). A reference picture index may indicate a reference picture by indicating a position of the reference picture in the reference picture list. In some possible implementations, the current picture may be associated with a combined reference picture list.

[0219] In some video encoders, the associated reference picture index is the reference picture index of the PU that contains the reference index source position associated with the current PU. In these video encoders, the reference index source position associated with the current PU is adjacent to the current PU, to the left of the current PU or above the current PU. In this application, a PU may "contain" a particular position if the picture block associated with the PU contains the particular position. In these video encoders, the video encoder may use reference picture index 0 if the reference index source position is unavailable.

[0220] However, in some examples, the reference index source position associated with the current PU is within the current CU. In these examples, the PU containing the reference index source position associated with the current PU may be considered available if the PU is above or to the left of the current CU. In this case, the video encoder may need to access motion information of another PU of the current CU to determine a reference picture including a co-located PU. Thus, these video encoders may use the motion information (i.e., reference picture index) of the PU belonging to the current CU to generate a temporal predictor motion vector candidate for the current PU. In other words, these video encoders may use the motion information of the PU belonging to the current CU to generate a temporal predictor motion vector candidate. Thus, the video encoder may not be able to generate in parallel a predictor motion vector candidate list for the current PU and a PU containing the reference index source position associated with the current PU.

[0221] According to the technology of the present application, the video encoder may explicitly set the associated reference picture index without referring to the reference picture index of any other PU. In this way, the video encoder can generate a predicted motion vector candidate list for the current PU and another PU of the current CU in parallel. Since the video encoder explicitly sets the associated reference picture index, the associated reference picture index is not based on the motion information of any other PU of the current CU. In some possible implementations in which the video encoder explicitly sets the associated reference picture index, the video encoder may always set the associated reference picture index to a fixed, predefined, and pre-set reference picture index (e.g., 0). In this way, the video encoder can generate a temporal predicted motion vector candidate based on the motion information of the co-located PU in the reference frame indicated by the pre-set reference picture index, and the temporal predicted motion vector candidate may be included in the predicted motion vector candidate list for the current CU.

[0222] In a possible implementation in which the video encoder explicitly sets the associated reference picture index, the video encoder may explicitly signal the associated reference picture index in a syntax structure (e.g., a picture header, a slice header, an APS, or another syntax structure). In this possible implementation, the video encoder may signal the associated reference picture index of each LCU (i.e., CTU), CU, PU, ​​TU, or another type of sub-block to the decoder side. For example, the video encoder may signal that the associated reference picture index of each PU of a CU is equal to "1".

[0223] In some possible implementations, the associated reference picture index may be set implicitly rather than explicitly. In these possible implementations, the video encoder may generate each temporal motion vector predictor candidate in the motion vector predictor candidate list for the PU of the current CU by using motion information of PUs in reference pictures indicated by reference picture indexes of PUs that encompass positions outside the current CU, even if these positions are not strictly adjacent to the current PU.

[0224] After generating the predictive motion vector candidate list for the current PU, the video encoder may generate a predictive picture block associated with the predictive motion vector candidate in the predictive motion vector candidate list (S204). The video encoder may determine motion information of the current PU based on the motion information of the indicated predictive motion vector candidate, and then generate a predictive picture block based on the one or more reference blocks indicated by the motion information of the current PU to generate the predictive picture block associated with the predictive motion vector candidate. Then, the video encoder may select one predictive motion vector candidate from the predictive motion vector candidate list (S206). The video encoder may select the predictive motion vector candidate in various manners. For example, the video encoder may select one predictive motion vector candidate by analyzing the rate-distortion cost of each of the predictive picture blocks associated with the predictive motion vector candidate.

[0225] After selecting the motion vector predictor candidate, the video encoder may output a motion vector predictor candidate index (S208). The motion vector predictor candidate index may indicate a position of the selected motion vector predictor candidate in the motion vector predictor candidate list. In some possible implementations, the motion vector predictor candidate index may be expressed as "merge_idx".

[0226] 6 is a flowchart of an implementation of an evolved motion vector prediction (AMVP) mode according to an embodiment of the present application. A video encoder (e.g., video encoder 100) may perform AMVP operations 210. The AMVP operations 210 may include: S211: generating one or more motion vectors for a current prediction unit; S212: generating a predicted video block for the current prediction unit; S213: generating a candidate list for the current prediction unit; S214: generating a motion vector differential; S215: selecting a candidate from the candidate list; and S216: outputting a reference picture index, a candidate index, and a motion vector differential used for the selected candidate. The candidates are motion vector candidates or motion information candidates.

[0227] After the video encoder starts the AMVP operation 210, the video encoder may generate one or more motion vectors for the current PU (S211). The video encoder may perform integer motion estimation and fractional motion estimation to generate a motion vector for the current PU. As described above, the current picture may be associated with two reference picture lists (list 0 and list 1). If the current PU is unidirectionally predicted, the video encoder may generate a list 0 motion vector or a list 1 motion vector for the current PU. The list 0 motion vector may indicate a spatial offset between a picture block corresponding to the current PU and a reference block in a reference picture in list 0. The list 1 motion vector may indicate a spatial offset between a picture block corresponding to the current PU and a reference block in a reference picture in list 1. If the current PU is bidirectionally predicted, the video encoder may generate a list 0 motion vector and a list 1 motion vector for the current PU.

[0228] After generating the one or more motion vectors for the current PU, the video encoder may generate a predictive picture block for the current PU (S212). The video encoder may generate the predictive picture block for the current PU based on the one or more reference blocks indicated by the one or more motion vectors of the current PU.

[0229] In addition, the video encoder may generate a predictive motion vector candidate list for the current PU (S213). The video encoder may generate the predictive motion vector candidate for the current PU in various manners. For example, the video encoder may generate the predictive motion vector candidate list for the current PU by using one or more of the possible implementations described below with reference to Figures 8 to 12. In some possible implementations, when the video encoder generates the predictive motion vector candidate list in the AMVP operation 210, the predictive motion vector candidate list may be limited to two predictive motion vector candidates. In contrast, when the video encoder generates the predictive motion vector candidate list in a merge operation, the predictive motion vector candidate list may include more predictive motion vector candidates (e.g., five predictive motion vector candidates).

[0230] After generating the motion vector predictor candidate for the current PU, the video encoder may generate one or more motion vector differentials (MVDs) for each motion vector predictor candidate in the motion vector predictor candidate list (S214). To generate a motion vector differential for the motion vector predictor candidate, the video encoder may determine a difference between a motion vector indicated by the motion vector predictor candidate and a corresponding motion vector of the current PU.

[0231] If the current PU is unidirectionally predicted, the video encoder may generate a single MVD for each predicted motion vector candidate. If the current PU is bidirectionally predicted, the video encoder may generate two MVDs for each predicted motion vector candidate. The first MVD may indicate the difference between the motion vector indicated by the predicted motion vector candidate and the list 0 motion vector of the current PU. The second MVD may indicate the difference between the motion vector indicated by the predicted motion vector candidate and the list 1 motion vector of the current PU.

[0232] The video encoder may select one or more motion vector predictor candidates from the motion vector predictor candidate list (S215). The video encoder may select one or more motion vector predictor candidates in various manners. For example, the video encoder may select a motion vector predictor candidate that matches with a minimal error a motion vector associated with the motion vector to be encoded. This can reduce the amount of bits required to represent a motion vector differential for the motion vector predictor candidate.

[0233] After selecting one or more predictor motion vector candidates, the video encoder may output one or more reference picture indexes for the current PU, one or more predictor motion vector candidate indexes for the current PU, and one or more motion vector differentials for the one or more selected predictor motion vector candidates (S216).

[0234] In an example in which the current picture is associated with two reference picture lists (list 0 and list 1) and the current PU is unidirectionally predicted, the video encoder may output a reference picture index for list 0 ("ref_idx_10") or a reference picture index for list 1 ("ref_idx_11"). The video encoder may further output a predictor motion vector candidate index ("mvp_10_flag") indicating a position of the selected predictor motion vector candidate relative to the list 0 motion vector of the current PU in the predictor motion vector candidate list. Alternatively, the video encoder may output a predictor motion vector candidate index ("mvp_11_flag") indicating a position of the selected predictor motion vector candidate relative to the list 1 motion vector of the current PU in the predictor motion vector candidate list. The video encoder may further output an MVD for the list 0 motion vector or list 1 motion vector of the current PU.

[0235] In an example in which the current picture is associated with two reference picture lists (list 0 and list 1) and the current PU is bidirectionally predicted, the video encoder may output a reference picture index for list 0 ("ref_idx_10") and a reference picture index for list 1 ("ref_idx_11"). The video encoder may further output a predictor motion vector candidate index ("mvp_10_flag") indicating a position of the selected predictor motion vector candidate relative to the list 0 motion vector of the current PU in the predictor motion vector candidate list. In addition, the video encoder may output a predictor motion vector candidate index ("mvp_11_flag") indicating a position of the selected predictor motion vector candidate relative to the list 1 motion vector of the current PU in the predictor motion vector candidate list. The video encoder may further output an MVD for the list 0 motion vector of the current PU and an MVD for the list 1 motion vector of the current PU.

[0236] FIG. 7 is a flowchart of an implementation of motion compensation performed by a video decoder (eg, video decoder 30) according to an embodiment of the present application.

[0237] When the video decoder performs the motion compensation operation 220, the video decoder may receive an indication to be used for a selected predicted motion vector candidate of the current PU (S222). For example, the video decoder may receive a predicted motion vector candidate index indicating a position of the selected predicted motion vector candidate in a predicted motion vector candidate list for the current PU.

[0238] When the motion information of the current PU is encoded based on the AMVP mode and the current PU is bidirectionally predicted, the video decoder may receive a first predicted motion vector candidate index and a second predicted motion vector candidate index. The first predicted motion vector candidate index indicates a position of the selected predicted motion vector candidate relative to the list 0 motion vector of the current PU in the predicted motion vector candidate list. The second predicted motion vector candidate index indicates a position of the selected predicted motion vector candidate relative to the list 1 motion vector of the current PU in the predicted motion vector candidate list. In some possible implementations, a single syntax element may be used to identify the two predicted motion vector candidate indexes.

[0239] In addition, the video decoder may generate a predictor motion vector candidate list for the current PU (S224). The video decoder may generate the predictor motion vector candidate list for the current PU in various manners. For example, the video decoder may generate the predictor motion vector candidate list for the current PU by using the techniques described below with reference to Figures 8 to 12. When the video decoder generates a temporal predictor motion vector candidate for the predictor motion vector candidate list, the video decoder may explicitly or implicitly set a reference picture index that identifies a reference picture that includes a co-located PU, as described above with reference to Figure 5.

[0240] After generating the predictive motion vector candidate list for the current PU, the video decoder may determine motion information of the current PU based on motion information indicated by one or more selected predictive motion vector candidates in the predictive motion vector candidate list for the current PU (S225). For example, if the motion information of the current PU is encoded based on a merge mode, the motion information of the current PU may be the same as the motion information indicated by the selected predictive motion vector candidate. If the motion information of the current PU is encoded based on an AMVP mode, the video decoder may reconstruct one or more motion vectors of the current PU by using one or more motion vectors indicated by the one or more selected predictive motion vector candidates and one or more MVDs indicated in the bitstream. The reference picture index and prediction direction indicator of the current PU may be the same as one or more reference picture indexes and prediction direction indicators of the one or more selected predictive motion vector candidates. After determining the motion information of the current PU, the video decoder may generate a predictive picture block for the current PU based on one or more reference blocks indicated by the motion information of the current PU (S226).

[0241] FIG. 8 is a schematic diagram of an example of a coding unit (CU) and neighboring picture blocks associated with the coding unit (CU) according to an embodiment of the present application. FIG. 8 is a schematic diagram for describing a CU 250 and exemplary predicted motion vector candidates at positions 252A to 252E associated with the CU 250. In the present application, the predicted motion vector candidates at positions 252A to 252E may be collectively referred to as the predicted motion vector candidates at position 252. The predicted motion vector candidates at position 252 represent spatial predicted motion vector candidates in the same picture as the CU 250. The predicted motion vector candidate at position 252A is located to the left of the CU 250. The predicted motion vector candidate at position 252B is located above the CU 250. The predicted motion vector candidate at position 252C is located to the top right of the CU 250. The predicted motion vector candidate at position 252D is located to the bottom left of the CU 250. The predicted motion vector candidate at position 252E is located to the top left of the CU 250. 8 shows a schematic implementation of the manner in which the inter prediction module 121 and the motion compensation module can generate a predicted motion vector candidate list. In the following, the implementation is described based on the inter prediction module 121. However, it should be understood that the motion compensation module can implement the same technique and therefore generate the same predicted motion vector candidate list.

[0242] FIG. 9 is a flowchart of an implementation of constructing a motion vector predictor candidate list according to an embodiment of the present application. Although the technique of FIG. 9 is described based on a list including five motion vector predictor candidates, the technique described herein may alternatively be used with a list having another size. The five motion vector predictor candidates may each have an index (e.g., 0 to 4). The technique of FIG. 9 is described based on a general video codec. The general video codec may be, for example, a video encoder (e.g., video encoder 100) or a video decoder (e.g., video decoder 30).

[0243] To build a predictor motion vector candidate list according to the implementation of FIG. 9, the video decoder first considers four spatial predictor motion vector candidates (902). The four spatial predictor motion vector candidates may include predictor motion vector candidates at positions 252A, 252B, 252C, and 252D. The four spatial predictor motion vector candidates may correspond to motion information of four PUs located in the same picture as the current CU (e.g., CU 250). The video decoder may consider the four spatial predictor motion vector candidates in the list in a specified order. For example, the predictor motion vector candidate at position 252A may be considered first. If the predictor motion vector candidate at position 252A is available, index 0 may be assigned to the predictor motion vector candidate at position 252A. If the predictor motion vector candidate at position 252A is unavailable, the video decoder may skip adding the predictor motion vector candidate at position 252A to the predictor motion vector candidate list. The predictor motion vector candidate at position may be unavailable for various reasons. For example, if the motion vector predictor candidate at the position is not in the current picture, the motion vector predictor candidate at the position may be unavailable. In another possible implementation, if the motion vector predictor candidate at the position undergoes intra prediction, the motion vector predictor candidate at the position may be unavailable. In another possible implementation, if the motion vector predictor candidate at the position is in a slice different from the slice corresponding to the current CU, the motion vector predictor candidate at the position may be unavailable.

[0244] After considering the motion vector predictor candidate at position 252A, the video decoder may consider the motion vector predictor candidate at position 252B. If the motion vector predictor candidate at position 252B is available and different from the motion vector predictor candidate at position 252A, the video decoder may add the motion vector predictor candidate at position 252B to the motion vector predictor candidate list. In this particular context, the term "same" or "different" means that the motion information associated with the motion vector predictor candidate at a position is the same or different. Thus, if the two motion vector predictor candidates at a position have the same motion information, the two motion vector predictor candidates at a position are considered to be the same, or if the two motion vector predictor candidates at a position have different motion information, the two motion vector predictor candidates at a position are considered to be different. If the motion vector predictor candidate at position 252A is unavailable, the video decoder may assign index 0 to the motion vector predictor candidate at position 252B. If the motion vector predictor candidate at position 252A is available, the video decoder may assign index 1 to the motion vector predictor candidate at position 252B. If the motion vector predictor candidate at location 252B is unavailable or is the same as the motion vector predictor candidate at location 252A, the video decoder skips adding the motion vector predictor candidate at location 252B to the motion vector predictor candidate list.

[0245] Similarly, the video decoder considers the motion vector predictor candidate at position 252C to determine whether to add the motion vector predictor candidate at position 252C to the list. If the motion vector predictor candidate at position 252C is available and different from the motion vector predictor candidates at positions 252B and 252A, the video decoder may assign the next available index to the motion vector predictor candidate at position 252C. If the motion vector predictor candidate at position 252C is unavailable or is the same as at least one of the motion vector predictor candidates at positions 252A and 252B, the video decoder skips adding the motion vector predictor candidate at position 252C to the motion vector predictor candidate list. Next, the video decoder considers the motion vector predictor candidate at position 252D. If the motion vector predictor candidate at position 252D is available and different from the motion vector predictor candidates at positions 252A, 252B, and 252C, the video decoder may assign the next available index to the motion vector predictor candidate at position 252D. If the motion vector predictor candidate at position 252D is unavailable or is the same as at least one of the motion vector predictor candidates at positions 252A, 252B, and 252C, the video decoder skips adding the motion vector predictor candidate at position 252D to the motion vector predictor candidate list. In the above implementation, an example is generally described in which the motion vector predictor candidates 252A to 252D are considered to determine whether to add the motion vector predictor candidates 252A to 252D in the motion vector predictor candidate list. However, in some implementations, all the motion vector predictor candidates 252A to 252D may be added to the motion vector predictor candidate list first, and then the repeated motion vector predictor candidates are removed from the motion vector predictor candidate list.

[0246] After the video decoder considers the first four spatial predictive motion vector candidates, the predictive motion vector candidate list may include four spatial predictive motion vector candidates, or the list may include less than four spatial predictive motion vector candidates. If the list includes four spatial predictive motion vector candidates (904, yes), the video decoder considers a temporal predictive motion vector candidate (906). The temporal predictive motion vector candidate may correspond to motion information of a co-located PU of a picture different from the current picture. If the temporal predictive motion vector candidate is available and different from the first four spatial predictive motion vector candidates, the video decoder assigns index 4 to the temporal predictive motion vector candidate. If the temporal predictive motion vector candidate is unavailable or is the same as one of the first four spatial predictive motion vector candidates, the video decoder skips adding the temporal predictive motion vector candidate to the predictive motion vector candidate list. Thus, after the video decoder considers the temporal motion vector predictor candidates (906), the motion vector predictor candidate list may include five motion vector predictor candidates (the first four spatial motion vector predictor candidates considered in 902 and the temporal motion vector predictor candidate considered in 906) or may include four motion vector predictor candidates (the first four spatial motion vector predictor candidates considered in 902). If the motion vector predictor candidate list includes five motion vector predictor candidates (908, yes), the video decoder completes constructing the list.

[0247] If the predictor motion vector candidate list includes four predictor motion vector candidates (908, no), the video decoder may consider a fifth spatial predictor motion vector candidate (910). The fifth spatial predictor motion vector candidate may (for example) correspond to the predictor motion vector candidate at position 252E. If the predictor motion vector candidate at position 252E is available and different from the predictor motion vector candidates at positions 252A, 252B, 252C, and 252D, the video decoder may add the fifth spatial predictor motion vector candidate to the predictor motion vector candidate list and assign index 4 to the fifth spatial predictor motion vector candidate. If the predictor motion vector candidate at position 252E is unavailable or is the same as one of the predictor motion vector candidates at positions 252A, 252B, 252C, and 252D, the video decoder may skip adding the predictor motion vector candidate at position 252E to the predictor motion vector candidate list. Thus, after the fifth spatial predictor motion vector candidate is considered (910), the list may include five predictor motion vector candidates (the first four spatial predictor motion vector candidates considered in 902 and the fifth spatial predictor motion vector candidate considered in 910) or may include four predictor motion vector candidates (the first four spatial predictor motion vector candidates considered in 902).

[0248] If the motion vector predictor candidate list includes five motion vector predictor candidates (912, yes), the video decoder completes the generation of the motion vector predictor candidate list. If the motion vector predictor candidate list includes four motion vector predictor candidates (912, no), the video decoder adds artificially generated motion vector predictor candidates (914) until the list includes five motion vector predictor candidates (916, yes).

[0249] If the list includes less than four spatial predictive motion vector candidates (904, no), the video decoder may consider a fifth spatial predictive motion vector candidate after the video decoder considers the first four spatial predictive motion vector candidates (918). The fifth spatial predictive motion vector candidate may (for example) correspond to the predictive motion vector candidate at position 252E. If the predictive motion vector candidate at position 252E is available and different from the existing predictive motion vector candidates in the predictive motion vector candidate list, the video decoder may add the fifth spatial predictive motion vector candidate to the predictive motion vector candidate list and assign the next available index to the fifth spatial predictive motion vector candidate. If the predictive motion vector candidate at position 252E is unavailable or is the same as one of the existing predictive motion vector candidates in the predictive motion vector candidate list, the video decoder may skip adding the predictive motion vector candidate at position 252E to the predictive motion vector candidate list. The video decoder may then consider a temporal predictive motion vector candidate (920). If the temporal motion vector predictor candidate is available and different from the existing motion vector predictor candidates in the motion vector predictor candidate list, the video decoder may add the temporal motion vector predictor candidate to the motion vector predictor candidate list and assign the next available index to the temporal motion vector predictor candidate. If the temporal motion vector predictor candidate is unavailable or is the same as one of the existing motion vector predictor candidates in the motion vector predictor candidate list, the video decoder may skip adding the temporal motion vector predictor candidate to the motion vector predictor candidate list.

[0250] If the motion vector predictor candidate list includes five motion vector predictor candidates (922, yes) after the fifth spatial motion vector predictor candidate (at 918) and the temporal motion vector predictor candidate (at 920) are considered, the video decoder completes the generation of the motion vector predictor candidate list. If the motion vector predictor candidate list includes fewer than five motion vector predictor candidates (922, no), the video decoder adds artificially generated motion vector predictor candidates (914) until the list includes five motion vector predictor candidates (916, yes).

[0251] According to the technology of the present application, an additional merge prediction motion vector candidate may be artificially generated after the spatial prediction motion vector candidate and the temporal prediction motion vector candidate, so that the size of the merge prediction motion vector candidate is constant, and the merge prediction motion vector candidate list includes a specified amount (e.g., five in the above-mentioned possible implementation of FIG. 9) of merge prediction motion vector candidates. The additional merge prediction motion vector candidates may include examples of a combined bi-predictive merge prediction motion vector candidate (predictive motion vector candidate 1), a scaled bi-predictive merge prediction motion vector candidate (predictive motion vector candidate 2), and a zero vector merge / AMVP predictive motion vector candidate (predictive motion vector candidate 3).

[0252] Figure 10 is a schematic diagram of an example of adding a combined motion vector candidate to a merge mode predicted motion vector candidate list according to an embodiment of the present application. A combined bi-predictive merge predicted motion vector candidate can be generated by combining original merge predicted motion vector candidates. Specifically, two original predicted motion vector candidates (which may have mvL0 and refIdxL0 or mvL1 and refIdxL1) can be used to generate a bi-predictive merge predicted motion vector candidate. In Figure 10, two predicted motion vector candidates are included in the original merge predicted motion vector candidate list. The prediction type of one predicted motion vector candidate is uni-prediction by using list 0, and the prediction type of the other predicted motion vector candidate is uni-prediction by using list 1. In this possible implementation, mvL0_A and ref0 are obtained from list 0, and mvL1_B and ref0 are obtained from list 1. Then, a bi-predictive merge prediction motion vector candidate (having mvL0_A and ref0 in list 0 and mvL1_B and ref0 in list 1) may be generated, and it is checked whether the bi-predictive merge prediction motion vector candidate is different from an existing predictive motion vector candidate in the predictive motion vector candidate list. If the bi-predictive merge prediction motion vector candidate is different from an existing predictive motion vector candidate, the video decoder may add the bi-predictive merge prediction motion vector candidate to the predictive motion vector candidate list.

[0253] FIG. 11 is a schematic diagram of an example of adding a scaled motion vector candidate to a merge mode predicted motion vector candidate list according to an embodiment of the present application. A scaled bi-predictive merge predicted motion vector candidate may be generated by scaling an original merge predicted motion vector candidate. Specifically, one original predicted motion vector candidate (having mvLX and refIdxLX) may be used to generate a bi-predictive merge predicted motion vector candidate. In a possible implementation of FIG. 11, two predicted motion vector candidates are included in the original merge predicted motion vector candidate list. The prediction type of one predicted motion vector candidate is uni-prediction by using list 0, and the prediction type of the other predicted motion vector candidate is uni-prediction by using list 1. In this possible implementation, mvL0_A and ref0 may be obtained from list 0, and ref0 may be copied to list 1, and is denoted as a reference index ref0'. Then, mvL0'_A may be calculated by scaling mvL0_A with ref0 and ref0'. The scaling may depend on a POC (Picture Order Count) distance. Then, a bi-predictive merge prediction motion vector candidate (having mvL0_A and ref0 in list 0 and ref0′ in list 1) may be generated, and it is checked whether the bi-predictive merge prediction motion vector candidate is repeated. If the bi-predictive merge prediction motion vector candidate is not repeated, the bi-predictive merge prediction motion vector candidate may be added to the merge prediction motion vector candidate list.

[0254] 12 is a schematic diagram of an example of adding a zero motion vector to a merge mode predicted motion vector candidate list according to an embodiment of the present application. A zero vector merge predicted motion vector candidate may be generated by combining a zero vector and a reference index that may be referenced. If a zero vector merge predicted motion vector candidate is not repeated, the zero vector merge predicted motion vector candidate may be added to the merge predicted motion vector candidate list. The motion information of each generated merge predicted motion vector candidate may be compared with the motion information of the previous predicted motion vector candidate in the list.

[0255] In one possible implementation, if the newly generated predictive motion vector candidate is different from the existing predictive motion vector candidate in the predictive motion vector candidate list, the generated predictive motion vector candidate is added to the merged predictive motion vector candidate list. The process of determining whether the predictive motion vector candidate is different from the existing predictive motion vector candidate in the predictive motion vector candidate list may be called pruning. Through pruning, each newly generated predictive motion vector candidate may be compared with the existing predictive motion vector candidates in the list. In some possible implementations, the pruning operation may include comparing one or more new predictive motion vector candidates with the existing predictive motion vector candidates in the predictive motion vector candidate list, and skipping adding the new predictive motion vector candidate that is the same as the existing predictive motion vector candidate in the predictive motion vector candidate list. In some other possible implementations, the pruning operation may include adding one or more new predictive motion vector candidates to the predictive motion vector candidate list, and then removing the repeated predictive motion vector candidate from the list.

[0256] The following describes some inter-prediction implementations. The first preset algorithm and the second preset algorithm in this application may include one or more of the inter-prediction implementations.

[0257] Inter-picture prediction uses temporal correlation between pictures to derive a motion-compensated prediction (MCP) for a block of picture samples.

[0258] In this block-based MCP, a video picture is divided into rectangular blocks. Assuming that homogeneous motion occurs inside a block and the moving object is larger than a block, for each block, a corresponding block in a previously decoded picture can be found, which serves as a predictor. With a translational motion model, the position of a block in a previously decoded picture is indicated by a motion vector (Δx, Δy), where Δx specifies the horizontal displacement relative to the position of the current block, and Δy specifies the vertical displacement relative to the position of the current block. The motion vector (Δx, Δy) may have fractional sample accuracy to capture the motion of the object behind more accurately. When the corresponding motion vector has fractional sample accuracy, an interpolation is applied to the reference picture to derive a prediction signal. The previously decoded picture is called a reference picture and is indicated by a reference index Δt corresponding to the reference picture list. These translational motion model parameters, i.e., the motion vector and the reference index, are further called motion data. Two types of inter-picture prediction are allowed in modern video coding standards: uni-prediction and bi-prediction.

[0259] In the case of bi-prediction, two sets of motion data (Δx0, Δy0, Δt0, and Δx1, Δy1, Δt1) are used to generate two MCPs (possibly from different pictures), which are then combined to obtain the final MCP. By default, this is done by averaging, but in the case of weighted prediction, different weights may be applied to each MCP, for example to compensate for scene fade-out. Reference pictures that may be used in bi-prediction are stored in two separate lists, namely list 0 and list 1. To constrain memory bandwidth in slices that allow bi-prediction, the HEVC standard constrains PUs with 4×8 or 8×4 luma prediction blocks to use only uni-prediction. The motion data is derived in the encoder by using a motion compensation process. Since motion estimation is not specified within the video standard, different encoders may utilize different complexity and quality trade-offs during the encoder implementation.

[0260] The motion data of a block is correlated with neighboring blocks. To exploit this correlation, the motion data is not directly coded in the bitstream, but is predictively coded based on the neighboring motion data. In HEVC, two concepts are used for this purpose. Predictive coding of motion vectors is improved in HEVC by introducing a new tool called advanced motion vector prediction (AMVP), where the best prediction value for each motion block is signaled to the decoder. In addition, a new technique called inter-prediction block merging is used to derive all the motion data of a block from neighboring blocks. In this way, the direct mode and skip mode in H.264 / AVC are replaced.

[0261] Evolutionary motion vector prediction

[0262] As described in previous video coding standards, HEVC motion vectors are coded as a difference to a motion vector prediction value (MVP) based on the horizontal (x) and vertical (y) components. The two motion vector difference (MVD) components are calculated according to equations (1.1) and (1.2). MVD X =Δx-MVP X (1.1) MVD Y =Δy-MVP Y (1.2)

[0263] The motion vector of the current block is usually correlated with the motion vectors of neighboring blocks in the current picture or in earlier coded pictures. This is because the neighboring blocks are more likely to correspond to the same moving object with similar motion, and the object motion is less likely to change suddenly with time. Therefore, using the motion vectors of neighboring blocks as predictors reduces the signaled motion vector differentials. The MVP is usually derived from already decoded motion vectors of spatial or temporal neighboring blocks in co-located pictures. In some cases, alternatively, the zero motion vector can be used as the MVP. In H.264 / AVC, this is implemented by taking a component-wise median of three spatially neighboring motion vectors. Using this approach, no signaling of the predictor is required. The temporal MVP from the co-located picture is only considered in the so-called temporal direct mode in H.264 / AVC. The H.264 / AVC direct mode is also used to derive motion data other than motion vectors.

[0264] In HEVC, the approach for implicitly deriving MVPs has been replaced by a technique known as motion vector compensation, which explicitly signals which MVP from a list of MVPs is used for motion vector derivation. The variable coding quadtree block structure in HEVC allows a block with several neighboring blocks with motion vectors to serve as a potential MVP candidate. The initial design of Advanced Motion Vector Prediction (AMVP) included five MVPs from three different classes of predictors: three motion vectors from spatial neighbors, the median of three spatial predictors, and a scaled motion vector from a collocated temporal neighbor. Furthermore, the list of predictors was modified by reordering to place the most probable motion predictor in the first position and by removing redundant candidates to ensure that signaling overhead is minimized. Significant simplifications to the AMVP design are then developed, such as removing the median predictor, reducing the amount of candidates in the list from five to two, fixing the order of candidates in the list, and reducing the amount of redundancy checking. The final design for AMVP candidate list construction contains the following two MVP candidates: a. up to two spatial MVP candidates derived from five spatial neighboring blocks, b. one temporal MVP candidate derived from two temporally co-located blocks when both spatial MVP candidates are unavailable or identical, and c. a zero motion vector when the spatial candidate, the temporal candidate, or both the spatial and temporal candidates are unavailable.

[0265] As already mentioned, two spatial MVP candidates A and B are derived from five spatial neighboring blocks. The location of the spatial block candidates is the same for both AMVP and inter-prediction block merging. For candidate A, the motion data of two blocks A0 and A1 in the lower left corner are considered in a two-pass approach. In the first pass, it is checked whether any of the candidate blocks contains a reference index equal to the reference index of the current block. The first motion vector found is used as candidate A. When all reference indexes from A0 to A1 point to different reference pictures than the reference index of the current block, the associated motion vector cannot be used as is. Therefore, in the second pass, the motion vector needs to be scaled based on the temporal distance between the reference picture candidate and the current reference picture. Equation (1.3) shows that the motion vector candidate mv cand is scaled based on the scaling factor. ScaleFactor is calculated based on the temporal distance between the current picture and the reference picture of the block candidate td and the temporal distance between the current picture and the reference picture of the current block tb. The temporal distance is expressed in terms of the difference between the picture order count (POC) values ​​that define the display order of the pictures. The scaling operation is essentially the same as the scheme used for the temporal direct mode of H.264 / AVC. This factor decomposition allows pre-computation of ScaleFactor at slice level, since it only depends on the reference picture list structure signaled in the slice header. Note that MV scaling is performed only when both the current reference picture and the reference picture candidate are short-term reference pictures. The parameter td is defined as the POC difference between the co-located picture and the reference picture for the co-located block candidate. mv=sign(mv cand ·ScaleFactor)((|mv cand ScaleFactor|+27)>>8) (1.3) ScaleFactor=clip(-2 12 ,2 12 -1, (tb·tx+2 5 )>>6) (1.4)

[0266]

number

[0267] For candidate B, candidates B0 to B2 are sequentially checked in the same manner that A0 and A1 are checked in the first pass. However, the second pass is only performed when blocks A0 and A1 do not contain any motion information, in other words, when blocks A0 and A1 are unavailable or are coded through intra-picture prediction. Then, if candidate A is found, candidate A is set equal to unscaled candidate B, and candidate B is set equal to a second unscaled or scaled variant of candidate B. In the second pass, a search is performed to derive unscaled and scaled MVs from candidates B0 to B2. Overall, this design allows A0 and A1 to be processed independently of B0, B1, and B2. The derivation of B should only be aware of the availability of both A0 and A1 in order to perform a search to derive scaled MVs or additional unscaled MVs from B0 to B2. This dependency is acceptable given that it greatly reduces the complex motion vector scaling operations for candidate B. Reducing the amount of motion vector scaling significantly reduces the complexity in the motion vector predictor derivation process.

[0268] In HEVC, the blocks at the bottom right and center of the current block were determined to be the most suitable blocks to yield a good temporal motion vector prediction value (TMVP). Among these candidates, C0 represents the bottom right neighboring block, and C1 represents the center block. Again, the motion data of C0 is considered first. If the motion data of C0 is unavailable, the motion data of the center co-located block candidate is used to derive the temporal MVP candidate C. The motion data of C0 is also considered unavailable when the associated PU belongs to a CTU beyond the current CTU row. This minimizes the memory bandwidth requirement for storing the co-located motion data. In contrast to spatial MVP candidates, whose motion vectors may point to the same reference picture, motion vector scaling is mandatory for TMVP. Therefore, the same scaling operation as for spatial MVP is used.

[0269] While the temporal direct mode in H.264 / AVC always references the second reference picture list, i.e. the first reference picture in list 1, and is only allowed in bi-predictive slices, HEVC is able to indicate for each picture which reference pictures are considered to be co-located pictures. This is implemented by signaling the co-located reference picture list and reference picture index in the slice header and by requiring that these syntax elements in all slices in a picture specify the same reference pictures.

[0270] Since the temporal MVP candidates introduce additional dependencies, the use of the temporal MVP candidates may need to be disabled for robustness against errors. In H.264 / AVC, it is possible to disable the temporal direct mode for bi-predictive slices in the slice header (direct_spatial_mv_pred_flag). The HEVC syntax extends this signaling by allowing disabling TMVP at the sequence level or picture level (sps / slice_temporal_mvp_enabled_flag). Although the flag is signaled in the slice header, there is a bitstream conformance requirement that the value of the flag must be the same for all slices in one picture. Since the signaling of the picture level flag depends on the SPS flag, the signaling of the picture level flag in the PPS may introduce parsing dependencies between the SPS and the PPS. Another advantage of this slice header signaling is that only the value of this flag in the PPS is preferably changed without changing another parameter, and there is no need to send a second PPS.

[0271] In general, motion data signaling in HEVC is similar to that in H / 264 / AVC. The inter-picture prediction syntax element inter_pred_idc signals whether reference list 0, reference list 1, or both reference list 0 and reference list 1 are used. For each MCP derived from one reference picture list, the corresponding reference picture (Δt) is signaled by the index ref_idx_l0 / 1 to the reference picture list, and the MV(Δx,Δy) is represented by the index mvp_l0 / 1_flag to the MVP, and the MVD of the MVP. A newly introduced flag mvd_l1_zero_flag in the slice header indicates whether the MVD for the second reference picture list is equal to 0 and therefore not signaled in the bitstream. Once the motion vectors are fully reconstructed, the final clipping operation is performed by multiplying the value of each component of the final motion vector by -2 inclusive. 15 From 2 15 Ensures that it is always within -1.

[0272] Inter-picture prediction Bloch merging

[0273] While the AMVP list only contains motion vectors for one reference list, the merge candidates contain all the motion data, including whether one or two reference picture lists are used, as well as information about the reference index and motion vector for each list. Overall, the merge candidate list is built based on the following candidates: a. up to four spatial merge candidates derived from five spatially neighboring blocks, b. one temporal merge candidate derived from two temporally co-located blocks, and c. an additional merge candidate including a combined bi-predictive candidate and a zero motion vector candidate.

[0274] The first candidate in the merge candidate list is a spatial neighbor. A1, B1, B0, A0, and B2 are checked sequentially and up to four candidates may be inserted into the merge list in this order.

[0275] Rather than just checking whether neighboring blocks are available and contain motion information, some additional redundancy checks are performed before all the motion data of neighboring blocks are used as merging candidates. These redundancy checks can be split into two categories with two different purposes: a. to ensure that no candidates with redundant motion data are in the list, and b. to prevent merging of two partitions that may be expressed by other means that may create redundant syntax.

[0276] When N represents the amount of spatial merge candidates, the complete redundancy check is

number

[0277] The motion vectors of temporal merge candidates are derived in the same manner as deriving TMVP. Since the merge candidate contains all the motion data and TMVP is only one motion vector, the derivation of the entire motion data depends only on the slice type. For bi-predictive slices, TMVP is derived for each reference picture list. Depending on the availability of TMVP for each list, the prediction type is set to bi-predictive or to the list where TMVP is available. All associated reference picture indices are set equal to 0. Therefore, for uni-predictive slices, only TMVP for list 0 is derived with reference picture index equal to 0.

[0278] When at least one TMVP is available and a temporal merge candidate is added to the list, no redundancy check is performed. This makes the construction of the merge list independent of the co-located pictures, thus increasing resilience to errors. Cases are considered where a temporal merge candidate is redundant and therefore not included in the merge candidate list, and when a co-located picture is lost, the decoder does not check whether the temporal candidate is redundant, since it cannot derive a temporal candidate. This affects the indexing of all subsequent candidates.

[0279] For robustness of the analysis, the merge candidate list has a fixed length. Even after spatial and temporal merge candidates are added, the list may not reach that fixed length. To compensate for the loss of coding efficiency associated with non-length-adaptive list index signaling, additional candidates are generated. Depending on the slice type, up to two types of candidates may be used to completely fill the list: a. combined bi-predictive candidates, and b. zero motion vector candidates.

[0280] In a bi-predictive slice, additional candidates may be generated based on existing candidates by combining the reference picture list 0 motion data of one candidate with the reference picture list 1 motion data of another candidate. This is performed by copying Δx0, Δy0, and Δt0 from one candidate, such as the first candidate, and copying Δx1, Δy1, and Δt1 from another candidate, such as the second candidate. Various combinations are predefined and are given in Table 1.1.

[0281] [Table 1]

[0282] When the list is still not filled after the combined bi-predictive candidates are added, or when the list is not filled for the uni-predictive slices, a zero motion vector candidate is calculated to complete the list. Every zero motion vector candidate has one zero disparity motion vector for the uni-predictive slices and two zero disparity motion vectors for the bi-predictive slices. The reference index is set equal to 0 and incremented by 1 for each additional candidate until a maximum amount of reference indexes is reached. In this case, if there are other candidates still missing, the reference index equal to 0 is used to create these candidates. For all additional candidates, a check for redundancy is not performed, since it is known that omitting these checks does not result in a loss of coding efficiency.

[0283] For each PU coded based on an inter-picture prediction mode, merge_flag indicates that block merging is used to derive motion data, and merge_idx is further used to determine which candidates are in the merge list and provide all the required motion data for the MCP. In addition to this PU-level signaling, the amount of candidates in the merge list is signaled in the slice header. Since the default value is 5, it is expressed as a delta to 5 (five_minus_max_num_merge_cand). Thus, 5 is signaled using a short codeword for 0. If only one candidate is used, 5 is signaled using a longer codeword for 4. With regard to the impact on the merge candidate list construction process, the overall process remains the same, but the process ends after the list contains the maximum amount of merge candidates. In the initial design, the maximum value for coding the merge index is given by the amount of available spatial and temporal candidates in the list. For example, when only two candidates are available, the index can be efficiently coded as a flag. However, to analyze the merge index, the entire merge candidate list must be constructed to know the actual amount of candidates. If a neighboring block is unavailable due to a transmission error, the merge index can no longer be analyzed.

[0284] An important application of the block merging concept in HEVC is its combination with skip mode. In previous video coding standards, motion data is inferred rather than explicitly signaled, and skip mode is used to indicate for a block that the prediction residual is zero. In other words, no transform coefficients are transmitted. In HEVC, at the beginning of each CU in an inter-picture predicted slice, a skip_flag is signaled, which indicates that a. the CU contains only one PU (2N×2N partition type); b. merge mode is used to derive the motion data (merge_flag equal to 1); and c. no residual data is present in the bitstream.

[0285] A parallel merge estimation level indicating the region is introduced in HEVC. The merge candidate list can be derived independently by checking whether the block candidates are located in the merge estimation region (MER). Block candidates in the same MER are not included in the merge candidate list. Therefore, the motion data of the candidate blocks does not need to be available during the construction of the list. When the level is, for example, 32, all prediction units in a 32×32 region can be used to build the merge candidate list in parallel, because not all merge candidates in the same 32×32 MER are inserted into the list. All potential merge candidates for the first PU 0 are available, because all potential merge candidates are outside the first 32×32 MER. For the second MER, when the merge estimation inside the MER should be independent, the merge candidate list for PUs 2 to 6 cannot include the motion data from these PUs. Thus, for example, when PU 5 is seen, no merge candidates are available and therefore are not inserted into the merge candidate list. In this case, the merge list for PU 5 includes only the temporal candidates (if available) and the zero MV candidates. To allow the encoder to trade off parallelism and coding efficiency, the parallel merge estimation level is adaptive and is signaled as log2_parallel_merge_level_minus2 in the picture parameter set.

[0286] Sub-CU based motion vector prediction

[0287] During the development of new video coding techniques, with QTBT, each CU may have at most one set of motion parameters for each prediction direction. Two sub-CU level motion vector prediction methods are considered in the encoder by partitioning a large CU into sub-CUs and deriving motion information for all sub-CUs of the large CU. The alternative temporal motion vector prediction (ATMP) method allows each CU to fetch multiple sets of motion information from multiple blocks smaller than the current CU in the co-located reference picture. In the spatial-temporal motion vector prediction (STMP) method, the motion vector of a sub-CU is derived recursively by using the temporal motion vector predictor and the spatial neighborhood motion vector.

[0288] In order to maintain a more accurate motion field for sub-CU motion prediction, motion compensation for reference frames is currently disabled.

[0289] Alternative Temporal Motion Vector Prediction

[0290] In the alternative temporal motion vector prediction (ATMP) method, the temporal motion vector prediction (TMVP) for motion vectors is modified by fetching multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU. A sub-CU is a square N×N block (N is set to 4 by default).

[0291] ATMVP predicts the motion vectors of sub-CUs in a CU in two steps. The first step is to identify the corresponding blocks in a reference picture by using a time vector. The reference picture is called a motion source picture. The second step is to partition the current CU into sub-CUs and obtain the motion vector and reference index of each sub-CU from the block corresponding to the sub-CU.

[0292] In the first step, the reference picture and the corresponding block are determined based on the motion information of the spatial neighboring blocks of the current CU. To avoid the repetitive scanning process for the neighboring blocks, the first merge candidate in the merge candidate list for the current CU is used. The first available motion vector and the reference index associated with the first available motion vector are set to be the temporal vector and index of the motion source picture. In this way, compared with TMVP, the corresponding block can be more accurately identified in ATMVP. In TMVP, the corresponding block (sometimes called the co-located block) is always located in the lower right or center position with respect to the current CU.

[0293] In the second step, the corresponding block of the sub-CU is identified by a time vector in the motion source picture by adding the time vector to the coordinates of the current CU. For each sub-CU, the motion information of the corresponding block of the sub-CU (the smallest motion grid containing the center sample) is used to derive the motion information for the sub-CU. After the motion information of the corresponding N×N block is identified, the motion information is converted into the motion vector and reference index of the current sub-CU in the same manner as TMVP in HEVC, and motion scaling and other procedures are applied. For example, the decoder checks whether the low-delay condition (i.e., the POC of all reference pictures of the current picture is smaller than the POC of the current picture) is met, and possibly uses the motion vector MVx (the motion vector corresponding to the reference picture list X) to predict the motion vector MVy (X is equal to 0 or 1, and Y is equal to 1-X) for each sub-CU.

[0294] Spatio-temporal motion vector prediction

[0295] In this method, the motion vectors of the sub-CUs are derived recursively according to the raster scan order. An 8x8 CU is considered to contain four 4x4 sub-CUs A, B, C, and D. The neighboring 4x4 blocks in the current frame are labeled as a, b, c, and d.

[0296] The motion derivation for sub-CU A starts by identifying two spatial neighbors of sub-CU A. The first neighbor is the N×N block above sub-CU A (i.e., block c). If block c is unavailable or intra-coded, the other N×N block above sub-CU A is checked (starting from block c, from left to right). The second neighbor is the block to the left of sub-CU A (i.e., block b). If block b is unavailable or intra-coded, the other block to the left of sub-CU A is checked (starting from block b, from top to bottom). The motion information obtained from the neighboring blocks for each list is scaled to the first reference frame for a given list. Then, the temporal motion vector prediction value (TMVP) of sub-block A is derived according to the same TMVP derivation procedure specified in HEVC. The motion information of the co-located block at position D is fetched and scaled accordingly. Finally, after the motion information is retrieved and scaled, all available motion vectors (up to three) are averaged separately for each reference list, and the averaged motion vector is assigned as the motion vector of the current sub-CU.

[0297] Combination with merge mode

[0298] As additional merge candidates, sub-CU mode is enabled, and no additional syntax elements are required to signal the mode. Two additional merge candidates are added to the merge candidate list for each CU to represent ATMVP and STMVP modes. If the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven merge candidates are used. The encoding logic of the additional merge candidates is the same as that of the merge candidates in HM. This means that for each CU in a P or B slice, two or more RD checks need to be performed for the two additional merge candidates.

[0299] Affine motion compensation prediction

[0300] The affine motion field of a block is described by using two control point motion vectors.

[0301] The motion vector field (MVF) of a block is described according to the following equation:

[0302]

number

[0303] Here, (v 0x ,v 0y ) represents the motion vector of the upper left corner control point, and (v 1x ,v 1y ) represents the motion vector of the upper right corner control point.

[0304] To further simplify the motion compensation prediction, subblock-based affine transformation prediction is applied. The subblock size M×N is derived according to formula (1.7), MvPre represents the motion vector fractional accuracy (e.g., 1 / 16), and (v 2x ,v 2y ) represents the motion vector of the bottom-left control point calculated according to equation (1.6).

[0305]

number

[0306] After the sub-block size M×N is derived according to equation (1.7), M and N may be adjusted downwards if necessary to make M and N divisors of w and h, respectively.

[0307] To derive the motion vector for each M×N sub-block, the motion vector of the center sample of the sub-block is calculated according to equation (1.6) and rounded to 1 / 16 fractional precision.

[0308] Affine Inter Mode

[0309] For CUs whose width and height are both greater than 8, the AF_INTER mode may be applied. An affine flag at the CU level is signaled in the bitstream to indicate whether the AF_INTER mode is used. In this mode, the motion vector pair {(v0,v1)|v0={v A ,v B ,v C},v1={v D ,v E A candidate list with}} is constructed by using neighboring blocks. v0 is selected from the motion vectors of blocks A, B, or C. The motion vectors from the neighboring blocks are scaled according to the reference list and according to the relationship between the POC of the reference for the neighboring block, the POC of the reference for the current CU, and the POC of the current CU. The method used to select v1 from neighboring blocks D and E is similar. If the amount of the candidate list is smaller than 2, the list is padded with motion vector pairs that are made by duplicating each AMVP candidate. When the amount of the candidate list is larger than 2, the candidates are first sorted according to the consistency of the neighboring motion vectors (the similarity of the two motion vectors in a pair candidate), and only the first two candidates are kept. To determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) of the current CU, the RD cost check is used, and an index indicating the position of the CPMVP in the candidate list is signaled in the bitstream. The difference between the CPMV and the CPMVP is signaled in the bitstream.

[0310] Affine Merge Mode

[0311] When a CU is applied in AF_MERGE mode, the first block coded based on affine mode is obtained from the valid neighboring reconstructed blocks. The selection order of the candidate blocks is left, top, top right, bottom left, top left. If the bottom left neighboring block A is coded based on affine mode, the top left, top right, and bottom left motion vectors v2, v3, and v4 of the CU containing block A are derived. In addition, the top left motion vector v0 of the current CU is calculated according to v2, v3, and v4. Then, the top right motion vector v1 of the current CU is calculated.

[0312] To specify whether the current CU is coded based on the AF_MERGE mode, when there is at least one neighboring block coded based on the affine mode, an affine flag is signaled in the bitstream.

[0313] Pattern matching motion vector derivation

[0314] The pattern matched motion vector derivation (PMMVD) mode is based on the Frame-Rate Up Conversion (FRUC) technique, in which the motion information of blocks is not signaled but is derived at the decoder side.

[0315] When the merge flag of a CU is true, the FRUC flag of the CU is signaled. When the FRUC flag is false, a merge index is signaled and regular merge mode is used. When the FRUC flag is true, an additional FRUC mode flag is signaled to indicate which method (bilateral matching or template matching) should be used to derive the motion information of the block.

[0316] At the encoder side, the decision of whether to use the FRUC merging mode for a CU is based on the RD cost selection as done for normal merging candidates. That is, two matching modes (bilateral matching and template matching) are both checked for the CU by using the RD cost selection. The mode that leads to the smallest cost is further compared with other CU modes. If the FRUC matching mode is the most efficient mode, the FRUC flag is set to true for the CU and the associated matching mode is used.

[0317] The motion derivation process in FRUC merge mode includes two steps. A CU-level motion search is performed first, followed by sub-CU-level motion refinement. At the CU level, an initial motion vector is derived for the entire CU based on bilateral matching or template matching. First, a list of MV candidates is generated, and the candidate leading to the smallest matching cost is selected as the starting point for further CU-level refinement. Then, a local search based on bilateral matching or template matching around the starting point is performed, and the MV leading to the smallest matching cost is used as the MV for the entire CU. Then, the motion information is further refined at the sub-CU level by using the derived CU motion vector as the starting point.

[0318] For example, the following derivation process is performed for deriving motion information of a W×H CU: In the first step, the MV of the whole W×H CU is derived. In the second step, the CU is further partitioned into M×M sub-CUs. The value of M is calculated according to formula (1.8), and D represents a predefined partition depth, which is set to 3 by default in JEM. Then, the MV of each sub-CU is derived.

[0319]

number

[0320] Bilateral matching is used to derive the motion information of the current CU by finding the closest match between two blocks along the motion trajectory of the current CU in two different reference pictures. Under the assumption of continuous motion trajectories, the motion vectors MV0 and MV1 pointing to two reference blocks shall be proportional to the temporal distances between the current picture and the two reference pictures, i.e., TD0 and TD1. When the current picture is between the two reference pictures in time and the temporal distances between the current picture and the two reference pictures are the same, the bilateral matching becomes mirror-based bidirectional MV.

[0321] In bilateral matching merge mode, bi-prediction is always applied because the motion information of the CU is derived based on the closest match between two blocks along the motion trajectory of the current CU in two different reference pictures. There is no such restriction in template matching merge mode. In template matching merge mode, the encoder can select uni-prediction from list 0, uni-prediction from list 1, or bi-prediction for the CU. The selection is based on the template matching cost as follows: If costBi<=factor*min(cost0,cost1), Bi-prediction is used; Otherwise, if cost0<=cost1, Uni-prediction from list 0 is used; Otherwise, The uni-prediction from List 1 is used.

[0322] cost0 represents the SAD of list 0 template matching, cost1 represents the SAD of list 1 template matching, and costBi represents the SAD of bi-predictive template matching. The value of the coefficient is equal to 1.25, which means that the selection process is biased towards bi-prediction. The selection of inter prediction direction is only applied to the CU-level template matching process.

[0323] Template matching is used to derive the motion information of the current CU by finding the closest match between a template in the current picture (a neighboring block above and / or to the left of the current CU) and a block in the reference picture (of the same size as the template). Except for the aforementioned FRUC merge mode, template matching is also applied to the AMVP mode. By using the template matching method, a new candidate is derived. If the new candidate derived through template matching is different from the first existing AMVP candidate, the new candidate is inserted at the beginning of the AMVP candidate list, and then the size of the list is set to 2 (which means that the second existing AMVP candidate is removed). When template matching is applied to the AMVP mode, only the CU-level search is applied.

[0324] The MV candidate set at the CU level includes: a. the original AMVP candidates selected if AMVP mode is used for the current CU, b. all merge candidates, c. some MVs in the interpolated MV field, and d. the top and left neighboring motion vectors.

[0325] It should be noted that the above mentioned interpolated MV field is generated before the whole picture is coded based on unilateral ME. Then, the motion field can be used later as MV candidate at CU level or sub-CU level. First, the motion field of each reference picture in the two reference lists is covered at 4×4 block level. For each 4×4 block, the motion associated with the block passes through the 4×4 block in the current picture, and if the block is not assigned any interpolated motion, the motion of the reference block is scaled to the current picture based on the temporal distances TD0 and TD1 (in the same manner as MV scaling in TMVP of HEVC), and the scaled motion is assigned to the block in the current frame. If no scaled MV is assigned to the 4×4 block, the motion of the block is marked as unavailable in the interpolated motion field.

[0326] When bilateral matching is used, the valid MV of each of the merge candidates is used as input to generate an MV pair under the assumption of bilateral matching. For example, the valid MV of one of the merge candidates is (MVa, refa) in reference list A. Then, the reference picture refb of the bilateral MV of the pair is found in the other reference list B. In this way, refa and refb are located on different sides of the current picture in time. If such refb is unavailable in reference list B, refb is determined as a reference different from refa, and the temporal distance between refb and the current picture is the smallest one in list B. After refb is determined, MVb is derived by scaling MVa based on the temporal distance between the current picture and refa and the temporal distance between the current picture and refb.

[0327] The four MVs from the interpolated MV field are also added to the CU-level candidate list: more specifically, the interpolated MVs at positions (0,0), (W / 2,0), (0,H / 2), and (W / 2,H / 2) of the current CU are added.

[0328] When FRUC is applied to AMVP mode, the original AMVP candidates are also added to the CU-level MV candidate set.

[0329] At the CU level, up to 15 MVs of the AMVP CU and up to 13 MVs of the merged CU are added to the candidate list.

[0330] The MV candidate set at the sub-CU level includes: a. the MV determined from the CU level search; b. the top, left, top-left, and top-right neighboring MVs; c. a scaled version of the co-located MV from the reference picture; d. up to four ATMVP candidates; and e. up to four STMVP candidates.

[0331] The scaled MV from the reference picture is derived as follows: all reference pictures in both lists are traversed, and the MV at the collocated position of the sub-CU in the reference picture is scaled to the reference of the MV at the start CU level.

[0332] ATMVP and STMVP candidates are limited to the four initial candidates.

[0333] At the sub-CU level, up to 17 MVs are added to the candidate list.

[0334] Motion Vector Improvement

[0335] The motion vectors can be improved by using various methods in combination with various inter prediction modes.

[0336] MV improvement in FRUC

[0337] MV refinement is a pattern-based MV search with bilateral matching cost or template matching cost criteria. In the current development, two search patterns are supported: unrestricted center-biased diamond search (UCBDS) and adaptive cross search, which are used for MV refinement at CU and sub-CU levels, respectively. For both CU and sub-CU level MV refinement, the MV is directly searched with MV accuracy of 1 / 4 luma sample, followed by MV accuracy of 1 / 8 luma sample. The search range of MV refinement for CU and sub-CU steps is set equal to 8 luma samples.

[0338] Decoder side motion vector improvement

[0339] In bi-prediction operation, for prediction of one block region, two prediction blocks formed using list 0 MV and list 1 MV respectively are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of bi-prediction are further refined by using a bilateral template matching process. The bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed sample in the reference picture, and obtain the refined MV without transmitting additional motion information.

[0340] In DMVR, a bilateral template is generated as a weighted combination (i.e., average) of two prediction blocks, from the first list 0 MV0 and list 1 MV1, respectively. The template matching operation involves calculating a cost measure between the generated template and a sample region (around the first prediction block) in the reference picture. For each of the two reference pictures, the MV that produces the smallest template cost is considered as the updated MV for the list, which should replace the original MV. In the current development, nine MV candidates are searched for each list. The nine MV candidates include the original MV and eight surrounding MVs with one luma sample offset to the original MV in either the horizontal and vertical directions or both the horizontal and vertical directions. Finally, two new MVs, namely MV 0' and MV 1', are used to generate the final bi-prediction result. The sum of absolute difference (SAD) is used as the cost measure.

[0341] DMVR is applied for bi-predictive merge mode with one MV from a past reference picture and another MV from a future reference picture without transmitting additional syntax elements.

[0342] Accuracy and storage of movement data

[0343] Motion data storage reduction

[0344] The use of TMVP in AMVP as well as in merge mode requires the storage of motion data (including motion vectors, reference indexes, and coding modes) in co-located reference pictures. Considering the granularity of the motion representation, the memory size required to store the motion data is enormous. HEVC utilizes motion data storage reduction (MDSR) to reduce the size of the motion data buffer and the associated memory access bandwidth by subsampling the motion data in the reference picture. In H.264 / AVC, information is stored every 4x4 block, while in HEVC 16x16 blocks are used. When subsampling a 4x4 grid, the information of the top left 4x4 block is stored. Due to this subsampling, MDSR impacts the quality of temporal prediction.

[0345] Furthermore, there is a close correlation between the locations of MVs used in co-located pictures and the locations of MVs stored by using MDSR. During the HEVC standardization process, it was found that storing the motion data of the top-left block inside the 16x16 region together with the bottom-right and center TMVP candidates provides the best trade-off between coding efficiency and memory bandwidth reduction.

[0346] Higher motion vector memory accuracy

[0347] In HEVC, the motion vector accuracy is 1 / 4 pel (1 / 4 luma sample and 1 / 8 chroma sample for 4:2:0 video). In the current development, the accuracy of the internal motion vector storage and merge candidates is increased to 1 / 16 pel. The higher motion vector accuracy (1 / 16 pel) is used in motion compensated inter prediction for CUs coded based on skip / merge mode. For CUs coded based on normal AMVP mode, either integer pel motion or quarter pel motion is used.

[0348] Adaptive motion vector differential resolution

[0349] In HEVC, motion vector difference (MVD) is signaled in units of quarter luma samples when use_integer_mv_flag is equal to 0 in the slice header. In current development, locally adaptive motion vector resolution (LAMVR) is introduced. MVD can be coded in units of quarter luma samples, integer luma samples, or 4 luma samples. MVD resolution is controlled at the coding unit (CU) level, and an MVD resolution flag is conditionally signaled for each CU that has at least one non-zero MVD component.

[0350] For CUs that have at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MV precision is used for the CU. When the first flag (which is equal to 1) indicates that quarter luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or 4 luma sample MV precision is used.

[0351] When the first MVD resolution flag of a CU is 0 or is not coded for a CU (which means all MVDs for the CU are 0), quarter luma sample MV resolution is used for the CU. When integer luma sample MV precision or 4 luma sample MV precision is used for a CU, the MVPs in the AMVP candidate list for the CU are rounded to the corresponding precision.

[0352] At the encoder, a CU-level RD check is used to determine which MVD resolution should be used for a CU, i.e., the CU-level RD check is performed three times for each MVD resolution.

[0353] Fractional Sample Interpolation Module

[0354] When the motion vectors point to fractional sample positions, motion compensated interpolation is required. As shown in Table 1.2, for luma interpolation filtering, an 8-tap separable DCT-based interpolation filter is used for 2 / 4 precision samples, and a 7-tap separable DCT-based interpolation filter is used for 1 / 4 precision samples.

[0355] [Table 2]

[0356] Similarly, a 4-tap separable DCT-based interpolation filter is used for the chroma interpolation filter, as shown in Table 1.3.

[0357] [Table 3]

[0358] For vertical interpolation for 4:2:2 chroma channels, and horizontal and vertical interpolation for 4:4:4 chroma channels, the odd positions in Table 1.3 are not used and result in 1 / 4 chroma interpolation.

[0359] In bi-prediction, the bit depth of the output of the interpolation filter is maintained at 14-bit accuracy, independent of the source bit depth, before averaging of the two prediction signals. The actual averaging process is performed implicitly together with the bit depth reduction process as follows: predSamples[x,y]=(predSamplesL0[x,y]+predSamplesL1[x,y]+offset)>>shift (1.9) shift=15-BitDepth (1.10) offset=1<<(shift-1) (1.11)

[0360] To reduce complexity, bilinear interpolation is used for both bilateral matching and template matching, instead of the usual 8-tap HEVC interpolation.

[0361] The calculation of the matching cost is slightly different in different steps. When a candidate is selected from the candidate set at the CU level, the matching cost is the SAD of bilateral matching or template matching. After the starting MV is determined, the matching cost C of bilateral matching in the sub-CU level search is calculated as follows:

[0362]

number

[0363] w represents a weighting factor empirically set to 4, and MV and MV S denote the current MV and the starting MV, respectively. Nevertheless, the SAD is used as the matching cost for template matching in the sub-CU level search.

[0364] In FRUC mode, the MV is derived by using only luma samples. The derived motion vectors will be used for both luma and chroma in MC inter prediction. After the MV is determined, the final MC is performed by using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma.

[0365] Motion Compensation Module

[0366] Overlapped Block Motion Compensation

[0367] In the current development, overlapped block motion compensation (OBMC) is performed for all motion compensation (MC) block boundaries except the right and bottom boundaries of a CU. Moreover, overlapped block motion compensation is applied to both luma and chroma components. An MC block corresponds to a coding block. When a CU is coded based on a sub-CU mode (including sub-CU merge mode, affine mode, and FRUC mode), each sub-block of a CU is an MC block. In order to uniformly handle the boundaries of a CU, OBMC is performed at the sub-block level for all MC block boundaries, and the sub-block size is set equal to 4×4.

[0368] When OBMC is applied to the current subblock, in addition to the current motion vector, if the motion vectors of the four connected neighboring subblocks are available and are not identical to the current motion vector, the motion vectors of the four connected neighboring subblocks are also used to derive a prediction block for the current subblock. These prediction blocks based on the multiple motion vectors are combined to generate a final prediction signal for the current subblock.

[0369] A prediction block based on the motion vector of a neighboring subblock is denoted as PN, where N represents the index of the upper, lower, left, or right neighboring subblock, and a prediction block based on the motion vector of the current subblock is denoted as PC. When PN is based on the motion information of a neighboring subblock that contains the same motion information as the current subblock, OBMC is not performed from PN. Otherwise, every single sample of PN is added to the same sample in PC. That is, four rows / columns of PN are added to PC. Weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for PN, and weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are used for PC. The exception is a small MC block (i.e., when the height or width of the coding block is equal to 4, or when a CU is coded based on a sub-CU mode), and for such a block, only two rows / columns of PN are added to PC. In this case, weighting factors {1 / 4, 1 / 8} are used for PN, and weighting factors {3 / 4, 7 / 8} are used for PC. For a PN generated based on the motion vectors of vertical (horizontal) neighboring subblocks, samples in the same row (column) of PN are added to PC with the same weighting factor.

[0370] In the current development, for CUs with a size of 256 luma samples or less, a CU level flag is signaled to indicate whether OBMC is applied to the current CU or not. For CUs with a size of more than 256 luma samples or CUs that are not coded based on AMVP mode, OBMC is applied by default. When OBMC is applied to a CU in the encoder, the influence of OBMC is taken into account in the motion estimation stage. The prediction signal formed through OBMC by using the motion information of the upper neighboring block and the left neighboring block is used to compensate the upper and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

[0371] Optimization Tools

[0372] Local Lighting Compensation

[0373] Local Illumination Compensation (LIC) is based on a linear model for illumination changes with a scaling factor a and an offset b. Local illumination compensation is adaptively enabled or disabled for each inter-mode coded coding unit (CU).

[0374] When LIC is applied to a CU, a least square error method is utilized to derive parameters a and b by using the neighboring samples of the current CU and the corresponding reference samples. The subsampled (2:1 subsampling) neighboring samples of the CU and the corresponding samples in the reference picture (specified by the motion information of the current CU or sub-CU) are used. The LIC parameters are derived and applied separately for each prediction direction.

[0375] When a CU is coded based on the merge mode, the LIC flag is copied from the neighboring blocks in a manner similar to copying motion information in the merge mode. Otherwise, the LIC flag is signaled to the CU to indicate whether LIC is applied.

[0376] When LIC is enabled for a picture, an additional CU-level RD check is required to determine whether LIC applies to a CU. When LIC is enabled for a CU, the mean-removed sum of absolute difference (MR-SAD) and the mean-removed sum of absolute Hadamard-transformed difference (MR-SATD) are used for integer-pel and fractional-pel motion searches, respectively, instead of SAD and SATD.

[0377] Bidirectional Optical Flow

[0378] Bi-directional Optical flow (BIO) is a sample-by-sample motion refinement that is performed on top of block-by-block motion compensation for bi-prediction. Sample-level motion refinement does not use signaling.

[0379] I (k) Let ∂I be the luma value from reference k (k=0,1) after block motion compensation. (k) / ∂x and ∂I (k) / ∂y are I (k) represents the horizontal and vertical components of the gradient of the motion vector field (v x ,v y ) is given by equation (1.13). ∂I (k) / ∂t+v x ∂I (k) / ∂x+v y ∂I (k) / ∂y=0 (1.13)

[0380] For each sample's motion trajectory, this optical flow equation is combined with Hermite interpolation to obtain the function value I (k) and the derivative ∂I (k) / ∂x and ∂I (k) / ∂y. The value of this polynomial at t=0 is the BIO predicted value. pred BIO =1 / 2(I (0) +I (l) +v x / 2(τ1∂I (1) / ∂x-τ0∂I (0) / ∂x)+v y / 2(τ1∂I (1) / ∂y-τ0∂I (0) / ∂y)) (1.14)

[0381] where τ0 and τ1 represent the distance to the reference frames. The distances τ0 and τ1 are calculated based on the POCs relative to Ref0 and Ref1, τ0=POC(current)-POC(Ref0), τ1=POC(Ref1)-POC(current). If both predictions come from the same temporal direction (either both from the past or both from the future), the signs are different (i.e., τ0·τ1<0). In this case, BIO applies only if the predictions are not from the same instant (i.e., τ0≠τ1), both referenced regions have non-zero motion vectors (MVx0,MVy0,MVx1,MVy1≠0), and the block motion vectors are proportional to the temporal distance (MVx0 / MVx1=MVy0 / MVy1=-τ0 / τ1).

[0382] Motion vector field (v x ,v y ) is determined by minimizing the difference Δ (the intersection of the motion trajectory and the reference frame plane) between the value at point A and the value at point B. The model uses only the first linear term of a local Taylor expansion for Δ.

[0383]

number

[0384] All values ​​in equation (1.15) depend on the sample position (i',j'), which has been omitted from the notation so far. Assuming that the motion is consistent in the local surrounding area, Δ is minimized inside a (2M+1)×(2M+1) square interval Ω centered at the current predicted point (i,j), where M is equal to 2.

[0385]

number

[0386] For this optimization problem, the current development uses a simplified approach that first minimizes in the vertical direction and then in the horizontal direction.

[0387]

number

[0388] Where:

[0389]

number

[0390] To avoid division by zero or very small values, regularization parameters r and m are introduced in equations (1.17) and (1.18). r=500·4 d-8 (1.20) m=700·4 d-8 (1.21)

[0391] Here, d represents the bit depth of the video samples.

[0392] In order to keep the memory access for BIO the same as for normal bi-predictive motion compensation, all the prediction and gradient values ​​I (k) , ∂I (k) / ∂x, ∂I (k) In equation (1.19), a (2M+1) × (2M+1) square window Ω centered on the current prediction point on the boundary of the prediction block needs to access positions outside the block. In the current development, I (k) , ∂I (k) / ∂x, ∂I (k) The value of / ∂y is set equal to the closest available value inside the block. For example, this could be implemented as padding.

[0393] With BIO, it is possible for the motion field to be refined for each sample. However, to reduce computational complexity, a block-based design of BIO can be used. The motion refinement is calculated on a 4×4 block basis. In block-based BIO, the s n The values ​​of are then aggregated, and then s n The aggregated values ​​of are used to derive the BIO motion vector offsets for 4x4 blocks. For block-based BIO derivation, the following formula is used:

[0394]

number

[0395] b k denotes the set of samples belonging to the kth 4 × 4 block of the prediction block. n is used to derive the associated motion vector offset ((s n ,b k )>>4) is replaced by.

[0396] In some cases, the large amount of MV in BIO may be unreliable due to noise or irregular motion. Therefore, in BIO, the magnitude of the large amount of MV is limited to a threshold thBIO. The threshold is determined based on whether the reference pictures of the current picture are all from one direction. If all the reference pictures of the current picture are from one direction, the value of the threshold is 12×2 14-d otherwise the value is set to 12 × 2 13-d is set to.

[0397] The gradients for BIO are calculated simultaneously with motion compensation interpolation by using operations consistent with the HEVC motion compensation process (2D separable FIR). The input of the 2D separable FIR is the same reference frame sample as the motion compensation process according to the fractional part of the block motion vector and the fractional position (fracX, fracY). For the horizontal gradient ∂I / ∂x signal, first, vertical interpolation is performed by using BIOfilterS corresponding to fractional position fracY with an unscaling shift of d-8, and then a gradient filter BIOfilterG is applied horizontally corresponding to fractional position fracX with an unscaling shift of 18-d. For the vertical gradient ∂I / ∂y, first, a gradient filter is applied vertically by using BIOfilterG corresponding to fractional position fracY with an unscaling shift of d-8, and then signal cancellation is performed by using BIOfilterS horizontally corresponding to fractional position fracX with an unscaling shift of 18-d. The length of the interpolation filter BIOfilterG for gradient calculation and the interpolation filter BIOfilterS for signal cancellation is shorter (6 taps) to maintain reasonable complexity. Table 1.4 shows the filters used for gradient calculation at different fractional positions of the block motion vector in BIO. Table 1.5 shows the interpolation filters for prediction signal generation in BIO.

[0398] [Table 4]

[0399] [Table 5]

[0400] In the current development, BIO is applied to all bi-predictive blocks when the two predictions are from different reference pictures. When LIC is enabled for a CU, BIO is disabled. OBMC is applied to a block after the normal MC process. To reduce the computational complexity, BIO is not applied in the OMBC process. This means that BIO is only applied in the MC process for a block when the MV of the block is used, but is not applied in the MC process when the MV of the neighboring block is used in the OBMC process.

[0401] Weighted Sample Prediction Module

[0402] As an optional tool, a weighted prediction (WP) tool is provided in HEVC. The principle of WP is to replace the inter prediction signal P by a linear weighted prediction signal P' (with weights w and offsets o). Uniform prediction: P' = w × P + 0 (1.23) Bi-prediction: P' = (w0 × P0 + o0 + w1 × P1 + o1) / 2 (1.24)

[0403] The applicable weights and the applicable offsets are selected by the encoder and conveyed in the bitstream. The L0 and L1 suffixes define list 0 and list 1 of the reference picture lists, respectively. Due to the interpolation filters, the bit depth is maintained at 14-bit accuracy before the prediction signal is averaged.

[0404] In the case of bi-prediction with at least one reference picture available in each of lists L0 and L1, the following formula applies for the explicit signaling of weighted prediction parameters for the luma channel: Corresponding formulas apply for the chroma channels and in the uni-prediction case.

[0405]

number

[0406] Where: log2WD=luma_log2_weight_denom+14-bitDepth, w0=LumaWeightL0[refIdxL0], wl=LumaWeightL[refIdxL1], o0=luma_offset_l0[refIdxL0]*highPrecisionScaleFactor, o1=luma_offset_l1[refIdxL1]*highPrecisionScaleFactor, highPrecisionScaleFactor=(1<<(bitDepth-8))

[0407] Boundary prediction filter is an intra-coding method used to further adjust the prediction samples of the leftmost column and the topmost row. In HEVC, after an intra-prediction block is generated for vertical or horizontal intra-mode, the prediction samples of the leftmost column and the topmost row are further adjusted, respectively. This method may be further extended to some diagonal intra-modes, where up to four columns or four rows of boundary samples are further adjusted by using a two-tap filter (for intra-modes 2 and 34) or a three-tap filter (for intra-modes 3 to 6 and 30 to 33).

[0408] In the HEVC standard and previous standards, reference frames are classified into two groups, forward reference frames and backward reference frames, and are arranged in two reference picture lists, respectively. The two reference frame lists are usually named List 0 and List 1. An inter prediction direction is used to indicate whether a prediction direction, forward prediction, backward prediction, or bi-prediction, is used for the current block, and a different reference frame list, i.e., List 0, List 1, or both List 0 and List 1, is selected based on the prediction direction. The reference frame in the selected reference frame list is indicated by a reference frame index. A motion vector is used to indicate the position offset of the reference block of the prediction block of the current block in the selected reference frame relative to the current block in the current frame. Then, a final prediction block is generated based on the prediction direction by using the prediction block obtained from the reference frames in List 0, List 1, or both List 0 and List 1. When the prediction direction is uni-prediction, the prediction block obtained from the reference frame in List 0 or List 1 is directly used. When the prediction direction is bi-predictive, the prediction blocks obtained from the reference frames in both list 0 and list 1 are combined through weighted averaging to obtain the final prediction block.

[0409] In order to solve the problems of the prior art that the prediction samples obtained based on the inter prediction mode are spatially discontinuous, the prediction efficiency is affected, and the prediction residual energy is relatively large, the embodiments of the present application provide an inter prediction method for filtering the prediction samples by using neighboring reconstructed samples after the prediction samples are generated, thereby improving the coding efficiency.

[0410] 13 is a schematic flowchart of an inter prediction method according to an embodiment of the present application. As shown in FIG. 13, the method includes steps S1301 to S1307.

[0411] S1301: Analyze the bitstream to obtain motion information of the picture block to be processed.

[0412] The current picture block may be referred to as the current block or the current CU.

[0413] It can be understood that step S1301 may be performed by the video decoder 200 of FIG.

[0414] For example, in this embodiment of the present application, a block-based motion compensation technique may be used to search the coded blocks for the best matching block of the current coding block, so that the residual between the prediction block and the current block is as small as possible and is used to calculate the offset MV of the current block.

[0415] For example, the picture block to be processed may be any block in a picture, and the size of the picture block to be processed may be 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, or 128×128, which is not limited in this embodiment of the present application.

[0416] For example, if the current picture block (current block) is encoded at the encoder side based on merge mode, the spatial candidate and the temporal candidate of the current block can be added to the merge motion information candidate list for the current block. The method is the same as that in HEVC. For example, any one of the techniques described in Figure 8 to Figure 12 can be used as a specific method for obtaining the merge motion information candidate list.

[0417] For example, if a merge mode is used for the current block, the motion information of the current block is determined based on a merge index carried in the bitstream. If an inter MVP mode is used for the current block, the motion information of the current block is determined based on an inter prediction direction, a reference frame index, a motion vector predictor index, and a motion vector residual value transmitted in the bitstream.

[0418] Step S1301 may be performed by using a method in HEVC or VTM, or may be performed by using another method for generating a motion vector prediction candidate list, which is not limited in this embodiment of the present application.

[0419] S1302: (Optional) It is determined to update the prediction block of the picture block to be processed.

[0420] It may be understood that step S1302 may be performed by the video decoder 200 of FIG.

[0421] The predictive block of the current picture block is a predictive block of the current block and may be obtained based on one or more coded blocks.

[0422] For example, whether to update the prediction block of the current picture block may be determined based on the update decision indicator of the current picture block. In other words, whether to perform spatial filtering on the current picture block may be determined based on the update decision indicator of the current picture block.

[0423] In one possible implementation form, the bitstream may be analyzed to obtain update decision indication information for the picture block to be processed, which is used to indicate whether to update a prediction block of the picture block to be processed, and based on the update decision indication information for the picture block to be processed, it is further determined that the prediction block of the picture block to be processed should be updated.

[0424] In another possible implementation form, pre-defined update decision indication information for a picture block to be processed may be obtained, and the pre-defined update decision indication information is used to indicate whether to update a prediction block of the picture block to be processed, and based on the pre-defined update decision indication information, it is further determined that the prediction block of the picture block to be processed should be updated.

[0425] For example, if the update decision indicator is true, it may be determined that the prediction block of the current picture block should be updated. In other words, it is determined that spatial filtering should be performed on the prediction block of the current picture block. If the update decision indicator is false, it is determined that the prediction block of the current picture block does not need to be updated. The specific form of the update decision indicator is not limited in this embodiment of the present application. Here, the update decision indicator being true or false is used merely as an example for explanation.

[0426] S1303: (Optional) Determine a prediction mode corresponding to the current picture block to be processed.

[0427] It may be understood that step S1303 may be performed by the video decoder 200 of FIG.

[0428] For example, the prediction mode corresponding to the picture block to be processed may be a merge mode and / or an inter-evolutionary motion vector prediction (inter AMVP) mode. This is not limited in this embodiment of the present application. It can be understood that the prediction mode corresponding to the picture block to be processed may be only a merge mode, only an inter AMVP mode, or a combination of a merge mode and an inter AMVP mode.

[0429] It should be noted that the inter evolutionary motion vector prediction (inter AMVP) mode may also be referred to as the inter motion vector prediction (inter MVP) mode.

[0430] For example, a method for determining a prediction mode corresponding to a picture block to be processed may include a step of parsing a bitstream to obtain a prediction mode corresponding to the picture block to be processed, and a step of determining that the prediction mode corresponding to the picture block to be processed is a merge mode and / or an inter AMVP mode.

[0431] It can be understood that in this embodiment of the present application, the spatial filtering method can be performed on the inter-coded blocks that are coded based on the merge mode and / or the inter-AMVP mode. In other words, the filtering process can be performed on the blocks that are coded based on the merge mode and / or the inter-AMVP mode at the decoder side during decoding.

[0432] S1304: Perform motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block.

[0433] The prediction block of the current picture contains the predicted values ​​of the target samples.

[0434] It may be understood that step S1304 may be performed by the video decoder 200 of FIG.

[0435] For example, during motion compensation, a current partial picture is predicted and compensated based on a reference picture, which can reduce redundant information in a frame sequence.

[0436] For example, when motion compensation is performed based on motion information, a prediction block of a current picture block can be obtained from a reference frame based on a reference frame direction, a reference frame sequence number, and a motion vector. The reference frame direction can be forward prediction, backward prediction, or bi-prediction. This is not limited in this embodiment of the present application.

[0437] For example, when the reference frame direction is forward prediction, a reference picture may be selected from a forward reference picture set for a current coding unit (CU) to obtain a reference block. When the reference frame direction is backward prediction, a reference picture may be selected from a backward reference picture set for a current coding unit (CU) to obtain a reference block. When the reference frame direction is bi-prediction, a reference picture may be selected from each of a forward reference picture set and a backward reference picture set for a current coding unit (CU) to obtain a reference block.

[0438] In step S1304, the method for performing motion compensation on the current picture block according to the motion information may be the method in HEVC or VTM, or may be another method for obtaining the prediction block of the current picture block, which is not limited in this embodiment of the present application.

[0439] S1306: To update the predicted value of the target sample, a weighted calculation is performed on the one or more reconstructed values ​​of the one or more reference samples and the predicted value of the target sample.

[0440] The reference sample has a pre-defined spatial relationship with the target sample.

[0441] It may be understood that step S1306 may be performed by the video decoder 200 of FIG.

[0442] For example, the target sample may be a sample in a predictive block of the current picture block, and the predicted value of the target sample may be determined based on a sample value of a sample in a reference block.

[0443] For example, the reference sample may be a reconstructed sample that is spatially adjacent to the current CU (the picture block to be processed). Specifically, the reference sample may be a reconstructed sample in a block other than the current CU block in the picture. For example, the reference sample may be a reconstructed sample in a CU block above or to the left of the current CU. This is not limited in this embodiment of the present application.

[0444] It can be seen that in step S1306, spatial filtering is performed on the predicted sample of the target sample by using the reconstructed sample that is spatially adjacent to the current CU. Specifically, weighted calculation is performed on the predicted sample of the target sample in the current block and the sample value of the reconstructed sample that is spatially adjacent to the current CU to obtain the updated predicted sample of the target sample.

[0445] In one possible implementation, the one or more reference samples may include a reconstructed sample having the same horizontal coordinate as the target sample and a preset vertical coordinate difference from the target sample, or a reconstructed sample having the same vertical coordinate as the target sample and a preset horizontal coordinate difference from the target sample.

[0446] For example, as shown in FIG. 14, the upper left corner of the picture is used as the origin of the coordinate system, the X-axis direction of the coordinate system extends to the right along the top side of the picture, and the Y-axis direction of the coordinate system extends to the bottom along the left side of the picture. If the coordinates of the target sample in the current picture block (current CU) are (xP, yP) and the coordinates of the top left sample of the current picture block are (xN, yN), the reference sample of the target sample may be a reconstructed sample in a block above or to the left of the current picture block. If the reference sample is a reconstructed sample in a block above the current picture block, the reference sample is a reconstructed sample in a block other than the current picture block, so that the vertical coordinate of the reference sample is a value obtained by subtracting a preset positional relationship N from the vertical coordinate of the top side of the current picture block, and the horizontal coordinate of the reference sample is the same as the horizontal coordinate of the target sample in the current picture block. Specifically, the coordinates of the reference sample are (xP, yN-N). When the reference sample is a reconstructed sample in a block to the left of the current picture block, the reference sample is a reconstructed sample in a block other than the current picture block, so the horizontal coordinate of the reference sample is a value obtained by subtracting a preset positional relationship M from the leftmost horizontal coordinate of the current picture block, and the vertical coordinate of the reference sample is the same as the vertical coordinate of the target sample in the current picture block. Specifically, the coordinate of the reference sample is (xN-M, yP). The specific spatial positional relationship (specific values ​​of M and N) between the reference sample and the target sample is not limited in this embodiment of the present application.

[0447] In one possible implementation, the predicted value of the target sample may be updated according to the following formula:

[0448]

number

[0449] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are preset constants, and M1 and M2 are preset positive integers.

[0450] The following describes a specific method for calculating the updated predicted value of the target sample based on different cases of the coordinates (xN, yN) of the top-left sample of the current picture block.

[0451] In the first case, when xN is greater than 0, yN is equal to 0, and the reference sample at position (xN-M1, yP) has been encoded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0452]

number

[0453] For example, as shown in FIG. 15, the size of the current picture block is 16×16, which is used as an example for explanation. If the current picture block is CU 1, the top-left sample (xN, yN) of the current picture block (CU 1) is (16,0), and the coordinate of the target sample (xP, yP) in the current picture block is (18,3). Since the horizontal coordinate xN of the top-left sample of the current CU (CU 1) is greater than 0, and the vertical coordinate yN of the top-left sample of the current CU (CU 1) is equal to 0, it can be determined that the current CU is located at the top of the picture. When spatial filtering is performed on the target sample in the current CU, the current CU is located at the top of the picture, and there is no reconstructed sample above the current CU, so the reference sample is the reconstructed sample to the left of the current CU. The reference sample has the same vertical coordinate as the target sample and may be denoted as (16-M1,3), where M1 represents the preset spatial positional relationship between the reference sample and the target sample. In this specification, M1 is used as an example for explanation that M1 is equal to 1. When M1 is equal to 1, the reference sample of the target sample (18, 3) can be (15, 3). Similarly, any reference sample of the target sample (xP, yP) can be (xN-1, yP).

[0454] If the reference sample at position (xN-1, yP) has been encoded and reconstructed, a weighted calculation may be performed on the reconstructed value of the reference sample recon(xN-1, yP) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0455] In the second case, when xN is equal to 0, yN is greater than 0, and the reference sample at position (xP, yN-M2) has been encoded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0456]

number

[0457] For example, as shown in FIG. 15, if the current picture block is CU 2, the top-left sample (xN, yN) of the current picture block (CU 2) is (0, 32), and the coordinate of the target sample (xP, yP) is (8, 35). The horizontal coordinate xN of the top-left sample of the current CU (CU 2) is equal to 0, and the vertical coordinate yN of the top-left sample of the current CU (CU 2) is greater than 0, so it can be determined that the current CU is located on the left side of the picture. When spatial filtering is performed on the target sample in the current CU, the current CU is located on the left side of the picture, and there is no reconstructed sample on the left side of the current CU, so the reference sample is the reconstructed sample above the current CU. The reference sample has the same horizontal coordinate as the target sample, and may be denoted as (8, 32-M2), where M2 represents the preset spatial positional relationship between the reference sample and the target sample. In this specification, M2 equal to 1 is used as an example for explanation. When M2 is equal to 1, the reference sample of the target sample (8,35) may be (8,31). Similarly, any reference sample of the target sample (xP,yP) may be (xP,yN-M2).

[0458] When the reference sample at position (xP, yN-M2) has been encoded and reconstructed, a weighted calculation may be performed on the reconstructed value of the reference sample recon(xP, yN-M2) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0459] In a third case, when xN is greater than 0, yN is greater than 0, and the reference samples at positions (xN-M1, yP) and (xP, yN-M2) have been encoded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0460]

number

[0461] For example, as shown in FIG. 15, if the current picture block is CU 3, the top-left sample (xN, yN) of the current picture block (CU 3) is (48, 32), and the coordinate of the target sample (xP, yP) is (56, 33). Since the horizontal coordinate xN of the top-left sample of the current CU (CU 3) is greater than 0, and the vertical coordinate yN of the top-left sample of the current CU (CU 3) is greater than 0, it may be determined that the current CU is not located at the edge of the picture. When spatial filtering is performed on the target sample in the current CU, the reference sample may be a reconstructed sample above the current CU and a reconstructed sample to the left of the current CU. When the reference sample is a reconstructed sample to the left of the current CU, the reconstructed sample has the same vertical coordinate as the target sample, and may be (xN-M1, 33). When the reference sample is a reconstructed sample above the current CU, the reconstructed sample has the same horizontal coordinate as the target sample and may be (56, yN-M2), where M1 and M2 respectively represent the preset spatial positional relationship between each of the reference sample and the target sample. In this specification, both M1 and M2 are equal to 1, and are used as an example for explanation. When both M1 and M2 are equal to 1, the reference samples of the target sample (56, 33) may be (47, 33) and (56, 31). Similarly, any reference sample of the target sample (xP, yP) may be (xN-M1, yP) and (xP, yN-M2).

[0462] When the reference samples at positions (xN-M1, yP) and (xP, yN-M2) have been encoded and reconstructed, a weighted calculation may be performed on the reconstructed values ​​of the reference samples recon(xN-M1, yP) and recon(xP, yN-M2) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0463] It should be noted that the values ​​of the weighting coefficients w1, w2, w3, w4, w5, and w6, as well as the values ​​of M1 and M2, are not limited in this embodiment of the present application, and both M1 and M2 being equal to 1 is used merely as an example.

[0464] For example, the weighting coefficient set (w1, w2), (w3, w4), or (w5, w6, w7) may be a value combination of w1+w2, w3+w4, or w5+w6+w7, where w1+w2, w3+w4, or w5+w6+w7 are equal to an integer power of 2. In this way, no division operation is performed anymore. For example, value combinations such as (6,2), (5,3), (4,4), (6,1,1), or (5,2,1) may be used. This is not limited to this embodiment of the present application, and the value combinations listed herein are used only as examples for illustration.

[0465] In another possible implementation, the predicted value of the target sample may be updated according to the following formula:

[0466]

number

[0467] Here, the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are preset constants, and M1 and M2 are preset positive integers.

[0468] For example, the weighting coefficient set (w1, w2, w3) may be a value combination of w1+w2+w3, where w1+w2+w3 is equal to an integer power of 2. In this way, no division operation is performed anymore. For example, value combinations such as (6, 1, 1) or (5, 2, 1) may be used. This is not limited in this embodiment of the present application, and the value combinations listed herein are used merely as examples for illustration.

[0469] Note that the difference between this implementation and the previous implementation is that the samples to the left of the current picture block and the samples above the current picture block are not considered in this implementation. When this implementation is used to update the predicted value of the target sample and the reconstructed value of the reference sample is not available, the method in the following steps S13061 and S13062 may be used to obtain a new reference sample and update the predicted value of the target sample based on the new reference sample.

[0470] In another possible implementation, the predicted value of the target sample may be updated according to the following formula:

[0471]

number

[0472] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and rec on(xP, yN-M4) respectively represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), and w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0473] The following specifically describes a method for calculating the updated predicted value of the target sample based on different cases of the coordinates (xN, yN) of the top-left sample of the current picture block.

[0474] In the first case, when xN is greater than 0, yN is equal to 0, and the reference samples at positions (xN-M1, yP) and (xN-M2, yP) have been encoded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0475]

number

[0476] It can be seen that the difference between this case of the present implementation and the first case of the first implementation is that there are two reference samples in the present implementation. If the picture block to be processed is CU 1, the top-left sample (xN, yN) of the picture block to be processed (CU 1) is (16,0), and the coordinate of the target sample (xP, yP) in the picture block to be processed is (18,3). The reference sample is a reconstructed sample to the left of the current CU. Each of the reference samples has the same vertical coordinate as the target sample, and may be written as (16-M1,3) and (16-M2,3), where M1 and M2 each represent a preset spatial positional relationship between each of the reference samples and the target sample. In this specification, M1 is equal to 1 and M2 is equal to 2 is used as an example for explanation. When M1 is equal to 1 and M2 is equal to 2, the reference samples of the target sample (18,3) may be (15,3) and (14,3). Similarly, any reference sample of a target sample (xP, yP) can be (xN-1, yP) and (xN-2, yP).

[0477] When the reference samples at positions (xN-1, yP) and (xN-2, yP) have been encoded and reconstructed, a weighted calculation may be performed on the reconstructed values ​​of the reference samples recon(xN-1, yP) and recon(xN-2, yP) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0478] In the second case, when xN is equal to 0, yN is greater than 0, and the reference samples at positions (xP, yN-M3) and (xP, yN-M4) have been encoded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0479]

number

[0480] For example, the difference between this case in this implementation and the second case in the first implementation is that there are two reference samples in this implementation. If the picture block to be processed is CU 2, the top-left sample (xN, yN) of the picture block to be processed (CU 2) is (0, 32), and the coordinates of the target sample (xP, yP) are (8, 35). The reference samples are reconstructed samples above the current CU. Each of the reference samples has the same horizontal coordinate as the target sample, and may be written as (8, 32-M3) and (8, 32-M4), where M3 and M4 each represent a preset spatial positional relationship between each of the reference samples and the target sample. In this specification, M3 is equal to 1 and M4 is equal to 2 is used as an example for explanation. When M3 is equal to 1 and M4 is equal to 2, the reference samples of the target sample (8, 35) may be (8, 31) and (8, 30). Similarly, any reference sample of a target sample (xP, yP) can be (xP, yN-1) and (xP, yN-2).

[0481] When the reference samples at positions (xP, yN-1) and (xP, yN-2) have been encoded and reconstructed, a weighted calculation may be performed on the reconstructed values ​​of the reference samples recon(xP, yN-1) and recon(xP, yN-2) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0482] In a third case, when xN is greater than 0, yN is greater than 0, and the reference samples at positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4) have been encoded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0483]

number

[0484] For example, the difference between this case in this implementation and the third case in the first implementation is that in this implementation, there are two reconstructed samples as reference samples above the current CU, and there are two reconstructed samples as reference samples to the left of the current CU. If the picture block to be processed is CU 3, the top-left sample (xN, yN) of the picture block to be processed (CU 3) is (48, 32), and the coordinates of the target sample (xP, yP) are (56, 33). When the reference sample is a reconstructed sample on the left of the current CU, the reconstructed sample has the same vertical coordinate as the target sample, and may be (48-M1,33) and (48-M2,33), and when the reference sample is a reconstructed sample on the top of the current CU, the reconstructed sample has the same horizontal coordinate as the target sample, and may be (56,32-M3) and (56,32-M4), where M1, M2, M3, and M4 each represent a preset spatial positional relationship between each of the reference samples and the target sample. In this specification, both M1 and M3 are equal to 1, and both M2 and M4 are equal to 2 are used as an example for explanation. When both M1 and M3 are equal to 1, and both M2 and M4 are equal to 2, the reference samples of the target sample (56,33) may be (47,33), (46,33), (56,31), and (56,30). Similarly, any reference sample of a target sample (xP, yP) can be (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4).

[0485] When the reference samples at positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4) have been encoded and reconstructed, a weighted calculation may be performed on the reconstructed values ​​of the reference samples recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0486] It should be noted that the values ​​of weighting coefficients w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11, and the values ​​of M1, M2, M3, and M4 are not limited in this embodiment of the present application. Both M1 and M3 are equal to 1, and both M2 and M4 are equal to 2 are used as an example for illustration only. It can be understood that in actual application, the values ​​of M1 and M3 may be the same or different, the values ​​of M2 and M4 may be the same or different, the values ​​of M1 and M2 may be different, and the values ​​of M3 and M4 may be different.

[0487] For example, the weighting coefficient set (w1, w2, w3), (w4, w5, w6), or (w7, w8, w9, w10, w11) may be a value combination of w1+w2+w3, w4+w5+w6, or w7+w8+w9+w10+w11, where w1+w2+w3, w4+w5+w6, or w7+w8+w9+w10+w11 are equal to an integer power of 2. In this way, no division operation is performed anymore. For example, value combinations such as (6, 1, 1), (5, 2, 1), or (3, 2, 1, 1, 1) may be used. This is not limited in this embodiment of the present application, and the value combinations listed herein are used only as examples for illustration.

[0488] In another implementation, the predicted value of the target sample may be updated according to the following equation:

[0489]

number

[0490] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, w1, w2, w3, w4, and w5 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0491] For example, the weighting coefficient set (w1, w2, w3, w4, w5) may be a combination of values ​​of w1+w2+w3+w4+w5, where w1+w2+w3+w4+w5 is equal to an integer power of 2. In this way, no division operation is performed anymore. For example, a combination of values ​​such as (3, 2, 1, 1, 1) may be used. This is not limited in this embodiment of the present application, and the combination of values ​​listed in this specification is used only as an example for explanation. It should be noted that the difference between this implementation and the previous implementation is that the samples to the left of the picture block to be processed and the samples above the picture block to be processed are not considered in this implementation. When this implementation is used to update the predicted value of the target sample and the reconstructed value of the reference sample is not available, the method in the following steps S13061 and S13062 may be used to obtain the available reference sample and update the predicted value of the target sample based on the available reference sample.

[0492] In one possible implementation, the one or more reference samples include one or more of the following samples: a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the top of the picture block to be processed, a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left of the picture block to be processed, a reconstructed sample at the top right of the picture block to be processed, a reconstructed sample at the bottom left of the picture block to be processed, or a reconstructed sample at the top left of the picture block to be processed.

[0493] In another possible implementation, the predicted value of the target sample may be updated according to the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) where the coordinates of the target pixel are (xP, yP), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target pixel, and w1 and w2 are preset constants.

[0494] Specifically, a second predicted sample value predP1(xP, yP) may be first obtained based on spatial neighboring samples and a planar (PLANAR) intra prediction mode. It can be seen that in the PLANAR mode, two linear filters in the horizontal and vertical directions are used, and the average value obtained by the two linear filters is used as the predicted value of the sample in the current block.

[0495] For example, the second predicted sample value predP1(xP, yP) may be obtained based on the PLANAR mode. predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-(yP-yN))*recon(xP,yN-1)+(yP-yN+1)*recon(xN-1,yN+nTbH))< <Log2(nTbW)、 predH(xP,yP)=((nTbH-1-(xP-xN))*recon(xN-1,yP)+(xP-xN+1)*recon(xN+nTbW,yN-1))< <Log2(nTbH) Here, as shown in FIG. 16, the coordinates of the top-left sample of the current picture block are (xN, yN), recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yP), and recon(xN+nTbW, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yN+nTbH), (xN-1, yP), and (xN+nTbW, yN-1), respectively, and nTbW and nTbH represent the width and height of the current CU (current picture block).

[0496] In another possible implementation, the predicted value of the target sample is updated according to the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) Here, predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the picture block to be processed are (0,0), and predP(xP,yP) is updated. where p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1), respectively; w1 and w2 are preset constants; and nTbW and nTbH represent the width and height of the picture block to be processed.

[0497] In another possible implementation, the predicted value of the target sample is updated according to the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Here, predV(xP, yP) = ((nTbH - 1 - yP) * p(xP, -1) + (yP + 1) * p(-1, nTbH)) << Log2(nTbW), predH(xP, yP) = ((nTbW - 1 - xP) * p(-1, yP) + (xP + 1) * p(nTbW, -1)) << Log2(nTbH), where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (0, 0), predP(xP, yP) represents the predicted value of the target sample before update, predQ(xP, yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values of the reference samples at (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1) respectively, w1 and w2 are pre-set constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0498] Note that the planar mode algorithm used to generate the second predicted sample value predP1(xP, yP) is not limited to the algorithm in VTM, and the planar algorithms in HEVC or H.264 can be alternatively used. This is not limited in this embodiment of the present application.

[0499] Note that the values of the weighting factors w1 and w2 are not limited in this embodiment of the present application. For example, the weighting factor set (w1, w2) may be a combination of values of w1 + w2, and w1 + w2 is equal to an integer power of 2. In this way, the division operation is no longer performed. For example, combinations of values such as (6, 2), (5, 3), or (4, 4) can be used. This is not limited in this embodiment of the present application, and the combinations of values listed in this specification are used only as examples for explanation.

[0500] In another possible implementation, the predicted value of the target sample can be updated according to the following formula. predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) Where: predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+1)>>1, predV(xP,yP)=((nTbH-1-(yP-yN))*recon(xP,yN-1)+(yP-yN+1)*recon(xN-1,yN+nTbH)+(nTbH>>1))>>Log2(nTbH), predH(xP,yP)=((nTbW-1-(xP-xN))*recon(xN-1,yP)+(xP-xN+1)*recon(xN+nTbW,yN-1)+(nTbW>>1))>>Log2(nTbW) where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yP), and recon(xN+nTbW, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yN+nTbH), (xN-1, yP), and (xN+nTbW, yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

[0501] In another possible implementation, the inter-prediction block may be processed by using a position-dependent intra prediction combination process technique used in intra prediction, and the updated predicted value predQ(xP, yP) of the target sample may be obtained by using a DC mode method in the intra prediction combination process technique in VTM. It can be understood that when the DC mode is used, the predicted value of the current block may be obtained based on the average value of the reference samples to the left and above the current block.

[0502] For example, the updated predicted value predQ(xP, yP) of the target sample may be obtained according to the following equation:

[0503]

number

[0504] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), and the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the picture block to be processed are (xP,yP), as shown in FIG. where the target is (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0505] It should be noted that the updated predicted value predQ(xP, yP) of the target sample can be obtained not only by using the intra-prediction combining processing technique in VTM, but also by using the algorithm in JEM.

[0506] In one possible implementation, the inter-prediction block may be processed by using the position-dependent intra-prediction combining processing technique used in intra-prediction, and the updated predicted value predQ(xP, yP) of the target sample may be obtained by using the PLANAR mode method in the intra-prediction combining processing technique in VTM. The following formula is used:

[0507]

number

[0508] Here, refL(xP, yP) = recon(xN-1, yP), refT(xP, yP) = recon(xP, yN-1), wT(yP) = 32>> ((yP<<1)>> nScale), wL(xP) = 32>> ((xP<<1)>> nScale), nScale = ((Log2(nTbW) + Log2(nTbH) -2)>> 2), and as shown in FIG. 16, the coordinates of the target sample are (xP, yP), and the coordinates of the top-left sample of the picture block to be processed are ( where predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-1, yP) and recon(xP, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-1, yP) and (xP, yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0509] It should be noted that the updated predicted value predQ(xP, yP) of the target sample can be obtained by using the algorithm in JEM instead of only using the algorithm in VTM.

[0510] In one possible implementation, the filtering process may be performed on the inter-prediction samples by using an intra-prediction boundary filtering technique. For details on the boundary filtering technique, please refer to the HEVC method. The details will not be described here.

[0511] Please note that when the predicted value of the target sample is updated in any one of the aforementioned manners, if the reconstructed value of the target sample is unavailable, step S1306 may further include the following steps S13601 and S13062.

[0512] S13061: When a reconstructed value of a reference sample is unavailable, determine the availability of adjacent samples above and to the left of the picture block to be processed in a predetermined order until a predetermined amount of available reference samples is obtained.

[0513] It can be understood that the case in which the reconstructed value of the reference sample is unavailable may be when the picture block to be processed is located at the top of the picture, and there is no reconstructed value of the reference sample at the coordinate position (xP, yN-M), or when the picture block to be processed is located at the left side of the picture, and there is no reconstructed value of the reference sample at the coordinate position (xN-N, yP), or when the reconstructed value cannot be obtained for the reference sample. The specific case in which the reconstructed value of the reference sample is unavailable is not limited in this embodiment of the present application, and this case is used here simply as an example for explanation.

[0514] In one implementation, as shown in FIG. 17, the preset order may be from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), and then from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). For example, to find available reference samples from the upper and left neighboring samples of the picture block to be processed, all samples may be covered in the order from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), and then from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). It should be noted that the specific preset order is not limited in this embodiment of the present application, and the preset order in this specification is merely used as an example for explanation.

[0515] For example, when there is at least one available reference sample among all reference samples, if the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is unavailable, the available samples are searched in a pre-set order from the coordinate (xN-1, yN+nTbH-1) to the coordinate (xN-1, yN-1), then from the coordinate (xN, yN-1) to the coordinate (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of the sample (x, y). If the reconstructed value of reference sample (x,y) is unavailable in the set containing reference sample (xN-1,yN+nTbH-M), then the reconstructed value of reference sample (x,y) is set to the reconstructed value of sample (x,y+1), where M is greater than or equal to 2 and less than or equal to nTbH+1. If the reconstructed value of reference sample (x,y) is unavailable in the set containing reference sample (xN+N,yN-1), then the reconstructed value of reference sample (x,y) is set to the reconstructed value of reference sample (x-1,y), where N is greater than or equal to 0 and less than or equal to nTbW-1.

[0516] For example, if a reconstructed value of a reference sample (xN-1, yN+nTbH-M) is unavailable, available reference samples may be searched in a pre-set order starting from coordinates (xN-1, yN+nTbH-M), where M is 1 or more and nTbH+1 or less. If the available reference sample is B, the reconstructed value of reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of reference sample B. If a reconstructed value of a reference sample with coordinates (xN+N, yN-1) is unavailable, available reference samples may be searched in a pre-set order starting from coordinates (xN+N, yN-1), where N is 0 or more and nTbW-1 or less. If the available reference sample is C, the reconstructed value of reference sample (xN+N, yN-1) may be set to the reconstructed value of reference sample C.

[0517] For example, if the reconstructed value of the reference sample (xN-1, yN+nTbH-3) is unavailable, the availability of the upper and left neighboring samples of the current picture block may be determined in the order from coordinate (xN-1, yN+nTbH-3) to coordinate (xN-1, yN-1) until a preset amount of available reference samples is obtained, and the reconstructed value of the reference sample (xN-1, yN+nTbH-3) may be set to the reconstructed value of the available reference sample. If the reconstructed value of the reference sample with coordinate (xN+3, yN-1) is unavailable, the available samples may be searched in the order from coordinate (xN+3, yN-1) to coordinate (xN+nTbW-1, yN-1), and the reconstructed value of the reference sample (xN+3, yN-1) may be set to the reconstructed value of the available reference sample.

[0518] For example, if the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a pre-set order from coordinates (xN-1, yN+nTbH-1) to coordinates (xN-1, yN-1), then from coordinates (xN, yN-1) to coordinates (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of sample (x, y). If the reconstructed value of the reference sample (xN-1, yN+nTbH-M) is unavailable, available reference samples may be searched in the reverse order to the pre-set order starting from coordinates (xN-1, yN+nTbH-M), where M is greater than 1 and less than or equal to nTbH+1. If the available reference sample is C, then the reconstructed value of reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of reference sample C. If the reconstructed value of the reference sample with coordinates (xN+N, yN-1) is unavailable, then the available reference samples may be searched in a reverse order to the pre-set order starting from coordinates (xN+N, yN-1), where N is greater than or equal to 0 and less than or equal to nTbW-1. If the available reference sample is D, then the reconstructed value of reference sample (xN+N, yN-1) may be set to the reconstructed value of reference sample D.

[0519] For example, if the reconstructed value of the reference sample (xN-1, yN+nTbH-3) is unavailable, the availability of the upper and left neighboring samples of the current picture block may be determined in the order from coordinate (xN-1, yN+nTbH-3) to coordinate (xN-1, yN+nTbH-1) until a preset amount of available reference samples is obtained, and the reconstructed value of the reference sample (xN-1, yN+nTbH-3) may be set to the reconstructed value of the available reference sample. If the reconstructed value of the reference sample with coordinate (xN+3, yN-1) is unavailable, the available samples may be searched in the order from coordinate (xN+3, yN-1) to coordinate (xN, yN-1), and the reconstructed value of the reference sample (xN+3, yN-1) may be set to the reconstructed value of the available reference sample.

[0520] It should be noted that the new reference sample may be the first available reference sample found in the preset order, or may be any available reference sample found in the preset order, which is not limited in this embodiment of the present application.

[0521] It can be understood that according to the method, when a reference sample is unavailable, an available reference sample may be searched for in a pre-set order from adjacent samples to the upper and left of the picture block to be processed, and the reconstructed value of the available reference sample is used as the reconstructed value of the unavailable reference sample.

[0522] S13062: To update the predicted value of the target sample, a weighted calculation is performed on the reconstructed values ​​of the available reference samples and the predicted value of the target sample.

[0523] For example, the predicted value of the target sample may be updated based on the reconstructed value of the new reference sample by using any one of the implementations described above.

[0524] If the reconstructed value of the reference sample is unavailable, in step S13061, it is determined that all the upper and left adjacent samples of the current picture block are unavailable, and the reconstructed value of the reference sample may be set to 1<<(bitDepth-1), where it is noted that bitDepth represents the bit depth of the sample value of the reference sample. For example, when the current picture block is located at the upper left corner of the picture, the coordinate of the upper left corner of the current picture block is (0,0), and all the upper and left adjacent samples of the current picture block are unavailable. In this case, the reconstructed value of the reference sample corresponding to the target sample in the current picture block may be set to 1<<(bitDepth-1).

[0525] In the above implementations, in the process of generating inter-prediction samples, spatial filtering is performed on the inter-prediction samples. Compared with the conventional technology, coding efficiency is improved.

[0526] In one possible implementation, before step S1306, the method may further include step S1305.

[0527] S1305: (Optionally) perform a filtering process on the reference samples.

[0528] It can be understood that step S1305 may be performed by the filter unit 206 in the video decoder of FIG.

[0529] For example, performing a filtering process on a reference sample may include performing a weighted calculation on a reconstructed value of the reference sample and reconstructed values ​​of neighboring samples to the left and right of the reference sample when the reference sample is located above the picture block to be processed, performing a weighted calculation on the reconstructed value of the reference sample and reconstructed values ​​of neighboring samples above and below the reference sample when the reference sample is located to the left of the picture block to be processed, and updating the reconstructed value of the reference sample by using the result of the weighted calculation.

[0530] It can be understood that after the filtering process is performed on the reference sample in step S1305, when step S1306 is executed, a weighted calculation can be performed on the updated reconstructed value of the reference sample after the filtering process and the predicted value of the target sample in order to update the predicted value of the target sample.

[0531] It should be noted that for a specific method for performing the filtering process on the reference samples, reference can be made to the filtering method in step S1306, and the details will not be described here.

[0532] It can be seen that in order to update the reconstructed value of the reference sample, a filtering process is performed on the reference sample, and a filtering process is performed on the target sample based on the updated reconstructed value of the reference sample. This can further improve coding efficiency and reduce prediction residual.

[0533] In one possible implementation, step S1307 may be further included before or after step S1306.

[0534] S1307: (Optionally) continue to perform inter prediction based on the motion information and bitstream information by using an inter coding technique other than this method.

[0535] It may be understood that step S1307 may be performed by the inter predictor 210 in the video decoder of FIG.

[0536] For example, techniques in HEVC or VTM may be used, including, but not limited to, bidirectional optical flow, decoder-side motion vector refinement, local illumination compensation (LIC) technique, generalized weighted prediction (GBI), overlapped block motion compensation (OBMC), and decoder-side motion vector compensation (DMVD) technique. Methods in HEVC or VTM, or other methods for generating a motion vector prediction candidate list may be used. This is not limited in this embodiment of the present application.

[0537] It should be noted that the order of performing steps S1301 to S1307 of the above method is not limited in this embodiment of the present application. For example, step S1305 may be performed before step S1307 or after step S1307. This is not limited in this embodiment of the present application.

[0538] In one possible implementation form, before performing motion compensation on the picture block to be processed based on the motion information, the method may further include a step of first updating the motion information by using a first pre-set algorithm, and correspondingly, the step of performing motion compensation on the picture block to be processed based on the motion information includes a step of first performing motion compensation on the picture block to be processed based on the updated motion information.

[0539] In another possible implementation form, after the predictive block of the picture block to be processed is obtained, the method may further include a step of pre-updating the predictive block by using a second pre-set algorithm, and correspondingly, the step of performing a weighted calculation on one or more reconstructed values ​​of the one or more reference samples and the predicted value of the target sample includes a step of performing a weighted calculation on one or more reconstructed values ​​of the one or more reference samples and the pre-updated predicted value of the target sample.

[0540] In another possible implementation, after performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, the method further includes updating the predicted value of the target sample by using a second pre-set algorithm.

[0541] It should be further understood that after the updated predicted value of the target sample is obtained, the method may further include adding the final inter-predicted picture and the residual picture to obtain a reconstructed picture of the current block. Specifically, if the current block has a residual, the residual information is added to the predicted picture to obtain a reconstructed picture of the current block. If the current block does not have a residual, the predicted picture is the reconstructed picture of the current block. In the above process, the same method as in HEVC or VTM, or another motion compensation method or picture reconstruction method may be used. This is not limited.

[0542] According to the inter prediction method provided in this embodiment of the present application, the bitstream is analyzed to obtain motion information of the current picture block, and motion compensation is performed on the current picture block based on the motion information to obtain a prediction block of the current picture block, and a weighted calculation is performed on one or more reconstructed values ​​of one or more reference samples and the prediction value of the target sample to update the prediction value of the target sample, and the reference sample has a preset spatial relationship with the target sample. In this embodiment of the present application, after the prediction value of the target sample in the current picture block is obtained, a filtering process is performed on the prediction value of the target sample by using the neighboring reconstructed samples. In this way, the coding compression efficiency can be improved, and the PSNR BDrate is reduced by 0.5%. Compared with the conventional technology, the spatial filtering performed on the inter prediction sample in the process of generating the inter prediction sample improves the coding efficiency.

[0543] An embodiment of the present application provides an inter prediction device. The device may be a video decoder. Specifically, the inter prediction device is configured to perform steps performed by a decoding device in the above-mentioned inter prediction method. The inter prediction device provided in this embodiment of the present application may include modules corresponding to corresponding steps.

[0544] In this embodiment of the present application, the inter-prediction device can be divided into functional modules according to the above-mentioned method examples. For example, the functional modules can be obtained through division based on corresponding functions, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware, or in the form of software function modules. In this embodiment of the present application, the division into modules is an example, and is merely a logical division of functions, and may be other divisions in actual implementation.

[0545] When the function modules are obtained through division based on the corresponding functions, Figure 18 is a possible schematic structural diagram of the inter prediction device 1800 of the above embodiment. As shown in Figure 18, the inter prediction device 1800 may include an analysis module 1801, a compensation module 1802 and a calculation module 1803. Specifically, the functions of the modules are as follows:

[0546] The parsing module 1801 is configured to parse the bitstream to obtain motion information of a current picture block.

[0547] The compensation module 1802 is configured to perform motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, where the prediction block of the current picture block comprises a prediction value of a target sample.

[0548] The calculation module 1803 is configured to perform a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, the reference samples having a pre-set spatial relationship with the target sample.

[0549] In one possible implementation, the one or more reference samples include a reconstructed sample having the same horizontal coordinate as the target sample and a preset vertical coordinate difference from the target sample, or a reconstructed sample having the same vertical coordinate as the target sample and a preset horizontal coordinate difference from the target sample.

[0550] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula:

[0551]

number

[0552] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are preset constants, and M1 and M2 are preset positive integers.

[0553] In one possible implementation, w1+w2=R, w3+w4=R, or w5+w6+w7=R, where R is the nth power of 2 and n is a non-negative integer.

[0554] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula:

[0555]

number

[0556] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and rec on(xP, yN-M4) represents the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0557] In one possible implementation, w1+w2+w3=S, w4+w5+w6=S, or w7+w8+w9+w10+w11=S, where S is the nth power of 2 and n is a non-negative integer.

[0558] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula:

[0559]

number

[0560] Here, the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are preset constants, and M1 and M2 are preset positive integers.

[0561] In one possible implementation, w1+w2+w3=R, where R is the nth power of 2, and n is a non-negative integer.

[0562] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula:

[0563]

number

[0564] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, w1, w2, w3, w4, and w5 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0565] In one possible implementation, w1+w2+w3+w4+w5=S, where S is the nth power of 2, and n is a non-negative integer.

[0566] In one possible implementation, the one or more reference samples include one or more of the following samples: a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the top of the picture...

Claims

1. 1. An inter prediction method, comprising: analyzing the bitstream to obtain motion information of a picture block to be processed; performing motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, where the prediction block of the current picture block includes a prediction value of a target sample; performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, the reference samples having a predefined spatial relationship with the target sample; The method includes:

2. The one or more reference samples a reconstructed sample having the same horizontal coordinate as the target sample and a pre-defined vertical coordinate difference with respect to the target sample; or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference with respect to the target sample; 2. The method of claim 1, comprising:

3. said step of updating said predicted value of said target sample further comprising: performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed values ​​of the reference samples to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [0010] 3. The method of claim 2, wherein the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are preset constants, and M1 and M2 are preset positive integers.

4. 4. The method of claim 3, wherein w1+w2=R1, w3+w4=R2, or w5+w6+w7=R3, and R1, R2, and R3 are each a power of 2, where n is a non-negative integer.

5. said step of updating said predicted value of said target sample further comprising: performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed values ​​of the reference samples to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [0025] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and 3. The method of claim 2, wherein recon(xP, yN-M4) represents reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

6. 6. The method of claim 5, wherein w1+w2+w3=S1, w4+w5+w6=S2, or w7+w8+w9+w10+w11=S3, and S1, S2, and S3 are each a power of 2, where n is a non-negative integer.

7. said step of updating said predicted value of said target sample further comprising: performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed values ​​of the reference samples to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [0030] 3. The method of claim 2, wherein the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are preset constants, and M1 and M2 are preset positive integers.

8. 8. The method of claim 7, wherein w1+w2+w3=R, where R is the nth power of 2, and n is a non-negative integer.

9. said step of updating said predicted value of said target sample further comprising: performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed values ​​of the reference samples to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [0045] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, 3. The method of claim 2, wherein recon(xP, yN-M3), recon(xN-M4), and recon(xP, yN-M4) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, and w5 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

10. 10. The method of claim 9, wherein w1+w2+w3+w4+w5=S, where S is the nth power of 2, and n is a non-negative integer.

11. The one or more reference samples a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the upper side of the current picture block; A reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left side of the picture block to be processed, A reconstructed sample at the upper right of the picture block to be processed, A reconstructed sample at the lower left of the picture block to be processed, or, A reconstructed sample at the upper left of the picture block to be processed The method according to claim 1, comprising one or more of the above.

12. The step of updating the predicted value of the target sample is Performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample Including, The updated predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) where predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values of the reference sample at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1) respectively, w1 and w2 are pre-set constants, nTbW and nTbH represent the width and height of the picture block to be processed, or, The predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the current picture block are (0,0), and predP(xP,yP) is the previous value before the update. where p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), respectively; w1, w2, and w3 are preset constants; and nTbW and nTbH represent the width and height of the current picture block; or The predicted value of the target sample is expressed by the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Updated according to, predV(xP,yP)=((nTbH - 1 - yP)*p(xP, - 1)+(yP + 1)*p(-1, nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW - 1 - xP)*p(-1, yP)+(xP + 1)*p(nTbW, - 1))<<Log2(nTbH), where the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP, - 1), p(-1, nTbH), p(-1, yP), and p(nTbW, - 1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP, - 1), (-1, nTbH), (-1, yP), and (nTbW, - 1), w1, w2, and w3 are preset constants, and nTbW and nTbH represent the width and height of the picture block to be processed, the method according to claim 11. [

13. ] The step of updating the predicted value of the target sample is executing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample including, the updated predicted value of the target sample is obtained according to the following formula [0050] predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+1)>>1, predV(xP,yP)=((nTbH - 1 - (yP - yN))*recon(xP, yN - 1)+(yP - yN + 1)*recon(xN - 1, yN + nTbH)+(nTbH>>1))>>Log2(nTbH), predH(xP,yP)=((nTbW - 1 - (xP - xN))*recon(xN - 1, yP)+(xP - xN + 1)*recon(xN + nTbW, yN - 1)+(nTbW>>1))>>Log2(nTbW) ​ where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yN+nTbH), 12. The method of claim 11, wherein recon(xN+nTbW,yN-1) represents reconstructed values ​​of the reference samples at coordinate positions (xP,yN-1), (xN-1,yN+nTbH), (xN-1,yP), and (xN+nTbW,yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

14. 14. The method of claim 12 or 13, wherein the sum of w1 and w2 is a power of 2, where n is a non-negative integer.

15. said step of updating said predicted value of said target sample further comprising: performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed values ​​of the reference samples to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [006] and refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the processing target picture block are (xN,yN), and pr 12. The method of claim 11, wherein edP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent reconstructed values ​​of the reference samples at coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents a clipping operation.

16. said step of updating said predicted value of said target sample further comprising: performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed values ​​of the reference samples to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [0070] where refL(xP, yP)=recon(xN-1, yP), refT(xP, yP)=recon(xP, yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), and predP 12. The method of claim 11, wherein (xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-1, yP) and recon(xP, yN-1) represent reconstructed values ​​of the reference samples at coordinate positions (xN-1, yP) and (xP, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents a clipping operation.

17. said step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and said predicted value of said target sample, when the reconstructed value of the reference sample is unavailable, determining availability of upper and left neighboring samples of the current picture block in a predetermined order until a predetermined amount of available reference samples is obtained; performing a weighted calculation on the reconstructed values ​​of the available reference samples and the predicted values ​​of the target samples; 17. The method of any one of claims 1 to 16, comprising:

18. said step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample in the current picture block, performing a weighted calculation on the reconstructed value of the reference sample and reconstructed values ​​of a left neighboring sample and a right neighboring sample of the reference sample when the reference sample is located above the current picture block; performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of an upper neighboring sample and a lower neighboring sample of the reference sample when the reference sample is located to the left of the current picture block; updating the reconstructed values ​​of the reference samples by using the results of the weighted calculations; 18. The method of any one of claims 1 to 17, comprising:

19. before the step of performing motion compensation on the current picture block based on the motion information, initially updating the motion information by using a first pre-defined algorithm; Further comprising: Correspondingly, the step of performing motion compensation on the current picture block based on the motion information comprises: first performing motion compensation on the current picture block based on the updated motion information; 19. The method of any one of claims 1 to 18, comprising:

20. After the prediction block of the current picture block is obtained, the method further comprises: pre-updating the prediction block by using a second pre-configured algorithm; Further comprising: Correspondingly, said step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and said predicted value of said target sample comprises: performing a weighted calculation on the one or more reconstructed values ​​of the one or more reference samples and the previously updated predicted values ​​of the target sample.

20. The method of any one of claims 1 to 19, comprising:

21. after the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, updating the predicted value of the target sample by using a second pre-defined algorithm.

20. The method of any one of claims 1 to 19, further comprising:

22. Prior to the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, analyzing the bitstream to obtain a prediction mode corresponding to the current picture block; determining that the prediction mode is a merge mode and / or an inter evolved motion vector prediction (inter AMVP) mode; 22. The method of any one of claims 1 to 21, further comprising:

23. Prior to the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, analyzing the bitstream to obtain update determination instruction information for the current picture block; determining that the update decision indication information is used to indicate updating the prediction block of the current picture block; 23. The method of any one of claims 1 to 22, further comprising:

24. Prior to the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, obtaining predetermined update determination instruction information for the picture block to be processed; determining that the update decision indication information is used to indicate updating the prediction block of the current picture block; 23. The method of any one of claims 1 to 22, further comprising:

25. 1. A method of prediction comprising: obtaining a first predicted value of a target sample through inter prediction; obtaining a second predicted value of the target sample through intra prediction; performing a weighting calculation on the first predicted value and the second predicted value to obtain an updated predicted value of the target sample, wherein a weighting factor of the first predicted value is different from a weighting factor of the second predicted value; The method includes:

26. The method of claim 25, wherein the second prediction value is obtained based on spatial neighboring samples by using a PLANAR intra prediction mode.

27. The step of obtaining a first predicted value of a target sample through inter prediction includes: analyzing the bitstream to obtain motion information of a picture block to be processed; performing motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, where the prediction block of the current picture block includes the first prediction value of the target sample; 26. The method of claim 25, comprising:

28. The updated prediction value predQ(xP, yP) is obtained based on w1*predP(xP, yP) and w2*predP1(xP, yP), 28. The method according to claim 25, wherein (xP, yP) represents the coefficients of the target sample, predP(xP, yP) represents the first predicted value, predP1(xP, yP) represents the second predicted value, w1 represents the weighting coefficient of the first predicted value, w2 represents the weighting coefficient of the second predicted value, w1 and w2 are preset constants, and w1 is not equal to w2.

29. 29. The method of claim 28, wherein the set of weighting coefficients (w1, w2) is (6, 2) or (5, 3).

30. 29. The method of claim 28, wherein the sum of w1 and w2 is a power of 2, where n is a non-negative integer.

31. An inter prediction device, an analysis module configured to analyze the bitstream to obtain motion information of a current picture block; a compensation module configured to perform motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, where the prediction block of the current picture block includes a prediction value of a target sample; and a calculation module configured to perform a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, the reference samples having a predefined spatial relationship with the target sample; An apparatus comprising:

32. The one or more reference samples a reconstructed sample having the same horizontal coordinate as the target sample and a pre-defined vertical coordinate difference with respect to the target sample; or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference with respect to the target sample; 32. The apparatus of claim 31 , comprising:

33. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [0080] wherein coordinates of the target sample are (xP, yP), coordinates of a top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are preset constants, and M1 and M2 are preset positive integers.

34. 34. The apparatus of claim 33, wherein w1+w2=R1, w3+w4=R2, or w5+w6+w7=R3, and R1, R2, and R3 are each a power of 2, where n is a non-negative integer.

35. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [0097] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and r 33. The apparatus of claim 32, wherein econ(xP, yN-M4) represents reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

36. 36. The apparatus of claim 35, wherein w1+w2+w3=S1, w4+w5+w6=S2, or w7+w8+w9+w10+w11=S3, and S1, S2, and S3 are each a power of 2, where n is a non-negative integer.

37. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [0010] wherein coordinates of the target sample are (xP, yP), coordinates of a top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are preset constants, and M1 and M2 are preset positive integers.

38. 38. The apparatus of claim 37, wherein w1+w2+w3=R, where R is the nth power of 2, and n is a non-negative integer.

39. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: ##EQU00011## where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, y 33. The apparatus of claim 32, wherein recon(xP, yN-M4), recon(xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and recon(xP, yN-M4) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, and w5 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

40. 40. The apparatus of claim 39, wherein w1+w2+w3+w4+w5=S, where S is a power of 2 and n is a non-negative integer.

41. The one or more reference samples a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the upper side of the current picture block; a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left of the current picture block; the top right reconstructed sample of the current picture block; the bottom-left reconstructed sample of the current picture block, or The top-left reconstructed sample of the current picture block 32. The apparatus of claim 31 , comprising one or more of:

42. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample; The updated predicted value of the target sample is expressed by the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) obtained according to predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), where the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), w1 and w2 are preset constants, nTbW and nTbH represent the width and height of the picture block to be processed, or, the predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the current picture block are (0,0), and predP(xP,yP) is the sample before the update. p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), respectively; w1 and w2 are preset constants; and nTbW and nTbH represent the width and height of the current picture block; or The predicted value of the target sample is expressed by the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Updated according to, predV(xP,yP)=((nTbH - 1 - yP)*p(xP, -1)+(yP + 1)*p(-1, nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW - 1 - xP)*p(-1, yP)+(xP + 1)*p(nTbW, -1))<<Log2(nTbH), where the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1), w1, w2, and w3 are preset constants, and nTbW and nTbH represent the width and height of the picture block to be processed, the apparatus according to claim 41.

43. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula ##EQU00012## obtained according to predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+1)>>1, predV(xP,yP)=((nTbH - 1 - (yP - yN))*recon(xP, yN - 1)+(yP - yN + 1)*recon(xN - 1, yN + nTbH)+(nTbH>>1))>>Log2(nTbH), predH(xP,yP)=((nTbW - 1 - (xP - xN))*recon(xN - 1, yP)+(xP - xN + 1)*recon(xN + nTbW, yN - 1)+(nTbW>>1))>>Log2(nTbW) where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yN+nTbH), 42. The apparatus of claim 41, wherein recon(xN+nTbW,yN-1) represents reconstructed values ​​of the reference samples at coordinate positions (xP,yN-1), (xN-1,yN+nTbH), (xN-1,yP), and (xN+nTbW,yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

44. 44. The apparatus of claim 42 or 43, wherein the sum of w1 and w2 is a power of 2, where n is a non-negative integer.

45. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: ##EQU00013## and refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the processing target picture block are (xN,yN), and pr 42. The apparatus of claim 41, wherein edP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent reconstructed values ​​of the reference samples at coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent width and height of the current picture block, and clip1Cmp represents a clipping operation.

46. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: ##EQU14## where refL(xP, yP)=recon(xN-1, yP), refT(xP, yP)=recon(xP, yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), and predP 42. The apparatus of claim 41, wherein (xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-1, yP) and recon(xP, yN-1) represent reconstructed values ​​of the reference samples at coordinate positions (xN-1, yP) and (xP, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents a clipping operation.

47. The computing module: when the reconstructed value of the reference sample is unavailable, determining availability of upper and left neighboring samples of the current picture block in a predetermined order until a predetermined amount of available reference samples is obtained; performing a weighted calculation on the reconstructed values ​​of the available reference samples and the predicted values ​​of the target samples; 47. The apparatus of any one of claims 31 to 46, further configured to:

48. The computing module: performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of a left neighboring sample and a right neighboring sample of the reference sample when the reference sample is located above the current picture block; performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of an upper neighboring sample and a lower neighboring sample of the reference sample when the reference sample is located to the left of the current picture block; updating the reconstructed values ​​of the reference samples by using the results of the weighted calculations 48. The apparatus of any one of claims 31 to 47, further configured to:

49. the calculation module is further configured to initially update the motion information by using a first pre-defined algorithm; The apparatus of claim 31 , wherein the compensation module is further configured to correspondingly perform motion compensation on the current picture block based on the first updated motion information.

50. The calculation module is further configured to pre-update the prediction block by using a second pre-configured algorithm; 50. The apparatus of claim 31, wherein the calculation module is further configured to perform a weighting calculation on the one or more reconstructed values ​​of the one or more reference samples and the previously updated predicted value of the target sample.

51. 50. The apparatus of claim 31, wherein the calculation module is further configured to update the predicted value of the target sample by using a second pre-configured algorithm.

52. The analysis module: Analyzing the bitstream to obtain a prediction mode corresponding to the current picture block; determining that the prediction mode is a merge mode and / or an inter-evolutionary motion vector prediction (inter AMVP) mode; 52. The apparatus of any one of claims 31 to 51, further configured to:

53. The analysis module: Analyzing the bitstream to obtain update determination instruction information for the processing target picture block; determining that the update decision indication information is used to indicate updating the prediction block of the current picture block; 53. The apparatus of any one of claims 31 to 52, further configured to:

54. The computing module: Obtaining predetermined update determination instruction information for the picture block to be processed; determining that the update decision indication information is used to indicate updating the prediction block of the current picture block; 53. The apparatus of any one of claims 31 to 52, further configured to:

55. 55. The apparatus of any one of claims 31 to 54, wherein weighting coefficients of the one or more reconstructed values ​​of the one or more reference samples differ from weighting coefficients of the predicted values ​​of the target sample.

56. 56. The apparatus of claim 31, wherein the one or more reconstructed values ​​of the one or more reference samples are obtained based on spatial neighboring samples by using a PLANAR intra prediction mode.

57. The updated prediction value predQ(xP, yP) is obtained based on w1*predP(xP, yP) and w2*predP1(xP, yP), 57. The apparatus of claim 31, wherein (xP, yP) represents coefficients of the target sample, predP(xP, yP) represents the predicted value of the target sample, predP1(xP, yP) represents the one or more reconstructed values ​​of the one or more reference samples, w1 represents a weighting coefficient of the one or more reconstructed values ​​of the one or more reference samples, and w2 represents a weighting coefficient of the predicted value of the target sample, where w1 and w2 are preset constants, and w1 is not equal to w2.

58. 58. The apparatus of claim 57, wherein the set of weighting coefficients (w1, w2) is (6, 2) or (5, 3).

59. 58. The apparatus of claim 57, wherein the sum of w1 and w2 is a power of 2, where n is 2 or 3.

60. 31. A computer storage medium having stored thereon computer program code which, when executed on a processor, enables the processor to carry out a method according to any one of claims 1 to 30.

Images