Inter prediction method and apparatus

Abstract

The present invention relates to an inter-prediction method and apparatus. [Solution] The embodiments of the present application relate to the field of video coding technology and disclose an inter-prediction method and apparatus for solving the problems in the prior art that predicted samples obtained in inter-prediction mode are spatially discontinuous, which affects prediction efficiency and results in relatively high prediction residual energy. A specific solution includes: analyzing a bitstream to obtain motion information of a current picture block; performing motion compensation on the current picture block according to the motion information to obtain a predicted block of the current picture block, where the predicted block of the current picture block includes a predicted value of a target sample; and performing weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, where the reference sample has a predetermined spatial position relationship with the target sample.

Description

[Technical Field]

[0001] TECHNICAL FIELD Embodiments of the present application relate to the field of video coding technology, and in particular to an inter prediction method and apparatus. [Background technology]

[0002] Digital video technologies may be widely used in a variety of digital video devices. Digital video devices may implement video coding techniques, such as those described in standards defined in MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG4 Part 10 Advanced Video Coding (AVC), ITU-T H.265 (also known as High Efficiency Video Coding, HEVC), and extensions to these standards. Digital video devices implement these video coding techniques to more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0003] Currently, inter-prediction and intra-prediction techniques are mainly used in video coding to eliminate temporal and spatial redundancy in video. Inter-prediction techniques only consider the temporal correlation between the same objects in adjacent frames of a picture, but do not consider the spatial correlation. As a result, the prediction samples obtained in existing inter-prediction modes are spatially discontinuous. Furthermore, prediction efficiency is affected, and the prediction residual energy is relatively large. Summary of the Invention [Means for solving the problem]

[0004] The embodiments of the present application provide an inter-prediction method and apparatus for performing spatial filtering on an inter-coded prediction block to improve coding efficiency.

[0005] To achieve this goal, the following technical solutions are used in the embodiments of this application.

[0006] According to a first aspect of an embodiment of the present application, there is provided an inter-prediction method, which includes: analyzing a bitstream to obtain motion information of a current picture block; performing motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, where the prediction block of the current picture block includes a prediction value of a target sample; and performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the prediction value of the target sample to update the prediction value of the target sample, where the reference sample has a predetermined spatial relationship with the target sample.

[0007] According to this solution, a spatial filtering process is performed on the predicted value of the target sample by using neighboring reconstructed samples to improve the compression coding efficiency.

[0008] Referring to the first aspect, in one possible implementation, the one or more reference samples include a reconstructed sample having the same horizontal coordinate as the target sample and a predetermined vertical coordinate difference from the target sample, or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference from the target sample.

[0009] According to this solution, a filtering process is performed on a target sample by using a reference sample that has a predetermined spatial relationship with the target sample, which improves coding efficiency compared with conventional techniques.

[0010] Referring to the first aspect and the above-mentioned possible implementations, in another possible implementation, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0011]

number

[0012] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are predetermined constants, and M1 and M2 are predetermined positive integers.

[0013] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0014] Referring to the first aspect and the possible implementations described above, in another possible implementation, w1+w2=R1, w3+w4=R2, or w5+w6+w7=R3, where R1, R2, and R3 are each a power of 2, and n is a non-negative integer.

[0015] Based on this solution, the coding efficiency can be further improved.

[0016] It should be understood that R1, R2, and R3 are each the nth power of 2. R1, R2, and R3 may be the same or different, and this is not limited thereto. For example, R1, R2, and R3 may all be 8, or R1, R2, and R3 may be 2, 4, and 16, respectively.

[0017] Referring to the first aspect and the above-mentioned possible implementations, in another possible implementation, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0018]

number

[0019] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and rec on(xP,yN-M4) represent the reconstructed values ​​of the reference samples at coordinate positions (xN-M1,yP), (xN-M2,yP), (xP,yN-M3), and (xP,yN-M4), respectively, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0020] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0021] Referring to the first aspect and the possible implementations described above, in another possible implementation, w1+w2+w3=S1, w4+w5+w6=S2, or w7+w8+w9+w10+w11=S3, where S1, S2, and S3 are each a power of 2, and n is a non-negative integer.

[0022] Based on this solution, the coding efficiency can be further improved.

[0023] It should be understood that S1, S2, and S3 are each the nth power of 2. S1, S2, and S3 may be the same or different, and this is not limited. For example, S1, S2, and S3 may all be 8, or S1, S2, and S3 may be 2, 4, and 16, respectively.

[0024] Referring to the first aspect and the above-mentioned possible implementations, in another possible implementation, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0025]

number

[0026] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are predetermined constants, and M1 and M2 are predetermined positive integers.

[0027] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0028] Referring to the first aspect and possible implementations described above, in another possible implementation, w1+w2+w3=R, where R is the nth power of 2 and n is a non-negative integer.

[0029] Referring to the first aspect and the above-mentioned possible implementations, in another possible implementation, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0030]

number

[0031] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, w1, w2, w3, w4, and w5 are predetermined constants, and M1, M2, M3, and M4 are predetermined positive integers.

[0032] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0033] Referring to the first aspect and possible implementations described above, in another possible implementation, w1+w2+w3+w4+w5=S, where S is the nth power of 2 and n is a non-negative integer.

[0034] Based on this solution, the coding efficiency can be further improved.

[0035] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the one or more reference samples include one or more of the following samples: a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the top side of the picture block to be processed, a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left side of the picture block to be processed, a reconstructed sample to the top right of the picture block to be processed, a reconstructed sample to the bottom left of the picture block to be processed, or a reconstructed sample to the top left of the picture block to be processed.

[0036] According to this solution, a filtering process is performed on a target sample by using a reference sample that has a predetermined spatial relationship with the target sample, which improves coding efficiency compared with conventional techniques.

[0037] Referring to the first aspect and the possible implementations described above, in another possible implementation, the step of updating the predicted value of the target sample includes performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is calculated according to the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) Obtained according to, predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0038] In a feasible implementation form of the first aspect, the predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the current picture block are (0,0), and predP(xP,yP) is updated according to the above formula. where p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), respectively; w1, w2, and w3 are preset constants; and nTbW and nTbH represent the width and height of the picture block to be processed.

[0039] In a possible implementation of the first aspect, the predicted value of the target sample is calculated using the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Updated according to the following, predV(xP, yP) = ((nTbH - 1 - yP) * p(xP, -1) + (yP + 1) * p(-1, nTbH)) << Log2(nTbW), predH(xP, yP) = ((nTbW - 1 - xP) * p(-1, yP) + (xP + 1) * p(nTbW, -1)) << Log2(nTbH), the coordinates of the target sample are (xP, yP), the coordinates of the upper left sample of the picture block to be processed are (0, 0), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1) respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0040] Based on this solution, the updated predicted value of the target sample can be obtained through filtering processing.

[0041] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of updating the predicted value of the target sample includes performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed values of the reference samples to obtain the updated predicted value of the target sample. The updated predicted value of the target sample is given by the following formula predQ(xP, yP) = (w1 * predP(xP, yP) + w2 * predP1(xP, yP) + ((w1 + w2) / 2)) / (w1 + w2) and is obtained according to predP1(xP, yP) = (predV(xP, yP) + predH(xP, yP) + 1) >> 1, predV(xP, yP) = ((nTbH - 1 - (yP - yN)) * recon(xP, yN - 1) + (yP - yN + 1) * recon(xN - 1, yN + nTbH) + (nTbH >> 1)) >> Log2(nTbH), predH(xP,yP)=((nTbW-1-(xP-xN))*recon(xN-1,yP)+(xP-xN+1)*recon(xN+nTbW,yN-1)+(nTbW>>1))>>Log2(nTbW) where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yP), and recon(xN+nTbW, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yN+nTbH), (xN-1, yP), and (xN+nTbW, yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

[0042] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0043] Referring to the first aspect and possible implementations described above, in another possible implementation, the sum of w1 and w2 is 2 to the nth power, where n is a non-negative integer.

[0044] Based on this solution, the coding efficiency can be further improved.

[0045] Referring to the first aspect and the above-mentioned possible implementations, in another possible implementation, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0046]

number

[0047] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0048] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0049] Referring to the first aspect and the above-mentioned possible implementations, in another possible implementation, the step of updating the predicted value of the target sample includes a step of performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0050]

number

[0051] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the picture block to be processed are (xN,yN). where predP(xP,yP) represents the predicted value of the target sample before updating, predQ(xP,yP) represents the updated predicted value of the target sample, recon(xN-1,yP) and recon(xP,yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-1,yP) and (xP,yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0052] Based on this solution, an updated predicted value of the target sample can be obtained through a filtering process.

[0053] Referring to the first aspect and the above-mentioned possible implementation forms, in another possible implementation form, the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample includes, when the reconstructed value of the reference sample is unavailable, determining the availability of samples adjacent to the upper and left sides of the picture block to be processed in a predetermined order until a predetermined amount of available reference samples is obtained, and performing a weighted calculation on the reconstructed values ​​of the available reference samples and the predicted value of the target sample.

[0054] According to this solution, when the reconstructed value of a reference sample is unavailable, the reference samples located to the left and above the current picture block and whose reconstructed values ​​are available can be searched in a predetermined order. In this way, the predicted value of the target sample can be updated by using the reconstructed values ​​of the available reference samples.

[0055] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, the step of determining the availability of samples adjacent to the upper and left sides of the picture block to be processed in a predetermined order until a predetermined amount of available reference samples is obtained includes the step of obtaining available reference samples in the order from coordinates (xN-1, yN+nTbH-1) to coordinates (xN-1, yN-1), and from coordinates (xN, yN-1) to coordinates (xN+nTbW-1, yN-1).

[0056] Based on this solution, reconstructed values ​​of available reference samples can be obtained.

[0057] Referring to the first aspect and the possible implementations described above, in another possible implementation, when there is at least one available reference sample among all reference samples, if the reconstructed value of reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a predetermined order from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), and from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of sample (x, y). If the reconstructed value of reference sample (x, y) is unavailable in the set containing reference sample (xN-1, yN+nTbH-M), then the reconstructed value of reference sample (x, y) is set to the reconstructed value of sample (x, y+1), where M is greater than or equal to 2 and less than or equal to nTbH+1. If the reconstructed value of reference sample (x, y) is unavailable in the set containing reference sample (xN+N, yN-1), then the reconstructed value of reference sample (x, y) is set to the reconstructed value of reference sample (x-1, y), where N is greater than or equal to 0 and less than or equal to nTbW-1.

[0058] Based on this solution, reconstructed values ​​of available reference samples can be obtained.

[0059] Referring to the first aspect and the possible implementations described above, in another possible implementation, if the reconstructed value of reference sample (xN-1, yN+nTbH-M) is unavailable, available reference samples may be searched in a preset order starting from coordinate (xN-1, yN+nTbH-M), where M is greater than or equal to 1 and less than or equal to nTbH+1. If the available reference sample is B, the reconstructed value of reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of reference sample B. If the reconstructed value of the reference sample at coordinate (xN+N, yN-1) is unavailable, available reference samples may be searched in a preset order starting from coordinate (xN+N, yN-1), where N is greater than or equal to 0 and less than or equal to nTbW-1. If the available reference sample is C, the reconstructed value of reference sample (xN+N, yN-1) may be set to the reconstructed value of reference sample C.

[0060] Based on this solution, reconstructed values ​​of available reference samples can be obtained.

[0061] Referring to the first aspect and the possible implementations described above, in another possible implementation, if the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a predetermined order from the coordinate (xN-1, yN+nTbH-1) to the coordinate (xN-1, yN-1), and from the coordinate (xN, yN-1) to the coordinate (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of the sample (x, y). If the reconstructed value of reference sample (xN-1, yN+nTbH-M) is unavailable, available reference samples may be searched in reverse of the preset order starting from coordinate (xN-1, yN+nTbH-M), where M is greater than or equal to 1 and less than or equal to nTbH+1. If the available reference sample is C, the reconstructed value of reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of reference sample C. If the reconstructed value of the reference sample at coordinate (xN+N, yN-1) is unavailable, available reference samples may be searched in reverse of the preset order starting from coordinate (xN+N, yN-1), where N is greater than or equal to 0 and less than or equal to nTbW-1. If the available reference sample is D, the reconstructed value of reference sample (xN+N, yN-1) may be set to the reconstructed value of reference sample D.

[0062] Based on this solution, reconstructed values ​​of available reference samples can be obtained.

[0063] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, if it is determined that all adjacent samples to the upper and left of the picture block to be processed are unavailable, the reconstructed value of the reference sample is set to 1<<(bitDepth-1), where bitDepth represents the bit depth of the sample value of the reference sample.

[0064] Based on this solution, the reconstructed value of the reference sample may be set based on the bit depth when both the reconstructed value of the reference sample and the reconstructed value of the new reference sample are unavailable.

[0065] Referring to the first aspect and the above-mentioned possible implementation forms, in another possible implementation form, before performing weighted calculations on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, the method includes the steps of: when the reference sample is located above the picture block to be processed, performing weighted calculations on the reconstructed value of the reference sample and the reconstructed values ​​of neighboring samples to the left and right of the reference sample; when the reference sample is located to the left of the picture block to be processed, performing weighted calculations on the reconstructed value of the reference sample and the reconstructed values ​​of neighboring samples above and below the reference sample; and updating the reconstructed value of the reference sample by using the result of the weighted calculations.

[0066] According to this solution, before the filtering operation is performed on the target samples, the filtering operation is performed on the reconstructed values ​​of the reference samples, thus further improving the coding efficiency and reducing the prediction residual.

[0067] Referring to the first aspect and the above-mentioned possible implementation forms, in another possible implementation form, before performing motion compensation on the picture block to be processed based on the motion information, the method further includes a step of first updating the motion information by using a first predetermined algorithm, and correspondingly, the step of performing motion compensation on the picture block to be processed based on the motion information includes a step of first performing motion compensation on the picture block to be processed based on the updated motion information.

[0068] According to this solution, the motion information is updated before the motion compensation is performed on the current block, and the motion compensation is performed based on the updated motion information. In this way, the prediction residual can be reduced.

[0069] Referring to the first aspect and the above-mentioned possible implementation forms, in another possible implementation form, after the prediction block of the picture block to be processed is obtained, the method further includes a step of pre-updating the prediction block by using a second pre-set algorithm, and correspondingly, the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the prediction value of the target sample includes a step of performing a weighted calculation on one or more reconstructed values ​​of the one or more reference samples and the pre-updated prediction value of the target sample.

[0070] According to this solution, the prediction block of the current block is pre-updated, and weighted calculations are performed based on the pre-updated prediction value and the reconstructed value of the reference sample, thus reducing the prediction residual.

[0071] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, after performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, the method further includes a step of updating the predicted value of the target sample by using a second preset algorithm.

[0072] Based on this solution, the predicted value of the target sample after spatial filtering can be updated by using a preset algorithm, thus reducing the prediction residual.

[0073] With reference to the first aspect and the possible implementations described above, in another possible implementation, before performing weighted calculations on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, the method further includes: analyzing the bitstream to obtain a prediction mode corresponding to the picture block to be processed, and determining that the prediction mode is a merge mode and / or an inter-evolutionary motion vector prediction (inter AMVP) mode. It may be understood that the inter-evolutionary motion vector prediction (inter AMVP) mode may also be referred to as an inter-motion vector prediction (inter MVP) mode.

[0074] According to this solution, the prediction mode corresponding to the current picture block can be determined before the filtering process.

[0075] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, before performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, the method further includes a step of analyzing the bitstream to obtain update decision indication information for the picture block to be processed, and a step of determining that the update decision indication information is used to indicate updating the predicted block of the picture block to be processed.

[0076] According to this solution, the update decision indication information of the current picture block can be obtained by analyzing the bitstream, and it is determined that the prediction block of the current picture block should be updated.

[0077] Referring to the first aspect and the possible implementation forms described above, in another possible implementation form, before performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, the method further includes a step of obtaining predetermined update decision indication information for the picture block to be processed, and a step of determining that the update decision indication information is used to indicate updating the predicted block of the picture block to be processed.

[0078] According to this solution, update decision indication information of the current picture block can be obtained, and it is determined that the prediction block of the current picture block should be updated based on the update decision indication information.

[0079] According to a second aspect of an embodiment of the present application, an inter prediction device is provided, which includes: an analysis module configured to analyze a bitstream to obtain motion information of a picture block to be processed; a compensation module configured to perform motion compensation on the picture block to be processed based on the motion information to obtain a prediction block of the picture block to be processed, where the prediction block of the picture block to be processed includes a prediction value of a target sample; and a calculation module configured to perform a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the prediction value of the target sample to update the prediction value of the target sample, where the reference sample has a predetermined spatial positional relationship with the target sample.

[0080] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the one or more reference samples include a reconstructed sample having the same horizontal coordinate as the target sample and a predetermined vertical coordinate difference from the target sample, or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference from the target sample.

[0081] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is particularly configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0082]

number

[0083] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are predetermined constants, and M1 and M2 are predetermined positive integers.

[0084] Referring to the second aspect and the possible implementations described above, in another possible implementation, w1+w2=R1, w3+w4=R2, or w5+w6+w7=R3, where R1, R2, and R3 are each a power of 2, and n is a non-negative integer.

[0085] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is further configured, in particular, to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0086]

number

[0087] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and rec on(xP,yN-M4) represent the reconstructed values ​​of the reference samples at coordinate positions (xN-M1,yP), (xN-M2,yP), (xP,yN-M3), and (xP,yN-M4), respectively, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0088] Referring to the second aspect and the possible implementations described above, in another possible implementation, w1+w2+w3=S1, w4+w5+w6=S2, or w7+w8+w9+w10+w11=S3, where S1, S2, and S3 are each a power of 2, and n is a non-negative integer.

[0089] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is further configured, in particular, to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is obtained according to the following formula:

[0090]

number

[0091] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are predetermined constants, and M1 and M2 are predetermined positive integers.

[0092] Referring to the second aspect and possible implementations described above, in another possible implementation, w1+w2+w3=R, where R is the nth power of 2, and n is a non-negative integer.

[0093] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is further configured, in particular, to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0094]

number

[0095] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, w1, w2, w3, w4, and w5 are predetermined constants, and M1, M2, M3, and M4 are predetermined positive integers.

[0096] Referring to the second aspect and possible implementations described above, in another possible implementation, w1+w2+w3+w4+w5=S, where S is the nth power of 2 and n is a non-negative integer.

[0097] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the one or more reference samples include one or more of the following samples: a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the top side of the picture block to be processed, a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left side of the picture block to be processed, a reconstructed sample to the top right of the picture block to be processed, a reconstructed sample to the bottom left of the picture block to be processed, or a reconstructed sample to the top left of the picture block to be processed.

[0098] Referring to the second aspect and the possible implementations described above, in another possible implementation, the calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) Obtained according to, predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), w1 and w2 are pre-set constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0099] In a realizable implementation form of the second aspect, the predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the current picture block are (0,0), and predP(xP,yP) is updated according to the above formula. where p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), respectively; w1, w2, and w3 are preset constants; and nTbW and nTbH represent the width and height of the picture block to be processed.

[0100] In a possible implementation of the second aspect, the predicted value of the target sample is calculated using the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Updated according to, predV(xP,yP) = ((nTbH - 1 - yP)*p(xP, -1) + (yP + 1)*p(-1, nTbH)) << Log2(nTbW), predH(xP,yP) = ((nTbW - 1 - xP)*p(-1, yP) + (xP + 1)*p(nTbW, -1)) << Log2(nTbH), the coordinates of the target sample are (xP,yP), the coordinates of the top - left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1) respectively, w1 and w2 are pre - set constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0101] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to perform a weighted calculation based on, in particular, the predicted value of the target sample before the update and the reconstructed values of the reference samples to obtain the updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1 + w2) / 2)) / (w1 + w2) is obtained according to, predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+1)>>1, predV(xP,yP)=((nTbH - 1-(yP - yN))*recon(xP,yN - 1)+(yP - yN + 1)*recon(xN - 1,yN + nTbH)+(nTbH>>1))>>Log2(nTbH), predH(xP,yP)=((nTbW-1-(xP-xN))*recon(xN-1,yP)+(xP-xN+1)*recon(xN+nTbW,yN-1)+(nTbW>>1))>>Log2(nTbW) where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yP), and recon(xN+nTbW, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yN+nTbH), (xN-1, yP), and (xN+nTbW, yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

[0102] Referring to the second aspect and possible implementations described above, in another possible implementation, the sum of w1 and w2 is 2 to the nth power, where n is a non-negative integer.

[0103] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is further configured, in particular, to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0104]

number

[0105] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0106] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is further configured, in particular, to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula:

[0107]

number

[0108] Here, refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), and the coordinates of the top-left sample of the picture block to be processed are (xN,yN). where predP(xP,yP) represents the predicted value of the target sample before updating, predQ(xP,yP) represents the updated predicted value of the target sample, recon(xN-1,yP) and recon(xP,yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-1,yP) and (xP,yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0109] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is further configured to: when the reconstructed value of the reference sample is unavailable, determine the availability of adjacent samples above and to the left of the picture block to be processed in a predetermined order until a predetermined amount of available reference samples is obtained, and perform weighted calculations on the reconstructed values ​​of the available reference samples and the predicted values ​​of the target sample.

[0110] With reference to the second aspect and the possible implementation forms mentioned above, in another possible implementation form, the calculation module is particularly configured to obtain reconstructed values ​​of the available reference samples in the order from the coordinates (xN-1,yN+nTbH-1) to the coordinates (xN-1,yN-1) and from the coordinates (xN,yN-1) to the coordinates (xN+nTbW-1,yN-1).

[0111] Referring to the second aspect and the possible implementations described above, in another possible implementation, when there is at least one available reference sample among all reference samples, if the reconstructed value of reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a predetermined order from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), and from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of sample (x, y). If the reconstructed value of reference sample (x, y) is unavailable in the set containing reference sample (xN-1, yN+nTbH-M), then the reconstructed value of reference sample (x, y) is set to the reconstructed value of sample (x, y+1), where M is greater than or equal to 2 and less than or equal to nTbH+1. If the reconstructed value of reference sample (x, y) is unavailable in the set containing reference sample (xN+N, yN-1), then the reconstructed value of reference sample (x, y) is set to the reconstructed value of reference sample (x-1, y), where N is greater than or equal to 0 and less than or equal to nTbW-1.

[0112] Referring to the second aspect and the possible implementations described above, in another possible implementation, the calculation module is particularly configured to: if the reconstructed value of the reference sample (xN-1, yN+nTbH-M) is unavailable, search for available reference samples in a preset order starting from coordinate (xN-1, yN+nTbH-M), where M is greater than or equal to 1 and less than or equal to nTbH+1. If the available reference sample is B, the reconstructed value of the reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of reference sample B. If the reconstructed value of the reference sample at coordinate (xN+N, yN-1) is unavailable, search for available reference samples in a preset order starting from coordinate (xN+N, yN-1), where N is greater than or equal to 0 and less than or equal to nTbW-1. If the available reference sample is C, the reconstructed value of the reference sample (xN+N, yN-1) may be set to the reconstructed value of reference sample C.

[0113] Referring to the second aspect and the possible implementations described above, in another possible implementation, if the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a predetermined order from the coordinate (xN-1, yN+nTbH-1) to the coordinate (xN-1, yN-1), and from the coordinate (xN, yN-1) to the coordinate (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of the reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of the sample (x, y). If the reconstructed value of reference sample (xN-1, yN+nTbH-M) is unavailable, available reference samples may be searched in reverse of the preset order starting from coordinate (xN-1, yN+nTbH-M), where M is greater than or equal to 1 and less than or equal to nTbH+1. If the available reference sample is C, the reconstructed value of reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of reference sample C. If the reconstructed value of the reference sample at coordinate (xN+N, yN-1) is unavailable, available reference samples may be searched in reverse of the preset order starting from coordinate (xN+N, yN-1), where N is greater than or equal to 0 and less than or equal to nTbW-1. If the available reference sample is D, the reconstructed value of reference sample (xN+N, yN-1) may be set to the reconstructed value of reference sample D.

[0114] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, if it is determined that all adjacent samples to the upper and left of the picture block to be processed are unavailable, the reconstructed value of the reference sample is set to 1<<(bitDepth-1), where bitDepth represents the bit depth of the sample value of the reference sample.

[0115] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is further configured to: when the reference sample is located above the picture block to be processed, perform a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of the neighboring sample on the left and right of the reference sample; when the reference sample is located to the left of the picture block to be processed, perform a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of the neighboring sample above and below the reference sample; and update the reconstructed value of the reference sample by using the result of the weighted calculation.

[0116] Referring to the second aspect and the above-mentioned possible implementation forms, in another possible implementation form, the calculation module is further configured to initially update the motion information by using a first pre-set algorithm, and correspondingly, the compensation module is particularly configured to perform motion compensation on the picture block to be processed based on the initially updated motion information.

[0117] Referring to the second aspect and the above-mentioned possible implementations, in another possible implementation, the calculation module is further configured to pre-update the prediction block by using a second pre-set algorithm, and correspondingly, the calculation module is particularly configured to perform a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the pre-updated predicted value of the target sample.

[0118] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the calculation module is further configured to update the predicted value of the target sample by using a second preset algorithm.

[0119] With reference to the second aspect and the above-mentioned possible implementations, in another possible implementation, the analysis module is further configured to analyze the bitstream to obtain a prediction mode corresponding to the picture block to be processed, and determine that the prediction mode is a merge mode and / or an inter-evolutionary motion vector prediction (inter AMVP) mode. It can be understood that the inter-evolutionary motion vector prediction (inter AMVP) mode can also be referred to as an inter-motion vector prediction (inter MVP) mode.

[0120] Referring to the second aspect and the possible implementation forms described above, in another possible implementation form, the analysis module is further configured to analyze the bitstream to obtain update decision indication information for the picture block to be processed, and determine that the update decision indication information is used to indicate updating of the prediction block of the picture block to be processed.

[0121] Referring to the second aspect and the above-mentioned possible implementation forms, in another possible implementation form, the calculation module is further configured to obtain predetermined update decision indication information of the picture block to be processed, and determine that the update decision indication information is used to indicate updating the prediction block of the picture block to be processed.

[0122] According to a third aspect of the present application, there is provided a motion information prediction device, comprising: a processor and a memory coupled to the processor, the processor configured to perform the method according to the first aspect.

[0123] According to a fourth aspect of the present application, there is provided a computer-readable storage medium storing instructions that, when executed on a computer, enable the computer to carry out a method according to the first aspect.

[0124] According to a fifth aspect of the present application there is provided a computer program product comprising instructions which, when executed on a computer, enable the computer to carry out a method according to the first aspect.

[0125] It should be understood that the technical solutions of the second to fifth aspects of the present application are consistent with the technical solutions of the first aspect of the present application. The beneficial effects achieved in all aspects and corresponding possible implementations of all aspects are similar. Therefore, details will not be described again. [Brief explanation of the drawings]

[0126] [Figure 1] FIG. 1 is a block diagram of an example video coding system that may be configured for use in embodiments of the present application. [Figure 2] FIG. 1 is a system block diagram of an example video encoder that may be configured for use in embodiments of the present application. [Figure 3] FIG. 2 is a system block diagram of an example video decoder that may be configured for use in embodiments of the present application. [Figure 4] FIG. 2 is a block diagram of an example inter-prediction module that may be configured for use in embodiments of the present application. [Figure 5] 1 is a flowchart of an example implementation of merge prediction mode. [Figure 6] 10 is a flowchart of an example implementation of an evolved motion vector prediction mode. [Figure 7] 1 is a flowchart of an example implementation of motion compensation performed by a video decoder that may be configured for use in embodiments of the present application. [Figure 8] 1 is a schematic diagram of an example of a coding unit and neighboring picture blocks associated with the coding unit. [Figure 9] 1 is a flowchart of an example implementation of building a motion vector predictor candidate list. [Figure 10]FIG. 10 is a schematic diagram of an example implementation of adding a combined motion vector candidate to a merge mode predicted motion vector candidate list. [Figure 11] FIG. 10 is a schematic diagram of an example implementation of adding a scaled motion vector candidate to a merge mode predicted motion vector candidate list. [Figure 12] FIG. 10 is a schematic diagram of an example implementation of adding a zero motion vector to a merge mode predicted motion vector candidate list. [Figure 13] 1 is a schematic flowchart of an inter prediction method according to an embodiment of the present application; [Figure 14] 1 is a schematic diagram 1 of the application of an inter prediction method according to an embodiment of the present application; [Figure 15] 2 is a schematic diagram 2 of the application of the inter prediction method according to an embodiment of the present application; [Figure 16] 3 is a schematic diagram 3 of the application of the inter prediction method according to an embodiment of the present application; [Figure 17] 4 is a schematic diagram of the application of the inter prediction method according to an embodiment of the present application; [Figure 18] 1 is a schematic block diagram of an inter prediction device according to an embodiment of the present application; [Figure 19] FIG. 2 is a schematic block diagram of another inter prediction device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0127] The following will clearly and completely describe the technical solutions of the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application.

[0128] 1 is a block diagram of an example video coding system according to an embodiment of the present application. As described herein, the term "video coder" generally refers to both a video encoder and a video decoder. In this application, the term "video coding" or "coding" may generally refer to video encoding or video decoding. A video encoder 100 and a video decoder 200 in the video coding system are configured to predict motion information, e.g., a motion vector, of a current coded picture block or a sub-block of the current coded picture block according to various example methods described based on any one of multiple new inter-prediction modes provided in this application, so that the predicted motion vector is maximally close to the motion vector obtained by using a motion estimation method, and no motion vector difference needs to be transmitted during coding. Furthermore, this improves coding performance.

[0129] As shown in FIG. 1 , the video coding system includes a source device 10 and a destination device 20. The source device 10 generates encoded video data. Therefore, the source device 10 may be referred to as a video encoding device. The destination device 20 may decode the encoded video data generated by the source device 10. Therefore, the destination device 20 may be referred to as a video decoding device. In various implementation solutions, the source device 10, the destination device 20, or both, the source device 10 and the destination device 20 may include one or more processors and memory coupled to the one or more processors. As described herein, the memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store required program code in the form of instructions or data structures and that can be accessed by a computer.

[0130] The source device 10 and destination device 20 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handheld telephone sets such as "smart" phones, television sets, cameras, display devices, digital media players, video game consoles, in-vehicle computers, and the like.

[0131] The destination device 20 may receive encoded video data from the source device 10 via a link 30. The link 30 may include one or more media or devices capable of transferring the encoded video data from the source device 10 to the destination device 20. In one example, the link 30 may include one or more communication media that enable the source device 10 to transmit the encoded video data directly to the destination device 20 in real time. In this example, the source device 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the destination device 20. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may constitute part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include a router, a switch, a base station, or another device that facilitates communication from the source device 10 to the destination device 20.

[0132] In another example, the encoded data may be output to storage device 40 through output interface 140. Similarly, the encoded data may be accessed from storage device 40 through input interface 240. Storage device 40 may include any one of a plurality of distributed or locally accessible data storage media, such as hard disk drives, Blu-ray discs, digital video discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium configured to store encoded video data.

[0133] In another example, storage device 40 may correspond to a file server or another intermediate storage device capable of storing encoded video generated by source device 10. Destination device 20 may access the stored video data from storage device 40 through streaming transmission or download. The file server may be any type of server capable of storing encoded video data and transmitting the encoded video data to destination device 20. In one example, the file server includes a network server (e.g., used for websites), a file transfer protocol (FTP) server, a network-attached storage (NAS) device, or a local disk drive. Destination device 20 may access the encoded video data through any standard data connection (including an Internet connection). Standard data connections may include a wireless channel (e.g., a wireless fidelity (Wi-Fi) connection), a wired connection (e.g., a digital subscriber line (DSL) or cable modem), or a combination of wireless and wired channels suitable for accessing encoded video data stored on a file server. The encoded video data may be transmitted from storage device 40 through streaming transmission, download transmission, or a combination thereof.

[0134] The motion vector prediction techniques of the present application are applicable to video coding to support multiple multimedia applications, such as terrestrial television broadcasting, cable television transmission, satellite television transmission, video streaming transmission (e.g., over the Internet), encoding video data stored on a data storage medium, decoding video data stored on a data storage medium, or another application. In some examples, a video coding system may be configured to support unidirectional or bidirectional video transmission to support applications such as video streaming transmission, video playback, video broadcasting, and / or video telephony.

[0135] 1 is merely an example, and the techniques of the present application are applicable to video coding settings (e.g., video encoding or video decoding) that do not necessarily include data communication between an encoder and a decoder. In another example, data is retrieved from local memory, streamed over a network, etc. A video encoder can encode data and store the data in memory, and / or a video decoder can retrieve data from memory and decode the data. In many examples, encoding and decoding are performed by devices that do not communicate with each other but simply encode data and store the data in memory and / or retrieve data from memory and decode the data.

[0136] 1, source device 10 includes video source 120, video encoder 100, and output interface 140. In some examples, output interface 140 may include a modulator / demodulator (modem) and / or a transmitter. Video source 120 may include a video capture device (e.g., a video camera), a video archive containing previously captured video data, a video feed-in interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of the aforementioned video data sources.

[0137] Video encoder 100 may encode video data from video source 120. In some examples, source device 10 transmits the encoded video data directly to destination device 20 through output interface 140. In other examples, the encoded video data may be further stored on storage device 40 so that destination device 20 subsequently accesses the encoded video data for decoding and / or playback.

[0138] 1, destination device 20 includes input interface 240, video decoder 200, and display device 220. In some examples, input interface 240 includes a receiver and / or a modem. Input interface 240 may receive encoded video data via link 30 and / or from storage device 40. Display device 220 may be integrated with destination device 20 or may be located external to destination device 20. Generally, display device 220 displays decoded video data. Display device 220 may include multiple types of display devices, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0139] 1, video encoder 100 and video decoder 200 may be integrated with an audio encoder and decoder, respectively, and may include appropriate multiplexer-demultiplexer units or other hardware and software for encoding both audio and video in the same data stream or separate data streams. In some examples, where applicable, the demultiplexer (MUX-DEMUX) units may conform to the International Telecommunication Union (ITU) H.223 multiplexer protocol or another protocol such as the User Datagram Protocol (UDP).

[0140] The video encoder 100 and the video decoder 200 may each be implemented as any one of multiple circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuits, hardware, or any combination thereof. If the present application is implemented in part using software, the device may store instructions for the software in an appropriate non-volatile computer-readable storage medium and use one or more processors to execute the instructions in hardware to implement the techniques of the present application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered as one or more processors. The video encoder 100 and the video decoder 200 may each be included in one or more encoders or decoders. Either the encoder or decoder may be integrated as part of a combined encoder / decoder (codec) in the corresponding device.

[0141] In this application, video encoder 100 may generally be referred to as a device that “signals” or “transmits” some information to another device, such as video decoder 200. The terms “signaling” or “transmitting” may generally refer to the transmission of syntax elements and / or other data used to decode compressed video data. The transmission may be performed in real time or near real time. Alternatively, the communication may be performed after a period of time, for example, when the syntax elements in the encoded bitstream are stored to a computer-readable storage medium during encoding. A decoding device may then retrieve the syntax elements at any time after the syntax elements are stored to the medium.

[0142] The JCT-VC developed the H.265 high efficiency video coding (HEVC) standard. The HEVC standardization is based on an evolved model of a video decoder, called the HEVC Test Model (HM). The latest H.265 standard document is available at http: / / www.itu.int / rec / T-REC-H.265. The latest version of the standard document is H.265(12 / 16), which is incorporated herein by reference in its entirety. In the HM, video decoders are assumed to have some additional capabilities compared to existing algorithms in ITU-TH.264 / AVC. For example, while H.264 provides nine intra-prediction coding modes, the HM can provide up to 35 intra-prediction coding modes.

[0143] JVET is tasked with developing the H.266 standard. The H.266 standardization process is based on an evolved model of a video decoding device, called the H.266 Test Model. The H.266 algorithm is available at http: / / phenix.int-evry.fr / jvet, and the latest algorithm description is included in JVET-F1001-v2. The algorithm description document is incorporated herein by reference in its entirety. Additionally, reference software for the JEM Test Model is available at http: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / , which is also incorporated herein by reference in its entirety.

[0144] Generally, as described in the HM working model, a video frame or picture may be divided into a sequence of treeblocks or largest coding units (LCUs), each containing both luma and chroma samples. LCUs are also called coding tree units (CTUs). A treeblock has a function similar to that of a macroblock in the H.264 standard. A slice includes several consecutive treeblocks in decoding order. A video frame or picture may be partitioned into one or more slices. Each treeblock may be divided into coding units based on a quadtree. For example, a treeblock, which is the root node of the quadtree, may be divided into four child nodes, each of which may also be a parent node and further divided into four other child nodes. The final, undividable child node, which is the leaf node of the quadtree, contains a decoding node, e.g., a decoded video block. The maximum number of times a treeblock may be divided and the minimum size of a decoding node may be defined in syntax data associated with the decoded bitstream.

[0145] A coding unit includes a decoding node, a prediction unit (PU), and a transform unit (TU) associated with the decoding node. The size of a CU corresponds to the size of the decoding node, and the shape of the CU must be square. The size of a CU may range from 8x8 pixels up to 64x64 pixels, or may be a larger treeblock size. Each CU may include one or more PUs and one or more TUs. For example, syntax data associated with a CU may describe the partitioning of one CU into one or more PUs. The partitioning mode may change when the CU is coded based on a skip mode or a direct mode, an intra-prediction mode, or an inter-prediction mode. PUs obtained through partitioning may have a non-square shape. For example, syntax data associated with a CU may alternatively describe the partitioning of one CU into one or more TUs based on a quadtree. The TUs may have a square or non-square shape.

[0146] The HEVC standard allows for TU-based transforms. TUs may be different for different CUs. The size of a TU is typically set based on the size of the PU within a given CU, which is defined for a partitioned LCU. However, this is not always the case. The size of a TU is typically equal to or less than the size of a PU. In some possible implementations, a quadtree structure called a "residual quadtree" (RQT) may be used to partition residual samples corresponding to a CU into smaller units. Leaf nodes of an RQT may be called TUs. Pixel differences associated with a TU may be transformed to generate transform coefficients, which may be quantized.

[0147] Generally, a PU includes data related to a prediction process. For example, when a PU is coded based on an intra prediction mode, the PU may include data describing the intra prediction mode corresponding to the PU. In another possible implementation, when a PU is coded based on an inter prediction mode, the PU may include data defining a motion vector of the PU. For example, the data defining the motion vector of the PU may describe the horizontal component of the motion vector, the vertical component of the motion vector, the resolution of the motion vector (e.g., 1 / 4 sample accuracy or 1 / 8 sample accuracy), the reference picture to which the motion vector points, and / or the reference picture list of the motion vector (e.g., list 0, list 1, or list C).

[0148] Generally, a transform and quantization process is used for the TU. A given CU, which includes one or more PUs, may also include one or more TUs. After prediction, video encoder 100 may calculate residual values ​​corresponding to the PU. The residual values ​​include pixel differences. The pixel differences may be transformed into transform coefficients, which are quantized and undergo TU traversal to generate serialized transform coefficients for entropy decoding. In this application, the term "video block" is generally used to refer to a decoding node of a CU. In some specific applications, the term "video block" in this application may also be used to refer to a tree block, such as an LCU or CU, which includes a decoding node, a PU, and a TU.

[0149] A video sequence typically includes a series of video frames or pictures. For example, a group of pictures (GOP) includes a series of video pictures, or one or more video pictures. A GOP may include syntax data in the GOP's header information, one or more picture header information, or elsewhere, which describes the amount of pictures included in the GOP. Each slice of a picture may include slice syntax data that describes the coding mode for the corresponding picture. Video encoder 100 typically performs operations on video blocks in video slices to encode video data. Video blocks may correspond to decoding nodes in a CU. The size of a video block may be constant or variable and may change with a specified decoding standard.

[0150] In a possible implementation, the HM supports prediction for PUs with various sizes. Assuming that the size of a given CU is 2Nx2N, the HM supports intra prediction for PUs with sizes of 2Nx2N or NxN, and inter prediction for symmetric PUs with sizes of 2Nx2N, 2NxN, Nx2N, or NxN. The HM also supports asymmetric prediction for inter prediction for PU sizes of 2NxnU, 2NxnD, nLx2N, or nRx2N. In asymmetric partitioning, the CU is not partitioned in one direction and is partitioned into two parts in the other direction, with one part occupying 25% of the CU and the other part occupying 75% of the CU. The part occupying 25% of the CU is indicated by an indicator including "U (Up)", "D (Down)", "L (Left)", or "R (Right)" after "n". So, for example, "2NxnU" refers to a horizontally partitioned 2Nx2N CU with a 2Nx0.5N PU on top and a 2Nx1.5N PU on the bottom.

[0151] As used herein, "NxN" and "N by N" are used interchangeably to refer to the pixel size of a video block in vertical and horizontal dimensions, e.g., 16x16 pixels or 16 by 16 pixels. Generally, a 16x16 block has 16 pixels vertically (y=16) and 16 pixels horizontally (x=16). Similarly, an NxN block has N pixels vertically and N pixels horizontally, where N is a non-negative integer value. The pixels within a block may be arranged in rows and columns. Additionally, the amount of pixels horizontally and the amount of pixels vertically within a block may not necessarily be the same. For example, a block may include NxM pixels, where M is not necessarily equal to N.

[0152] After intra-predictive decoding or inter-predictive decoding is performed on the PUs of a CU, video encoder 100 may calculate residual data of the TUs in the CU. The PUs may include pixel data in the spatial domain (also referred to as the pixel domain). The TUs may include coefficients in the transform domain after a transform (e.g., a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform) is applied to the residual video data. The residual data may correspond to pixel differences between pixels of the uncoded picture and predicted values ​​corresponding to the PU. Video encoder 100 may generate TUs including the residual data of the CU and then transform the TUs to generate transform coefficients for the CU.

[0153] After performing any transforms to generate transform coefficients, video encoder 100 may quantize the transform coefficients. Quantization refers to the process of quantifying coefficients, e.g., to reduce the amount of data used to represent the coefficients and further implement compression. The quantization process can reduce the bit depth associated with some or all of the coefficients. For example, during quantization, an n-bit value may be reduced to an m-bit value through rounding, where n is greater than m.

[0154] The JEM model further improves the video picture coding structure. Specifically, it introduces a block coding structure called the "quadtree plus binary tree" (QTBT) structure. Instead of using concepts like CU, PU, ​​and TU in HEVC, the QTBT structure supports more flexible CU partitioning shapes. CUs can be square or rectangular. Quadtree partitioning is first performed on CTUs, and binary tree partitioning is then performed on the leaf nodes of the quadtree. Additionally, binary tree partitioning has two partitioning modes: symmetric horizontal partitioning and symmetric vertical partitioning. The leaf nodes of the binary tree are called CUs. CUs in the JEM model cannot be further partitioned during prediction and transform. In other words, CUs, PUs, and TUs in the JEM model have the same block size. In the existing JEM model, the maximum CTU size is 256 × 256 luma pixels.

[0155] In some possible implementations, video encoder 100 may scan the quantized transform coefficients in a predetermined scan order to generate a serialized vector that can be entropy coded. In other possible implementations, video encoder 100 may perform adaptive scanning. After scanning the quantized transform coefficients to generate a one-dimensional vector, video encoder 100 may perform entropy coding on the one-dimensional vector by using context-based adaptive variable-length coding (CAVLC), context-based adaptive binary arithmetic coding (CABAC), syntax-based adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method. Video encoder 100 may further perform entropy coding on syntax elements associated with the coded video data in order for video decoder 200 to decode the video data.

[0156] To perform CABAC, video encoder 100 may assign a context in a context model to a symbol to be transmitted. The context may relate to whether neighboring values ​​of the symbol are non-zero. To perform CAVLC, video encoder 100 may select a variable-length code for the symbol to be transmitted. Codewords in variable-length coding (VLC) may be constructed such that shorter codes correspond to more probable symbols and longer codes correspond to less probable symbols. In this way, bitrates can be reduced by using VLC compared to using codewords of equal length for all symbols to be transmitted. Probabilities in CABAC may be determined based on the context assigned to the symbol.

[0157] In this embodiment of the present application, a video encoder may perform inter prediction to reduce temporal redundancy between pictures. As described above, a CU may have one or more prediction units (PUs) according to different video compression coding standards. In other words, multiple PUs may belong to one CU, or a PU and a CU may have the same size. In this specification, when a CU and a PU have the same size, the partitioning mode corresponding to the CU is, for illustration purposes, that the CU is not partitioned or that the CU is partitioned into one PU, and the PU is used uniformly. When a video encoder performs inter prediction, the video encoder may signal motion information of the PU to a video decoder. For example, the motion information of the PU may include a reference picture index, a motion vector, and a prediction direction indicator. The motion vector may indicate the offset between a picture block (also referred to as a video block, pixel block, pixel set, etc.) of the PU and a reference block of the PU. The reference block of the PU may be part of the same reference picture as the picture block of the PU. The reference block may be located in a reference picture indicated by the reference picture index and the prediction direction indicator.

[0158] To reduce the amount of coding bits required to represent the motion information of a PU, a video encoder may generate a motion vector (MV) candidate list for each PU based on a merge prediction mode or an evolved motion vector prediction mode. Each motion vector candidate in the motion vector candidate list for a PU may indicate motion information. The motion information indicated by some motion vector candidates in the motion vector candidate list may be based on the motion information of other PUs. When a motion vector candidate indicates the motion information of one of a spatially predicted motion vector candidate designated at a position or a temporally predicted motion vector candidate designated at a position, the motion vector candidate may be referred to as an "original" motion vector candidate in this application. For example, in a merge mode, also referred to herein as a merge prediction mode, there may be five original spatially predicted motion vector candidates at a position and one original temporally predicted motion vector candidate at a position. In some examples, a video encoder may generate additional motion vector candidates by combining some motion vectors from different original motion vector candidates, modifying the original motion vector candidate, or inserting only a zero motion vector as a motion vector candidate. The additional motion vector candidate predictors are considered as original motion vector candidate predictors and may be referred to in this application as artificially generated motion vector candidate predictors.

[0159] The technology of the present application generally includes a technique for generating a motion vector predictor candidate list on a video encoder and a technique for generating the same motion vector predictor candidate list on a video decoder. The video encoder and the video decoder may generate the same motion vector predictor candidate list by implementing the same technique for constructing the motion vector predictor candidate list. For example, the video encoder and the video decoder may construct a list with the same number of motion vector predictor candidates (e.g., five motion vector predictor candidates). The video encoder and the video decoder may first consider spatial motion vector predictor candidates (e.g., adjacent blocks in the same picture), then consider temporal motion vector predictor candidates (e.g., motion vector predictor candidates in different pictures), and finally consider artificially generated motion vector predictor candidates until a required number of motion vector predictor candidates are added to the list. According to the technology of the present application, during construction of the motion vector predictor candidate list, a pruning operation may be performed on some types of motion vector predictor candidates to remove repetitive motion vector predictor candidates from the motion vector predictor candidate list, while a pruning operation may not be performed on other types of motion vector predictor candidates to reduce decoder complexity. For example, a pruning operation may be performed on a set of spatial motion vector predictor candidates and temporal motion vector predictor candidates to remove motion vector predictor candidates with the same motion information from the motion vector predictor candidate list. However, an artificially generated motion vector predictor candidate may be added to the motion vector predictor candidate list without being pruned.

[0160] After generating the motion vector predictor candidate for the PU of the CU, the video encoder may select a motion vector predictor candidate from the motion vector predictor candidate list and output a motion vector predictor candidate index in the bitstream. The selected motion vector predictor candidate may be the motion vector predictor candidate for generating a motion vector that best matches a predicted value of the target PU being decoded. The motion vector predictor candidate index may indicate a position of the selected motion vector predictor candidate in the motion vector predictor candidate list. The video encoder may further generate a predictive picture block of the PU based on the reference block indicated by the motion information of the PU. The motion information of the PU may be determined based on the motion information indicated by the selected motion vector predictor candidate. For example, in merge mode, the motion information of the PU may be the same as the motion information indicated by the selected motion vector predictor candidate. In AMVP mode, the motion information of the PU may be determined based on the motion vector differential for the PU and the motion information indicated by the selected motion vector predictor candidate. The video encoder may generate one or more residual picture blocks of the CU based on the predictive picture block of the PU of the CU and the original picture block of the CU. The video encoder may then encode the one or more residual picture blocks and output the one or more residual picture blocks in a bitstream.

[0161] The bitstream may include data identifying a selected motion vector predictor candidate in the motion vector predictor candidate list for the PU. The video decoder may determine motion information of the PU based on the motion information indicated by the selected motion vector predictor candidate in the motion vector predictor candidate list for the PU. The video decoder may identify one or more reference blocks for the PU based on the motion information of the PU. After identifying the one or more reference blocks for the PU, the video decoder may generate a predictive picture block for the PU based on the one or more reference blocks of the PU. The video decoder may reconstruct a picture block of the CU based on the predictive picture block of the PU and one or more residual picture blocks of the CU.

[0162] For ease of explanation, in this application, a location or picture block may be described as a location or picture block having various spatial relationships with a CU or PU. This description may be expressed as a location or picture block having various spatial relationships with a picture block associated with a CU or PU. In addition, in this application, a PU currently being decoded by a video decoder may be referred to as a current PU, and may also be referred to as a currently processed picture block. In this application, a CU currently being decoded by a video decoder may be referred to as a current CU. In this application, a picture currently being decoded by a video decoder may be referred to as a current picture. This application is also applicable when a PU and a CU have the same size, or when a PU is a CU. For explanation purposes, a PU is used uniformly.

[0163] As briefly described above, video encoder 100 may generate predicted picture blocks and motion information for a PU of a CU through inter prediction. In many examples, the motion information of a given PU may be identical to or similar to the motion information of one or more neighboring PUs (i.e., PUs whose picture blocks are spatially or temporally adjacent to the picture block of the given PU). Because neighboring PUs often have similar motion information, video encoder 100 may encode the motion information of the given PU based on the motion information of the neighboring PUs. Encoding the motion information of the given PU based on the motion information of the neighboring PUs can reduce the amount of coding bits in the bitstream needed to represent the motion information of the given PU.

[0164] Video encoder 100 may encode the motion information of a given PU based on the motion information of a neighboring PU in various manners. For example, video encoder 100 may indicate that the motion information of a given PU is the same as that of a neighboring PU. In the present application, merge mode may be used to indicate that the motion information of a given PU is the same as or can be derived from that of a neighboring PU. In another possible implementation, video encoder 100 may calculate a motion vector difference (MVD) for a given PU. The MVD indicates the difference between the motion vector of the given PU and the motion vector of a neighboring PU. Video encoder 100 may include the MVD in the motion information of the given PU instead of the motion vector of the given PU. In the bitstream, the amount of coding bits required to represent the MVD is less than the amount of coding bits required to represent the motion vector of the given PU. In the present application, evolved motion vector prediction mode may be used to indicate that the motion information of a given PU is signaled to the decoder side by using the MVD and an index value used to identify a motion vector candidate.

[0165] To signal the motion information of a given PU to the decoder side based on merge mode or AMVP mode, video encoder 100 may generate a motion vector predictor candidate list for the given PU. The motion vector predictor candidate list may include one or more motion vector predictor candidates. Each of the motion vector predictor candidates in the motion vector predictor candidate list for the given PU may indicate motion information. The motion information indicated by each motion vector predictor candidate may include a motion vector, a reference picture index, and a prediction direction indicator. The motion vector predictor candidates in the motion vector predictor candidate list may include the "original" motion vector predictor candidate. Each of the motion vector predictor candidates indicates motion information in one of the specified motion vector predictor candidates at a position within a PU different from the given PU.

[0166] After generating the motion vector predictor candidate list for the PU, video encoder 100 may select one motion vector predictor candidate from the motion vector predictor candidate list to be used for the PU. For example, the video encoder may compare each motion vector predictor candidate with the PU being decoded and may select a motion vector predictor candidate with a required rate-distortion cost. Video encoder 100 may output a motion vector predictor candidate index for the PU. The motion vector predictor candidate index may identify the position of the selected motion vector predictor candidate in the motion vector predictor candidate list.

[0167] In addition, video encoder 100 may generate a predictive picture block of the PU based on a reference block indicated by the motion information of the PU. The motion information of the PU may be determined based on the motion information indicated by a selected predictive motion vector candidate in a predictive motion vector candidate list for the PU. For example, in merge mode, the motion information of the PU may be the same as the motion information indicated by the selected predictive motion vector candidate. In AMVP mode, the motion information of the PU may be determined based on the motion vector differential for the PU and the motion information indicated by the selected predictive motion vector candidate. As described above, video encoder 100 may process the predictive picture block of the PU.

[0168] When video decoder 200 receives the bitstream, video decoder 200 may generate a motion vector predictor candidate list for each PU of a CU. The motion vector predictor candidate list generated by video decoder 200 for a PU may be the same as the motion vector predictor candidate list generated by video encoder 100 for the PU. A syntax element obtained by parsing the bitstream may indicate the position of a selected motion vector predictor candidate in the motion vector predictor candidate list for the PU. After generating the motion vector predictor candidate list for the PU, video decoder 200 may generate a predictive picture block of the PU based on one or more reference blocks indicated by the motion information of the PU. Video decoder 200 may determine the motion information of the PU based on the motion information indicated by the selected motion vector predictor candidate in the motion vector predictor candidate list for the PU. Video decoder 200 may reconstruct a picture block of the CU based on the predictive picture block of the PU and the residual picture block of the CU.

[0169] In one possible implementation, at the decoder side, constructing a predicted motion vector candidate list and analyzing the bitstream to obtain the position of the selected predicted motion vector candidate in the predicted motion vector candidate list are independent of each other and can be performed in any order or in parallel.

[0170] In another possible implementation, at the decoder side, the position of the selected motion vector predictor candidate in the motion vector predictor candidate list is first obtained by analyzing the bitstream, and then the motion vector predictor candidate list is constructed based on the position obtained through analysis.In this implementation, it is not necessary to construct all motion vector predictor candidate lists.Specifically, as long as the motion vector predictor candidate at the position obtained through analysis can be determined, only the motion vector predictor candidate list at that position needs to be constructed.For example, when it is obtained by analyzing the bitstream that the selected motion vector predictor candidate is the motion vector predictor candidate whose index is 3 in the motion vector predictor candidate list, only the motion vector predictor candidate list including the motion vector predictor candidate whose index is 0 to the motion vector predictor candidate whose index is 3 needs to be constructed, and the motion vector predictor candidate whose index is 3 can be determined.This can reduce complexity and improve decoding efficiency.

[0171] 2 is a block diagram of an example video encoder 100 according to an embodiment of the present application. The video encoder 100 is configured to output video to a post-processing entity 41. The post-processing entity 41 represents an example of a video entity that can process encoded video data from the video encoder 100. For example, the video entity is a media-aware network element (MANE) or a splicing / editing device. In some cases, the post-processing entity 41 may be an example of a network entity. In some video coding systems, the post-processing entity 41 and the video encoder 100 may be components of separate devices. In other cases, the functions of the post-processing entity 41 may be performed by the same device that includes the video encoder 100. In one example, the post-processing entity 41 is an example of the storage device 40 of FIG. 1.

[0172] In the example of FIG. 2, video encoder 100 includes prediction processing unit 108, filter unit 106, decoded picture buffer (DPB) 107, summer 112, transformer 101, quantizer 102, and entropy encoder 103. Prediction processing unit 108 includes inter predictor 110 and intra predictor 109. To reconstruct picture blocks, video encoder 100 further includes inverse quantizer 104, inverse transformer 105, and summer 111. Filter unit 106 is intended to represent one or more loop filters, e.g., a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 106 is shown in FIG. 2 as an in-loop filter, in another implementation, filter unit 106 may be implemented as a post-loop filter. In some examples, video encoder 100 may further include a video data memory and a partitioning unit (not shown).

[0173] The video data memory may store video data to be encoded by components of video encoder 100. The video data stored in video encoder 100 may be obtained from video source 120. DPB 107 may be a reference picture memory that stores reference video data used by video encoder 100 to encode video data in intra-coding or inter-coding modes. The video data memory and DPB 107 may include any one of several types of memory devices, such as dynamic random access memory (DRAM), including synchronous dynamic random access memory (SDRAM), magnetic random access memory (MRAM), resistive random access memory (RRAM), or another type of memory device. The video data memory and DPB 107 may be provided by the same memory device or separate memory devices. In various examples, the video data memory may be integrated on a chip with other components of video encoder 100 or may be disposed off-chip relative to those components.

[0174] As shown in FIG. 2, video encoder 100 receives video data and stores the video data in a video data memory. A partition unit partitions the video data into several picture blocks and may further partition the picture blocks into smaller blocks, for example, based on a quadtree structure or a binary tree structure. This partitioning may further include partitioning into slices, tiles, or other larger units. Video encoder 100 is typically a component that encodes picture blocks in a video slice to be encoded. A slice may be partitioned into multiple picture blocks (and into sets of picture blocks called tiles). Prediction processing unit 108 may select one of multiple possible coding modes to be used for the current picture block, for example, one of multiple intra-coding modes or one or more inter-coding modes. Prediction processing unit 108 may provide the obtained intra-coded or inter-coded blocks for adder 112 to generate residual blocks and may provide the obtained intra-coded or inter-coded blocks for adder 111 to reconstruct coded blocks to be used as reference pictures.

[0175] An intra predictor 109 in prediction processing unit 108 may perform intra predictive coding on the current block to be coded relative to one or more neighboring blocks in the same frame or slice as the current picture block to remove spatial redundancy. An inter predictor 110 in prediction processing unit 108 may perform inter predictive coding on the current picture block relative to one or more predictive blocks in one or more reference pictures to remove temporal redundancy.

[0176] Specifically, inter predictor 110 may be configured to determine an inter prediction mode to be used to encode a current picture block. For example, inter predictor 110 may calculate rate-distortion values ​​for various inter prediction modes in an inter prediction mode candidate set through rate-distortion analysis and select an inter prediction mode with optimal rate-distortion characteristics from the inter prediction mode candidate set. Rate-distortion analysis is typically used to determine the amount of distortion (or error) between a coded block and an original uncoded block from which the coded block is generated through encoding, and the bit rate (i.e., the amount of bits) used to generate the coded block. For example, inter predictor 110 may determine that an inter prediction mode in the inter prediction mode candidate set used to encode the current picture block that has the smallest bit rate-distortion cost is the inter prediction mode to be used to perform inter prediction on the current picture block.

[0177] The inter predictor 110 is configured to predict motion information (e.g., motion vectors) of one or more sub-blocks of the current picture block based on the determined inter prediction mode, and obtain or generate a predictive block of the current picture block based on the motion information (e.g., motion vectors) of the one or more sub-blocks of the current picture block. The inter predictor 110 may locate the predictive block indicated by the motion vector in one of the reference picture lists. The inter predictor 110 may further generate syntax elements associated with the picture block and the video slice, so that the video decoder 200 decodes the picture block in the video slice using the syntax elements. Alternatively, in one example, the inter predictor 110 performs a motion compensation process based on the motion information of each sub-block to generate a predictive block of each sub-block and obtain a predictive block of the current picture block. It should be understood that the inter predictor 110 herein performs a motion estimation process and a motion compensation process.

[0178] Specifically, after selecting an inter prediction mode for the current picture block, the inter predictor 110 may provide information indicating the inter prediction mode selected for the current picture block to the entropy encoder 103, so that the entropy encoder 103 encodes the information indicating the selected inter prediction mode.

[0179] The intra predictor 109 may perform intra prediction on the current picture block. It is clear that the intra predictor 109 may determine the intra prediction mode to be used to encode the current block. For example, the intra predictor 109 may calculate rate-distortion values ​​for various test intra prediction modes through rate-distortion analysis and select the intra prediction mode with the optimal rate-distortion characteristics from the test modes. In either case, after selecting the intra prediction mode for the picture block, the intra predictor 109 may provide information indicating the intra prediction mode selected for the current picture block to the entropy encoder 103, so that the entropy encoder 103 encodes the information indicating the selected intra prediction mode.

[0180] After prediction processing unit 108 generates a predictive block for a current picture block through inter-prediction and intra-prediction, video encoder 100 obtains a residual picture block by subtracting the predictive block from the current picture block to be encoded. Adder 112 represents one or more components that perform the subtraction operation. The residual video data in the residual block may be included in one or more TUs and is applied to transformer 101. Transformer 101 converts the residual video data into residual transform coefficients through a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform. Transformer 101 may convert the residual video data from the pixel value domain to a transform domain, for example, the frequency domain.

[0181] The transformer 101 may send the obtained transform coefficients to the quantizer 102. The quantizer 102 quantizes the transform coefficients to further reduce the bit rate. In some examples, the quantizer 102 may further scan a matrix containing the quantized transform coefficients. Alternatively, the entropy encoder 103 may perform the scanning.

[0182] After quantization, entropy encoder 103 performs entropy coding on the quantized transform coefficients. For example, entropy encoder 103 may perform context-adaptive variable length coding (CALVC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioned entropy (PIPE) coding, or another entropy coding method or technique. After performing entropy coding, entropy encoder 103 may transmit the coded bitstream to video decoder 200 or archive the coded bitstream for later transmission or retrieval by video decoder 200. Entropy encoder 103 may also perform entropy coding on syntax elements of the current picture block to be coded.

[0183] The inverse quantizer 104 and the inverse transformer 105 perform inverse quantization and inverse transformation, respectively, to reconstruct the residual block in the pixel domain, e.g., for later use as a reference block of a reference picture. The adder 111 adds the reconstructed residual block to a predictive block generated by the inter predictor 110 or the intra predictor 109 to generate a reconstructed picture block. A filter unit 106 can be applied to the reconstructed picture block to reduce distortion, e.g., block artifacts. The reconstructed picture block is then used as a reference block and stored in the decoded picture buffer 107, and can be used by the inter predictor 110 as a reference block to perform inter prediction on blocks in subsequent video frames or pictures.

[0184] It should be understood that other structural variations of video encoder 100 may be used to encode the video stream. For example, for some picture blocks or picture frames, video encoder 100 may directly quantize the residual signal, eliminating the need for processing by transformer 101 and, correspondingly, by inverse transformer 105. Alternatively, for some picture blocks or picture frames, video encoder 100 does not generate residual data, eliminating the need for processing by transformer 101, quantizer 102, inverse quantizer 104, and inverse transformer 105. Alternatively, video encoder 100 may directly store reconstructed picture blocks as reference blocks, eliminating the need for processing by filter unit 106. Alternatively, quantizer 102 and inverse quantizer 104 in video encoder 100 may be combined.

[0185] 3 is a block diagram of an example video decoder 200 according to an embodiment of the present application. In the example of FIG. 3, the video decoder 200 includes an entropy decoder 203, a prediction processing unit 208, an inverse quantizer 204, an inverse transformer 205, an adder 211, a filter unit 206, and a DPB 207. The prediction processing unit 208 may include an inter predictor 210 and an intra predictor 209. In some examples, the video decoder 200 may perform a decoding process that is generally the inverse of the encoding process described with reference to the video encoder 100 of FIG. 2.

[0186] During decoding, video decoder 200 receives from video encoder 100 an encoded video bitstream representing picture blocks in an encoded video slice and associated syntax elements. Video decoder 200 may receive video data from network entity 42 and, optionally, may further store the video data in a video data memory (not shown). The video data memory may store video data, e.g., an encoded video bitstream, to be decoded by components of video decoder 200. The video data stored in the video data memory may be obtained, for example, from a local video source such as storage device 40, or from a camera through wired or wireless network communication of the video data, or by accessing a physical data storage medium. The video data memory may be used as a decoded picture buffer (CPB) for storing encoded video data from the encoded video bitstream. Thus, although the video data memory is not shown in FIG. 3, the video data memory and DPB 207 may be the same memory or may be separately disposed memories. The video data memory and DPB 207 may each include any one of several types of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetic RAM (MRAM), resistive RAM (RRAM), or another type of memory device. In various examples, the video data memory may be integrated on a chip with other components of video decoder 200 or may be disposed off-chip relative to those components.

[0187] Network entity 42 may be, for example, a server, a MANE, a video editor / splitter, or another device configured to implement one or more of the techniques described above. Network entity 42 may or may not include a video encoder, e.g., video encoder 100. Before network entity 42 transmits the encoded video bitstream to video decoder 200, network entity 42 may implement some of the techniques described in this application. In some video decoding systems, network entity 42 and video decoder 200 may be components of separate devices. In other cases, the functions of network entity 42 may be performed by the same device that includes video decoder 200. In some cases, network entity 42 may be an example of storage device 40 of FIG. 1.

[0188] The entropy decoder 203 of the video decoder 200 performs entropy decoding on the bitstream to generate quantized coefficients and some syntax elements. The entropy decoder 203 forwards the syntax elements to the prediction processing unit 208. The video decoder 200 may receive the syntax elements at the video slice level and / or the picture block level.

[0189] When a video slice is decoded into an intra-decoded (I) slice, intra predictor 209 in prediction processing unit 208 may generate a predictive block of a picture block in the current video slice based on the signaled intra-prediction mode and data of previously decoded blocks of the current frame or picture. When a video slice is decoded into an inter-decoded (i.e., B or P) slice, inter predictor 210 in prediction processing unit 208 may determine an inter-prediction mode to be used to decode the current picture block in the current video slice based on syntax elements received from entropy decoder 203, and decode (e.g., perform inter prediction on) the current picture block based on the determined inter-prediction mode. Specifically, inter predictor 210 may determine whether a new inter-prediction mode will be used to predict the current picture block in the current video slice. When a syntax element indicates that a new inter prediction mode is used to predict a current picture block, motion information of the current picture block or a sub-block of the current picture block in the current video slice is predicted based on the new inter prediction mode (e.g., a new inter prediction mode specified by using the syntax element or a default new inter prediction mode), and the predictive block of the current picture block or a sub-block of the current picture block is obtained or generated by using the predicted motion information of the current picture block or a sub-block of the current picture block in a motion compensation process. The motion information herein may include reference picture information and motion vectors. The reference picture information may include, but is not limited to, uni-prediction / bi-prediction information, a reference picture list number, and a reference picture index corresponding to the reference picture list. In inter prediction, the predictive block may be generated from one of the reference pictures in one of the reference picture lists.Video decoder 200 may construct reference picture lists, namely, List 0 and List 1, based on the reference pictures stored in DPB 207. Reference frame indices for the current picture may be included in one or both of Reference Frame List 0 and Reference Frame List 1. In some examples, video encoder 100 may signal whether a new inter-prediction mode is used to decode a particular syntax element of a particular block, or signal whether and which new inter-prediction mode is used to decode a particular syntax element of a particular block. It should be understood that inter predictor 210 herein performs a motion compensation process.

[0190] The inverse quantizer 204 performs inverse quantization, i.e., dequantizes, the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 203. The inverse quantization process may involve determining the degree of quantization to be applied by using quantization parameters calculated by the video encoder 100 for each picture block in the video slice, and in the same manner determining the degree of inverse quantization to be applied. The inverse transformer 205 performs an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, on the transform coefficients to generate residual blocks in the pixel domain.

[0191] After the inter predictor 210 generates a predictive block for the current picture block or a sub-block of the current picture block, the video decoder 200 adds the residual block from the inverse transformer 205 and the corresponding predictive block generated by the inter predictor 210 to obtain a reconstructed block, i.e., a decoded picture block. The adder 211 represents a component that performs the addition operation. When necessary, a loop filter (either in the decoding loop or after the decoding loop) may be used to further smooth pixel transitions or otherwise improve video quality. The filter unit 206 may represent one or more loop filters, e.g., a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although the filter unit 206 is shown as an in-loop filter in FIG. 2, in another implementation, the filter unit 206 may be implemented as a post-loop filter. In one example, the filter unit 206 can be applied to the reconstructed block to reduce blockiness, and the result is output as a decoded video stream. Additionally, decoded picture blocks in a given frame or picture may be further stored in DPB 207, which stores reference pictures used for subsequent motion compensation. DPB 207 may be part of a memory and may further store decoded video for later presentation on a display device (e.g., display device 220 of FIG. 1). Alternatively, DPB 207 may be separate from such memory.

[0192] It should be understood that other structural variations of the video decoder 200 may be used to decode an encoded video bitstream. For example, the video decoder 200 may generate an output video stream, and no processing by the filter unit 206 is required. Alternatively, for some picture blocks or picture frames, the entropy decoder 203 in the video decoder 200 does not obtain quantized coefficients through decoding, and correspondingly, no processing by the inverse quantizer 204 and the inverse transformer 205 is required.

[0193] As described above, the techniques of the present application relate to, for example, inter-decoding. It should be understood that the techniques of the present application may be performed by any one of the video codecs described in the present application, and that the video decoder includes (for example) the video encoder 100 and video decoder 200 shown and described in FIG. 1 through FIG. 3. Specifically, in one possible implementation, the inter predictor 110 described in FIG. 2 may perform certain techniques described below when inter prediction is performed during encoding of a block of video data. In another possible implementation, the inter predictor 210 described in FIG. 3 may perform certain techniques described below when inter prediction is performed during decoding of a block of video data. Accordingly, references to a "video encoder" or a "video decoder" may include the video encoder 100, the video decoder 200, or another video encoding or decoding unit.

[0194] It should be understood that in the encoder 100 and decoder 200 of the present application, the processing result of a step may be further processed and then output to the next step. For example, after a step such as interpolation filtering, motion vector derivation, or loop filtering, a further operation such as clipping or shifting is performed on the processing result of the corresponding step.

[0195] For example, the value of a motion vector may be constrained to be within a particular bit depth range. Assuming the allowed bit depth of a motion vector is bitDepth, the value of the motion vector ranges from -2^(bitDepth-1) to 2^(bitDepth-1)-1, where the symbol "^" represents exponentiation. If bitDepth is 16, the value ranges from -32768 to 32767. If bitDepth is 18, the value ranges from -131072 to 131071. The value of the motion vector may be constrained in one of the following two ways:

[0196] Method 1: The overflow most significant bit of the motion vector is removed. ux=(vx+2 bitDepth )%2 bitDepth vx=(ux≧2 bitDepth-1 )?(ux-2 bitDepth ):ux uy=(vy+2 bitDepth )%2 bitDepth vy=(uy≧2 bitDepth-1 )?(uy-2 bitDepth ):uy

[0197] For example, suppose the value of vx is -32769, and 32767 is derived according to the formula above. Values ​​are stored in a computer as two's complement numbers, and the two's complement representation of -32769 is 1, 0111, 1111, 1111, 1111 (17 bits), and the process performed by the computer due to overflow is to discard the most significant bits. Therefore, the value of vx is 0111, 1111, 1111, 1111, or 32767. This value is consistent with the result derived through processing according to the formula.

[0198] Method 2: Clipping is performed on the motion vectors, using the following formula: vx=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vx) vy=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vy)

[0199] In the above formula, Clip3 is defined as clipping the value of z to the range [x,y].

[0200]

number

[0201] 4 is a schematic block diagram of the inter prediction module 121 according to an embodiment of the present application. For example, the inter prediction module 121 may include a motion estimation unit and a motion compensation unit. The relationship between PUs and CUs varies with video compression coding standards. The inter prediction module 121 may partition the current CU into PUs based on multiple partition modes. For example, the inter prediction module 121 may partition the current CU into PUs based on 2N×2N, 2N×N, N×2N, and N×N partition modes. In another embodiment, the current CU is the current PU. This is not limited.

[0202] Inter prediction module 121 may perform integer motion estimation (IME) and then fractional motion estimation (FME) for each PU. When inter prediction module 121 performs IME on a PU, inter prediction module 121 may search one or more reference pictures for the PU's reference block. After finding the PU's reference block, inter prediction module 121 may generate a motion vector that indicates the spatial displacement between the PU and the PU's reference block with integer precision. When inter prediction module 121 performs FME on a PU, inter prediction module 121 may refine the motion vector generated by performing IME on the PU. The motion vector generated by performing FME on the PU may have sub-integer precision (e.g., ½ sample precision or ¼ sample precision). After generating the motion vector for the PU, inter prediction module 121 may generate a predicted picture block for the PU by using the PU's motion vector.

[0203] In some possible implementations in which the inter prediction module 121 signals motion information of a PU to the decoder side based on the AMVP mode, the inter prediction module 121 may generate a motion vector predictor candidate list for the PU. The motion vector predictor candidate list may include one or more original motion vector predictor candidates and one or more additional motion vector predictor candidates derived from the one or more original motion vector predictor candidates. After generating the motion vector predictor candidate list for the PU, the inter prediction module 121 may select a motion vector predictor candidate from the motion vector predictor candidate list and generate a motion vector differential (MVD) for the PU. The MVD for the PU may indicate the difference between the motion vector indicated by the selected motion vector predictor candidate and the motion vector generated for the PU through the IME and FME. In these possible implementations, the inter prediction module 121 may output a motion vector predictor candidate index used to locate the selected motion vector predictor candidate in the motion vector predictor candidate list. The inter prediction module 121 may further output the MVD for the PU. The following describes in detail a possible implementation of the Advanced Motion Vector Prediction (AMVP) mode of FIG. 6 in this embodiment of the present application.

[0204] In addition to performing IME and FME on a PU to generate motion information for the PU, inter prediction module 121 may further perform a merge operation on each PU. When inter prediction module 121 performs a merge operation on a PU, inter prediction module 121 may generate a motion vector predictor candidate list for the PU. The motion vector predictor candidate list for the PU may include one or more original motion vector predictor candidates and one or more additional motion vector predictor candidates derived from the one or more original motion vector predictor candidates. The original motion vector predictor candidates in the motion vector predictor candidate list may include one or more spatial motion vector predictor candidates and temporal motion vector predictor candidates. The spatial motion vector predictor candidate may indicate motion information of another PU of the current picture. The temporal motion vector predictor candidate may be based on motion information of a corresponding PU of a picture different from the current picture. The temporal motion vector predictor candidate may also be referred to as temporal motion vector prediction (TMVP).

[0205] After generating the motion vector predictor candidate list, inter prediction module 121 may select one motion vector predictor candidate from the motion vector predictor candidate list. Then, inter prediction module 121 may generate a predicted picture block for the PU based on the reference block indicated by the motion information of the PU. In merge mode, the motion information of the PU may be the same as the motion information indicated by the selected motion vector predictor candidate. Figure 5, described below, is a flowchart of an example of merge mode.

[0206] After generating the predicted picture block of the PU through IME and FME and generating the predicted picture block of the PU through a merge operation, inter prediction module 121 may select the predicted picture block generated through the FME operation or the predicted picture block generated through a merge operation. In some possible implementations, inter prediction module 121 may select the predicted picture block of the PU by analyzing the rate-distortion costs of the predicted picture block generated through the FME operation and the predicted picture block generated through the merge operation.

[0207] After inter prediction module 121 selects a predictive picture block of a PU generated by partitioning the current CU based on each partition mode (in some implementations, after a coding tree unit CTU is divided into CUs, the CU is not further divided into smaller PUs, in which case the PU is equivalent to a CU), inter prediction module 121 may select a partition mode for the current CU. In some implementations, inter prediction module 121 may select a partition mode for the current CU by analyzing a rate-distortion cost of the selected predictive picture block of the PU generated by partitioning the current CU based on each partition mode. Inter prediction module 121 may output a predictive picture block associated with the PU belonging to the selected partition mode to residual generation module 102. Inter prediction module 121 may output a syntax element indicating motion information of the PU belonging to the selected partition mode to the entropy coding module.

[0208] In the schematic diagram of FIG. 4, inter-prediction module 121 includes IME modules 180A to 180N (collectively referred to as "IME modules 180"), FME modules 182A to 182N (collectively referred to as "FME modules 182"), merge modules 184A to 184N (collectively referred to as "merge modules 184"), PU mode decision-making modules 186A to 186N (collectively referred to as "PU mode decision-making modules 186"), and CU mode decision-making module 188 (and may further perform the CTU to CU mode decision-making process).

[0209] IME module 180, FME module 182, and merge module 184 may perform IME operations, FME operations, and merge operations, respectively, on the PUs of the current CU. In the schematic diagram of FIG. 4, inter prediction module 121 is illustrated as including a separate IME module 180, a separate FME module 182, and a separate merge module 184 for each PU in each partition mode for the CU. In another possible implementation, inter prediction module 121 does not include a separate IME module 180, a separate FME module 182, or a separate merge module 184 for each PU in each partition mode for the CU.

[0210] 4, IME module 180A, FME module 182A, and merge module 184A may perform IME operations, FME operations, and merge operations, respectively, on PUs generated by partitioning CUs based on a 2N×2N partition mode. PU mode decision-making module 186A may select one of the predictive picture blocks generated by IME module 180A, FME module 182A, and merge module 184A.

[0211] IME module 180B, FME module 182B, and merge module 184B may perform IME operations, FME operations, and merge operations, respectively, on the left PU generated by partitioning the CU based on the N×2N partition mode. PU mode decision-making module 186B may select one of the predictive picture blocks generated by IME module 180B, FME module 182B, and merge module 184B.

[0212] The IME module 180C, the FME module 182C, and the merge module 184C may perform an IME operation, an FME operation, and a merge operation, respectively, on the right PU generated by partitioning the CU based on the N×2N partition mode. The PU mode decision-making module 186C may select one of the predictive picture blocks generated by the IME module 180C, the FME module 182C, and the merge module 184C.

[0213] The IME module 180N, the FME module 182N, and the merge module 184N may perform an IME operation, an FME operation, and a merge operation, respectively, on the bottom-right PU generated by partitioning the CU based on the N×N partition mode. The PU mode decision-making module 186N may select one of the predictive picture blocks generated by the IME module 180N, the FME module 182N, and the merge module 184N.

[0214] The PU mode decision-making module 186 may select a predicted picture block by analyzing the rate-distortion costs of multiple possible predicted picture blocks and select a predicted picture block that results in an optimal rate-distortion cost in a given decoding scenario. For example, in an application with limited bandwidth, the PU mode decision-making module 186 may prefer to select a predicted picture block with an increased compression ratio, while in another application, the PU mode decision-making module 186 may prefer to select a predicted picture block with an improved quality of the reconstructed video. After the PU mode decision-making module 186 selects a predicted picture block for a PU in the current CU, the CU mode decision-making module 188 selects a partitioning mode for the current CU and outputs the predicted picture block and motion information of the PU belonging to the selected partitioning mode.

[0215] 5 is a flowchart of an implementation of a merge mode according to an embodiment of the present application. A video encoder (e.g., video encoder 100) may perform a merge operation 201. The merge operation 201 may include: S202: generating a candidate list for a current prediction unit; S204: generating a prediction video block associated with a candidate in the candidate list; S206: selecting a candidate from the candidate list; and S208: outputting the candidate. The candidate is a motion vector candidate or a motion information candidate.

[0216] In another possible implementation, the video encoder may perform a merging operation different from merging operation 201. For example, in another possible implementation, the video encoder may perform a merging operation in which the video encoder performs more or fewer steps than, or different steps from, the steps of merging operation 201. In another possible implementation, the video encoder may perform the steps of merging operation 201 in a different order or in parallel. The encoder may further perform merging operation 201 on a PU that is encoded based on a skip mode.

[0217] After the video encoder starts the merging operation 201, the video encoder may generate a motion vector predictor candidate list for the current PU (S202). The video encoder may generate the motion vector predictor candidate list for the current PU in various manners. For example, the video encoder may generate the motion vector predictor candidate list for the current PU by using one of the example techniques described below with reference to Figures 8 to 12.

[0218] As described above, the motion vector predictor candidate list for the current PU may include a temporal motion vector predictor candidate. The temporal motion vector predictor candidate may indicate motion information of a corresponding temporal (co-located) PU. A co-located PU may be spatially located at the same position as the current PU in a picture frame, but may be in a reference picture other than the current picture. In this application, a reference picture including a corresponding temporal PU may be referred to as an associated reference picture. In this application, a reference picture index of an associated reference picture may be referred to as an associated reference picture index. As described above, the current picture may be associated with one or more reference picture lists (e.g., list 0 and list 1). A reference picture index may indicate a reference picture by indicating the position of the reference picture in the reference picture list. In some possible implementations, the current picture may be associated with a combined reference picture list.

[0219] In some video encoders, the associated reference picture index is the reference picture index of the PU that contains the reference index source position associated with the current PU. In these video encoders, the reference index source position associated with the current PU is adjacent to the current PU, to the left of the current PU or above the current PU. In this application, a PU may "contain" a particular position if the picture block associated with the PU contains the particular position. In these video encoders, the video encoder may use reference picture index 0 if the reference index source position is unavailable.

[0220] However, in some examples, the reference index source position associated with the current PU is within the current CU. In these examples, the PU containing the reference index source position associated with the current PU may be considered available if the PU is above or to the left of the current CU. In this case, the video encoder may need to access motion information of another PU of the current CU to determine a reference picture containing a co-located PU. Therefore, these video encoders may use the motion information (i.e., reference picture index) of the PU belonging to the current CU to generate a temporal motion vector predictor candidate for the current PU. In other words, these video encoders may use the motion information of the PU belonging to the current CU to generate a temporal motion vector predictor candidate. Therefore, the video encoder may not be able to generate in parallel the motion vector predictor candidate list for the current PU and the PU containing the reference index source position associated with the current PU.

[0221] According to the technology of the present application, a video encoder may explicitly set an associated reference picture index without referring to the reference picture index of any other PU. In this way, the video encoder can generate motion vector predictor candidate lists for a current PU and another PU of a current CU in parallel. Because the video encoder explicitly sets the associated reference picture index, the associated reference picture index is not based on motion information of any other PU of the current CU. In some possible implementations in which the video encoder explicitly sets the associated reference picture index, the video encoder may always set the associated reference picture index to a fixed, predetermined, and preset reference picture index (e.g., 0). In this way, the video encoder can generate a temporal motion vector predictor candidate based on the motion information of the co-located PU in the reference frame indicated by the preset reference picture index, and the temporal motion vector predictor candidate may be included in the motion vector predictor candidate list for the current CU.

[0222] In a possible implementation in which the video encoder explicitly sets the associated reference picture index, the video encoder may explicitly signal the associated reference picture index in a syntax structure (e.g., a picture header, a slice header, an APS, or another syntax structure). In this possible implementation, the video encoder may signal the associated reference picture index of each LCU (i.e., a CTU), CU, PU, ​​TU, or another type of sub-block to the decoder side. For example, the video encoder may signal that the associated reference picture index of each PU of a CU is equal to “1.”

[0223] In some possible implementations, the associated reference picture index may be set implicitly rather than explicitly. In these possible implementations, the video encoder may generate each temporal motion vector predictor candidate in the motion vector predictor candidate list for a PU of the current CU by using motion information of PUs in reference pictures indicated by reference picture indexes of PUs that encompass positions outside the current CU, even if these positions are not strictly adjacent to the current PU.

[0224] After generating the motion vector predictor candidate list for the current PU, the video encoder may generate a predictive picture block associated with the motion vector predictor candidate in the motion vector predictor candidate list (S204). The video encoder may determine motion information of the current PU based on the motion information of the indicated motion vector predictor candidate, and then generate a predictive picture block based on one or more reference blocks indicated by the motion information of the current PU to generate the predictive picture block associated with the motion vector predictor candidate. The video encoder may then select one motion vector predictor candidate from the motion vector predictor candidate list (S206). The video encoder may select one motion vector predictor candidate in various manners. For example, the video encoder may select one motion vector predictor candidate by analyzing the rate-distortion cost of each of the predictive picture blocks associated with the motion vector predictor candidate.

[0225] After selecting the motion vector predictor candidate, the video encoder may output a motion vector predictor candidate index (S208). The motion vector predictor candidate index may indicate the position of the selected motion vector predictor candidate in the motion vector predictor candidate list. In some possible implementations, the motion vector predictor candidate index may be expressed as "merge_idx".

[0226] 6 is a flowchart of an implementation of an evolved motion vector prediction (AMVP) mode according to an embodiment of the present application. A video encoder (e.g., video encoder 100) may perform AMVP operations 210. The AMVP operations 210 may include: S211: generating one or more motion vectors for a current prediction unit; S212: generating a predictive video block for the current prediction unit; S213: generating a candidate list for the current prediction unit; S214: generating a motion vector differential; S215: selecting a candidate from the candidate list; and S216: outputting a reference picture index, a candidate index, and a motion vector differential used for the selected candidate. The candidates are motion vector candidates or motion information candidates.

[0227] After the video encoder starts AMVP operation 210, the video encoder may generate one or more motion vectors for the current PU (S211). The video encoder may perform integer motion estimation and fractional motion estimation to generate a motion vector for the current PU. As described above, the current picture may be associated with two reference picture lists (list 0 and list 1). If the current PU is unidirectionally predicted, the video encoder may generate a list 0 motion vector or a list 1 motion vector for the current PU. The list 0 motion vector may indicate a spatial offset between a picture block corresponding to the current PU and a reference block in a reference picture in list 0. The list 1 motion vector may indicate a spatial offset between a picture block corresponding to the current PU and a reference block in a reference picture in list 1. If the current PU is bidirectionally predicted, the video encoder may generate a list 0 motion vector and a list 1 motion vector for the current PU.

[0228] After generating the one or more motion vectors for the current PU, the video encoder may generate a predictive picture block for the current PU (S212). The video encoder may generate the predictive picture block for the current PU based on the one or more reference blocks indicated by the one or more motion vectors of the current PU.

[0229] In addition, the video encoder may generate a motion vector predictor candidate list for the current PU (S213). The video encoder may generate the motion vector predictor candidate list for the current PU in various manners. For example, the video encoder may generate the motion vector predictor candidate list for the current PU by using one or more of the possible implementations described below with reference to Figures 8 to 12. In some possible implementations, when the video encoder generates the motion vector predictor candidate list in the AMVP operation 210, the motion vector predictor candidate list may be limited to two motion vector predictor candidates. In contrast, when the video encoder generates the motion vector predictor candidate list in a merge operation, the motion vector predictor candidate list may include more motion vector predictor candidates (e.g., five motion vector predictor candidates).

[0230] After generating the motion vector predictor candidate for the current PU, the video encoder may generate one or more motion vector differentials (MVDs) for each motion vector predictor candidate in the motion vector predictor candidate list (S214). To generate a motion vector differential for the motion vector predictor candidate, the video encoder may determine a difference between the motion vector indicated by the motion vector predictor candidate and the corresponding motion vector of the current PU.

[0231] If the current PU is unidirectionally predicted, the video encoder may generate a single MVD for each motion vector predictor candidate. If the current PU is bidirectionally predicted, the video encoder may generate two MVDs for each motion vector predictor candidate. The first MVD may indicate the difference between the motion vector indicated by the motion vector predictor candidate and the list 0 motion vector of the current PU. The second MVD may indicate the difference between the motion vector indicated by the motion vector predictor candidate and the list 1 motion vector of the current PU.

[0232] The video encoder may select one or more motion vector predictor candidate from the motion vector predictor candidate list (S215). The video encoder may select one or more motion vector predictor candidate in various manners. For example, the video encoder may select a motion vector predictor candidate that matches a motion vector associated with a motion vector to be encoded with a minimum error. This can reduce the amount of bits required to represent the motion vector differential for the motion vector predictor candidate.

[0233] After selecting one or more predictor motion vector candidates, the video encoder may output one or more reference picture indexes for the current PU, one or more predictor motion vector candidate indexes for the current PU, and one or more motion vector differentials for the one or more selected predictor motion vector candidates (S216).

[0234] In an example where the current picture is associated with two reference picture lists (list 0 and list 1) and the current PU is unidirectionally predicted, the video encoder may output a reference picture index for list 0 ("ref_idx_10") or a reference picture index for list 1 ("ref_idx_11"). The video encoder may further output a predictor motion vector candidate index ("mvp_10_flag") that indicates the position of the selected predictor motion vector candidate relative to the list 0 motion vector of the current PU in the predictor motion vector candidate list. Alternatively, the video encoder may output a predictor motion vector candidate index ("mvp_11_flag") that indicates the position of the selected predictor motion vector candidate relative to the list 1 motion vector of the current PU in the predictor motion vector candidate list. The video encoder may further output the MVD for the list 0 motion vector or the list 1 motion vector of the current PU.

[0235] In an example where the current picture is associated with two reference picture lists (list 0 and list 1) and the current PU is bidirectionally predicted, the video encoder may output a reference picture index for list 0 ("ref_idx_10") and a reference picture index for list 1 ("ref_idx_11"). The video encoder may further output a motion vector predictor candidate index ("mvp_10_flag") that indicates the position of the selected motion vector predictor candidate relative to the list 0 motion vector of the current PU in the motion vector predictor candidate list. In addition, the video encoder may output a motion vector predictor candidate index ("mvp_11_flag") that indicates the position of the selected motion vector predictor candidate relative to the list 1 motion vector of the current PU in the motion vector predictor candidate list. The video encoder may further output the MVD for the list 0 motion vector of the current PU and the MVD for the list 1 motion vector of the current PU.

[0236] FIG. 7 is a flowchart of an implementation of motion compensation performed by a video decoder (eg, video decoder 30) according to an embodiment of the present application.

[0237] When the video decoder performs motion compensation operation 220, the video decoder may receive an indication to be used for a selected motion vector candidate predictor of the current PU (S222). For example, the video decoder may receive a motion vector candidate predictor index that indicates the position of the selected motion vector candidate predictor in a motion vector candidate predictor list for the current PU.

[0238] When the motion information of the current PU is encoded based on AMVP mode and the current PU is bidirectionally predicted, the video decoder may receive a first motion vector predictor candidate index and a second motion vector predictor candidate index. The first motion vector predictor candidate index indicates the position of the selected motion vector predictor candidate relative to the list 0 motion vector of the current PU in the motion vector predictor candidate list. The second motion vector predictor candidate index indicates the position of the selected motion vector predictor candidate relative to the list 1 motion vector of the current PU in the motion vector predictor candidate list. In some possible implementations, a single syntax element may be used to identify the two motion vector predictor candidate indexes.

[0239] In addition, the video decoder may generate a motion vector predictor candidate list for the current PU (S224). The video decoder may generate the motion vector predictor candidate list for the current PU in various manners. For example, the video decoder may generate the motion vector predictor candidate list for the current PU by using the techniques described below with reference to Figures 8 to 12. When the video decoder generates the temporal motion vector predictor candidate for the motion vector predictor candidate list, the video decoder may explicitly or implicitly set a reference picture index that identifies a reference picture that includes the co-located PU, as described above with reference to Figure 5.

[0240] After generating the predicted motion vector candidate list for the current PU, the video decoder may determine motion information of the current PU based on motion information indicated by one or more selected predicted motion vector candidates in the predicted motion vector candidate list for the current PU (S225). For example, if the motion information of the current PU is encoded based on merge mode, the motion information of the current PU may be the same as the motion information indicated by the selected predicted motion vector candidate. If the motion information of the current PU is encoded based on AMVP mode, the video decoder may reconstruct one or more motion vectors of the current PU by using one or more motion vectors indicated by the one or more selected predicted motion vector candidates and one or more MVDs indicated in the bitstream. The reference picture index and prediction direction indicator of the current PU may be the same as one or more reference picture indexes and prediction direction indicators of the one or more selected predicted motion vector candidates. After determining the motion information of the current PU, the video decoder may generate a predicted picture block for the current PU based on one or more reference blocks indicated by the motion information of the current PU (S226).

[0241] 8 is a schematic diagram of an example of a coding unit (CU) and neighboring picture blocks associated with the coding unit (CU) according to an embodiment of the present application. FIG. 8 is a schematic diagram for describing a CU 250 and exemplary motion vector predictor candidates at positions 252A to 252E associated with the CU 250. In the present application, the motion vector predictor candidates at positions 252A to 252E may be collectively referred to as the motion vector predictor candidate at position 252. The motion vector predictor candidate at position 252 represents a spatial motion vector predictor candidate in the same picture as the CU 250. The motion vector predictor candidate at position 252A is located to the left of the CU 250. The motion vector predictor candidate at position 252B is located above the CU 250. The motion vector predictor candidate at position 252C is located to the top right of the CU 250. The motion vector predictor candidate at position 252D is located to the bottom left of the CU 250. The motion vector predictor candidate at position 252E is located to the top left of the CU 250. 8 shows a schematic implementation of how the inter prediction module 121 and the motion compensation module may generate a motion vector predictor candidate list. In the following, the implementation is described based on the inter prediction module 121. However, it should be understood that the motion compensation module can implement the same technique and therefore generate the same motion vector predictor candidate list.

[0242] 9 is a flowchart of an implementation of constructing a motion vector predictor candidate list according to an embodiment of the present application. Although the technique of FIG. 9 is described based on a list including five motion vector predictor candidate, the technique described herein may alternatively be used with a list having another size. The five motion vector predictor candidate may each have an index (e.g., 0 to 4). The technique of FIG. 9 is described based on a general video codec. The general video codec may be, for example, a video encoder (e.g., video encoder 100) or a video decoder (e.g., video decoder 30).

[0243] To construct a motion vector predictor candidate list according to the implementation of FIG. 9, the video decoder first considers four spatial motion vector predictor candidates (902). The four spatial motion vector predictor candidates may include motion vector predictor candidates at positions 252A, 252B, 252C, and 252D. The four spatial motion vector predictor candidates may correspond to motion information of four PUs located in the same picture as the current CU (e.g., CU 250). The video decoder may consider the four spatial motion vector predictor candidates in the list in a specified order. For example, the motion vector predictor candidate at position 252A may be considered first. If the motion vector predictor candidate at position 252A is available, index 0 may be assigned to the motion vector predictor candidate at position 252A. If the motion vector predictor candidate at position 252A is unavailable, the video decoder may skip adding the motion vector predictor candidate at position 252A to the motion vector predictor candidate list. The motion vector predictor candidate at position * may be unavailable for various reasons. For example, if the motion vector predictor candidate at the position is not in the current picture, the motion vector predictor candidate at the position may be unavailable. In another possible implementation, if the motion vector predictor candidate at the position is subjected to intra prediction, the motion vector predictor candidate at the position may be unavailable. In another possible implementation, if the motion vector predictor candidate at the position is in a slice different from the slice corresponding to the current CU, the motion vector predictor candidate at the position may be unavailable.

[0244] After considering the motion vector predictor candidate at position 252A, the video decoder may consider the motion vector predictor candidate at position 252B. If the motion vector predictor candidate at position 252B is available and different from the motion vector predictor candidate at position 252A, the video decoder may add the motion vector predictor candidate at position 252B to the motion vector predictor candidate list. In this particular context, the terms "same" or "different" mean that the motion information associated with the motion vector predictor candidate at a position is the same or different. Thus, if two motion vector predictor candidate at a position have the same motion information, the two motion vector predictor candidate at the position are considered to be the same, or if two motion vector predictor candidate at a position have different motion information, the two motion vector predictor candidate at the position are considered to be different. If the motion vector predictor candidate at position 252A is unavailable, the video decoder may assign index 0 to the motion vector predictor candidate at position 252B. If the motion vector predictor candidate at position 252A is available, the video decoder may assign index 1 to the motion vector predictor candidate at position 252B. If the motion vector predictor candidate at location 252B is unavailable or is the same as the motion vector predictor candidate at location 252A, the video decoder skips adding the motion vector predictor candidate at location 252B to the motion vector predictor candidate list.

[0245] Similarly, the video decoder considers the motion vector predictor candidate at position 252C to determine whether to add the motion vector predictor candidate at position 252C to the list. If the motion vector predictor candidate at position 252C is available and different from the motion vector predictor candidate at positions 252B and 252A, the video decoder may assign the next available index to the motion vector predictor candidate at position 252C. If the motion vector predictor candidate at position 252C is unavailable or is the same as at least one of the motion vector predictor candidate at positions 252A and 252B, the video decoder skips adding the motion vector predictor candidate at position 252C to the motion vector predictor candidate list. Next, the video decoder considers the motion vector predictor candidate at position 252D. If the motion vector predictor candidate at position 252D is available and different from the motion vector predictor candidate at positions 252A, 252B, and 252C, the video decoder may assign the next available index to the motion vector predictor candidate at position 252D. If the motion vector predictor candidate at position 252D is unavailable or is the same as at least one of the motion vector predictor candidate at positions 252A, 252B, and 252C, the video decoder skips adding the motion vector predictor candidate at position 252D to the motion vector predictor candidate list. In the above-mentioned implementation, an example is generally described in which the motion vector predictor candidate 252A to 252D are considered to determine whether to add the motion vector predictor candidate 252A to 252D in the motion vector predictor candidate list. However, in some implementations, all the motion vector predictor candidate 252A to 252D may be added to the motion vector predictor candidate list first, and then repeated motion vector predictor candidate is removed from the motion vector predictor candidate list.

[0246] After the video decoder considers the first four spatial motion vector predictor candidates, the motion vector predictor candidate list may include four spatial motion vector predictor candidates, or the list may include fewer than four spatial motion vector predictor candidates. If the list includes four spatial motion vector predictor candidates (904, yes), the video decoder considers a temporal motion vector predictor candidate (906). The temporal motion vector predictor candidate may correspond to motion information of a co-located PU of a picture different from the current picture. If a temporal motion vector predictor candidate is available and different from the first four spatial motion vector predictor candidates, the video decoder assigns index 4 to the temporal motion vector predictor candidate. If the temporal motion vector predictor candidate is unavailable or is the same as one of the first four spatial motion vector predictor candidates, the video decoder skips adding the temporal motion vector predictor candidate to the motion vector predictor candidate list. Thus, after the video decoder considers the temporal motion vector predictor candidates (906), the motion vector predictor candidate list may include five motion vector predictor candidates (the first four spatial motion vector predictor candidates considered in 902 and the temporal motion vector predictor candidate considered in 906), or may include four motion vector predictor candidates (the first four spatial motion vector predictor candidates considered in 902). If the motion vector predictor candidate list includes five motion vector predictor candidates (908, yes), the video decoder completes building the list.

[0247] If the motion vector predictor candidate list includes four motion vector predictor candidates (908, no), the video decoder may consider a fifth spatial motion vector predictor candidate (910). The fifth spatial motion vector predictor candidate may (for example) correspond to the motion vector predictor candidate at position 252E. If the motion vector predictor candidate at position 252E is available and different from the motion vector predictor candidate at positions 252A, 252B, 252C, and 252D, the video decoder may add the fifth spatial motion vector predictor candidate to the motion vector predictor candidate list and assign index 4 to the fifth spatial motion vector predictor candidate. If the motion vector predictor candidate at position 252E is unavailable or is the same as one of the motion vector predictor candidate at positions 252A, 252B, 252C, and 252D, the video decoder may skip adding the motion vector predictor candidate at position 252E to the motion vector predictor candidate list. Thus, after the fifth spatial predictor motion vector candidate is considered (910), the list may include five predictor motion vector candidates (the first four spatial predictor motion vector candidates considered in 902 and the fifth spatial predictor motion vector candidate considered in 910), or may include four predictor motion vector candidates (the first four spatial predictor motion vector candidates considered in 902).

[0248] If the motion vector predictor candidate list includes five motion vector predictor candidates (912, yes), the video decoder completes generating the motion vector predictor candidate list. If the motion vector predictor candidate list includes four motion vector predictor candidates (912, no), the video decoder adds artificially generated motion vector predictor candidates (914) until the list includes five motion vector predictor candidates (916, yes).

[0249] If the list includes fewer than four spatial motion vector predictor candidates (904, no), after the video decoder considers the first four spatial motion vector predictor candidates, the video decoder may consider a fifth spatial motion vector predictor candidate (918). The fifth spatial motion vector predictor candidate may (for example) correspond to the motion vector predictor candidate at position 252E. If the motion vector predictor candidate at position 252E is available and different from the existing motion vector predictor candidate in the motion vector predictor candidate list, the video decoder may add the fifth spatial motion vector predictor candidate to the motion vector predictor candidate list and assign the next available index to the fifth spatial motion vector predictor candidate. If the motion vector predictor candidate at position 252E is unavailable or is the same as one of the existing motion vector predictor candidates in the motion vector predictor candidate list, the video decoder may skip adding the motion vector predictor candidate at position 252E to the motion vector predictor candidate list. Next, the video decoder may consider temporal motion vector predictor candidates (920). If the temporal motion vector predictor candidate is available and different from the existing motion vector predictor candidate in the motion vector predictor candidate list, the video decoder may add the temporal motion vector predictor candidate to the motion vector predictor candidate list and assign the next available index to the temporal motion vector predictor candidate. If the temporal motion vector predictor candidate is unavailable or is the same as one of the existing motion vector predictor candidates in the motion vector predictor candidate list, the video decoder may skip adding the temporal motion vector predictor candidate to the motion vector predictor candidate list.

[0250] If the motion vector predictor candidate list includes five motion vector predictor candidates (922, yes) after considering the fifth spatial motion vector predictor candidate (at 918) and the temporal motion vector predictor candidate (at 920), the video decoder completes generating the motion vector predictor candidate list. If the motion vector predictor candidate list includes fewer than five motion vector predictor candidates (922, no), the video decoder adds artificially generated motion vector predictor candidates (914) until the list includes five motion vector predictor candidates (916, yes).

[0251] According to the technology of the present application, additional merge prediction motion vector candidates may be artificially generated after the spatial prediction motion vector candidates and the temporal prediction motion vector candidates, so that the size of the merge prediction motion vector candidates is constant and the merge prediction motion vector candidate list includes a specified amount (e.g., five in the above-mentioned possible implementation of FIG. 9) of merge prediction motion vector candidates. The additional merge prediction motion vector candidates may include examples of a combined bi-predictive merge prediction motion vector candidate (predictive motion vector candidate 1), a scaled bi-predictive merge prediction motion vector candidate (predictive motion vector candidate 2), and a zero vector merge / AMVP predictive motion vector candidate (predictive motion vector candidate 3).

[0252] Figure 10 is a schematic diagram of an example of adding a combined motion vector candidate to a merge mode predicted motion vector candidate list according to an embodiment of the present application. A combined bi-predictive merge prediction motion vector candidate can be generated by combining original merge prediction motion vector candidates. Specifically, two original predicted motion vector candidates (which may have mvL0 and refIdxL0 or mvL1 and refIdxL1) can be used to generate a bi-predictive merge prediction motion vector candidate. In Figure 10, two predicted motion vector candidates are included in the original merge prediction motion vector candidate list. The prediction type of one predicted motion vector candidate is uni-predictive by using list 0, and the prediction type of the other predicted motion vector candidate is uni-predictive by using list 1. In this possible implementation, mvL0_A and ref0 are obtained from list 0, and mvL1_B and ref0 are obtained from list 1. Then, a bi-predictive merge prediction motion vector candidate (having mvL0_A and ref0 in list 0 and mvL1_B and ref0 in list 1) may be generated, and it is checked whether the bi-predictive merge prediction motion vector candidate is different from an existing predictive motion vector candidate in the predictive motion vector candidate list. If the bi-predictive merge prediction motion vector candidate is different from an existing predictive motion vector candidate, the video decoder may add the bi-predictive merge prediction motion vector candidate to the predictive motion vector candidate list.

[0253] 11 is a schematic diagram of an example of adding a scaled motion vector candidate to a merge mode predicted motion vector candidate list according to an embodiment of the present application. A scaled bi-predictive merge prediction motion vector candidate may be generated by scaling an original merge prediction motion vector candidate. Specifically, one original predicted motion vector candidate (having mvLX and refIdxLX) may be used to generate a bi-predictive merge prediction motion vector candidate. In a possible implementation of FIG. 11, two predicted motion vector candidates are included in the original merge prediction motion vector candidate list. The prediction type of one predicted motion vector candidate is uni-predictive by using List 0, and the prediction type of the other predicted motion vector candidate is uni-predictive by using List 1. In this possible implementation, mvL0_A and ref0 may be obtained from List 0, and ref0 may be copied to List 1, denoted as reference index ref0'. Then, mvL0'_A may be calculated by scaling mvL0_A with ref0 and ref0'. The scaling may depend on a POC (Picture Order Count) distance. Then, a bi-predictive merge prediction motion vector candidate (having mvL0_A and ref0 in list 0 and ref0′ in list 1) may be generated, and it is checked whether the bi-predictive merge prediction motion vector candidate is repeated. If the bi-predictive merge prediction motion vector candidate is not repeated, the bi-predictive merge prediction motion vector candidate may be added to the merge prediction motion vector candidate list.

[0254] 12 is a schematic diagram of an example of adding a zero motion vector to a merge mode predicted motion vector candidate list according to an embodiment of the present application. A zero vector merge predicted motion vector candidate may be generated by combining a zero vector with a reference index that may be referenced. If a zero vector merge predicted motion vector candidate is not repeated, the zero vector merge predicted motion vector candidate may be added to the merge predicted motion vector candidate list. The motion information of each generated merge predicted motion vector candidate may be compared with the motion information of the previous predicted motion vector candidate in the list.

[0255] In some possible implementations, if a newly generated motion vector predictor candidate is different from an existing motion vector predictor candidate in the motion vector predictor candidate list, the generated motion vector predictor candidate is added to a merged motion vector predictor candidate list. The process of determining whether a motion vector predictor candidate is different from an existing motion vector predictor candidate in the motion vector predictor candidate list may be called pruning. Through pruning, each newly generated motion vector predictor candidate may be compared with the existing motion vector predictor candidate in the list. In some possible implementations, the pruning operation may include comparing one or more new motion vector predictor candidates with the existing motion vector predictor candidates in the motion vector predictor candidate list and skipping adding new motion vector predictor candidates that are the same as the existing motion vector predictor candidates in the motion vector predictor candidate list. In some other possible implementations, the pruning operation may include adding one or more new motion vector predictor candidates to the motion vector predictor candidate list and then removing repeated motion vector predictor candidates from the list.

[0256] The following describes some inter-prediction implementations. The first preset algorithm and the second preset algorithm in the present application may include one or more of the inter-prediction implementations.

[0257] Inter-picture prediction uses the temporal correlation between pictures to derive a motion-compensated prediction (MCP) for a block of picture samples.

[0258] In this block-based MCP, a video picture is divided into rectangular blocks. Assuming that homogeneous motion occurs within a block and the moving object is larger than a single block, for each block, a corresponding block in a previously decoded picture can be found, which serves as a predictor. Using a translational motion model, the position of a block in a previously decoded picture is indicated by a motion vector (Δx, Δy), where Δx specifies the horizontal displacement relative to the position of the current block and Δy specifies the vertical displacement relative to the position of the current block. The motion vector (Δx, Δy) may have fractional sample accuracy to more accurately capture the motion of background objects. When the corresponding motion vector has fractional sample accuracy, interpolation is applied to reference pictures to derive a prediction signal. The previously decoded picture is called a reference picture and is indicated by a reference index Δt corresponding to the reference picture list. These translational motion model parameters, i.e., the motion vector and reference index, are further referred to as motion data. Two types of inter-picture prediction are allowed in modern video coding standards: uni-prediction and bi-prediction.

[0259] In the case of bi-prediction, two sets of motion data (Δx0, Δy0, Δt0, and Δx1, Δy1, Δt1) are used to generate two MCPs (possibly from different pictures), which are then combined to obtain the final MCP. By default, this is done by averaging, but in the case of weighted prediction, different weights may be applied to each MCP, for example, to compensate for scene fade-out. Reference pictures that may be used in bi-prediction are stored in two separate lists: List 0 and List 1. To constrain memory bandwidth in slices that allow bi-prediction, the HEVC standard restricts PUs with 4x8 or 8x4 luma prediction blocks to use only uni-prediction. Motion data is derived in the encoder by using a motion compensation process. Because motion estimation is not specified in the video standard, various encoders can utilize various complexity and quality tradeoffs during their implementation.

[0260] The motion data of a block is correlated with neighboring blocks. To exploit this correlation, the motion data is not directly coded in the bitstream, but is predictively coded based on the neighboring motion data. In HEVC, two concepts are used for this purpose. Predictive coding of motion vectors is improved in HEVC by introducing a new tool called advanced motion vector prediction (AMVP), in which the best prediction value for each motion block is signaled to the decoder. In addition, a new technique called inter-prediction block merging is used to derive all the motion data of a block from neighboring blocks. In this way, the direct mode and skip mode in H.264 / AVC are replaced.

[0261] Evolutionary Motion Vector Prediction

[0262] As described in previous video coding standards, HEVC motion vectors are coded as differential motion vector prediction values ​​(MVPs) based on horizontal (x) and vertical (y) components. The two motion vector differential (MVD) components are calculated according to equations (1.1) and (1.2). MVD X =Δx-MVP X (1.1) MVD Y =Δy-MVP Y (1.2)

[0263] The motion vector of a current block is usually correlated with the motion vectors of neighboring blocks in the current picture or a previous coded picture. This is because neighboring blocks are likely to correspond to the same moving object with similar motion, and the object's motion is unlikely to change suddenly over time. Therefore, using the motion vector of a neighboring block as a predictor reduces the signaled motion vector differential. The MVP is usually derived from already decoded motion vectors of spatially or temporally neighboring blocks in a co-located picture. In some cases, a zero motion vector can alternatively be used as the MVP. In H.264 / AVC, this is implemented by taking the component-wise median of three spatially neighboring motion vectors. Using this approach, signaling of a predictor is not required. The temporal MVP from a co-located picture is only considered in the so-called temporal direct mode in H.264 / AVC. H.264 / AVC direct mode is also used to derive motion data other than motion vectors.

[0264] In HEVC, the technique for implicitly deriving an MVP has been replaced by a technique known as motion vector compensation. This technique explicitly signals which MVP from a list of MVPs is used for motion vector derivation. The variable coding quadtree block structure in HEVC allows a block with several neighboring blocks with motion vectors to serve as a potential MVP candidate. The initial design of Advanced Motion Vector Prediction (AMVP) included five MVPs from three different classes of predictors: three motion vectors from spatial neighbors, the median of three spatial predictors, and a scaled motion vector from a co-located temporal neighbor. Furthermore, the list of predictors was modified by reordering to place the most probable motion predictor in the first position and by removing redundant candidates to ensure that signaling overhead is minimized. Significant simplifications to the AMVP design are then developed, such as removing the median predictor, reducing the amount of candidates in the list from five to two, fixing the order of candidates in the list, and reducing the amount of redundancy checking. The final design for AMVP candidate list construction includes two MVP candidates: a. up to two spatial MVP candidates derived from five spatially neighboring blocks, b. one temporal MVP candidate derived from two temporally co-located blocks when both spatial MVP candidates are unavailable or identical, and c. a zero motion vector when the spatial candidate, the temporal candidate, or both the spatial and temporal candidates are unavailable.

[0265] As already mentioned, two spatial MVP candidates A and B are derived from five spatial neighboring blocks. The location of the spatial block candidate is the same for both AMVP and inter-prediction block merging. For candidate A, the motion data of two blocks A0 and A1 in the bottom left corner are considered in a two-pass approach. In the first pass, it is checked whether any of the candidate blocks contains a reference index equal to the reference index of the current block. The first motion vector found is used as candidate A. When all reference indexes A0 to A1 point to reference pictures different from the reference index of the current block, the associated motion vector cannot be used as is. Therefore, in the second pass, the motion vector needs to be scaled based on the temporal distance between the reference picture candidate and the current reference picture. Equation (1.3) shows that the motion vector candidate mv cand is scaled based on a scaling factor. ScaleFactor is calculated based on the temporal distance between the current picture and the reference picture of the block candidate td and the temporal distance between the current picture and the reference picture of the current block tb. The temporal distance is expressed in terms of the difference between the picture order count (POC) values ​​that define the display order of the pictures. The scaling operation is essentially the same as the scheme used for the temporal direct mode of H.264 / AVC. This factorization allows pre-computation of ScaleFactor at the slice level, since it relies only on the reference picture list structure signaled in the slice header. Note that MV scaling is performed only when both the current reference picture and the reference picture candidate are short-term reference pictures. The parameter td is defined as the POC difference between the co-located picture and the reference picture for the co-located block candidate. mv=sign(mv cand ·ScaleFactor)((|mv cand ScaleFactor+27)>>8) (1.3) ScaleFactor=clip(-2 12 ,2 12 -1,(tb·tx+2 5 )>>6) (1.4)

[0266]

number

[0267] For candidate B, candidates B0 through B2 are sequentially checked in the same manner as A0 and A1 were checked in the first pass. However, the second pass is only performed when blocks A0 and A1 do not contain any motion information, i.e., when blocks A0 and A1 are unavailable or coded via intra-picture prediction. Then, if candidate A is found, candidate A is set equal to unscaled candidate B, and candidate B is set equal to a second unscaled or scaled variant of candidate B. In the second pass, a search is performed to derive unscaled and scaled motion vectors from candidates B0 through B2. Overall, this design allows A0 and A1 to be processed independently of B0, B1, and B2. The derivation of B should only be aware of the availability of both A0 and A1 to perform a search to derive scaled motion vectors or additional unscaled motion vectors from B0 through B2. This dependency is acceptable given that it significantly reduces the complex motion vector scaling operations for candidate B. Reducing the amount of motion vector scaling significantly reduces the complexity in the motion vector predictor derivation process.

[0268] In HEVC, the blocks to the lower right and center of the current block are determined to be the most suitable blocks for producing a good temporal motion vector prediction value (TMVP). Among these candidates, C0 represents the lower-right neighboring block, and C1 represents the center block. Again, the motion data of C0 is considered first. If the motion data of C0 is unavailable, the motion data of the center co-located block candidate is used to derive the temporal MVP candidate C. The motion data of C0 is also considered unavailable when the associated PU belongs to a CTU beyond the current CTU row. This minimizes the memory bandwidth requirements for storing the co-located motion data. In contrast to spatial MVP candidates, whose motion vectors may point to the same reference picture, motion vector scaling is mandatory for TMVP. Therefore, the same scaling operation as for spatial MVP is used.

[0269] While temporal direct mode in H.264 / AVC always references the second reference picture list, i.e., the first reference picture in list 1, and is only allowed in bi-predictive slices, HEVC can indicate for each picture which reference pictures are considered to be co-located pictures. This is implemented by signaling the co-located reference picture list and reference picture index in the slice header and by requiring these syntax elements in all slices in a picture to specify the same reference pictures.

[0270] Because temporal MVP candidates introduce additional dependencies, their use may need to be disabled for error robustness. In H.264 / AVC, it is possible to disable temporal direct mode for bi-predictive slices in the slice header (direct_spatial_mv_pred_flag). The HEVC syntax extends this signaling by allowing TMVP to be disabled at the sequence or picture level (sps / slice_temporal_mvp_enabled_flag). Although the flag is signaled in the slice header, there is a bitstream conformance requirement that the value of the flag must be the same for all slices in a picture. Because picture-level flag signaling depends on the SPS flag, signaling picture-level flags in the PPS may introduce parsing dependencies between the SPS and the PPS. Another advantage of this slice header signaling is that it is desirable to change only the value of this flag in the PPS without changing other parameters, eliminating the need to send a second PPS.

[0271] In general, motion data signaling in HEVC is similar to that in H / 264 / AVC. The inter-picture prediction syntax element inter_pred_idc signals whether reference list 0, reference list 1, or both reference lists 0 and 1 are used. For each MCP derived from one reference picture list, the corresponding reference picture (Δt) is signaled by the index ref_idx_l0 / 1 to the reference picture list, and the MV(Δx, Δy) is represented by the index mvp_l0 / 1_flag to the MVP and the MVD of the MVP. A newly introduced flag mvd_l1_zero_flag in the slice header indicates whether the MVD for the second reference picture list is equal to 0 and therefore not signaled in the bitstream. Once the motion vector is fully reconstructed, the final clipping operation is performed when the value of each component of the final motion vector is equal to -2 inclusive. 15 From 2 15 Ensures that it is always within the -1 range.

[0272] Inter-picture prediction Bloch merging

[0273] While an AMVP list contains only motion vectors for one reference list, merge candidates contain all motion data, including information on whether one or two reference picture lists are used, as well as the reference index and motion vector for each list. Overall, the merge candidate list is built based on the following candidates: a. up to four spatial merge candidates derived from five spatially neighboring blocks, b. one temporal merge candidate derived from two temporally co-located blocks, and c. additional merge candidates including combined bi-predictive candidates and zero motion vector candidates.

[0274] The first candidate in the merge candidate list is a spatial neighbor. A1, B1, B0, A0, and B2 are checked sequentially, and up to four candidates can be inserted into the merge list in that order.

[0275] Rather than simply checking whether neighboring blocks are available and contain motion information, some additional redundancy checks are performed before all motion data of neighboring blocks are used as merging candidates. These redundancy checks can be divided into two categories with two different purposes: a. to ensure that no candidates with redundant motion data are in the list, and b. to prevent merging of two partitions that may be expressed by other means that may create redundant syntax.

[0276] When N represents the amount of spatial merge candidates, the complete redundancy check is

number

[0277] The motion vectors of temporal merge candidates are derived in the same manner as deriving TMVP. Because the merge candidate contains all motion data and TMVP is only one motion vector, the derivation of the entire motion data depends only on the slice type. For bi-predictive slices, TMVP is derived for each reference picture list. Depending on the availability of TMVP for each list, the prediction type is set to bi-predictive or to the list where TMVP is available. All associated reference picture indices are set equal to 0. Therefore, for uni-predictive slices, only TMVP for list 0 is derived, with the reference picture index equal to 0.

[0278] When at least one TMVP is available and a temporal merge candidate is added to the list, redundancy checking is not performed. This makes the construction of the merge list independent of the co-located picture, thereby increasing resilience to errors. Cases are considered in which a temporal merge candidate is redundant and therefore not included in the merge candidate list. When a co-located picture is lost, the decoder cannot derive a temporal candidate and therefore does not check whether the temporal candidate is redundant. This affects the indexing of all subsequent candidates.

[0279] For robustness of the analysis, the merge candidate list has a fixed length. Even after spatial and temporal merge candidates are added, the list may not reach that fixed length. To compensate for the loss of coding efficiency associated with non-length-adaptive list index signaling, additional candidates are generated. Depending on the slice type, up to two types of candidates may be used to completely fill the list: a. combined bi-predictive candidates, and b. zero motion vector candidates.

[0280] In a bi-predictive slice, additional candidates may be generated based on existing candidates by combining the reference picture list 0 motion data of one candidate with the reference picture list 1 motion data of another candidate. This is performed by copying Δx0, Δy0, and Δt0 from one candidate, such as the first candidate, and copying Δx1, Δy1, and Δt1 from another candidate, such as the second candidate. Various combinations are predefined and are given in Table 1.1.

[0281] [Table 1]

[0282] When the list is still not filled after adding the combined bi-predictive candidates, or when the list is not filled for the uni-predictive slices, zero motion vector candidates are calculated to complete the list. All zero motion vector candidates have one zero disparity motion vector for the uni-predictive slices and two zero disparity motion vectors for the bi-predictive slices. The reference index is set equal to 0 and incremented by 1 for each additional candidate until the maximum amount of reference indexes is reached. In this case, if there are other candidates still missing, a reference index equal to 0 is used to create these candidates. Redundancy checks are not performed for all additional candidates, since it is known that omitting these checks does not result in a loss of coding efficiency.

[0283] For each PU coded based on an inter-picture prediction mode, merge_flag indicates that block merging is used to derive motion data, and merge_idx is further used to determine which candidates are in the merge list and provide all the motion data required for the MCP. In addition to this PU-level signaling, the number of candidates in the merge list is signaled in the slice header. Since the default value is 5, it is expressed as a difference to 5 (five_minus_max_num_merge_cand). Thus, 5 is signaled using a short codeword for 0. If only one candidate is used, 5 is signaled using a longer codeword for 4. Regarding the impact on the merge candidate list construction process, the overall process remains the same, but the process terminates after the list contains the maximum amount of merge candidates. In the initial design, the maximum value for coding the merge index is given by the amount of available spatial and temporal candidates in the list. For example, when only two candidates are available, the index can be efficiently coded as a flag. However, to analyze the merge index, the entire merge candidate list must be constructed to know the actual amount of candidates. If a neighboring block becomes unavailable due to a transmission error, the merge index can no longer be analyzed.

[0284] An important application of the block merging concept in HEVC is its combination with skip mode. In previous video coding standards, motion data is inferred rather than explicitly signaled, and skip mode is used to indicate for a block that the prediction residual is zero. In other words, no transform coefficients are transmitted. In HEVC, a skip_flag is signaled at the beginning of each CU in an inter-picture predicted slice, indicating: a. that the CU contains only one PU (2N×2N partition type); b. that merge mode is used to derive motion data (merge_flag equal to 1); and c. that no residual data is present in the bitstream.

[0285] A parallel merge estimation level indicating a region is introduced in HEVC. The merge candidate list can be derived independently by checking whether block candidates are located in the merge estimation region (MER). Block candidates within the same MER are not included in the merge candidate list. Therefore, motion data of candidate blocks does not need to be available during list construction. When this level is, for example, 32, all prediction units within a 32x32 region can be used to construct the merge candidate list in parallel, because not all merge candidates within the same 32x32 MER are inserted into the list. All potential merge candidates for the first PU 0 are available because they are outside the first 32x32 MER. For the second MER, when merge estimation within the MER should be independent, the merge candidate list for PUs 2 to 6 cannot include motion data from these PUs. Therefore, for example, when PU 5 is seen, no merge candidate is available and therefore not inserted into the merge candidate list. In this case, the merge list for PU 5 includes only temporal candidates (if available) and zero MV candidates. To allow the encoder to trade off parallelism and coding efficiency, the parallel merge estimation level is adaptive and is signaled as log2_parallel_merge_level_minus2 in the picture parameter set.

[0286] Sub-CU-based motion vector prediction

[0287] During the development of new video coding techniques, with QTBT, each CU may have at most one set of motion parameters for each prediction direction. Two sub-CU level motion vector prediction methods are considered in the encoder by partitioning a large CU into sub-CUs and deriving motion information for all sub-CUs of the large CU. The alternative temporal motion vector prediction (ATMP) method allows each CU to fetch multiple sets of motion information from multiple blocks smaller than the current CU in a co-located reference picture. In the spatial-temporal motion vector prediction (STMP) method, the motion vector of a sub-CU is recursively derived by using the temporal motion vector predictor and spatial neighboring motion vectors.

[0288] To maintain a more accurate motion field for sub-CU motion estimation, motion compensation for reference frames is currently disabled.

[0289] Alternative Temporal Motion Vector Prediction

[0290] In the alternative temporal motion vector prediction (ATMP) method, the temporal motion vector prediction (TMVP) for motion vectors is modified by fetching multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. A sub-CU is a square NxN block (N is set to 4 by default).

[0291] ATMVP predicts the motion vectors of sub-CUs in a CU in two steps. The first step is to identify the corresponding block in a reference picture by using the temporal vector. The reference picture is called the motion source picture. The second step is to partition the current CU into sub-CUs and obtain the motion vector and reference index of each sub-CU from the block corresponding to the sub-CU.

[0292] In the first step, a reference picture and corresponding blocks are determined based on the motion information of the spatial neighboring blocks of the current CU. To avoid a repetitive scanning process for neighboring blocks, the first merge candidate in the merge candidate list for the current CU is used. The first available motion vector and the reference index associated with the first available motion vector are set to be the temporal vector and index of the motion source picture. In this way, compared to TMVP, ATMVP can more accurately identify corresponding blocks. In TMVP, corresponding blocks (sometimes called co-located blocks) are always located in the lower right or center position relative to the current CU.

[0293] In the second step, the corresponding block of the sub-CU is identified by a time vector in the motion source picture by adding the time vector to the coordinates of the current CU. For each sub-CU, the motion information of the corresponding block of the sub-CU (the smallest motion grid containing the center sample) is used to derive the motion information for the sub-CU. After the motion information of the corresponding N×N block is identified, the motion information is converted into a motion vector and reference index of the current sub-CU in the same manner as TMVP in HEVC, and motion scaling and other procedures are applied. For example, the decoder checks whether the low-latency condition (i.e., the POCs of all reference pictures of the current picture are smaller than the POC of the current picture) is met, and possibly uses the motion vector MVx (the motion vector corresponding to reference picture list X) to predict the motion vector MVy (X equals 0 or 1, and Y equals 1-X) for each sub-CU.

[0294] Spatiotemporal Motion Vector Prediction

[0295] In this method, the motion vectors of sub-CUs are derived recursively according to the raster scan order. An 8x8 CU is considered to contain four 4x4 sub-CUs A, B, C, and D. The neighboring 4x4 blocks in the current frame are labeled a, b, c, and d.

[0296] Motion derivation for sub-CU A begins by identifying two spatial neighbors of sub-CU A. The first neighbor is the N×N block above sub-CU A (i.e., block c). If block c is unavailable or intra-coded, other N×N blocks above sub-CU A are checked (starting from block c, from left to right). The second neighbor is the block to the left of sub-CU A (i.e., block b). If block b is unavailable or intra-coded, other blocks to the left of sub-CU A are checked (starting from block b, from top to bottom). The motion information obtained from the neighboring blocks for each list is scaled to the first reference frame for a given list. Next, the temporal motion vector prediction value (TMVP) of sub-block A is derived according to the same TMVP derivation procedure specified in HEVC. The motion information of the co-located block at position D is fetched and scaled accordingly. Finally, after the motion information is extracted and scaled, all available motion vectors (maximum three) are averaged separately for each reference list, and the averaged motion vector is assigned as the motion vector of the current sub-CU.

[0297] Combination with merge mode

[0298] For additional merge candidates, sub-CU mode is enabled, and no additional syntax elements are required to signal the mode. Two additional merge candidates are added to the merge candidate list for each CU to represent ATMVP mode and STMVP mode. If the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven merge candidates are used. The encoding logic for the additional merge candidates is the same as the encoding logic for merge candidates in HM. This means that for each CU in a P or B slice, two or more RD checks need to be performed for the two additional merge candidates.

[0299] Affine motion compensated prediction

[0300] The affine motion field of a block is described by using two control point motion vectors.

[0301] The motion vector field (MVF) of a block is described according to the following equation:

[0302]

number

[0303] where (v 0x ,v 0y ) represents the motion vector of the control point in the upper left corner, and (v 1x ,v 1y ) represents the motion vector of the control point in the upper right corner.

[0304] To further simplify the motion compensation prediction, sub-block-based affine transformation prediction is applied. The sub-block size M×N is derived according to equation (1.7), MvPre represents the motion vector fractional accuracy (e.g., 1 / 16), and (v 2x ,v 2y ) represents the motion vector of the bottom-left control point calculated according to equation (1.6).

[0305]

number

[0306] After the sub-block size M×N is derived according to equation (1.7), M and N can be adjusted downwards if necessary to make M and N divisors of w and h, respectively.

[0307] To derive the motion vector for each M×N sub-block, the motion vector of the center sample of the sub-block is calculated according to equation (1.6) and rounded to 1 / 16 fractional precision.

[0308] Affine Intermode

[0309] For CUs whose width and height are both greater than 8, the AF_INTER mode can be applied. An affine flag at the CU level is signaled in the bitstream to indicate whether the AF_INTER mode is used. In this mode, the motion vector pair {(v0, v1)|v0={v A ,v B ,v C},v1={v D ,v E A candidate list with}} is constructed by using neighboring blocks. v0 is selected from the motion vectors of blocks A, B, or C. The motion vectors from neighboring blocks are scaled according to the reference list and the relationship between the POC of the reference for the neighboring block, the POC of the reference for the current CU, and the POC of the current CU. The method used to select v1 from neighboring blocks D and E is similar. If the amount of the candidate list is less than 2, the list is padded with motion vector pairs formed by duplicating each AMVP candidate. If the amount of the candidate list is greater than 2, the candidates are first sorted according to the consistency of the neighboring motion vectors (the similarity of the two motion vectors in a pair candidate), and only the first two candidates are kept. To determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) for the current CU, RD cost check is used, and an index indicating the position of the CPMVP in the candidate list is signaled in the bitstream. The difference between CPMV and CPMVP is signaled in the bitstream.

[0310] Affine Merge Mode

[0311] When a CU is applied in AF_MERGE mode, the first block coded based on the affine mode is obtained from the valid neighboring reconstructed blocks. The selection order of candidate blocks is left, top, top right, bottom left, top left. If the bottom left neighboring block A is coded based on the affine mode, the top left, top right, and bottom left motion vectors v2, v3, and v4 of the CU containing block A are derived. In addition, the top left motion vector v0 of the current CU is calculated according to v2, v3, and v4. Then, the top right motion vector v1 of the current CU is calculated.

[0312] To identify whether the current CU is coded based on the AF_MERGE mode, an affine flag is signaled in the bitstream when there is at least one neighboring block coded based on the affine mode.

[0313] Pattern matching motion vector derivation

[0314] The pattern matched motion vector derivation (PMMVD) mode is based on the Frame-Rate Up Conversion (FRUC) technique, in which the motion information of blocks is not signaled but is derived at the decoder side.

[0315] When the merge flag of a CU is true, the FRUC flag of the CU is signaled. When the FRUC flag is false, a merge index is signaled and regular merge mode is used. When the FRUC flag is true, an additional FRUC mode flag is signaled to indicate which method (bilateral matching or template matching) should be used to derive motion information for the block.

[0316] At the encoder side, the decision to use the FRUC merge mode for a CU is based on the RD cost selection as done for regular merge candidates. That is, two matching modes (bilateral matching and template matching) are both checked for the CU by using the RD cost selection. The mode that leads to the smallest cost is further compared with other CU modes. If the FRUC matching mode is the most efficient mode, the FRUC flag is set to true for the CU and the associated matching mode is used.

[0317] The motion derivation process in FRUC merge mode includes two steps. A CU-level motion search is performed first, followed by sub-CU-level motion refinement. At the CU level, an initial motion vector is derived for the entire CU based on bilateral matching or template matching. First, a list of MV candidates is generated, and the candidate that leads to the smallest matching cost is selected as the starting point for further CU-level refinement. Then, a local search based on bilateral matching or template matching is performed around the starting point, and the MV that leads to the smallest matching cost is used as the MV for the entire CU. Subsequently, the motion information is further refined at the sub-CU level by using the derived CU motion vector as the starting point.

[0318] For example, the following derivation process is performed for deriving motion information of a W×H CU: In the first step, the MV of the entire W×H CU is derived. In the second step, the CU is further partitioned into M×M sub-CUs. The value of M is calculated according to formula (1.8), and D represents a predetermined partition depth, which is set to 3 by default in JEM. Then, the MV of each sub-CU is derived.

[0319]

number

[0320] Bilateral matching is used to derive the motion information of the current CU by finding the closest match between two blocks along the motion trajectory of the current CU in two different reference pictures. Under the assumption of continuous motion trajectories, the motion vectors MV0 and MV1 pointing to two reference blocks are proportional to the temporal distances between the current picture and the two reference pictures, i.e., TD0 and TD1. When the current picture is temporally located between the two reference pictures and the temporal distances between the current picture and the two reference pictures are the same, bilateral matching becomes mirror-based bidirectional MV.

[0321] In bilateral matching merge mode, bi-prediction is always applied because the motion information of a CU is derived based on the closest match between two blocks along the motion trajectory of the current CU in two different reference pictures. There is no such restriction in template matching merge mode. In template matching merge mode, the encoder can select uni-prediction from list 0, uni-prediction from list 1, or bi-prediction for a CU. The selection is based on the template matching cost as follows: If costBi<=factor*min(cost0,cost1), Bi-prediction is used; Otherwise, if cost0<=cost1, Uniprediction from list 0 is used; Otherwise, The uni-prediction from List 1 is used.

[0322] cost0 represents the SAD of list 0 template matching, cost1 represents the SAD of list 1 template matching, and costBi represents the SAD of bi-predictive template matching. The value of the coefficient is equal to 1.25, which means the selection process is biased towards bi-prediction. The inter prediction direction selection is only applied to the CU-level template matching process.

[0323] Template matching is used to derive motion information for the current CU by finding the closest match between a template in the current picture (a neighboring block above and / or to the left of the current CU) and a block in the reference picture (of the same size as the template). Except for the FRUC merge mode mentioned above, template matching is also applied to AMVP mode. New candidates are derived using the template matching method. If the new candidate derived through template matching is different from the first existing AMVP candidate, the new candidate is inserted at the beginning of the AMVP candidate list, and then the list size is set to 2 (which means the second existing AMVP candidate is removed). When template matching is applied to AMVP mode, only CU-level search is applied.

[0324] The MV candidate set at the CU level includes: a. the original AMVP candidates selected if AMVP mode is used for the current CU; b. all merge candidates; c. some MVs in the interpolated MV field; and d. the top and left neighboring motion vectors.

[0325] Note that the interpolated MV field mentioned above is generated before the entire picture is coded based on unilateral ME. The motion field can then be later used as a CU-level or sub-CU-level MV candidate. First, the motion field of each reference picture in the two reference lists is exhausted at the 4x4 block level. For each 4x4 block, the motion associated with the block passes through the 4x4 block in the current picture, and if the block is not assigned any interpolated motion, the motion of the reference block is scaled to the current picture based on the temporal distances TD0 and TD1 (in the same manner as MV scaling in TMVP of HEVC), and the scaled motion is assigned to the block in the current frame. If scaled MV is not assigned to the 4x4 block, the motion of the block is marked as unavailable in the interpolated motion field.

[0326] When bilateral matching is used, each valid MV of the merge candidates is used as input to generate an MV pair under the assumption of bilateral matching. For example, one valid MV of the merge candidates is (MVa, refa) in reference list A. Then, the reference picture refb of the pair of bilateral MVs is found in the other reference list B. In this way, refa and refb are temporally located on different sides of the current picture. If such refb is unavailable in reference list B, refb is determined as a reference different from refa, and the temporal distance between refb and the current picture is the smallest in list B. After refb is determined, MVb is derived by scaling MVa based on the temporal distance between the current picture and refa and the temporal distance between the current picture and refb.

[0327] Four MVs from the interpolated MV field are also added to the CU-level candidate list. More specifically, the interpolated MVs at positions (0,0), (W / 2,0), (0,H / 2), and (W / 2,H / 2) of the current CU are added.

[0328] When FRUC is applied to AMVP mode, the original AMVP candidates are also added to the CU-level MV candidate set.

[0329] At the CU level, up to 15 MVs of the AMVP CU and up to 13 MVs of the merged CU are added to the candidate list.

[0330] The MV candidate set at the sub-CU level includes: a. the MV determined from the CU-level search; b. the neighboring MVs above, to the left, above-left, and above-right; c. scaled versions of the co-located MVs from the reference picture; d. up to four ATMVP candidates; and e. up to four STMVP candidates.

[0331] The scaled MV from the reference picture is derived as follows: all reference pictures in both lists are traversed, and the MV at the co-located position of the sub-CU in the reference picture is scaled to the reference of the MV at the starting CU level.

[0332] ATMVP and STMVP candidates are limited to the four initial candidates.

[0333] At the sub-CU level, up to 17 MVs are added to the candidate list.

[0334] Motion Vector Improvement

[0335] The motion vectors can be improved by using various methods in combination with various inter prediction modes.

[0336] MV improvement in FRUC

[0337] MV refinement is a pattern-based MV search using bilateral matching cost or template matching cost criteria. In the current development, two search patterns are supported: unrestricted center-biased diamond search (UCBDS) and adaptive cross search, which are used for MV refinement at the CU and sub-CU levels, respectively. For both CU and sub-CU level MV refinement, the MV is directly searched with MV accuracy of one-quarter luma sample, followed by MV accuracy of one-eighth luma sample. The search range for MV refinement for the CU and sub-CU steps is set equal to eight luma samples.

[0338] Decoder-side motion vector improvement

[0339] In bi-predictive operation, for prediction of one block region, two prediction blocks formed using list 0 MV and list 1 MV, respectively, are combined to form a single prediction signal. In decoder-side motion vector refinement (DMVR) method, the two motion vectors of bi-prediction are further refined by using a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between a bilateral template and a reconstructed sample in a reference picture to obtain an refined MV without transmitting additional motion information.

[0340] In DMVR, a bilateral template is generated as a weighted combination (i.e., average) of two prediction blocks from the first list 0 MV0 and list 1 MV1, respectively. The template matching operation involves calculating a cost measure between the generated template and a sample region (around the first prediction block) in the reference picture. For each of the two reference pictures, the MV that yields the smallest template cost is considered as the updated MV for the list, which should replace the original MV. In the current development, nine MV candidates are searched for each list. The nine MV candidates include the original MV and eight surrounding MVs with one luma sample offset from the original MV in either the horizontal or vertical direction or both the horizontal and vertical directions. Finally, two new MVs, namely MV0' and MV1', are used to generate the final bi-prediction result. The sum of absolute difference (SAD) is used as the cost measure.

[0341] DMVR is applied for bi-predictive merge mode using one MV from a past reference picture and another MV from a future reference picture without transmitting additional syntax elements.

[0342] Accuracy and storage of movement data

[0343] Motion data storage reduction

[0344] The use of TMVP in AMVP and in merge mode requires the storage of motion data (including motion vectors, reference indices, and coding modes) in co-located reference pictures. Considering the granularity of motion representation, the memory size required to store the motion data is enormous. HEVC utilizes motion data storage reduction (MDSR) to reduce the size of the motion data buffer and the associated memory access bandwidth by subsampling the motion data within the reference picture. In H.264 / AVC, information is stored every 4x4 block, while in HEVC, 16x16 blocks are used. When subsampling a 4x4 grid, information from the top-left 4x4 block is stored. Due to this subsampling, MDSR affects the quality of temporal prediction.

[0345] Furthermore, there is a close correlation between the locations of MVs used in co-located pictures and the locations of MVs stored by using MDSR. During the HEVC standardization process, it was found that storing the motion data of the top-left block inside a 16x16 region, along with the bottom-right and center TMVP candidates, provides the best trade-off between coding efficiency and memory bandwidth reduction.

[0346] Higher motion vector memory accuracy

[0347] In HEVC, the motion vector accuracy is 1 / 4 pel (1 / 4 luma sample and 1 / 8 chroma sample for 4:2:0 video). In current developments, the accuracy of the internal motion vector storage and merge candidates is increased to 1 / 16 pel. The higher motion vector accuracy (1 / 16 pel) is used in motion compensated inter prediction for CUs coded based on skip / merge mode. For CUs coded based on regular AMVP mode, either integer pel motion or quarter pel motion is used.

[0348] Adaptive Motion Vector Differential Resolution

[0349] In HEVC, motion vector difference (MVD) is signaled in units of quarter luma samples when use_integer_mv_flag is equal to 0 in the slice header. In current development, locally adaptive motion vector resolution (LAMVR) is introduced. MVD can be coded in units of quarter luma samples, integer luma samples, or 4 luma samples. MVD resolution is controlled at the coding unit (CU) level, and an MVD resolution flag is conditionally signaled for each CU that has at least one non-zero MVD component.

[0350] For a CU that has at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MV precision is used for the CU. When the first flag (which is equal to 1) indicates that quarter luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or 4 luma sample MV precision is used.

[0351] When the first MVD resolution flag of a CU is 0 or is not coded for a CU (which means all MVDs for the CU are 0), quarter luma sample MV resolution is used for the CU. When integer luma sample MV precision or 4 luma sample MV precision is used for a CU, the MVPs in the AMVP candidate list for the CU are rounded to the corresponding precision.

[0352] At the encoder, a CU-level RD check is used to determine which MVD resolution should be used for a CU, i.e., three CU-level RD checks are performed for each MVD resolution.

[0353] Fractional Sample Interpolation Module

[0354] When the motion vector points to a fractional sample position, motion compensated interpolation is required. For luma interpolation filtering, an 8-tap separable DCT-based interpolation filter is used for 2 / 4 precision samples, and a 7-tap separable DCT-based interpolation filter is used for 1 / 4 precision samples, as shown in Table 1.2.

[0355] [Table 2]

[0356] Similarly, a 4-tap separable DCT-based interpolation filter is used for the chroma interpolation filter, as shown in Table 1.3.

[0357] [Table 3]

[0358] For vertical interpolation for 4:2:2 chroma channels, and horizontal and vertical interpolation for 4:4:4 chroma channels, odd positions in Table 1.3 are not used and result in 1 / 4 chroma interpolation.

[0359] In bi-prediction, the bit depth of the output of the interpolation filter is maintained at 14-bit accuracy, independent of the source bit depth, before averaging the two prediction signals. The actual averaging process is performed implicitly along with the bit depth reduction process as follows: predSamples[x,y]=(predSamplesL0[x,y]+predSamplesL1[x,y]+offset)>>shift (1.9) shift=15-BitDepth (1.10) offset=1<<(shift-1) (1.11)

[0360] To reduce complexity, bilinear interpolation is used for both bilateral matching and template matching instead of the usual 8-tap HEVC interpolation.

[0361] The calculation of the matching cost is slightly different in different steps. When a candidate is selected from the candidate set at the CU level, the matching cost is the SAD of bilateral matching or template matching. After the starting MV is determined, the matching cost C of bilateral matching in the sub-CU level search is calculated as follows:

[0362]

number

[0363] w represents a weighting factor empirically set to 4, and MV and MV S denote the current MV and the starting MV, respectively. Nevertheless, the SAD is used as the matching cost for template matching in the sub-CU level search.

[0364] In FRUC mode, MV is derived by using only luma samples. The derived motion vectors will be used for both luma and chroma in MC inter prediction. After MV is determined, the final MC is performed by using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma.

[0365] Motion Compensation Module

[0366] Overlapped Block Motion Compensation

[0367] In the current development, overlapped block motion compensation (OBMC) is performed on all motion compensation (MC) block boundaries except the right and bottom boundaries of a CU. Moreover, overlapped block motion compensation is applied to both the luma and chroma components. An MC block corresponds to a coding block. When a CU is coded based on a sub-CU mode (including sub-CU merge mode, affine mode, and FRUC mode), each sub-block of the CU is an MC block. To process CU boundaries uniformly, OBMC is performed at the sub-block level for all MC block boundaries, and the sub-block size is set equal to 4 × 4.

[0368] When OBMC is applied to a current subblock, in addition to the current motion vector, if the motion vectors of the four connected neighboring subblocks are available and are not identical to the current motion vector, the motion vectors of the four connected neighboring subblocks are also used to derive a prediction block for the current subblock. These prediction blocks based on the multiple motion vectors are combined to generate a final prediction signal for the current subblock.

[0369] A prediction block based on the motion vector of a neighboring subblock is denoted as PN, where N represents the index of the neighboring subblock above, below, left, or right, and a prediction block based on the motion vector of the current subblock is denoted as PC. When PN is based on the motion information of a neighboring subblock that contains the same motion information as the current subblock, OBMC is not performed from PN. Otherwise, every sample in PN is added to the same sample in PC. That is, four rows / columns of PN are added to PC. Weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for PN, and weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are used for PC. The exception is small MC blocks (i.e., when the height or width of the coding block is equal to 4 or when the CU is coded based on subCU mode), and for such blocks, only two rows / columns of PN are added to PC. In this case, weighting factors {1 / 4, 1 / 8} are used for PN, and weighting factors {3 / 4, 7 / 8} are used for PC. For PNs generated based on the motion vectors of vertical (horizontal) neighboring sub-blocks, samples in the same row (column) of PN are added to PC using the same weighting factor.

[0370] In the current development, for CUs with a size of 256 luma samples or less, a CU-level flag is signaled to indicate whether OBMC is applied to the current CU. For CUs with a size greater than 256 luma samples or CUs that are not coded based on the AMVP mode, OBMC is applied by default. In the encoder, when OBMC is applied to a CU, the influence of OBMC is taken into account in the motion estimation stage. The predicted signal formed through OBMC by using the motion information of the upper and left neighboring blocks is used to compensate the upper and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

[0371] Optimization Tools

[0372] Local Lighting Compensation

[0373] Local illumination compensation (LIC) is based on a linear model for illumination changes using a scaling factor a and an offset b. Local illumination compensation is adaptively enabled or disabled for each inter-mode coded coding unit (CU).

[0374] When LIC is applied to a CU, a least squares error method is used to derive parameters a and b by using neighboring samples of the current CU and corresponding reference samples. Subsampled (2:1 subsampling) neighboring samples of the CU and corresponding samples in the reference picture (identified by the motion information of the current CU or sub-CU) are used. LIC parameters are derived and applied separately for each prediction direction.

[0375] When a CU is coded based on merge mode, the LIC flag is copied from neighboring blocks in a manner similar to copying motion information in merge mode. Otherwise, the LIC flag is signaled to the CU to indicate whether LIC is applied.

[0376] When LIC is enabled for a picture, an additional CU-level RD check is required to determine whether LIC applies to the CU. When LIC is enabled for a CU, instead of SAD and SATD, the mean-removed sum of absolute difference (MR-SAD) and the mean-removed sum of absolute Hadamard-transformed difference (MR-SATD) are used for integer-pel and fractional-pel motion searches, respectively.

[0377] Bidirectional Optical Flow

[0378] Bi-directional Optical flow (BIO) is a sample-by-sample motion refinement performed in addition to block-by-block motion compensation for bi-prediction. Sample-level motion refinement does not use signaling.

[0379] I (k) Let ∂I be the luma value from reference k (k=0,1) after block motion compensation. (k) / ∂x and ∂I (k) / ∂y are I (k) represents the horizontal and vertical components of the gradient of the motion vector field (v x ,v y ) is given according to equation (1.13). ∂I (k) / ∂t+v x ∂I (k) / ∂x+v y ∂I (k) / ∂y=0 (1.13)

[0380] For each sample's motion trajectory, this optical flow formula is combined with Hermite interpolation to obtain the function value I (k) and the derivative ∂I (k) / ∂x and ∂I (k) A unique third-order polynomial is obtained that matches both ∂y and ∂y. The value of this polynomial at t=0 is the BIO predicted value. pred BIO =1 / 2(I (0) +I (l) +v x / 2(τ1∂I (1) / ∂x-τ0∂I (0) / ∂x)+v y / 2(τ1∂I (1) / ∂y-τ0∂I (0) / ∂y)) (1.14)

[0381] Here, τ0 and τ1 represent the distance to the reference frame. The distances τ0 and τ1 are calculated based on the POCs relative to Ref0 and Ref1: τ0 = POC(current) - POC(Ref0) and τ1 = POC(Ref1) - POC(current). If both predictions come from the same temporal direction (either both from the past or both from the future), the signs are different (i.e., τ0 · τ1 < 0). In this case, BIO applies only when the predictions are not from the same instant (i.e., τ0 ≠ τ1), both referenced regions have non-zero motion vectors (MVx0, MVy0, MVx1, MVy1 ≠ 0), and the block motion vector is proportional to the temporal distance (MVx0 / MVx1 = MVy0 / MVy1 = -τ0 / τ1).

[0382] Motion vector field (v x ,v y ) is determined by minimizing the difference Δ (the intersection of the motion trajectory and the reference frame plane) between the value at point A and the value at point B. The model uses only the first linear term of a local Taylor expansion for Δ.

[0383]

number

[0384] All values ​​in equation (1.15) depend on the sample position (i',j'), which has been omitted from the notation so far. Assuming that the motion is consistent in the local surrounding area, Δ is minimized inside a (2M+1) × (2M+1) square interval Ω centered at the current predicted point (i,j), where M is equal to 2.

[0385]

number

[0386] For this optimization problem, the current development uses a simplified approach that minimizes first in the vertical direction and then in the horizontal direction.

[0387]

number

[0388] where:

[0389]

number

[0390] To avoid division by 0 or very small values, regularization parameters r and m are introduced in equations (1.17) and (1.18). r=500·4 d-8 (1.20) m=700·4 d-8 (1.21)

[0391] where d represents the bit depth of the video samples.

[0392] To keep the memory access for BIO the same as for regular bi-predictive motion compensation, all the predictors and gradient values ​​I (k) , ∂I (k) / ∂x, ∂I (k) / ∂y is calculated only for positions inside the current block. In equation (1.19), a (2M+1) × (2M+1) square box Ω centered on the current prediction point on the boundary of the prediction block needs to access positions outside the block. In the current development, I outside the block (k) , ∂I (k) / ∂x, ∂I (k) The value of / ∂y is set equal to the closest available value inside the block. For example, this can be implemented as padding.

[0393] With BIO, the motion field can be improved for each sample. However, to reduce the computational complexity, a block-based design of BIO can be used. The motion improvement is calculated based on a 4x4 block. In block-based BIO, the s in equation (1.19) for all samples in the 4x4 block is n The values ​​of are aggregated and then s n The aggregated values ​​of are used to derive the BIO motion vector offset for the 4x4 block. For block-based BIO derivation, the following formula is used:

[0394]

number

[0395] b k denotes the set of samples belonging to the kth 4x4 block of the prediction block. n is used to derive the associated motion vector offsets ((s n ,b k )>>4) is replaced by

[0396] In some cases, the large amount of MV in BIO may be unreliable due to noise or irregular motion. Therefore, in BIO, the magnitude of the large amount of MV is limited to a threshold thBIO. The threshold is determined based on whether the reference pictures of the current picture are all from one direction. If all the reference pictures of the current picture are from one direction, the value of the threshold is 12 × 2. 14-d otherwise the value is set to 12 × 2 13-d is set to

[0397] The gradients for BIO are calculated simultaneously with the motion compensation interpolation by using operations consistent with the HEVC motion compensation process (2D separable FIR). The input of the 2D separable FIR is the same reference frame sample as the motion compensation process according to the fractional part of the block motion vector and the fractional position (fracX, fracY). For the horizontal gradient ∂I / ∂x signal, first, vertical interpolation is performed by using BIOfilterS corresponding to fractional position fracY with an unscaling shift of d-8, and then a gradient filter BIOfilterG is applied horizontally corresponding to fractional position fracX with an unscaling shift of 18-d. For the vertical gradient ∂I / ∂y, first, a gradient filter is applied vertically by using BIOfilterG corresponding to fractional position fracY with an unscaling shift of d-8, and then signal cancellation is performed by using BIOfilterS horizontally corresponding to fractional position fracX with an unscaling shift of 18-d. The lengths of the interpolation filter BIOfilterG for gradient calculation and the interpolation filter BIOfilterS for signal cancellation are shorter (6 taps) to maintain reasonable complexity. Table 1.4 shows the filters used for gradient calculation at different fractional positions of the block motion vector in BIO. Table 1.5 shows the interpolation filters for prediction signal generation in BIO.

[0398] [Table 4]

[0399] [Table 5]

[0400] In the current development, BIO is applied to all bi-predictive blocks when the two predictions are from different reference pictures. When LIC is enabled for a CU, BIO is disabled. OBMC is applied to a block after the normal MC process. To reduce computational complexity, BIO is not applied in the OMBC process. This means that BIO is only applied in the MC process for a block when the MV of the block is used, but is not applied in the MC process when the MV of a neighboring block is used in the OBMC process.

[0401] Weighted Sample Prediction Module

[0402] As an optional tool, a weighted prediction (WP) tool is provided in HEVC. The principle of WP is to replace the inter prediction signal P by a linear weighted prediction signal P′ (with weights w and offsets o). Simple prediction: P' = w × P + 0 (1.23) Biprediction: P' = (w0 × P0 + o0 + w1 × P1 + o1) / 2 (1.24)

[0403] The applicable weights and applicable offsets are selected by the encoder and conveyed in the bitstream. The L0 and L1 suffixes define list 0 and list 1 of the reference picture lists, respectively. Due to the interpolation filters, the bit depth is maintained at 14 bits of accuracy before the prediction signal is averaged.

[0404] In the bi-prediction case with at least one reference picture available in each of lists L0 and L1, the following formula applies for the explicit signaling of weighted prediction parameters for the luma channel: Corresponding formulas apply for the chroma channels and in the uni-prediction case.

[0405]

number

[0406] where: log2WD=luma_log2_weight_denom+14-bitDepth, w0=LumaWeightL0[refIdxL0], wl=LumaWeightL[refIdxL1], o0=luma_offset_l0[refIdxL0]*highPrecisionScaleFactor, o1=luma_offset_l1[refIdxL1]*highPrecisionScaleFactor, highPrecisionScaleFactor=(1<<(bitDepth-8))

[0407] A boundary prediction filter is an intra-coding method used to further adjust the prediction samples in the leftmost column and the topmost row. In HEVC, after an intra-prediction block is generated for a vertical or horizontal intra-mode, the prediction samples in the leftmost column and the topmost row are further adjusted, respectively. This method may be further extended to some diagonal intra-modes, where up to four columns or four rows of boundary samples are further adjusted by using a two-tap filter (for intra-modes 2 and 34) or a three-tap filter (for intra-modes 3 to 6 and 30 to 33).

[0408] In the HEVC standard and previous standards, reference frames are classified into two groups, forward reference frames and backward reference frames, and are arranged in two reference frame lists, respectively. The two reference frame lists are usually named List 0 and List 1. An inter-prediction direction is used to indicate whether the prediction direction of forward prediction, backward prediction, or bi-prediction is used for the current block, and a different reference frame list, i.e., List 0, List 1, or both List 0 and List 1, is selected based on the prediction direction. A reference frame in the selected reference frame list is indicated by a reference frame index. A motion vector is used to indicate the position offset of the reference block of the prediction block of the current block in the selected reference frame relative to the current block in the current frame. Then, a final prediction block is generated based on the prediction direction by using the prediction block obtained from the reference frames in List 0, List 1, or both List 0 and List 1. When the prediction direction is uni-prediction, the prediction block obtained from the reference frame in List 0 or List 1 is directly used. When the prediction direction is bi-predictive, the prediction blocks obtained from the reference frames in both list 0 and list 1 are combined through weighted averaging to obtain the final prediction block.

[0409] To solve the problems of the prior art that the prediction samples obtained based on the inter prediction mode are spatially discontinuous, the prediction efficiency is affected, and the prediction residual energy is relatively large, the embodiments of the present application provide an inter prediction method for filtering the prediction samples by using neighboring reconstructed samples after the prediction samples are generated, thereby improving the coding efficiency.

[0410] 13 is a schematic flowchart of an inter prediction method according to an embodiment of the present application. As shown in FIG. 13, the method includes steps S1301 to S1307.

[0411] S1301: Analyze the bitstream to obtain motion information of the picture block to be processed.

[0412] The current picture block may be referred to as the current block or the current CU.

[0413] It can be understood that step S1301 can be performed by the video decoder 200 of FIG.

[0414] For example, in this embodiment of the present application, a block-based motion compensation technique may be used to search the coded block for the best matching block of the current coding block, so that the residual between the predicted block and the current block is as small as possible and is used to calculate the offset MV of the current block.

[0415] For example, the picture block to be processed may be any block in a picture, and the size of the picture block to be processed may be 2x2, 4x4, 8x8, 16x16, 32x32, 64x64, or 128x128, which is not limited in this embodiment of the present application.

[0416] For example, if the current picture block is encoded at the encoder side based on a merge mode, the spatial candidates and temporal candidates of the current block may be added to a merge motion information candidate list for the current block. The method is the same as that in HEVC. For example, any one of the techniques described in Figures 8 to 12 may be used as a specific method for obtaining the merge motion information candidate list.

[0417] For example, if a merge mode is used for the current block, the motion information of the current block is determined based on the merge index carried in the bitstream. If an inter MVP mode is used for the current block, the motion information of the current block is determined based on the inter prediction direction, the reference frame index, the motion vector predictor index, and the motion vector residual value transmitted in the bitstream.

[0418] Step S1301 may be performed by using a method in HEVC or VTM, or by using another method for generating a motion vector prediction candidate list, which is not limited in this embodiment of the present application.

[0419] S1302: (Optional) Determine to update the prediction block of the picture block to be processed.

[0420] It can be understood that step S1302 can be performed by the video decoder 200 of FIG.

[0421] The predictive block of the current picture block is a predictive block of the current block and may be obtained based on one or more coded blocks.

[0422] For example, whether to update the prediction block of the current picture block may be determined based on the update decision indicator of the current picture block. In other words, whether to perform spatial filtering on the current picture block may be determined based on the update decision indicator of the current picture block.

[0423] In one possible implementation form, the bitstream may be analyzed to obtain update decision indication information for the picture block to be processed, which is used to indicate whether to update the prediction block of the picture block to be processed, and based on the update decision indication information for the picture block to be processed, it is further determined that the prediction block of the picture block to be processed should be updated.

[0424] In another possible implementation form, predetermined update decision instruction information for the picture block to be processed may be obtained, and the predetermined update decision instruction information is used to indicate whether to update the prediction block of the picture block to be processed, and based on the predetermined update decision instruction information, it is further determined that the prediction block of the picture block to be processed should be updated.

[0425] For example, if the update decision indicator is true, it may be determined that the prediction block of the current picture block should be updated. In other words, it is determined that spatial filtering should be performed on the prediction block of the current picture block. If the update decision indicator is false, it is determined that the prediction block of the current picture block does not need to be updated. The specific form of the update decision indicator is not limited in this embodiment of the present application. Here, the update decision indicator being true or false is used merely as an example for explanation.

[0426] S1303: (Optional) Determine a prediction mode corresponding to the picture block to be processed.

[0427] It can be understood that step S1303 can be performed by the video decoder 200 of FIG.

[0428] For example, the prediction mode corresponding to the current picture block may be a merge mode and / or an inter-evolutionary motion vector prediction (inter AMVP) mode. This is not limited in this embodiment of the present application. It may be understood that the prediction mode corresponding to the current picture block may be only a merge mode, only an inter AMVP mode, or a combination of a merge mode and an inter AMVP mode.

[0429] It should be noted that the inter evolutionary motion vector prediction (inter AMVP) mode may also be referred to as the inter motion vector prediction (inter MVP) mode.

[0430] For example, a method for determining a prediction mode corresponding to a picture block to be processed may include the steps of analyzing a bitstream to obtain a prediction mode corresponding to the picture block to be processed, and determining that the prediction mode corresponding to the picture block to be processed is merge mode and / or inter AMVP mode.

[0431] It can be understood that in this embodiment of the present application, the spatial filtering method can be performed on inter-coded blocks that are coded based on merge mode and / or inter AMVP mode. In other words, the filtering process can be performed on blocks that are coded based on merge mode and / or inter AMVP mode at the decoder side during decoding.

[0432] S1304: Perform motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block.

[0433] The predicted block of the current picture contains predicted values ​​of the target samples.

[0434] It can be understood that step S1304 can be performed by the video decoder 200 of FIG.

[0435] For example, during motion compensation, the current partial picture is predicted and compensated based on the reference picture, which can reduce redundant information in the frame sequence.

[0436] For example, when motion compensation is performed based on motion information, a prediction block of a current picture block may be obtained from a reference frame based on a reference frame direction, a reference frame sequence number, and a motion vector. The reference frame direction may be forward prediction, backward prediction, or bi-prediction. This is not limited in this embodiment of the present application.

[0437] For example, when the reference frame direction is forward prediction, a reference picture may be selected from a forward reference picture set for a current coding unit (CU) to obtain a reference block. When the reference frame direction is backward prediction, a reference picture may be selected from a backward reference picture set for a current coding unit (CU) to obtain a reference block. When the reference frame direction is bi-predictive, a reference picture may be selected from each of a forward reference picture set and a backward reference picture set for a current coding unit (CU) to obtain a reference block.

[0438] In step S1304, the method for performing motion compensation on the current picture block based on the motion information may be a method in HEVC or VTM, or may be another method for obtaining a prediction block of the current picture block, which is not limited in this embodiment of the present application.

[0439] S1306: Perform a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample.

[0440] The reference sample has a predetermined spatial relationship with the target sample.

[0441] It can be understood that step S1306 can be performed by the video decoder 200 of FIG.

[0442] For example, the target sample may be a sample in a predictive block of the current picture block, and the predicted value of the target sample may be determined based on the sample value of a sample in a reference block.

[0443] For example, the reference sample may be a reconstructed sample spatially adjacent to the current CU (current picture block). Specifically, the reference sample may be a reconstructed sample in a block other than the current CU block in the picture. For example, the reference sample may be a reconstructed sample in a CU block above or to the left of the current CU. This is not limited in this embodiment of the present application.

[0444] It can be seen that in step S1306, spatial filtering is performed on the predicted sample of the target sample by using the reconstructed sample that is spatially adjacent to the current CU. Specifically, to obtain an updated predicted sample of the target sample, weighted calculations are performed on the predicted sample of the target sample in the current block and the sample values ​​of the reconstructed sample that is spatially adjacent to the current CU.

[0445] In one possible implementation, the one or more reference samples may include a reconstructed sample having the same horizontal coordinate as the target sample and a predetermined vertical coordinate difference from the target sample, or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference from the target sample.

[0446] For example, as shown in FIG. 14, the upper left corner of the picture is used as the origin of the coordinate system, with the X-axis direction of the coordinate system extending to the right along the top side of the picture and the Y-axis direction of the coordinate system extending down along the left side of the picture. If the coordinates of the target sample in the current picture block (current CU) are (xP, yP) and the coordinates of the top left sample of the current picture block are (xN, yN), the reference sample of the target sample may be a reconstructed sample in a block above or to the left of the current picture block. If the reference sample is a reconstructed sample in a block above the current picture block, the reference sample is a reconstructed sample in a block other than the current picture block. Therefore, the vertical coordinate of the reference sample is a value obtained by subtracting a predetermined positional relationship N from the vertical coordinate of the top side of the current picture block, and the horizontal coordinate of the reference sample is the same as the horizontal coordinate of the target sample in the current picture block. Specifically, the coordinates of the reference sample are (xP, yN-N). If the reference sample is a reconstructed sample in a block to the left of the current picture block, the reference sample is a reconstructed sample in a block other than the current picture block, so the horizontal coordinate of the reference sample is a value obtained by subtracting a predetermined positional relationship M from the leftmost horizontal coordinate of the current picture block, and the vertical coordinate of the reference sample is the same as the vertical coordinate of the target sample in the current picture block. Specifically, the coordinate of the reference sample is (xN-M, yP). The specific spatial positional relationship between the reference sample and the target sample (the specific values ​​of M and N) is not limited in this embodiment of the present application.

[0447] In one possible implementation, the predicted value of the target sample may be updated according to the following equation:

[0448]

number

[0449] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are predetermined constants, and M1 and M2 are predetermined positive integers.

[0450] The following describes a specific method for calculating the updated predicted value of the target sample based on different cases of the coordinates (xN, yN) of the top-left sample of the current picture block.

[0451] In the first case, when xN is greater than 0, yN is equal to 0, and the reference sample at position (xN-M1, yP) has been encoded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0452]

number

[0453] For example, as shown in FIG. 15, a size of a current picture block is 16×16, which is used as an example for explanation. If the current picture block is CU 1, the top-left sample (xN, yN) of the current picture block (CU 1) is (16, 0), and the coordinate of the target sample (xP, yP) in the current picture block is (18, 3). Since the horizontal coordinate xN of the top-left sample of the current CU (CU 1) is greater than 0 and the vertical coordinate yN of the top-left sample of the current CU (CU 1) is equal to 0, it can be determined that the current CU is located at the top of the picture. When spatial filtering is performed on the target sample in the current CU, since the current CU is located at the top of the picture and there is no reconstructed sample above the current CU, the reference sample is the reconstructed sample to the left of the current CU. The reference sample has the same vertical coordinate as the target sample and may be denoted as (16−M1, 3), where M1 represents the predetermined spatial relationship between the reference sample and the target sample. In this specification, M1 equal to 1 is used as an example for explanation. When M1 equals 1, the reference sample of the target sample (18, 3) can be (15, 3). Similarly, any reference sample of the target sample (xP, yP) can be (xN-1, yP).

[0454] When the reference sample at position (xN-1, yP) has been coded and reconstructed, a weighted calculation may be performed on the reconstructed value of the reference sample recon(xN-1, yP) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0455] In the second case, when xN is equal to 0, yN is greater than 0, and the reference sample at position (xP, yN-M2) has been coded and reconstructed, the updated predicted value of the target sample can be obtained according to the following formula:

[0456]

number

[0457] For example, as shown in FIG. 15, if the current picture block is CU 2, the coordinates (xN, yN) of the top-left sample of the current picture block (CU 2) are (0, 32), and the coordinates of the target sample (xP, yP) are (8, 35). Since the horizontal coordinate xN of the top-left sample of the current CU (CU 2) is equal to 0 and the vertical coordinate yN of the top-left sample of the current CU (CU 2) is greater than 0, it can be determined that the current CU is located on the left side of the picture. When spatial filtering is performed on the target sample in the current CU, since the current CU is located on the left side of the picture and there is no reconstructed sample on the left side of the current CU, the reference sample is the reconstructed sample above the current CU. The reference sample has the same horizontal coordinate as the target sample and may be denoted as (8, 32-M2), where M2 represents the predetermined spatial relationship between the reference sample and the target sample. In this specification, M2 equal to 1 is used as an example for explanation. When M2 is equal to 1, the reference sample of the target sample (8,35) can be (8,31). Similarly, any reference sample of the target sample (xP,yP) can be (xP,yN-M2).

[0458] When the reference sample at position (xP, yN-M2) has been coded and reconstructed, a weighted calculation may be performed on the reconstructed value of the reference sample recon(xP, yN-M2) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0459] In the third case, when xN is greater than 0, yN is greater than 0, and the reference samples at positions (xN-M1, yP) and (xP, yN-M2) have been coded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0460]

number

[0461] For example, as shown in FIG. 15, if the current picture block is CU 3, the coordinates (xN, yN) of the top-left sample of the current picture block (CU 3) are (48, 32), and the coordinates of the target sample (xP, yP) are (56, 33). Since the horizontal coordinate xN of the top-left sample of the current CU (CU 3) is greater than 0 and the vertical coordinate yN of the top-left sample of the current CU (CU 3) is greater than 0, it may be determined that the current CU is not located at the edge of the picture. When spatial filtering is performed on the target sample within the current CU, the reference sample may be a reconstructed sample above the current CU and a reconstructed sample to the left of the current CU. When the reference sample is a reconstructed sample to the left of the current CU, the reconstructed sample has the same vertical coordinate as the target sample, which may be (xN-M1, 33). When the reference sample is a reconstructed sample above the current CU, the reconstructed sample has the same horizontal coordinate as the target sample and may be (56, yN-M2), where M1 and M2 respectively represent the predetermined spatial positional relationship between the reference sample and each of the target samples. In this specification, M1 and M2 are both equal to 1 as an example for explanation. When M1 and M2 are both equal to 1, the reference samples of target sample (56, 33) may be (47, 33) and (56, 31). Similarly, any reference sample of target sample (xP, yP) may be (xN-M1, yP) and (xP, yN-M2).

[0462] When the reference samples at positions (xN-M1, yP) and (xP, yN-M2) have been coded and reconstructed, a weighted calculation may be performed on the reconstructed values ​​of the reference samples recon(xN-M1, yP) and recon(xP, yN-M2) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0463] It should be noted that the values ​​of the weighting coefficients w1, w2, w3, w4, w5, and w6, as well as the values ​​of M1 and M2, are not limited in this embodiment of the present application, and both M1 and M2 being equal to 1 is used merely as an example.

[0464] For example, the weighting coefficient set (w1, w2), (w3, w4), or (w5, w6, w7) may be a value combination of w1+w2, w3+w4, or w5+w6+w7, where w1+w2, w3+w4, or w5+w6+w7 is equal to an integer power of 2. In this way, no division operation is performed. For example, value combinations such as (6, 2), (5, 3), (4, 4), (6, 1, 1), or (5, 2, 1) may be used. This is not limited to this embodiment of the present application, and the value combinations listed herein are used merely as examples for illustration.

[0465] In another possible implementation, the predicted value of the target sample may be updated according to the following equation:

[0466]

number

[0467] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are predetermined constants, and M1 and M2 are predetermined positive integers.

[0468] For example, the weighting coefficient set (w1, w2, w3) may be a value combination of w1+w2+w3, where w1+w2+w3 is equal to an integer power of 2. In this way, no division operation is performed. For example, value combinations such as (6, 1, 1) or (5, 2, 1) may be used. This is not limited in this embodiment of the present application, and the value combinations listed herein are used merely as examples for illustration.

[0469] Note that the difference between this implementation and the previous implementation is that samples to the left of the current picture block and samples above the current picture block are not considered in this implementation. When this implementation is used to update the predicted value of the target sample and the reconstructed value of the reference sample is unavailable, the methods in steps S13061 and S13062 below may be used to obtain a new reference sample and update the predicted value of the target sample based on the new reference sample.

[0470] In another possible implementation, the predicted value of the target sample may be updated according to the following equation:

[0471]

number

[0472] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and rec on(xP,yN-M4) represent the reconstructed values ​​of the reference samples at coordinate positions (xN-M1,yP), (xN-M2,yP), (xP,yN-M3), and (xP,yN-M4), respectively, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0473] The following specifically describes a method for calculating the updated predicted value of the target sample based on different coordinates (xN, yN) of the top-left sample of the current picture block.

[0474] In the first case, when xN is greater than 0, yN is equal to 0, and the reference samples at positions (xN-M1, yP) and (xN-M2, yP) have been coded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0475]

number

[0476] It can be understood that the difference between this case in this implementation and the first case in the first implementation is that there are two reference samples in this implementation. If the picture block to be processed is CU 1, the top-left sample (xN, yN) of the picture block to be processed (CU 1) is (16,0), and the coordinates of the target sample (xP, yP) in the picture block to be processed are (18,3). The reference samples are reconstructed samples to the left of the current CU. Each of the reference samples has the same vertical coordinate as the target sample and may be written as (16-M1,3) and (16-M2,3), where M1 and M2 respectively represent the predetermined spatial relationship between each of the reference samples and the target sample. In this specification, M1 equals 1 and M2 equals 2 is used as an example for explanation. When M1 equals 1 and M2 equals 2, the reference samples of the target sample (18,3) may be (15,3) and (14,3). Similarly, any reference samples of a target sample (xP, yP) can be (xN-1, yP) and (xN-2, yP).

[0477] When the reference samples at positions (xN-1, yP) and (xN-2, yP) have been coded and reconstructed, a weighted calculation may be performed on the reconstructed values ​​of the reference samples recon(xN-1, yP) and recon(xN-2, yP) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0478] In the second case, when xN is equal to 0, yN is greater than 0, and the reference samples at positions (xP, yN-M3) and (xP, yN-M4) have been coded and reconstructed, the updated predicted value of the target sample can be obtained according to the following formula:

[0479]

number

[0480] For example, the difference between this case in this implementation and the second case in the first implementation is that there are two reference samples in this implementation. If the picture block to be processed is CU 2, the coordinates (xN, yN) of the top-left sample of the picture block to be processed (CU 2) are (0, 32), and the coordinates of the target sample (xP, yP) are (8, 35). The reference samples are reconstructed samples above the current CU. Each of the reference samples has the same horizontal coordinate as the target sample and may be denoted as (8, 32-M3) and (8, 32-M4), where M3 and M4 represent the predetermined spatial relationship between each of the reference samples and the target sample, respectively. In this specification, M3 equals 1 and M4 equals 2 are used as an example for explanation. When M3 equals 1 and M4 equals 2, the reference samples of the target sample (8, 35) may be (8, 31) and (8, 30). Similarly, any reference samples of a target sample (xP, yP) can be (xP, yN-1) and (xP, yN-2).

[0481] When the reference samples at positions (xP, yN-1) and (xP, yN-2) have been coded and reconstructed, a weighted calculation can be performed on the reconstructed values ​​of the reference samples recon(xP, yN-1) and recon(xP, yN-2) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0482] In the third case, when xN is greater than 0, yN is greater than 0, and the reference samples at positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4) have been coded and reconstructed, the updated predicted value of the target sample can be obtained according to the following equation:

[0483]

number

[0484] For example, the difference between this case in this implementation and the third case in the first implementation is that in this implementation, there are two reconstructed samples as reference samples above the current CU, and two reconstructed samples as reference samples to the left of the current CU. If the picture block to be processed is CU 3, the coordinates of the top-left sample (xN, yN) of the picture block to be processed (CU 3) are (48, 32), and the coordinates of the target sample (xP, yP) are (56, 33). When the reference samples are reconstructed samples to the left of the current CU, the reconstructed samples each have the same vertical coordinates as the target sample, which may be (48-M1,33) and (48-M2,33). When the reference samples are reconstructed samples above the current CU, the reconstructed samples each have the same horizontal coordinates as the target sample, which may be (56,32-M3) and (56,32-M4), where M1, M2, M3, and M4 each represent a predetermined spatial relationship between each of the reference samples and the target sample. In this specification, the case where both M1 and M3 are equal to 1 and both M2 and M4 are equal to 2 is used as an example for explanation. When both M1 and M3 are equal to 1 and both M2 and M4 are equal to 2, the reference samples for target sample (56,33) may be (47,33), (46,33), (56,31), and (56,30). Similarly, any reference samples of a target sample (xP, yP) can be (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4).

[0485] When the reference samples at positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4) have been encoded and reconstructed, a weighted calculation may be performed on the reconstructed values ​​of the reference samples recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) and the predicted value of the target sample predP(xP, yP) to obtain an updated predicted value of the target sample predQ(xP, yP).

[0486] It should be noted that the values ​​of the weighting coefficients w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11, and the values ​​of M1, M2, M3, and M4 are not limited in this embodiment of the present application. It is used merely as an example for illustration purposes that both M1 and M3 are equal to 1 and both M2 and M4 are equal to 2. It can be understood that in actual applications, the values ​​of M1 and M3 may be the same or different, the values ​​of M2 and M4 may be the same or different, the values ​​of M1 and M2 may be different, and the values ​​of M3 and M4 may be different.

[0487] For example, the weighting coefficient set (w1, w2, w3), (w4, w5, w6), or (w7, w8, w9, w10, w11) may be a value combination of w1+w2+w3, w4+w5+w6, or w7+w8+w9+w10+w11, where w1+w2+w3, w4+w5+w6, or w7+w8+w9+w10+w11 is equal to an integer power of 2. In this way, no division operation is performed. For example, value combinations such as (6, 1, 1), (5, 2, 1), or (3, 2, 1, 1, 1) may be used. This is not limited in this embodiment of the present application, and the value combinations listed herein are used merely as examples for illustration.

[0488] In another implementation, the predicted value of the target sample may be updated according to the following equation:

[0489]

number

[0490] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, w1, w2, w3, w4, and w5 are predetermined constants, and M1, M2, M3, and M4 are predetermined positive integers.

[0491] For example, the weighting coefficient set (w1, w2, w3, w4, w5) may be a value combination of w1+w2+w3+w4+w5, where w1+w2+w3+w4+w5 is equal to an integer power of 2. In this way, a division operation is no longer performed. For example, a value combination such as (3, 2, 1, 1, 1) may be used. This is not limited in this embodiment of the present application, and the value combinations listed herein are used merely as examples for explanation. Note that the difference between this implementation and previous implementations is that samples to the left of the current picture block and samples above the current picture block are not considered in this implementation. When this implementation is used to update the predicted value of a target sample and the reconstructed value of the reference sample is unavailable, the methods in the following steps S13061 and S13062 may be used to obtain available reference samples and update the predicted value of the target sample based on the available reference samples.

[0492] In one possible implementation, the one or more reference samples include one or more of the following samples: a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the top of the picture block to be processed, a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left of the picture block to be processed, a reconstructed sample to the top right of the picture block to be processed, a reconstructed sample to the bottom left of the picture block to be processed, or a reconstructed sample to the top left of the picture block to be processed.

[0493] In another possible implementation, the predicted value of the target sample may be updated according to the following equation: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) where the coordinates of the target pixel are (xP, yP), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target pixel, and w1 and w2 are preset constants.

[0494] Specifically, a second predicted sample value predP1(xP, yP) may be first obtained based on spatial neighboring samples and a planar (PLANAR) intra prediction mode. It can be understood that in the PLANAR mode, two linear filters in the horizontal and vertical directions are used, and the average value obtained by the two linear filters is used as the predicted value of the sample in the current block.

[0495] For example, the second predicted sample value predP1(xP, yP) may be obtained based on the PLANAR mode. predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-(yP-yN))*recon(xP,yN-1)+(yP-yN+1)*recon(xN-1,yN+nTbH))< <Log2(nTbW)、 predH(xP,yP)=((nTbH-1-(xP-xN))*recon(xN-1,yP)+(xP-xN+1)*recon(xN+nTbW,yN-1))< <Log2(nTbH) Here, as shown in Figure 16, the coordinates of the top-left sample of the current picture block are (xN, yN), recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yP), and recon(xN+nTbW, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yN+nTbH), (xN-1, yP), and (xN+nTbW, yN-1), respectively, and nTbW and nTbH represent the width and height of the current CU (current picture block).

[0496] In another possible implementation, the predicted value of the target sample is updated according to the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) Here, predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the picture block to be processed are (0,0), and predP(xP,yP) is updated. where p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), respectively; w1 and w2 are preset constants; and nTbW and nTbH represent the width and height of the picture block to be processed.

[0497] In another possible implementation, the predicted value of the target sample is updated according to the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Here, predV(xP, yP) = ((nTbH - 1 - yP) * p(xP, -1) + (yP + 1) * p(-1, nTbH)) << Log2(nTbW), predH(xP, yP) = ((nTbW - 1 - xP) * p(-1, yP) + (xP + 1) * p(nTbW, -1)) << Log2(nTbH), where the coordinates of the target sample are (xP, yP), the coordinates of the upper left sample of the picture block to be processed are (0, 0), predP(xP, yP) represents the predicted value of the target sample before update, predQ(xP, yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values of the reference samples at (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1) respectively, w1 and w2 are pre-set constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0498] Note that the planar mode algorithm used to generate the second predicted sample value predP1(xP, yP) is not limited to the algorithm in VTM, and the planar algorithms in HEVC or H.264 can be alternatively used. This is not limited in this embodiment of the present application.

[0499] Note that the values of the weighting factors w1 and w2 are not limited in this embodiment of the present application. For example, the weighting factor set (w1, w2) may be a combination of values of w1 + w2, and w1 + w2 is equal to an integer power of 2. In this way, the division operation is no longer performed. For example, combinations of values such as (6, 2), (5, 3), or (4, 4) can be used. This is not limited in this embodiment of the present application, and the combinations of values listed in this specification are used only as examples for illustration.

[0500] In another possible implementation, the predicted value of the target sample can be updated according to the following formula. predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) where: predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+1)>>1, predV(xP,yP)=((nTbH-1-(yP-yN))*recon(xP,yN-1)+(yP-yN+1)*recon(xN-1,yN+nTbH)+(nTbH>>1))>>Log2(nTbH), predH(xP,yP)=((nTbW-1-(xP-xN))*recon(xN-1,yP)+(xP-xN+1)*recon(xN+nTbW,yN-1)+(nTbW>>1))>>Log2(nTbW) where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, yP), and recon(xN+nTbW, yN-1) represent the reconstructed values ​​of the reference samples at the coordinate positions (xP, yN-1), (xN-1, yN+nTbH), (xN-1, yP), and (xN+nTbW, yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

[0501] In another possible implementation, the inter-predicted block may be processed by using a position-dependent intra prediction combination process technique used in intra prediction, and the updated predicted value predQ(xP, yP) of the target sample may be obtained by using a DC mode method in the intra prediction combination process technique in the VTM. It can be understood that when the DC mode is used, the predicted value of the current block may be obtained based on the average value of the reference samples to the left and above the current block.

[0502] For example, the updated predicted value predQ(xP, yP) of the target sample may be obtained according to the following equation:

[0503]

number

[0504] Here, refL(xP, yP) = recon(xN-1, yP), refT(xP, yP) = recon(xP, yN-1), wT(yP) = 32>> ((yP<<1)>> nScale), wL(xP) = 32>> ((xP<<1)>> nScale), wTL(xP, yP) = ((wL(xP)>> 4) + (wT(yP)>> 4)), nScale = ((Log2(nTbW) + Log2(nTbH)-2)>> 2), and as shown in Figure 16, the coordinates of the target sample are (xP, yP), and the coordinate of the top-left sample of the picture block to be processed is (xP, yP). where the target is (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents the clipping operation.

[0505] It should be noted that the updated predicted value predQ(xP, yP) of the target sample can be obtained by not only using the intra-prediction combining processing technique in VTM, but also by using the algorithm in JEM.

[0506] In one possible implementation, the inter-prediction block may be processed by using the position-dependent intra-prediction combining processing technique used in intra-prediction, and the updated predicted value predQ(xP, yP) of the target sample may be obtained by using the PLANAR mode method in the intra-prediction combining processing technique in VTM. The following formula is used:

[0507]

number

[0508] Here, refL(xP, yP) = recon(xN-1, yP), refT(xP, yP) = recon(xP, yN-1), wT(yP) = 32>> ((yP<<1)>> nScale), wL(xP) = 32>> ((xP<<1)>> nScale), nScale = ((Log2(nTbW) + Log2(nTbH)-2)>> 2), and as shown in Figure 16, the coordinates of the target sample are (xP, yP), and the coordinates of the top-left sample of the picture block to be processed are ( xN,yN), predP(xP,yP) represents the predicted value of the target sample before updating, predQ(xP,yP) represents the updated predicted value of the target sample, recon(xN-1,yP) and recon(xP,yN-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xN-1,yP) and (xP,yN-1), respectively, nTbW and nTbH represent the width and height of the picture block to be processed, and clip1Cmp represents the clipping operation.

[0509] It should be noted that the updated predicted value predQ(xP, yP) of the target sample can be obtained by using the algorithm in JEM as well as the algorithm in VTM.

[0510] In one possible implementation, the filtering process can be performed on the inter-predicted samples by using intra-prediction boundary filtering techniques. For details of boundary filtering techniques, please refer to the HEVC method. Details will not be described here.

[0511] It should be noted that when the predicted value of the target sample is updated in any one of the above-mentioned manners, if the reconstructed value of the target sample is unavailable, step S1306 may further include the following steps S13601 and S13062.

[0512] S13061: When the reconstructed value of a reference sample is unavailable, determine the availability of samples adjacent to the upper and left sides of the picture block to be processed in a predetermined order until a predetermined amount of available reference samples is obtained.

[0513] It can be understood that the case in which the reconstructed value of the reference sample is unavailable may be when the current picture block is located at the top of the picture and there is no reconstructed value of the reference sample at the coordinate position (xP, yN-M), or when the current picture block is located at the left of the picture and there is no reconstructed value of the reference sample at the coordinate position (xN-N, yP), or when a reconstructed value cannot be obtained for the reference sample. The specific case in which the reconstructed value of the reference sample is unavailable is not limited in this embodiment of the present application, and this case is used here merely as an example for explanation.

[0514] 17, the preset order may be from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), and then from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). For example, to search for available reference samples from neighboring samples above and to the left of the current picture block, all samples may be traversed in the order from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), and then from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). It should be noted that the specific preset order is not limited in this embodiment of the present application, and the preset order in this specification is merely used as an example for explanation.

[0515] For example, when there is at least one available reference sample among all reference samples, if the reconstructed value of reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a predetermined order from coordinate (xN-1, yN+nTbH-1) to coordinate (xN-1, yN-1), then from coordinate (xN, yN-1) to coordinate (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of sample (x, y). If the reconstructed value of reference sample (x, y) is unavailable in the set containing reference sample (xN-1, yN+nTbH-M), then the reconstructed value of reference sample (x, y) is set to the reconstructed value of sample (x, y+1), where M is greater than or equal to 2 and less than or equal to nTbH+1. If the reconstructed value of reference sample (x, y) is unavailable in the set containing reference sample (xN+N, yN-1), then the reconstructed value of reference sample (x, y) is set to the reconstructed value of reference sample (x-1, y), where N is greater than or equal to 0 and less than or equal to nTbW-1.

[0516] For example, if the reconstructed value of reference sample (xN-1, yN+nTbH-M) is unavailable, available reference samples may be searched in a preset order starting from coordinate (xN-1, yN+nTbH-M), where M is greater than or equal to 1 and less than or equal to nTbH+1. If the available reference sample is B, the reconstructed value of reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of reference sample B. If the reconstructed value of the reference sample with coordinate (xN+N, yN-1) is unavailable, available reference samples may be searched in a preset order starting from coordinate (xN+N, yN-1), where N is greater than or equal to 0 and less than or equal to nTbW-1. If the available reference sample is C, the reconstructed value of reference sample (xN+N, yN-1) may be set to the reconstructed value of reference sample C.

[0517] For example, if the reconstructed value of reference sample (xN-1, yN+nTbH-3) is unavailable, the availability of upper and left neighboring samples of the current picture block may be determined in order from coordinates (xN-1, yN+nTbH-3) to coordinates (xN-1, yN-1) until a predetermined amount of available reference samples is obtained, and the reconstructed value of reference sample (xN-1, yN+nTbH-3) may be set to the reconstructed value of the available reference sample. If the reconstructed value of the reference sample with coordinates (xN+3, yN-1) is unavailable, available samples may be searched in order from coordinates (xN+3, yN-1) to coordinates (xN+nTbW-1, yN-1), and the reconstructed value of reference sample (xN+3, yN-1) may be set to the reconstructed value of the available reference sample.

[0518] For example, if the reconstructed value of reference sample (xN-1, yN+nTbH-1) is unavailable, available samples are searched in a preset order from coordinates (xN-1, yN+nTbH-1) to coordinates (xN-1, yN-1), then from coordinates (xN, yN-1) to coordinates (xN+nTbW-1, yN-1). When an available sample is found, the search ends. If the available sample is (x, y), the reconstructed value of reference sample (xN-1, yN+nTbH-1) is set to the reconstructed value of sample (x, y). If the reconstructed value of reference sample (xN-1, yN+nTbH-M) is unavailable, available reference samples may be searched in the reverse order of the preset order starting from coordinates (xN-1, yN+nTbH-M), where M is greater than 1 and less than or equal to nTbH+1. If the available reference sample is C, the reconstructed value of reference sample (xN-1, yN+nTbH-M) may be set to the reconstructed value of reference sample C. If the reconstructed value of the reference sample with coordinates (xN+N, yN-1) is unavailable, the available reference samples may be searched in reverse order from the preset order starting from coordinates (xN+N, yN-1), where N is greater than or equal to 0 and less than or equal to nTbW-1. If the available reference sample is D, the reconstructed value of reference sample (xN+N, yN-1) may be set to the reconstructed value of reference sample D.

[0519] For example, if the reconstructed value of reference sample (xN-1, yN+nTbH-3) is unavailable, the availability of upper and left neighboring samples of the current picture block may be determined in order from coordinate (xN-1, yN+nTbH-3) to coordinate (xN-1, yN+nTbH-1) until a predetermined amount of available reference samples is obtained, and the reconstructed value of reference sample (xN-1, yN+nTbH-3) may be set to the reconstructed value of the available reference sample. If the reconstructed value of the reference sample with coordinate (xN+3, yN-1) is unavailable, available samples may be searched in order from coordinate (xN+3, yN-1) to coordinate (xN, yN-1), and the reconstructed value of reference sample (xN+3, yN-1) may be set to the reconstructed value of the available reference sample.

[0520] It should be noted that the new reference sample may be the first available reference sample found in a preset order, or may be any available reference sample found in a preset order, which is not limited in this embodiment of the present application.

[0521] It can be understood that according to the method, when a reference sample is unavailable, an available reference sample may be searched for in a predetermined order from the samples adjacent to the upper and left sides of the picture block to be processed, and the reconstructed value of the available reference sample is used as the reconstructed value of the unavailable reference sample.

[0522] S13062: Perform weighted calculations on the reconstructed values ​​of available reference samples and the predicted values ​​of the target sample to update the predicted values ​​of the target sample.

[0523] For example, the predicted value of the target sample may be updated based on the reconstructed value of the new reference sample by using any one of the implementations described above.

[0524] If the reconstructed value of the reference sample is unavailable, in step S13061, it is determined that all samples adjacent to the upper and left sides of the current picture block are unavailable, and the reconstructed value of the reference sample may be set to 1<<(bitDepth-1), where bitDepth represents the bit depth of the sample value of the reference sample. For example, when the current picture block is located at the upper left corner of a picture, the coordinates of the upper left corner of the current picture block are (0,0), and all samples adjacent to the upper and left sides of the current picture block are unavailable. In this case, the reconstructed value of the reference sample corresponding to the target sample in the current picture block may be set to 1<<(bitDepth-1).

[0525] In the above-mentioned implementations, spatial filtering is performed on the inter-predicted samples in the process of generating the inter-predicted samples, which improves coding efficiency compared to conventional techniques.

[0526] In one possible implementation, before step S1306, the method may further include step S1305.

[0527] S1305: (Optionally) perform a filtering process on the reference sample.

[0528] It can be understood that step S1305 may be performed by the filter unit 206 in the video decoder of FIG.

[0529] For example, performing a filtering process on a reference sample may include performing a weighted calculation on a reconstructed value of the reference sample and reconstructed values ​​of neighboring samples to the left and right of the reference sample when the reference sample is located above the picture block to be processed, performing a weighted calculation on the reconstructed value of the reference sample and reconstructed values ​​of neighboring samples above and below the reference sample when the reference sample is located to the left of the picture block to be processed, and updating the reconstructed value of the reference sample by using the result of the weighted calculation.

[0530] It can be understood that after the filtering process is performed on the reference sample in step S1305, when step S1306 is performed, a weighted calculation can be performed on the updated reconstructed value of the reference sample after the filtering process and the predicted value of the target sample in order to update the predicted value of the target sample.

[0531] It should be noted that for a specific method for performing the filtering process on the reference samples, reference can be made to the filtering method in step S1306, and the details will not be described here.

[0532] It can be understood that to update the reconstructed values ​​of the reference samples, a filtering operation is performed on the reference samples, and based on the updated reconstructed values ​​of the reference samples, a filtering operation is performed on the target samples, which can further improve coding efficiency and reduce prediction residuals.

[0533] In one possible implementation, step S1307 may be further included before or after step S1306.

[0534] S1307: (Optionally) continue to perform inter prediction based on motion information and bitstream information by using an inter coding technique other than this method.

[0535] It may be understood that step S1307 may be performed by the inter predictor 210 in the video decoder of FIG.

[0536] For example, techniques in HEVC or VTM may be used, including, but not limited to, bidirectional optical flow, decoder-side motion vector refinement, local illumination compensation (LIC) technique, generalized weighted prediction (GBI), overlapped block motion compensation (OBMC), and decoder-side motion vector compensation (DMVD) technique. Methods in HEVC or VTM, or other methods for generating a motion vector prediction candidate list may be used. This is not limited in this embodiment of the present application.

[0537] It should be noted that the order of performing steps S1301 to S1307 of the above-mentioned method is not limited in this embodiment of the present application. For example, step S1305 may be performed before step S1307 or after step S1307. This is not limited in this embodiment of the present application.

[0538] In one possible implementation form, before performing motion compensation on the picture block to be processed based on the motion information, the method may further include a step of first updating the motion information by using a first predetermined algorithm, and correspondingly, the step of performing motion compensation on the picture block to be processed based on the motion information includes a step of first performing motion compensation on the picture block to be processed based on the updated motion information.

[0539] In another possible implementation form, after the prediction block of the picture block to be processed is obtained, the method may further include a step of pre-updating the prediction block by using a second pre-set algorithm, and correspondingly, the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the prediction value of the target sample includes a step of performing a weighted calculation on one or more reconstructed values ​​of the one or more reference samples and the pre-updated prediction value of the target sample.

[0540] In another possible implementation, after performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, the method further includes updating the predicted value of the target sample by using a second preset algorithm.

[0541] It should be further understood that after the updated predicted value of the target sample is obtained, the method may further include adding the final inter-predicted picture and the residual picture to obtain a reconstructed picture of the current block. Specifically, if the current block has a residual, the residual information is added to the predicted picture to obtain a reconstructed picture of the current block. If the current block does not have a residual, the predicted picture is the reconstructed picture of the current block. In the above process, the same method as in HEVC or VTM, or another motion compensation method or picture reconstruction method may be used. This is not limited.

[0542] According to the inter prediction method provided in this embodiment of the present application, a bitstream is analyzed to obtain motion information of a current picture block; motion compensation is performed on the current picture block based on the motion information to obtain a predicted block of the current picture block; and weighted calculation is performed on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, where the reference samples have a predetermined spatial relationship with the target sample. In this embodiment of the present application, after the predicted value of the target sample in the current picture block is obtained, a filtering process is performed on the predicted value of the target sample by using neighboring reconstructed samples. In this way, coding compression efficiency can be improved, and the PSNR BD rate is reduced by 0.5%. Compared with conventional techniques, the spatial filtering performed on the inter prediction sample in the process of generating the inter prediction sample improves coding efficiency.

[0543] An embodiment of the present application provides an inter prediction device. The device may be a video decoder. Specifically, the inter prediction device is configured to perform steps performed by a decoding device in the above-mentioned inter prediction method. The inter prediction device provided in this embodiment of the present application may include modules corresponding to the corresponding steps.

[0544] In this embodiment of the present application, the inter-prediction device may be divided into functional modules based on the above-mentioned method examples. For example, the functional modules may be obtained through division based on corresponding functions, or two or more functions may be integrated into one processing module. The integrated module may be implemented in the form of hardware or in the form of a software functional module. In this embodiment of the present application, the division into modules is an example, and is merely a logical division of functions, and other divisions may be used in actual implementation.

[0545] When the functional modules are obtained through division based on the corresponding functions, Figure 18 is a possible schematic structural diagram of the inter prediction device 1800 of the aforementioned embodiment. As shown in Figure 18, the inter prediction device 1800 may include an analysis module 1801, a compensation module 1802, and a calculation module 1803. Specifically, the functions of the modules are as follows:

[0546] The parsing module 1801 is configured to parse the bitstream to obtain motion information of a current picture block.

[0547] The compensation module 1802 is configured to perform motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, where the prediction block of the current picture block comprises a prediction value of a target sample.

[0548] The calculation module 1803 is configured to perform a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, the reference samples having a predetermined spatial relationship with the target sample.

[0549] In one possible implementation, the one or more reference samples include a reconstructed sample having the same horizontal coordinate as the target sample and a predetermined vertical coordinate difference from the target sample, or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference from the target sample.

[0550] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula:

[0551]

number

[0552] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are predetermined constants, and M1 and M2 are predetermined positive integers.

[0553] In one possible implementation, w1+w2=R, w3+w4=R, or w5+w6+w7=R, where R is the nth power of 2 and n is a non-negative integer.

[0554] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula:

[0555]

number

[0556] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and rec on(xP,yN-M4) represent the reconstructed values ​​of the reference samples at coordinate positions (xN-M1,yP), (xN-M2,yP), (xP,yN-M3), and (xP,yN-M4), respectively, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are preset constants, and M1, M2, M3, and M4 are preset positive integers.

[0557] In one possible implementation, w1+w2+w3=S, w4+w5+w6=S, or w7+w8+w9+w10+w11=S, where S is the nth power of 2 and n is a non-negative integer.

[0558] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula:

[0559]

number

[0560] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are predetermined constants, and M1 and M2 are predetermined positive integers.

[0561] In one possible implementation, w1+w2+w3=R, where R is the nth power of 2 and n is a non-negative integer.

[0562] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula:

[0563]

number

[0564] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the picture block to be processed are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before updating, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and recon(xP, yN-M4) represent the reconstructed values ​​of the reference samples at the coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, w1, w2, w3, w4, and w5 are predetermined constants, and M1, M2, M3, and M4 are predetermined positive integers.

[0565] In one possible implementation, w1+w2+w3+w4+w5=S, where S is the nth power of 2, and n is a non-negative integer.

[0566] In one possible implementation, the one or more reference samples include one or more of the following samples: a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the top of the picture block to be processed, a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left of the picture block to be processed, a reconstructed sample to the top right of the picture block to be processed, a reconstructed sample to the bottom left of the picture block to be processed, or a reconstructed sample to the top left of the picture block to be processed.

[0567] In one possible implementation, the calculation module 1803 updates the predicted value of the target sample according to the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) Here, predP1(xP,yP) = (predV(xP,yP) + predH(xP,yP) + nTbW * nTbH) >> (Log2(nTbW) + Log2(nTbH) + 1), predV(xP,yP) = ((nTbH - 1 - yP) * p(xP, -1) + (yP + 1) * p(-1, nTbH)) << Log2(nTbW), predH(xP,yP) = ((nTbW - 1 - xP) * p(-1, yP) + (xP + 1) * p(nTbW, -1)) << Log2(nTbH), where the coordinates of the target sample are (xP,yP), the coordinates of the upper-left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1), w1 and w2 are pre-set constants, and nTbW and nTbH represent the width and height of the picture block to be processed.

[0568] In a possible implementation form, the predicted value of the target sample is updated...

Claims

1. 1. An inter prediction method, comprising: analyzing the bitstream to obtain motion information of the picture block to be processed; performing motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, wherein the prediction block of the current picture block includes a prediction value of a target sample; performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, wherein the reference samples have a predetermined spatial relationship with the target sample; A method comprising:

2. the one or more reference samples a reconstructed sample having the same horizontal coordinate as the target sample and a predetermined vertical coordinate difference relative to the target sample; or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference relative to the target sample; 2. The method of claim 1, comprising:

3. updating the predicted value of the target sample; performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [Equation 1] 3. The method of claim 2, wherein coordinates of the target sample are (xP, yP), coordinates of an upper-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are predetermined constants, and M1 and M2 are predetermined positive integers.

4. 4. The method of claim 3, wherein w1+w2=R1, w3+w4=R2, or w5+w6+w7=R3, and R1, R2, and R3 are each a power of 2, where n is a non-negative integer.

5. updating the predicted value of the target sample; performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [Equation 2] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and 3. The method of claim 2, wherein recon(xP, yN-M4) represents the reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are predetermined constants, and M1, M2, M3, and M4 are predetermined positive integers.

6. 6. The method of claim 5, wherein w1+w2+w3=S1, w4+w5+w6=S2, or w7+w8+w9+w10+w11=S3, and S1, S2, and S3 are each a power of 2, where n is a non-negative integer.

7. updating the predicted value of the target sample; performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [Equation 3] 3. The method of claim 2, wherein the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are predetermined constants, and M1 and M2 are predetermined positive integers.

8. 8. The method of claim 7, wherein w1+w2+w3=R, R is the nth power of 2, and n is a non-negative integer.

9. updating the predicted value of the target sample; performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [Equation 4] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, 3. The method of claim 2, wherein recon(xP, yN-M4), recon(xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and recon(xP, yN-M4) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, where w1, w2, w3, w4, and w5 are predetermined constants, and M1, M2, M3, and M4 are predetermined positive integers.

10. 10. The method of claim 9, wherein w1+w2+w3+w4+w5=S, where S is the nth power of 2, and n is a non-negative integer.

11. the one or more reference samples a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the upper side of the current picture block; A reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left side of the picture block to be processed, A reconstructed sample at the upper right of the picture block to be processed, A reconstructed sample at the lower left of the picture block to be processed, or, A reconstructed sample at the upper left of the picture block to be processed The method according to claim 1, comprising one or more of the above.

12. The step of updating the predicted value of the target sample is Performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample including, The updated predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) where predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values of the reference sample at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1) respectively, w1 and w2 are pre-set constants, nTbW and nTbH represent the width and height of the picture block to be processed, or, The predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the current picture block are (0,0), and predP(xP,yP) is the previous value before the update. where p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), respectively; w1, w2, and w3 are preset constants; and nTbW and nTbH represent the width and height of the current picture block; or The predicted value of the target sample is expressed by the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Updated according to the following, predV(xP, yP) = ((nTbH - 1 - yP) * p(xP, -1) + (yP + 1) * p(-1, nTbH)) << Log2(nTbW), predH(xP, yP) = ((nTbW - 1 - xP) * p(-1, yP) + (xP + 1) * p(nTbW, -1)) << Log2(nTbH), where the coordinates of the target sample are (xP, yP), the coordinates of the upper left sample of the picture block to be processed are (0, 0), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1), w1, w2, and w3 are pre-set constants, and nTbW and nTbH represent the width and height of the picture block to be processed, the method according to claim 11. [

13. ] The step of updating the predicted value of the target sample is executing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample including, the updated predicted value of the target sample being obtained according to the following formula [Equation 5] predP1(xP, yP) = (predV(xP, yP) + predH(xP, yP) + 1) >> 1, predV(xP, yP) = ((nTbH - 1 - (yP - yN)) * recon(xP, yN - 1) + (yP - yN + 1) * recon(xN - 1, yN + nTbH) + (nTbH >> 1)) >> Log2(nTbH), predH(xP, yP) = ((nTbW - 1 - (xP - xN)) * recon(xN - 1, yP) + (xP - xN + 1) * recon(xN + nTbW, yN - 1) + (nTbW >> 1)) >> Log2(nTbW) ​ where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, y 12. The method of claim 11, wherein recon(xN+nTbW,yN-1) represents reconstructed values ​​of the reference samples at coordinate positions (xP,yN-1), (xN-1,yN+nTbH), (xN-1,yP), and (xN+nTbW,yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

14. 14. The method of claim 12 or 13, wherein the sum of w1 and w2 is the nth power of 2, where n is a non-negative integer.

15. updating the predicted value of the target sample; performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [Equation 6] where refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the picture block to be processed are (xN,yN), and pr 12. The method of claim 11, wherein edP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent reconstructed values ​​of the reference samples at coordinate positions (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents a clipping operation.

16. updating the predicted value of the target sample; performing a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample. and the updated predicted value of the target sample is expressed by the following formula: [Equation 7] where refL(xP, yP)=recon(xN-1, yP), refT(xP, yP)=recon(xP, yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), and predP 12. The method of claim 11, wherein (xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-1, yP) and recon(xP, yN-1) represent reconstructed values ​​of the reference samples at coordinate locations (xN-1, yP) and (xP, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents a clipping operation.

17. performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample; when the reconstructed value of the reference sample is unavailable, determining availability of upper and left neighboring samples of the current picture block in a predetermined order until a predetermined amount of available reference samples is obtained; performing a weighted calculation on the reconstructed values ​​of the available reference samples and the predicted values ​​of the target samples; 17. The method of any one of claims 1 to 16, comprising:

18. performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted values ​​of the target samples in the current picture block; performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of neighboring samples to the left and right of the reference sample when the reference sample is located above the current picture block; performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of upper and lower neighboring samples of the reference sample when the reference sample is located to the left of the current picture block; updating the reconstructed values ​​of the reference samples by using the results of the weighted calculations; 18. The method of any one of claims 1 to 17, comprising:

19. before the step of performing motion compensation on the current picture block based on the motion information, initially updating the motion information by using a first preset algorithm; further comprising Correspondingly, the step of performing motion compensation on the current picture block based on the motion information comprises: first performing motion compensation on the current picture block based on the updated motion information; 19. The method of any one of claims 1 to 18, comprising:

20. After the prediction block of the current picture block is obtained, the method further comprises: Pre-updating the prediction block by using a second pre-defined algorithm. further comprising Correspondingly, said step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and said predicted value of said target sample comprises: performing a weighted calculation on the one or more reconstructed values ​​of the one or more reference samples and the previously updated predicted values ​​of the target sample; 20. The method of any one of claims 1 to 19, comprising:

21. after the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, updating the predicted value of the target sample by using a second preset algorithm; 20. The method of any one of claims 1 to 19, further comprising:

22. before the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, analyzing the bitstream to obtain a prediction mode corresponding to the current picture block; determining that the prediction mode is a merge mode and / or an inter-evolutionary motion vector prediction (AMVP) mode; 22. The method of any one of claims 1 to 21, further comprising:

23. before the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, analyzing the bitstream to obtain update determination instruction information for the current picture block; determining that the update decision indication information is used to indicate that the prediction block of the current picture block is to be updated; 23. The method of any one of claims 1 to 22, further comprising:

24. before the step of performing a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample, obtaining predetermined update determination instruction information for the picture block to be processed; determining that the update decision indication information is used to indicate that the prediction block of the current picture block is to be updated; 23. The method of any one of claims 1 to 22, further comprising:

25. 1. A prediction method comprising: obtaining a first predicted value of a target sample through inter prediction; obtaining a second predicted value of the target sample through intra prediction; performing a weighting calculation on the first predicted value and the second predicted value to obtain an updated predicted value of the target sample, wherein a weighting coefficient of the first predicted value is different from a weighting coefficient of the second predicted value; A method comprising:

26. The method of claim 25 , wherein the second prediction value is obtained based on spatial neighboring samples by using a planar intra prediction mode.

27. The step of obtaining a first predicted value of a target sample through inter prediction includes: analyzing the bitstream to obtain motion information of the picture block to be processed; performing motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, wherein the prediction block of the current picture block includes the first prediction value of the target sample; 26. The method of claim 25, comprising:

28. the updated prediction value predQ(xP,yP) is obtained based on w1*predP(xP,yP) and w2*predP1(xP,yP); 28. The method of claim 25, wherein (xP, yP) represents the coefficients of the target sample, predP(xP, yP) represents the first predicted value, predP1(xP, yP) represents the second predicted value, w1 represents the weighting coefficient of the first predicted value, w2 represents the weighting coefficient of the second predicted value, w1 and w2 are preset constants, and w1 is not equal to w2.

29. 29. The method of claim 28, wherein the weighting coefficient set (w1, w2) is (6, 2) or (5, 3).

30. 29. The method of claim 28, wherein the sum of w1 and w2 is the nth power of 2, where n is a non-negative integer.

31. An inter prediction device, an analysis module configured to analyze the bitstream to obtain motion information of the current picture block; a compensation module configured to perform motion compensation on the current picture block based on the motion information to obtain a prediction block of the current picture block, wherein the prediction block of the current picture block includes a prediction value of a target sample; and a calculation module configured to perform a weighted calculation on one or more reconstructed values ​​of one or more reference samples and the predicted value of the target sample to update the predicted value of the target sample, wherein the reference samples have a predetermined spatial relationship with the target sample; An apparatus comprising:

32. the one or more reference samples a reconstructed sample having the same horizontal coordinate as the target sample and a predetermined vertical coordinate difference relative to the target sample; or a reconstructed sample having the same vertical coordinate as the target sample and a predetermined horizontal coordinate difference relative to the target sample; 32. The apparatus of claim 31 , comprising:

33. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [Equation 8] wherein coordinates of the target sample are (xP, yP), coordinates of a top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, w3, w4, w5, and w6 are predetermined constants, and M1 and M2 are predetermined positive integers.

34. 34. The apparatus of claim 33, wherein w1+w2=R1, w3+w4=R2, or w5+w6+w7=R3, and R1, R2, and R3 are each a power of 2, where n is a non-negative integer.

35. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [Equation 9] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, yN-M3), and r 33. The apparatus of claim 32, wherein econ(xP, yN-M4) represents reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, and w11 are predetermined constants, and M1, M2, M3, and M4 are predetermined positive integers.

36. 36. The apparatus of claim 35, wherein w1+w2+w3=S1, w4+w5+w6=S2, or w7+w8+w9+w10+w11=S3, and S1, S2, and S3 are each a power of 2, where n is a non-negative integer.

37. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [Equation 10] wherein coordinates of the target sample are (xP, yP), coordinates of a top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-M1, yP) and recon(xP, yN-M2) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP) and (xP, yN-M2), respectively, w1, w2, and w3 are predetermined constants, and M1 and M2 are predetermined positive integers.

38. 38. The apparatus of claim 37, wherein w1 + w2 + w3 = R, where R is the nth power of 2, and n is a non-negative integer.

39. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [0011] where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xN-M1, yP), recon(xN-M2, yP), recon(xP, y 33. The apparatus of claim 32, wherein w1, w2, w3, w4, and w5 are predetermined constants, and M1, M2, M3, and M4 are predetermined positive integers, and recon(xP, yN-M4), recon(xP, yN-M1, yP), (xN-M2, yP), (xP, yN-M3), and recon(xP, yN-M4) represent reconstructed values ​​of the reference samples at coordinate positions (xN-M1, yP), (xN-M2, yP), (xP, yN-M3), and (xP, yN-M4), respectively, and w1, w2, w3, w4, and w5 are predetermined positive integers.

40. 40. The apparatus of claim 39, wherein w1+w2+w3+w4+w5=S, where S is the nth power of 2, and n is a non-negative integer.

41. the one or more reference samples a reconstructed sample having the same horizontal coordinate as the target sample and adjacent to the upper side of the current picture block; a reconstructed sample having the same vertical coordinate as the target sample and adjacent to the left of the current picture block; the top right reconstructed sample of the current picture block; the bottom-left reconstructed sample of the current picture block, or The upper left reconstructed sample of the current picture block 32. The apparatus of claim 31, comprising one or more of:

42. the calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample; The updated predicted value of the target sample is calculated using the following formula: predQ(xP,yP)=(w1*predP(xP,yP)+w2*predP1(xP,yP)+((w1+w2) / 2)) / (w1+w2) obtained according to, predP1(xP,yP)=(predV(xP,yP)+predH(xP,yP)+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1), predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH))<<Log2(nTbW), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1))<<Log2(nTbH), where the coordinates of the target sample are (xP,yP), the coordinates of the upper left sample of the picture block to be processed are (0,0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP,-1), p(-1,nTbH), p(-1,yP), and p(nTbW,-1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP,-1), (-1,nTbH), (-1,yP), and (nTbW,-1), w1 and w2 are preset constants, nTbW and nTbH represent the width and height of the picture block to be processed, or, the predicted value of the target sample is given by the following formula predQ(xP,yP)=(w1*predP(xP,yP)+w2*predV(xP,yP)+w3*predH(xP,yP)+((w1+w2+w3) / 2)) / (w1+w2+w3) predV(xP,yP)=((nTbH-1-yP)*p(xP,-1)+(yP+1)*p(-1,nTbH)+nTbH / 2)>>Log2(nTbH), predH(xP,yP)=((nTbW-1-xP)*p(-1,yP)+(xP+1)*p(nTbW,-1)+nTbW / 2)>>Log2(nTbW), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the current picture block are (0,0), and predP(xP,yP) is the value before the update. where p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) represent the reconstructed values ​​of the reference samples at coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1), respectively; w1 and w2 are preset constants; and nTbW and nTbH represent the width and height of the current picture block; or The predicted value of the target sample is expressed by the following formula: predQ(xP,yP)=(((w1*predP(xP,yP))<<(Log2(nTbW)+Log2(nTbH)+1))+w2*predV(xP,yP)+w3*predH(xP, yP)+(((w1+w2+w3) / 2)<<(Log2(nTbW)+Log2(nTbH)+1))) / (((w1+w2+w3)<<(Log2(nTbW)+Log2(nTbH)+1))) Updated according to the following, predV(xP,yP) = ((nTbH - 1 - yP) * p(xP, -1) + (yP + 1) * p(-1, nTbH)) << Log2(nTbW), predH(xP,yP) = ((nTbW - 1 - xP) * p(-1, yP) + (xP + 1) * p(nTbW, -1)) << Log2(nTbH), where the coordinates of the target sample are (xP, yP), the coordinates of the top - left sample of the picture block to be processed are (0, 0), predP(xP,yP) represents the predicted value of the target sample before the update, predQ(xP,yP) represents the updated predicted value of the target sample, p(xP, -1), p(-1, nTbH), p(-1, yP), and p(nTbW, -1) respectively represent the reconstructed values of the reference samples at the coordinate positions (xP, -1), (-1, nTbH), (-1, yP), and (nTbW, -1), w1, w2, and w3 are pre - set constants, and nTbW and nTbH represent the width and height of the picture block to be processed, the apparatus according to claim 41.

43. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain the updated predicted value of the target sample, and the updated predicted value of the target sample is obtained according to the following formula [0012] and is obtained as follows predP1(xP,yP) = (predV(xP,yP) + predH(xP,yP) + 1) >> 1, predV(xP,yP) = ((nTbH - 1 - (yP - yN)) * recon(xP, yN - 1) + (yP - yN + 1) * recon(xN - 1, yN + nTbH) + (nTbH >> 1)) >> Log2(nTbH), predH(xP,yP) = ((nTbW - 1 - (xP - xN)) * recon(xN - 1, yP) + (xP - xN + 1) * recon(xN + nTbW, yN - 1) + (nTbW >> 1)) >> Log2(nTbW) where the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), predP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, and recon(xP, yN-1), recon(xN-1, yN+nTbH), recon(xN-1, y 42. The apparatus of claim 41, wherein recon(xN+nTbW,yN-1) represents reconstructed values ​​of the reference samples at coordinate locations (xP,yN-1), (xN-1,yN+nTbH), (xN-1,yP), and (xN+nTbW,yN-1), respectively, w1 and w2 are preset constants, and nTbW and nTbH represent the width and height of the current picture block.

44. 44. The apparatus of claim 42 or 43, wherein the sum of w1 and w2 is a power of 2, where n is a non-negative integer.

45. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [0013] where refL(xP,yP)=recon(xN-1,yP), refT(xP,yP)=recon(xP,yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), wTL(xP,yP)=((wL(xP)>>4)+(wT(yP)>>4)), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP,yP), the coordinates of the top-left sample of the picture block to be processed are (xN,yN), and pr 42. The apparatus of claim 41, wherein edP(xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xP, yN-1), recon(xN-1, yP), and recon(xN-1, yN-1) represent reconstructed values ​​of the reference samples at coordinate locations (xP, yN-1), (xN-1, yP), and (xN-1, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents a clipping operation.

46. The calculation module is further configured to perform a weighted calculation based on the predicted value of the target sample before the update and the reconstructed value of the reference sample to obtain an updated predicted value of the target sample, wherein the updated predicted value of the target sample is expressed by the following formula: [0014] where refL(xP, yP)=recon(xN-1, yP), refT(xP, yP)=recon(xP, yN-1), wT(yP)=32>>((yP<<1)>>nScale), wL(xP)=32>>((xP<<1)>>nScale), nScale=((Log2(nTbW)+Log2(nTbH)-2)>>2), the coordinates of the target sample are (xP, yP), the coordinates of the top-left sample of the current picture block are (xN, yN), and predP 42. The apparatus of claim 41, wherein (xP, yP) represents the predicted value of the target sample before the update, predQ(xP, yP) represents the updated predicted value of the target sample, recon(xN-1, yP) and recon(xP, yN-1) represent reconstructed values ​​of the reference sample at coordinate locations (xN-1, yP) and (xP, yN-1), respectively, nTbW and nTbH represent the width and height of the current picture block, and clip1Cmp represents a clipping operation.

47. the computing module: when the reconstructed value of the reference sample is unavailable, determining availability of upper and left neighboring samples of the current picture block in a predetermined order until a predetermined amount of available reference samples is obtained; performing a weighted calculation on the reconstructed values ​​of the available reference samples and the predicted values ​​of the target samples; 47. The apparatus of any one of claims 31 to 46, further configured to:

48. the computing module: performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of a left neighboring sample and a right neighboring sample of the reference sample when the reference sample is located above the current picture block; When the reference sample is located to the left of the current picture block, performing a weighted calculation on the reconstructed value of the reference sample and the reconstructed values ​​of the upper and lower neighboring samples of the reference sample; updating the reconstructed values ​​of the reference samples by using the results of the weighting calculations; 48. The apparatus of any one of claims 31 to 47, further configured to:

49. the calculation module is further configured to initially update the motion information by using a first preset algorithm; 49. The apparatus of claim 31, wherein the compensation module is further configured to correspondingly perform motion compensation on the current picture block based on the first updated motion information.

50. the calculation module is further configured to pre-update the prediction block by using a second pre-set algorithm; 50. The apparatus of claim 31, wherein the calculation module is further configured to perform a weighting calculation on the one or more reconstructed values ​​of the one or more reference samples and the previously updated predicted value of the target sample.

51. 50. The apparatus of claim 31, wherein the calculation module is further configured to update the predicted value of the target sample by using a second preset algorithm.

52. The analysis module: Analyzing the bitstream to obtain a prediction mode corresponding to the current picture block; determining that the prediction mode is a merge mode and / or an inter-evolutionary motion vector prediction (AMVP) mode; 52. The apparatus of any one of claims 31 to 51, further configured to:

53. The analysis module: analyzing the bitstream to obtain update determination instruction information for the picture block to be processed; determining that the update decision indication information is used to indicate updating of the prediction block of the current picture block; 53. The apparatus of any one of claims 31 to 52, further configured to:

54. the computing module: obtaining predetermined update determination instruction information for the picture block to be processed; determining that the update decision indication information is used to indicate updating of the prediction block of the current picture block; 53. The apparatus of any one of claims 31 to 52, further configured to:

55. 55. The apparatus of claim 31, wherein weighting coefficients of the one or more reconstructed values ​​of the one or more reference samples differ from weighting coefficients of the predicted values ​​of the target sample.

56. 56. The apparatus of claim 31, wherein the one or more reconstructed values ​​of the one or more reference samples are obtained based on spatial neighboring samples by using a planar intra prediction mode.

57. the updated prediction value predQ(xP,yP) is obtained based on w1*predP(xP,yP) and w2*predP1(xP,yP); 57. The apparatus of claim 31, wherein (xP, yP) represents a coefficient of the target sample, predP(xP, yP) represents the predicted value of the target sample, predP1(xP, yP) represents the one or more reconstructed values ​​of the one or more reference samples, w1 represents a weighting coefficient of the one or more reconstructed values ​​of the one or more reference samples, and w2 represents a weighting coefficient of the predicted value of the target sample, where w1 and w2 are predetermined constants and w1 is not equal to w2.

58. 58. The apparatus of claim 57, wherein the weighting coefficient set (w1, w2) is (6, 2) or (5, 3).

59. 58. The device of claim 57, wherein the sum of w1 and w2 is a power of 2, where n is 2 or 3.

60. 31. A computer storage medium having stored thereon computer program code which, when executed on a processor, enables the processor to carry out a method according to any one of claims 1 to 30.

Images