Adaptive weighting method and apparatus for overlapped block motion compensation in video coding system

Abstract

A method and apparatus for overlapping boundary motion compensation (OBMC). A current MV (motion vector) of a current block / sub-block is determined, and a neighboring MV of a neighboring block / sub-block associated with the current block / sub-block is determined. A number of rows, one or more OMBC weights related to the number of rows, or both, to be applied to OBMC (Overlapped Block Motion Compensation) is determined based on one or more boundary matching costs between a current block / sub-block and a neighboring block / sub-block, an MV difference between a current MV and a neighboring MV, or predicted luma data associated with the current block / sub-block. The OBMC is applied to at least one of the current block / sub-block and the neighboring block / sub-block according to an OBMC parameter including a number of rows, the one or more weights, or both.

Description

[0001] Cross-references

[0002] The present invention is a non-provisional application, claiming the priority of the U.S. Provisional Patent Application No. 63 / 336,395 filed on April 29, 2022. The above-mentioned U.S. Provisional Patent Application is incorporated into this specification by reference in its entirety. Technical Field

[0003] The present invention relates to video coding and decoding, and more particularly to adaptive design of number of lines and weightings for the lines related to Overlapped Block Motion Compensation (OBMC) processing in a video coding and decoding system. Background Art

[0004] Versatile video coding (VVC) is the latest international video coding standard jointly developed by the ITU-T Video Coding Experts Group (VCEG) and the Joint Video Experts Group (JVET) of the ISO / IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology-Coded representation of immersive media-Part 3:Versatile Video Coding, released in February 2021. VVC improves the coding efficiency by adding more coding tools on the basis of its predecessor HEVC (High Efficiency Video Coding), and can also process various types of video sources, including 3-dimensional (3D) video signals.

[0005] Figure 1AAn exemplary adaptive Inter / Intra video coding system including loop processing is illustrated. For intra prediction, prediction data is derived from previously coded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112 and the selected prediction data is provided to adder 116 to form a prediction error, also called a residual. The prediction error is then processed by transform (T) 118 and subsequent quantization (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 to be included in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with side information (e.g., motion and codec modes associated with intra-frame prediction and inter-frame prediction) and other information (e.g., parameters associated with the loop filter applied to the underlying image area). Side information associated with intra-frame prediction 110, inter-frame prediction 112, and in-loop filter 130 is provided to entropy encoder 122, as shown in FIG. Figure 1A As shown. When using inter-frame prediction mode, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to the prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in a reference picture buffer 134 and used to predict other frames.

[0006] like Figure 1A As shown, the input video data undergoes a series of processes in the encoding system. Due to the series of processes, the reconstructed video data from REC 128 may be subject to various impairments. Therefore, the loop filter 130 is often applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve the video quality. For example, a deblocking filter (DF), a sample adaptive offset (SAO), and an adaptive loop filter (ALF) may be used. It may be necessary to merge the loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 to be merged into the bitstream. In Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to a High Efficiency Video Codec (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use similar or identical functional blocks as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. Instead of the entropy encoder 122, the decoder uses an entropy decoder 140 to decode the video bitstream into quantized transform coefficients and required codec information (such as ILPF information, intra-frame prediction information, and inter-frame prediction information). The intra-frame prediction 150 on the decoder side does not need to perform a pattern search. Instead, the decoder only needs to generate an intra-frame prediction based on the intra-frame prediction information received from the entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) 140 based on the inter-frame prediction information received from the entropy decoder without motion estimation.

[0008] According to VVC, similar to HEVC, the input picture is partitioned into non-overlapped square block regions called CTUs (Codec Tree Units). Each CTU can be divided into one or more smaller-sized Codec Units (CUs). The generated CU partitions can be square or rectangular. In addition, VVC divides CTUs into Prediction Units (PUs) as units for applying prediction processing (e.g., inter-frame prediction, intra-frame prediction, etc.).

[0009] The VVC standard incorporates various new codec tools to further improve the codec efficiency over the HEVC standard. In addition, various new codec tools have been proposed for consideration in the development of new codec standards other than VVC. Among the various new codec tools, a brief description of some codec tools related to the present invention is as follows.

[0010] Overlapped Block Motion Compensation (OBMC)

[0011] Overlapped block motion compensation (OBMC) finds a linear minimum mean square error (LMMSE) estimate of a pixel intensity value based on a motion-compensated signal derived from its nearby block motion vectors (MVs). From an estimation-theoretic point of view, these MVs are viewed as different plausible hypotheses of their true motion, and to maximize codec efficiency, their weights should minimize the mean square prediction error subject to a unity gain constraint.

[0012] When High Efficiency Video Codec (HEVC) was developed, several proposals were made to use OBMC to provide codec gains. Some of them are described below.

[0013] In JCTVC-C251 (Peisong Chen et al., "Overlapped block motion compensation in TMuC", Joint Collaborative Group on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 3rd Meeting: Guangzhou, China, October 7-15, 2010, Document: JCTVC-C251), OBMC is applied to geometric partitioning. In geometric partitioning, a transform block is likely to contain pixels belonging to different partitions. In geometric partitioning, since two different motion vectors are used for motion compensation, the pixels at the partition boundary may have large discontinuities, resulting in visual artifacts similar to blocking effects. This in turn reduces the transform efficiency. The two regions created by geometric partitioning are denoted by region 1 and region 2. A pixel from region 1 (2) is defined as a boundary pixel if any of its four connected neighbors (left, top, right, and bottom) belongs to region 2 (1). Figure 2 An example is shown where the gray pixel belongs to the boundary of region 1 (gray region) and the white pixel belongs to the boundary of region 2 (white region). If the pixel is a boundary pixel, motion compensation is performed using a weighted sum of motion predictions from two motion vectors. The weight is 3 / 4 for the prediction using the motion vector of the region containing the boundary pixel and 1 / 4 for the prediction using the motion vector of the other region. Overlapping boundaries improves the visual quality of the reconstructed video while also providing BD rate gains.

[0014] In JCTVC-F299 (Liwei Guo et al., "CE2: Overlapped Block Motion Compensation for 2NxN and Nx2N Motion Partitions", ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 Joint Collaboration Group on Video Coding (JCT-VC), 6th Meeting: Torino, July 14-22, 2011, document: JCTVC-F299), OBMC is applied to symmetrical motion partitions. If the codec unit (CU) is divided into 2 2NxN or Nx2N prediction units (PUs), OBMC is applied to the horizontal boundary of two 2NxN prediction blocks and the vertical boundary of two Nx2N prediction blocks. Since these partitions may have different motion vectors, the pixels at the partition boundary may have large discontinuities, which may produce visual artifacts and also reduce the transform / codec efficiency. In JCTVC-F299, OBMC is introduced to smooth the boundaries of motion partitions.

[0015] Figure 3A -B illustrates the use of 2NxN ( Figure 3A ) and Nx2N blocks ( Figure 3B ). Gray pixels are pixels belonging to partition 0, and white pixels are pixels belonging to partition 1. The overlapping area in the luminance component is defined as 2 rows (columns) of pixels on either side of the horizontal (vertical) boundary. For pixels 1 row (column) away from the partition boundary, that is, Figure 3A - For the pixel marked as A in B, the OBMC weighting factor is (3 / 4, 1 / 4). For the pixel 2 columns (columns) away from the partition boundary, that is, Figure 3A - For pixels marked as B in B, the OBMC weighting factor is (7 / 8, 1 / 8). For chrominance components, the overlap area is defined as 1 column (bar) of pixels on each side of the horizontal (vertical) boundary, and the weighting factor is (3 / 4, 1 / 4).

[0016] Currently, OBMC is performed after normal MC, and BIO is also applied in these two MC processes separately. That is, the MC result of the overlapping area between two CUs or PUs is generated by another process instead of in the normal MC process. Bi-Directional Optical Flow (BIO) is then applied to improve these two MC results. This helps skip redundant OBMC and BIO processes when two adjacent MVs are the same. However, the bandwidth and MC operations required for the overlapping area are increased compared to integrating the OBMC process into the normal MC process. For example, the current PU size is 16x8, the overlapping area is 16x2, and the interpolation filter in MC is 8-tap. If OBMC is performed after normal MC, then for the current PU and the related OBMC, we need (16+7)x(8+7)+(16+7)x(2+7)=552 reference pixels for each reference list. If the OBMC operation is merged into one stage with the normal MC, each reference list of the current PU and the related OBMC has only (16+7)x(8+2+7)=391 reference pixels. Therefore, in the following, in order to reduce the computational complexity or memory bandwidth of BIO, several methods are proposed when BIO and OBMC are enabled at the same time.

[0017] In JEM (Joint Exploration Model), OBMC is also applied. In JEM, unlike H.263, OBMC can be turned on and off using CU-level syntax. When OBMC is used in JEM, OBMC is performed on all motion compensation (MC) block boundaries except the right and bottom boundaries of the CU. In addition, it applies to both luminance and chrominance components. In JEM, one MC block corresponds to one codec block. When the CU is encoded and decoded in sub-CU mode (including sub-CU merging, affine, and FRUC modes), each sub-block of the CU is an MC block. In order to handle CU boundaries in a unified manner, OBMC is performed on all MC block boundaries at the sub-block level, where the sub-block size is set to 4×4, such as Figure 4A as well as Figure 4B shown.

[0018] When OBMC is applied to the current sub-block, in addition to the current motion vector, the motion vectors of four adjacent neighboring sub-blocks (if available and different from the current motion vector) are also used to derive the prediction block of the current sub-block. These multiple prediction blocks based on multiple motion vectors are combined to generate the final prediction signal of the current sub-block. The prediction block based on the motion vector of the adjacent sub-block is denoted as PN, N represents the index of the adjacent upper, lower, left, and right sub-blocks, and the prediction block based on the motion vector of the current sub-block is denoted as PC. Figure 4AAn example of OBMC of a sub-block of the current CU 410 is shown, which uses an adjacent upper sub-block (i.e., when the current sub-block is PN1), a left adjacent sub-block (i.e., when the current sub-block is PN2), and a left and upper sub-block (i.e., when the current sub-block is PN3). Figure 4B An example of OBMC for the current CU 420 for ATMVP mode is illustrated, where the block PN uses MVs from four neighboring sub-blocks for OBMC. When the PN is based on the motion information of a neighboring sub-block containing the same motion information as the current sub-block, OBMC is not performed from the PN. Otherwise, each sample of the PN is added to the same sample in the PC, that is, four columns / columns of the PN are added to the PC. Weight factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for the PN, and weight factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are used for the PC. Small MC blocks (i.e., when the height or width of the codec block is equal to 4 or the CU is encoded and decoded in sub-CU mode) are exceptions, and only two columns / columns of the PN are added to the PC. In this case, weight factors {1 / 4, 1 / 8} are used for the PN, and weight factors {3 / 4, 7 / 8} are used for the PC. For PNs generated based on motion vectors of vertically (horizontally) neighboring sub-blocks, samples in the same column (column) of the PN are added to the PC with the same weight factor.

[0019] In JEM, for CUs with a size less than or equal to 256 luma samples, a CU level flag is sent to indicate whether OBMC is applied to the current CU. For CUs with a size greater than 256 luma samples or not encoded and decoded using AMVP mode, OBMC is applied by default. At the encoder, when OBMC is applied to a CU, its impact is taken into account in the motion estimation stage. The prediction signal formed by OBMC using the motion information of the top neighboring blocks and the left neighboring blocks is used to compensate the top and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

[0020] In the Joint Exploration Model for VVC development (JEM), OBMC is applied. Figure 5As shown, for the current block 510, if the block above and the block on the left are coded and decoded in inter-frame mode, the MV of the block above is taken to generate OBMC block A, and the MV of the block on the left is taken to generate OBMC block L. The prediction sub-blocks of OBMC block A and OBMC block L are blended with the current prediction sub-block. In order to reduce the memory bandwidth of OBMC, it is recommended to execute the MC of the 4 rows above, the MC of the 4 columns on the left, and the adjacent blocks. For example, when executing the MC of the block above, 4 additional columns are taken out to generate a block of (above block + OBMC block A) (above block + OBMC block A). The prediction sub-block of OBMC block A is stored in the buffer for coding and decoding the current block. When executing the MC of the block on the left, 4 additional columns are taken out to generate a block of (left block + OBMC block L) (left block + OBMC block L). The prediction sub-block of OBMC block L is stored in the buffer for coding and decoding the current block. Therefore, when performing MC on the current block, four columns and four columns of reference pixels are additionally taken out to generate the prediction sub, OBMC block B and OBMC block R of the current block, as shown in FIG. Fig. 6A As shown (it may also generate Figure 6B OBMC block B and OBMC block R are stored in a buffer for OBMC processing of the bottom neighboring block and the right neighboring block.

[0021] For MxN blocks, if MV is not an integer and an 8-tap interpolation filter is applied, a reference block of size (M+7)x(N+7) is used for motion compensation. However, if BIO and OBMC are applied, additional reference pixels are required, which increases the worst-case memory bandwidth.

[0022] There are two different schemes to implement OBMC. In the first scheme, OBMC blocks are generated in advance when motion compensation is performed on each block. These OBMC blocks will be stored in a local buffer for neighboring blocks. In the second scheme, OBMC blocks are generated before the blending process of each block when OBMC is performed.

[0023] In both schemes, several methods are proposed to reduce the computational complexity, especially for interpolation filtering, and the additional bandwidth requirement of OBMC.

[0024] Template Matching based OBMC

[0025] Recently, a template matching-based OBMC scheme (JVET-Y0076) was proposed for the emerging international coding standards. Figure 7As shown, for each top block of size 4×4 at the top CU boundary, the above template size is equal to 4×1. Figure 7 In the example, block 710 corresponds to a CU. If N neighboring blocks have the same motion information, the above template size is enlarged to 4N×1 because the MC operation can be processed at one time, which is the same as in ECM-OBMC. For each left block of size 4×4 at the left CU boundary, the left template size is equal to 1×4 or 1×4N.

[0026] For each 4×4 top block (or N 4×4 block groups), the prediction values ​​of the boundary samples are derived according to the following steps:

[0027] – Take block A as the current block and its above neighbor block AboveNeighbour_A as an example. The operation of the left block is performed in the same way.

[0028] – First, three template matching costs (Cost1, Cost2, Cost3) are measured between the reconstructed samples of the template and the corresponding reference samples derived by the MC process according to the following three types of motion information:

[0029] i.Cost1 is calculated based on A's motion information.

[0030] ii. Cost2 is calculated based on the motion information of AboveNeighbour_A.

[0031] iii. Cost3 is calculated based on the weighted prediction of the motion information of A and AboveNeighbour_A, with weighting factors of 3 / 4 and 1 / 4 respectively.

[0032] –Secondly, choose one of the three methods to calculate the final prediction result of the boundary samples by comparing Cost1, Cost2 and Cost 3.

[0033] The original MC result using the motion information of the current block is denoted as Pixel1, and the MC result using the motion information of the neighboring block is denoted as Pixel2. The final prediction result is denoted as NewPixel.

[0034] If Cost1 is the smallest, NewPixel(i,j)=Pixel1(i,j).

[0035] If Cost2 is the smallest, use blending mode 1.

[0036] For luma blocks, the number of blending pixel rows is 4.

[0037] -NewPixel(i,0)=(26×Pixel1(i,0)+6×Pixel2(i,0)+16)>>5

[0038] -NewPixel(i,1)=(7×Pixel1(i,1)+Pixel2(i,1)+4)>>3

[0039] -NewPixel(i,2)=(15×Pixel1(i,2)+Pixel2(i,2)+8)>>4

[0040] -NewPixel(i,3)=(31×Pixel1(i,3)+Pixel2(i,3)+16)>>5For the chroma block, the number of mixed pixel columns is 1.

[0041] -NewPixel(i,0)=(26×Pixel1(i,0)+6×Pixel2(i,0)+16)>>5

[0042] If Cost3 is the smallest, use blending mode 2.

[0043] For luma blocks, the number of mixed pixel columns is 2.

[0044] -NewPixel(i,0)=(15×Pixel1(i,0)+Pixel2(i,0)+8)>>4

[0045] -NewPixel(i,1)=(31×Pixel1(i,1)+Pixel2(i,1)+16)>>5

[0046] For chroma blocks, the number of mixed pixel columns is 1.

[0047] -NewPixel(i,0)=(15×Pixel1(i,0)+Pixel2(i,0)+8)>>4

[0048] In this application, an adaptive OBMC method is disclosed to improve the coding efficiency. Summary of the invention

[0049] A method and apparatus for video encoding using OBMC are disclosed. According to the method, input data associated with a current block / subblock is received, wherein the input data includes pixel data of the current block / subblock to be encoded on the encoder side or coded data associated with the current block / subblock to be decoded on the decoder side. A current MV (motion vector) of the current block / subblock is determined, and adjacent MVs of adjacent blocks / subblocks associated with the current block / subblock are determined. The number of rows to be applied to OBMC (overlapped block motion compensation), one or more weights of the number of rows associated with OMBC, or both are determined based on one or more boundary matching costs between the current block / subblock and the adjacent block / subblock, MV differences between the current MV and the adjacent MV, or predicted brightness data associated with the current block / subblock. OBMC is applied to at least one of the current block / subblock and the adjacent block / subblock based on OBMC parameters including the number of rows, the one or more weights, or both.

[0050] In one embodiment, the number of rows, the one or more weights, or both are determined by comparing the MV difference to a threshold. In one embodiment, if the MV difference is less than the threshold, a first weight is used, and if the MV difference is greater than the threshold, a second weight is used. In one embodiment, the first weight is greater than the second weight. In another embodiment, the second weight is greater than the first weight.

[0051] In one embodiment, the number of rows, the one or more weights, or both are determined based on the one or more boundary matching costs including a first boundary matching cost and a second boundary matching cost. The first boundary matching cost is calculated between the one or more first predicted or reconstructed samples of the current block / subblock and the one or more reconstructed samples of the adjacent block / subblock, and the second boundary matching cost is calculated between the one or more second predicted or reconstructed samples of the current block / subblock and the one or more reconstructed samples of the adjacent block / subblock. The one or more first predicted or reconstructed samples of the current block / subblock are determined using the current MV, and the one or more second predicted or reconstructed samples of the current block / subblock are determined using the adjacent MV.

[0052] In one embodiment, the one or more weights include first weights and second weights for the one or more first predicted or reconstructed samples of the current block / sub-block and the one or more second predicted or reconstructed samples of the current block / sub-block, respectively, to form one or more final predicted or reconstructed samples of the current block / sub-block, wherein the ratio of the first weight and the second weight corresponds to the ratio of the second boundary matching cost to the first boundary matching cost. In one embodiment, the number of rows, the one or more weights, or both are determined in response to comparing one of the first boundary matching cost and the second boundary matching cost with a threshold. In one embodiment, if one of the first boundary matching cost and the second boundary matching cost is less than the threshold, the number of rows, the one or more weights, or both are reduced or increased accordingly. In one embodiment, if the one of the first boundary matching cost and the second boundary matching cost is greater than the threshold, the number of rows, the one or more weights, or both are reduced or increased accordingly.

[0053] In one embodiment, the threshold depends on the block width or height of the current block / sub-block, the aspect ratio or block shape, the current MV, the adjacent MV, the weighted or unweighted MV difference, the boundary similarity, the predicted luminance signal, the MV gradient at one or more boundaries between the current block / sub-block and the adjacent block / sub-block, the cost indicator or a combination thereof.

[0054] In one embodiment, the number of rows, the one or more weights, or both are determined according to predicted luma data associated with the current block / sub-block.In one embodiment, the predicted luma data corresponds to an average value of the current block / sub-block. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1A An exemplary adaptive Inter / Intra video coding system incorporating loop processing is described.

[0056] Figure 1B Shows Figure 1A The corresponding decoder of the encoder in .

[0057] Figure 2 An example of overlapped motion compensation for geometric partitions is illustrated.

[0058] Figure 3A -B shows the Figure 3A ) and Nx2N blocks ( Figure 3B )’s example of OBMC.

[0059] Figure 4A An example of a sub-block to which OBMC is applied is illustrated, wherein the example includes a sub-block at a CU / PU boundary.

[0060] Figure 4B An example of a sub-block to which OBMC is applied is shown, the example including a sub-block coded and decoded in AMVP mode.

[0061] Figure 5 An example of OBMC processing using upper and left neighboring blocks for a current block is shown.

[0062] Fig. 6A An example of OBMC processing of the right and bottom portions of the current block using neighboring blocks from the right and bottom is shown.

[0063] Figure 6B An example of OBMC processing of the right and lower portions of the current block using neighboring blocks from the right, bottom, and right-bottom is shown.

[0064] Figure 7 An example of template matching based OBMC is shown, where for each top block having a 4×4 size at the top CU boundary, the above template size is equal to 4×1.

[0065] Figure 8 A flowchart of an exemplary overlapped block motion compensation (OBMC) process in a video coding system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0066] It will be readily appreciated that the components of the present invention as generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of the embodiments of the systems and methods of the present invention as shown in the figures is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. References throughout this specification to "one embodiment," "an embodiment," or similar language mean that a particular feature, structure, or characteristic described in conjunction with that embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing throughout this specification do not necessarily all refer to the same embodiment.

[0067] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. However, those skilled in the relevant art will recognize that the present invention may be practiced without one or more specific details, or practiced using other methods, components, etc. In other cases, well-known structures or detailed descriptions of operations are not shown or are not shown to avoid obscuring aspects of the present invention. The illustrated embodiments of the present invention will be best understood with reference to the accompanying drawings, in which like parts are represented by like numbers throughout. The following description is intended to be exemplary only, and simply illustrates certain selected embodiments of the apparatus and methods consistent with the present invention as claimed herein.

[0068] A new OBMC hybrid adaptive weighting method is disclosed. In this method, when OBMC is applied to a current block, the number of rows or the weights of the rows in OBMC can be adaptively changed, for example, according to its block width or height, aspect ratio, block shape, MV of the current block / subblock, MV of neighboring blocks / subblocks, weighted or unweighted MV difference between the current subblock and neighboring blocks, boundary similarity, predictor luminance signal, MV gradient at the boundary, cost metric or a combination thereof. The number of rows or the weights of the rows in OBMC can be changed according to some specific features in the CU, or it can always remain the same regardless of the CU features.

[0069] In one embodiment, when the current block area (ie, block width×block height) is smaller than or larger than a threshold, the number of rows in the OBMC or the weight of the row will be reduced or increased accordingly.

[0070] In another embodiment, when the aspect ratio of the current block is less than or greater than a threshold, the number of rows in the OBMC or the weight of the row will be reduced or increased accordingly. For example, if the MV difference is less than a threshold, a first weighting may be used, and if the MV difference is greater than a threshold, a second weighting may be used. In one example, the first weight is greater than the second weight and in another example, the second weight is greater than the first weight.

[0071] In another embodiment, the MV of the current sub-block is compared with the MV of the adjacent block to obtain an MV difference. If the MV difference is less than or greater than a threshold, the number of rows or the weight of the row in the OBMC will be reduced or increased accordingly. In another example, after obtaining the MV difference, the MV difference is weighted by multiplying the MV difference by another value to obtain a weighted MV difference, and then the weighted MV difference is compared with the threshold.

[0072] In another embodiment, in order to measure the boundary similarity, when determining to perform OBMC on the current sub-block, the boundary matching cost is first calculated by comparing the current sample with the neighboring samples. The current sample can be a predicted sample or a reconstructed sample. Similarly, the neighboring samples can be neighboring predicted samples or neighboring reconstructed samples. In order to determine the number of rows or the weight of the rows in OBMC, the boundary matching cost can be used as follows:

[0073] Cost 1 = BoundaryMatchingCost (Neighboring Reconstruction, Current Predictor or Current Reconstruction)

[0074] Cost 2 = BoundaryMatchingCost (NeighboringReconstruction, NeighboringPredictor)

[0075] For neighboring predictors: Weighting = Cost 1 / (Cost 1+Cost 2)

[0076] For the current predictor: Weighting = Cost 2 / (Cost 1+Cost 2)

[0077] In the above, the predicted or reconstructed samples of the current block or subblock are derived based on the MV of the current block or subblock. The neighboring predictor refers to the predictor generated based on the MV of the neighboring block or subblock. It can be seen that the ratio of the weight of the neighboring predictor to the weight of the current predictor corresponds to the ratio of the boundary matching cost 1 to the boundary matching cost 2.

[0078] In another example, instead of always using weighting according to cost 1 and cost 2, one cost can be selected from cost 1 and cost 2 and compared with a threshold. If the selected cost is less than or greater than the threshold, the number of rows in the OBMC or the weight of the row will be reduced or increased accordingly. In another embodiment, the boundary matching cost or the template matching cost is calculated. Different comparisons can be applied to the cost and the threshold. For example, the comparison may correspond to "less than or equal to", "less than", "greater than or equal to", or "greater than". Depending on the comparison result, the blending line number or blending weighting can be reduced or increased. The threshold can depend on block width or height, aspect ratio, block shape, MV of the current block / subblock, MV of adjacent blocks / subblocks, weighted or unweighted MV difference between the current subblock and adjacent blocks, boundary similarity, predicted sub-luminance signal, MV gradient at the boundary, cost metric, or a combination thereof.

[0079] In another embodiment, when it is determined to perform OBMC on the current sub-block, the luminance signal of the current predictor is first calculated. If the luminance value (or the average value of the current block / sub-block) is less than or greater than a threshold, the number of rows in OBMC or the weight of the row will be reduced or increased accordingly.

[0080] In another embodiment, when it is determined to perform OBMC on the current sub-block, the sample level offset is first calculated using the spatial gradient to obtain the MV gradient. If the MV gradient is less than or greater than a threshold, the number of rows in the OBMC or the weight of the row will be reduced or increased accordingly. In another example, when performing a comparison with a threshold, optical flow derivation will be applied based on the MV gradient.

[0081] Any of the aforementioned proposed OBMC methods can be implemented in an encoder and / or decoder. For example, at the encoder end, in a predictor derivation module (e.g., Figure 1A The required processing is implemented in the prediction submodule (part of the inter-frame prediction unit shown in FIG. 1 ). However, the encoder may also use additional processing units to implement the required processing. For the decoder, the required processing may be implemented in the prediction submodule, such as Figure 1B 152 is shown as part of the MC unit 152. However, the decoder may also use additional processing units to implement the required processing. Inter-frame prediction 112 and MC 152 are shown as separate processing units, which may correspond to executable software or firmware code stored on a medium (such as a hard disk or flash memory) for a CPU (central processing unit) or a programmable device (such as a DSP (digital signal) processor) or an FPGA (field programmable gate array)). Alternatively, any of the proposed methods can be implemented as a circuit coupled to the predictor derivation module of the encoder and / or the predictor derivation module of the decoder to provide the information required by the predictor derivation module.

[0082] Figure 8A flowchart of an exemplary overlapped block motion compensation (OBMC) process in a video coding and decoding system according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program codes that can be executed on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, input data associated with the current block is received in step 810, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. In step 820, the current MV (motion vector) of the current block / subblock is determined. In step 830, the neighboring MV of the neighboring block / subblock associated with the current block / subblock is determined. In step 840, the number of rows to be applied to OBMC (overlapped block motion compensation) is determined according to one or more boundary matching costs between the current block / subblock and the neighboring block / subblock, the MV difference between the current MV and the neighboring MV, or the predicted luminance data associated with the current block / subblock, one or more weights associated with the OMBC of the number of rows, or both. In step 850, OBMC is applied to at least one of the current block / sub-block and the neighboring blocks / sub-blocks according to OBMC parameters including the number of rows, the one or more weights, or both.

[0083] The flowchart shown is intended to illustrate an example of video coding according to the present invention. Without departing from the spirit of the present invention, those skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to implement the present invention. In the present disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. Without departing from the spirit of the present invention, those skilled in the art may implement the present invention by replacing syntax and semantics with equivalent syntax and semantics.

[0084] The above description is provided to enable those skilled in the art to practice the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are illustrated to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be implemented.

[0085] The embodiments of the present invention as described above can be implemented with various hardware, software codes or a combination of the two. For example, one embodiment of the present invention can be one or more circuits integrated into a video compression chip or a program code integrated into a video compression software to perform the processing described herein. An embodiment of the present invention can also be a program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software codes or firmware codes that define the specific methods embodied by the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. It is also possible to compile software code for different target platforms. However, the different code formats, styles and languages ​​of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0086] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered in all respects as illustrative and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes that fall within the meaning and range of equivalents of the claims should be included within their scope.

Claims

1. A video encoding and decoding method, the method comprising: Receiving input data associated with a current block / sub-block, wherein the input data includes pixel data of the current block / sub-block to be encoded at an encoder side or encoded data associated with the current block / sub-block to be decoded at a decoder side; Determine a current motion vector of the current block / sub-block; Determining neighboring motion vectors of neighboring blocks / sub-blocks associated with the current block / sub-block; determining a row number to be applied for overlapped block motion compensation, one or more weights associated with the OMBC for the row number, or both based on one or more boundary matching costs between the current block / subblock and the neighboring block / subblock, a motion vector difference between the current motion vector and the neighboring motion vector, or predicted luma data associated with the current block / subblock; and The overlapped block motion compensation is applied to at least one of the current block / sub-block and the neighboring block / sub-block according to overlapped block motion compensation parameters including the number of rows, the one or more weights, or both.

2. The method according to claim 1, characterized in that The number of rows, the one or more weights, or both are determined by comparing the motion vector difference to a threshold.

3. The method according to claim 2, characterized in that If the motion vector difference is smaller than the threshold, the first weight is used, and if the motion vector difference is larger than the threshold, the second weight is used.

4. The method according to claim 3, characterized in that The first weight is greater than the second weight.

5. The method according to claim 3, characterized in that: The second weight is greater than the first weight.

6. The method according to claim 1, characterized in that The number of rows, the one or more weights, or both are determined according to the one or more boundary matching costs including a first boundary matching cost and a second boundary matching cost, wherein the first boundary matching cost is calculated between the one or more first predicted or reconstructed samples of the current block / subblock and the one or more reconstructed samples of the neighboring block / subblock, and the second boundary matching cost is calculated between the one or more second predicted or reconstructed samples of the current block / subblock and the one or more reconstructed samples of the neighboring block / subblock, and wherein the one or more first predicted or reconstructed samples of the current block / subblock are determined using the current motion vector, and the one or more second predicted or reconstructed samples of the current block / subblock are determined using the neighboring motion vector.

7. The method according to claim 6, characterized in that The one or more weights include first weights and second weights respectively used for the one or more first predicted or reconstructed samples of the current block / sub-block and the one or more second predicted or reconstructed samples of the current block / sub-block to form one or more final predicted or reconstructed samples of the current block / sub-block, wherein a ratio of the first weight to the second weight corresponds to a ratio of the second boundary matching cost to the first boundary matching cost.

8. The method according to claim 6, characterized in that The number of rows, the one or more weights, or both are determined in response to comparing one of the first boundary matching cost and the second boundary matching cost to a threshold.

9. The method according to claim 8, characterized in that If one of the first boundary matching cost and the second boundary matching cost is less than the threshold, the number of rows, the one or more weights, or both are reduced or increased accordingly.

10. The method according to claim 8, characterized in that If one of the first boundary matching cost and the second boundary matching cost is greater than the threshold, the number of rows, the one or more weights, or both are reduced or increased accordingly.

11. The method according to claim 8, characterized in that The threshold depends on the block width or height, aspect ratio or block shape of the current block / sub-block, the current motion vector, the neighboring motion vector, weighted or unweighted motion vector difference, boundary similarity, predicted sub-luminance signal strength, motion vector gradients at one or more boundaries between the current block / sub-block and the neighboring block / sub-block, a cost metric, or a combination thereof.

12. The method according to claim 1, characterized in that The number of rows, the one or more weights, or both are determined based on the predicted luma data associated with the current block / sub-block.

13. The method according to claim 12, characterized in that The predicted brightness data corresponds to the average value of the current block / sub-block.

14. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors arranged to: Receive input data associated with the current block / subblock, where The input data includes pixel data of the current block / sub-block to be encoded at the encoder side or encoded data associated with the current block / sub-block to be decoded at the decoder side; Determine a current motion vector of the current block / sub-block; Determining neighboring motion vectors of neighboring blocks / sub-blocks associated with the current block / sub-block; determining a number of rows to be applied for overlapped block motion compensation, one or more weights associated with the overlapped block motion compensation for the number of rows, or both based on one or more boundary matching costs between the current block / subblock and the neighboring block / subblock, a motion vector difference between the current motion vector and the neighboring motion vector, or predicted luminance data associated with the current block / subblock; and The overlapped block motion compensation is applied to at least one of the current block / sub-block and the neighboring block / sub-block according to overlapped block motion compensation parameters including the number of rows, the one or more weights, or both.

Images