Method and device for video processing with overlapping block motion compensation in video coding and decoding system

Abstract

An exemplary video processing method and apparatus for encoding and decoding a current block splits the current block into a plurality of overlapping sub-blocks by overlapping sub-block motion compensation, determines a sub-block MV for each overlapping sub-block, derives an original predictor for each sub-block by motion compensation using the sub-block MV, derives a final predictor for each overlapping region by mixing a plurality of original predictors of the plurality of overlapping sub-blocks of the overlapping region, and encodes or decodes the current block based on the final predictor. An exemplary video processing method and apparatus for encoding and decoding a block with OBMC generates a converted MV by changing the MV into an integer MV or changing the MV components into integer components, derives an OBMC region by motion compensation using the converted MV, and encodes or decodes a block by mixing the OBMC predictor with the original predictor.

Description

[0001] Related references

[0002] The present invention claims priority to U.S. Provisional Patent Application No. 62 / 657,995, filed on April 16, 2018, entitled “Simplified overlapped block motion compensation for subblock mode,” U.S. Provisional Patent Application No. 62 / 750,279, filed on October 25, 2018, entitled “Methods of Overlapped Blocks Motion Compensation with modified MV,” and U.S. Provisional Patent Application No. 62 / 751,755, filed on October 29, 2018, entitled “Method of Overlapped Blocks Motion Compensation with modified MV and MV constraints,” which are hereby incorporated by reference. Technical Field

[0003] The present invention relates to a video processing method and apparatus in a video encoding and decoding system, and in particular, to overlapped sub-block motion compensation or simplified overlapped block motion compensation. Background Art

[0004] High-Efficiency Video Coding (HEVC) is the latest video codec standard developed by the Joint Team on Video Coding (JCT-VC) of video codec experts from the TTU-T research group. The HEVC standard improves the video compression performance of its predecessor standard H.264 / AVC to meet the needs of higher image resolution, higher frame rate and better video quality. During the development of the HEVC standard, many proposals related to Overlapped Block Motion Compensation (OBMC) were proposed to improve codec efficiency.

[0005] OBMC The basic principle of OBMC is to find the linear minimum mean square error (LMMSE) estimate of the pixel intensity value based on the motion compensation signal derived from the block motion vectors in its vicinity. From the point of view of estimation theory, these MVs are considered to be different likelihood hypotheses of its true motion, and in order to maximize the codec efficiency, the weights of the MVs are determined to minimize the mean square prediction error constrained by the unit gain. OBMC is proposed to improve the visual quality of the reconstructed video while providing codec gain for boundary pixels. In the example of applying OBMC to geometric partitions, because two different MVs are used for motion compensation, the pixels at the partition boundary usually have large discontinuities and cause visual artifacts such as block artifacts. These discontinuities reduce the conversion efficiency. For example, two regions created by several partitions are labeled as region 1 and region 2, if any of the four connected neighboring pixels (i.e., left, top, right, and bottom pixels) of a pixel from region 1 belongs to region 2, it is defined as a boundary pixel, and if any of the four adjacent neighboring pixels from a pixel from region 2 belongs to region 1, it is defined as a boundary pixel. Figure 1 An example of a boundary pixel between two regions of a block is marked. The grey shaded pixel 122 belongs to the boundary of the first region 12 in the upper left half of the block, and the white shaded pixel 142 belongs to the boundary of the second region 14 in the lower right half of the block. For each boundary pixel, motion compensation is performed using a weighted sum of motion predictors retrieved from the MVs of the first region 12 and the second region 14. For the predictor retrieved using the MV of the region containing the boundary pixel, the weight is 3 / 4, and for the predictor retrieved using the MV of the other region, the weight is 1 / 4.

[0006] OBMC is also used to smooth the boundary pixels of symmetric motion partitions, such as two 2N×N or N×2N prediction units (PUs) split from a 2N×2N coding unit (CU). OBMC is applied to the horizontal boundary of two 2N×N PUs and the vertical boundary of two N×2N PUs. Because the partitions are reconstructed using different MVs, the pixels at the partition boundaries may have large discontinuities, and OBMC is applied to mitigate visual artifacts and improve conversion / encoding efficiency. Figure 2A shows an example of applying OBMC to two 2N×N blocks and Figure 2B An example of applying OBMC to two N×2N blocks is shown. Figure 2A or Figure 2BThe gray pixels in are pixels belonging to partition 0 and the white pixels are pixels belonging to partition 1. The overlapping area in the luminance (luma) component is defined as two rows of pixels on each side of the horizontal boundary and two columns of pixels on each side of the vertical boundary. For a row or column of pixels outside the partition boundary, that is, in Figure 2A as well as Figure 2B For pixels marked as A in , the OBMC weight factors for the original predictor and the OBMC predictor are (3 / 4, 1 / 4), respectively. For pixels in two rows or columns outside the partition boundary, i.e. pixels marked as B, the OBMC weight factors for the original predictor and the OBMC predictor are (7 / 8, 1 / 8), respectively. For the chroma component, the overlapping area is defined as a row of pixels on each side of the horizontal boundary and a column of pixels on each side of the vertical boundary, and the weight factors for the original predictor and the OBMC predictor are (3 / 4, 1 / 4), respectively.

[0007] Skip and Merge Skip and merge modes are proposed and adopted in the HEVC standard to increase the coding and decoding efficiency of motion information by inheriting motion information from spatial neighboring blocks or temporal co-located blocks. In order to encode a PU in skip or merge mode, only the index representing the final candidate selected from the candidate set is signaled without signaling the motion information. The motion information reused by the PU encoded in skip or merge mode includes the motion vector (MV), the inter prediction indicator, and the reference image index of the selected final candidate. Note that if the selected final candidate is a temporal motion candidate, the reference image index is usually set to 0. When the PU is encoded in merge mode, the prediction residual is encoded, however, because the residual data of the PU encoded in skip mode is forced to 0, the skip mode further skips the signaling of the prediction residual.

[0008] The merge candidate set in the HEVC standard for the current PU 30 includes four spatial motion candidates related to the neighboring blocks of the current PU 30 and one temporal motion candidate related to the co-located PU 32 of the current PU 30. Figure 3As shown, the first merge candidate is the left predictor A1 312, the second merge candidate is the top predictor B1 314, the third merge candidate is the upper right predictor B0 313, and the fourth merge candidate is the lower left predictor A0 311. The upper left predictor B2 315 is included in the merge candidate set to replace the unavailable spatial predictor. The fifth merge candidate is the temporal predictor of the first available temporal predictor TBR321 and TCTR 322. Based on motion vector compensation as determined by rate-distortion optimization (RDO), the encoder selects a final candidate from the candidate set for each PU encoded in skip or merge mode, and the index representing the selected final candidate is signaled to the decoder. The decoder selects the same final candidate from the candidate set according to the index transmitted in the video bitstream. Because the derivation of skip and merge candidates is similar, for convenience, the "merge" mode mentioned below can correspond to merge mode as well as skip mode.

[0009] In many recently developed codecs, such as subblock Temporal Motion Vector Prediction (sbTMVP), Spatial-Temporal Motion Vector Prediction (STMVP), Pattern-based Motion Vector Derivation (PMVD), and Affine Motion Compensation Prediction (MCP), subblock motion compensation is used to increase the accuracy of the prediction process. The CU or PU in subblock motion compensation is divided into multiple subblocks, and these subblocks in the CU or PU may have different reference images and different MVs. Therefore, for subblock motion compensation, high bandwidth is required, especially when the MV of each subblock is very diverse. Some subblock motion compensation codecs are described in the following paragraphs.

[0010] Sub-PU TMVP. Subblock Temporal Motion Vector Prediction (Subblock TMVP, SbTMVP) is applied to merge mode by including at least one SbTMVP candidate as a merge candidate in the candidate set. SbTMVP is also called Alternative Temporal Motion Vector Prediction (ATMVP). The current PU is divided into multiple smaller sub-PUs, and the corresponding temporal co-located motion vectors of the multiple sub-PUs are searched. Figure 4An example of the SbTMVP technique is shown in FIG, where a current PU 41 of size M×N is divided into (M / P)×(N / Q) sub-PUs, each of which has a size of P×Q, where M is divisible by P and N is divisible by Q. The specific algorithm of the SbTMVP mode can be described in the following three steps.

[0011] In step 1, an initial motion vector is assigned to the current PU 41, denoted as vec_init. The initial motion vector is usually the first available candidate among multiple spatial neighboring blocks. For example, List X is the first list used to search for co-location information, and vec_init is set to the List X MV of the first available spatial neighboring block, where X is 0 or 1. The value of X (0 or 1) depends on which list is more suitable for inheriting motion information, for example, when the picture order count (POC) distance between the reference image and the current image is closer than the POC distance in List 1, List 0 is the first list used for searching. List X allocation can be performed at the slice level or the picture level. After obtaining the initial motion vector, a "co-location picture search process" is started to find the main co-location picture for all sub-PUs in the current PU, denoted as main_colpic. The reference picture selected by the first available spatial neighboring block is searched first, and then all reference pictures of the current picture are searched sequentially. For B slices, after searching the reference image selected by the first available spatial neighboring block, the search starts from the first list (List 0 or List 1) reference index 0, then index 1, then index 2, until the last reference image in the first list, when all the reference images in the first list are searched, the reference images in the second list are searched one by one. For P slices, the reference image selected by the first available spatial neighboring block is searched first, followed by all the reference images in the list starting from reference index 0, then index 1, then index 2 and so on. During the co-located image search process, for each searched image, an "availability check" checks whether the co-located sub-PUs around the center position of the current PU indicated by vec_init_scaled are coded by inter or intra mode. Vec_init_scaled is the MV appropriately scaled from vec_init. Some embodiments that determine "around the center position" are the center pixels (M / 2, N / 2) of a PU of size M×N, the center pixels in the center sub-PU, or a mixture of the center pixels or the center pixels of the current sub-PU depending on the shape of the current PU. The availability check result is true when the co-located sub-PU around the center position pointed out by vec_init_scaled is coded by inter mode. The currently searched picture is recorded as the main co-located picture main_colpic and when the availability check result for the currently searched picture is true, the co-located picture search process is completed. If the availability check result is true, the MV around the center pixel is used and scaled for the current block to derive the default MV. If the availability check result is false, then when the co-located sub-PU around the center position pointed out by vec_init_scaled is coded by intra mode, it continues to search for the next reference picture.When the reference image of vec_init is not equal to the original reference image, MV scaling is required in the co-located image search process. The MV is scaled based on the temporal distance between the current image and the reference image of vec_init and the searched reference image, respectively. After MV scaling, the scaled MV is defined as vec_init_scaled.

[0012] In step 2, the co-location position in main_colpic is located for each sub-PU. For example, the corresponding position 421 and position 422 for sub-PU 411 and sub-PU 412 are first located in the temporal co-location image 42 (main_colpic). The co-location position for the current sub-PU i is calculated as follows:

[0013] Parity position x = Sub-PU_i_x + vec_init_scaled_i_x (integer part) + shift_x,

[0014] Co-location position y = Sub-PU_i_y + vec_init_scaled_i_y (integer part) + shift_y,

[0015] Where Sub-PU_i_x represents the horizontal upper left position of sub-PU i in the current image, Sub-PU_i_y represents the vertical upper left position of sub-PU i in the current image content, vec_init_scaled_i_x represents the horizontal component of the scaled initial motion vector for sub-PU i (vec_init_scaled_i), vec_init_scaled_i_y represents the scaled vertical component of vec_init_scaled_i, and shift_x and shift_y represent the horizontal shift value and the vertical shift value, respectively. In order to reduce the computational complexity, only the integer positions of Sub-PU_i_x and Sub-PU_i_y, and the integer parts of vec_init_scaled_i_x and vec_init_scaled_i_y are used for the calculation. Figure 4 , the co-located position 425 located by vec_init_sub_0 423 from position 421 of sub-PU 411 and the co-located position 426 located by vec_init_sub_1 424 from position 422 of sub-PU 412.

[0016] In step 3 of SbTMVP, the motion information (MI) for each sub-PU is obtained from collocated_picture_i_L0 and collocated_picture_i_L1 at the co-located position x and co-located position y, marked as SubPU_MI_i. MI is defined as a set of {MV_x, MV_y, reference list, reference index and other merge mode sensitive information, such as local brightness compensation flag}. In addition, MV_x and MV_y can be scaled according to the temporal distance relationship between the co-located picture, the current picture and the reference picture of the co-located MV. If MI is not available for some sub-PUs, the MI of the sub-PUs close to the center position will be used, or in other words, the default MV will be used. Figure 4 As shown, subPU0_MV 427 obtained from co-located position 425 and subPU1_MV 428 obtained from co-located position 426 are used to derive predictors for sub-PU 411 and sub-PU 412, respectively. Each sub-PU in current PU 41 derives its own predictor based on the MI obtained at the corresponding co-located position.

[0017] STMVP In JEM-3.0, Spatial Temporal Motion Vector Prediction (STMVP) is used to derive new candidates to be included in the candidate set for Skip or Merge mode. Motion vectors for sub-blocks are derived recursively following raster scan order using temporal as well as spatial motion vector predictors. Figure 5 An example of a CU with four sub-blocks and its neighboring blocks being used to derive STMVP candidates is shown. Figure 5 The CU in is an 8×8 block containing four 4×4 sub-blocks A, B, C and D, and the neighboring N×N blocks in the current image are labeled a, b, c and d. The STMVP candidate derivation for sub-block A begins by identifying two spatial neighboring blocks of sub-block A. The first neighboring block c is the N×N block above sub-block A, and the second neighboring block b is the N×N block to the left of sub-block A. If block c is not available or block c is intra-coded, check the other N×N blocks above sub-block A from left to right, starting from block c. If block b is not available or block b is intra-coded, check the other N×N blocks to the left of sub-block A from top to bottom, starting from block b. The motion information obtained from the two neighboring blocks for each list is scaled to the first reference image of the given list. The temporal motion neighbor predictor (TMVP) of sub-block A is then derived by the same process as the subsequent TMVP derivation specified in the HEVC standard. The motion information of the co-located block at position D is extracted and scaled accordingly. Finally, all available motion neighbors are averaged for each reference list separately. The averaged motion vector is assigned as the motion vector of the current sub-block.

[0018] PMVD The pattern-based MV derivation (PMVD) method, also known as FRUC (Frame Rate Up Conversion) or DMVR (Decoder-side MV Refinement), includes bidirectional matching for bidirectional predicted blocks and template matching for unidirectional blocks. When the merge or skip flag is true, FRUC_mrg_flag is signaled, and if FRUC_mrg_flag is true, FRUC_merge_mode is signaled to indicate whether the bidirectional matching (both bidirectional matching merge mode and template matching merge mode) includes two-stage matching: the first stage is PU-level matching, and the second stage is sub-PU-level matching. In PU-level matching, multiple initial MVs in LIST_0 and LIST_1 are selected, respectively. These MVs include multiple MVs from merge candidates (i.e., traditional merge candidates as specified in the HEVC standard) and multiple MVs from temporally derived MVPs. Two different starting MV sets are generated for the two lists. For each MV in one list, an MV pair is generated by synthesizing this MV and a mirrored MV derived by scaling the MV to another list. For each MV pair, two reference blocks are compensated by using this MV pair. The sum of the absolute differences (SAD) of the two blocks is calculated. The MV pair with the smallest SAD is selected as the best MV pair. A diamond search is then performed to refine the MV pair. The refinement accuracy is 1 / 8 pixel. The refinement search range is limited to ±8 pixels. The final MV pair is the MV pair derived at the PU level.

[0019] The sub-PU level search in the second stage searches for the best MV pair for each sub-PU. The current PU is divided into multiple sub-PUs, where the depth of the sub-PU is signaled in the sequence parameter set with a minimum sub-PU size of 4×4. Several starting MVs in List 0 and List 1 are selected for each sub-PU, including the MV pair derived at the PU level, 0MV, the HEVC co-located TMVP of the current sub-PU and the lower right block, the time-derived MVP of the current sub-PU, and the MV of the left and upper PU or sub-PU. The best MV pair is selected for each sub-PU using a similar mechanism in the PU level search. A diamond search is then performed to refine the best MV pair. Motion compensation for each sub-PU is then performed to generate a predictor for each sub-PU.

[0020] Affine MCP Affine motion compensated prediction (affine MCP) is a technique developed for predicting various types of motion other than translational motion. For example, rotation, enlargement, reduction, perspective motion, and other irregular motions. Fig. 6AAs shown, an exemplary simplified affine transformation MCP is applied to JEM-3.0 to improve the coding efficiency. The affine motion field of the current block 61 is described by the motion vectors 613 and 614 of the two control points 611 and 612. The motion vector field (MVF) of the block is described by the following equation:

[0021]

[0022] Where (V 0x ,V 0y ) represents the motion vector 613 of the upper left corner control point 611, and (V 1x ,V 1y ) represents the motion vector 614 of the upper right corner control point 612.

[0023] Block-based affine transformation prediction is applied instead of pixel-based affine transformation prediction in order to further simplify the affine motion compensated prediction. Figure 6B FIG. 6 shows dividing the current block 62 into a plurality of sub-blocks and applying MCP to each sub-block. Figure 6B As shown, the motion vector of the center sample of each 4×4 sub-block is calculated according to the above equation and then rounded to 1 / 16 fractional accuracy, where (V 0x ,V 0y ) represents the motion vector 623 of the upper left corner control point 621, and (V 1x ,V 1y ) represents the motion vector 624 of the upper right control point 622. Motion compensated interpolation is applied based on the derived motion vector to generate a predictor for each sub-block. After performing motion compensated prediction, the high-precision motion vector of each sub-block is rounded and stored with the same precision as the normal motion vector.

[0024] When encoding in one of the multiple sub-block motion compensation codecs, a CU or PU is divided into multiple sub-blocks, and these sub-blocks may have different reference images and different MVs. Sub-block motion compensation requires high bandwidth and applying OBMC to blocks encoded in sub-block motion compensation requires high computational complexity. Fig. 7A An example of applying OBMC to a CU that is not encoded in sub-block motion compensation mode is shown, and Figure 7B An example of applying OBMC to a CU encoded in sub-block motion compensation mode is shown. When OBMC is applied to the current sub-block, in addition to the current motion vector, the motion vectors of four connected neighboring sub-blocks are also used to derive the final predictor of the current sub-block if they are available and not equal to the current motion vector. Multiple predictors derived based on multiple motion vectors are combined to generate the final predictor. Fig. 7AIn , the final predictor of the current CU is calculated by using the weighted sum of the current motion compensation predictor C derived from the current MV, the OBMC predictor A' derived from the MV of the upper neighboring block A, and the OBMC predictor B' derived from the MV of the left neighboring block B. Figure 7B In the figure, the final predictor of the current sub-block is calculated by using the weighted sum of the current motion compensation predictor C derived from the current MV, the OBMC predictor A' derived from the MV of the upper neighboring block, the OBMC predictor B' derived from the MV of the left neighboring block, the OBMC predictor D' derived from the MV of the right sub-block D, and the OBMC predictor E' derived from the MV of the lower sub-block E.

[0025] The OBMC predictor based on the MV of the neighboring block / subblock is marked as PN, and N indicates the index of the upper, lower, left and right neighboring blocks / subblocks. The original predictor based on the MV of the current block / subblock is marked as PC. If the PN is based on the motion information of the neighboring block / subblock containing the same motion information as the current block / subblock, this PN does not perform PBMC. Otherwise, each sample of the PN is added to the same sample in the PC. In JEM, four columns or four rows of PN are added to the PC, and the weight factors for PN are {1 / 4, 1 / 8, 1 / 16, 1 / 32} and the weight factors for PC are {3 / 4, 7 / 8, 15 / 16, 31 / 32} respectively. In the case of applying OBMC in a small MC block, when the height or width of the codec block is equal to 4 or when the CU is encoded in sub-CU mode, only two rows or two columns of PN are added to the PC. The weight factors for PN and PC are {1 / 4, 1 / 8} and {3 / 4, 7 / 8} respectively. For PN generated based on motion vectors of vertically (horizontally) adjacent sub-blocks, samples in the same column (row) of PN are added to PC with the same weight factor. The OBMC process of generating the final predictor by weighted sum is performed sequentially one by one, which introduces high computational complexity and data dependency.

[0026] When the CU size is less than or equal to 256 luma samples in JEM, OBMC can be turned on and off according to the CU level flag. For CUs with a size greater than 256 luma samples or not coded in AMVP mode, OBMC is applied by default. When OBMC is enabled, OBMC is performed on all motion compensation (MC) block boundaries except the right and bottom boundaries of the CU. OBMC is applied to both luma and chroma components. If the CU is encoded without sub-block MC, the MC block corresponds to the coding block, or if it is encoded with sub-block MC, the MC block corresponds to the sub-block in the CU.

[0027] At the encoder, when OBMC is applied to a CU, the impact is considered at the motion estimation stage. The OBMC predictor derived using the top and left neighboring blocks is used to compensate the top and left boundaries of the original predictor of the current CU, and then the normal motion estimation process is applied.

[0028] OBMC can be performed after normal motion compensation (MC). If OBMC is performed after normal MC, bidirectional optical flow (BDOF) is applied in both OBMC and normal MC, respectively. That is, MC results for overlapping areas between two CUs or PUs are generated by the OBMC process instead of in the normal MC process. BDOF is applied to refine the two MC results. When two adjacent MVs are the same, redundant OBMC and BDOF processes can be skipped. However, compared to integrating the OBMC process into the normal MC process, the bandwidth and MC operations required for overlapping areas are increased. Because fractional pixel motion vectors are supported in the new codec standard, additional reference pixels around the reference block are retrieved according to the number of interpolation taps used for interpolation calculation. In one example, the current PU size is 16×8, the OBMC area is 16×2, and an 8-tap interpolation filter is used in MC. If OBMC is performed after normal MC, each reference list requires (16+7)×(8+7)+(16+7)×(2+7)=522 reference pixels for generating the current PU and the associated OBMC area. If the OBMC operation is combined with normal MC into one stage, each reference list only needs (16+7)×(8+2+7)=391 reference pixels for the current PU and the associated OBMC.

[0029] There are two different implementation schemes for integrating OBMC in regular MC: pre-generation and real-time. When the current block is processed by OBMC, the first scheme is to pre-generate the OBMC region and store the OBMC predictor of the OBMC region in the local buffer for the neighboring blocks. Therefore, the corresponding OBMC predictor in the local buffer is available when processing the neighboring blocks. Fig. 8A Reference block extraction for generating a predictor for a current block without generating an OBMC region is shown. Figure 8B The reference block extraction for generating the current block predictor and the OBMC region is shown. Fig. 8A as well as Figure 8BIn this example, the size of the current block is W×H, an 8-tap interpolation filter is used for motion compensation, the width of the right OBMC region is W', and the height of the bottom OBMC region is H'. In one example, W' is four pixels and H' is also four pixels. In this case, four additional rows are extracted to generate the right OBMC region and four additional rows are extracted to generate the bottom OBMC region. Fig. 8A As shown in , the number of reference samples in the reference block that needs to be extracted from the memory is (3+W+4)×(3+H+4). Figure 8B As shown, the number of reference samples in the reference block extracted from the memory for generating the predictor of the current block and the two OBMC regions increases to (3+W+W'+4)×(3+H+H'+4). The right OBMC region and the bottom OBMC region are stored in the buffer for the OBMC process of the right and bottom neighboring blocks. An additional linear buffer across the coding tree unit (CTU) is required to store the MC results of the bottom OBMC region. The second embodiment generates an OBMC region for the current block just before mixing the OBMC predictor and the original predictor of the current block. For example, when OBMC is applied to the current sub-block, the OBMC predictor is not yet available in the local buffer, so the original predictor is derived based on the MV of the current sub-block, and one or more OBMC predictors are also derived based on the MV of one or more neighboring blocks, and then the original predictor is mixed with the one or more OBMC predictors. Summary of the invention

[0030] An exemplary method of video processing in a video coding system performs overlapped sub-block motion compensation. The exemplary video processing method receives input video data related to a current block in a current image, splits the current block into a plurality of overlapping sub-blocks according to overlapping sub-block partitions, and determines one or more sub-block MVs for each sub-block. According to the overlapping sub-block partitions, each sub-block in the current block overlaps with one or more other sub-blocks in a horizontal direction, a vertical direction, or both the horizontal and vertical directions. The selection of the overlapping sub-block partitions is predefined, explicitly signaled at a sequence level, a picture level, a tile group level, or a slice level of a video bitstream, or implicitly determined according to the current block motion information, a sub-block size, or a prediction mode. The exemplary video processing method derives an original predictor for each sub-block in the current block by motion compensation using the one or more sub-block MVs. In some embodiments, the current block contains only overlapping regions, and in some other embodiments, the current block contains both overlapping regions and non-overlapping regions. A final predictor is derived for each overlapping region by blending the multiple original predictors for the overlapping regions. For the non-overlapping regions, since there is only one original predictor associated with each non-overlapping region, the original predictor is used. The current block is encoded or decoded based on the final predictor of the overlapping region and the original predictor of the non-overlapping region (if available).

[0031] In some embodiments, the final predictor is derived by using weighted and blended the multiple original predictors of the overlapping area.The weighting factors for the multiple original predictors may be position based or may be based on the number of overlapping blocks.

[0032] Some exemplary video processing methods for processing a block with overlapped block motion compensation (OBMC) in a video coding system receive input video data related to a current block in a current image, determine one or more MVs, such as one MV for unidirectional prediction or two MVs for bidirectional prediction, generate one or more converted MVs by changing the one or more MVs into one or more integer MVs or changing an MV component of the one or more MVs into an integer component, and derive an OBMC region by motion compensation using the one or more converted MVs. The exemplary video processing method applies OBMC by blending an OBMC predictor in the OBMC region with an original predictor, and encodes or decodes the current block.

[0033] When processing the current block, a first OBMC implementation scheme pregenerates at least one OBMC region for at least one neighboring block, whereby the OBMC region is derived from a converted MV generated from one or more current MVs of the current block. In some embodiments, the converted MV is used to pregenerate a right or bottom OBMC region for a right or bottom neighboring block of the current block, wherein the OBMC predictor in the OBMC region is blended with the original predictor of the right or bottom neighboring block when processing the right or bottom neighboring block. According to one embodiment, the converted MV may be used to pregenerate both the right and bottom OBMC regions for the right and bottom neighboring blocks. In another embodiment, to derive the right OBMC region for the right neighboring block, the converted MV is generated by changing a horizontal component of the MV of the current block to an integer, and to derive the bottom OBMC region for the bottom neighboring block, the converted MV is generated by changing a vertical component of the MV of the current block to an integer.

[0034] In some other embodiments, the OBMC area is derived using the converted MV or the original MV based on the prediction direction of the current block, the neighboring block, or both. For example, if the prediction direction of the current block is bidirectional prediction, the OBMC area is derived by motion compensation using the converted MV, otherwise if the current block is unidirectional prediction, the OBMC area is derived using the MV without conversion. In another example, if the current block or the neighboring block is bidirectional prediction, the OBMC area is derived by motion compensation using the converted MV, otherwise if both the current block and the neighboring block are unidirectional prediction, the OBMC area is derived using the MV without conversion. According to one or more criteria in some embodiments, the pre-generation of the OBMC area can be adaptive. In one embodiment, the OBMC area is pre-generated only when both the horizontal and vertical components of the MV of the current block are not integers, or in another embodiment, the OBMC area is pre-generated only when one of the horizontal and vertical components of the MV of the current block is not an integer. In yet another embodiment, the OBMC region is pre-generated only when a predetermined component of the MV of the current block is not an integer, and the predetermined component depends on whether the OBMC region is used for the right or bottom neighboring block. In yet another embodiment, when the current block is bi-directionally predicted with an integer MV and a fractional MV, the OBMC region is pre-generated by one of List 0 and List 1 MVs, and the OBMC region is pre-generated by a converted MV generated from the fractional MV. Both the horizontal and vertical components of the fractional MV are fractional or only the predetermined component of the fractional MV is a fraction. When the OBMC predictor is mixed with the original predictor of the right or bottom neighboring block, the weight factor of the OBMC predictor in the pre-generated OBMC region is reduced. For example, the weight factor is reduced to half of the original weight factor, wherein the original weight factor is used to pre-generate the OBMC predictor in the OBMC region when the current block is not bi-directionally predicted with an integer MV and a fractional MV.

[0035] When processing the current block, a second OBMC implementation generates both the OBMC predictor in the OBMC region and the original predictor for the current block. The OBMC region is derived from a converted MV generated from one or more neighboring MVs of a neighboring block, the OBMC region is derived for the current block, and the OBMC region is mixed with the original predictor of the current block. Some embodiments of the second OBMC implementation use the converted MV to derive the OBMC region if the prediction direction of the neighboring block is bidirectional prediction, or use the MV without conversion if the prediction direction of the neighboring block is unidirectional prediction. Some other embodiments use the converted MV to derive the OBMC region if the current block or the neighboring block is bidirectional prediction, or use the MV without conversion if both the current block and the neighboring block are unidirectional prediction. In another embodiment, the OBMC region is derived using the converted MV only when the current block is bidirectionally predicted with an integer MV and the neighboring block is bidirectionally predicted.

[0036] Some examples of changing an MV to an integer MV are truncating or rounding to an integer MV. In some exemplary embodiments, the method checks similarity of multiple MVs of the current block and neighboring blocks, and adaptively skips blending the OBMC predictor with the original predictor in the OBMC region based on the similarity of the multiple MVs. The MV similarity check may be performed before or after generating the converted MV.

[0037] In one variation of the video processing method, when the current block is a luma block, one embodiment sets the maximum number of OBMC blending lines in the OBMC area to 3, and when the current block is a chroma block, sets the maximum number of OBMC blending lines to 1 or 2. If the fractional part of the absolute value of the MV is greater than 0.5 or greater than or equal to 0.5, another embodiment of the video processing method sets the number of OBMC blending lines in the OBMC area for the luma component to 3, otherwise the number of OBMC blending lines in the OBMC area for the luma component is set to 4. If the fractional part of the absolute value of the MV is greater than 0.5 or greater than or equal to 0.5, one embodiment of the video processing method sets the number of OBMC blending lines in the OBMC area for the chroma component to 1, otherwise the number of OBMC blending lines in the OBMC area for the chroma component is set to 2. In another embodiment of the video processing method, if the number of the OBMC blending lines in the OBMC area for the luma component is reduced to 3, the number of the OBMC blending lines in the OBMC area for the chroma component is set to 1, otherwise the number of the OBMC blending lines in the OBMC area for the chroma component is set to 2. In another embodiment, when the OBMC area is derived from the converted MV of a top neighboring block, the number of the OBMC blending lines in the OBMC area for the luma component is 3 only when the fractional part of the absolute value of the MV in the vertical direction is greater than 0.5 or greater than or equal to 0.5, otherwise the number of the OBMC blending lines in the OBMC area for the luma component is 4. When the OBMC area is derived from the converted MV of a left neighboring block, the number of the OBMC blending lines in the OBMC area for the luma component is 3 only when the fractional part of the absolute value of the MV in the horizontal direction is greater than 0.5 or greater than or equal to 0.5, otherwise the number of the OBMC blending lines in the OBMC area for the luma component is 4. Similarly, when the OBMC area is derived from the converted MV of a top-neighboring block, the number of OBMC blending lines in the OBMC area for the chrominance component is 1 only when the fractional part of the absolute value of the MV in the vertical direction is greater than 0.5 or greater than or equal to 0.5, or the number of OBMC blending lines for the luminance component is reduced to 3, otherwise, the number of OBMC blending lines for the chrominance component is 2.When the OBMC area is derived from the converted MV of the left adjacent block, the number of OBMC blending lines in the OBMC area for the chrominance component is 1 only when the fractional part of the absolute value of the MV in the horizontal direction is greater than 0.5 or greater than or equal to 0.5, or the number of OBMC blending lines for the luminance component is reduced to 3, otherwise the number of OBMC blending lines for the chrominance component is 2.

[0038] Some embodiments of applying OBMC adaptively determine the number of OBMC blending lines for blending the original predictor with the OBMC predictor of the current block. In one embodiment, the number of the OBMC blending lines for the left boundary of the current block is determined according to the width of the current block, and / or the number of the OBMC blending lines for the top boundary of the current block is determined according to the height of the current block. The original predictor of the current block is derived by motion compensation using one or more current MVs of the current block. The OBMC predictor for the left OBMC region of the left boundary with the number of OBMC blending lines is derived by motion compensation using one or more MVs of the left neighboring block of the current block. The OBMC predictor for the top OBMC region of the top boundary with the number of OBMC blending lines is derived by motion compensation using one or more MVs of the top neighboring block of the current block. By blending the OBMC predictor with the original predictor of the current block for the number of OBMC blending lines, the video encoding or decoding system applies OBMC to the current block and encodes or decodes the current block. For example, the width of the luma block is compared to a predetermined threshold to determine whether to use 2 OBMC blend lines or 4 OBMC blend lines at the left boundary of the current block. The height of the luma is compared to a predetermined threshold to determine whether to use 2 OBMC blend lines or 4 OBMC blend lines at the top boundary of the current block. Fewer OBMC blend lines for the left or top boundary are used for blocks with a width or length shorter than the predetermined threshold. Similarly, the width of the chroma block can be used to determine the number of OBMC blend lines, for example, if the width of the chroma block is less than a predetermined threshold, 1 OBMC blend line is used at the left boundary, otherwise 2 OBMC blend lines are used; and / or if the width of the chroma block is less than a predetermined threshold, 1 OBMC blend line is used at the top boundary, otherwise 2 OBMC blend lines are used. The number of OBMC blend lines is determined based on the adaptability of the length of the interpolation filter used in motion compensation, for example, more OBMC blend lines are required when a longer interpolation filter is used.

[0039] According to the flag, the method of adaptively determining the number of OBMC blending lines can be enabled or disabled, for example, if the flag indicates that the number of adaptive OBMC blending lines is disabled, 4 OBMC blending lines are used for luma components and 2 OBMC blending lines are used for chroma components. In one embodiment, the number of OBMC blending lines for the luma component is adaptively determined according to the width or length of the current block, and the number of OBMC blending lines for the chroma components is determined according to the number of OBMC blending lines for the luma component.

[0040] One aspect of the present invention further provides an embodiment of an apparatus for processing video data in a video coding and decoding system. One embodiment of the apparatus includes one or more electronic circuits for receiving input data of a current block in a current image, partitioning the current block into a plurality of overlapping sub-blocks according to overlapping sub-block partitions, determining one or more sub-block MVs for each sub-block; deriving an original predictor for each sub-block by motion compensation using the one or more sub-block MVs, deriving a final predictor for each overlapping region by mixing a plurality of original predictors of the plurality of overlapping sub-blocks in the overlapping region, and encoding or decoding the current block based on the final predictor. Another embodiment of the apparatus includes one or more electronic circuits for receiving input video data of the current block, determining one or more MVs, generating one or more converted MVs by changing the one or more MVs into one or more integer MVs or changing MV components of the one or more MVs into integer components, deriving an OBMC region by motion compensation using the one or more converted MVs, applying OBMC by mixing the OBMC predictor with the original predictor in the OBMC region, and encoding or decoding the current block.

[0041] One aspect of the present invention further provides a non-transitory computer readable medium storing program instructions so that the processing circuit of the device performs a video processing method to encode or decode the current block using overlapping sub-blocks according to some embodiments, or to encode or decode the current block using OBMC and OBMC area derived using integer MV according to some embodiments. Other aspects and features of the present invention will be apparent to those of ordinary skill in the art after reading the subsequent description of specific embodiments.

[0042] The present invention can reduce the bandwidth and interpolation requirements required for generating the prediction sub-units of the OBMC region by using the OBMC processing block. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Various embodiments of the present invention, presented by way of example, will be described in detail with reference to the following drawings, and in which:

[0044] Figure 1An example of motion compensation for geometric segmentation overlap is shown.

[0045] Figure 2A as well as Figure 2B Examples of OBMC footprints for 2NxN blocks and Nx2N blocks are shown where the boundary pixels have different weights.

[0046] Figure 3 The positions of spatial and temporal MV candidates used to reconstruct the merge candidate set are shown.

[0047] Figure 4 An example of determining a sub-block motion vector of a sub-block in a current PU according to the SbTMVP technique is shown.

[0048] Figure 5 An example of determining merge candidates for a CU having four sub-blocks according to the STMVP technique is shown.

[0049] Fig. 6A An example of applying affine motion vector compensated prediction to a current block having two control points is shown.

[0050] Figure 6B An example of applying block-based affine motion compensated prediction with two control points is shown.

[0051] Fig. 7A An example of applying OBMC to a block without adopting the sub-block motion compensation mode is shown.

[0052] Figure 7B An example of a block to which OBMC is applied in the case of adopting the sub-block motion compensation mode is shown.

[0053] Fig. 8A An example of fetching a reference block from memory for generating a predictor for the current block is marked.

[0054] Figure 8B An example of fetching a reference block from memory for generating a predictor for a current block and two OBMC predictors for neighboring blocks is indicated.

[0055] Fig.9A Exemplary non-overlapping sub-block partitions are shown.

[0056] Fig. 9B Exemplary overlapping sub-block partitions with overlapping areas in the horizontal direction are shown.

[0057] Fig. 9C Exemplary overlapping sub-block partitions with overlapping areas in the vertical direction are shown.

[0058] Fig.9DSub-block partitions are shown with exemplary overlaps in both the horizontal and vertical directions.

[0059] Fig.10 is a flow chart illustrating an exemplary embodiment of processing a current block with overlapped sub-block motion compensation.

[0060] Fig.11 An example is shown in which a reference block is fetched from memory for generating a predictor for a current block and two OBMC predictors for neighboring blocks when the current MV is rounded to an integer MV for generating an OBMC predictor.

[0061] Fig. 12A is a flow chart illustrating an exemplary embodiment of processing a current block with OBMC using the first OBMC implementation.

[0062] Fig. 12B is a flow chart illustrating an exemplary embodiment of processing a current block with OBMC using the second OBMC implementation.

[0063] Fig.13 An exemplary system block diagram of a video encoding system combined with a video processing method according to an embodiment of the present invention is shown.

[0064] Fig.14 An exemplary system block diagram of a video decoding system combined with a video processing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0065] It will be readily appreciated that the elements of the present invention, as generally described and illustrated in the diagrams herein, may be arranged and designed in a wide variety of different configurations. Therefore, the subsequent more detailed description of the embodiments of the systems and methods of the present invention as illustrated in the diagrams is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. In the present invention, systems and methods are described for motion compensation with overlapping sub-block partitions or overlapped block motion compensation (OBMC), and each or a combination of the methods may be implemented in a video encoder or a video decoder. Exemplary video encoders and decoders implementing one or a combination of the methods are described in the following, respectively. Fig.13 as well as Fig.14. Various embodiments of the present invention reduce computational complexity, especially for applying interpolation filtering in motion compensation, and reducing the additional bandwidth required for OBMC in one or both OBMC implementations. The systems and methods described herein are organized into the following sections. Section "Overlapping sub-block partitioning" shows an exemplary method for overlapping sub-block motion compensation, which achieves similar effects as the OBMC technique in reducing artifacts. Section "Directional OBMC" describes an exemplary method for applying OBMC only in one or more specific directions. Section "Short tap interpolation filter for OBMC area" describes an exemplary method for using a short tap interpolation filter when generating one or more OBMC areas. Section "Using integer MV for generating OBMC area", followed by sections "Conditional change to integer MV", "Conditional pre-generated OBMC area", "Conditional skipping OBMC blending", "Conditional generation of OBMC area" and "Reducing blending lines for OBMC" describe various exemplary methods for converting MV or MV components into integer MV or integer MV components when generating OBMC areas. The section "Adaptive Number of OBMC Blending Lines" describes an exemplary method for generating an OBMC region with adaptive determination of the number of OBMC blending lines. The section "Exemplary Flowchart" describes an exemplary method for generating an OBMC region using one or more converted MVs, where the converted MVs are integer MVs or MVs with integer components. The sections "OBMC Interaction with BiCW" and "OBMC Interaction with BDOF" describe some examples of implementing OBMC with bi-prediction with CU weights (BiCW) and BDOF techniques. The sections "Video Encoder and Decoder Implementations" and Fig.13 as well as Fig.14 A video encoding system and a video decoding system incorporating one or a combination of the described video processing methods are shown together.

[0066] Overlapped Sub-Block Partitioning In order to reduce the computational complexity of applying OBMC, affine prediction mode or other sub-block based prediction modes to non-overlapping sub-blocks, embodiments of the present invention apply overlapped sub-block motion compensation instead of OBMC. Fig.9A An example of partitioning a CU into 16 non-overlapping sub-blocks is shown. Fig. 9B An example of partitioning a CU into 6 overlapping sub-blocks is shown. Fig. 9C Another example of partitioning a CU into 6 overlapping sub-blocks is shown. Fig. 9B In the example of , sub-block 0 partially overlaps with sub-block 1, sub-block 1 partially overlaps with sub-block 2, sub-block 3 partially overlaps with sub-block 4, and sub-block 4 partially overlaps with sub-block 5. Fig. 9BThe overlapping sub-blocks in the video have only overlapping areas located at the right and / or left boundaries of each sub-block. The video encoder or decoder derives the original predictor of each sub-block of sub-blocks 0, 1, 2, 3, 4, and 5 according to one or more corresponding MVs of the sub-blocks, and then the video encoder or decoder derives the final predictor for the CU by mixing and combining the 6 original predictors. For example, the original predictor of the left part of sub-block 1 is mixed with the original predictor of the right part of sub-block 0, and the original predictor of the right part of sub-block 1 is mixed with the original predictor of the left part of sub-block 2. The original predictor of each non-overlapping area is the final predictor for the non-overlapping area, for example, the original predictor of the left part of sub-block 0 is also the final predictor for the left part of sub-block 0.

[0067] exist Fig. 9C , sub-block 0 partially overlaps with sub-block 2, sub-block 2 partially overlaps with sub-block 4, sub-block 1 partially overlaps with sub-block 3, and sub-block 3 partially overlaps with sub-block 5. Fig. 9C The overlapping subblocks in the above embodiment have only overlapping areas located at the top and / or bottom borders of each subblock. A raw predictor is derived for each subblock, and the raw predictor of the top portion of subblock 2 is mixed with the raw predictor of the bottom portion of subblock 0, and the raw predictor of the bottom portion of subblock 2 is mixed with the raw predictor of the top portion of subblock 4. Similarly, the raw predictor of the top portion of subblock 3 is mixed with the raw predictor of the bottom portion of subblock 1, and the raw predictor of the bottom portion of subblock 3 is mixed with the raw predictor of the top portion of subblock 5.

[0068] In some other embodiments, Fig.9D The overlapping areas at the top border, bottom border, left border and right border are shown in FIG. Fig.9D As shown, the CU is first partitioned into 16 sub-blocks and one or more corresponding MVs are derived for each sub-block. A first original predictor is located for each sub-block of the 16 sub-block partitions according to the one or more corresponding MVs. The same CU is also partitioned into 25 sub-blocks, and each of the 25 sub-blocks overlaps with one or more sub-blocks of the 16 sub-blocks in both horizontal and vertical directions. New MVs are derived for the 25 sub-blocks for locating the second original predictor. Each pixel of the final predictor for the overlapping area is calculated by mixing or combining the pixel values ​​of the two corresponding original predictors, where one predictor is the first original predictor derived from the 16 sub-block partitions and the other predictor is the second original predictor derived from the 25 sub-block partitions.

[0069] The overlapping sub-block partitions for the split block may be explicitly signaled, implicitly determined, or predefined. For example, the selection of the overlapping sub-block partitions is signaled at the sequence level, picture level, block group level, or slice level of the video bitstream. An example of implicit determination determines the overlapping sub-block partitions based on one or a combination of motion information of the current block, the size of the sub-block, or the prediction mode of the current block. The original predictors generated by the overlapping regions in the current block may be combined or mixed using a weighted sum to generate a final predictor for the current block. According to one embodiment, the weights used to generate the final predictor in the overlapping region may be position dependent, or according to other embodiments, each weight of the original predictor depends on the number of overlapping sub-blocks.

[0070] Exemplary Flowchart Fig.10 An exemplary flow chart of a video encoding or decoding system processing video data by overlapping sub-block motion compensation is marked. In step S1010, the video encoding or decoding system receives input video data related to a current block in a current image. On the encoder side, the input video data corresponds to pixel data to be encoded. On the decoder side, the input data corresponds to encoded data or prediction residuals to be decoded. In step S1020, the video encoding or decoding system divides the current block into a plurality of overlapping sub-blocks according to overlapping sub-block partitions. Each sub-block in the current block overlaps with one or more other sub-blocks at the left and / or right boundaries, at the top and / or bottom boundaries, or at one or a combination of the left, right, top and bottom boundaries. The overlapping sub-block partitions can be predefined, explicitly signaled at the sequence level, picture level, tile group level or slice level in the video bitstream, or implicitly determined based on one or a combination of motion information, sub-block size and prediction mode of the current block. In step S1030, if the subblocks are predicted in unidirectional prediction, a subblock MV is determined for each overlapping subblock, and if the subblocks are predicted in bidirectional prediction, List 0 and List 1 subblock MVs are determined for each overlapping subblock. In step S1040, each overlapping subblock in the current block is motion compensated by the subblock MV to derive an original predictor from the reference image. In step S1050, the video encoding or decoding system then derives a final predictor for each overlapping area by mixing the original predictors of the overlapping areas. According to various overlapping subblock partitions, there may be non-overlapping areas as well as overlapping areas in the current block, or there may be only overlapping areas in the current block. In step 1060, the video encoding or decoding system encodes or decodes the current block based on the final predictor of the overlapping areas. If there are also non-overlapping areas in the current block, the video encoding or decoding system encodes or decodes the current block based on the final predictor of the overlapping areas and the original predictor of the non-overlapping areas.

[0071] Directed OBMC For CUs encoded in sub-block mode, conventional OBMC is applied to each sub-block in four directions. Some embodiments of directional OBMC reduce the number of directions in the OBMC sub-block process. For example, only the OBMC predictor generated from the left block is used to blend with the original predictor of the current sub-block. In another embodiment, only the OBMC predictor generated from the upper block is used. The selected one or more directions for applying the OBMC process can be implicitly derived or explicitly signaled. An example of implicitly selecting the OBMC direction determines the direction based on the motion information of the current block and the neighboring blocks. The motion information includes one or a combination of motion vector, reference frame index, prediction direction, prediction mode, CU size, and sub-block size. In one embodiment, the direction of OBMC is applied based on the magnitude of the motion vector difference between the current block and the neighboring block. The direction with a larger MVD between the current block and the neighboring block is selected. In another embodiment, the direction with a smaller but non-0 MVD is selected. In one embodiment, the direction with a smaller average CU size of the neighboring blocks is selected. The selection can be explicitly signaled to the decoder at the sequence level, picture level, CTU level, CU level, or block level. The current block and the adjacent block of the present invention may be the current block or the current sub-block and the adjacent block or the adjacent sub-block respectively.

[0072] Short tap interpolation filter for OBMC area In order to reduce the computational complexity of the OBMC process, the interpolation filter length of the interpolation filter used in the motion compensation of the OBMC process can be reduced. The length of the interpolation filter used in the OBMC process is shorter than the length of the interpolation filter used in the normal MC process. Typically, in the normal MC process, an 8-tap interpolation filter is used for luminance samples and a 4-tap interpolation filter is used for chrominance samples. An embodiment of the present invention uses a 4-tap interpolation filter for performing OBMC on luminance samples and a 2-tap interpolation filter for performing OBMC on chrominance samples. In another example, a 1-tap interpolation filter is used for luminance samples and a 2-tap interpolation filter is used for chrominance samples. In yet another example, a 1-tap interpolation filter is used in the OBMC process for both luminance and chrominance samples. By reducing the number of interpolation filter taps, the computational complexity of OBMC can be reduced and the extra memory bandwidth used for the OBMC area in the worst case can be saved.

[0073] In the subsequent description, the current block may be the current CU, PU or sub-block, and the adjacent block may be an adjacent CU, PU or sub-block. Some embodiments of the present invention allow only integer MVs in the OBMC process to further simplify the calculation and reduce the memory bandwidth of motion compensation in the OBMC process. It is equivalent to using a 1-tap interpolation filter in the OBMC process. The exemplary embodiment determines one or more MVs for generating the OBMC region, and generates one or more converted MVs by changing one or more MVs into one or more integer MVs. One or more converted MVs of each adjacent block are aligned to integer samples in the real-time OBMC process, or one or more converted MVs of the current block are aligned to integer samples in the pre-generated OBMC process to avoid fractional motion compensation calculations and access to additional reference samples in the OBMC process. One or more converted MVs are used to derive the OBMC region by motion compensation. The OBMC predictor in the OBMC region is mixed with the original predictor in the OBMC process. For example, the OBMC predictor is mixed with the original predictor of the adjacent block in the pre-generated OBMC process, and the OBMC predictor is mixed with the original predictor of the current block in the real-time OBMC process.

[0074] Some examples of changing an MV to an integer MV for generating an OBMC region include truncating or rounding the MV to an integer MV. For example, changing the MV by discarding the fractional part of the MV, rounding the MV to the nearest integer, rounding to an integer MV when the rounding offset is equal to 0.5 (e.g., offset = (1 << (shift_bit-1), where shift_bit is the rounding bit), or rounding to an integer MV when the rounding offset is less than 0.5 (e.g., offset = (1 << (shift_bit-1) - 1)). If the distance between the original MV and the two integer MVs is the same, the integer MV closer to 0 can be selected, or in another example, the integer MV closer to infinity can be selected.

[0075] In some embodiments, only the MV of the luma component is converted to an integer MV, and in another embodiment, the MVs of the chroma components are also rounded or truncated to integer MVs.

[0076] Under the first implementation scheme of pre-generating OBMC areas for adjacent blocks when performing motion compensation on the current block, according to one embodiment, the MV in only the horizontal direction for deriving the right OBMC area is changed into an integer MV. Before generating the right OBMC area for the right adjacent block of the current block, the horizontal component of the MV is truncated or rounded to an integer value. That is, only the horizontal component in the MV of the current block is converted into an integer for deriving the right OBMC area. In another embodiment, the MV in only the vertical direction for deriving the bottom OBMC area is changed into an integer MV. Before generating the bottom OBMC area for the bottom adjacent block, the vertical component of the MV is truncated or rounded to an integer value. Only the vertical component in the MV of the current block is converted into an integer for deriving the bottom OBMC area. In yet another embodiment, the MV in the horizontal direction for deriving the right OBMC block is changed into an integer MV and the MV in the vertical direction for deriving the bottom OBMC area is changed into an integer MV.

[0077] Conditionally Changing to Integer MV In some embodiments, when the first implementation is applied, i.e., pre-generation of OBMC regions, OBMC regions are derived by changing the current MV or MV components of the current MV to integer MVs or integers if the motion information of the current block and / or neighboring blocks meets certain criteria. In the first implementation, each OBMC region generated by the motion information of the current block is used for blending with one or more predictors of neighboring blocks.

[0078] For example, if the prediction direction of the current block is bidirectional prediction, the current MV of the current block is changed to an integer MV before generating the OBMC area. If the prediction direction of the current block is unidirectional prediction, the current MV is used to generate the OBMC area of ​​the adjacent block without conversion. In another example, the motion information of the adjacent block is also considered, and if the prediction direction of the current block or the adjacent block is bidirectional prediction, the current MV of the current block is converted to an integer MV. That is, only when the current block and the adjacent block are unidirectional prediction, the current MV is not converted to generate the OBMC area for the adjacent block.

[0079] When the second embodiment is applied, if the motion information of the neighboring block and / or the current block meets a specific criterion, the MV or MV component of the neighboring block is converted into an integer MV or a classification of integer MVs to generate an OBMC region for the current block. In the second embodiment, one or more OBMC regions generated by the motion information of one or more neighboring blocks are used to process the current block by mixing one or more OBMC regions with the current predictor. In some embodiments, the MV of the neighboring block is conditionally corrected to integer precision depending on the motion information of the neighboring block and / or the current block. In one embodiment, if the prediction direction of the neighboring block is bidirectional prediction, the MV of the neighboring block is changed to an integer MV. For example, when both the current block and the neighboring block are predicted by unidirectional prediction, the MV from the neighboring block is not converted into an integer MV; when the current block is unidirectionally predicted and the neighboring block is bidirectionally predicted, the MV from the neighboring block is converted into an integer MV; and when the current block is bidirectionally predicted with an integer MV and the neighboring block is bidirectionally predicted, the MV from the neighboring block is converted into an integer MV. In another embodiment, if the prediction direction of the current block or the neighboring block is bidirectional prediction, the MV of the neighboring block is converted into an integer MV to generate an OBMC region for the current block. In yet another embodiment, only when the current block is bi-directionally predicted with integer MVs and the neighboring blocks are also bi-directionally predicted, the MVs of the neighboring blocks are converted into integer MVs to generate the OBMC region for the current block.

[0080] Conditionally pre-generate OBMC regions Some embodiments of the first implementation scheme always convert the MV into an integer MV before generating one or more OBMC regions, however, one or more OBMC regions for one or more adjacent blocks can be conditionally pre-generated. In one embodiment, the right OBMC and bottom OBMC regions are pre-generated only when the MV component of the current MV is not an integer in both the horizontal and vertical directions, and if the MV component of the current MV is not an integer in both directions, the current MV is changed to an integer MV for generating the OBMC region. In this embodiment, before pre-generating the right OBMC region for the right adjacent block, the MV component of the current block is first checked, and if one of the MV components in the horizontal and vertical directions is an integer, the right OBMC region will not be pre-generated. Similarly, the MV component is checked before pre-generating the bottom OBMC region, and if one of the MV components in the horizontal and vertical directions is an integer, the bottom OBMC region will not be pre-generated. In another embodiment, when the MV component of the current MV in the horizontal or vertical direction is not an integer, the right and bottom OBMC regions are pre-generated. When the MV component in the horizontal or vertical direction is not an integer, the OBMC region is pre-generated by changing the MV component or the current MV to an integer or an integer MV. In this embodiment, the current MV of the current block is first checked, and if the MV components in the horizontal and vertical directions are integers, the right OBMC region is not pre-generated. Similarly, if the MV components in the horizontal and vertical directions are integers, the bottom OBMC region is not generated.

[0081] Another embodiment of conditionally pregenerating OBMC areas checks the current MV for the right or bottom OBMC area in a predetermined direction, and when the current MV in the predetermined direction is not an integer, the current MV is changed to an integer MV. For example, the right OBMC area is generated only when the horizontal component of the MV is not an integer, and when the right OBMC area is generated, the horizontal MV component or MV components in all directions are changed to integers. In another example, the bottom OBMC area is generated only when the vertical component of the MV is not an integer, and when the bottom OBMC area is generated, the vertical MV component or all MV components are changed to integers. In yet another example, each right and bottom OBMC area is generated only when the MV components in the horizontal and vertical directions are not integers, respectively. When the right or bottom OBMC area is generated, the corresponding MV component or MV is changed to an integer or an integer MV for generating the right or bottom OBMC area.

[0082] In some embodiments, the current block is bidirectionally predicted and only one of the multiple MVs in List 0 and List 1 is an integer MV in both horizontal and vertical directions, while the other MVs are fractional MVs in at least one direction, or only one of the multiple MVs in List 0 and List 1 has an integer MV component in a predetermined direction, while the other MVs have a fractional MV component in a predetermined direction. By changing the fractional MV to an integer MV, the OBMC area will be pre-generated using the fractional MV, or by changing the fractional MV component to an integer or changing all MV components to an integer, the OBMC area will be pre-generated using the fractional MV component in a predetermined direction. For example, the prediction direction for the current block is bidirectional prediction, and the MVs in List 0 are integer MVs in both horizontal and vertical directions, while the MVs in List 1 are not integer MVs in both directions. The MVs in List 1 are selected and changed to integer MVs for generating the OBMC area. When one of these OBMC areas is used by a neighboring block, it is mixed with the original predictor derived from the motion information of the neighboring block, and the weight factor of the OBMC predictor used for the OBMC area can be reduced, for example. The weight factor can be reduced to half of the original weight factor. In another example, only one of the multiple current MVs in List 0 and List 1 has an integer MV component in the horizontal direction, and the right OBMC region is generated using an MV with a fractional MV component in the horizontal direction. Before generating the right OBMC region, the MV is changed to have an integer MV component in the horizontal direction or is changed to have an integer MV component in all directions. When the OBMC predictor of this right OBMC region is mixed with another predictor, the weight factor used for the OBMC predictor may be lower than the original weight factor, for example, the weight factor is reduced to half of the original weight factor. For generating the bottom OBMC region according to the bidirectional prediction motion information of the current block, if only one of the multiple MVs in List 0 and List 1 has an integer MV component in the vertical direction, the bottom OBMC region is generated using an MV with a fractional MV component in the vertical direction. Before generating the bottom OBMC region, the MV is changed to an integer MV component in the vertical direction or is changed to an integer MV component in both directions. The weight factor of the OBMC predictor used for this type of bottom OBMC region may be reduced, for example, reduced to half of the original weight factor used for the normal bottom OBMC region.

[0083] Conditionally skip OBMC blending In some embodiments, OBMC blending depends on the similarity of the MV of the current block and the adjacent blocks. For example, the MV is checked by calculating the MV difference between the MV of the current block and the adjacent blocks and comparing it with a predetermined threshold. If the MV difference is greater than the predetermined threshold, the OBMC blending between the current block and the adjacent blocks is skipped. In one embodiment, the MV similarity check is performed before the MV is changed to an integer MV for generating the OBMC region. In another embodiment, the MV similarity check is performed after the MV is changed to an integer MV for generating the OBMC region. In a first embodiment, the MV used to generate the OBMC region is the MV of the current block, and in a second embodiment, the MV used to generate the OBMC region is the MV of the adjacent blocks. In another embodiment, skipping OBMC blending is disabled based on MV similarity, and the MV is changed to an integer MV before generating the OBMC region.

[0084] Conditionally Generating an OBMC Region In some examples of the first embodiment, an OBMC region is generated only when the size, width, or height of the current block is greater than or equal to a predetermined threshold. Some examples of predetermined thresholds for size are 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, and 16384. For example, an OBMC region is generated only when the current block has a size greater than or equal to 64. In one embodiment, an OBMC region is generated only when the width of the current block is greater than or equal to a first predetermined threshold, and the height of the current block is greater than or equal to a second predetermined threshold. Some examples of combinations of the first and second predetermined thresholds are (4,4), (4,8), (4,16), (4,32), (4,64), (4,128), (8,4), (8,8), (8,16), (8,32), (8,64), (8,128), (16,4), (16,8), (16,16), (16,32), (16,64), (16,128), (32,4), (32,8), (32,16), (32,32), (32,64), (32,128), (64,4), (64,8), (64,16), (64,32), (64,64), (64,128), (128,4), (128,8), (128,16), (128,32), (128,64), and (128,128). For example, an OBMC area is generated only when the width of the current block is greater than or equal to 8 and the height is greater than or equal to 16.

[0085] In some examples of the second embodiment, some constraints for deciding whether to generate an OBMC region are applied to one or more neighboring blocks. For example, an OBMC region is generated only when the width of the neighboring block is greater than or equal to 8 and the height of the neighboring block is greater than or equal to 16.

[0086] The conditional OBMC region generation method may be applied to the luma component only, or may be applied to both luma and chroma components.

[0087] Reduce the number of blending lines for OBMC The number of blending lines for OBMC is the number of pixels in the horizontal direction in the right OBMC area or the number of pixels in the vertical direction in the bottom OBMC area. The number of blending lines for OBMC is also defined as the number of pixel columns on the horizontal boundary or the number of pixel rows on the vertical boundary processed by OBMC blending. In some embodiments of converting a fractional MV to an integer MV, the fractional MV is rounded to an integer MV when the fractional part is greater than 0.5 or when the fractional part is greater than or equal to 0.5. Fig.11 An example of extracting reference samples from a reference image for a current block and two OBMC regions is shown. In this example, the size of the current block is W×H, and the number of blending lines for OBMC is 4. Because the fractional current MV is rounded to an integer MV, the OBMC predictors for the right and bottom OBMC regions (OBMC R 1120 and OBMC B 1130) are 1 pixel apart from the current predictor 1110 for the current block in the reference image. In this case, the reference samples that need to be extracted for the current block and the two OBMC regions are the larger reference block (W+8)×(H+8) instead of (W+7)×(H+7). In order to keep the maximum reference block size of extracted reference pixels for the current block and OBMC regions within a certain range, the OBMC process may only apply 3 pixel lines instead of 4 at the block boundary. One embodiment reduces the maximum number of OBMC blending lines by rounding the MV to an integer MV when the OBMC regions are pre-generated from an integer MV so that the bandwidth will not increase in the worst case. For example, the maximum number of OBMC blending lines for the luma component is reduced from 4 to 3. According to one embodiment, the maximum number of OBMC blending lines for chroma components is reduced from 2 to 1, or according to another embodiment, remains at 2. In one embodiment, the maximum number of blending lines for luma and chroma components is reduced from 4 to 3 and from 2 to 1, respectively.

[0088] Some other embodiments determine the number of OBMC blending lines for each block based on one or more predetermined criteria. The exemplary system determines the number of OBMC blending lines based on the MV used to generate one or more OBMC regions. For example, the exemplary system checks whether the fractional part of the absolute value of the MV in list 0 or list 1 is greater than 0.5, and only when the fractional part is greater than 0.5, the number of OBMC blending lines for the luma component is reduced from 4 to 3. If the MV is an integer MV or if the fractional part is less than or equal to 0.5, the number of blending lines remains at 4. In an implementation that considers both luma and chroma components, the number of blending lines for the luma component is reduced from 4 to 3 only when the fractional part of the absolute value of the luma MV is greater than 0.5, and the number of blending lines for the chroma components is reduced from 2 to 1 only when the fractional part of the absolute value of the chroma MV is greater than 0.5. In one embodiment considering both luma as well as chroma components, the number of blending lines for luma components is reduced from 4 to 3 only when the fractional part of the absolute value of the luma MV is greater than 0.5, and the number of blending lines for chroma components is reduced from 2 to 1 if the number of blending lines for luma components is reduced from 4 to 3. In another embodiment, the number of OBMC blending lines for luma components is reduced from 4 to 3 only when the fractional part of the absolute value of the luma MV in list0 or list1 is greater than or equal to 0.5. In one embodiment, the number of OBMC blending lines for luma components is reduced from 4 to 3 only when the fractional part of the absolute value of the luma MV is greater than or equal to 0.5, and the number of OBMC blending lines for chroma components is reduced from 2 to 1 only when the fractional part of the absolute value of the chroma MV is greater than or equal to 0.5. In one embodiment, the number of OBMC blending lines for the luma component is reduced from 4 to 3 only when the fractional part of the absolute value of the luma MV is greater than or equal to 0.5, and the number of OBMC blending lines for the chroma components is reduced from 2 to 1 if the number of OBMC blending lines for the luma component is reduced from 4 to 3.

[0089] Under the second embodiment, the OBMC region is generated only before the OBMC blending process of the current block. The embodiment of reducing the number of OBMC blending lines first checks the luminance or chrominance MV of the adjacent blocks respectively to derive the OBMC region for the luminance component or the chrominance component respectively. For example, the exemplary system reduces the number of blending lines at the top block boundary for the luminance component from 4 to 3 only when the fractional part of the absolute value of the luminance MV in the vertical direction is greater than 0.5, otherwise the number of blending lines at the top block boundary for the luminance component is 4. Similarly, the number of OBMC blending lines at the left block boundary for the luminance component is 3 only when the fractional part of the absolute value of the luminance MV in the horizontal direction is greater than 0.5. In another example, the number of OBMC blending lines at the top or left block boundary for the luminance component is reduced from 4 to 3 only when the fractional part of the absolute value of the luminance MV in the vertical or horizontal direction is greater than 0.5, and the number of OBMC blending lines at the top or left block boundary for the chrominance component is reduced from 2 to 1 only when the fractional part of the absolute value of the chrominance MV in the vertical or horizontal direction is greater than 0.5. In another example, only when the fractional part of the absolute value of the luma MV in the vertical or horizontal direction is greater than 0.5, the number of OBMC blending lines at the top or left block boundary for the luma component is reduced from 4 to 3, and if the number of OBMC blending lines at the top or left block boundary for the luma component is reduced from 4 to 3, the number of OBMC blending lines at the top or left block boundary for the chroma component is reduced from 2 to 1. Similarly, the above embodiment can be modified to determine whether to reduce the OBMC blending lines when the fractional part of the absolute value of the luma or chroma MV is greater than or equal to 0.5.

[0090] Adaptive Number of OBMC Blending Lines Some embodiments of the present invention adaptively determine the number of OBMC blending lines for a current block based on the width or height of the current block depending on the direction of the OBMC blending. For example, the number of OBMC blending lines for the left border of the current block depends on the width of the current block, and the number of OBMC blending lines for the top border of the current block depends on the height of the current block. In an exemplary embodiment, if the width or height of the current block is less than a predetermined threshold, the number of OBMC blending lines for the luma component is reduced from 4 to 2. For example, if the width of the current block is less than a first predetermined threshold, the number of OBMC blending lines for the luma component at the left block border is reduced from 4 to 2, and if the width of the current block is greater than or equal to the first predetermined threshold, the number of OBMC blending lines is 4. If the height of the current block is less than a second predetermined threshold, the number of OBMC blending lines for the luma component at the top block border is reduced from 4 to 2, and if the height of the current block is greater than or equal to the second predetermined threshold, the number of OBMC blending lines is 4. The first or and the second predetermined thresholds may be the same or different. In subsequent examples, the first and the second predetermined thresholds are both 8. In one example, if the width of the current block is less than 8 and the height of the current block is greater than or equal to 8, the number of blending lines for the luma component at the left boundary is reduced from 4 to 2, however, the number of blending lines for the luma component at the top boundary remains at 4. In another example, if the width of the current block is less than 8 and the height of the current block is also less than 8, the number of blending lines for the luma component at the left and top boundaries is reduced from 4 to 2. In another example, if the height of the current block is less than 8 and the width of the current block is greater than or equal to 8, the number of OBMC blending lines for the luma component at the top boundary is reduced from 4 to 2, and the number of blending lines for the left boundary remains at 4.

[0091] Some other embodiments of adaptively determining the number of OBMC blend lines determine the number of OBMC blend lines based on the length of the interpolation filter used in motion compensation. The length of the interpolation filter is also referred to as the number of taps in the interpolation filter. For example, when a longer interpolation filter is used, more OBMC blend lines are blended. In a specific embodiment, when the length of the interpolation filter is L, (L / 2)-1 OBMC blend lines are used.

[0092] In one embodiment, adaptive determination of the number of OBMC blending lines may be enabled or disabled according to a flag, and when the flag indicates that adaptive number of OBMC blending lines is disabled, the number of blending lines for luma components is always equal to 4 and for chroma components is 2.

[0093] In one embodiment, the number of OBMC blending lines for chroma components is reduced according to the luma component. For example, when the number of blending lines in the luma component is reduced from 4 to 2, the number of blending lines in the chroma component is reduced from 2 to 1; otherwise, the number of blending lines for the luma component is 4 and for the chroma components is 2.

[0094] The above-mentioned OBMC mixed line reduction or determination method can be combined with one of the methods for generating OBMC areas by integer MVs. For example, when the OBMC area is generated by integer MVs converted by rounding to the nearest integer, the number of OBMC mixed lines is reduced from 4 to 3. In some embodiments, before generating the OBMC area, all MVs are converted into integer MVs, and in some other embodiments, when the current block or adjacent block meets a predetermined standard, the MV used to generate the OBMC area is converted into an integer MV. The predetermined standard can be related to the size of the current or adjacent block, the width of the current or adjacent block, the height of the current or adjacent block, the prediction direction of the current or adjacent block, the fractional part of the MV of the current or adjacent block, and one or a combination of the MVs of the current / adjacent block. The embodiment of conditionally changing the MV into an integer MV for the chrominance component depends on whether the corresponding MV for the luminance component is changed into an integer MV. For example, if the MV for the luminance component is converted into an integer MV, the MV for the chrominance component is converted into an integer MV. In another embodiment, when the luminance MV is changed into an integer MV, the MV for the chrominance component is derived from the integer MV of the luminance component. In yet another embodiment, when the luma MV is changed to an integer MV, the MVs for the chroma components are also changed to integer MVs, and the number of OBMC blending lines for the luma component is reduced from 4 to 3 and the number of OBMC blending lines for the chroma components is reduced from 2 to 1.

[0095] Exemplary Flowchart Fig. 12A as well as Fig. 12B Two exemplary flow charts of video encoding or decoding systems for encoding or decoding blocks with overlapped block motion compensation are shown. Fig. 12A shows an example of processing the current block according to the first OBMC implementation, and Fig. 12B An example of processing the current block according to the second OBMC implementation is shown. Fig. 12AIn step S1210, the video encoding or decoding system receives input data related to the current block in the current image. On the encoder side, the input data corresponds to pixel data to be encoded. On the decoder side, the input data corresponds to the encoded data or prediction residual to be decoded. In step S1212, it is determined to generate at least one OBMC area for the current MV of one or more neighboring blocks, wherein when the current block is bidirectionally predicted, the current MV may have List 0MV and List 1MV. In step S1214, the current MV is changed to a converted MV by rounding or truncating to an integer MV or changing the MV components to integer components. In step S1216, the video encoding or decoding system derives the original predictor for the current block by using motion compensation of the current MV and derives one or more OBMC areas for one or more neighboring blocks by using motion compensation of the converted MV. The one or more OBMC areas are pre-generated for one or more neighboring blocks. For example, the converted MV is used to locate the OBMC predictor for the right OBMC area and the bottom OBMC area, and the right and bottom OBMC areas are subsequently used to process the right and bottom neighboring blocks. The video encoding or decoding system retrieves one or more OBMC predictors in one or more OBMC regions associated with the current block from a memory, and stores the one or more OBMC regions for one or more neighboring blocks in the memory. In step S1218, the current block is encoded or decoded by mixing the original predictor for the current block with the one or more OBMC predictors associated with the current block.

[0096] exist Fig. 12B In step S1220, input video data of a current block in a current image is received. In step S1222, MVs of neighboring blocks for generating an OBMC region of the current block are determined. Fig. 12B In the example shown, OBMC is applied only to one side of the current block, so only one OBMC region generated by one neighboring block is required, however, when OBMC is applied to one or more sides of the current block, one or more OBMC regions may be generated. In step S1224, the video encoding or decoding system generates a converted MV by changing the MV of the neighboring block to an integer MV or by changing the MV component of the MV to an integer component. In step S1226, the original predictor for the current block is derived by motion compensation using the current MV of the current block, and the OBMC predictor in the OBMC region of the current block is derived by motion compensation using the converted MV. In step S1228, the video encoding or decoding system encodes or decodes the current block by mixing the original predictor for the current block with the OBMC predictor for the current block.

[0097] Interaction of OBMC with BiCW Bi-prediction with CU weights (BiCW), also known as generalized bi-prediction (GBI), is a technique for encoding a current block using a first reference block selected from a first reference image and a second reference block selected from a second reference image. Each reference block is associated with a weight, and the current block is predicted by the sum of the weights of the two reference blocks. In an embodiment where OBMC is applied to a current block with neighboring blocks encoded in BiCW, OBMC regions are generated with equal weights regardless of the actual BiCW weights of the neighboring blocks. In another embodiment where OBCM is applied to a current block with neighboring blocks encoded in BiCW, the BiCW weights of the neighboring blocks are stored and the OBMC regions are generated based on the actual BiCW weights of the neighboring blocks.

[0098] Interaction of OBMC and BDOF Typically, a video codec system performs bidirectional optical flow (BDOF) during motion compensation. Generally, the motion vector of the current block identifies the position of the reference block in the reference image relative to the current block. When BDOF is applied to the current block, the video codec system modifies the motion vector for the current block on a per-pixel basis. That is, according to BDOF, the video codec system determines a per-pixel modification of the motion vector for the current block, instead of retrieving each pixel of the reference block as a block unit, and reconstructing the reference block so that the reference block includes the reference pixel identified by the motion vector of the corresponding pixel of the current block and each pixel modification. In one embodiment of applying BDOF to generate an OBMC area, the video codec system retrieves the reference pixel identified by the original MV and each pixel modification of the corresponding pixel for the OBMC area. In another embodiment, the BDOF technique is prohibited from being used to generate an OBMC area.

[0099] Video encoder and decoder implementation The video processing method described above can be implemented in a video encoder or decoder. For example, the proposed video processing method is implemented in a predictor derivation module of an encoder and / or a predictor derivation module of a decoder. In another example, the proposed video processing method is implemented in a motion compensation module of an encoder and / or a motion compensation module of a decoder. Alternatively, any of the proposed methods is implemented as a circuit coupled to a predictor derivation or motion compensation module of an encoder and / or a predictor derivation module or motion compensation module of a decoder to provide information required by the predictor derivation module or motion compensation module.

[0100] Fig.13An exemplary system block diagram of a video encoder 1300 for implementing various embodiments of the present invention is shown. Intra-frame prediction 1310 provides intra-frame predictors based on reconstructed video data of a current image. Inter-frame prediction 1312 performs motion estimation (ME) and motion compensation (MC) based on video data from one or more other images to provide inter-frame predictors. According to some embodiments of the present invention, in order to encode a current block by an overlapped sub-block motion compensation codec, each overlapped area in the current block is predicted by mixing two or more original predictors derived from sub-block MVs corresponding to the overlapped area. In some other embodiments, inter-frame prediction 1312 derives OBMC predictors in OBMC areas by motion compensation using converted MVs, wherein the converted MVs are generated by changing MVs into integer MVs or changing MV components into integer components. The final predictor for each block is generated by mixing one or more OBMC predictors with the original predictors in inter-frame prediction. Intra-frame prediction 1310 or inter-frame prediction 1312 provides the selected predictor to adder 1316 to form a prediction error, also known as a prediction residual. The prediction residual of the current block is further processed by transform (T) 1318 followed by quantization (Q) 1320. The transformed and quantized residual signal is then encoded by entropy encoder 1332 to form a video bitstream. The video bitstream is then packed together with the side information. The transformed and quantized residual signal of the current block is processed by inverse quantization (IQ) 1322 and inverse transform (IT) 1324 to recover the prediction residual. Fig.13 As shown, reconstructed video data is generated by adding back the selected predictor to restore the prediction residual at reconstruction (REC) 1326. The reconstructed video data can be stored in a reference picture buffer (Ref.Pict.Buffer) 1330 and used for prediction of other pictures. Due to the encoding process, the reconstructed video data restored from REC 1326 may suffer from various impairments, so a loop processing filter (Filter) 1328 is applied to the reconstructed video data before being stored in the reference picture buffer 1330 to further enhance the image quality.

[0101] Fig.14 The following figure shows the method for decoding the Fig.13The video decoder 1400 corresponding to the video bitstream of the video encoder 1300 in FIG. The video bitstream is input to the video decoder 1400 and decoded by the entropy decoder 1410 to parse and recover the transformed and quantized residual signal and other system information. The decoding process of the decoder 1400 is similar to the reconstruction loop in the encoder 1300, except that the decoder 1400 only needs the motion compensation predictor in the inter prediction 1414. Each block is decoded by the intra prediction 1412 or the inter prediction 1414. The switch 1416 selects the intra predictor from the intra prediction 1412 or the inter predictor from the inter prediction 1414 according to the decoded mode information. The inter prediction 1414 performs overlapped sub-block motion compensation on the current block by mixing the original predictor derived from the overlapped sub-block MV according to some exemplary embodiments. According to some other exemplary embodiments, the inter prediction 1414 uses the OBMC area generated by one or more derived MVs for mixing with the original predictor. One or more derived MVs are generated by changing one or more MVs into one or more integer MVs or changing MV components of one or more MVs into integer components. The transformed and quantized residual signals associated with each block are recovered by inverse quantization (IQ) 1420 and inverse transform (IT) 1422. The reconstructed video is generated by adding back the predictor in REC 1418 to reconstruct the recovered residual signal. The reconstructed video is further processed by a loop processing filter (filter) 1424 to generate the final decoded video. If the current decoded picture is a reference picture for subsequent pictures in decoding order, the reconstructed video of the current decoded picture is also stored in the reference picture buffer 1426.

[0102] Fig.13 as well as Fig.14 The various elements of the video encoder 1300 and the video decoder 1400 can be implemented by hardware elements, one or more processors for executing program instructions stored in a memory, or a combination of hardware and processing. For example, the processor executes program instructions to control the reception of input data related to the current image. The processor is equipped with a single or multiple processing cores. In some examples, the processor executes program instructions to perform the functions of some elements in the encoder 1300 and the decoder 1400, and the memory electrically coupled to the processor is used to store program instructions, information corresponding to the reconstructed image of the block and / or intermediate data during the encoding or decoding process. The memory in some embodiments includes a non-transitory computer-readable medium, such as a semiconductor or solid-state memory, a random access memory (RAM), a read-only memory (ROM), a hard disk, an optical disk, or other suitable storage medium. The memory can also be a combination of two or more of the non-transitory computer-readable media listed above. Such as Fig.13 as well as Fig.14As shown, the encoder 1300 and the decoder 1400 may be implemented in the same electronic device, and thus various functional elements of the encoder 1300 and the decoder 1400 may be shared or reused if implemented in the same electronic device.

[0103] Embodiments of the video processing methods for encoding or decoding can be implemented in a circuit integrated into a video compression chip or in a program code integrated into video compression software to perform the above-described processing. For example, the determination of a candidate set including an average candidate for encoding a current block can be implemented in a program code that will be executed on a computer processor, a digital signal processor (DSP), a microprocessor, or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software code or firmware code that defines a specific method implemented by the present invention.

[0104] References throughout this specification to "one embodiment", "some embodiments" or similar language mean that specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the appearance of the phrases "in one embodiment" or "in some embodiments" in various locations throughout this specification does not all refer to the same embodiment, which may be implemented individually or in conjunction with one or more other embodiments. In addition, in one or more embodiments, the described features, structures or characteristics may be combined in any suitable manner. However, those skilled in the relevant art will recognize that the present invention may be practiced without one or more specific details, or using other methods, elements, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid blurring aspects of the present invention.

[0105] Without departing from the spirit or essential features of the present invention, the present invention may be implemented in other specific forms. The described examples are considered to be illustrative and not restrictive in all respects. Therefore, the scope of the present invention is indicated by the appended claims rather than the foregoing description. All changes within the equivalent meanings and scope of the claims are included within their scope.

Claims

1. A video processing method using overlapping block motion compensation processing blocks in a video coding and decoding system, characterized in that: The method comprises: receiving input video data associated with a current block in a current image; Determining the number of overlapped block motion compensation mixed lines for the left boundary of the current block according to the width of the current block, or determining the number of overlapped block motion compensation mixed lines for the top boundary of the current block according to the height of the current block; deriving an original predictor for the current block by motion compensation using one or more motion vectors for the current block; deriving an overlapped block motion compensation predictor for a left overlapped block motion compensation area having the number of overlapped block motion compensation blend lines for the left boundary by motion compensation using one or more motion vectors of a left neighboring block of the current block, or deriving an overlapped block motion compensation predictor for a top overlapped block motion compensation area having the number of overlapped block motion compensation blend lines for the top boundary by motion compensation using one or more motion vectors of a top neighboring block of the current block; applying overlapped block motion compensation to the current block by blending the overlapped block motion compensation predictor with the original predictor of the current block for the number of overlapped block motion compensation blend lines; and The current block is encoded or decoded.

2. The video processing method for processing blocks using overlapping block motion compensation in a video encoding and decoding system according to claim 1, characterized in that: Wherein if the width of the current block is less than a predetermined threshold, the number of the overlapping block motion compensation mixed lines at the left boundary of the luminance component is 2, and if the width of the current block is greater than or equal to the predetermined threshold, the number of the overlapping block motion compensation mixed lines at the left boundary of the luminance component is 4; or if the height of the current block is less than a predetermined threshold, the number of the overlapping block motion compensation mixed lines at the top boundary of the luminance component is 2, and if the height of the current block is greater than or equal to the predetermined threshold, the number of the overlapping block motion compensation mixed lines at the top boundary of the luminance component is 4.

3. The video processing method using overlapping block motion compensation processing blocks in a video encoding and decoding system as claimed in claim 1, characterized in that: Wherein, if the width of the current block is less than a predetermined threshold, the number of the overlapping block motion compensation mixed lines at the left boundary of the chroma component is 1, and if the width of the current block is greater than or equal to the predetermined threshold, the number of the overlapping block motion compensation mixed lines at the left boundary of the chroma component is 2; or if the height of the current block is less than a predetermined threshold, the number of the overlapping block motion compensation mixed lines at the top boundary of the chroma component is 1, and if the height of the current block is greater than or equal to the predetermined threshold, the number of the overlapping block motion compensation mixed lines at the top boundary of the chroma component is 2.

4. The video processing method using overlapping block motion compensation processing blocks in a video encoding and decoding system as claimed in claim 1, characterized in that: Further comprising determining a flag indicating whether the number of overlapped block motion compensation blend lines is enabled or disabled, if the flag indicates that the number of overlapped block motion compensation blend lines is disabled, the number of overlapped block motion compensation blend lines for luma components is always 4 and the number of overlapped block motion compensation blend lines for chroma components is always 2.

5. The video processing method using overlapping block motion compensation processing blocks in a video encoding and decoding system as claimed in claim 1, characterized in that: The number of the overlapped block motion compensation mixed lines of the luminance component is determined according to the width or the height, and the number of the overlapped block motion compensation mixed lines of the chrominance component is determined according to the number of the overlapped block motion compensation mixed lines for the luminance component.

6. A device for processing video data in a video coding and decoding system, the device comprising one or more electronic circuits and a memory, wherein the one or more electronic circuits are used to: receiving input video data associated with a current block in a current image; Determining the number of overlapped block motion compensation mixed lines for the left boundary of the current block according to the width of the current block, or determining the number of overlapped block motion compensation mixed lines for the top boundary of the current block according to the height of the current block; deriving an original predictor for the current block by motion compensation using one or more motion vectors for the current block; deriving an overlapped block motion compensation predictor for a left overlapped block motion compensation area having the number of overlapped block motion compensation blend lines for the left boundary by motion compensation using one or more motion vectors of a left neighboring block of the current block, or deriving an overlapped block motion compensation predictor for a top overlapped block motion compensation area having the number of overlapped block motion compensation blend lines for the top boundary by motion compensation using one or more motion vectors of a top neighboring block of the current block; applying overlapped block motion compensation to the current block by blending the overlapped block motion compensation predictor with the original predictor of the current block for the number of overlapped block motion compensation blend lines; and The current block is encoded or decoded.

7. A non-transitory computer-readable medium storing program instructions, wherein the program instructions cause a processing circuit of a device to execute a video processing method, characterized in that: The method comprises: receiving input video data associated with a current block in a current image; Determining the number of overlapped block motion compensation mixed lines for the left boundary of the current block according to the width of the current block, or determining the number of overlapped block motion compensation mixed lines for the top boundary of the current block according to the height of the current block; deriving an original predictor for the current block by motion compensation using one or more motion vectors for the current block; deriving an overlapped block motion compensation predictor for a left overlapped block motion compensation area having the number of overlapped block motion compensation blend lines for the left boundary by motion compensation using one or more motion vectors of a left neighboring block of the current block, or deriving an overlapped block motion compensation predictor for a top overlapped block motion compensation area having the number of overlapped block motion compensation blend lines for the top boundary by motion compensation using one or more motion vectors of a top neighboring block of the current block; applying overlapped block motion compensation to the current block by blending the overlapped block motion compensation predictor with the original predictor of the current block for the number of overlapped block motion compensation blend lines; and The current block is encoded or decoded.

Images