Prediction generation with border crossing check in video coding

Abstract

A method of encoding a video picture using predictive coding and border crossing check (OOB) is provided. A video encoder receives data to be encoded or decoded as a current block of a current picture of a video. A video encoder identifies a first reference block in a first reference picture based on a first block vector of a current block, and identifies a second reference block in a second reference picture based on a second block vector of the current block. The video encoder performs an out-of-border check (OOB) on the first and second reference blocks. The video encoder generates a predictor for the current block based on the first and second reference blocks and based on the out-of-bounds check, so that the use of bidirectional motion compensation can be constrained. The video encoder encodes or decodes the current block using the generated predictor.

Description

[0001] Cross-references

[0002] The present invention is a non-provisional application, claiming the priority of the U.S. Provisional Patent Application No. 63 / 367,738 filed on July 6, 2022. The above-mentioned U.S. Provisional Patent Application is incorporated into this specification by reference in its entirety. Technical Field

[0003] The present invention relates to video coding and decoding. More specifically, the present invention relates to a method for coding and decoding pixel blocks by inter-frame and intra-frame prediction. Background Art

[0004] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed later and are not admitted to be prior art by inclusion in this section.

[0005] High Efficiency Video Codec (HEVC) is an international video codec standard developed by the Joint Collaboration on Video Codecs (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT-like transform coding architecture. The basic unit of compression, called a codec unit (CU), is a 2Nx2N square pixel block, and each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile Video Codec (VVC) is the latest international video codec standard developed by ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 Joint Video Experts Group (JVET). The input video signal is predicted from a reconstructed signal derived from a coded picture region. The prediction residual signal is processed by block transform. The transform coefficients are quantized and entropy coded along with other auxiliary information in the bitstream. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal after inverse transforming the dequantized transform coefficients. The reconstructed signal is further processed by loop filtering to remove coding artifacts. The decoded pictures are stored in a frame buffer and used to predict future pictures in the input video signal.

[0007] In VVC, the coded picture is divided into non-overlapping square block areas represented by associated codec tree units (CTUs). The leaf nodes of the codec tree correspond to codec units (CUs). The coded picture can be represented by a set of slices, each slice containing an integer number of CTUs. Each CTU in a slice is processed in raster scan order. Bi-predictive (abbreviated as B) slices can be decoded using intra prediction or inter prediction, where there are at most two motion vectors and reference indices to predict the sample values ​​of each block. Predictive (abbreviated as P) slices are decoded using intra prediction or inter prediction with at most one motion vector and reference index to predict the sample values ​​of each block. Intra (I) slices are decoded only using intra prediction.

[0008] A quadtree (QT) with a nested multi-type-tree (MTT) structure can be used to divide a CTU into one or more non-overlapping codec units (CUs) to accommodate various local motion and texture features. A CU can be further split into smaller CUs using one of five partitioning types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning.

[0009] Each CU contains one or more prediction units (PUs). The prediction unit, together with the associated CU syntax, serves as the basic unit for sending prediction information. A specified prediction process is used to predict the values ​​of the relevant pixel samples within the PU. Each CU may contain one or more transform units (TUs) for representing prediction residual blocks. A transform unit (TU) consists of a transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, each TB corresponding to a residual block of samples from one color component. An integer transform is applied to the transform block. The level values ​​of the quantized coefficients are entropy encoded and decoded in the bitstream along with other auxiliary information. The terms codec tree block (CTB), codec block (CB), prediction block (PB), and transform block (TB) are defined as 2-D sample arrays specifying one color component associated with a CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU consists of a luma CTB, two chroma CTBs, and related syntax elements. Similar relationships are valid for CUs, PUs, and TUs.

[0010] For each inter-predicted CU, motion parameters consisting of motion vector, reference picture index and reference picture list usage index, as well as additional information, are used for inter-prediction sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is encoded or decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector increments or reference picture indexes. To specify the merge mode, the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in VVC. The merge mode can be applied to any inter-predicted CU. An alternative to the merge mode is the explicit transmission of motion parameters, where the motion vector, the corresponding reference picture index and reference picture list usage flag for each reference picture list, as well as other required information, are explicitly sent per CU. Summary of the invention

[0011] The following is for illustrative purposes only and is not intended to be limiting in any way. That is, the following summary is intended to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious technology described herein. Selected, but not all, embodiments will be further described in the detailed description below. Therefore, the following summary is not intended to identify the essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

[0012] Some embodiments of the present disclosure provide a method for encoding a video picture using predictive coding and out-of-bounds check (OOB). A video encoder receives data of a current block of a current picture to be encoded or decoded as a video. The video encoder identifies a first reference block in a first reference picture based on a first block vector of the current block, and identifies a second reference block in a second reference picture based on a second block vector of the current block. The video encoder performs an out-of-bounds check (OOB) on the first and second reference blocks. The video encoder generates a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check. The video encoder encodes or decodes the current block using the generated predictor.

[0013] The first and second block vectors may be motion vectors.In some embodiments, the first reference picture or the second reference picture is a current picture, and the first block vector or the second block vector is an intra prediction direction or mode.

[0014] In some embodiments, when the sample of the first reference block is out of bounds, bidirectional motion compensation is not applied in at least one of the horizontal direction and the vertical direction. In some embodiments, when the sample of the first reference block is out of bounds on the left boundary or the right boundary of the first reference picture, or when the sample of the first reference block is out of bounds on the top boundary or the bottom boundary of the first reference picture, bidirectional motion compensation is not applied. In some embodiments, when the sample of the first reference block is a non-out-of-bounds sample, the corresponding prediction sample is generated by mixing the non-out-of-bounds sample of the first reference block with the corresponding non-out-of-bounds sample of the second reference block. When the sample of the first reference block is an out-of-bounds sample, the corresponding prediction sample is generated by using the corresponding non-out-of-bounds sample of the second reference block without mixing.

[0015] In some embodiments, the video may be a 360-degree video in an equirectangular projection (ERP) format. When a sample of a first reference block crosses a left boundary or a right boundary of a first reference picture, a corresponding prediction sample is generated by blending horizontal surrounding samples from the first reference picture with corresponding non-crossing samples of a second reference block in a second reference picture. When a sample of the first reference block crosses a top boundary or a bottom boundary of the first reference picture, a corresponding prediction sample is generated by using corresponding non-crossing samples from the second reference block without blending.

[0016] In some embodiments, the current block is divided into a plurality of geometric partitions, a first reference block is used to generate a first predictor for the first geometric partition, and a second reference block is used to generate a second predictor for the second geometric partition. Corresponding prediction samples are generated by blending samples of the first and second predictors along a boundary between the first and second geometric partitions, wherein out-of-bounds samples of the first and second predictors are excluded from the blending.

[0017] In some embodiments, the generated predictor is a combined inter-intra prediction (CIIP), wherein the first reference block provides inter prediction samples and the second reference block provides intra prediction samples. In some embodiments, the current block is encoded using multiple hypothesis prediction (MHP), such that the encoder performs an additional out-of-bounds check on the third reference block, and the predictor is further generated based on the third reference block and the additional out-of-bounds check.

[0018] In some embodiments, the first block vector is refined to minimize a template matching cost calculated between a reference template adjacent to the first reference block and a current template adjacent to the current block, wherein out-of-bounds samples of the reference template are excluded from the cost calculation. In some embodiments, the first and second block vectors are refined to minimize a bilateral matching cost based on the first and second reference blocks, wherein out-of-bounds samples in the first and second reference blocks are excluded from the cost calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of the present disclosure. The accompanying drawings illustrate embodiments of the present disclosure and together with the description are used to explain the principles of the present disclosure. It is worth noting that the drawings are not necessarily drawn to scale, as certain components may be shown out of scale in actual implementations in order to clearly illustrate the concepts of the present disclosure.

[0020] Figure 1 The diagram illustrates bidirectional prediction using an out-of-boundary (OOB) reference block.

[0021] Figure 2 Multiple Hypothesis Prediction (MHP) for encoding and decoding a current block is conceptually illustrated.

[0022] Figure 3 The partitioning of the CU by the geometric partitioning mode (GPM) is indicated.

[0023] Figure 4 An exemplary uni-directional prediction candidate list for a GPM partition and the selection of a uni-directional prediction MV for a GPM are shown.

[0024] Figure 5 An example partition edge blending process for a GPM of a CU is shown.

[0025] Figure 6 Performing template matching based on a search area around an initial motion vector (MV) is conceptually illustrated.

[0026] Figure 7 A current block 700 with sub-block motion is conceptually illustrated.

[0027] Figure 8 The refinement of prediction candidates (eg, merge candidates) through bilateral matching (BM) is conceptually illustrated.

[0028] Fig. 9 The intra prediction modes for different directions are shown.

[0029] Fig. 10A -B conceptually illustrates wide-angle directional mode with extended length top and left reference templates to support non-square blocks of different aspect ratios.

[0030] Fig.11A -C conceptually illustrates horizontal surround motion compensation with out-of-bounds (OOB) checking.

[0031] Fig.12 Intra prediction using OOB reference samples is conceptually illustrated.

[0032] Fig.13An example video encoder that can implement predictive coding is identified.

[0033] Fig.14 The portion of a video encoder that implements out-of-bounds checking for predictive codecs is explained.

[0034] Fig.15 The process of predictive coding for a block of pixels using out-of-bounds checking is conceptually illustrated.

[0035] Fig.16 An example video decoder that can implement predictive coding is identified.

[0036] Fig.17 Portions of a video decoder that implements out-of-bounds checking for predictive codecs are explained.

[0037] Fig.18 The process of predictive coding and decoding for a pixel block using out-of-bounds checking is conceptually illustrated.

[0038] Fig.19 An electronic system that implements some embodiments of the present disclosure is conceptually illustrated. DETAILED DESCRIPTION

[0039] In the detailed description below, many specific details are set forth by way of example to provide a thorough understanding of the relevant teachings. Any variations, derivations, and / or extensions based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, procedures, components, and / or circuits associated with one or more example implementations disclosed herein may be described at a relatively high level without details to avoid unnecessarily obscuring aspects of the teachings of the present disclosure.

[0040] I. Motion Compensation Out-of-Boundary Check

[0041] An inter-predicted CU may have reference blocks that are partially or completely outside the reference picture. Figure 1The diagram illustrates bidirectional prediction using an Out-of-Boundary (OOB) reference block. In the figure, bidirectional motion compensation is performed to generate an inter-frame prediction block (predictor) of a current block 105 in a current picture 100. List 0 reference block 120 partially exceeds the reference block picture 110. List 1 reference block 121 is completely located inside the reference picture 111. When the video codec performs the prediction generation process 130, the OOB portion 140 of the reference block 120 is filled with repetitive padding samples. The prediction generation process 130 generates a prediction block 135 for the current block 105 using reference blocks 120 and 121. Since the OOB portion 140 of the reference block 120 is filled with repeated samples derived from boundary samples within the reference picture 110, the portion of the motion compensation block 135 corresponding to the OOB portion 140 may provide lower prediction efficiency.

[0042] In some embodiments, a unidirectional motion compensated sample is considered OOB when one of its reference samples is more than half a sample outside the reference picture. For each prediction sample in a bidirectional motion compensated block, when one of its unidirectional prediction samples is OOB and the other is non-OOB, the corresponding bidirectional sample is set equal to the non-OOB sample instead of using the average of the OOB and non-OOB samples.

[0043] In some embodiments, when more than one prediction block is combined, OOB prediction samples are discarded and only non-OOB predictors are used to generate the final predictor. Specifically, let Pos_x i,j and Pos_y i,j Represents the x and y position of a predicted sample (i, j) in the current block, MV LX _x i,j and MV LX _y i,j (LX = L0 or L1) indicates the MV of the current block; Pos LeftBdry 、Pos RightBdry 、Pos TopBdry 、Pos BottomBdry are the positions of the four boundaries of the picture. A prediction sample is considered OOB when at least one of the following conditions is met:

[0044] (Pos_x i,j +MV LX _x i,j )>(Pos RightBdry +half_pixel),

[0045] (Pos_x i,j +MV LX _x i,j )<(Pos LeftBdry+half_pixel),

[0046] (Pos_y i,j +MV LX _y i,j )>(Pos BottomBdry +half_pixel),

[0047] (Pos_y i,j +MV LX _y i,j )>(Pos TopBdry +half_pixel)

[0048] Where half_pixel is equal to 8, indicating the half-pixel sample distance in 1 / 16 pixel sample accuracy. After checking the OOB status of each sample, the final prediction samples of a bidirectional block are generated as follows:

[0049] If P L0 i,j is OOB and P L1 i,j If P is non-OOB, final i,j =P L1 i,j

[0050] Otherwise, if P L0 i,j is non-OOB and P L1 i,j is OOB, then P final i,j =P L0 i,j

[0051] Otherwise P final i,j =(P L0 i,j +P L1 i,j +1)>>1

[0052] II. Block Codec Prediction Tool

[0053] a. Horizontal Wrap Around Motion Compensation

[0054] Horizontal surround motion compensation is a 360-degree specific codec tool designed to improve the visual quality of reconstructed 360-degree videos in equirectangular projection (ERP) format. In traditional motion compensation, when a motion vector refers to a sample outside the picture boundaries of a reference picture, repeated padding is applied to derive the value of the out-of-bounds sample by copying from the nearest neighbor on the corresponding picture boundary. For 360-degree video, this method of repeated padding is inappropriate and may cause visual artifacts called "seam artifacts" in the reconstructed viewport video. Since 360-degree video is captured on a sphere and has no "borders" in essence, reference samples that are outside the reference picture boundaries in the projection domain can always be obtained from neighboring samples in the spherical domain.

[0055] For general projection formats, it may be difficult to derive the corresponding neighboring samples in the spherical domain, because doing so involves 2D to 3D and 3D to 2D coordinate conversions, as well as sample interpolation at fractional sample positions. This problem is much simpler for the left and right boundaries of the ERP projection format, because the spherical neighborhood outside the left picture boundary can be obtained from samples inside the right picture boundary, and vice versa. Therefore, horizontal surround motion compensation can be used to improve the visual quality of 360-degree videos encoded and decoded in the ERP projection format.

[0056] In some embodiments, horizontal surround motion compensation can be combined with the (non-normative) padding method often used in 360-degree video codecs. This can be achieved by signaling a high-level syntax element to indicate a surround offset, which can be set to the padding width applied to the ERP picture. This syntax can be used to adjust the position of the horizontal surround accordingly. This syntax is not affected by the specific padding amount of the left and right picture boundaries, so it naturally supports asymmetric padding of ERP pictures, i.e. when the left and right padding are different. When the reference sample is outside the left and right boundaries of the reference picture, horizontal surround motion compensation provides more meaningful information for motion compensation.

[0057] b. Multi-Hypothesis Prediction (MHP)

[0058] In the multi-hypothesis inter-frame prediction mode (MHP), in addition to the traditional bidirectional prediction signal, one or more additional motion compensated prediction signals are signaled. The final overall prediction signal is obtained by weighted superposition of samples. bi and the first additional inter-frame prediction signal / hypothesis h3, the resulting prediction signal p3 is obtained according to the following formula

[0059] p3=(1–α)p bi +αh3

[0060] The weighting factor α (or mixing ratio) is specified by the syntax element add_hyp_weight_idx in the codec video as 1 / 4 or -1 / 8, etc.

[0061] Figure 2 The multi-hypothesis prediction (MHP) for encoding and decoding a current block 220 is conceptually illustrated. The current block 220 is in a current picture 200. The current block has bi-directional prediction motion vectors MV0 and MV1 used to obtain predictors 230 (in a reference picture 210) and 231 (in a reference picture 211), respectively. The bi-directional prediction predictors are used to generate a bi-directional prediction signal p bi In addition to bidirectional prediction, the current block is also encoded and decoded using MHP prediction. MHP prediction is based on an additional hypothesis selected from a plurality of MHP candidates A, B, C and D, which locate predictors or prediction signals 241, 242, 243 and 244 in different reference pictures, respectively. In the example, hypothesis C is selected to generate an additional prediction signal h3, i.e., prediction signal 243.

[0062] In some embodiments, more than one additional prediction signal may be used. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal. For example, a second additional hypothesis h4 may be added to the prediction according to the following condition:

[0063] p4=(1-α)p3+αh4

[0064] Or more generally:

[0065] p n+1 =(1-α n+1 ) n +α n+1 h n+1

[0066] The predicted signal p n+1 As the last p n (the current accumulated forecast) and the latest additional forecast signal h n+1 The first existing prediction of the current block is p0. For example, if MHP is applied to the merge candidate, p0 is indicated by the existing merge index. n+1 and p n+1 The mixture of is a weighted sum based on a mixture ratio or weighting factor α. In some embodiments, for each additional hypothesis for prediction, a weighting index may be signaled / parsed to indicate the weighting factor.

[0067] c. Combined inter and intra prediction (CIIP)

[0068] Combined inter-frame and intra-frame prediction (CIIP) combines the inter-frame prediction signal with the intra-frame prediction signal. The inter-frame prediction signal P in CIIP mode inter is derived using the same inter-frame prediction process applied to the regular merge mode; while the intra-frame prediction signal P intra It is derived after a conventional intra prediction process using a planar mode or one or more intra prediction modes derived from a predefined mechanism. For example, in the following, the predefined mechanism is based on a neighboring reference area (template) of the current block. The intra prediction mode of the CU is implicitly derived from the neighboring templates at the encoder and decoder, rather than being sent to the decoder as an exact intra prediction mode bit. The prediction samples of the template are generated using the reference samples of the template for each candidate mode. The cost is calculated as the SATD between the prediction samples and the reconstructed samples of the template. The intra prediction mode with the minimum cost and / or some intra prediction modes with smaller costs are selected and used for intra prediction of the CU. The candidate modes can be all MPMs and / or any subset of MPMs, such as 67 intra prediction modes in VVC or extended to 131 intra prediction modes. The intra and inter prediction signals are combined using weighted averaging, where the weight values ​​are calculated based on the encoding and decoding modes of the top and left neighboring blocks. CIIP predicts P CIIP It is formed as follows: (wt is the weight value)

[0069] P CIIP =((4–wt)*P inter +wt*P intra +2)>>2

[0070] In some embodiments, when a CU is encoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag may be signaled to indicate whether the CIIP mode applies to the current CU.

[0071] d. Geometric partitioning mode (GPM)

[0072] The geometric partition mode is a codec tool for inter-frame prediction that splits the block currently being encoded into two or more parts. The partition mode of each GPM partition or GPM split is characterized by the distance-angle pairing that defines the bisecting line or segmenting line. The geometric partition mode (GPM) is sent as a merge mode using a CU-level flag. Other merge modes include normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size w×h=2 m ×2 n (where m,n∈{3…6}, excluding 8x64 and 64x8), the geometric partitioning mode supports a total of 64 partitions.

[0073] Figure 3 The partitioning of a CU by a geometric partitioning mode (GPM) is marked. The figure shows an example of a GPM split grouped by the same angle. As shown in the figure, when GPM is used, the CU is divided into at least two parts by a geometrically positioned straight line. The position of the partition line is mathematically derived from the angle and offset parameters of the specific partition.

[0074] Each partition in a CU formed by the partitioning pattern of GPM uses its own motion (vector) for inter-frame prediction. In some embodiments, each partition only allows unidirectional prediction, that is, each part has one motion vector and one reference index. Similar to traditional bidirectional prediction, unidirectional prediction motion constraints are applied to ensure that only two motion compensation predictions are performed for each CU.

[0075] If GPM is used for the current CU, a geometric partition index indicating the partition mode (angle and offset) of the geometric partition and two merge indices (one for each partition) are further sent. Each of the at least two partitions created by geometric partitioning according to the partition can be assigned a merge index to select a candidate from a unidirectional prediction candidate list (also called a GPM candidate list). Therefore, the merge index pair of the two partitions selects a pair of merge candidates. The maximum number of candidates in the GPM candidate list can be explicitly sent in the SPS to specify the syntactic binarization of the GPM merge index. After predicting each of the at least two partitions, the sample values ​​along the edges of the geometric partitions are adjusted using a hybrid process with adaptive weights. This is the prediction signal for the entire CU, and the transform and quantization processes will be applied to the entire CU as in other prediction modes. The motion field of the CU predicted by GPM is then stored.

[0076] The unidirectional prediction candidate list (GPM candidate list) of the GPM partition can be directly derived from the merge candidate list of the current CU. Figure 4An exemplary unidirectional prediction candidate list 400 for a GPM partition and the selection of a unidirectional prediction MV for the GPM are shown. The GPM candidate list 400 is constructed in an even-odds manner, with only unidirectional prediction candidates alternating between L0MV and L1MV. Set n to the index of the unidirectional prediction motion in the unidirectional prediction candidate list for the GPM. The LX (i.e., L0 or L1) motion vector of the nth extended merge candidate, where X is equal to the parity of n, is used as the nth unidirectional prediction motion vector for the GPM. (These motion vectors are marked with "x" in the figure.) In the absence of a corresponding LX motion vector for the nth extended merge candidate, the L(1-X) motion vector of the same candidate is used as the unidirectional prediction motion vector for the GPM.

[0077] As mentioned above, the sample values ​​along the edges of geometric partitions are adjusted using a blending process with adaptive weights. Specifically, after predicting each part of the geometric partition using its own motion, blending is applied to at least two prediction signals to derive samples around the edges of the geometric partitions. The blending weights for each position of the CU are derived based on the distance between the corresponding position and the partition edge. The distance from the position (x, y) to the partition edge is derived as follows:

[0078]

[0079]

[0080] where i,j are the angle and offset indices of the geometry partition, which depends on the geometry partition index sent. x,j and ρ y,j The sign of depends on the angle index i. The weight of each part of the geometric partition is derived as follows:

[0081] wIdxL(x,y)=partIdx? 32+d(x,y):32-d(x,y)

[0082]

[0083] w1(x,y)=1-w0(x,y)

[0084] The variable partIdx depends on the angle index i. Figure 5 An example partition edge blending process is shown for the GPM of CU 500. In the figure, blending weights are generated based on initial blending weights w0.

[0085] As described above, the motion field of a CU predicted using GPM is stored. Specifically, Mv1 from the first part of the geometric partition, Mv2 from the second part of the geometric partition, and the combination Mv of Mv1 and Mv2 are stored in the motion field of the CU encoded by GPM. The type of stored motion vectors for each individual position in the motion field is determined as:

[0086] sType=abs(motionIdx)<32?

[0087] 2: (motionIdx≤0? (1-partIdx):partIdx)

[0088] Where motionIdx is equal to d(4x+2,4y+2), which is recalculated from equation (2). partIdx depends on the angle index i. If sType is equal to 0 or 1, Mv0 or Mv1 is stored in the corresponding motion field, otherwise if sType is equal to 2, the combined Mv from Mv0 and Mv2 is stored. The combined Mv is generated using the following process: (i) If Mv1 and Mv2 come from different reference picture lists (one from L0 and the other from L1), Mv1 and Mv2 are simply combined to form a bidirectional prediction motion vector; (ii) Otherwise, if Mv1 and Mv2 come from the same list, only the unidirectional prediction motion Mv2 is stored.

[0089] e. Template Matching (TM)

[0090] Template matching (TM) is a decoder-side MV derivation method used to refine the motion information of the current CU by finding the closest match between the template of the current CU in the current picture (e.g., the top and / or left neighboring blocks of the current CU) and a set of pixels (i.e., the same size as the template) in the reference picture.

[0091] Figure 6 The performance of template matching based on a search area around an initial motion vector (MV) is conceptually illustrated. As shown, for a current CU 605 in a current picture 600, the video codec searches a reference picture or frame 601 within a [-8, +8] pixel search range around an initial MV 610 to obtain a better or refined MV 611. The initial MV 610 identifies an initial reference template 630. The refined MV 611 identifies a refined reference template 631. The search is based on minimizing the difference (or cost) between a current template 620 adjacent to the current block 605 and the refined reference template 631. Template matching can be performed using a search step size determined based on an adaptive motion vector resolution (AMVR) mode. The template matching process can be cascaded with a bilateral matching process in a merge mode.

[0092] In the Advanced Motion Vector Prediction (AMVP) mode, the MVP candidate is determined based on the template matching error to select the MVP candidate with the smallest difference between the current block template and the reference block template, and then TM is performed for MV refinement only on this specific MVP candidate. The TM process refines this MVP candidate by using an iterative diamond search starting with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode) within the [–8, +8] pixel search range. The AMVP candidate can be further refined by using a cross search with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode), and then using half-pixels and quarter-pixels in turn according to the AMVR mode search pattern.

[0093] Adaptive Reordering of Merge Candidates with Template Matching (ARMC-TM) is a method of reordering merge candidates based on template matching (TM) cost, where the signaling efficiency is improved by sorting the merge candidates in ascending order of TM cost. For ARMC-TM or TM merge mode, the merge candidates are reordered before the refinement process. The template matching cost of the merge candidates can be measured by the sum of absolute differences (SAD) between the samples of the current template 620 of the current block and their corresponding reference samples in the reference template 621.

[0094] In some embodiments, after constructing the merge candidate list, the merge candidates are divided into several subgroups. For the normal merge mode and the TM merge mode, the subgroup size is set to 5. For the affine merge mode, the subgroup size is set to 3. The merge candidates in each subgroup are reordered in ascending order according to the cost value based on template matching. In some embodiments, the merge candidates in the last but not the first subgroup are not reordered.

[0095] For sub-block based merge candidates with sub-block size equal to Wsub×Hsub, the template above may include several sub-templates of size Wsub×1, and the template on the left may include several sub-templates of size 1×Hsub. In some embodiments, the motion information of the sub-blocks in the first row and first column of the current block is used to derive reference samples for each sub-template.

[0096] Figure 7A current block 700 with sub-block motion is conceptually illustrated. The current block 700 is encoded using motion information of a sub-block (sub-block AG) in the first row and first column of the current block. The motion information of the sub-block AG is used to identify a reference sub-block A'-G' in a reference picture. The current block 700 has a neighboring template 710, which includes sub-templates 711-717 that are adjacent to the sub-block AG above and to the left of the current block 700, respectively. The reference sub-block A'-G' has a corresponding neighboring reference sub-block 721-727 in the reference picture. The TM cost of motion information of different sub-blocks can be calculated by matching the sub-templates 711-717 with the corresponding sub-templates 721-727.

[0097] f. Decoder Motion Vector Refinement (DM VR)

[0098] In order to increase the accuracy of the MV in merge mode, decoder-side motion vector refinement based on bilateral-matching (BM) can be applied to refine the MV. In the bidirectional prediction operation, the refined MV is searched around the initial MV in the reference picture list L0 and the reference picture list L1. The BM method calculates the distortion between two candidate blocks in the reference picture list L0 and the list L1. The MV candidate with the lowest SA D becomes the refined MV and is used to generate the bidirectional prediction signal.

[0099] In some embodiments, if the selected merge candidate satisfies the DMVR condition, a multi-pass decoder-side motion vector refinement (MP-DMVR) method is applied in a conventional merge mode. In the first channel, bilateral matching (BM) is applied to the codec block. In the second channel, BM is applied to each 16x16 sub-block within the codec block. In the third channel, the MV in each 8x8 sub-block is refined by applying bidirectional optical flow (BDOF). BM refines a pair of motion vectors MV 0 and MV1 under the constraint that the motion vector difference MVD0 (i.e., MV0'-MV0) is exactly opposite in sign to the motion vector difference MVD1 (i.e., MV1'-MV1).

[0100] Figure 8Conceptually illustrates refinement of a prediction candidate (e.g., a merge candidate) by bilateral matching (BM). MV0 is an initial motion vector or prediction candidate, and MV1 is a mirror image of MV0. A current block 800 in a current picture 801 refers to an initial reference block 820 in a reference picture 810 via MV0. MV1 refers to an initial reference block 821 in a reference picture 811. The figure shows that MV0 and MV1 are refined to form MV0' and MV1', which update reference blocks 830 and 831, respectively. The refinement is performed based on bilateral matching, so that the refined motion vector pair MV0' and MV1' has a better bilateral matching cost than the initial motion vector pair MV0 and MV1. MV0'-MV0 (i.e., MVD0) and MV1'-MV1 (i.e., MVD1) are constrained to be equal in size but opposite in direction. In some embodiments, the bilateral matching cost for a pair of mirrored motion vectors (eg, MV0 and MV1) is calculated based on the difference between two reference blocks referenced by the mirrored motion vectors (eg, the difference between reference blocks 820 and 821).

[0101] g. Intra-frame prediction

[0102] The intra prediction method uses a reference layer (referencetier) adjacent to the current prediction unit (PU) and an intra prediction mode to generate a prediction sub-unit for the current PU. The intra prediction direction can be selected from a mode set containing multiple prediction directions. For each PU of the intra prediction codec, an index will be used and encoded to select an intra prediction mode. The corresponding prediction will be generated, and then the residual can be derived and transformed.

[0103] Fig. 9 Intra-frame prediction modes in different directions are shown. These intra-frame prediction modes are called directional modes and do not include DC mode or planar mode. As shown in the figure, there are 33 directional modes (V: vertical direction; H: horizontal direction), so H, H+1~H+8, H-1~H-7, V, V+1~V+8, V-1~V-8 are used. Usually the directional mode can be expressed as H+k or V+k mode, where k=±1, ±2,..., ±8. Each of these intra-frame prediction modes can also be called an intra-frame prediction angle. In order to capture arbitrary edge directions presented in natural videos, the number of directional intra-frame modes can be extended from 33 used in HEVC to 65 directional modes so that k ranges from ±1 to ±16. These denser directional intra-frame prediction modes are applicable to all block sizes and luminance and chrominance intra-frame predictions. By including DC and planar modes, the number of intra-frame prediction modes is 35 (or 67).

[0104] Of the 35 (or 67) intra prediction modes, some modes (e.g., 3 or 5) are identified as a set of most probable modes (MPMs) for intra prediction in the current prediction block. The encoder can reduce the bit rate by signaling the index of selecting one of the MPMs instead of selecting the index of one of the 35 (or 67) intra prediction modes. For example, the intra prediction mode used in the left prediction block and the intra prediction mode used in the prediction block above are used as MPMs. When the intra prediction modes in two neighboring blocks use the same intra prediction mode, the intra prediction mode can be used as an MPM. When only one of the two neighboring blocks is available and encoded in a directional mode, the two neighboring directions adjacent to the directional mode can be used as MPMs. DC mode and planar mode are also considered as MPMs to fill in the available points in the MPM set, especially if the upper or top neighboring block is not available or is not encoded in intra prediction, or if the intra prediction mode in the neighboring block is not a directional mode. If the intra prediction mode of the current prediction block is one of the modes in the MPM set, 1 or 2 bits are used to signal which one it is. Otherwise, the intra prediction mode of the current block is different from any entry in the MPM set, and the current block will be encoded into non-MPM mode. There are 32 such non-MPM modes in total, and a (5-bit) fixed-length codec method is used to signal this mode.

[0105] The MPM list is constructed based on the intra-frame modes of the left neighbor block and the upper neighbor block. Assuming that the mode of the left neighbor block is recorded as Left and the mode of the upper neighbor block is recorded as Above, the unified MPM list can be constructed as follows:

[0106] – When a neighboring block is not available, its intra mode is set to planar by default.

[0107] – If both Left and Above modes are non-angle modes:

[0108] MPM list → {Planar,DC,V,H,V-4,V+4}

[0109] – If one of the Left and Above modes is an angle mode and the other is a non-angle mode:

[0110] Set the mode Max to the larger mode between Left and Above

[0111] MPM list → {Planar,Max,Max-1,Max+1,Max-2,Max+2}

[0112] – If Left and Above are both angled and they are different:

[0113] Set the mode Max to the larger mode between Left and Above

[0114] Set the Min mode to the smaller of Left and Above.

[0115] If Max–Min is equal to 1:

[0116] –MPM list → {Planar,Left,Above,Min–1,Max+1,Min–2}

[0117] Otherwise, if Max–Min is greater than or equal to 62:

[0118] –MPM list → {Planar,Left,Above,Min+1,Max–1,Min+2}

[0119] Otherwise, if Max–Min equals 2:

[0120] –MPM list → {Planar,Left,Above,Min+1,Min–1,Max+1}

[0122] ·otherwise:

[0123] –MPM list → {Planar,Left,Above,Min–1,Min+1,Max–1}

[0125] – If Left and Above are both angled and they are the same:

[0126] ·MPM list→{Planar,Left,Left-1,Left+1,Left–2,Left+2}

[0127] The traditional angular intra prediction direction is defined as clockwise from 45 degrees to -135 degrees. In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are signaled using the original mode indices, which are remapped to the indices of the wide-angle mode after parsing.

[0128] For some embodiments, the total number of intra prediction modes remains unchanged, ie, 67, and the intra mode encoding and decoding method remains unchanged. To support these prediction directions, a top reference template of length 2W+1 and a left reference template of length 2H+1 are defined. Fig. 10A -B conceptually illustrates wide-angle directional mode with extended length top and left reference templates to support non-square blocks of different aspect ratios.

[0129] h. Overlapped Block Motion Compensation (OB MC)

[0130] Overlapped Block Motion Compensation (OBMC) is a video codec tool that reduces blocking artifacts by refining the top and left boundary pixels of a CU using the motion information of neighboring blocks with weighted prediction, specifically:

[0131] Inter' predY =((128–w1)×Inter predY +w1×OBMC predY ) / 128

[0132] PredY=((4–w0)×FwdMap(Inter' predY )+w0×Intra predY ) / 4

[0133] Among them, Inter predY Intra represents the sample of motion prediction of the current block in the original domain. predY Represents the sample predicted in the image domain, OBMC predY Represents the samples of the original domain neighboring block motion prediction, w0 and w1 are weighted.

[0134] Subblock boundary OBMC is performed by applying the same blending to the top, left, bottom, and right subblock boundary pixels using the motion information of the neighboring subblocks. OBMC can be enabled for subblock-based codec tools such as affine AMVP mode, affine merge mode, and subblock-based temporal motion vector prediction, as well as subblock-based bilateral matching. When OBMC mode uses luma mapping with chroma scaling (LMCS) in CIIP mode, inter blending is performed before LMCS mapping of inter samples. LMCS is applied to the blended inter samples, which are combined with intra samples to which LMCS is applied in CIIP mode.

[0135] III. Out-Of-Boundary Check for PredictionCoding Tools

[0136] In order to improve the coding efficiency of bidirectional motion compensation with OOB check, some embodiments of the present disclosure provide a method for performing OOB check on a prediction coding mode other than bidirectional motion compensation.

[0137] A. OOB inspection of ERP video

[0138] In some embodiments, for the horizontal wrap around motion compensation process or for motion compensation of 360-degree video in equirectangular projection (ERP) format, the bidirectional motion compensation with OOB check is applied only in the vertical direction, i.e., horizontal wrap around motion compensation is used for OOB samples in the horizontal direction, and bidirectional motion compensation with OOB check is used for OOB samples in the vertical direction. In some embodiments, for the horizontal wrap around motion compensation process or for motion compensation of 360-degree video in ERP projection format, the bidirectional motion compensation with OOB check is not applied.

[0139] Fig.11A - C conceptually illustrates horizontal surround motion compensation with out-of-bounds (OOB) check. A current block 1105 in a current picture 1100 is encoded by bi-directional prediction using motion vectors MV0 and MV1. Fig.11A MV0 is shown pointing to reference block 1120 in reference picture 1110. Fig. 11B MV1 is shown pointing to reference block 1121 in L1 reference picture 1111.

[0140] As shown, a portion 1130 of the reference block 1120 is OOB because it is outside the left (or right) boundary of the projection domain of the reference picture 1110. Instead of repeat padding, samples for the OOB portion 1130 of the reference block 1120 are generated by horizontal surround motion compensation, specifically by copying from a spherical neighbor 1131 located within the reference picture 1110 to the right boundary in the projection domain. A portion 1140 of the reference block 1121 is OOB because it is outside the top (or bottom) boundary of the projection domain of the L1 reference picture 1111. In some embodiments, repeat padding is applied to generate samples of the OOB portion 1140.

[0141] Fig. 11C The diagram illustrates generating a bidirectional motion compensated predictor 1135 based on OOB and non-OOB samples of the ERP video. For some embodiments, the method described in Section I above may be used to generate a predictor 1135 for the current block 1110 based on the reference blocks 1120 and 1121 and their respective OOB states. For some embodiments, samples generated by the horizontal surround of the OOB portion 1130 of the reference block 1120 are not considered OOB samples.

[0142] As shown, at positions not corresponding to OOB samples, non-OOB samples from the reference block 1120 (including horizontal surrounding samples 1130) and non-OOB samples from the reference block 1121 are mixed (by, for example, weighted averaging) to generate prediction samples for the predictor 1135. On the other hand, at sample positions corresponding to OOB samples (e.g., positions in the OOB portion 1140), non-OOB samples from the reference block 1120 (including horizontal surrounding samples 1130) are used (thus the OOB samples are not included in the predictor 1135).

[0143] B. OOB Check for Intra Prediction Blocks

[0144] In some embodiments, OOB checking is applied to intra-predicted blocks. In some embodiments, a video codec may treat a sample as OOB if the sample is generated by a padded L-neighbor in an intra block (eg, at a picture boundary). Fig.12 Conceptually illustrate intra prediction using OOB reference samples. As shown, the current block 1210 is intra predicted. The samples of the upper / top L-neighbor 1220 of the current block are true reconstructed samples, while the samples of the left L-neighbor 1222 of the current block are padding samples. Block 1210 is intra predicted from the diagonal direction. The samples in the upper triangle 1230 are non-OOB and the samples in the lower triangle 1232 are OOB samples.

[0145] C. OOB for blocks with decoder-side MV refinement

[0146] In some embodiments, OOB checking is applied to DMVR sub-blocks (or codec block-level MP-D MVR, or sub-block-level MP-DMVR). Specifically, if a search point (e.g., initial MV and MV offset) in L0 and / or L1 reference pictures is OOB, the search point is designated as unavailable or deprioritized relative to non-OOB search points (e.g., by applying an additional cost or a predefined large cost). In some embodiments, if all search points and the initial MV are OOB, the refined MV is set to the initial MV.

[0147] In some embodiments, even if the initial MV is OOB, the video codec still compares the initial MV with other non-OOB search points. The video encoder does not set the initial MV to be unavailable, or set the initial MV to a lower priority than non-OOB search points, or add additional costs, or apply a predefined large cost.

[0148] D. OOB of sub-blocks in the reference image

[0149] In some embodiments, when bidirectional prediction uses a subblock in L0 and / or L1 reference pictures, if samples in the subblock are OOB, then the subblock in the L0 or L1 reference picture is OOB. In some embodiments, the subblock is OOB if the number of OOB samples in the subblock is greater than a predefined threshold, or if the ratio of OOB samples in the subblock is greater than a predefined ratio, or if all samples in the subblock are OOB. In some embodiments, the search point is OOB if both subblocks in L0 and L1 reference pictures are OOB, or any one of the subblocks in L0 or L1 reference pictures is OOB.

[0150] E. OOB of GPM Coded Blocks

[0151] In some embodiments, OOB check is applied to two (or more) partitions of a GPM codec block (e.g., by GPM-TM, GPM-MMVD, GPM-INTER-INTRA, GPM-INTRA-INTRA). Specifically, OOB check can be performed on 2 inter-frame partitions, or 1 inter-frame partition and 1 intra-frame partition, or 2 intra-frame partitions. GPM mixing is then applied to the result of the OOB check according to the following:

[0152] If P i,j part0 is OOB and P i,j part1 If P is non-OOB, i,j final =P i,j part1

[0153] Otherwise if P i,j part0 is non-OOB and P i,j part1 is OOB, then P i,j final =P i,j part0

[0154] Otherwise P i,j final =GPM_blending(P i,j part0 ,P i,j part1 )

[0155] In some embodiments, the OOB check is not applied to the intra-frame part but only to the inter-frame part, so that the OOB check result of the intra-frame part is always non-OOB. A GPM mixture of a GPM with one inter-frame part and one intra-frame part can be calculated according to:

[0156] If Pi,j partX is OOB and partX is inter-frame and part 1-X is intra-frame, then P i,j final =P i,j part(1-X)

[0157] Otherwise P i,j final =GPM_blending(P i,j part0 ,P i,j part1 )

[0158] If the samples of the inter part are OOB, the samples of the intra part are used as predictors. Otherwise, a blend is applied to the samples of both parts. A GPM blend with a GPM of two intra parts is the same as a GPM blend with a GPM of two inter parts. In some embodiments, if the inter prediction signal of the inter part is bidirectionally predicted, the OOB check result can be obtained from the OOB result of the L0 reference and the OOB result of the L1 reference:

[0159] If the inter-frame part is bidirectionally predicted,

[0160] P i,j Inter OOB = P i,j Inter,L0 OOB&P i,j Inter,L1 OOB

[0161] Otherwise if the inter part is bi-predicted (alternative method):

[0162] P i,j Inter OOB = P i,j Inter,L0 OOB|P i,j Inter,L1 OOB

[0163] F. OOB of CIIP codec block

[0164] In some embodiments, OOB checking is applied to both the inter-frame portion and the intra-frame portion of the CIIP, and then CIIP blending is applied to the results of the OOB checking according to the following:

[0165] If the inter part is unidirectional prediction (L0):

[0166] P i,j Inter OOB = P i,j Inter,L0 OOB

[0167] If the inter part is unidirectional prediction (L1):

[0168] P i,j Inter OOB = P i,j Inter,L1 OOB

[0169] If the inter part is bidirectionally predicted:

[0170] P i,j Inter OOB = P i,j Inter,L0 OOB&P i,j Inter,L1 OOB

[0171] Or if the inter part is bi-predicted (alternative method):

[0172] P i,j Inter OOB = P i,j Inter,L0 OOB|P i,j Inter,L1 OOB

[0173] Afterwards:

[0174] If P i,j Inter is OOB and P i,j Intra Non-OOB

[0175] P i,j final =P i,j Intra

[0176] Otherwise, if P i,j Inter is non-OOB and P i,j Intra OOB

[0177] P i,j final =P i,j Inter

[0178] otherwise

[0179] P i,j final =CIIP_blending(P i,j Inter ,P i,j Intra )

[0180] In some embodiments, the OOB check is not applied to the intra-frame part but only to the inter-frame part, that is, the OOB check result of the intra-frame part is always non-OOB. Then the CIIP mixing of the CIIP with one inter-frame part and one intra-frame part is performed according to the following:

[0181] If P i,j Inter OOB

[0182] P i,j final =P i,j Intra

[0183] otherwise

[0184] P i,j final =CIIP_blending(P i,j Inter ,P i,j Intra )

[0185] If a sample of the inter part is OOB, the corresponding sample of the intra part is used as the predictor, otherwise CIIP blending is applied to the samples of both CIIP parts.

[0186] G. OOB for Template Matching

[0187] In some embodiments, an OOB check is applied to a template region of a template match (TM) or ARMC-TM, and then the results of the OOB check are blended. For example, the video codec may perform motion vector refinement based on minimizing the TM matching cost between a reference template (e.g., reference template 630) adjacent to the reference block and a current template (e.g., current template 620) adjacent to the current block. The TM cost may be calculated based on an OOB check of the reference template and / or the current template. For example, OOB samples in the reference template will not be used to determine the TM cost.

[0188] OOB of H.OBMC

[0189] In some embodiments, the OOB check is applied to overlapped block motion compensation (OBMC), and then the additional inter prediction is mixed with the result of the OOB check. If the OBMC inter prediction signal is OOB, the OBMC inter prediction signal is not mixed with the original prediction signal. Otherwise, if the OBMC inter prediction signal is not OOB, the OBMC inter prediction signal is mixed with the original prediction signal. In some embodiments, if the additional inter prediction signal is bidirectional prediction, the OOB check result can be derived from the OOB result of the L0 prediction (or reference block) and the OOB result of the L1 prediction (or reference block) according to the following:

[0190] If the inter part is bidirectionally predicted:

[0191] P i,j Inter OOB = P i,j Inter,L0 OOB&P i,j Inter,L1 OOB

[0192] Otherwise if the inter part is bi-predicted (alternative method):

[0193] P i,j Inter OOB = P i,j Inter,L0 OOB|P i,j Inter,L1 OOB

[0194] I. OOB of MHP

[0195] In some embodiments, the video codec applies an OOB check to a multiple hypothesis prediction (MHP) and then adds the additional inter prediction signal / hypothesis to the result of the OOB check. If the additional inter prediction signal is OOB, the additional inter prediction signal is not added to the original prediction signal. Otherwise, if the additional inter prediction signal is not OOB, the additional inter prediction signal is added to the original prediction signal. In some embodiments, if the additional inter prediction signal is bidirectional prediction, the OOB check result can be derived from the OOB result of the L0 prediction (reference block) and the OOB result of the L1 prediction (reference block) according to the following:

[0196] If the inter-frame part is bidirectionally predicted:

[0197] P i,j Inter OOB = P i,j Inter,L0 OOB&P i,j Inter,L1OOB

[0198] Otherwise if the inter part is bi-predicted (alternative method):

[0199] P i,j Inter OOB = P i,j Inter,L0 OOB|P i,j Inter,L1 OOB

[0200] The aforementioned proposed method can be implemented in an encoder and / or a decoder. For example, the proposed method can be implemented in an inter-frame prediction module and / or an intra-frame block copy prediction module of an encoder and / or an inter-frame prediction module (and / or an intra-frame block copy prediction module) of a decoder.

[0201] IV. Example Video Encoder

[0202] Fig.13 An example video encoder 1300 that can implement predictive coding is labeled. As shown, the video encoder 1300 receives an input video signal from a video source 1305 and encodes the signal into a bitstream 1395. The video encoder 1300 has several components or modules for encoding the signal from the video source 1305, including at least some components selected from the following: a transform module 1310, a quantization module 1311, an inverse quantization module 1314, an inverse transform module 1315, an intra-picture estimation module 1320, an intra-frame prediction module 1325, a motion compensation module 1330, a motion estimation module 1335, a loop filter 1345, a reconstructed picture buffer 1350, an MV buffer 1365, an MV prediction module 1375, and an entropy encoder 1390. The motion compensation module 1330 and the motion estimation module 1335 are part of the inter-frame prediction module 1340.

[0203] In some embodiments, modules 1310-1390 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 1310-1390 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1310-1390 are shown as separate modules, some modules may be combined into a single module.

[0204] The video source 1305 provides a raw video signal that presents the pixel data of each video frame without compression. The subtractor 1308 calculates the difference between the raw video pixel data of the video source 1305 and the predicted pixel data 1313 from the motion compensation module 1330 or the intra-frame prediction module 1325. The transform module 1310 converts the difference (or residual pixel data or residual signal 1308) into transform coefficients (e.g., by performing a discrete cosine transform or DCT). The quantization module 1311 quantizes the transform coefficients into quantized data (or quantized coefficients) 1312, which are encoded into a bitstream 1395 by the entropy encoder 1390.

[0205] The inverse quantization module 1314 dequantizes the quantized data (or quantized coefficients) 1312 to obtain transform coefficients, and the inverse transform module 1315 performs an inverse transform on the transform coefficients to generate a reconstructed residual 1319. The reconstructed residual 1319 is added to the predicted pixel data 1313 to generate reconstructed pixel data 1317. In some embodiments, the reconstructed pixel data 1317 is temporarily stored in a line buffer (linebuffer not shown) for intra-picture prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 1345 and stored in the reconstructed picture buffer 1350. In some embodiments, the reconstructed picture buffer 1350 is a memory external to the video encoder 1300. In some embodiments, the reconstructed picture buffer 1350 is a memory internal to the video encoder 1300.

[0206] The intra-picture estimation module 1320 performs intra-prediction based on the reconstructed pixel data 1317 to generate intra-prediction data. The intra-prediction data is provided to the entropy encoder 1390 to be encoded into a bitstream 1395. The intra-prediction data is also used by the intra-prediction module 1325 to generate predicted pixel data 1313.

[0207] The motion estimation module 1335 performs inter-frame prediction by generating MVs to refer to pixel data of previously decoded frames stored in the reconstructed picture buffer 1350. These MVs are provided to the motion compensation module 1330 to generate predicted pixel data.

[0208] Instead of encoding the complete actual MV in the bitstream, the video encoder 1300 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 1395.

[0209] The MV prediction module 1375 generates a predicted MV based on a reference MV generated for encoding a previous video frame, i.e., a motion compensated MV for performing motion compensation. The MV prediction module 1375 obtains the reference MV from the previous video frame from the MV buffer 1365. The video encoder 1300 stores the MV generated for the current video frame in the MV buffer 1365 as a reference MV for generating a predicted MV.

[0210] The MV prediction module 1375 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame is encoded into the bitstream 1395 by the entropy encoder 1390.

[0211] The entropy encoder 1390 encodes various parameters and data into a bitstream 1395 by using an entropy coding technique such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 1390 encodes various header elements, flags along with quantized transform coefficients 1312 and residual motion data as syntax elements into the bitstream 1395. The bitstream 1395 is then stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0212] The loop filter 1345 performs filtering or smoothing operations on the reconstructed pixel data 1317 to reduce encoding and decoding artifacts, especially at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the loop filter 1345 include deblocking filtering (DBF), sample adaptive offset (SAO), and / or adaptive loop filtering (ALF).

[0213] Fig.14 Portions of a video encoder 1300 that implements out-of-bounds checking for predictive codecs are illustrated. Specifically, the figure illustrates components of a motion compensation module 1330 of the video encoder 1300.

[0214] As shown, the motion compensation module 1330 has a candidate selector 1410, which selects a block vector (including a motion vector and an intra prediction mode) from the MV buffer 1365 and the intra prediction module 1325. The selection is controlled by the motion estimation module 1335. The selected block vector is checked by the OOB check module 1430, which determines whether the reference block identified by the block vector is out of bounds. The selected block vector is also provided to the predictor generator module 1420.

[0215] The predictor generator module 1420 generates a predictor or prediction block for the current block by obtaining samples from the reconstructed picture buffer 1350 as the predicted pixel data 1313. The motion estimation module 1335 selects one or more prediction codec tools (e.g., bidirectional prediction, GPM, CIIP, MHP, intra prediction, IBC, etc.) for the current block, and the predictor generator module 1420 generates a predictor based on the selected prediction tool. The motion estimation module 1335 also provides the prediction tool selection to the entropy encoder 1390 for signaling in the bitstream 1395.

[0216] The predictor generator module 1420 also uses the OOB check result from the OOB check module 1430 to determine whether to exclude certain reference samples identified by the selected block vector when performing blending to generate the predictor. For example, if the current picture is an ERP picture and the reference block identified by the block vector has OOB samples, the predictor generator module 1420 can use the horizontal surrounding samples obtained from the reconstructed picture buffer 1650 for blending if the reference block is OOB at the left boundary or the right boundary, or exclude the OOB samples from blending if the reference block is OOB at the top or the bottom boundary. The predictor generator module 1420 can also use the OOB result to determine whether to exclude samples from the BM cost calculation or the TM cost calculation.

[0217] Fig.15 The process 1500 for predictive encoding of a pixel block using out-of-bounds checking is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 1300 perform the process 1500 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the encoder 1300 performs the process 1500.

[0218] The encoder receives (at block 1510) data of a current pixel block in a current picture to be encoded as a video. The encoder identifies (at block 1520) a first reference block in a first reference picture based on a first block vector of the current block. The encoder identifies (at block 1530) a second reference block in a second reference picture based on a second block vector of the current block. The first and second block vectors may be motion vectors. In some embodiments, the first reference picture or the second reference picture is a current picture, and the first block vector or the second block vector is an intra prediction direction or mode.

[0219] The encoder (at block 1540) performs an out-of-bounds (OOB) check on the first and second reference blocks. The encoder (at block 1550) generates a predictor for the current block based on the first and second reference blocks and based on the OOB check. In some embodiments, when the samples of the first reference block are OOB, bidirectional motion compensation is not applied to at least one of the horizontal direction and the vertical direction. In some embodiments, when the samples of the first reference block are OOB on the left or right boundary of the first reference picture, or when the samples of the first reference block are OOB on the top or bottom boundary of the first reference picture, bidirectional motion compensation is not applied. In some embodiments, when the samples of the first reference block are non-OOB samples, the corresponding prediction samples are generated by mixing the non-OOB samples of the first reference block with the corresponding non-OOB samples of the second reference block. When the samples of the first reference block are OOB samples, the corresponding prediction samples are generated using the corresponding non-OOB samples of the second reference block without mixing.

[0220] In some embodiments, the video may be a 360-degree video in an equirectangular projection (ERP) format. When a sample of a first reference block is OOB on a left or right boundary of a first reference picture, a corresponding prediction sub-sample is generated by blending horizontal surrounding samples from the first reference picture with corresponding non-OOB samples of a second reference block in a second reference picture. When a sample of the first reference block is OOB on a top or bottom boundary of the first reference picture, a corresponding prediction sub-sample is generated by using corresponding non-OOB samples from a second reference block without blending.

[0221] In some embodiments, the current block is divided into a plurality of geometric partitions, a first reference block is used to generate a first predictor for the first geometric partition and a second reference block is used to generate a second predictor for the second geometric partition. Corresponding predictor samples are generated by blending samples of the first and second predictors along a boundary between the first and second geometric partitions, wherein out-of-bounds samples of the first and second predictors are excluded from the blending.

[0222] In some embodiments, the generated predictor is a combined inter-intra prediction (CIIP), wherein the first reference block provides inter-prediction samples and the second reference block provides intra-prediction samples. In some embodiments, the current block is decoded using MHP, such that the encoder performs an additional OOB check on the third reference block, and further generates the predictor based on the third reference block and the additional OOB check.

[0223] In some embodiments, the first block vector is refined to minimize a calculated template matching cost between a reference template adjacent to the first reference block and a current template adjacent to the current block, wherein OOB samples of the reference template are excluded from the cost calculation. In some embodiments, the first and second block vectors are refined to minimize a bilateral matching cost based on the first and second reference blocks, wherein OOB samples in the first and second reference blocks are excluded from the cost calculation.

[0224] The encoder (at block 1560) encodes the current block using the generated predictor to produce a prediction residual.

[0225] V. Example Video Decoder

[0226] In some embodiments, the encoder may send (or generate) one or more syntax elements in a bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0227] Fig.16 An example video decoder 1600 that can implement predictive coding is labeled. As shown, the video decoder 1600 is an image decoding or video decoding circuit that receives a bitstream 1695 and decodes the contents of the bitstream into pixel data of a video frame for display. The video decoder 1600 has several components or modules for decoding the bitstream 1695, including components selected from the following: an inverse quantization module 1611, an inverse transform module 1610, an intra-frame prediction module 1625, a motion compensation module 1630, a loop filter 1645, a decoded picture buffer 1650, an MV buffer 1665, an MV prediction module 1675, and a parser 1690. The motion compensation module 1630 is part of the inter-frame prediction module 1640.

[0228] In some embodiments, modules 1610-1690 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1610-1690 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 1610-1690 are shown as separate modules, some modules may be combined into a single module.

[0229] The parser 1690 (or entropy decoder) receives the bitstream 1695 and performs initial parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1612. The parser 1690 decodes the bitstream 1695 using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman encoding.

[0230] The inverse quantization module 1611 dequantizes the quantized data (or quantized coefficients) 1612 to obtain transform coefficients, and the inverse transform module 1610 performs an inverse transform on the transform coefficients 1616 to generate a reconstructed residual signal 1619. The reconstructed residual signal 1619 is added to the predicted pixel data 1613 from the intra prediction module 1625 or the motion compensation module 1630 to generate decoded pixel data 1617. The decoded pixel data is filtered by the loop filter 1645 and stored in the decoded picture buffer 1650. In some embodiments, the decoded picture buffer 1650 is a memory external to the video decoder 1600. In some embodiments, the decoded picture buffer 1650 is a memory internal to the video decoder 1600.

[0231] The intra prediction module 1625 receives intra prediction data from the bitstream 1695 and, based thereon, generates predicted pixel data 1613 from decoded pixel data 1617 stored in the decoded picture buffer 1650. In some embodiments, the decoded pixel data 1617 is also stored in a line buffer (not shown) for intra picture prediction and spatial MV prediction.

[0232] In some embodiments, the contents of decoded picture buffer 1650 are used for display. Display device 1655 either obtains the contents of decoded picture buffer 1650 for direct display or obtains the contents of decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from decoded picture buffer 1650 by pixel transfer.

[0233] The motion compensation module 1630 generates predicted pixel data 1613 from the decoded pixel data 1617 stored in the decoded picture buffer 1650 according to motion compensated MVs (MC MVs). These motion compensated MVs are decoded by adding the residual motion data received from the bitstream 1695 to the predicted MVs received from the MV prediction module 1675.

[0234] The MV prediction module 1675 generates a predicted MV based on a reference MV generated for decoding a previous video frame (e.g., a motion compensated MV for performing motion compensation). The MV prediction module 1675 obtains the reference MV of the previous video frame from the MV buffer 1665. The video decoder 1600 stores the motion compensated MV generated for decoding the current video frame in the MV buffer 1665 as a reference MV for generating a predicted MV.

[0235] The loop filter 1645 performs filtering or smoothing operations on the decoded pixel data 1617 to reduce encoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the loop filter 1645 include deblocking filtering (DBF), sample adaptive offset (SAO), and / or adaptive filtering (ALF).

[0236] Fig.17 Portions of a video decoder 1600 implementing out-of-bounds checking for predictive codecs are illustrated. Specifically, the figure illustrates components of a motion compensation module 1630 of the video decoder 1600.

[0237] As shown, the motion compensation module 1630 has a candidate selector 1710, which selects a block vector (including a motion vector and an intra prediction mode) from the MV buffer 1665 and the intra prediction module 1625. The selection is provided by the entropy decoder 1690 based on syntax elements parsed from the bitstream 1695. The selected block vector is checked by the OOB check module 1730, which determines whether the reference block identified by the block vector is out of bounds. The selected block vector is also provided to the predictor generator module 1720.

[0238] The predictor generator module 1720 generates a predictor or a prediction block of the current block as predicted pixel data 1613 by obtaining samples from the decoded picture buffer 1650. The entropy decoder 1690 selects one or more prediction codec tools (e.g., bidirectional prediction, GPM, CIIP, MHP, intra prediction, IBC, etc.) for the current block, and the predictor generator module 1720 generates a predictor based on the selected prediction tool.

[0239] The predictor generator module 1720 also uses the OOB check result from the OOB check module 1730 to determine whether to exclude certain reference samples identified by the selected block vector when performing blending to generate the predictor. For example, if the current picture is an ERP picture and the reference block identified by the block vector has OOB samples, the predictor generator module 1720 can use the horizontal surrounding samples obtained from the decoded picture buffer 1650 for blending if the reference block is OOB at the left or right boundary, or exclude the OOB samples from blending if the reference block is OOB at the top or bottom boundary. The predictor generator module 1720 can also use the OOB result to determine whether to exclude samples from the BM cost calculation or the TM cost calculation.

[0240] Fig.18 The process 1800 of predictive coding for a pixel block using out-of-bounds checking is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the decoder 1600 perform the process 1800 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the decoder 1600 performs the process 1800.

[0241] The decoder receives (at block 1810) data of a current pixel block in a current picture to be decoded as a video. The decoder identifies (at block 1820) a first reference block in a first reference picture based on a first block vector of the current block. The decoder identifies (at block 1830) a second reference block in a second reference picture based on a second block vector of the current block. The first and second block vectors may be motion vectors. In some embodiments, the first reference picture or the second reference picture is a current picture, and the first block vector or the second block vector is an intra prediction direction or mode.

[0242] The decoder (at block 1840) performs an out-of-bounds (OOB) check on the first and second reference blocks. The decoder (at block 1850) generates a predictor for the current block based on the first and second reference blocks and based on the OOB check. In some embodiments, when the samples of the first reference block are OOB, bidirectional motion compensation is not applied to at least one of the horizontal direction and the vertical direction. In some embodiments, when the samples of the first reference block are OOB on the left or right boundary of the first reference picture, or when the samples of the first reference block are OOB on the top or bottom boundary of the first reference picture, bidirectional motion compensation is not applied. In some embodiments, when the samples of the first reference block are non-OOB samples, corresponding prediction samples are generated by mixing the non-OOB samples of the first reference block with corresponding non-OOB samples of the second reference block. When the samples of the first reference block are OOB samples, corresponding prediction samples are generated using corresponding non-OOB samples of the second reference block without mixing.

[0243] In some embodiments, the video may be a 360-degree video in an equirectangular projection (ERP) format. When a sample of a first reference block is OOB on a left or right boundary of a first reference picture, a corresponding prediction sample is generated by blending horizontal surrounding samples from the first reference picture with corresponding non-OOB samples of a second reference block in a second reference picture. When a sample of the first reference block is OOB on a top or bottom boundary of the first reference picture, a corresponding prediction subsample is generated by using corresponding non-OOB samples from a second reference block without blending.

[0244] In some embodiments, the current block is divided into a plurality of geometric partitions, a first reference block is used to generate a first predictor for the first geometric partition and a second reference block is used to generate a second predictor for the second geometric partition. Corresponding predictor samples are generated by blending samples of the first and second predictors along a boundary between the first and second geometric partitions, wherein out-of-bounds samples of the first and second predictors are excluded from the blending.

[0245] In some embodiments, the generated predictor is a combined inter-intra prediction (CIIP), wherein the first reference block provides inter-prediction samples and the second reference block provides intra-prediction samples. In some embodiments, the current block is encoded and decoded using MHP, so that the decoder performs an additional OOB check on the third reference block, and further generates the predictor based on the third reference block and the additional OOB check.

[0246] In some embodiments, the first block vector is refined to minimize a template matching cost calculated between a reference template adjacent to the first reference block and a current template adjacent to the current block, wherein OOB samples of the reference template are excluded from the cost calculation. In some embodiments, the first and second block vectors are refined to minimize a bilateral matching cost based on the first and second reference blocks, wherein OOB samples in the first and second reference blocks are excluded from the cost calculation.

[0247] The decoder reconstructs the current block (at block 1860) by using the generated predictor. The decoder may then provide the reconstructed current block for display as part of a reconstructed current picture.

[0248] VI. Example Electronic System

[0249] Many of the above features and applications are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, compact disc read-only memory (CD-ROM), flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted by wireless or wired connections.

[0250] In this specification, the term "software" is intended to include firmware residing in read-only memory or application programs stored in magnetic memory, which can be read into memory for processing by a processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that jointly implement the software inventions described herein are within the scope of this disclosure. In some embodiments, the software program, when installed to run on one or more electronic systems, defines one or more specific machine implementations that process and execute the operations of the software program.

[0251] Fig.19 An electronic system 1900 is conceptually shown for implementing some embodiments of the present disclosure. The electronic system 1900 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1900 includes a bus 1905, a processing unit 1910, a graphics processing unit (GPU) 1915, a system memory 1920, a network 1925, a read-only memory 1930, a permanent storage device 1935, an input device 1940, and an output device 1945.

[0252] Bus 1905 collectively represents all system, peripheral, and chipset buses that communicatively couple numerous internal devices of electronic system 1900. For example, bus 1905 communicatively couples processing unit 1910 with GPU 1915, read-only memory 1930, system memory 1920, and permanent storage 1935.

[0253] The processing unit 1910 obtains instructions to be executed and data to be processed from these various memory units in order to perform the processing of the present disclosure. In different embodiments, the processing unit can be a single processor or a multi-core processor. Some instructions are passed to and executed by the GPU 1915. The GPU 1915 can offload various calculations or supplement the image processing provided by the processing unit 1910.

[0254] Read-only-memory (ROM) 1930 stores static data and instructions used by processing unit 1910 and other modules of the electronic system. On the other hand, permanent storage device 1935 is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when electronic system 1900 is turned off. Some embodiments of the present disclosure use a large-capacity storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as permanent storage device 1935.

[0255] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1935, system memory 1920 is a read-write memory device. However, unlike permanent storage device 1935, system memory 1920 is a volatile read-write memory, such as random access memory. System memory 1920 stores some instructions and data used by the processor at runtime. In some embodiments, processing according to the present disclosure is stored in system memory 1920, permanent storage device 1935 and / or read-only memory 1930. For example, according to some embodiments of the present disclosure, various memory units include instructions for processing multimedia clips. From these various memory units, processing unit 1910 obtains instructions to be executed and data to be processed in order to perform processing of some embodiments.

[0256] The bus 1905 is also connected to an input device 1940 and an output device 1945. The input device 1940 enables a user to convey information and select commands to the electronic system. The input device 1940 includes an alphanumeric keyboard and a pointing device (also referred to as a "cursor control device"), a camera (e.g., a webcam), a microphone, or a similar device for receiving voice commands, etc. The output device 1945 displays images generated by the electronic system or outputs data. The output device 1945 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices used as input and output devices, such as a touch screen.

[0257] Finally, if Fig.19 As shown, bus 1905 also couples electronic system 1900 to a network 1925 via a network adapter (not shown). In this manner, the computer may be part of a computer network, such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network of multiple networks, such as the Internet. Any or all components of electronic system 1900 may be used in conjunction with the present disclosure.

[0258] Some embodiments include electronic components, such as microprocessors, storage devices, and memory, which store computer program instructions in machine-readable or computer-readable media (or otherwise referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, double-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer program or computer code may be a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code such as produced by a compiler, and files including high-level code that is executed by a computer, an electronic component, or a microprocessor using an interpreter.

[0259] Although the above discussion mainly involves microprocessors or multi-core processors executing software, many of the above features and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0260] As used in this specification and any claims of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or display refer to displaying on an electronic device. As used in this specification and any claims of this application, the terms "computer-readable medium," "computer-readable medium," and "machine-readable medium" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, and any other transient signals.

[0261] Although the present disclosure has been described with reference to many specific details, those skilled in the art will recognize that the present disclosure may be implemented in other specific forms without departing from the spirit of the present disclosure. Fig.15 and Fig.18 ) conceptually illustrates the processes. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the process may be implemented using several sub-processes or as part of a larger macro process. Therefore, it will be understood by those skilled in the art that the present disclosure is not limited by the foregoing illustrative details, but is defined by the appended claims.

[0262] Additional Notes

[0263] The subject matter described herein sometimes represents different components, which are contained in or connected to other different components. It is understood that the described structure is only an example, and in fact, it can be implemented by many other structures to achieve the same function. Conceptually, the arrangement of any components that achieve the same function is actually "associated" to achieve the desired function. Therefore, regardless of the structure or intermediate components, any two components combined to achieve a specific function are considered to be "interrelated" to achieve the desired function. Similarly, any two associated components are considered to be "operably connected" or "operably coupled" to achieve a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to achieve a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to achieve a specific function. Specific examples of operable connections include, but are not limited to, physically pairable and / or physically interacting components, and / or wirelessly interactive and / or wirelessly interactive components, and / or logically interactive and / or logically interactive components.

[0264] Furthermore, with respect to the use of substantially any plural and / or singular terms, those of ordinary skill in the art can translate from the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application. For clarity, the present invention expressly sets forth various singular / plural arrangements.

[0265] In addition, it will be understood by those of ordinary skill in the art that, in general, the terms used in the present invention, especially in the claims, and the subject matter of the claims, are generally used as "open" terms, for example, "including" should be interpreted as "including but not limited to", "having" should be interpreted as "at least having", "including" should be interpreted as "including but not limited to", etc. It will be further understood by those of ordinary skill in the art that if a specific number of claim contents are intended to be introduced, it will be explicitly indicated in the claim, and will not be displayed in the absence of such contents. For example, to aid understanding, the following claims may contain the phrases "at least one" and "one or more" to introduce the contents of the claims. However, the use of these phrases should not be understood to imply the use of the indefinite article "a" or "an" to introduce the claim contents and limit the scope of any particular patent. Even when the same claim includes the introductory phrases "one or more" or "at least one", the indefinite article, such as "a" or "an", should be interpreted to mean at least one or more, and the same is true for the use of the explicit description used to introduce the claim. In addition, even if the introductory content explicitly refers to a specific number, a person of ordinary skill in the art will recognize that such content should be interpreted as indicating the number of references, for example, "two references" without other modifications means at least two references, or two or more references. In addition, when a statement similar to "at least one of A, B, and C" is used, it is generally stated so that a person of ordinary skill in the art can understand the statement, for example, "a system includes at least one of A, B, and C" will include but is not limited to a system with A alone, a system with B alone, a system with C alone, a system with A and B, a system with A and C, a system with B and C, and / or a system with A, B, and C, etc. A person of ordinary skill in the art will further understand that any separated words and / or phrases represented by two or more alternative terms, whether in the specification, claims, or drawings, should be understood to include the possibility of one of these terms, one of them, or both of these terms. For example, "A or B" should be understood as the possibility of "A", or "B", or "A and B".

[0266] As can be seen from the foregoing, various embodiments have been described for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and application are indicated by the claims.

Claims

1. A video encoding and decoding method, comprising: Receiving data of a current block of a current picture to be encoded or decoded into a video; identifying a first reference block in a first reference picture based on a first block vector of the current block; identifying a second reference block in a second reference picture based on a second block vector of the current block; performing an out-of-bounds check (OOB) on the first and second reference blocks; generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied in at least one of a horizontal direction and a vertical direction; and The current block is encoded or decoded using the generated predictor.

2. The video encoding and decoding method according to claim 1, characterized in that: When samples of the first reference block are out-of-bounds on a left boundary or a right boundary of the first reference picture, generating corresponding prediction samples by mixing horizontal surrounding samples from the first reference picture with corresponding non-out-of-bounds samples of the second reference block in the second reference picture, When samples of the first reference block are out-of-bounds on a top or bottom boundary of the first reference picture, corresponding prediction samples are generated by using corresponding non-out-of-bounds samples from the second reference block without blending.

3. The video encoding and decoding method according to claim 2, characterized in that: The video is a 360-degree video in equirectangular projection (ERP) format.

4. The video encoding and decoding method according to claim 1, characterized in that: When samples of the first reference block are out-of-bounds on a left boundary or a right boundary of the first reference picture, or when samples of the first reference block are out-of-bounds on a top boundary or a bottom boundary of the first reference picture, the bidirectional motion compensation is not applied.

5. The video encoding and decoding method according to claim 1, characterized in that: The first reference picture or the second reference picture is the current picture.

6. The video encoding and decoding method according to claim 5, characterized in that: The first block vector or the second block vector is an intra prediction direction.

7. The video encoding and decoding method according to claim 1, characterized in that: The current block includes a plurality of geometric partitions, The first reference block is used to generate a first predictor for the first geometric partition, The second reference block is used to generate a second predictor for the second geometric partition, The corresponding predicted sub-samples are generated by blending the samples of the first and second predicted sub-samples along the boundary between the first and second geometric partitions, wherein out-of-bounds samples of the first and second predicted sub-samples are excluded from the blending.

8. The video encoding and decoding method according to claim 1, characterized in that: The generated predictor is a combined inter-intra prediction (CIIP), where the first reference block provides inter-prediction samples and the second reference block provides intra-prediction samples.

9. The video encoding and decoding method according to claim 1, characterized in that: Also included is performing an additional out-of-bounds check on a third reference block, wherein the predictor is generated further based on the third reference block and the additional out-of-bounds check.

10. The video encoding and decoding method according to claim 1, characterized in that: The first block vector is refined to minimize a template matching cost calculated between a reference template adjacent to the first reference block and a current template adjacent to the current block, wherein out-of-bounds samples of the reference template are excluded from the cost calculation.

11. The video encoding and decoding method according to claim 1, characterized in that: The first and second block vectors are refined based on the first and second reference blocks to minimize a bilateral matching cost, wherein out-of-bounds samples in the first and second reference blocks are excluded from the cost calculation.

12. An electronic device comprising: A video encoder circuit configured to perform operations including: Receiving data of a current block of a current picture to be encoded or decoded into a video; identifying a first reference block in a first reference picture based on a first block vector of the current block; identifying a second reference block in a second reference picture based on a second block vector of the current block; performing an out-of-bounds check (OOB) on the first and second reference blocks; generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied in at least one of a horizontal direction and a vertical direction; and The current block is encoded or decoded using the generated predictor.

13. A video decoding method, comprising: Receiving data of a current block of a current picture to be decoded into a video; identifying a first reference block in a first reference picture based on a first block vector of the current block; identifying a second reference block in a second reference picture based on a second block vector of the current block; performing an out-of-bounds check (OOB) on the first and second reference blocks; generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied in at least one of a horizontal direction and a vertical direction; and The current block is reconstructed using the generated predictor.

Images