Multi-reference line based intra prediction

Abstract

Provided is a method for performing intra prediction of a block of pixels using a plurality of reference lines. A video codec receives data for a block of pixels to be encoded or decoded as a current block of a current picture of a video. A video codec receives or signals a first reference line and a second reference line selected from among a plurality of reference lines adjacent to the current block. A video codec mixes samples of a first reference line and a second reference line, each mixed sample being generated by combining samples in the first reference line with samples in the second reference line identified according to an intra prediction mode. A video codec generates a prediction of a current block in an intra prediction mode by using mixed samples. The video codec encodes or decodes the current block by using the generated prediction.

Description

[0001] Cross-references

[0002] The present invention is a non-provisional application and claims priority to U.S. Provisional Patent Application Nos. 63 / 369,773 and 63 / 380,395, filed on July 29, 2022 and October 21, 2022, respectively. The entire contents of the U.S. Provisional Patent Application are incorporated herein by reference. Technical Field

[0003] The present invention relates to a video coding and decoding system, and in particular to a method for coding and decoding pixel blocks by intra-frame prediction. Background Art

[0004] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0005] High Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaboration Group on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion compensation categorical DCT transform coding architecture. The basic unit of compression is called a coding unit (CU), which is a 2Nx2N pixel block. Each CU can be recursively split into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile Video Coding (VVC) is the latest international video coding standard developed by ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 Joint Video Experts Group (JVET). The input video signal is predicted from a reconstructed signal derived from a coded picture region. The prediction residual signal is processed by block transform. The transform coefficients are quantized and entropy coded along with other side information in the bitstream. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal after inverse transforming the dequantized transform coefficients. The reconstructed signal is further processed by loop filtering to remove coding artifacts. The decoded pictures are stored in a frame buffer and used to predict future pictures in the input video signal.

[0007] In VVC, a coded picture is divided into non-overlapping square block regions represented by associated coding tree units (CTUs). The leaf nodes of the coding tree correspond to coding units (CUs). A coded picture can be represented by a set of slices, each slice containing an integer number of CTUs. Each CTU in a slice is processed in raster scan order. Bidirectionally predicted (B) slices can be decoded using intra prediction or inter prediction, with up to two motion vectors and reference indices to predict the sample values ​​of each block. Predicted (P) slices are decoded using intra prediction or inter prediction with up to one motion vector and reference index to predict the sample values ​​of each block. Intra (I) slices are decoded using only intra prediction.

[0008] A CTU can be divided into one or more non-overlapping coding units (CUs) using a quadtree (QT) with a nested multi-type tree (MTT) structure to accommodate various local motion and texture characteristics. A CU can be further split into smaller CUs using one of five partitioning types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning.

[0009] Each CU contains one or more prediction units (PUs). The prediction unit, together with the associated CU syntax, serves as the basic unit for sending prediction information. A specified prediction process is used to predict the values ​​of the relevant pixel samples within the PU. Each CU may contain one or more transform units (TUs) for representing prediction residual blocks. A transform unit (TU) consists of a transform block (TB) of luma samples and two corresponding chroma sample TBs, each TB corresponding to a residual block of samples from one color component. An integer transform is applied to the transform block. The values ​​of the quantized coefficients are entropy encoded in the bitstream along with other auxiliary information. The terms coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are defined as 2-dimensional sample arrays that specify one color component associated with a CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU consists of one luma CTB, two chroma CTBs, and related syntax elements. Similar relationships are valid for CUs, PUs, and TUs.

[0010] For each inter-predicted CU, motion parameters consisting of motion vector, reference picture index and reference picture list usage index, as well as additional information, are used for inter-prediction sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is encoded using skip mode, this CU is associated with one PU and has no significant residual coefficients, no encoded motion vector increments or reference picture indices. A merge mode is specified, whereby the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates and additional scheduling introduced in VVC. Merge mode can be applied to any inter-predicted CU. An alternative to merge mode is the explicit transmission of motion parameters, where motion vectors, corresponding reference picture indices for each reference picture list and reference picture list usage flags, as well as other required information, are explicitly signaled for each CU. Summary of the invention

[0011] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious technologies described herein. Selected but not all embodiments are further described in the detailed description below. Therefore, the following summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

[0012] A method for performing intra-frame prediction of a pixel block using multiple reference lines is provided. A video codec receives data of a pixel block of a current block of a current picture to be encoded or decoded as a video. The video codec receives or signals a first reference line and a second reference line selected from a plurality of reference lines adjacent to the current block. The video codec mixes samples of the first reference line and the second reference line, each mixed sample being generated by combining a sample in the first reference line with a sample in the second reference line identified according to an intra-frame prediction mode. The video codec generates a prediction of the current block in the intra-frame prediction mode using the mixed samples. The video codec encodes or decodes the current block using the generated prediction.

[0013] In some embodiments, when the intra-frame prediction mode is an intra-frame prediction direction, the position of the sample to be mixed in the first reference line and the position of the sample to be mixed in the second reference line follow the following direction: the intra-frame prediction mode; when the intra-frame prediction mode is not an intra-frame prediction direction (for example, DC or planar), the position of the sample to be mixed in the first reference line and the position of the sample to be mixed in the second reference line are the same.

[0014] In some embodiments, the samples in the first reference line that are blended may be samples of a top reference line portion or a left reference line portion of the first reference line. The top reference line portion of the first reference line may be identified by a first reference line index, and the top reference line portion of the second reference line may be identified by a second reference line index. Additionally, the left reference line portion of the first reference line may be identified by the first reference line index, and the left reference line portion of the second reference line may be identified by the second reference line index. In some embodiments, the first reference line index is explicitly signaled, and the second reference line index is signaled by an offset relative to the explicitly signaled first line index.

[0015] In some embodiments, when the mixed samples are located at non-integer positions (e.g., due to intra-prediction directions), the encoder may apply an interpolation filter (e.g., a 4-tap filter) to the mixed samples. In some embodiments, when the samples are located at non-integer positions, the encoder applies a first interpolation filter to the samples in the first reference line to be mixed (the samples to be mixed). When the samples are located at non-integer positions, the encoder may also apply a second interpolation filter to the samples in the second reference line to be mixed. The first and second interpolation filters may be the same or different (e.g., in terms of filter taps).

[0016] In some embodiments, when a specific tool for intra prediction using samples from multiple reference lines is explicitly marked to be enabled, a prediction is generated by using mixed samples. In some embodiments, a prediction is generated by using mixed samples only when a specific tool for using samples from multiple reference lines is explicitly marked to be enabled. In some embodiments, when a difference between samples of a first reference line and a second reference line used to create the mixed sample fails to meet a specific criterion (e.g., a difference ≥ a threshold or < a threshold), the mixed sample is not used to generate a prediction for the current block. In some embodiments, whether to mix samples of the first and second reference lines is determined based on an implicit indication based on, for example, a block width, height, or area of ​​the current block. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of the present disclosure. The accompanying drawings illustrate embodiments of the present disclosure and, together with the description, are used to explain the principles of the present disclosure. It is worth noting that the accompanying drawings are not necessarily drawn to scale, because in order to clearly illustrate the concepts of the present disclosure, some components may be shown as being out of proportion to the sizes in actual implementation.

[0018] Figure 1 Different directional intra prediction modes are shown.

[0019] 2A-B conceptually illustrate top and left reference templates with extended lengths for supporting wide angle directional modes for non-square blocks of different aspect ratios.

[0020] Figure 3 Reference lines of a current block that may be used for intra prediction of the current block are conceptually illustrated.

[0021] Figure 4 The top and left reference line portions are shown providing samples to be mixed for intra prediction.

[0022] Figure 5 Blending of reference lines based on the intra prediction mode of the current block is shown.

[0023] Figure 6 An example video encoder 600 implementing intra prediction is labeled.

[0024] Figure 7 The portion of the video encoder 600 that implements intra prediction by blending multiple reference lines is labeled.

[0025] Figure 8 The process of performing intra prediction by blending multiple reference lines is conceptually illustrated.

[0026] Fig. 9 An example video decoder implementing intra prediction is indicated.

[0027] Fig.10 Shown is a portion of a video decoder that implements intra prediction by blending multiple reference lines.

[0028] Fig.11 The process of performing intra prediction by blending multiple reference lines is conceptually illustrated.

[0029] Fig.12 An electronic system for implementing some embodiments of the present disclosure is conceptually illustrated. DETAILED DESCRIPTION

[0030] In the detailed description below, many specific details are set forth by way of example to provide a thorough understanding of the relevant teachings. Any variation, derivation and / or extension based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, processes, components and / or circuits related to one or more example implementations disclosed herein may be described at a relatively high level without detail to avoid unnecessarily obscuring aspects of the teachings of the present disclosure.

[0031] Intra prediction mode

[0032] The intra prediction method uses a reference unit adjacent to the current prediction unit (PU) and one of the intra prediction modes to generate a predictor for the current PU. The intra prediction direction can be selected from a mode set containing multiple prediction directions. For each PU coded via intra prediction, an index is used and encoded to select one of the intra prediction modes. The corresponding prediction is generated and the residual can then be derived and transformed.

[0033] Figure 1 Intra-frame prediction modes of different directions are shown. These intra-frame prediction modes are called directional modes, excluding DC mode or planar mode. As shown in the figure, there are 33 directional modes (V: vertical direction; H: horizontal direction), so H, H+1~H+8, H-1~H-7, V, V+1~V+8, V-1~V-8 are used. In general, the directional mode can be expressed as H+k or V+k mode, where k=±1, ±2,...,±8. Each of such intra-frame prediction modes can also be called an intra-frame prediction angle. In order to capture arbitrary edge directions presented in natural pictures, the number of directional intra-frame modes can be extended from the 33 directional modes used in HEVC to 65 directional modes, so that k ranges from ±1 to ±16. These denser directional intra-frame prediction modes are applicable to all block sizes and luminance and chrominance intra-frame prediction. By including DC and planar modes, the number of intra-frame prediction modes is 35 (or 67).

[0034] Of the 35 (or 67) intra prediction modes, some modes (e.g., 3 or 5) are identified as the most probable mode (MPM) set for intra prediction in the current prediction block. The encoder can reduce the bit rate by signaling an index for selecting one of the MPMs instead of an index for selecting one of the 35 (or 67) intra prediction modes. For example, the intra prediction mode used in the left prediction block and the intra prediction mode used in the above prediction block are used as MPMs. When the intra prediction modes of two neighboring blocks use the same intra prediction mode, the intra prediction mode can be used as an MPM. When only one of the two neighboring blocks is available and coded in a directional mode, the two neighboring directions immediately adjacent to the directional mode can be used as MPMs. DC mode and Planar mode are also considered as MPMs to fill in available spots in the MPM set, especially if the above or top neighboring block is not available or is not coded in intra prediction, or if the intra prediction mode in the neighboring block is not a directional mode. If the intra prediction mode of the current prediction block is one of the modes in the MPM set, 1 or 2 bits are used to signal which one it is. Otherwise, the intra prediction mode of the current block is different from any entry in the MPM set, and the current block will be encoded as non-MPM mode. There are 32 such non-MPM modes in total, and a (5-bit) fixed length coding method is applied to signal the mode.

[0035] The MPM list is constructed based on the intra mode of the upper left neighboring block. Assuming that the mode of the left neighboring block is represented as Left and the mode of the upper neighboring block is represented as Above, the unified MPM list can be constructed as follows:

[0036] – When neighboring blocks are not available, their intra mode defaults to Planar.

[0037] – If both “Left” and “Above” modes are non-angle modes:

[0038] MPM List → {plane, DC, V, H, V-4, V+4}

[0039] – If one of the “Left” and “Above” modes is an angle mode and the other is a non-angle mode:

[0040] Sets the Max mode to the larger of Left and Above.

[0041] MPM list → {plane, Max, Max-1, Max+1, Max––2, Max+2}

[0042] – If Left and Above are both angled and they are different:

[0043] Sets the Max mode to the larger of Left and Above.

[0044] Set mode "Min" to the smaller of "Left" and "Above"

[0045] If Max–Min equals 1:

[0046] – MPM list → {Plane, Left, Above, Min–1, Max+1, Min–2}

[0047] Otherwise, if Max–Min is greater than or equal to 62:

[0048] – MPM list → {Plane, Left, Above, Min+1, Max–1, Min+2}

[0049] Otherwise, if Max–Min equals 2:

[0050] –MPM list → {Plane, Left, Above, Min+1, Min–1, Max+1}

[0051] otherwise:

[0052] – MPM list → {Plane, Left, Above, Min–1, Min+1, Max–1}

[0053] – If Left and Above are both angled and they are the same:

[0054] MPM List → {Plane, Left, Left-1, Left+1, Left-2, Left+2}.

[0055] The traditional angular intra prediction direction is defined as clockwise from 45 degrees to -135 degrees. In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes. The replaced mode is represented by the original mode index, which is remapped to the index of the wide-angle mode after parsing.

[0056] For some embodiments, the total number of intra prediction modes remains unchanged, i.e., 67, and the intra mode encoding method remains unchanged. To support these prediction directions, a top reference template of length 2W+1 and a left reference template of length 2H+1 are defined. FIGS. 2A-B conceptually illustrate top and left reference templates with extended lengths to support wide-angle direction modes for non-square blocks of different aspect ratios.

[0057] The number of replacement modes in the wide-angle direction mode depends on the aspect ratio of the block. The replaced intra prediction modes for different blocks with different aspect ratios are shown in Table 1 below.

[0058] Table 1: Wide-angle mode replaces intra-prediction mode

[0059]

[0060] Reference sample smoothing using interpolation filters

[0061] In some embodiments, a four-tap intra-frame interpolation filter can be used to improve the accuracy of directional intra-frame prediction (i.e., excluding planar and DC predictors). In HEVC, a two-tap linear interpolation filter has been used to generate intra-frame prediction blocks for directional prediction mode. In VVC, two sets of 4-tap IFs replace the low-precision linear interpolation in HEVC, one set is a DCT-based interpolation filter (DCTIF) and the other set is a 4-tap smoothing interpolation filter (SIF). The DCTIF is constructed in the same way as the chroma component motion compensation in HEVC and VVC. SIF is obtained by convolving a 2-tap linear interpolation filter with a [1 2 1] / 4 filter.

[0062] Depending on the intra prediction mode, the following reference sample processing can be performed. Directional intra prediction modes fall into one of the following groups:

[0063] Group A: vertical or horizontal mode (HOR_IDX, VER_IDX),

[0064] Group B: Directional modes and plane modes representing non-fractional angles (-14, -12, -10, -6, 2, 34, 66, 72, 76, 78, 80),

[0065] Group C: Remaining direction patterns.

[0066] If the directional intra prediction mode is classified as belonging to group A, no filter is applied to the reference samples to produce the prediction samples. Otherwise, if the mode belongs to group B and the mode is a directional mode, and certain conditions are all true (for example, refIdx equals 0 or no MRL, TU size greater than 32, Luma, no ISP block), a [1,2,1] reference sample filter (depending on the MDIS condition) can be applied to the reference samples to further copy these filter values ​​into the intra predictor according to the selected direction, but no interpolation filter is applied. Otherwise, if the mode is classified as belonging to group C, the MRL index is equal to 0, and the current block is not an ISP block, only the intra reference sample interpolation filter is applied to the reference samples to produce prediction samples that fall into fractional or integer positions between the reference samples according to the selected direction (no reference sample filtering is performed). The interpolation filter type is determined as follows:

[0067] – Set minDistVerHor equal to Min(Abs(predModeIntra-50),Abs(predModeIntra-18))

[0068] – Set nTbS equal to (Log2(W)+Log2(H))>>1

[0069] – Set intraHorVerDistThres[nTbS] as specified below:

[0070]

[0071] – If minDistVerHor is greater than intraHorVerDistThres[nTbS], use SIF for interpolation

[0072] – Otherwise, use DCTIF for interpolation

[0073] Intra prediction based on additional reference lines

[0074] When generating intra prediction of the current block, the video codec may use adjacent reconstructed samples of the current block located at a portion of an adjacent top reference line (top line 0) and / or a portion of an adjacent left reference line (left line 0), or samples located at a non-adjacent top reference line (e.g., top line n, where n>0) and a non-adjacent left reference line (e.g., left line m, where m>0 and m=n) as reference samples. In some embodiments of the present invention, the video codec uses more than one reference line (which may include more than one left reference line segment and / or more than one top reference line segment) to generate prediction samples of the current block.

[0075] Figure 3 The reference lines of the current block that can be used for intra prediction of the current block are conceptually shown. This figure shows the current block 300 and samples adjacent to the current block 300. The top and right adjacent samples are used as reference samples for intra prediction. The reference samples are grouped into multiple reference lines 0 to N. Reference line 0 is the reference line closest to the current block and is adjacent to the current block 300. Reference lines 1 to N are non-adjacent reference lines. Each reference line includes a top reference line portion and a left reference line portion. When a reference line is used for intra prediction, the reference samples in the reference line (whether in the top or left reference line portion) are used as references to generate prediction samples.

[0076] In some embodiments, whether to use more than one reference line to generate intra-prediction samples is determined by an explicit indication encoded in the bitstream. In some embodiments, the explicit indication is encoded using one or more contexts. In some embodiments, multiple contexts are used and the context selection depends on the width, height, or area of ​​the block. If the area of ​​the current block is less than a predefined threshold, one context is used, otherwise another context is used. In some embodiments, multiple contexts are used and the context selection depends on neighboring mode information. In some embodiments, if both the top and left neighboring blocks are encoded and decoded using intra-frame prediction, one context is used. Otherwise, if only one of the top and left neighboring blocks is encoded using intra-frame prediction, another context is used.

[0077] In some embodiments, in some cases, an explicit indication of whether to use multiple reference lines for intra prediction is not signaled / parsed and is inferred to prohibit the use of more than one reference line to generate intra prediction samples. For example, in some embodiments, the video codec prohibits the use of multiple top and left reference lines for intra prediction when the area of ​​the current block is smaller than a predefined threshold, or when the current block has a longer side greater than a predetermined threshold, or when the long side of the current block is smaller than a predefined threshold multiplied by the short side.

[0078] In some embodiments, whether to use multiple reference lines to generate intra-frame prediction samples is determined based on an implicit indication. In some embodiments, the implicit indication depends on the block width, height, or area (of the current block). For example, if the area of ​​the current block is greater than a predefined threshold, the video codec may use multiple top and left reference lines for generating intra-frame predictions for the current block, otherwise the video codec does not use multiple reference lines to generate predictions. In some embodiments, if most of the neighboring blocks of the current block (e.g., the top and left neighboring blocks) are encoded and decoded using inter-frame predictions, the video codec may use multiple reference lines to generate intra-frame predictions for the current block, otherwise the video codec does not use multiple reference lines to generate intra-frame predictions to encode and decode the current block.

[0079] In some embodiments, the implicit indication depends on the intra prediction of the current block. If the intra prediction mode of the current block produces non-integer prediction samples (e.g., samples at non-integer positions), the video codec can use multiple reference lines for intra prediction. Otherwise, multiple reference lines are not used to generate the prediction of the current block. When the intra prediction mode is not directional (e.g., DC or planar), the position of the sample to be mixed in the first reference line and the position of the corresponding sample to be mixed in the second reference line are the same.

[0080] In some embodiments, if one or more prediction samples fall within a directional reference sample according to an intra-prediction mode, the video codec may apply an interpolation filter to perform intra-prediction. In some embodiments, if the reference samples in the first and / or second reference lines used are not located at integer positions, the video codec may apply an interpolation filter (rather than nearest neighbor rounding) to generate the samples to be mixed by using spatially adjacent samples (e.g., in the first and second reference lines used). The interpolation filters used for the first reference line used and the second reference line used may be the same or different. For example, the tap length of the first reference line used may be greater than (or less than) the tap length of the second reference line used.

[0081] In some embodiments, when the implicit indication specifies that multiple reference lines are not used to generate intra prediction, only one reference line is used (which may include a top reference line portion and a left reference line portion). In some embodiments, the video codec always uses multiple reference lines for intra prediction. In some embodiments, whether to use a single reference line or multiple reference lines for intra prediction is determined based on block-level syntax control or high-level syntax control (e.g., SPS, PPS, PH, SH, or CTU flags).

[0082] In some embodiments, when multiple reference lines are used for intra-frame prediction of the current block, the reference samples used to generate the intra-frame prediction include: (i) a mixed sample of the first top reference line portion used (denoted as top line n1) and the second top reference line portion used (denoted as top line n2), and (ii) a mixed sample of the first left reference line portion used (denoted as left line m1) and the second left reference line portion used (denoted as left line m2). Figure 4 It is shown that top and left reference line portions to be mixed for intra prediction are provided, including top reference line portions n1 and n2 and left reference line portions m1 and m2. In some embodiments, n1 and m1 may belong to the same or different reference lines, and n2 and m2 may belong to the same or different reference lines. In some embodiments, n1 is different from n2, and m1 is different from m2. Top reference line portions n1 and n2 may belong to reference lines that are adjacent or non-adjacent to each other. Left reference line portions m1 and m2 may belong to reference lines that are adjacent or non-adjacent to each other. Reference line segments n1, n2, m1, m2 may or may not belong to reference lines adjacent to the current block.

[0083] In some embodiments, an explicit indication is used to jointly determine n1 and m1, and an implicit indication is used to determine n2 and m2 based on the determined n1 and m1. (Assume that n1 and m1 belong to the same reference line, and n2 and m2 belong to the same reference line.) The explicit indication of n1 and m1 can be the reference line index of the top reference line 0 and the left reference line 0, or the explicit indication of n1 and m1 can be a reference line index that refers to the upper reference line that is not adjacent to the upper boundary of the current block and the left reference line that is not adjacent to the left boundary of the current block. For example, assuming that n2 is inferred to be an adjacent row of n1, then index (n2) = index (n1) ± 1. For another example, assuming that m2 is inferred to be an adjacent row of m1, then index (m2) = index (m1) ± 1. For another example, assuming that n2 is inferred to be a non-adjacent row of n1, then index (n2) = index (n1) ± k, where k is predefined and k>1. As another example, assuming that m2 is inferred to be a non-adjacent row of m1, then index (m2) = index (m1) ± k, where k is predefined and k>1.

[0084] In some embodiments, an explicit indication is used to jointly determine n1 and m1, and another explicit indication is used to jointly determine n2 and m2. (Assume that n1 and m1 are on the same reference line; n2 and m2 are on the same reference line.) The explicit indication of n1 and m1 may refer to the top reference line 0 and the left reference line 0, or the explicit indication of n1 and m1 may refer to the top reference line that is not adjacent to the top boundary of the current block and the left reference line that is not adjacent to the left boundary of the current block. For example, the explicit indication of n2 and m2 (denoted as index 2) may depend on the explicit indication of n1 and m1 (denoted as index 1) (assuming that the maximum number of reference lines is 3), then index (n1) = index1 (ranging from 0 to 2), and index (n2) = index 2 (ranging from 0 to 1) + ((index 2 ≥ index (n1))? 1:0). As another example, the explicit indication of n2 and m2 refers to the difference between the first reference line used and the second reference line used, such that index (n2) = index (n1) ± difference, or index (m2) = index (m1) ± difference.

[0085] In some embodiments, when the first reference line used and the second reference line used are mixed, the weights (w1, w2) of the first and second reference lines used are applied. For example, a uniform w1 is applied to each sample in the first reference line used, and a uniform w2 is applied to each sample in the second reference line used. For another example, the weights (w1, w2) are fixed to apply (1.5, 0.5) or (3, 1)>>2 after mixing. For another example, the weights (w1, w2) depend on the block width, height, area, or neighboring pattern information.

[0086] In some embodiments, a blending process is applied to blend each sample in the first used reference line with its collocated sample in the second used reference line. In some embodiments, when blending the first used reference line and the second used reference line, the intra prediction mode of the current block is taken into account. For example, if the intra prediction mode is directional, the position of the sample to be blended in the first used reference line and the position of the corresponding sample to be blended in the second used reference line follows the direction of the intra prediction mode. Figure 5 The reference lines are shown to be blended based on the intra prediction mode of the current block. In the figure, reference sample 510 (s1 of reference line 1) will be blended with reference sample 520 (s2 of reference line 2) at a position identified according to the intra prediction angle or direction.

[0087] In some embodiments, when the intra prediction mode is not directional (e.g., DC or planar), the position of the samples to be mixed in the first reference line used and the collocation position of the samples to be mixed in the second reference line used are the same. In some embodiments, regardless of the intra prediction mode of the current block, the position of the samples to be mixed in the first reference line used and the collocation position of the samples to be mixed in the second reference line used are the same.

[0088] In some embodiments, blending of the used first reference line and the used second reference line is applied before reference sample smoothing is performed on each used reference line. The video codec may also apply reference sample smoothing to the blended reference samples or skip reference sample smoothing. In some embodiments, the video codec performs blending of the used first reference line and the used second reference line after reference sample smoothing is performed on each used reference line.

[0089] In some embodiments, the mixed reference samples are stored in an additional memory buffer (different from the buffer storing the first and second used reference lines) for generating intra predictions of the current block. In some embodiments, the mixed reference samples are stored in an existing buffer for the first or second used reference line for generating intra predictions. That is, if the mixed reference samples are stored in an existing buffer for the first used reference line, the mixing process can be viewed as adding samples from the second used reference line to samples from the first used reference line.

[0090] In some embodiments, the process for mixing the first and second reference lines is not performed before generating the intra-frame prediction, but is performed when generating the intra-frame prediction. Therefore, there is no need to pre-search a buffer to place the mixed reference samples. That is, when generating each prediction sample of the current block, the video program code not only accesses the reference samples in the first reference line used, but also accesses the reference samples in the second reference line used. A weighted average is applied when mixing the reference samples of the two reference lines. The mixed sample is then used as the prediction sample of the current block.

[0091] In some embodiments, before applying the blending process to the sample in the first reference line used (s1) and the collocated sample in the second reference line used (s2), a predefined criterion is checked. Only if the criterion is met, the video codec may apply the blending process to blend s1 and s2. This criterion may be that the difference between s1 and s2 is less than a predefined threshold, or that the difference between s1 and s2 is greater than a predefined threshold.

[0092] In some embodiments, using multiple reference lines for intra prediction is not limited to using only two reference lines (a first reference line used and a second reference line used) and can be extended to use more than two reference lines.

[0093] The methods proposed in the present disclosure may enable and / or disable levels according to implicit rules (e.g., block width, height, or area) or according to explicit rules (e.g., syntax about blocks, tiles, slices, pictures, SPS, or PPS). The term "block" in the present disclosure may refer to TU / TB, CU / CB, PU / PB, predefined area, or CTU / CTB. Any combination of the methods proposed in the present disclosure may be applied.

[0094] Any of the aforementioned proposed methods may be implemented in an encoder and / or a decoder. For example, any of the proposed methods may be implemented in an inter-frame / intra-frame / prediction module of an encoder and / or an inter-frame / intra-frame / prediction module of a decoder. Alternatively, any of the proposed methods may be implemented as a circuit coupled to an inter-frame / intra-frame / prediction module of an encoder and / or an inter-frame / intra-frame / prediction module of a decoder to provide information required by the inter-frame / intra-frame / prediction module.

[0095] II. Example Video Encoder

[0096] Figure 6 An example video encoder 600 that performs inter-frame prediction is illustrated. As shown, the video encoder 600 receives a video source 605 input video signal and encodes the signal into a bitstream 695. The video encoder 600 has several components or modules for encoding the signal from the video source 605, including at least some components selected from the following: a transform module 610, a quantization module 611, an inverse quantization module 614, an inverse transform module 615, an intra-frame estimation module 620, an intra-frame prediction module 625, a motion compensation module 630, a motion estimation module 635, a loop filter 645, a reconstructed picture buffer 650, an MV buffer 665, an MV prediction module 675, and an entropy encoder 690. The motion compensation module 630 and the motion estimation module 635 are part of the inter-frame prediction module 640.

[0097] In some embodiments, modules 610-690 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 610-690 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 610-690 are shown as separate modules, some modules can be combined into a single module.

[0098] The video source 605 provides an uncompressed raw video signal that presents pixel data for each video frame. A subtractor 608 calculates the difference between the raw video pixel data of the video source 605 and the predicted pixel data 613 from the motion compensation module 630 or the intra-frame prediction module 625 as a prediction residual 609. The transform module 610 converts the difference (or residual pixel data or residual signal 608) into transform coefficients (e.g., by performing a discrete cosine transform or DCT). The quantization module 611 quantizes the transform coefficients into quantized data (or quantized coefficients) 612, which are encoded into a bitstream 695 by an entropy encoder 690.

[0099] The inverse quantization module 614 inversely quantizes the quantized data (or quantized coefficients) 612 to obtain transform coefficients, and the inverse transform module 615 inversely transforms the transform coefficients to generate a reconstructed residual 619. The reconstructed residual 619 is added to the predicted pixel data 613 to generate reconstructed pixel data 617. In some embodiments, the reconstructed pixel data 617 is temporarily stored in a line buffer (not shown) for intra-frame prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 645 and stored in the reconstructed picture buffer 650. In some embodiments, the reconstructed picture buffer 650 is a memory external to the video encoder 600. In some embodiments, the reconstructed picture buffer 650 is a memory internal to the video encoder 600.

[0100] The intra estimation module 620 performs intra prediction based on the reconstructed pixel data 617 to generate intra prediction data. The intra prediction data is provided to the entropy encoder 690 to be encoded into a bitstream 695. The intra prediction data is also used by the intra prediction module 625 to generate predicted pixel data 613.

[0101] The motion estimation module 635 performs inter-frame prediction by generating MVs to refer to pixel data of a previously decoded frame stored in the reconstructed picture buffer 650. These MVs are provided to the motion compensation module 630 to generate predicted pixel data.

[0102] Instead of encoding the complete actual MV in the bitstream, the video encoder 600 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 695.

[0103] The MV prediction module 675 generates a predicted MV based on a reference MV generated for a previously encoded video frame, i.e., a motion compensated MV for performing motion compensation. The MV prediction module 675 retrieves a reference MV from a previous video frame from the MV buffer 665. The video encoder 600 stores the MV generated for the current video frame in the MV buffer 665 as a reference MV for generating a predicted MV.

[0104] The MV prediction module 675 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The entropy encoder 690 encodes the difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame into a bitstream 695.

[0105] The entropy encoder 690 encodes various parameters and data into a bitstream 695 by using an entropy coding technique such as context adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 690 encodes various header elements, flags along with quantized transform coefficients 612 and residual motion data as syntax elements into the bitstream 695. The bitstream 695 is in turn stored in a storage device or transmitted to a decoder over a network communication medium, such as a network.

[0106] The in-loop filter 645 performs a filtering or smoothing operation on the reconstructed pixel data 617 to reduce encoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operation or smoothing operation performed by the in-loop filter 645 includes a sample adaptive offset (SAO), a deblocking filter (DBF), and / or an adaptive loop filter (ALF).

[0107] Figure 7 The portion of the video encoder 600 that implements intra prediction by blending multiple reference lines is marked. Figure 7 , the reference line selection module 710 selects one or more reference lines. An indication of the selected reference line is provided to the entropy encoder 690, which may signal an index indicating a combination including the selected reference line, or multiple indices indicating the selected reference lines individually.

[0108] Based on the reference line selection, corresponding samples are retrieved from the reconstructed picture buffer 650. The retrieved samples are provided to the reference line blending module 720, which uses the retrieved samples to generate a blended reference line with blended or fused samples.

[0109] The intra estimation module 620 specifies the intra prediction mode of the current block to the reference line blending module 720. The reference line blending module 720 selects samples from different reference lines for blending based on the direction of the intra prediction of the prediction mode of the current block. When the samples to be blended are located at non-integer positions due to the intra prediction mode, an interpolation filter 715 may be applied before the samples are blended.

[0110] The mixed samples of the mixed reference line are in turn provided to the intra prediction module 625 , which uses the mixed samples and other samples from the reconstructed picture buffer 650 to generate an intra prediction prediction of the current block as the predicted pixel data 613 .

[0111] Figure 8 A process 800 for performing intra prediction by blending multiple reference lines is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 600 perform the process 800 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the encoder 600 performs the process 800.

[0112] The encoder receives (at block 810) data to be encoded as a current pixel block in a current picture of a video.The encoder signals (at block 815) a first reference line and a second reference line selected from among a plurality of reference lines adjacent to the current block.

[0113] The encoder mixes (at block 820) samples of the first and second reference lines. Each mixed sample is generated by combining samples in the first reference line with samples in the second reference line identified according to the intra prediction mode. In some embodiments, when the intra prediction mode is an intra prediction direction, the position of the sample to be mixed in the first reference line and the position of the sample to be mixed in the second reference line follow the direction: the intra prediction mode; when the intra prediction mode is not an intra prediction direction (e.g., DC or planar), the position of the sample to be mixed in the first reference line and the position of the sample to be mixed in the second reference line are the same.

[0114] In some embodiments, the samples in the first reference line being blended may be samples of a top reference line portion (e.g., n1) or a left reference line portion (e.g., m1) of the first reference line. The top reference line portion (n1) of the first reference line may be identified by a first line index, and the top reference line portion (n2) of the second reference line may be identified by a second line index. Additionally, the left reference line portion (m1) of the first reference line may be identified by the first line index, and the left reference line portion (m2) of the second reference line may be identified by the second line index. In some embodiments, the first reference line index is explicitly signaled, and the second reference line index is signaled via an explicitly signaled offset relative to the first line index.

[0115] In some embodiments, when the mixed samples are located at non-integer positions (e.g., due to intra prediction directions), the encoder may apply an interpolation filter (e.g., a 4-tap filter) to the mixed samples. In some embodiments, when the samples are located at non-integer positions, the encoder applies a first interpolation filter to the samples in the first reference line to be mixed. When the samples are located at non-integer positions, the encoder may also apply a second interpolation filter to the samples in the second reference line to be mixed. The first and second interpolation filters may be the same or different (e.g., in terms of filter taps).

[0116] The encoder generates (at block 830) a prediction of a current block in an intra prediction mode using the mixed samples. In some embodiments, the prediction is generated using the mixed samples when an explicit flag enables a particular tool that uses samples from multiple reference lines for intra prediction. In some embodiments, the prediction is generated using the mixed samples only when an explicit flag enables a particular tool that uses samples from multiple reference lines. In some embodiments, when a difference between samples of a first reference line and a second reference line used to create the mixed sample fails to meet a particular criterion (e.g., a difference ≥ a threshold or < a threshold), the mixed sample is not used to generate a prediction of the current block. In some embodiments, whether to mix samples of the first and second reference lines is determined based on an implicit indication based on, for example, a block width, height, or area of ​​the current block.

[0117] The encoder encodes (at block 840) the current block using the generated prediction to generate a prediction residual.

[0118] III. Example Video Decoder

[0119] In some embodiments, the encoder may signal (or generate) one or more syntax elements in a bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0120] Fig. 9 An example video decoder 900 that performs inter-frame prediction is illustrated. As shown, the video decoder 900 is an image decoding or video decoding circuit that receives a bitstream 995 and decodes the contents of the bitstream into pixel data of a video frame for display. The video decoder 900 has several components or modules for decoding the bitstream 995, including some components selected from an inverse quantization module 911, an inverse transform module 910, an intra-frame prediction module 925, a motion compensation module 930, a loop filter 945, a decoded picture buffer 950, an MV buffer 965, an MV prediction module 975, and a parser 990. The motion compensation module 930 is part of the inter-frame prediction module 940.

[0121] In some embodiments, modules 910-990 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 910-990 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 910-990 are shown as separate modules, some modules may be combined into a single module.

[0122] The parser 990 (or entropy decoder) receives the bitstream 995 and performs initial parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 912. The parser 990 parses out the various syntax elements using entropy coding techniques such as context adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0123] The inverse quantization module 911 inversely quantizes the quantized data (or quantized coefficients) 912 to obtain transform coefficients, and the inverse transform module 910 inversely transforms the transform coefficients 916 to obtain a reconstructed residual signal 919. The reconstructed residual signal 919 is added to the predicted pixel data 913 from the intra-frame prediction module 925 or the motion compensation module 930 to generate decoded pixel data 917. The decoded pixel data is filtered by the in-loop filter 945 and stored in the decoded picture buffer 950. As shown in the figure, in some embodiments, the decoded picture buffer 950 is a storage outside the video decoder 900. In some embodiments, the decoded picture buffer 950 is a storage inside the video decoder 900.

[0124] The intra prediction module 925 receives intra prediction data from the bitstream 995 and generates predicted pixel data 913 from decoded pixel data 917 stored in the decoded picture buffer 950. In some embodiments, the decoded pixel data 917 is also stored in a line buffer (not shown) for intra prediction and spatial MV prediction.

[0125] In some embodiments, the contents of the decoded picture buffer 950 are used for display. The display device 955 either retrieves the contents of the decoded picture buffer 950 for direct display or retrieves the contents of the decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded picture buffer 950 via pixel transfer.

[0126] The motion compensation module 930 generates predicted pixel data 913 from the decoded pixel data 917 stored in the decoded picture buffer 950 according to the motion compensation MV (MC MV). These motion compensation MVs are decoded by adding the residual motion data received from the bitstream 995 to the predicted MV received from the MV prediction module 975.

[0127] The MV prediction module 975 generates a predicted MV, for example, a motion compensated MV for performing motion compensation, based on a reference MV generated for decoding a previous video frame. The MV prediction module 975 retrieves a reference MV of a previous video frame from the MV buffer 965. The video decoder 900 stores the motion compensated MV generated for decoding a current video frame in the MV buffer 965 as a reference MV for generating a predicted MV.

[0128] The in-loop filter 945 performs filtering or smoothing operations on the decoded pixel data 917 to reduce coding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operation or smoothing operation performed by the in-loop filter 445 includes a sample adaptive offset (SAO), a deblocking filter (DBF), and / or an adaptive loop filter (ALF).

[0129] Fig.10 The portion of the video decoder 900 that implements intra prediction by blending multiple reference lines is marked. Fig.10 , the reference line selection module 1010 selects one or more reference lines. An indication of the selected reference line is provided by the entropy decoder 990. The entropy decoder may receive signaling indicating an index of a combination including the selected reference lines, or signaling indicating multiple indices of the selected reference lines individually.

[0130] Based on the reference line selection, corresponding samples are fetched from the decoded picture buffer 950. The fetched samples are provided to the reference line blending module 1020, which uses the fetched samples to generate a blended reference line with blended or fused samples.

[0131] The entropy decoder 950 specifies the intra prediction mode of the current block to the intra prediction module 925 and the reference line blending module 1020. The reference line blending module 1020 selects samples from different reference lines to be blended based on the direction of the intra prediction mode of the current block. When the samples to be blended are located at non-integer positions due to the intra prediction mode, an interpolation filter 1015 may be applied before the samples are blended.

[0132] The mixed samples of the mixed reference line are in turn provided to the intra prediction module 925 , which uses the mixed samples and other samples from the decoded picture buffer 950 to generate an intra prediction prediction of the current block as the predicted pixel data 913 .

[0133] Fig.11 A process 1100 for performing intra prediction by blending multiple reference lines is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 900 perform process 1100 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 900 performs process 1100.

[0134] The decoder receives (at block 1110) data to be decoded as a current pixel block in a current picture of a video.The decoder receives (at block 1115) a selection of a first reference line and a second reference line among a plurality of reference lines adjacent to the current block.

[0135] The decoder mixes (at block 1120) samples of the first and second reference lines. Each mixed sample is generated by combining samples in the first reference line with samples in the second reference line identified according to the intra prediction mode. In some embodiments, when the intra prediction mode is an intra prediction direction, the position of the sample to be mixed in the first reference line and the position of the sample to be mixed in the second reference line follow the direction: the intra prediction mode; when the intra prediction mode is not an intra prediction direction (e.g., DC or planar), the position of the sample to be mixed in the first reference line and the position of the sample to be mixed in the second reference line are the same.

[0136] In some embodiments, the samples in the first reference line being blended may be samples of a top reference line portion (e.g., n1) or a left reference line portion (e.g., m1) of the first reference line. The top reference line portion (n1) of the first reference line may be identified by a first line index, and the top reference line portion (n2) of the second reference line may be identified by a second line index. Additionally, the left reference line portion (m1) of the first reference line may be identified by the first line index, and the left reference line portion (m2) of the second reference line may be identified by the second line index. In some embodiments, the first reference line index is explicitly signaled, and the second reference line index is signaled via an explicitly signaled offset relative to the first line index.

[0137] In some embodiments, when the mixed samples are located at non-integer positions (e.g., due to intra-prediction directions), the decoder may apply an interpolation filter (e.g., a 4-tap filter) to the mixed samples. In some embodiments, when the samples are located at non-integer positions, the decoder applies a first interpolation filter to the samples in the first reference line to be mixed (the samples to be mixed) . The decoder may also apply a second interpolation filter to the samples in the second reference line to be mixed when the samples are located at non-integer positions. The first and second interpolation filters may be the same or different (e.g., in terms of filter taps).

[0138] The decoder (at block 1130) generates a prediction of the current block in an intra prediction mode by using the mixed samples. In some embodiments, the prediction is generated by using the mixed samples when an explicit flag enables a specific tool for intra prediction using samples from multiple reference lines. In some embodiments, the samples are mixed to generate the prediction only when an explicit flag enables a specific tool for using samples from multiple reference lines. In some embodiments, when a difference between samples of a first reference line and a second reference line used to create the mixed sample fails to meet a specific criterion (e.g., a difference ≥ a threshold or < a threshold), the mixed sample is not used to generate the prediction of the current block. In some embodiments, whether to mix samples of the first and second reference lines is determined based on an implicit indication based on, for example, a block width, height, or area of ​​the current block.

[0139] The decoder reconstructs the current block (at block 1140) by using the generated prediction to produce a prediction residual. The decoder may then provide the reconstructed current block for display as part of a reconstructed current picture.

[0140] IV. Example Electronic Systems

[0141] Many of the above features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or via wired connections.

[0142] In this specification, the term "software" is meant to include firmware residing in a read-only memory or an application stored in a magnetic storage device, which can be read into the memory for processing by the processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that implement the software inventions described herein together are within the scope of this disclosure. In some embodiments, when the software program is installed to run on one or more electronic systems, one or more specific machine implementations for executing and performing the operations of the software program are defined.

[0143] Fig.12 An electronic system 1200 is conceptually illustrated to implement some embodiments of the present disclosure. The electronic system 1200 may be a computer (e.g., a desktop calculator, a personal calculator, a tablet calculator, etc.), a phone, a PDA, or any other kind of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1200 includes a bus 1205, a processing unit 1210, a graphics processing unit (GPU) 1215, a system memory 1220, a network 1225, a read-only memory 1230, a permanent storage device 1235, an input device 1240, and an output device 1245.

[0144] The bus 1205 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 1200. For example, the bus 1205 communicatively connects the processing unit 1210 and the GPU 1215, the read-only memory 1230, the system memory 1220, and the permanent storage device 1235.

[0145] From these various memory units, the processing unit 1210 retrieves instructions to be executed and data to be processed in order to perform the processes of the present disclosure. In different embodiments, the processing unit may be a single processor or a multi-core processor. Some instructions are passed to and executed by the GPU 1215. The GPU 1215 may offload various calculations or supplement the image processing provided by the processing unit 1210.

[0146] Read-only memory (ROM) 1230 stores static data and instructions used by processing unit 1210 and other modules of the electronic system. On the other hand, permanent storage device 1235 is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when electronic system 1200 is turned off. Some embodiments of the present disclosure use a mass storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as permanent storage device 1235.

[0147] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc. and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1235, system memory 1220 is a read-write storage device. However, unlike storage device 1235, system memory 1220 is a volatile read-write memory, such as random access memory. System memory 1220 stores some instructions and data used by the processor at runtime. In some embodiments, processing according to the present disclosure is stored in system memory 1220, permanent storage device 1235 and / or read-only memory 1230. For example, various memory units include instructions for processing multimedia clips and some embodiments. From these different memory units, processing unit 1210 retrieves instructions to be executed and data to be processed in order to perform processing of some embodiments.

[0148] The bus 1205 is also connected to input and output devices 1240 and 1245. The input device 1240 enables a user to transmit information and select commands to the electronic system. The input device 1240 includes an alphanumeric keyboard and a pointing device (also known as a "cursor control device"), a camera (e.g., a webcam), a microphone or a similar device for receiving voice commands, etc. The output device 1245 displays images generated by the electronic system or outputs data in other ways. The output device 1245 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices that function as both input and output devices, such as a touch screen.

[0149] Finally, if Fig.12 As shown, bus 1205 also couples electronic system 1200 to a network 1225 via a network adapter (not shown). In this manner, the computer can be part of a network of computers, such as a local area network ("LAN"), a wide area network ("WAN") or an intranet, or a network of networks. Any or all components of electronic system 1500 may be used in conjunction with the present disclosure.

[0150] Some embodiments include electronic components, such as microprocessors, memory, and storage that store computer program instructions in machine-readable or computer-readable media (or alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), digital versatile disk-read only (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer readable medium may store a computer program executable by at least one processing unit and including a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that generated by a compiler, and files including high-level code that is executed by a computer, an electronic component, or a microprocessor using an interpreter.

[0151] Although the above discussion primarily involves microprocessors or multi-core processors executing software, many of the above functions and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0152] As used in this specification and any claims of this application, the terms "computer", "server", "processor" and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or display means display on an electronic device. As used in this specification and any claims of this application, the terms "computer readable medium", "computer readable medium" and "machine readable medium" are entirely limited to tangible physical objects that store information in a readable form. These terms do not include any wireless signals, wired download signals, and any other temporary signals.

[0153] Although the present disclosure has been described with reference to many specific details, those skilled in the art will recognize that the present disclosure may be implemented in other specific forms without departing from the spirit of the present disclosure. Figure 8 and Fig.11 ) conceptually illustrate processes. The specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the process may be implemented using multiple sub-processes or as part of a larger macro process. Therefore, it will be understood by those skilled in the art that the present disclosure is not limited by the foregoing illustrative details, but is defined by the appended claims.

[0154] Additional notes

[0155] The subject matter described herein sometimes illustrates different components contained within or connected to different other components. It should be understood that the architectures depicted in this way are merely examples, and that many other architectures that implement the same functionality can actually be implemented. Conceptually, any arrangement of components that implement the same functionality is effectively "associated" to achieve the desired functionality. Therefore, any two components combined herein to achieve a particular functionality regardless of the architecture or intermediate components can be considered to be "associated" with each other to achieve the desired functionality. Similarly, any two components so associated may also be considered to be "operably connected" or "operably coupled" to each other to achieve the desired functionality, and any two components that can be so associated may also be considered to be "operably connected" or "coupled" to each other to achieve the desired functionality. Specific examples of operably coupled include, but are not limited to, physically matable and / or physically interactive components and / or wirelessly interactive and / or wirelessly interactive components and / or logically interactive and / or logically interactive components.

[0156] Furthermore, with respect to the use of substantially any plural and / or singular terms herein, those skilled in the art may translate from the plural to the singular and / or from the singular to the plural, depending on the context and / or application. For the sake of clarity, the various singular / plural arrangements may be expressly set forth herein.

[0157] Furthermore, those skilled in the art will appreciate that, in general, the terms used herein, and particularly the terms used in the appended claims, such as the bodies of the appended claims, are generally intended as "open" terms, e.g., the term "comprising" should be interpreted as "including but not limited to," and "having" should be interpreted as "at least having." Those skilled in the art will further appreciate that if a specific number of claim statements are intended to be introduced, such intent will be explicitly stated in the claim, and such intent is absent in the absence of such statement. For example, to aid understanding, the following appended claims may contain statements that use the introductory phrases "at least one" and "one or more" to introduce the claim statements. However, the use of such phrases should not be interpreted as implying that a claim introduced by the indefinite article "a" or "an" limits any particular claim that contains such introduced claim to an implementation that contains only one such statement, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "an," which should be interpreted as "at least one" or "one or more." The same applies to the use of definite articles to introduce claim statements. Furthermore, even if a specific number of the claims recited by the introduction is explicitly cited, a person skilled in the art will recognize that such a recitation should be interpreted as indicating at least the number of citations, e.g., a reference to "two iterations," without other modifiers, means at least two iterations, or two or more iterations. Furthermore, in those cases where the convention uses something like "at least one of A, B, and C, etc.", generally, such a construction is intended to be understood by a person skilled in the art in the sense of the convention, e.g., "a system having at least one of A, B, and C" would include, but is not limited to, systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. In those cases where the convention uses something like "at least one of A, B, or C," generally, such a construction is intended to be understood by a person skilled in the art in the sense of the convention, e.g., "a system having at least one of A, B, or C" would include, but is not limited to, systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. Those skilled in the art will further understand that in practice, any disjunctive words and / or phrases that appear as two or more alternative terms, whether in the specification, claims or drawings, should be understood to include the possibility of one term, one term or two terms. For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B".

[0158] It can be understood from the above that various embodiments of the present disclosure have been described herein for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the present disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are indicated by the attached claims.

Claims

1. A video encoding and decoding method, comprising: Receiving data of a pixel block to be encoded or decoded as a current block of a current picture of a video; receiving or sending a signal notifying a first reference line and a second reference line selected from a plurality of reference lines adjacent to the current block; blending samples of the first reference line and the second reference line, each blended sample being generated by combining a sample in the first reference line with a sample in the second reference line identified according to an intra prediction mode; generating a prediction of a current block using the mixed sample in an intra prediction mode; as well as The current block is encoded or decoded using the generated prediction.

2. The video encoding and decoding method according to claim 1, characterized in that: The samples in the first reference line that are blended are samples of the top reference line portion or the left reference line portion of the first reference line.

3. The video encoding and decoding method according to claim 2, characterized in that: A top reference line portion of the first reference line is identified by a first reference line index and a top reference line portion of the second reference line is identified by a second reference line index, wherein the reference line index is explicitly signaled and the second reference line index is signaled by an offset relative to the first reference line index that is explicitly signaled.

4. The video encoding and decoding method according to claim 3, characterized in that: A left reference line portion of the first reference line is identified by the first reference line index, and a left reference line portion of the second reference line is identified by the second reference line index.

5. The video encoding and decoding method according to claim 1, characterized in that Also included is applying an interpolation filter to the mixed samples when the mixed samples are located at non-integer positions.

6. The video encoding and decoding method according to claim 1, characterized in that The method further includes applying a first interpolation filter to samples in the first reference line to be mixed when the samples in the first reference line to be mixed are located at non-integer positions.

7. The video encoding and decoding method according to claim 6, characterized in that The method further includes applying a second interpolation filter to the samples when the samples in the second reference line to be blended are located at non-integer positions, wherein the first interpolation filter and the second interpolation filter are different.

8. The video encoding and decoding method according to claim 1, characterized in that: When a specific tool using the multiple reference lines for intra prediction of the current block is explicitly enabled, the prediction is generated by using the mixed samples.

9. The video encoding and decoding method according to claim 1, characterized in that: When the difference between the samples of the first and second reference lines used to create the mixed sample does not meet a certain criterion, the mixed sample is not used to generate a prediction of the current block.

10. The video encoding and decoding method according to claim 1, characterized in that: Further comprising determining whether to blend samples of the first reference line and the second reference line based on an implicit indication.

11. The video encoding and decoding method according to claim 10, characterized in that: The implicit indication depends on a block width, height or area of ​​the current block.

12. The video encoding and decoding method according to claim 1, characterized in that: When the intra prediction mode is not an intra prediction direction, positions of the samples to be mixed in the first reference line are the same as positions of the samples to be mixed in the second reference line.

13. The video encoding and decoding method according to claim 1, characterized in that: When the intra prediction mode is an intra prediction direction, the positions of the samples to be mixed in the first reference line and the positions of the samples to be mixed in the second reference line follow the direction of the intra prediction mode.

14. An electronic device comprising: The video codec circuit is configured to perform operations including the following: Receiving data of a pixel block of a current block of a current picture of a video to be encoded or decoded; receiving or sending a signal notifying a selection of a first reference line and a second reference line among a plurality of reference lines adjacent to the current block; blending samples of the first reference line and the second reference line, each blended sample being generated by combining a sample in the first reference line with a sample in the second reference line identified according to an intra prediction mode; generating a prediction of the current block using the mixed samples in an intra prediction mode; as well as The current block is encoded or decoded using the generated prediction.

15. A video decoding method, comprising: Receiving data of a pixel block to be decoded as a current block of a current picture of a video; receiving a selection of a first reference line and a second reference line among a plurality of reference lines adjacent to the current block; blending samples of the first reference line and the second reference line, each blended sample being generated by combining a sample in the first reference line with a sample in the second reference line identified according to an intra prediction mode; generating a prediction of the current block using an intra prediction mode of mixed samples; as well as The current block is reconstructed by using the generated prediction.

16. A video encoding method, comprising: Receiving data of a pixel block to be encoded as a current block of a current picture of a video; signaling selection of a first reference line and a second reference line among a plurality of reference lines adjacent to the current block; blending samples of the first reference line and the second reference line, each blended sample being generated by combining a sample in the first reference line with a sample in the second reference line identified according to an intra prediction mode; generating a prediction of the current block using the mixed samples in an intra prediction mode; as well as The current block is encoded using the generated prediction.

Images