Filter-based prediction

Abstract

The present disclosure provides a method of coding a pixel block based on filter prediction. A video coder receives data to be encoded or data to be decoded as a current pixel block of a current picture of a video; wherein the current pixel block is coded using a motion vector, a block vector, or a geometric partition mode. The video coder determines filter parameters for generating a target sample prediction. The video coder generates a filter prediction of the current pixel block based on the filter. The video coder generates a final prediction of the current pixel block based on the filter prediction. The video coder encodes or decodes the current pixel block based on the final prediction.

Description

[0001] Cross-referencing of related patent applications This disclosure is part of a non-provisional application that claims priority to U.S. Provisional Patent Application No. 63 / 621,638, filed January 17, 2024, and U.S. Provisional Patent Application No. 63 / 550,674, filed February 7, 2024. The contents of the above applications are incorporated herein by reference. Technical Field

[0002] This disclosure relates to video encoding and decoding, and more particularly to a method for encoding and decoding pixel blocks based on a filter-based prediction pattern. Background Technology

[0003] Unless otherwise stated herein, the methods described in this section are not prior art with respect to the claims listed below, and are not acknowledged as prior art by virtue of their inclusion in this section.

[0004] High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT transform coding architecture. Its basic compression unit is called a coding unit (CU), which is a 2Nx2N square pixel block. Each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0005] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted based on the reconstructed signal extracted from the encoded image region; the predicted residual signal is processed by block transform; the transform coefficients, along with other auxiliary information, are quantized and entropy-encoded in the bitstream; the reconstructed signal is generated jointly from the predicted signal and the reconstructed residual signal, wherein the reconstructed residual signal is obtained by inverse transforming the dequantized transform coefficients; the reconstructed signal is further processed by loop filtering to remove coding artifacts; the decoded image is stored in the frame buffer for predicting subsequent images in the input video signal.

[0006] In VVC, the encoded image is segmented into non-overlapping square block regions represented by relevant coding tree units (CTUs). Leaf nodes of the coding tree correspond to coding units (CUs). The encoded image can consist of multiple slices, each containing an integer number of CTUs. The CTUs within a slice are processed in raster scan order. Double-prediction (B) slices are decoded via intra-frame prediction or inter-frame prediction; inter-frame prediction uses at most two motion vectors (MVs) and a reference index to predict the sample value for each block. Predictive (P) slices are decoded via intra-frame prediction or inter-frame prediction; inter-frame prediction uses at most one motion vector and a reference index to predict the sample value for each block. Intra-frame (I) slices are decoded only via intra-frame prediction.

[0007] Using a nested multi-type tree (MTT) quadtree structure, the CTU can be divided into one or more non-overlapping codec units (CUs) to accommodate different local motion and texture characteristics. Furthermore, a CU can be divided into smaller CUs using one of five segmentation types: quadtree segmentation, vertical binary tree segmentation, horizontal binary tree segmentation, vertical center-side ternary tree segmentation, and horizontal center-side ternary tree segmentation.

[0008] Each CU contains one or more prediction units (PUs). The prediction unit, together with the associated CU syntax, constitutes the basic unit for conveying prediction result information. The values ​​of associated pixel samples within a PU are predicted using a specified prediction process. Each CU may contain one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) consists of one luma transform block (TB) and two corresponding chroma transform blocks, each TB corresponding to a residual sample block from a color component. Integer transforms are applied to the transform blocks. The level values ​​of the quantization coefficients, along with other auxiliary information, are entropy-encoded in the bitstream. Coding Tree Blocks (CTBs), Coding Blocks (CBs), Prediction Blocks (PBs), and Transform Blocks (TBs) are used to specify two-dimensional sample arrays of a single color component associated with the CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU contains one luma CTB, two chroma CTBs, and associated syntax elements. CUs, PUs, and TUs have similar relationships.

[0009] For each inter-frame predicted CU, motion parameters include motion vectors, reference image indices, reference image list usage indices, and other additional information used to generate inter-frame predicted samples. Motion parameters can be passed explicitly or implicitly. When a CU is encoded / decoded in skip mode, it is associated with only one PU and does not contain significant residual coefficients, encoding / decoding motion vector increments, or reference image indices. Merging mode specifies that the motion parameters of the current CU are available from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in VVC. Merging mode can be applied to any inter-frame predicted CU. An alternative to merging mode is explicit transmission of motion parameters, where the motion vectors of each CU, the corresponding reference image indices for each reference image list, reference image list usage flags, and other necessary information are explicitly passed.

[0010] Intra Block Copy (IBC) or Current Picture Referencing (CPR) refers to encoding and decoding pixel blocks by referencing the same pixel location in the current image as the current pixel block using block vectors. Summary of the Invention

[0011] The following abstract is for illustrative purposes only and is not intended to be limiting in any way. That is, the abstract aims to introduce the concepts, key points, benefits, and advantages of the novel and non-obvious techniques described herein. Some, but not all, of the embodiments will be further described in the detailed description below; therefore, this abstract is neither intended to define the essential features of the claimed subject matter nor to determine its scope.

[0012] Several embodiments of this disclosure provide a method for encoding and decoding pixel blocks based on filter prediction. A video codec receives data to be encoded or decoded as the current pixel block of the current video image. The current pixel block can be encoded or decoded using motion vectors, block vectors, or geometric segmentation modes. The video codec first determines filter parameters for generating target sample predictions. Subsequently, the video codec generates a filter prediction for the current pixel block based on the filter predictions, and generates a final prediction for the current pixel block based on the filter predictions. Finally, the video codec encodes or decodes the current pixel block based on the final prediction.

[0013] In some embodiments, the parameters of the filter are derived based on a reference block, wherein the reference block is located based on the motion vector or block vector of the current pixel block.

[0014] The parameters of the filter are inherited from the candidates in the candidate list, wherein each candidate provides a set of filter information for configuring the filter.

[0015] The parameters of the filter can be derived based on samples from the reference block or samples from the template region adjacent to the reference block or the current pixel block.

[0016] In some embodiments, the video codec generates a filter prediction for the current pixel block based on the filter by using samples from a reference block or by using samples from neighboring reference blocks or the current pixel block as input to the filter.

[0017] A flag can be used to indicate whether the filter prediction is used to generate the final prediction. In some embodiments, the final prediction is generated by merging the filter prediction with a non-intra-frame prediction generated based on the motion vector or block vector of the current pixel block. The weight allocation in the merging process can be determined based on the position, width, height, area, or segmentation mode of the current pixel block.

[0018] In some embodiments, the current pixel block can be segmented into at least a first region and a second region by geometric segmentation (e.g., by a geometric segmentation mode), thereby generating a prediction result for at least one region in the first region and the second region using the filter.

[0019] In some embodiments, the video codec applies a predefined processing flow to the final prediction to determine a representative intra-prediction pattern. The predefined processing flow may include gradient analysis. The video codec can use the representative intra-prediction pattern to determine one or more transform settings (e.g., a transform set and / or transform kernel and / or transform transpose flags) for encoding or decoding the current pixel block. The representative intra-prediction pattern may be stored for reference in subsequent blocks. Attached Figure Description

[0020] The accompanying drawings are included in this specification to provide a further understanding of this disclosure and form part of this disclosure. The drawings illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure. It will be understood that the drawings are not necessarily drawn to scale, and for the purpose of clearly illustrating the concepts of this disclosure, the displayed dimensions of some components may not be proportional to their actual dimensions in an implementation.

[0021] Figure 1 It showcases 67 intra-frame prediction modes, including 65 directional or angular intra-frame prediction modes (from 2 to 66).

[0022] Figure 2A -C indicates the three defined types of reconstructed regions.

[0023] Figure 3A-C indicates three defined filter shapes that have 15 inputs and produce one output.

[0024] Figure 4 An example is shown that predictions are generated for different positions of the current pixel block in diagonal order.

[0025] Figure 5 This demonstrates how to implicitly derive the intra-prediction mode for the current pixel block using Decoder-side Intra Mode Derivation (DIMD).

[0026] Figure 6 The locations of the spatial merging candidates are shown.

[0027] Figure 7 This demonstrates an example of segmenting the coding unit (CU) using Geometric Partitioning Mode (GPM).

[0028] Figure 8 This demonstrates a CU example where intra-frame prediction is used for encoding and decoding of all GPM segmented regions.

[0029] Figure 9A -C is the IPM candidate that GPM can use when inter-frame prediction and intra-frame prediction are employed.

[0030] Figure 10 The MxN pattern around the location of the sample to be predicted by the filter is shown.

[0031] Figure 11A -C represents a subset of the MxN region surrounding the location to be predicted.

[0032] Figure 12 The current pixel block template used to derive filter parameters is displayed.

[0033] Figure 13A -B demonstrates the process of generating samples to be predicted by the filter in different parts of the current pixel block.

[0034] Figure 14A -B illustrates the process of deriving filter predictions based on the reference block when the current pixel block is encoded and decoded in a non-intra-frame manner.

[0035] Figure 15 This demonstrates how the current pixel block is geometrically divided into multiple regions using a segmentation mode.

[0036] Figure 16 This provides a conceptual example flowchart for enabling and using target filter prediction.

[0037] Figure 17 An example of a video encoder that enables filter-based prediction is shown.

[0038] Figure 18 This demonstrates the portion of the video encoder that implements intra-frame prediction mode or non-intra-frame prediction mode based on the target filter.

[0039] Figure 19 This concept demonstrates the process of encoding pixel blocks using a prediction mode based on a target filter.

[0040] Figure 20 An example of a video decoder that enables filter-based prediction is shown.

[0041] Figure 21 This demonstrates the portion of the video decoder that implements either intra-frame prediction mode or non-intra-frame prediction mode based on the target filter.

[0042] Figure 22 This concept demonstrates the process of decoding pixel blocks using a prediction mode based on a target filter.

[0043] Figure 23 This invention conceptually demonstrates an electronic system that implements certain embodiments of the present disclosure. Detailed Implementation

[0044] In the following detailed description, numerous specific details are illustrated by way of example to provide a full understanding of the related technologies. Any variations, derivatives, and / or extensions based on the technologies described herein are within the scope of this disclosure. In some cases, to avoid affecting the understanding of the key technical points of this disclosure, well-known methods, processes, components, and / or circuits involving one or more exemplary embodiments may be described in general terms only, without providing specific details.

[0045] I. Intra-frame prediction A. Intra-mode encoding and decoding with 67 intra-prediction modes To capture arbitrary edge directions presented in natural video, the number of directional intra-frame modes has been expanded from 33 (such as High Efficiency Video Coding (HEVC)) to 65 (such as Versatile Video Coding (VVC)). Figure 1 It showcases 67 intra-frame prediction modes, including 65 directional or angular intra-frame prediction modes (from 2 to 66).

[0046] B. Extrapolation filter-based IntraPrediction (EIP) Intra-frame prediction based on extrapolation filters, or EIP, is a method for generating predictions for each sample in a block.

[0047] In some implementations, there are three reconstruction region types and three filter shapes. Figure 2A -C indicates the three defined types of reconstructed regions. Figure 3A -C indicates three defined filter shapes that have 15 inputs and produce one output.

[0048] When predicting the current pixel block using EIP mode, the decoder decodes the relevant syntax elements to determine the type of reconstructed region and the filter shape selected based on the current pixel block. The size of the reconstructed region depends on min(block width, block height) and the selected filter shape. For example, when the current pixel block is 8x16 and the selected filter shape is 4x4, the upper size (aboveSize) of the reconstructed region is min(8, 16) + 4 – 1 = 11, and the left size (leftsize) of the reconstructed region is also min(8, 16) + 4 – 1 = 11.

[0049] The selected filter moves horizontally or vertically within the selected reconstruction region in single-pixel strides to construct the autocorrelation matrix and cross-correlation vector. The coefficients are calculated from the autocorrelation matrix and cross-correlation vector in the same way as the Convolutional Cross-Component Model (CCCM), which uses a convolutional model to improve chroma prediction performance. Its model coefficients are calculated by minimizing the mean-square error (MSE) between the predicted and reconstructed chroma samples in a reference region that is spatially adjacent to the current pixel block.

[0050] EIP mode generates predictions for the current pixel block by predicting diagonally from the top left to the bottom right position, such as... Figure 4 As shown in the figure, an example is illustrated where predictions are generated for different positions of the current pixel block in diagonal order from the top left to the bottom right. In some implementations, the predicted samples are calculated as follows: .

[0051] in, It is the predicted value at the current pixel block (x, y) position; It is the i-th coefficient of the selected EIP filter, and the coefficient index is from 0 to 14; It is a reconstructed or predicted value used to predict the current location; and These are the position offsets of the current position along the x-axis and y-axis, respectively.

[0052] In some implementations, the coefficients of the current coding unit (CU) can be derived based on neighboring reconstructed pixels. To derive the coefficients of the EIP filter based on neighboring reconstructed pixels, the decoder decodes the relevant syntax elements to determine the type of reconstructed region and the filter shape selected based on the current pixel block.

[0053] The filter shape and filter coefficients can also be inherited from previously decoded blocks using EIP or EIP merging modes. In some implementations, the decoder decodes the EIP merging flag to determine whether the proposed merging mode should be used when the current pixel block uses EIP mode. When the EIP merging flag is true, the merging index is further decoded. The EIP merging list includes spatially neighboring and non-neighboring candidates, temporal candidates, and historical candidates. The constructed EIP merging list can contain up to 12 candidates, and can be reduced to a maximum of 6 candidates through a reordering process based on the SAD cost measured on an L-shaped template (column width and row height are both 1). In SAD calculation, the EIP filter's prediction of the template region is generated only from reconstructed samples (neighboring samples and template samples), allowing the EIP filter to be applied in parallel rather than serially.

[0054] C. Decoder-Side Intra Mode Derivation (DIMD) Decoder-side Intra-Modal Derivation (DIMD) is a technique that first derives two intra-prediction modes from the reconstructed neighboring samples (templates) of a block. Then, it weights the predictions generated based on these two modes with those generated based on the planar mode, where the weights are calculated based on gradients. The DIMD mode serves as an alternative prediction mode and is consistently checked in the highly complex RDO mode. To implicitly derive the intra-prediction mode of a block, both the encoder and decoder perform texture gradient analysis. This process begins with an empty Histogram of Gradients (HoG), which has 65 entries corresponding to 65 angular intra-prediction modes. The magnitudes of these entries are determined during the texture gradient analysis.

[0055] The video codec implementing DIMD performs the following steps: First, the video codec selects a template region consisting of T=3 columns and rows from the left and top of the current pixel block, respectively, and uses this template region as a reference for gradient-based intra-prediction mode derivation. Second, horizontal and vertical Sobel filters are applied at all 3×3 window positions centered on the middle row of the template. At each window position, the Sobel filter calculates the intensity in the purely horizontal and purely vertical directions, respectively. and Then, the texture angle of the window is calculated using the formula shown below: .

[0056] This angle can be converted into one of 65 intra-prediction modes. Once the intra-prediction mode index of the current window is derived as idx, its amplitude in HoG[idx] will be updated by addition as shown below:

[0057] Figure 5 This demonstrates how decoder-side intra-mode derivation (DIMD) is used to implicitly derive the intra-prediction mode of the current pixel block. The figure shows an example of the histogram of gradients (HoG) 510 calculated after applying the above operation to all pixel locations in the template region 515 containing the neighboring rows of pixel samples around the current pixel block 500.

[0058] II. Inter-frame prediction For each codec unit (CU) in inter-frame prediction, generating inter-frame prediction samples requires motion parameters and other additional information, including motion vectors, reference image indices, and reference image list indexes. These motion parameters can be passed explicitly or implicitly. When a CU is encoded in skip mode, it is associated with only one prediction unit (PU) and does not contain significant residual coefficients, nor does it contain the encoded motion vector increments or reference image indices. In this case, a merging mode must be specified, and the motion parameters of the current CU are obtained from neighboring CUs and additional scheduling schemes introduced in the VVC (Variable Frame Control). Neighboring CUs include spatially adjacent CUs and temporally adjacent CUs.

[0059] The merging mode can be applied to any inter-frame prediction CU, not just those using skip mode coding. An alternative to the merging mode is the explicit delivery of motion parameters, whereby for each CU, its motion vector, the reference image index corresponding to each reference image list, the reference image list usage flag, and other necessary information are explicitly delivered.

[0060] The merged candidate list can be constructed by including the following candidates in order: Predicted spatial motion vectors from neighboring CUs (MVP) (spatial merging candidate) Predicted temporal motion vectors from co-located CUs (MVP) (temporal merging candidate) Motion vector prediction based on historical information from a first-in, first-out (FIFO) table (MVP) (HMVP merging candidate) Figure 6 The locations of spatial merging candidates are shown. A maximum of four merging candidates can be selected from the candidates at the locations shown in the figure. The derivation order is B0, A0, B1, A1, and B2. Position B2 is considered only if one or more CUs at positions B0, A0, B1, and A1 are unavailable (e.g., because they belong to another segment or block) or if it is intra-frame encoding / decoding. After adding a candidate at position A0, the addition of remaining candidates requires a redundancy check to ensure that candidates with the same motion information are excluded from the list, thereby improving encoding / decoding efficiency.

[0061] For time merging candidates, only one candidate is added to the list. Specifically, during the derivation of this time merging candidate, a scaled motion vector is derived based on the co-located CUs belonging to the co-located reference image. The list of reference images and reference indices used to derive the co-located CUs are explicitly passed in the slice header.

[0062] Historical motion vector prediction (HMVP) candidates are added to the merging list after spatial motion vector prediction (MVP) and temporal motion vector prediction (TMVP). In this method, motion information from previous codec blocks is stored in a table and used as the current CU's motion vector prediction (MVP). During encoding / decoding, this table, containing multiple HMVP candidates, is continuously updated and reset (cleared) when a new codec tree unit (CTU) row is encountered.

[0063] III. Geometric Partitioning (GPM) A. Geometric segmentation mode As shown in the figure, when using the Geometric Partitioning (GPM) mode, the CU is divided into at least two parts by a geometrically defined straight line. The position of this partition line is derived mathematically from the angle and offset parameters of the specific partition. Each GPM partition or GPM partitioning operation is a partitioning mode characterized by defining the partition line through a distance-angle combination. Figure 7 An example of segmenting the CU700 using the Geometric Segmentation Mode (GPM) is shown, where the segmentation edges are specified by a combination of distance and angle.

[0064] B. Geometric segmentation mode using a combination of inter-frame / intra-frame prediction and intra-frame prediction In some embodiments, a GPM segmentation region of a CU can be predicted by inter-frame or intra-frame prediction, while another GPM segmentation region of the CU can be predicted by intra-frame prediction. Figure 8 This demonstrates a CU example where intra-frame prediction is used for encoding and decoding of all GPM segmented regions.

[0065] like Figure 9A As shown in -C, for each GPM segmentation region, its final predicted sample is generated by weighting the inter-frame predicted sample and the intra-frame predicted sample. The inter-frame predicted sample is derived from the inter-frame GPM, while the intra-frame predicted sample is derived from the Intra Prediction Mode (IPM) candidate list and the index signal from the encoder. The size of the IPM candidate list is predefined as 3. Figure 9A -C indicates the IPM candidates that GPM can use when employing inter-frame prediction and intra-frame prediction methods. Figure 9A This demonstrates a parallel mode for GPM-segmented regions using intra-frame prediction, which is an angular intra-frame prediction direction parallel to the GPM segmentation edges. Figure 9B This demonstrates a vertical mode for GPM segmentation regions using intra-frame prediction, which is an angular intra-frame prediction direction perpendicular to the GPM segmentation edges. Figure 9C Intra-planar mode intra-prediction is demonstrated for GPM segmentation regions using intra-frame prediction.

[0066] IV. Filter-based prediction modes Filter-based intra-prediction modes can be used to apply filters to generate predictions for the current pixel block. The EIP described above is an example of a filter-based intra-prediction mode. Target-filter-based intra-prediction mode (or target intra-prediction mode) is a filter-based intra-prediction mode that applies filters to samples spatially adjacent to the target sample. Therefore, EIP is an example of a target-filter-based intra-prediction mode, but it is not limited to EIP and can be any variant of a filter-based intra-prediction mode. On the other hand, target-filter-based non-intra-prediction mode (or target non-intra-prediction mode) refers to a coding / decoding mode in which, for each target sample in the current pixel block, a prediction for the current pixel block is generated by applying filters to samples at the same location within or adjacent to a reference block determined by motion vectors or block vectors.

[0067] In some embodiments, intra-frame prediction modes based on the target filter and / or non-intra-frame prediction modes based on the target filter can be used to form filter predictions for target intra-frame blocks (intra-codec blocks that serve as filter targets). In some embodiments, intra-frame prediction modes based on the target filter and / or non-intra-frame prediction modes based on the target filter can be used to form filter predictions for target non-intra-frame blocks (non-intra-codec blocks that serve as filter targets). Here, a non-intra-frame block may refer to a block encoded or decoded via inter-frame prediction or block vector prediction.

[0068] A. Intra-frame prediction mode based on target filter In some embodiments, the filter (or model) for the intra-prediction mode based on the target filter includes a method for determining the filter shape and / or filter parameters. After selecting a filter shape from one or more candidate filter shapes and / or a method for determining the filter parameters from one or more candidate filter parameter determination methods, the filter for the current pixel block is obtained. This filter is then applied to each sample in the current pixel block to be predicted by the filter.

[0069] In some implementations, filters are applied in a predefined order, such as horizontal scanning, vertical scanning, diagonal scanning, and / or any scanning order. For example, the predefined order is diagonal scanning. That is, the prediction sample located at the first position in the diagonal scanning order of the current pixel block is output first. Then, the prediction samples located at the second, third, ..., (block area) positions in the diagonal scanning order of the current pixel block are output sequentially. When applying the filter, the filter input is combined with the filter parameters (or convolved) to obtain the filter output. In some implementations, the input samples refer to the reconstructed and / or filter predictions and / or prediction samples associated with the current pixel block, as well as the reconstructed and / or filter predictions and / or prediction samples of reference regions in the same image that were reconstructed before or adjacent to the current pixel block. Finally, after applying the filter to each prediction sample to be filtered in the current pixel block, the filter prediction for that current pixel block is obtained.

[0070] In some implementations, the filter shape uses a pattern surrounding (excluding) the location (x, y) of the sample to be filtered for prediction. This pattern is an MxN region or any subset of an MxN region surrounding the sample to be filtered for prediction, where M is a first predefined positive number greater than 1, and / or N is a second predefined positive number greater than 1. Figure 10 The MxN pattern around the location of the sample to be filtered and predicted is shown.

[0071] In some embodiments, M and N may be the same or different. For example, M and / or N is equal to 1, 2, 3, 4, ……, or any number specified in the standard. For another example, if the block width of the current pixel block is greater than the block height, then M is greater than N. For another example, if the block height of the current pixel block is greater than the block width, then N is greater than M. For another example, M and / or N are adaptively adjusted according to the position (x, y).

[0072] In some embodiments, a subset of the MxN region around the sample to be filtered and predicted may be: (a) the region in the MxN region excluding the region where the abscissa > x; and / or (b) the region in the MxN region excluding the region where the ordinate > y. Figure 11A -C shows a subset of the MxN region around the position of the sample to be filtered and predicted. The MxN region excludes the region where the abscissa > x and excludes the region where the ordinate > y. Figure 11A An example where M = N is shown (so the MxN region is square). The method of using the subset of the MxN region is not limited to the case where M is equal to N, and / or can be applied to the cases where M > N and / or N > M. For another example, the subset of MxN is the MxN region that simultaneously satisfies (a) and (b) and M > N. For another example, the subset of MxN is the MxN region that simultaneously satisfies (a) and (b) and N > M. Figure 11B An example where M > N is shown. Figure 11C An example where M < N is shown.

[0073] In some embodiments, multiple filter shapes are supported, and the selection from multiple filter shapes depends on explicit signaling at the block level, Coding Tree Unit (CTU) level, slice level, tile level, Sequence Parameter Set (SPS) level, Picture Parameter Set (PPS) level, picture level (such as picture header), and / or sequence level. If the total number of candidate filter shapes in the list containing multiple filter shapes is K, then a signaling of a syntax element is required to select the filter shape. For example, the syntax element can use truncated unary coding and use the shortest codeword to indicate the first candidate filter shape in the list.

[0074] In some embodiments, multiple ways of determining filter parameters are supported, and the selection from multiple ways of determining filter parameters depends on explicit signaling at the block level, CTU level, slice level, tile level, SPS level, PPS level, picture level (such as picture header), and / or sequence level.

[0075] In some implementations, filter parameters can be determined by deriving them using a template (neighborhood region) of the current pixel block. In other implementations, filter parameters can be determined by inheriting filter information from previously encoded blocks. For the purposes of this document, the process of deriving filter parameters using a template of the current pixel block is referred to as a "deriving path," while the process of inheriting filter information from previously encoded blocks is referred to as an "inheriting path." In some implementations, the video codec can select between the deriving path and the inheriting path to determine filter parameters based on explicit signaling at the block level, codec tree unit (CTU) level, slice level, tile level, SPS level, PPS level, image level (e.g., image header), and / or sequence level.

[0076] In some implementations, when deriving filter parameters using a template (neighborhood region) of the current pixel block, the reconstructed samples of the templates adjacent to the current pixel block can be used as "golden samples." Any method can be employed to minimize the difference between the "golden samples" and the filtering result to obtain filter parameters that allow the filtering result to match the "golden samples" well. For example, regression-based methods, such as Gaussian elimination and / or arbitrary regression methods consistent with cross-component model derivation methods for cross-component chromaticity modes, can be used to find filter parameters that minimize the difference between the "golden samples" and the filtering result.

[0077] Figure 12 This displays the current pixel block template used to derive filter parameters. This template can refer to the top template, left template, top-left template, and / or any combination of the above templates for the current pixel block.

[0078] In some implementations, when deriving filter parameters using a template (neighborhood region) of the current pixel block, the selection of a template from multiple candidate templates may depend on the width, height, area, and / or position of the current pixel block. For example, if the current pixel block is located at the top boundary of the image and / or the top boundary of a Codec Tree Unit (CTU) row, the top template and / or the upper-left template are not used to derive filter parameters. Similarly, if the current pixel block is located at the left boundary of the image and / or the left boundary of a CTU, the left template and / or the upper-left template are not used to derive filter parameters. Furthermore, different templates used to derive filter parameters correspond to multiple methods of determining filter parameters. The selection from these multiple methods may depend on explicit signaling at the block level, CTU level, slice level, tile level, SPS level, PPS level, image level (e.g., image header), and / or sequence level. For example, the size of the template depends on the position of the block. When the block is located at the right boundary of the image or any predefined boundary, the top template does not include template regions outside the current image or any predefined range. When the block is located at the bottom boundary of the image or any predefined boundary, the left template does not include the template area outside the current image or any predefined range.

[0079] In some implementations, when inheriting filter information from previously encoded blocks, the filter information of the previously encoded blocks can be used to determine the filter for the current pixel block. This filter information includes filter shape and / or filter parameters and / or some filter parameters. The previously encoded block may be located in the same image (or frame) as the current pixel block, in a region reconstructed before the current pixel block; or it may be located in a region of an image reconstructed before the current image (e.g., an image at the same location as the current pixel block).

[0080] In some implementations, filter information can be inherited from previously encoded blocks via spatially adjacent candidates (including left-side and / or top-side adjacent blocks) and / or spatially non-adjacent candidates, and / or history-based candidates, and / or time-based candidates, and / or propagation candidates. Spatially adjacent candidates can come from the left and / or top and / or top-left and / or top-right and / or bottom-left adjacent blocks of the current pixel block, and / or any subset of the aforementioned positions. Spatially non-adjacent candidates can come from any predefined position in the search pattern around the current pixel block, and / or any subset of the aforementioned positions. History-based candidates can come from a history buffer that stores filter information from previously encoded blocks. The history buffer is empty at predefined times. For example, the history buffer is empty at the beginning or end of a slice, CTU / CTB, CTU / CTB line, image, block, sequence, and / or any predefined unit. Timing candidates can come from a buffer that stores filter information for a reference location in a reference frame (or reference image) and / or a predefined co-location image, and / or filter information for any predefined locations near the reference location. For example, the reference location is a co-location block within a co-location image in inter-frame prediction.

[0081] In other implementations, the reference location can be indicated by motion information from neighboring blocks or any predefined blocks associated with the current pixel block. Propagation candidates can be filter information from one or more reference locations indicated by the motion information of neighboring blocks or any predefined blocks associated with the current pixel block. In one case, a predefined order is specified to examine all or any subset of the inherited candidates described above. If a certain inherited candidate cannot find filter information, it is skipped, and the first available filter information is used for the current pixel block.

[0082] In some implementations, a candidate list containing multiple inheritance candidates is constructed, each candidate providing a set of filter information for configuring the filter, including filter parameters. The size of this list can be fixed to a predefined number specified in the video codec standard. A predefined order is set, and all or any subset of the inheritance candidates are inserted into the list. If insufficient filter information can be found for all or any subset of the inheritance candidates, default filter information is inserted. In some implementations, the list size is adaptively adjusted based on the filter information available using all or any subset of the inheritance candidates.

[0083] In some implementations, inherited filter information is selected from a list as follows: If the total number of inherited candidates in a list containing multiple candidates is K, a signaling element is transmitted to select inherited filter information from the original list or the reordered list. This syntax may employ truncated unary encoding and decoding, using the shortest codeword to indicate the first candidate in the list.

[0084] The list can be reordered (or sorted) based on the measurement results of each candidate on the template. The measurement results of the candidates may depend on the distortion between the reconstructed sample and the predicted sample of the template, where the predicted sample is generated by applying the filter information of the candidate to the template. During the measurement process, candidates with smaller distortion on the template are considered foreground candidates and are reordered to the front of the list. In some implementations, it is not necessary to indicate the candidates in the reordering list via syntax elements. During the measurement process, only the most foreground candidates are retained, and after checking each candidate in the list, the most foreground candidate with the smallest distortion is selected as the candidate for the current pixel block.

[0085] In some implementations, when generating the prediction sample to be filtered at the top-left corner of the current pixel block, all input samples for filtering use reconstructed samples from neighboring pixel blocks. When generating the prediction sample to be filtered near the top and / or left boundary of the current pixel block, some input samples for filtering use reconstructed samples from neighboring pixel blocks, and some input samples use prior prediction samples within the current pixel block. On the other hand, when generating the prediction sample to be filtered within the current pixel block, and / or when generating the prediction sample to be filtered away from the top and / or left boundary of the current pixel block, all input samples for filtering use prior prediction samples within the current pixel block. Figure 13A -B indicates that the filterable prediction samples are generated at different positions in the current pixel block. Figure 13A This shows the samples to be filtered and predicted that are close to the upper boundary of the current pixel block. Figure 13B This displays the samples to be filtered for prediction that are inside and / or far from the upper and left boundaries of the current pixel block.

[0086] In some implementations, an intra-frame prediction mode based on the target filter is used to generate the luma prediction result. Therefore, the current pixel block refers to the luma component. In some implementations employing a single-tree structure, the current pixel block is a luma codec block (CB) composed of one or more luma codec blocks (CBs) and one or more chroma codec blocks (CBs) within the codec unit. In some implementations employing a dual-tree structure, the current pixel block is a luma codec block (CB) composed of one or more luma codec blocks (CBs) within the codec unit.

[0087] In some implementations, an intra-frame prediction mode based on the target filter is used to generate chroma prediction results. Therefore, the current pixel block may refer to one or more chroma components, such as Cb and / or Cr. With a single-tree structure, the current pixel block is a chroma codec block (CB) composed of one or more luma codec blocks (CBs) and one or more chroma codec blocks (CBs) in the codec unit. With a dual-tree structure, the current pixel block is a chroma codec block (CB) composed of one or more chroma codec blocks (CBs) in the codec unit.

[0088] B. Non-intra-frame prediction mode based on target filter When the current pixel block belongs to a non-intra-frame coding mode, block vector information (including block vectors) and / or motion vector information (including motion vectors and / or a reference image) is used to point to a reference block. If the non-intra-frame coding mode of the current pixel block is inter-frame prediction, the motion information of the current pixel block points to a reference block in the reference image or a reference block in any predefined co-located image different from the current image. If the non-intra-frame coding mode of the current pixel block is intra-block copy (IBC) and / or intra-template matching prediction (intra-TMP), where intra-template matching prediction involves searching within a predefined region and finding a reference block (whose template matches the template of the current pixel block) to predict the current pixel block, the block vector information of the current pixel block points to a reference block in the current image. Similar to the target filter-based intra-frame prediction mode, the reference block of the current pixel block is used to determine the filter (including the determination of the filter shape and / or filter parameters).

[0089] In some implementations, the video codec utilizes a template of a reference block (i.e., a neighboring region) to derive `filter_ref` by applying a derivation process based on that template in the derivation path. This involves using reconstructed samples from templates of neighboring reference blocks as "golden samples" for filter parameter derivation, rather than using reconstructed samples from templates of the current pixel block. The video codec then applies `filter_ref` to derive a prediction for the reference block (denoted as `filter_pred_ref`). This prediction (`filter_pred_ref`) can then be used to determine the prediction for the current pixel block.

[0090] Figure 14A -B illustrates the process of deriving filter predictions based on the reference block when the current pixel block is encoded and decoded in a non-intra-frame manner. Figure 14AAn example is shown where the current pixel block 1410 in the current image has a motion vector (MV) pointing to a reference block 1420 in the reference image. Reconstructed samples from template regions 1425 of neighboring reference blocks are used to derive the filter parameters of filter_ref. The filter_ref is then used to generate a prediction (filter_pred_ref) for reference block 1420. This prediction (filter_pred_ref) can be used to determine the prediction for the current pixel block 1410.

[0091] Figure 14B An example is shown where the current pixel block 1410 has a block vector (BV) pointing to a reference block 1430 in the current image. Reconstructed samples from template regions 1435 adjacent to the reference block are used to derive the filter parameters of filter_ref. The filter_ref is then used to generate a prediction (filter_pred_ref) of the reference block 1430. This prediction (filter_pred_ref) can be used to determine the prediction of the current pixel block 1410.

[0092] In some implementations, besides the template of the current pixel block or a template unrelated to the current pixel block, the video codec uses template samples from neighboring reference blocks and / or samples from the reference block itself as all or part of the filter input to derive the filter parameters of `filter_ref` (i.e., using reconstructed samples from templates of neighboring current pixel blocks as "golden samples" for filter parameter derivation). The video codec then applies `filter_ref` to generate a prediction `filter_pred_ref` for the current pixel block. Here, `filter_ref` can use input from the reference block. This prediction (`filter_pred_ref`) can then be used to determine the prediction for the current pixel block.

[0093] In some implementations, filter_pred_ref is used as the prediction for the current pixel block.

[0094] In some implementations, filter_ref can be used as input to the filter along with samples within the current pixel block or samples from neighboring current pixel blocks to generate a prediction for the current pixel block.

[0095] In some implementations, the filter prediction of the current pixel block using the intra-frame prediction mode based on the target filter is combined with filter_pred_ref, and the combined filter prediction is used to generate the final prediction of the current pixel block.

[0096] In some implementations, a prediction residual (R) of the reference block is generated (calculated by comparing `filter_pred_ref` with reconstructed samples of the reference block), and this prediction residual is used to correct the filter prediction of the current pixel block obtained using the intra-frame prediction mode based on the target filter. In some implementations, the prediction residual is calculated as the difference between `filter_pred_ref` and reconstructed samples of the reference block, and the filter prediction obtained using the intra-frame prediction mode based on the target filter is adjusted by adding or subtracting this prediction residual. The adjusted filter prediction is used to generate the final prediction for the current pixel block.

[0097] In some implementations, for a filter (filter_ref) determined using a reference block, its filter parameters can be determined by inheriting filter information from previously encoded / decoded blocks through an inheritance process in the inheritance path (as described in Section IV.A above), where this inheritance process is based on the reference block rather than the current pixel block. For example, spatially adjacent or spatially non-adjacent candidates in the candidate list can come from blocks that are adjacent to or not adjacent to the reference block, rather than from blocks that are adjacent to or not adjacent to the current pixel block. Then, filter_pred_ref is derived by applying filter_ref to the reference block or applying filter_ref to the current pixel block, and this filter_pred_ref can be used to generate the final prediction for the current pixel block.

[0098] In some implementations, the process of deriving filter_pred_ref based on the reference block is similar to the process described in Section IV.A regarding applying the filter to each filter-predicted sample in the current pixel block for the target filter-based intra-prediction mode. Furthermore, the signaling methods related to filter determination in the derivation and / or inheritance paths may also be consistent with or similar to the methods mentioned in Section IV.A above for the target filter-based intra-prediction mode.

[0099] C. Prediction settings for the current pixel block In some implementations, prediction settings are used to generate a final prediction for the current pixel block. For a current pixel block encoded or decoded in a target intra-frame mode or a target non-intra-frame mode, the prediction settings use at least one of the following four methods to determine the final prediction for the current pixel block: 1. Color components are predicted using only one filter prediction (or one or more filter prediction hypotheses) from the target filter-based intra-prediction mode and / or the target filter-based non-intra-prediction mode.

[0100] 2. Color components are predicted using a combined prediction of existing prediction hypotheses from non-intra-prediction modes and one or more additional filter prediction hypotheses from target-filter-based intra-prediction modes.

[0101] 3. Color components are predicted using a combined prediction of existing prediction hypotheses from non-intra-prediction modes and one or more additional filter prediction hypotheses from non-intra-prediction modes based on target filters.

[0102] 4. Predict color components using a combined prediction of existing prediction hypotheses from an intra-prediction mode (which may be any predefined or existing intra-prediction mode) and one or more filter prediction hypotheses attached from a target-filter-based intra-prediction mode and / or a target-filter-based non-intra-prediction mode.

[0103] In some implementations, if the non-intra-frame mode is an inter-frame prediction mode, the existing prediction assumptions from the non-intra-frame prediction mode can be motion-compensated prediction signals. The generation of these motion-compensated prediction signals follows the original process of using the inter-frame prediction mode of the non-intra-frame mode.

[0104] In some implementations, if the non-intra-frame mode is Intra Block Copy (IBC) and / or Intra TMP, the existing prediction assumption from the non-intra-frame prediction mode can be a block vector prediction signal, wherein the mode searches for and finds a reference block (whose template matches well with the template of the current pixel block) within a predefined region to predict the current pixel block. The generation of the block vector prediction signal follows the original process of using the block vector prediction mode of the non-intra-frame mode.

[0105] In some implementations, a predefined processing procedure (e.g., gradient analysis, such as DIMD) is performed on the data of the current pixel block (e.g., prediction samples or reconstructed samples) to obtain a representative prediction pattern (e.g., a representative intra-frame prediction pattern). This representative prediction pattern is stored and / or referenced by the current pixel block or subsequent blocks.

[0106] In some implementations, if the predefined processing flow is DIMD, gradient analysis is applied to the member predictions (e.g., filter predictions) of the final prediction (e.g., merge prediction or refine prediction) or to the final prediction itself to obtain a representative intra-prediction pattern with the highest histogram value.

[0107] In some embodiments, the current pixel block selects the transform set and / or transform kernel and / or transform transpose flag for the transform process based on a representative intra-frame prediction mode. The transform process can refer to an inseparable or separable quadratic transform and / or an inseparable or separable primary transform. The transform process can also refer to a low-frequency inseparable transform. The transform process can also refer to an inseparable primary transform. The transform process can also refer to multiple transform sets or multiple transform selections.

[0108] In some embodiments, the filter prediction from the target filter-based intra-prediction mode may be the filter prediction or adjusted filter prediction described in Section IV.A and / or Section IV.B.

[0109] In some embodiments, when using method 1, 2, 3, or 4 described in this section, a fixed weighting value is set when merging forecasts. In some embodiments, the weight of the existing forecasting hypothesis is greater than the weight of the additional forecasting hypothesis. For example, the weight of the existing forecasting hypothesis is 3, and the weight of the additional forecasting hypothesis is 1. In some embodiments, the weight of the existing forecasting hypothesis is less than the weight of the additional forecasting hypothesis. For example, the weight of the existing forecasting hypothesis is 1, and the weight of the additional forecasting hypothesis is 3.

[0110] In some embodiments, when using method 1, 2, 3, or 4 described in this section, the weighting value varies based on the block position, width, height, area, and / or template processing of the current pixel block during merging prediction.

[0111] In some embodiments, when using method 1, 2, 3, or 4 described in this section, multiple prediction hypotheses (including at least one filter prediction) are combined to form a final prediction for the current pixel block. The weighting values ​​used in the combination vary depending on the block location, width, height, area, stencil processing, and / or segmentation mode of the current pixel block. This segmentation mode can geometrically divide the current pixel block into multiple regions.

[0112] Figure 15 This illustrates how the current pixel block is geometrically segmented into multiple regions using a segmentation mode. As shown, the current pixel block 1500 is segmented into a first region, a second region, and a third region. In the first region, a first filter is used to generate the prediction result. In the second region, a second filter is used to generate the prediction result. In the third region (the blending region), which contains only the samples to be predicted near the segmentation lines determined by the segmentation mode, the prediction results generated using the first filter and the prediction results generated using the second filter are combined using a weighted method to form the merged prediction result.

[0113] For example, in some embodiments, the segmentation line is uniform with or similar to the GPM segmentation boundary. In some embodiments, the segmentation line is represented by distance and / or angle. In some embodiments, the segmentation mode is determined by implicit and / or explicit conditions. Implicit conditions may rely on predefined rules or mapping tables in a standard and / or the width and / or height and / or area of ​​the current pixel block to obtain the segmentation mode of the current pixel block. Explicit conditions may rely on a signaling index to select a segmentation mode from multiple candidate segmentation modes.

[0114] In some embodiments, for a sample to be predicted in a first region, the prediction weight from the second filter is set to 0, while the prediction weight from the first filter is set to the maximum weight value in the weighting; and / or for a sample to be predicted in a second region, the prediction weight from the first filter is set to 0, while the prediction weight from the second filter is set to the maximum weight value in the weighting.

[0115] In some embodiments, for a sample to be predicted in the third region, the weight value is derived based on the segmentation mode, explicit signaling (e.g., explicit signaling of the weight index and / or explicit signaling of the segmentation mode), the width of the current pixel block, the height of the current pixel block, and / or the block area of ​​the current pixel block.

[0116] In some embodiments, the size of the third region is determined based on an explicit weight index or implicitly determined weight option, the width of the current pixel block, the height of the current pixel block, and / or the block area of ​​the current pixel block. An explicit weight index can indicate one of several weight candidates. For example, one candidate is a regular weight option, such as the VVC GPM weight design used to derive the size of the third region. More candidates come from a set of sizes including “1 / k multiplied by a regular weight option” (referring to a smaller blending region size) and / or “k multiplied by a regular weight option” (referring to a larger blending region size), where k is a positive integer greater than 1.

[0117] Implicitly determined weight options can be selected from multiple weight candidates. For example, one candidate is a regular weight option, such as the VVC GPM weight design used to derive the third region size. More candidates come from a set of sizes including "1 / k multiplied by a regular weight option" (referring to a smaller blending region size) and / or "k multiplied by a regular weight option" (referring to a larger blending region size), where k is a positive integer greater than 1.

[0118] In some embodiments, the position of the third region is derived based on an explicit segmentation mode index or an implicitly determined segmentation option, the width of the current pixel block, the height of the current pixel block, and / or the block area of ​​the current pixel block. In some embodiments, the current pixel block consists of a first region, a second region, and a third region, meaning the third region is smaller than the current pixel block. In some embodiments, the current pixel block consists of a first region and a second region, meaning there is no blending region in the current pixel block.

[0119] D. Syntax and storage of the current pixel block Enabling the methods described in Section IV can rely on explicit signaling and / or implicit checks. As an example of explicit signaling, for the current pixel block using a target intra-frame mode or a target non-intra-frame mode, in addition to the existing syntax, an additional flag is signaled to indicate whether the proposed filter prediction is used to generate the final prediction result for the current pixel block. For example, the existing syntax could indicate any intra-frame mode or non-intra-frame mode in the standard, such as a geometric segmentation mode (GPM) that combines inter-frame prediction or block vector prediction with intra-frame prediction (which, in some embodiments, is generated using filter prediction) or a GPM that combines intra-frame prediction with intra-frame prediction (which, in some embodiments, is generated using filter prediction).

[0120] In some embodiments, the method described in Section IV may be an option for existing tools. Therefore, the additional flag is signaled only when the existing syntax indication is enabled. When the additional flag indication is applied, a filter is obtained using candidate filters with derivation parameters (from the "derivation path") and / or inheritance parameters (from the "inheritance path"). In some embodiments, when a candidate filter is obtained from the inheritance path, an additional index is signaled to select a candidate filter from a list (which may be a reordered list or the original list). In some embodiments, when a candidate filter is obtained from the derivation path, an additional index is signaled to select a candidate filter from a plurality of candidate filters in the derivation path.

[0121] In some embodiments, when candidate filters are obtained from the inheritance path, a predefined candidate filter in the inheritance path is selected (without passing an additional index) to generate the filter prediction for the current pixel block. This predefined candidate filter may be a candidate filter at a predefined (e.g., first) position in the reordering list. In some embodiments, when candidate filters are obtained from both the inheritance path and the derivation path, there may be insufficient candidate filters in the inheritance path to reach the maximum list size. In this case, one or more predefined candidate filters from the derivation path can be inserted into the list, and / or an additional index can be signaled to select a candidate filter from the list (reordering list or original list).

[0122] In some embodiments, when obtaining candidate filters from the inheritance path and the derivation path, there may be no candidates in the list during the inheritance process, resulting in an empty list after checking all inherited candidates. In this case, predefined candidate filters from the derivation path can be used to generate filter predictions.

[0123] In some embodiments, when a target intra-frame mode or a target non-intra-frame mode is used and filter-based prediction is not employed, filter information is not stored for the current pixel block. On the other hand, when a target intra-frame mode or a target non-intra-frame mode is used and filter-based prediction is employed, all or any subset of filter information (filter shapes and / or filter parameters) is stored for the current pixel block, and / or subsequent blocks may reference all or any subset of the filter information (filter shapes and / or filter parameters). For example, the filter information stored for the current pixel block is inherited by subsequent blocks in their inheritance path.

[0124] Figure 16 A conceptual flowchart illustrating the enabling and use of target filter-based prediction is provided. This flowchart is process 1600, which can be executed by one or more processing units (e.g., a processor) of a computing device implementing a video encoder or video decoder (video codec). The video codec can execute process 1600 by executing instructions stored in a computer-readable medium. In some embodiments, process 1600 is executed by an electronic device implementing a video encoder or video decoder.

[0125] As shown in the figure, at block 1610, the video codec determines whether the current pixel block is encoded or decoded using the target intra-frame mode or a non-intra-frame mode. At block 1620, the video codec determines whether to use a prediction mode based on the target filter. If not, at block 1630, the video codec generates the final prediction result for the current pixel block using either raw intra-frame prediction or raw non-intra-frame prediction. If yes, at block 1640, the video codec generates the final prediction result for the current pixel block using filter prediction, and at block 1650, it stores one or more filter information for the current pixel block for reference in subsequent blocks.

[0126] The methods described in this specification can be enabled and / or disabled based on implicit rules (e.g., block width, height, or area) or explicit rules (e.g., syntax at the block, partition, slice, image, sequence parameter set (SPS), or picture parameter set (PPS) level). For example, the proposed methods are applied when the block area is less than / greater than a threshold. The term "block" in this disclosure can refer to a transform unit / transform block (TU / TB), a codec unit / codec block (CU / CB), a prediction unit / prediction block (PU / PB), a predefined region, or a codec tree unit / codec tree block (CTU / CTB).

[0127] Any combination of the methods proposed in this disclosure may be applied. Any of the proposed methods described above may be implemented in the encoder and / or decoder. For example, any of the proposed methods may be implemented in the inter-frame and / or intra-frame and / or block copy (IBC) and / or prediction and / or transform modules of the encoder, and / or in the inter-frame and / or intra-frame and / or block copy (IBC) and / or prediction and / or transform modules of the decoder. Alternatively, any of the proposed methods may be coupled as circuitry to the inter-frame and / or intra-frame and / or block copy (IBC) and / or prediction and / or transform modules of the encoder, and / or the inter-frame and / or intra-frame and / or block copy (IBC) and / or prediction and / or transform modules of the decoder, in order to provide the required information to the inter-frame and / or intra-frame and / or block copy (IBC) and / or prediction and / or transform modules.

[0128] V. Example Video Encoder Figure 17 An example video encoder 1700 capable of implementing filter-based prediction is illustrated. As shown, the video encoder 1700 receives an input video signal from a video source 1705 and encodes the signal into a bitstream 1795. The video encoder 1700 has multiple components or modules for encoding the signal from the video source 1705, including at least several components selected from: a transform module 1710, a quantization module 1711, an inverse quantization module 1714, an inverse transform module 1715, an intra-frame image estimation module 1724, an intra-frame prediction module 1725, a motion compensation module 1730, a motion estimation module 1735, a loop filter 1745, a reconstructed image buffer 1750, a motion vector buffer 1765, a motion vector prediction module 1775, and an entropy encoder 1790. The motion compensation module 1730 and the motion estimation module 1735 belong to the inter-frame prediction module 1740. Intra-frame prediction module 1725 and intra-frame prediction estimation module 1724 belong to the current image prediction module 1720, which uses the current image reconstruction sample as the reference sample for predicting the current pixel block.

[0129] In some embodiments, modules 1710–1790 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 1710–1790 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1710–1790 are illustrated as independent modules, some of these modules may be combined into a single module.

[0130] Video source 1705 provides an uncompressed raw video signal, presenting pixel data for each video frame. Subtractor 1708 calculates the difference between the raw video pixel data of video source 1705 and the predicted pixel data 1713 of motion compensation module 1730 or intra-frame prediction module 1725, as the prediction residual 1709. Transform module 1710 converts this difference (or residual pixel data or residual signal 1708) into transform coefficients (e.g., by performing a Discrete Cosine Transform, or DCT for short). Quantization module 1711 quantizes the transform coefficients into quantized data (or quantization coefficients) 1712, which is then encoded into bitstream 1795 by entropy encoder 1790.

[0131] The inverse quantization module 1714 inverse-quantizes the quantized data (or quantization coefficients) 1712 to obtain transform coefficients 1718. The inverse transform module 1715 performs an inverse transform on the transform coefficients 1718 to generate a reconstruction residual 1719. The reconstruction residual 1719 is added to the predicted pixel data 1713 to generate reconstructed pixel data 1717. In some embodiments, the reconstructed pixel data 1717 is temporarily stored in the line buffer 1727 (or the intra-frame prediction buffer) for intra-frame image prediction and spatial motion vector prediction. After being filtered by the loop filter 1745, the reconstructed pixels are stored in the reconstructed image buffer 1750. In some embodiments, the reconstructed image buffer 1750 is external memory to the video encoder 1700. In some embodiments, the reconstructed image buffer 1750 is internal memory to the video encoder 1700.

[0132] Intra-frame image estimation module 1724 performs intra-frame prediction based on reconstructed pixel data 1717 to generate intra-frame prediction data. This intra-frame prediction data is provided to entropy encoder 1790 for encoding into bitstream 1795. This intra-frame prediction data is also used by intra-frame prediction module 1725 to generate predicted pixel data 1713.

[0133] The motion estimation module 1735 generates motion vectors (MVs) by referencing pixel data of previously decoded frames stored in the reconstructed image buffer 1750 to perform inter-frame prediction. These motion vectors are then provided to the motion compensation module 1730 to generate predicted pixel data.

[0134] The video encoder 1700 does not encode the complete actual motion vector into the bitstream. Instead, it uses motion vector prediction to generate predicted motion vectors and encodes the difference between the motion vector used for motion compensation and the predicted motion vector as residual motion data, which is stored in bitstream 1795.

[0135] The motion vector prediction module 1775 generates predicted motion vectors based on reference motion vectors generated from the encoding of previous video frames. These reference motion vectors are the motion compensation motion vectors used to perform motion compensation. The motion vector prediction module 1775 retrieves reference motion vectors from previous video frames from the motion vector buffer 1765. The video encoder 1700 stores the motion vectors generated for the current video frame in the motion vector buffer 1765 as reference motion vectors for generating the predicted motion vectors.

[0136] The motion vector prediction module 1775 creates a predicted motion vector using a reference motion vector. The predicted motion vector can be calculated via spatial motion vector prediction or temporal motion vector prediction. The difference (residual motion data) between the predicted motion vector and the motion-compensated motion vector (MC motion vector) of the current frame is encoded into the bitstream 1795 by the entropy encoder 1790.

[0137] The entropy encoder 1790 encodes various parameters and data into a bitstream 1795 using entropy encoding / decoding techniques such as Context-Adaptive Binary Arithmetic Coding (CABAC) or Huffman coding. The entropy encoder 1790 encodes various header elements, flags, quantization transform coefficients 1712, and residual motion data as syntax elements into the bitstream 1795. The bitstream 1795 is then stored in a storage device or transmitted to the decoder via a communication medium such as a network.

[0138] The loop filter 1745 performs filtering or smoothing operations on the reconstructed pixel data 1717 to reduce encoding / decoding artifacts, especially at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by the loop filter 1745 include a deblocking filter (DBF), sample adaptive offset (SAO), and / or adaptive loop filter (ALF). In some embodiments, luminance-mapped chrominance scaling (LMCS) is performed before the loop filter.

[0139] Figure 18This section illustrates the portion of the video encoder 1700 that implements either a target filter-based intra-prediction mode or a non-intra-prediction mode. The filter derivation module 1805 derives the filter parameters for either the target filter-based intra-prediction mode or the non-intra-prediction mode. This derivation can be performed using the reconstructed image buffer 1750 and / or the line buffer 1727 for regression calculation. For the target filter-based intra-prediction mode, the filter derivation module 1805 retrieves reconstructed samples of template regions adjacent to the current pixel block for regression calculation. For the target filter-based non-intra-prediction mode, the filter derivation module 1805 retrieves reconstructed samples of a reference block or reconstructed samples of template regions adjacent to the reference block for regression calculation. The reference block can be located in the reference image using motion vectors or in the current image using block vectors. The motion vectors and / or block vectors can be obtained by the entropy encoder 1790 from the inter-frame prediction module 1740 or the IBC module (not shown). The filter parameters derived by the filter derivation module 1805 can be stored in the filter parameter memory 1810 for inheritance by subsequent blocks. In some embodiments, the entropy encoder 1790 controls whether to execute an intra-frame prediction mode based on a target filter or a non-intra-frame prediction mode based on a target filter.

[0140] Filter selector 1825 selects one or more filters 1820 to generate one or more filter predictions 1830. If entropy encoder 1790 indicates that the current pixel block inherits candidate filters (“inheritance path”), filter selector 1825 may select one or more filters from filter parameter memory 1810. Filter selector 1825 may also select currently derived filter parameters generated for the current pixel block by filter derivation module 1805. The selected filter 1820 then uses reconstructed samples provided by reconstructed image buffer 1750 or line buffer 1727 as filter input to generate the corresponding filter prediction 1830. The samples used as filter input may be template region samples adjacent to the current pixel block, template region samples adjacent to a reference block, or samples from the reference block itself.

[0141] Filter prediction 1830 generates predicted pixel data 1713 as the final prediction. Prediction fusion module 1850 can perform weighted merging of different predictions, including incorporating filter prediction 1830 as part of a multi-hypothesis prediction, or incorporating different filter predictions as predictions for different partitions of the current pixel block according to a segmentation mode (e.g., GPM).

[0142] In some embodiments, filter prediction 1830 or final prediction may be used by a predefined processing flow (e.g., DIMD) to determine a representative intra-prediction mode. The representative intra-prediction mode can be used to determine transform settings for configuring transform module 1710 (e.g., transform set, transform kernel, transform transpose flag, etc.). The representative intra-prediction mode may be stored and / or used by subsequent blocks.

[0143] Figure 19 A conceptual illustration of a process 1900 for encoding pixel blocks using a prediction mode based on a target filter is presented. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 1700 execute process 1900 by executing instructions stored in a computer-readable medium. In some embodiments, process 1900 is executed by an electronic device implementing encoder 1700.

[0144] The encoder receives data to be encoded at block 1910, representing the current pixel block in the current image. The current pixel block can be encoded using motion vectors or block vectors, or via a geometric segmentation pattern. At block 1920, the encoder determines filter parameters for generating target sample predictions. In some embodiments, the filter parameters are derived based on samples from a reference block or samples from a template region of a neighboring reference block or the current pixel block. These parameters can be derived based on a reference block located by the motion vector or block vector of the current pixel block. The filter parameters can be inherited from candidates in a candidate list, each candidate providing a set of filter information for configuring the filter.

[0145] The encoder applies the filter at block 1930 to generate a filter prediction for the current pixel block. In some embodiments, the encoder uses samples from a reference block or samples from a neighboring reference block or the current pixel block as filter inputs and applies the filter to generate a filter prediction for the current pixel block.

[0146] The encoder generates a final prediction for the current pixel block based on filter-based prediction at block 1940. A flag can be signaled to indicate whether the filter prediction was used to generate the final prediction. In some implementations, the final prediction is generated by combining the filter prediction with a non-intra-frame prediction generated based on the motion vector or block vector of the current pixel block. The weighting of this combination can be determined based on block position, width, height, area, or the segmentation mode of the current pixel block.

[0147] In some implementations, the current pixel block can be segmented into at least a first region and a second region by geometric segmentation (e.g., by a geometrical partitioning mode (GPM)), such that the filter is used to generate a prediction result for at least one of the first region and the second region.

[0148] At block 1950, the encoder encodes the current pixel block using this final prediction to produce a prediction residual. In some implementations, the encoder may apply a predefined processing flow to this final prediction to determine a representative intra-prediction mode. This predefined processing flow may be a gradient-analysis-based DIMD. The encoder can use this representative intra-prediction mode to determine one or more transform settings (e.g., transform set and / or transform kernel and / or transform transpose flag, for the transform process) for encoding the current pixel block. This representative intra-prediction mode may be stored for reference in subsequent blocks.

[0149] VI. Example Video Decoder In some implementations, the encoder may signal (or generate) one or more syntax elements in the bitstream, so that the decoder can parse the one or more syntax elements from the bitstream.

[0150] Figure 20 A video decoder 2000 capable of filter-based prediction is demonstrated. As shown, the video decoder 2000 is an image decoder or video decoder that receives a bitstream 2095 and decodes its contents into pixel data of video frames for display. The video decoder 2000 has multiple components or modules for decoding the bitstream 2095, including several components selected from: an inverse quantization module 2014, an inverse transform module 2015, an intra-frame prediction module 2025, a motion compensation module 2030, a loop filter 2045, a decoded image buffer 2050, a motion vector buffer 2065, a motion vector prediction module 2075, and a resolver 2090. The motion compensation module 2030 is part of the inter-frame prediction module 2040. The intra-frame prediction module 2025 is part of the current image prediction module 2020, which uses reconstructed samples of the current image as reference samples for predicting the current pixel block.

[0151] In some implementations, modules 2014–2090 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some implementations, modules 2014–2090 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 2014–2090 are illustrated as independent modules, some of these modules may be combined into a single module.

[0152] Parser 2090 (or entropy decoder) receives bitstream 2095 and performs initial parsing according to the syntax defined by the video or image codec standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 2012. Parser 2090 then uses entropy coding techniques, such as Context-Adaptive Binary Arithmetic Coding (CABAC) or Huffman coding, to further parse the syntax elements.

[0153] The inverse quantization module 2014 inverse-quantizes the quantized data (or quantization coefficients) 2012 to obtain transform coefficients. The inverse transform module 2015 inverse-transforms the transform coefficients 2018 to generate a reconstructed residual signal 2019. The reconstructed residual signal 2019 is added to the predicted pixel data 2013 from the intra-frame prediction module 2025 or the motion compensation module 2030 to generate decoded pixel data 2017. The decoded pixel data is filtered by the loop filter 2045 and then stored in the decoded image buffer 2050. In some embodiments, the decoded image buffer 2050 is an external memory of the video decoder 2000. In some embodiments, the decoded image buffer 2050 is an internal memory of the video decoder 2000.

[0154] Intra-frame prediction module 2025 receives intra-frame prediction data from bitstream 2095 and generates prediction pixel data 2013 based on the intra-frame prediction data and using the decoded pixel data 2017 stored in decoded image buffer 2050. In some embodiments, the decoded pixel data 2017 is also stored in line buffer 2027 (or intra-frame prediction buffer) for intra-frame image prediction and spatial motion vector prediction.

[0155] In some embodiments, the contents of the decoded image buffer 2050 are displayed. The display device 2005 can directly extract the contents of the decoded image buffer 2050 for display, or it can extract the contents of the decoded image buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 2050 via pixel transfer.

[0156] The motion compensation module 2030 generates predicted pixel data 2013 based on the motion compensation motion vectors (MC MVs) and the decoded pixel data 2017 stored in the decoded image buffer 2050. These motion compensation motion vectors are obtained by decoding the residual motion data received from the bitstream 2095 and the predicted motion vector received from the motion vector prediction module 2075.

[0157] The motion vector prediction module 2075 generates predicted motion vectors based on reference motion vectors generated from decoding the previous video frame. These reference motion vectors are the motion compensation motion vectors used to perform motion compensation. The motion vector prediction module 2075 extracts the reference motion vectors of the previous video frame from the motion vector buffer 2065. The video decoder 2000 stores the motion compensation motion vectors generated from decoding the current video frame in the motion vector buffer 2065 as reference motion vectors for generating the predicted motion vectors.

[0158] The loop filter 2045 performs filtering or smoothing operations on the decoded pixel data 2017 to reduce encoding / decoding artifacts, particularly at pixel block boundaries. In some implementations, the filtering or smoothing operations performed by the loop filter 2045 include a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF). In some implementations, luma mapping chroma scaling (LMCS) is performed before the loop filter.

[0159] Figure 21 This demonstrates a portion of the video decoder 2000 that implements intra-frame prediction mode or non-intra-frame prediction mode based on a target filter. A filter derivation module 2105 is used to derive filter parameters for the intra-frame prediction mode or non-intra-frame prediction mode based on the target filter. This derivation can be performed using regression calculations in the decoded image buffer 2050 and / or line buffer 2027. For the intra-frame mode based on the target filter, the filter derivation module 2105 can retrieve reconstructed samples of template regions adjacent to the current pixel block for regression calculations. For the non-intra-frame mode based on the target filter, the filter derivation module 2105 can retrieve reconstructed samples of a reference block or reconstructed samples of template regions adjacent to the reference block for regression calculations. The reference block can be located in the reference image by motion vector (MV) or in the current image by block vector (BV). MV and / or BV can be obtained by the entropy decoder 2090 from the bitstream 2095. The filter parameters derived by the filter derivation module 2105 can be stored in the filter parameter memory 2110 for inheritance by subsequent blocks. In some embodiments, the entropy decoder 2090 controls whether to execute an intra-frame mode based on the target filter or a non-intra-frame mode based on the target filter.

[0160] Filter selector 2125 selects one or more filters 2120 to generate one or more filter predictions 2130. If entropy decoder 2090 indicates that the current pixel block inherits candidate filters (“inheritance path”), filter selector 2125 may select one or more filters from filter parameter memory 2110. Filter selector 2125 may also select currently derived filter parameters generated for the current block by filter derivation module 2105. The selected filter 2120 then uses reconstructed samples provided by decoded image buffer 2050 or line buffer 2027 as filter input to generate the corresponding filter prediction 2130. The samples used as filter input may be samples from template regions adjacent to the current pixel block, samples from template regions adjacent to a reference block, or samples from the reference block itself.

[0161] Filter prediction 2130 is used to generate predicted pixel data 2013 as the final prediction. Prediction fusion module 2150 can perform weighted merging of different predictions, including incorporating filter prediction 2130 as part of a multi-hypothesis prediction, or incorporating different filter predictions as predictions for different partitions of the current pixel block according to a segmentation mode (e.g., geometric segmentation mode (GPM)).

[0162] In some embodiments, filter prediction 2130 or final prediction may be used by a predefined processing flow (e.g., DIMD) to determine a representative intra-prediction mode. The representative intra-prediction mode can be used to determine transform settings (e.g., transform set, transform kernel, transform transpose flag, etc.) for configuring the inverse transform module 2015. The representative intra-prediction mode may be stored and / or used by subsequent blocks.

[0163] Figure 22 A conceptual illustration of process 2200 for decoding pixel blocks using a prediction mode based on a target filter is provided. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 2000 execute process 2200 by executing instructions stored in a computer-readable medium. In some embodiments, process 2200 is executed by an electronic device implementing decoder 2000.

[0164] At block 2210, the decoder receives the data to be decoded as the current pixel block in the current image. The current pixel block can be encoded or decoded using motion vectors or block vectors, or via a geometric segmentation pattern. At block 2220, the decoder determines filter parameters for generating target sample predictions. In some embodiments, the filter parameters are derived based on samples from a reference block or samples from a template region of a neighboring reference block or the current pixel block. These parameters can be derived based on a reference block located by the motion vector or block vector of the current pixel block. The filter parameters can also be inherited from candidates in a candidate list, each candidate providing a set of filter information for configuring the filter.

[0165] The decoder applies the filter at block 2230 to generate a filter prediction for the current pixel block. In some embodiments, the decoder uses samples from a reference block or neighboring reference blocks or the current pixel block as filter inputs and applies the filter to generate a filter prediction for the current pixel block.

[0166] At block 2240, the decoder generates a final prediction for the current pixel block based on the filter prediction. A flag can be signaled to indicate whether the filter prediction is used to generate the final prediction. In some embodiments, the final prediction is generated by combining the filter prediction with a non-intra-frame prediction generated based on the motion vector or block vector of the current pixel block. The weighting of this combination can be determined based on block location, width, height, area, or the segmentation mode of the current pixel block.

[0167] In some embodiments, the current pixel block can be geometrically segmented into at least a first region and a second region using a geometric segmentation mode (e.g., GPM), such that the filter is used to generate a prediction result for at least one of the first and second regions.

[0168] At block 2250, the decoder reconstructs the current pixel block using a final prediction with prediction residuals. The decoder can then display the reconstructed current pixel block as part of the reconstructed current image. In some embodiments, the decoder may apply a predefined processing flow to the filter prediction or final prediction to determine a representative intra-prediction mode. This predefined processing flow may be DIMD, including gradient analysis. The decoder may use the representative intra-prediction mode to determine one or more transform settings for decoding the current pixel block (e.g., transform set and / or transform kernel and / or transform transpose flags for the transform process). The representative intra-prediction mode may be stored for reference in subsequent blocks.

[0169] VII. Example Electronic System Many of the features and applications described above are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). When these instructions are executed by one or more computing units or processing units (e.g., one or more processors, processor cores, or other processing units), the processing unit performs the operations indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard disks, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted via wireless or wired connections.

[0170] In this specification, the term "software" includes firmware residing in read-only memory or application programs stored in magnetic memory, which can be read into memory for processor processing. Furthermore, in some embodiments, multiple software inventions may be implemented as sub-parts of a larger program while remaining independent software inventions. In some embodiments, multiple software inventions may also be implemented as independent programs. Finally, any combination of independent programs that collectively implement the software inventions of this disclosure is within the scope of this disclosure. In some embodiments, when a software program is installed on one or more electronic systems and runs, one or more specific machine implementation schemes are defined to perform the various operations of the software program.

[0171] Figure 23 This invention conceptually illustrates an electronic system 2300 that may implement certain embodiments of the present disclosure. The electronic system 2300 may be a computer (e.g., a desktop computer, personal computer, tablet computer, etc.), a telephone, a personal digital assistant (PDA), or any other type of electronic device. Such electronic systems include various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 2300 includes a bus 2305, a processing unit 2310, a graphics-processing unit (GPU) 2315, system memory 2320, a network 2325, read-only memory 2330, permanent storage device 2335, input device 2340, and output device 2345.

[0172] Bus 2305 represents all system, peripheral, and chipset buses used for communication connections to numerous internal devices of electronic system 2300. For example, bus 2305 communicates with processing unit 2310 and GPU 2315, read-only memory 2330, system memory 2320, and permanent storage device 2335.

[0173] Processing unit 2310 retrieves instructions to be executed and data to be processed from these different storage units to execute the processing flow of this disclosure. In different embodiments, the processing unit may be a single processor or a multi-core processor. Some instructions are passed to GPU 2315 and executed thereon. GPU 2315 may offload various computing or supplementary image processing provided by processing unit 2310.

[0174] Read-only memory (ROM) 2330 stores static data and instructions used by processing unit 2310 and other modules of the electronic system. Permanent storage device 2335 is a read-write storage device. This device is a non-volatile storage unit that can store instructions and data even when the electronic system 2300 is powered off. Some embodiments of this disclosure use mass storage devices (such as disks or optical discs and their corresponding disk drives) as permanent storage device 2335.

[0175] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Similar to permanent storage device 2335, system memory 2320 is also a read-write storage device. However, unlike storage device 2335, system memory 2320 is volatile read-write memory, such as random access memory. System memory 2320 stores some instructions and data used by the processor during runtime. In some embodiments, processing flows conforming to this disclosure are stored in system memory 2320, permanent storage device 2335, and / or read-only memory 2330. For example, various storage units store instructions, including those for processing multimedia segments in some embodiments, and then processing unit 2310 retrieves the instructions to be executed and the data to be processed from these different storage units to execute the processes of some embodiments.

[0176] Bus 2305 also connects input device 2340 and output device 2345. Input device 2340 enables a user to pass information to the electronic system and select commands. Input device 2340 includes an alphanumeric keypad and pointer device (also known as a "cursor control device"), a camera (e.g., a webcam), a microphone, or similar device for receiving voice commands. Output device 2345 displays images generated by the electronic system or otherwise outputs data. Output device 2345 includes printers and display devices such as cathode ray tube (CRT) or liquid crystal display (LCD), as well as speakers or similar audio output devices. Some embodiments include devices such as touchscreens that function as both input and output devices.

[0177] Finally, as Figure 23As shown, bus 2305 also connects electronic system 2300 to network 2325 via a network adapter (not shown). In this way, the computer can become part of a computer network (such as a local area network (LAN), wide area network (WAN), or intranet, or a network of networks such as the Internet). Any or all components of electronic system 2300 can be used in conjunction with this disclosure.

[0178] Some embodiments include electronic components, such as microprocessors, storage, and memory, that store computer program instructions in a machine-readable or computer-readable medium (also referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Examples of such computer-readable media include random access memory (RAM), read-only memory (ROM), read-only optical disc (CD-ROM), recordable optical disc (CD-R), rewritable optical disc (CD-RW), read-only digital versatile optical discs (such as DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (such as DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (such as SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state drives, read-only and recordable Blu-ray discs, ultra-high-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable medium may store a computer program executable by at least one processing unit and contain a set of instructions for performing various operations. Examples of computer programs or computer code include machine code (such as code generated by a compiler) and files containing high-level code that is executed by a computer, electronic component, or microprocessor via an interpreter.

[0179] While the foregoing discussion primarily concerns microprocessors or multi-core processors that execute software, many of the aforementioned features and applications are implemented by one or more integrated circuits, such as Application Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself. Furthermore, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0180] In this specification and any claim of this disclosure, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include individuals or groups. For the purposes of this specification, the term "display" or "show" means "displayed on an electronic device." In this specification and any claim of this disclosure, the terms "computer-readable medium," "computer-readable media," and "machine-readable medium" are fully defined as tangible, physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0181] Although this disclosure has been described in conjunction with numerous specific details, those skilled in the art will recognize that this disclosure may be implemented in other specific forms without departing from its spirit. Furthermore, many figures (including...) Figure 16 , Figure 19 and Figure 22 This provides a conceptual description of the process. The specific operations of these processes may not be performed in the order shown and described. Specific operations may not be performed in a continuous sequence of operations; different specific operations may be performed in different embodiments. Furthermore, the process may be implemented through multiple sub-processes or as part of a larger macro-process. Therefore, those skilled in the art will understand that this disclosure should not be limited to the foregoing illustrative details but should be defined by the appended claims.

[0182] Additional Notes The topics described herein sometimes demonstrate different components contained within or connected to other different components. It should be understood that these illustrated architectures are merely examples, and many other architectures can actually be implemented to achieve the same functionality. Conceptually, any arrangement of components to achieve the same function is effectively “associated” to achieve the desired functionality. Therefore, any two components combined in this document to achieve a specific function can be considered “associated” with each other to achieve the desired functionality, regardless of the architecture or intermediate components. Similarly, any two components so associated can also be considered “operably connected” or “operably coupled” to achieve the desired functionality, and any two components that can be so associated can also be considered “operably coupled” to achieve the desired functionality. Specific examples of operable coupling include, but are not limited to, components that are physically matable and / or physically interact and / or wirelessly interact and / or logically interact and / or logically interactable.

[0183] Furthermore, regarding the use of almost all plural and / or singular terms in this document, those skilled in the art can appropriately convert plural to singular and / or singular to plural depending on the context and / or application. For clarity, various singular / plural permutations may be explicitly listed herein.

[0184] Furthermore, those skilled in the art will understand that, generally, the terms used herein, especially in appended claims, such as the body portion of appended claims, are generally considered "open" terms. For example, the word "comprising" should be interpreted as "including but not limited to," the word "having" should be interpreted as "having at least," and the word "including" should be interpreted as "including but not limited to," etc. Those skilled in the art will also understand that if a specific number of claim elements is expressly stated in the claim, then that intention is expressly stated in the claim; if it is not stated, then that intention does not exist. For example, for ease of understanding, the following appended claims may contain the use of the introductory phrases "at least one" and "one or more" to introduce claim elements. However, the use of such phrases should not be interpreted as limiting any particular claim containing that introduced claim element to containing only one of that element, even if the same claim contains the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "one," for example, "a" and / or "one" should be interpreted as "at least one" or "one or more"; the same applies to definite articles used to introduce claim elements. Furthermore, even when a specific number of elements in a claim is explicitly stated, those skilled in the art will recognize that such a statement should be interpreted as at least the stated number. For example, the phrase "two elements" alone, without any other modifiers, implies at least two elements, or two or more elements. Additionally, when using conventions such as "at least one A, B, and C, etc.", this structure is generally intended for those skilled in the art to understand the meaning of the convention. For example, "a system having at least one A, B, and C" includes, but is not limited to, systems with only A, only B, only C, A and B, A and C, B and C, and systems with A, B, and C, etc. Similarly, when using conventions such as "at least one A, B, or C, etc.", this structure is generally intended for those skilled in the art to understand the meaning of the convention. For example, "a system having at least one A, B, or C" includes, but is not limited to, systems with only A, only B, only C, A and B, A and C, B and C, and systems with A, B, and C, etc. Those skilled in the art will also understand that virtually any disjunctive words and / or phrases appearing in the specification, claims, or drawings, presenting two or more alternative terms, should be understood to include one, any, or both terms. For example, the phrase “A or B” should be understood as including the possibility of “A” or “B” or “A and B”.

[0185] As can be seen from the foregoing, various embodiments of this disclosure have been described herein for illustrative purposes, and various modifications can be made without departing from the scope and spirit of this disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are indicated by the following claims.

Claims

1. A video encoding / decoding method, comprising: Receive data to be encoded or data to be decoded as the current pixel block of the current video image; wherein, the current pixel block is encoded or decoded using motion vector, block vector, or geometric segmentation mode; Determine the parameters of the filter used to generate target sample predictions; Based on the filter, a filter prediction for the current pixel block is generated; The final prediction for the current pixel block is generated based on the filter prediction; and The current pixel block is encoded or decoded based on the final prediction.

2. The video encoding / decoding method according to claim 1, wherein, The parameters of the filter are derived based on a reference block, wherein the reference block is located based on the motion vector or block vector of the current pixel block.

3. The video encoding / decoding method according to claim 1, wherein, The parameters of the filter are inherited from the candidates in the candidate list, wherein each candidate provides a set of filter information for configuring the filter.

4. The video encoding and decoding method according to claim 1, wherein, The parameters of the filter are derived based on samples from the reference block; or, The parameters of the filter are derived based on samples from the template region adjacent to the reference block; or, The parameters of the filter are derived based on samples from the template region adjacent to the current pixel block.

5. The video encoding / decoding method according to claim 1, wherein, The filter prediction for generating the current pixel block based on the filter includes: Use samples from the reference block as input to the filter; or, Use samples adjacent to the reference block as input to the filter; or, The filter uses samples from neighboring pixels as input.

6. The video encoding / decoding method according to claim 1, wherein, The final prediction is generated by merging the filter prediction with a non-intra-frame prediction generated based on the motion vector or block vector of the current pixel block.

7. The video encoding / decoding method according to claim 6, wherein, The weight allocation in the merging process is determined based on the block position, width, height, area, or segmentation mode of the current block pixel.

8. The video encoding / decoding method according to claim 1, wherein, The current pixel block is geometrically segmented into at least a first region and a second region; wherein, a prediction result for at least one region in the first region and the second region is generated based on the filter.

9. The video encoding / decoding method according to claim 1, further comprising: A predefined processing flow is applied to the final prediction to determine the representative intra-frame prediction mode; One or more transform settings for encoding or decoding the current pixel block are determined based on the representative intra-frame prediction mode.

10. The video encoding / decoding method according to claim 9, wherein, The predefined processing flow includes gradient analysis.

11. The video encoding / decoding method according to claim 9, further comprising: The representative intra-frame prediction patterns are stored for reference in subsequent blocks.

12. The video encoding / decoding method according to claim 1, wherein, A flag indicates whether the filter is used to generate the final prediction.

13. An electronic device, comprising: The video codec circuit is configured to perform the following operations: Receive data to be encoded or data to be decoded as the current pixel block of the current video image; wherein, the current pixel block is encoded or decoded using motion vector, block vector, or geometric segmentation mode; Determine the filter parameters used to generate target sample predictions; Based on the filter, a filter prediction for the current pixel block is generated; The final prediction for the current pixel block is generated based on the filter prediction; and The current pixel block is encoded or decoded based on the final prediction.

14. A video decoding method, comprising: Receive the data to be decoded as the current pixel block of the current video image; wherein, the current pixel block is encoded and decoded using motion vector, block vector, or geometric segmentation mode; Determine the filter parameters used to generate target sample predictions; Based on the filter, a filter prediction for the current pixel block is generated; The final prediction for the current pixel block is generated based on the filter prediction; and The current pixel block is reconstructed based on the final prediction.

15. A video coding method, comprising: Receive data to be encoded as the current pixel block of the current video image; wherein the current pixel block is encoded and decoded using motion vector, block vector, or geometric segmentation modes; Determine the filter parameters used to generate target sample predictions; Based on the filter, a filter prediction for the current pixel block is generated; The final prediction for the current pixel block is generated based on the filter prediction; and The current pixel block is encoded based on the final prediction.

Images