Encoding method, decoding method, code stream, encoder, decoder, and storage medium

By optimizing the selection of the reference region for interpolation filtering in the video coding standard, and combining the size parameters of the current block and the target filtering mode, the filtering coefficients are calculated for intra-frame prediction, which solves the problem of low encoding and decoding performance and time complexity in the existing technology, and realizes a more efficient encoding and decoding process.

WO2024259568A1PCT designated stage expired Publication Date: 2024-12-26GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD

Patent Information

Application Number
PCT/CN2023/101156
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-06-19
Publication Date
2024-12-26

AI Technical Summary

Technical Problem

Existing video coding standards suffer from low encoding/decoding performance and time complexity due to interpolation-based intra-frame prediction techniques.

Method used

By determining the target filtering mode of the current block, and combining its size parameters and reference region, the filtering coefficients are calculated to perform intra-frame prediction, thereby optimizing the selection of the reference region for interpolation filtering, reducing computational complexity, and improving prediction accuracy.

Benefits of technology

While ensuring encoding and decoding performance, it reduces time complexity and improves the accuracy of intra-frame prediction and encoding and decoding efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023101156_26122024_PF_FP_ABST
    Figure CN2023101156_26122024_PF_FP_ABST
Patent Text Reader

Abstract

Embodiments of the present application disclose an encoding method, a decoding method, a code stream, an encoder, a decoder, and a storage medium. The decoding method, applied to a decoder, comprises: decoding a code stream, and determining a target filtering mode of a current block; determining a reference area of the current block on the basis of a size parameter of the current block and the target filtering mode; determining a filter coefficient of the current block on the basis of the reference area of the current block; and performing intra-frame prediction on the current block on the basis of the filter coefficient, and determining a predicted value of the current block. Therefore, the time complexity can be reduced while the encoding and decoding performance is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] As demand for video display quality increases, high-resolution video, such as HD and UHD, has emerged. However, high-resolution video typically contains more information and therefore requires more bandwidth. To reduce bandwidth requirements, video coding standards involving video compression have been introduced.

[0003] Currently, video coding standards have proposed intra-frame prediction techniques based on interpolation. Specifically, this technique uses the reconstructed pixel values ​​around the current block to obtain interpolation filter coefficients, which are then used to perform intra-frame prediction on the current block. However, existing technical solutions still have some shortcomings, resulting in a low cost-effectiveness in terms of encoding and decoding performance and time complexity.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium, which can reduce time complexity while ensuring coding and decoding performance.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Decode the code stream and determine the target filtering mode of the current block;

[0009] Determine a reference area of ​​the current block according to a size parameter of the current block and a target filtering mode;

[0010] Determining a filter coefficient of the current block according to a reference area of ​​the current block;

[0011] Perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0012] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0013] Determine the target filtering mode for the current block;

[0014] Determine a reference area of ​​the current block according to a size parameter of the current block and a target filtering mode;

[0015] Determining a filter coefficient of the current block according to a reference area of ​​the current block;

[0016] Perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0017] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:

[0018] A target filtering mode of a current block, a residual value of the current block, and a transform kernel index value of the current block.

[0019] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determination unit and a first prediction unit, wherein:

[0020] A first determining unit is configured to determine a target filtering mode of the current block; and determine a reference area of ​​the current block according to a size parameter of the current block and the target filtering mode;

[0021] The first prediction unit is configured to determine a filter coefficient of the current block according to a reference area of ​​the current block; and perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0022] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor; wherein,

[0023] a first memory for storing a computer program capable of running on the first processor;

[0024] The first processor is configured to execute the method according to the second aspect when running a computer program.

[0025] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a decoding unit, a second determining unit, and a second predicting unit, wherein:

[0026] a decoding unit configured to decode the bitstream and determine a target filtering mode for a current block;

[0027] a second determining unit configured to determine a reference area of ​​the current block according to a size parameter of the current block and a target filtering mode;

[0028] The second prediction unit is configured to determine a filter coefficient of the current block according to a reference area of ​​the current block; and perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0029] In a seventh aspect, an embodiment of the present application provides a decoder, the decoder comprising a second memory and a second processor; wherein,

[0030] a second memory for storing a computer program capable of running on the second processor;

[0031] The second processor is configured to execute the method according to the first aspect when running a computer program.

[0032] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed, implements the method described in the first aspect or the method described in the second aspect.

[0033] The embodiment of the present application provides a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium. Whether at the encoding end or the decoding end, after determining the target filtering mode of the current block, the reference area of ​​the current block is determined according to the size parameters of the current block and the target filtering mode; then, based on the reference area of ​​the current block, the filter coefficient of the current block is determined; and then, based on the filter coefficient, intra-frame prediction is performed on the current block to determine the predicted value of the current block. In this way, the intra-frame prediction technology based on interpolation filtering, when determining the reference area for calculating the filter coefficient, is not only related to the target filtering mode, but also to the size parameters of the current block. For example, when the size of the current block is large, a large reference area can be used, and when the size of the current block is small, a small reference area can be used. In this way, the computational complexity can be reduced and the encoding time can be reduced. At the same time, the accuracy of intra-frame prediction can be improved, thereby improving the coding and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1A is a schematic diagram of a calculation method for obtaining the value m;

[0035] FIG1B is a second schematic diagram of a calculation method for obtaining the value m;

[0036] FIG1C is a third schematic diagram of a calculation method for obtaining the value m;

[0037] FIG2A is a schematic diagram showing a positional relationship between a current block and a reconstruction area;

[0038] FIG2B is a second schematic diagram of the positional relationship between the current block and the reconstruction area;

[0039] FIG2C is a third schematic diagram of the positional relationship between the current block and the reconstruction area;

[0040] FIG3A is a schematic diagram of a shape of an interpolation filter;

[0041] FIG3B is a second schematic diagram of the shape of an interpolation filter;

[0042] FIG3C is a third schematic diagram of the shape of an interpolation filter;

[0043] FIG4 is a schematic diagram of a structure for obtaining input and output at possible positions of an interpolation filter;

[0044] FIG5 is a schematic diagram of a prediction direction based on interpolation filtering;

[0045] FIG6 is a schematic diagram of an angle mode for intra-frame prediction;

[0046] FIG7 is a schematic diagram of a 3×3 window sliding in a prediction block;

[0047] FIG8 is a schematic diagram of the accumulated gradient amplitude values ​​at different angles;

[0048] FIG9A is a schematic block diagram of an encoder provided in an embodiment of the present application;

[0049] FIG9B is a schematic block diagram of a decoder according to an embodiment of the present application;

[0050] FIG10 is a schematic diagram of a network architecture of a coding and decoding system provided in an embodiment of the present application;

[0051] FIG11 is a flowchart diagram 1 of a decoding method provided in an embodiment of the present application;

[0052] FIG12A is a first schematic diagram of a reference area of ​​a current block provided by an embodiment of the present application;

[0053] FIG12B is a second schematic diagram of a reference area of ​​a current block provided by an embodiment of the present application;

[0054] FIG12C is a third schematic diagram of a reference area of ​​a current block provided by an embodiment of the present application;

[0055] FIG13 is a second flow chart of a decoding method provided in an embodiment of the present application;

[0056] FIG14A is a first schematic diagram showing the distribution of linear and nonlinear terms of a target filter provided by an embodiment of the present application;

[0057] FIG14B is a second schematic diagram of the distribution of linear terms and nonlinear terms of an interpolation filter provided in an embodiment of the present application;

[0058] FIG14C is a third schematic diagram of the distribution of linear terms and nonlinear terms of an interpolation filter provided in an embodiment of the present application;

[0059] FIG15A is a first schematic diagram showing the distribution of linear and nonlinear terms of another target filter provided in an embodiment of the present application;

[0060] FIG15B is a second schematic diagram showing the distribution of linear and nonlinear terms of another interpolation filter provided in an embodiment of the present application;

[0061] FIG15C is a third schematic diagram showing the distribution of linear and nonlinear terms of another interpolation filter provided in an embodiment of the present application;

[0062] FIG16A is a first schematic diagram showing the distribution of linear and nonlinear terms of another target filter provided in an embodiment of the present application;

[0063] FIG16B is a second schematic diagram showing the distribution of linear terms and nonlinear terms of another interpolation filter provided in an embodiment of the present application;

[0064] FIG16C is a third schematic diagram showing the distribution of linear and nonlinear terms of another interpolation filter provided in an embodiment of the present application;

[0065] FIG17 is a flowchart diagram 1 of an encoding method provided in an embodiment of the present application;

[0066] FIG18A is a first schematic diagram of a division of a reference area provided in an embodiment of the present application;

[0067] FIG18B is a second schematic diagram of division of a reference area provided in an embodiment of the present application;

[0068] FIG18C is a third schematic diagram of division of a reference area provided in an embodiment of the present application;

[0069] FIG19 is a second flow chart of an encoding method provided in an embodiment of the present application;

[0070] FIG20 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0071] FIG21 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0072] FIG22 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0073] FIG23 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;

[0074] FIG24 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0077] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0078] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0079] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0080] Joint Video Exploration Team (JVET);

[0081] H.266 / Versatile Video Coding (VVC);

[0082] VVC Test Model (VTM), a reference software testing platform for VVC;

[0083] Enhanced Compression Mode (ECM);

[0084] Extrapolation Intra Prediction (EIP)

[0085] Multiple Transform Selection (MTS);

[0086] Discrete Cosine Transform (DCT);

[0087] Discrete Sine Transform (DST);

[0088] Non-Separable Primary Transform (NSPT);

[0089] Low Frequency Non-separable Secondary Transform (LFNST);

[0090] Direct Current (DC) mode;

[0091] Planar mode (PLANAR);

[0092] Direct Mode (DM);

[0093] Intra Block Copy (IBC);

[0094] Wide Angle Intra Prediction (WAIP);

[0095] Sum of Squares for Error (SSE);

[0096] Mean Squared Error (MSE);

[0097] Sum of Absolute Difference (SAD).

[0098] It can be understood that interpolation-based intra-frame prediction technology refers to a technology that uses the reconstructed pixel values ​​around the current block to obtain interpolation filter coefficients for intra-frame prediction of the current block. Specifically, interpolation-based intra-frame prediction technology can include one or more of the following features:

[0099] (a) The number of taps of the interpolation filter should be greater than or equal to 2. The interpolation filter can have various shapes. The shape of the selected interpolation filter is controlled using syntax elements.

[0100] (b) The reconstructed pixels used to obtain the interpolation filter coefficients should be within one or several regions around the current block, and the region used to obtain the interpolation filter coefficients is selected using syntax elements.

[0101] (c) Interpolation filter prediction can be used for intra block prediction of luma or chroma.

[0102] (d) When using interpolation filtering to predict the current block, prediction should be performed in a certain order starting from the upper left corner to the lower right corner of the block.

[0103] (e) The input of the interpolation filter is the reconstructed pixel value and / or the predicted pixel value, or the reconstructed value and the predicted value with a certain value subtracted.

[0104] (f) Corresponding to (e), if a certain value is subtracted from the input of the interpolation filter, then the interpolation result should add this value back.

[0105] (g) The maximum and minimum values ​​can be obtained from the reconstructed pixel values ​​around the current block to limit the output range of the interpolation filter.

[0106] Furthermore, the specific intra-frame prediction technology based on interpolation filtering can be described in detail through the following aspects.

[0107] (1) Obtain the mean, minimum, and maximum values.

[0108] In a possible embodiment, the maximum and minimum values ​​of the reconstructed pixels are found in a reconstruction area of ​​13 rows and 13 columns around the current block. The maximum and minimum values ​​here can be used to limit the range of the prediction result.

[0109] In a possible embodiment, the value m to be subtracted from the input and added to the output of the interpolation filter is obtained according to the following method, where m is a value used in DC mode prediction and is a positive integer.

[0110] For example, taking FIG. 1A , FIG. 1B , and FIG. 1C as examples, the calculation method for obtaining the value m can be divided into three cases:

[0111] (i) When the width of the current block is equal to its height, m ​​is equal to the average of the reconstructed pixels in the row above and the column to the left of the current block, as shown in FIG1A ;

[0112] (ii) When the width of the current block is greater than its height, m ​​is equal to the average value of the reconstructed pixels in a row above the current block, as shown in FIG1B ;

[0113] (iii) When the height of the current block is greater than its width, m is equal to the average value of the reconstructed pixels in a column on the left side of the current block, as shown in FIG1C .

[0114] In the implementation, the calculation method here can also be summarized as shown in Table 1.

[0115] Table 1

[0116] (2) Obtaining interpolation filter coefficients.

[0117] In a possible embodiment, three 15-tap interpolation filters and three reconstruction areas are defined.

[0118] Figure 2A shows a schematic diagram of the positional relationship between a current block and a reconstruction region. As shown in Figure 2A, the reconstruction region may include an upper adjacent region adjacent to the upper side of the current block and a left adjacent region adjacent to the left side of the current block; wherein the length of the upper adjacent region is 2×Width+13 and the width is 13; and the length of the left adjacent region is 2×Height+13 and the width is 13. Figure 2B shows another schematic diagram of the positional relationship between a current block and a reconstruction region. As shown in Figure 2B, the reconstruction region may include an upper adjacent region adjacent to the upper side of the current block; wherein the length of the upper adjacent region is 2×Width+13 and the width is 13. Figure 2C shows yet another schematic diagram of the positional relationship between a current block and a reconstruction region. As shown in Figure 2C, the reconstruction region may include a left adjacent region adjacent to the left side of the current block; wherein the length of the left adjacent region is 2×Height+13 and the width is 13. In Figures 2A, 2B, and 2C, Height and Width represent the current height and width, respectively. It should be noted that, for the reconstructed areas in FIG. 2A , FIG. 2B and FIG. 2C , that is, the reconstructed pixels in 13 rows and / or 13 columns around the current block can be used to obtain interpolation filter coefficients.

[0119] Figure 3A shows a schematic diagram of the shape of an interpolation filter. As shown in Figure 3A, the interpolation filter is in the shape of a 4×4 square. Figure 3B shows a schematic diagram of the shape of another interpolation filter. As shown in Figure 3B, the interpolation filter is in the shape of a 2×8 rectangle. Figure 3C shows a schematic diagram of the shape of yet another interpolation filter. As shown in Figure 3C, the interpolation filter is in the shape of an 8×2 rectangle. In Figures 3A, 3B, and 3C, the grid-filled portions represent the input positions of the interpolation filters, and the black-filled portions represent the output positions of the interpolation filters.

[0120] In this way, 3×3 different filtering modes can be derived by combining the three reconstruction areas with the three interpolation filter shapes in different ways (each filter shape combined with each reconstruction area can derive a filtering mode). The encoder decides a combination of filter shape and reconstruction area based on the rate-distortion cost. In this way, when predicting the current block, the encoder and decoder first determine the coefficients of the interpolation filter based on the determined filter shape and reconstruction area.

[0121] In one possible embodiment, the input to the interpolation filter is the de-meaned pixel value (i.e., the reconstructed pixel value minus the mean). When obtaining parameters, the selected interpolation filter is slid across the selected area with a horizontal and vertical sliding step of 1 pixel. Specifically, see FIG4 , which illustrates a schematic diagram of the structure of a 4×4 interpolation filter for obtaining the input and output of the interpolation filter at possible positions within the selected reconstruction area. The obtained inputs and outputs are used to construct an autocorrelation coefficient matrix and a cross-correlation coefficient vector. If the selected reconstruction area includes unreconstructed pixel values, these pixel values ​​are not included in the samples used to obtain the interpolation filter parameters.

[0122] In this case, for the constructed Wienerhof equation, specifically the autocorrelation coefficient matrix and the cross-correlation coefficient vector are constructed, the linear equation system is as follows:

[0123] in, represents the selected reconstruction area, t represents the reconstructed pixel value, r represents the coordinate position in the reconstruction area, p0…p N-1 Represents the coordinate relationship relative to position r, and the relative coordinates they refer to are the relative coordinate relationship between the input position and output position of the interpolation filter. c0…c N-1 is the coefficient of the interpolation filter to be solved (also called "filter coefficient"), and m is a value subtracted from the input of the interpolation filter (a value that needs to be added to the output in this case).

[0124] (3) Predict the current block.

[0125] In one possible embodiment, the prediction process starts from the upper left corner of the current block and proceeds to the lower left corner in a certain order. The prediction formula is as follows:

[0126] Among them, a and b in Clip(a,b,c) represent the output range of the limited prediction results. r is the prediction result of position r in the current block, min, max are the minimum and maximum values ​​obtained above, and m is a certain value obtained above. Represents the input of the interpolation filter, which needs to be subtracted from m and further multiplied by the corresponding filter coefficient and summed. When it is located in the reconstruction area, the reconstructed value is used as the input of the interpolation filter. When it is located in the current block, the already obtained predicted value is used as the input of the interpolation filter.

[0127] For example, as shown in Figure 5, the interpolation filter is predicted along the diagonal direction, where the grid-filled area represents the input position of the interpolation filter, and the black-filled area represents the output position of the interpolation filter. In addition, from an implementation perspective, the predicted points on the same diagonal line can be predicted in parallel.

[0128] (4) Classification of prediction blocks and selection of transform kernels.

[0129] After predicting the current block, a prediction block of the current block can be obtained, which includes a predicted value of at least one pixel in the current block. For the prediction block, different angle modes are suitable for using different transforms, including primary transform MTS, NSPT and secondary transform LFNST.

[0130] MTS includes some traditional transforms, such as DCT and DST. NSPT and LFNST are a series of transform coefficients obtained from a universal training set based on the optimal transform. The difference between NSPT and LFNST is that NSPT is used directly to transform the residual coefficients, while LFNST further transforms the transform coefficients after the DCT2 transform.

[0131] For traditional prediction modes (PLANAR mode, DC mode, and angle mode), non-separable primary transform (NSPT) or non-separable secondary transform (LFNST), different traditional prediction modes can be transformed corresponding to different groups of transform kernels according to a mapping such as a table lookup.

[0132] In the reference software ECM, as shown in FIG6 , conventional intra prediction modes may include:

[0133] (i) PLANAR mode: intra prediction mode index is 0;

[0134] (ii) DC mode: intra prediction mode index is 1;

[0135] (iii) Angular mode: Intra prediction mode index ranges from 2 to 66.

[0136] It should also be noted that, as shown in Figure 6, the intra-frame prediction mode can include angle modes of 2 to 66, as well as wide angle modes of -1 to 14 and 67 to 80. The arrows in Figure 6 point to the directions predicted by the angle modes existing in VVC, and the intra-frame prediction mode indexes used during encoding and decoding are 2 to 66. When the current block is a non-square block, some angle directions will be replaced by wide angle modes (such as -1 to -14 and 67 to 80 in Figure 6).

[0137] In the reference software ECM, NSPT and LFNST divide the transformation kernels of the traditional prediction mode into 35 groups, each with 3 selectable transformation kernels. Table 2 shows the correspondence between the traditional prediction modes and the transformation kernel groups.

[0138] Table 2

[0139] In one possible embodiment, a method is proposed herein for matching a prediction block based on interpolation filtering to a traditional prediction mode. Then, the prediction block based on interpolation filtering is matched to different transform kernels of a preset primary transform (separable or non-separable) or secondary transform (separable or non-separable) through the matched traditional prediction mode. Specifically, the prediction block based on interpolation filtering is matched to a PLANAR mode or a mode of angle directions 2 to 66 through the prediction value in the prediction block. Specifically, the following steps may be included:

[0140] The first step is to use a sliding 3×3 window to calculate the horizontal and vertical gradient values ​​G of each 3×3 window in the prediction block based on interpolation filtering. x and G y Among them, G x and G y They are respectively composed of 3×3 horizontal gradient operators M x and the vertical gradient operator M y It is obtained by multiplying the predicted value within the window position.

[0141] Figure 7 is a schematic diagram of a 3×3 window sliding in the prediction block, which can slide in the horizontal and vertical directions. Assuming that the prediction block based on interpolation filtering is a block with a width and height of (w,h), the sliding 3×3 window can calculate the G of the (w-2)×(h-2) positions at the center of the prediction block. x and G y .

[0142] The second step is to calculate the G at each position. x and G y The traditional angle direction O corresponding to each position is calculated according to the following formula, and the gradient amplitude value G corresponding to the angle of each position is calculated, as shown below: G=|G x |+|G y | (5)

[0143] In some embodiments, the calculation process of atan() can be simplified and completed by table lookup or some transformation.

[0144] In the third step, the gradient magnitude G at each position is accumulated over the derived traditional angle modes, resulting in a gradient magnitude histogram as shown in Figure 8. Finally, the traditional angle mode with the largest cumulative gradient magnitude is selected from the histogram as the prediction mode for the current block. In particular, when the gradient magnitudes derived from all traditional angle modes are zero, the current block is matched to the traditional PLANAR mode as the corresponding prediction mode.

[0145] It should also be noted that, in the embodiment of the present application, the traditional prediction mode derived from the interpolation filter prediction will be used for the selection of the transformation kernel group of NSPT and LFNST.

[0146] However, after the interpolation filter intra-frame prediction technology was proposed at the JVET conference, feedback has shown that the cost-effectiveness of codec performance and encoder complexity needs to be improved. Using the latest ECM-8.0 reference software, the encoding time complexity of the related technical solution described here is 108% to 109%.

[0147] Based on this, an embodiment of the present application proposes an encoding method to determine the target filtering mode of the current block; determine the reference area of ​​the current block based on the size parameters and the target filtering mode of the current block; determine the filtering coefficient of the current block based on the reference area of ​​the current block; perform intra-frame prediction on the current block based on the filtering coefficient to determine the prediction value of the current block.

[0148] An embodiment of the present application proposes a decoding method, which decodes a code stream and determines a target filtering mode for a current block; determines a reference area for the current block based on a size parameter of the current block and the target filtering mode; determines a filtering coefficient for the current block based on the reference area of ​​the current block; and performs intra-frame prediction on the current block based on the filtering coefficient to determine a predicted value for the current block.

[0149] In this way, the intra-frame prediction technology based on interpolation filtering, when determining the reference area for calculating the filter coefficient, is not only related to the target filtering mode, but also related to the size parameters of the current block. For example, when the size of the current block is large, a large reference area can be used, and when the size of the current block is small, a small reference area can be used. In this way, while ensuring the encoding and decoding performance, the computational complexity can be reduced, the encoding time can be reduced, and the cost-effectiveness of the encoding and decoding performance and encoding complexity can be improved. At the same time, the intra-frame prediction accuracy can be improved, thereby improving the encoding and decoding efficiency.

[0150] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0151] Referring to Figure 9A, which shows a schematic block diagram of the composition of an encoder provided in an embodiment of the present application. As shown in Figure 9A, the encoder (specifically, a "video encoder") 100 may include a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image cache unit 110, etc., wherein the filtering unit 108 can implement deblocking filtering and sample adaptive offset (SAO) filtering, and the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (CABAC).For the input original video signal, a video coding block can be obtained by dividing the coding tree unit (CTU). Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transformation and quantization unit 101, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block. Specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction coding on the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is the process of generating a motion vector, which can estimate the motion of the video coding block. The motion compensation unit 104 then calculates the motion vector based on the motion vector determined by the motion estimation unit 105. After determining the intra-frame prediction mode, the intra-frame prediction unit 103 is further configured to provide the selected intra-frame prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the encoding unit 109. In addition, the inverse transform and inverse quantization unit 106 is configured to reconstruct the video coding block and reconstruct a residual block in the pixel domain. The reconstructed residual block is subjected to the filter control analysis unit 107 and the filtering unit 108 to remove the block effect artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block. The encoding unit 109 is configured to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode, and output the code stream of the video signal. The decoded image buffer unit 110 is configured to store the reconstructed video coding block for prediction reference. As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 110 .

[0152] 9B , which shows a schematic block diagram of a decoder provided in an embodiment of the present application. As shown in FIG9B , the decoder (specifically, a "video decoder") 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206. The decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. After the input video signal is encoded as shown in FIG9A , a code stream of the video signal is output; the code stream is input to the decoder 200 and first passes through the decoding unit 201 to obtain decoded transform coefficients; the transform coefficients are processed by the inverse transform and inverse quantization unit 202 to generate a residual block in the pixel domain; the intra-frame prediction unit 203 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from the previously decoded block of the current frame or picture; the motion compensation unit 204 determines the prediction information for the video decoding block by analyzing the motion vector and other associated syntax elements, and uses The prediction information is used to generate a predictive block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal passes through the filtering unit 205 to remove blocking artifacts, thereby improving video quality; the decoded video block is then stored in the decoded image buffer unit 206, which stores reference images used for subsequent intra-frame prediction or motion compensation, and is also used for outputting the video signal, thereby obtaining the restored original video signal.

[0153] Furthermore, the embodiment of the present application also provides a network architecture of a coding and decoding system including an encoder and a decoder, wherein Figure 10 shows a schematic diagram of a network architecture of a coding and decoding system provided by an embodiment of the present application. As shown in Figure 10, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During implementation, the electronic device can be various types of devices with video coding and decoding functions. For example, the electronic device can include a smart phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensing device, a server, etc., and the embodiment of the present application is not specifically limited. Here, the decoder or encoder described in the embodiment of the present application can be the above-mentioned electronic device.

[0154] It should be noted that the method of the embodiment of the present application is mainly applied to the intra-frame prediction unit 103 shown in Figure 9A and the intra-frame prediction unit 203 shown in Figure 9B. In other words, the embodiment of the present application can be applied to both the encoder and the decoder, and can even be applied to both the encoder and the decoder simultaneously, but the embodiment of the present application is not specifically limited thereto.

[0155] It should also be noted that, when applied to the intra-frame prediction unit 103, the "current block" specifically refers to the coding block currently to be intra-frame predicted; when applied to the intra-frame prediction unit 203, the "current block" specifically refers to the decoding block currently to be intra-frame predicted.

[0156] In one embodiment of the present application, referring to FIG11 , a flowchart of a decoding method provided by an embodiment of the present application is shown. As shown in FIG11 , the method may include:

[0157] S1101: Decode the code stream and determine the target filtering mode of the current block.

[0158] It should be noted that the decoding method of the embodiment of the present application can be an intra-frame prediction method, specifically an improvement of the intra-frame prediction mode based on interpolation filtering to improve the cost-effectiveness of performance and complexity.

[0159] It should also be noted that in this embodiment of the present application, the current block includes at least a first color component and a second color component. For the first color component of the current block, the block can be simply referred to as a first color component block; and when the first color component is a luminance component, the first color component block can also be referred to as a luminance block. Similarly, for the second color component of the current block, the block can be simply referred to as a second color component block; and when the second color component is a chrominance component, the second color component block can also be referred to as a chrominance block.

[0160] It should also be noted that, in the embodiment of the present application, the target filtering mode may refer to a mode for performing intra-frame prediction on the current block using a target filter, wherein the target filter may refer to an interpolation filter.

[0161] It should also be noted that, in embodiments of the present application, the target filtering mode can be implemented using the first syntax element identification information. That is, in some embodiments, the bitstream is decoded and the value of the first syntax element identification information is determined; when the value of the first syntax element identification information is a first value, the prediction mode of the current block is determined to be the target filtering mode; and when the value of the first syntax element identification information is a second value, the prediction mode of the current block is determined to be a non-target filtering mode.

[0162] In the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the first syntax element identification information can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0163] For example, for the first value and the second value, the first value can be set to 1 and the second value can be set to 0; or the first value can be set to 0 and the second value can be set to 1; or the first value can be set to true and the second value can be set to false; or the first value can be set to false and the second value can be set to true. In the embodiment of the present application, the first value is set to 1 and the second value is set to 0, but this is not a specific limitation.

[0164] It should also be noted that, in the embodiment of the present application, the target filtering mode may include the reference area category of the current block and the shape of the target filter.

[0165] In some embodiments, the reference area category of the current block may include a first category, a second category, and a third category. The method may further include:

[0166] If the reference area category of the current block is the first category, determining that the reference area of ​​the current block includes an upper adjacent area and a left adjacent area;

[0167] If the reference area category of the current block is the second category, determining that the reference area of ​​the current block includes the upper adjacent area;

[0168] If the reference region category of the current block is the third category, it is determined that the reference region of the current block includes the left adjacent region.

[0169] In the embodiment of the present application, the reference area of ​​the current block refers to the reconstructed area surrounding the current block. The upper adjacent area may refer to the reconstructed area adjacent to the upper side of the current block, and the left adjacent area may refer to the reconstructed area adjacent to the left side of the current block.

[0170] Exemplarily, the reference area category shown in FIG. 2A is the first category, the reference area category shown in FIG. 2B is the second category, and the reference area category shown in FIG. 2C is the third category.

[0171] In some embodiments, the shape of the target filter may include a first shape, a second shape, and a third shape, wherein the first shape may be a 4×4 square, the second shape may be a 2×8 rectangle, and the third shape may be an 8×2 rectangle; however, this is not specifically limited here.

[0172] Exemplarily, the target filter shown in FIG. 3A is of a first shape, the target filter shown in FIG. 3B is of a second shape, and the target filter shown in FIG. 3C is of a third shape.

[0173] In this way, the candidate filter modes for the current block can be obtained by combining the three reference region categories and the three target filter shapes. For example, a total of nine candidate filter modes can be combined here, and the target filter mode is one of the nine candidate filter modes.

[0174] S1102: Determine a reference area of ​​the current block according to a size parameter of the current block and a target filtering mode.

[0175] It should be noted that in the embodiment of the present application, after decoding the target filtering mode of the current block, the reference area of ​​the current block can be determined in combination with the size parameter of the current block, where the size parameter of the current block may include the height and width of the current block.

[0176] In some embodiments, the size parameters of the current block include the height and width of the current block; determining the reference area of ​​the current block based on the size parameters of the current block and the target filtering mode may include: determining a minimum parameter from the height and width of the current block; and determining the reference area of ​​the current block based on the minimum parameter and the target filtering mode.

[0177] In the embodiments of the present application, the size of the reference area of ​​the current block is associated with the shape and minimum parameters of the target filter. Simply put, if the current block is large, a large reference area can be used; if the current block is small, a small reference area can be used. The number of rows and columns in the reference area can be derived based on the size of the current block.

[0178] For example, Figure 12A is a schematic diagram of a reference area for a current block, Figure 12B is a schematic diagram of another reference area for a current block, and Figure 12C is a schematic diagram of yet another reference area for a current block. As shown in Figures 12A, 12B, and 12C, the area within the dashed box is the reference area for the current block, which depends on the shape of the target filter used for the current block and the size of the variable tplSize. The size of the variable tplSize is equal to the smaller of the width and height of the current block. For example, for a 4×8 current block, the value of the variable tplSize is 4; for a 16×16 current block, the value of the variable tplSize is 16.

[0179] It is understandable that in the embodiment of the present application, the enabling of the filtering mode can also be limited according to the size parameters of the current block. Specifically, taking Figure 12C as an example, for a current block with a width of 16 and a height of 4, the value of the variable tplSize is 4, which means that there are 4×16=64 pixels to be predicted in the current block; and there are tplSize×(tplSize+4×2)=48 samples used to obtain the filter coefficients. In other words, if the left adjacent area is used to obtain the filter coefficients, there are many pixels to be predicted, but few samples in the reference area used to obtain the filter coefficients. The filter coefficients obtained with too few samples often result in poor prediction results.

[0180] In some embodiments, the method may further include:

[0181] If the multiple of the width of the current block and the first factor is smaller than the height of the current block, determining the reference region category in the target prediction mode to be any one except the second category;

[0182] If the multiple of the height of the current block and the first factor is smaller than the width of the current block, the reference region category in the target prediction mode is determined to be any one except the third category.

[0183] In the embodiment of the present application, the value of the first factor may be a first preset constant. For example, the value of the first factor may be set to 2, but this is not specifically limited.

[0184] In an embodiment of the present application, it is possible to determine whether certain filtering modes are disabled based on the ratio of the width to the height of the current block. Specifically, if the ratio of the width to the height of the current block is less than the inverse of the first factor, that is, the width of the current block and the multiple of the first factor are less than the height of the current block, then the reference area category of the current block can be disabled as the second category, that is, it is prohibited to use the upper adjacent area of ​​the current block for calculating the filter coefficient. At this time, the reference area category in the target prediction mode may only be the first category or the third category; if the ratio of the width to the height of the current block is greater than the first factor, that is, the height of the current block and the multiple of the first factor are less than the width of the current block, then the reference area category of the current block can be disabled as the third category, that is, it is prohibited to use the left adjacent area of ​​the current block for calculating the filter coefficient. At this time, the reference area category in the target prediction mode may only be the first category or the second category.

[0185] Thus, assuming that the target filter still has three shapes, the number of candidate filter modes will be reduced accordingly because some reference area categories are disabled. For example, if the reference area category of the current block is disabled as the second category (i.e., the use of the upper adjacent area of ​​the current block for filter coefficient calculation is prohibited), the number of candidate filter modes will be reduced to six. In other words, since some interpolation filter modes are restricted according to the ratio of the width to the height of the current block (referred to as the "aspect ratio"), the number of candidate filter modes allowed for use under different aspect ratios is different; therefore, decoding can be performed based on the context model when parsing the target filter mode.

[0186] In some embodiments, decoding the code stream and determining the target filtering mode of the current block may include: determining a context model of the current block; and decoding the code stream based on the context model to determine the target filtering mode of the current block.

[0187] In the embodiment of the present application, the determination of the context model is associated with at least one of the following parameters:

[0188] The shape of the current block;

[0189] The ratio of the width to the height of the current block.

[0190] That is to say, in an embodiment of the present application, the selection of a context model may be related to factors such as the shape and aspect ratio of the current block. Specifically, there are multiple context models in the decoding end, and which context model to use for decoding may be determined based on factors such as the shape and aspect ratio of the current block. The reason is that for a current block with a narrow shape, since there may be fewer interpolation filter modes to select, the length of the codeword required to indicate a selected interpolation filter mode is short; while current blocks of other shapes allow different numbers of interpolation filter modes to be selected, and the length of the codeword required to indicate a certain interpolation filter mode is also long, which makes the probability of selecting an interpolation filter mode under different shapes different. In this way, different probabilities require the selection of different context models, and here different context model indexes can be used to determine which specific context model is used.

[0191] It should also be noted that in the embodiment of the present application, after selecting the corresponding context model, the value of the first syntax element identification information can be decoded according to the context model to determine the target filtering mode for the current block. This not only improves prediction accuracy but also reduces computational complexity.

[0192] S1103: Determine a filter coefficient of the current block according to the reference area of ​​the current block.

[0193] It should be noted that, in the embodiment of the present application, the filter coefficient of the current block is mainly determined based on the reference area of ​​the current block and the shape of the target filter. In some embodiments, determining the filter coefficient of the current block based on the reference area of ​​the current block may include:

[0194] According to the reference area of ​​the current block and the shape of the target filter, the input value of the target filter corresponding to at least one reference pixel in the reference area and the output value of the target filter are determined; according to the input value of the target filter corresponding to at least one reference pixel, an autocorrelation coefficient matrix is ​​determined; according to the input value of the target filter corresponding to at least one reference pixel and the output value of the target filter, a cross-correlation coefficient vector is determined; according to the autocorrelation coefficient matrix and the cross-correlation coefficient vector, the coefficients of the target filter are determined; and the coefficients of the target filter are determined as the filter coefficients of the current block.

[0195] For example, the intra-frame prediction technology based on interpolation filtering determines the filter shape and reference area category corresponding to the current block by parsing relevant syntax elements at the decoding end, and then traverses each position in the reference area to construct the autocorrelation coefficient matrix and the cross-correlation coefficient vector, and then obtains the filter coefficient by solving the equation group.

[0196] Among them, the autocorrelation coefficient matrix can be represented by A, and the cross-correlation coefficient vector can be represented by Y, as follows:

[0197] Furthermore, the linear equations are constructed as follows,

[0198] Here, represents the selected reference area, t represents the reconstructed pixel value, r represents the coordinate position of the reference area, p0…p N-1 Represents the coordinate relationship relative to position r, and the relative coordinates they refer to are the relative coordinate relationship between the input position and output position of the target filter. c0…c N-1 is the filter coefficient to be solved, and m is a value subtracted from the input of the target filter (a value added to the output at this time).

[0199] S1104: Perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0200] It should be noted that, in an embodiment of the present application, intra-frame prediction is performed on the pixels in the current block based on the filter coefficient to determine the predicted values ​​of the pixels in the current block, which may include: determining the reference sample value corresponding to the pixel to be predicted in the current block; and determining the predicted value of the pixel to be predicted in the current block based on the reference sample value and the filter coefficient corresponding to the pixel to be predicted in the current block.

[0201] In some embodiments, determining the reference sample value corresponding to the pixel to be predicted in the current block may include: based on the shape of the target filter, if the reference sample value is located in the reference area of ​​the current block, determining the reconstructed value at the corresponding position in the reference area as the reference sample value; if the reference sample value is located inside the current block, determining the predicted value at the corresponding position in the current block as the reference sample value.

[0202] It should also be noted that, in an embodiment of the present application, for the input of the target filter, that is, the reference sample value corresponding to the pixel to be predicted in the current block, if the corresponding position is within the reference area, then the reconstructed value is used as the input of the target filter; or, if the corresponding position is within the current block, then the predicted value that has been predicted is used as the input of the target filter.

[0203] It should also be noted that, in the embodiment of the present application, for the target filter, the interpolation filtering is predicted in the diagonal direction; and the pixels to be predicted on the same diagonal line can be predicted in parallel, as shown in FIG5 .

[0204] In some embodiments, for determining the predicted value of a pixel to be predicted in the current block, referring to FIG. 13 , the method may include:

[0205] S1301: Determine a first input value of a target filter based on reference sample values ​​corresponding to pixels to be predicted in a current block.

[0206] It should be noted that, in embodiments of the present application, the input to the target filter is obtained by subtracting a certain value from the reference sample value, which is then multiplied by the filter coefficient and summed. Therefore, in some embodiments, determining the first input value of the target filter based on the reference sample value corresponding to the pixel to be predicted in the current block may include: determining a second factor; and performing a subtraction operation on the reference sample value and the second factor to obtain the first input value of the target filter.

[0207] S1302: Determine a first output value of a target filter based on a first input value and a filter coefficient.

[0208] It should be noted that, in an embodiment of the present application, determining the first output value of the target filter based on the first input value and the filter coefficient may include: determining the second output value of the target filter based on the first input value and the filter coefficient; performing a first processing on the second output value to determine the first output value of the target filter.

[0209] In some embodiments, determining the second output value of the target filter based on the first input value and the filter coefficient may include: calculating the product of the first input value and the corresponding filter coefficient; setting the second output value of the target filter to be equal to the sum of n products; wherein n represents the number of input items corresponding to the target filter, and n is a positive integer.

[0210] For example, assuming that the reference sample value corresponding to the pixel r to be predicted in the current block can be The second factor can be represented by m, ci represents the i-th filter coefficient; i = 0, 1, 2, ... n-1. Then the second output value of the target filter is represented by P out1 It is expressed as follows:

[0211] In a specific implementation, performing the first processing on the second output value to determine the first output value of the target filter may include: performing an addition operation on the second output value and a second factor to obtain the first output value of the target filter.

[0212] It should be noted that, in the embodiment of the present application, if the input of the target filter is subtracted by a certain value, the output of the target filter also needs to increase by the value. Therefore, the first output value of the target filter can be expressed as P out2 Indicates that,

[0213] In some embodiments, the value of the second factor may be a second preset constant. Alternatively, in some embodiments, the method may further include: determining a reconstructed value of at least one reference pixel in the reference area; performing mean calculation on the reconstructed value of the at least one reference pixel to obtain a first mean value; and setting the value of the second factor to be equal to the first mean value.

[0214] That is, the second factor can be calculated by averaging the reconstructed values ​​in the reference area, or it can be a preset constant, or even a specific value, such as the reconstructed value of the upper left corner of the current block, without specific limitation. For example, if the second factor is the mean of the reference area, then the mean is subtracted from the input of the target filter, and correspondingly, the mean is added to the output of the target filter to obtain the final prediction result.

[0215] In another specific implementation, performing a first processing on the second output value to determine the first output value of the target filter may include: determining the third output value of the target filter; determining the fourth output value of the target filter based on the second output value and the third output value; and performing an addition operation on the fourth output value and the second factor to obtain the first output value of the target filter.

[0216] It should be noted that in an embodiment of the present application, when calculating the output of the target filter, the number of input items includes not only the number of linear items, but also the number of nonlinear items and / or the number of bias items. Here, the third output value can be calculated based on the number of nonlinear items and / or the number of bias items, and the second output value can be calculated based on the number of linear items. In this case, for the second output value of the target filter, specifically: calculate the product of the first input value and the corresponding filter coefficient; set the second output value of the target filter to be equal to the sum of n products; where n represents the number of first-type input items corresponding to the target filter, and n is a positive integer.

[0217] In one specific implementation, the third output value is calculated based on the number of nonlinear terms. In some embodiments, determining the third output value of the target filter may include: determining the number of first-type input terms corresponding to the target filter based on the shape of the target filter; if the number of first-type input terms corresponding to the target filter is p, determining p+q filter coefficients of the target filter, where p and q are both positive integers; and determining the third output value of the target filter based on q filter coefficients and the number of q second-type input terms among the p+q filter coefficients.

[0218] In another specific implementation, the third output value is calculated based on the number of bias terms. In some embodiments, determining the third output value of the target filter may include: determining the number of first-type input terms corresponding to the target filter based on the shape of the target filter; if the number of first-type input terms corresponding to the target filter is p, determining p+m filter coefficients of the target filter, where p and m are both positive integers; and determining the third output value of the target filter based on m filter coefficients of the p+m filter coefficients and the m number of third-type input terms.

[0219] In another specific implementation, the third output value is calculated based on the number of nonlinear terms and the number of bias terms. In some embodiments, determining the third output value of the target filter may include: determining the number of first-type input terms corresponding to the target filter based on the shape of the target filter; if the number of first-type input terms corresponding to the target filter is p, determining p+k filter coefficients of the target filter, where p and k are both positive integers; determining the third output value of the target filter based on i filter coefficients and i second-type input terms among the p+k filter coefficients, and j filter coefficients and j third-type input terms among the p+k filter coefficients; wherein i and j are both positive integers, and k=i+j.

[0220] In the embodiment of the present application, the first type of input item number has a linear relationship with the reference sample value, the second type of input item number has a nonlinear relationship with the reference sample value, and the third type of input item number is preset bias information. In simple terms, the first type of input item number is the linear number of items, the second type of input item number is the nonlinear number of items, and the third type of input item number is the bias number of items.

[0221] For example, assume that the 15-tap linear term of the target filter is shown in Figures 3A, 3B, and 3C. The black-filled position indicates the current position to be predicted. Based on this, a three-tap nonlinear term can be added. The reconstructed pixel positions used by the nonlinear term are shown in Figures 14A, 14B, and 14C, specifically the three dot-filled positions.

[0222] Here, the interpolation input p of 15 linear terms i =t i -m, i takes values ​​from 0 to 14, corresponding to the 14 grid filling positions around the current predicted position, t i is the reconstructed value or predicted value at the grid filling position (depending on whether the input required for the current predicted position is located in the current block or in the reference area), and m is a subtracted value, which can be the reconstructed value of the upper left corner of the current block, or the average value of the reference area, which is not specifically limited here.

[0223] Here, the interpolation input p of the three nonlinear terms i =((t i -m)×(t i -m)+midVal)>>bitDepth, i is the three positions filled with points, p i is the value of the nonlinear term, midVal and bitDepth are equal to 512 and 10 in the case of 10 bits. Thus, when the nonlinear term is added, for the current prediction position, the calculation formula of the first output value of the current position is:

[0224] It should also be noted that in obtaining the filter coefficients, the corresponding nonlinear term values ​​should also be added when constructing the autocorrelation coefficient matrix and the cross-correlation coefficient vector; in addition, when there is a bias term, the bias term value should also be further increased; the setting here is based on actual conditions and is not specifically limited here.

[0225] It can also be understood that, still taking the 15-tap linear term of the target filter shown in Figures 3A, 3B, and 3C as an example, on this basis, the three tap nonlinear terms added here can also be shown in Figures 15A, 15B, and 15C, where the nonlinear terms are specifically three dot-filled positions, and the black-filled position represents the current position to be predicted. Among them, Figures 15A, 15B, and 15C are compared with Figures 14A, 14B, and 14C, although three nonlinear terms are added, because different filter shapes use the same nonlinear terms, the calculation is simpler, further reducing the complexity.

[0226] It can also be understood that, with respect to the number of nonlinear terms, in addition to using 3 nonlinear terms, the embodiments of the present application may also use more nonlinear terms. For example, 5 nonlinear terms are used in FIG16A , FIG16B , and FIG16C , and the positions of the 5 nonlinear terms are specifically five positions filled with dots.

[0227] Thus, in the embodiment of the present application, the number of nonlinear terms should be a positive integer, the specific number is not limited, and different designs can be made according to performance complexity requirements.

[0228] S1303: Determine a predicted value of a pixel to be predicted in the current block according to the first output value.

[0229] It should be noted that, in the embodiment of the present application, determining the predicted value of the pixel to be predicted in the current block based on the first output value may include: performing a second processing on the first output value to obtain the predicted value of the pixel to be predicted in the current block.

[0230] In a specific implementation, the second processing may be setting the predicted value of the pixel to be predicted in the current block to be equal to the first output value.

[0231] In another specific implementation, the second processing may be to limit the first output value to a preset value range, which may also be referred to as a "clamping operation." The lower limit of the preset value range is the minimum reconstructed value (min) in the reference region, and the upper limit of the preset value range is the maximum reconstructed value (max) in the reference region.

[0232] That is to say, in the embodiment of the present application, the preset numerical range is between min and max. If the first output value is within the preset numerical range, then the first output value can be used as the predicted value of the pixel to be predicted in the current block; if the first output value is greater than max, then max can be used as the predicted value of the pixel to be predicted in the current block; if the first output value is less than min, then min can be used as the predicted value of the pixel to be predicted in the current block. Specifically, it can be expressed using the following formula: pred=Clip(min,max,P out2) (12)

[0233] In this way, after the correction operation is performed on the first output value, it can be ensured that the predicted values ​​of all pixels in the current block are between min and max.

[0234] Furthermore, in some embodiments, the method may further include:

[0235] If the luminance component of the current block uses intra prediction based on the filter coefficients, determining a derived intra prediction mode for the luminance component of the current block;

[0236] If the chrominance component of the current block uses direct mode intra prediction, the direct mode is set as the derived intra prediction mode to determine the prediction value of the chrominance component of the current block.

[0237] It should be noted that, in the embodiment of the present application, the derived intra-frame prediction mode may be a traditional PLANAR mode, a DC mode, or an angle mode, etc., which may be specifically determined according to the aforementioned method of constructing a gradient histogram.

[0238] Here, for the DM mode (i.e. "direct mode" or "derived mode"), it is an efficient intra-frame chrominance prediction mode used in many standards when performing intra-frame prediction. When the chrominance block selects to use the DM mode, the chrominance block will obtain the mode selected by the luminance block at the corresponding position for intra-frame prediction.

[0239] Specifically, the interpolation filtering technique described in the previous embodiments only applies to intra-frame prediction of luma blocks. A straightforward approach is to extend this mode to chroma, but this requires deriving filter coefficients for chroma as well, which introduces significant computational complexity. In related art, there is no intra-frame prediction mode for chroma based on interpolation filtering. When the DM mode is selected for a chroma block, the DM mode is set to the PLANAR mode for prediction.

[0240] However, in an embodiment of the present application, for a luminance block using an interpolation filtering mode, a traditional prediction mode can be derived by constructing a gradient histogram. This traditional mode can be used when the chrominance mode selects the DM mode and the interpolation filtering mode is selected for the luminance block at the corresponding position.

[0241] Furthermore, in some embodiments, the method may further include:

[0242] When the current block meets a preset condition, determining a reference block of the current block;

[0243] If the reference block uses intra prediction based on filter coefficients, determining a derived intra prediction mode for the reference block;

[0244] The derived intra prediction mode is added to the intra prediction mode candidate list for the current block.

[0245] In this embodiment of the present application, the current block satisfies a preset condition, including at least one of the following:

[0246] The current block is an inter-frame prediction block;

[0247] The current block is an IBC block.

[0248] It should be noted that in the embodiment of the present application, in the IBC block and inter-frame block, they are not intra-coded blocks, so they do not have intra-frame prediction mode, and the initial reference blocks of the IBC block and inter-frame block are all intra-frame prediction blocks. In the related art, when the reference block is obtained through the inter-frame block and the IBC block, the intra-frame prediction mode of the reference block is also transferred to the current block. These intra-frame prediction modes are traditional intra-frame prediction modes (PLANAR, DC, angle mode). These transferred traditional intra-frame prediction modes will be used when the current block constructs the intra-frame prediction mode candidate list when the surrounding blocks are IBC blocks or inter-frame blocks.

[0249] In this way, when the position of the IBC block or inter-frame block reference is the interpolation filter mode, the traditional intra-frame prediction mode corresponding to the interpolation filter mode is used for transmission.

[0250] Furthermore, in some embodiments, the method may further include:

[0251] Decode the code stream and determine the residual value of the current block;

[0252] A reconstructed value of the current block is determined according to the predicted value of the current block and the residual value of the current block.

[0253] In an embodiment of the present application, decoding the code stream and determining the residual value of the current block may include: decoding the code stream to determine the quantization coefficient of the current block; performing inverse quantization processing on the quantization coefficient to obtain the transformation coefficient of the current block; and performing inverse transformation processing on the transformation coefficient to obtain the residual value of the current block.

[0254] It should be noted that in the embodiment of the present application, after the current block is predicted, the encoder calculates a residual value based on the original value and the predicted value. The residual value is further transformed and quantized to obtain a quantized coefficient, which is then transmitted to the decoder via the bitstream. In this way, the decoder can obtain the quantized coefficient of the current block through decoding, and then obtain the residual value of the current block through inverse quantization and inverse transformation. The residual value of the current block is then added to the predicted value of the current block to obtain the reconstructed value of the current block.

[0255] Furthermore, in some embodiments, performing inverse transform processing on the transform coefficients to obtain the residual value of the current block may include: when the current block uses a multi-transform selection mode and the target filtering mode is an interpolation filtering mode, determining the target transform kernel of the current block; performing inverse transform processing on the transform coefficients according to the target transform kernel to obtain the residual value of the current block.

[0256] In the embodiment of the present application, the determination of the target transformation kernel may be associated with at least one of the following parameters:

[0257] The target filtering mode of the current block;

[0258] The size parameters of the current block;

[0259] The shape of the current block.

[0260] It should also be noted that in this embodiment of the present application, the prediction results of the interpolation filter prediction are derived into a gradient histogram and matched to the traditional prediction mode, further selecting the method of the non-separable transform kernel. For other basic transform kernels other than the non-separable transform kernel, the transform kernel selection method is the same as that of the PLANAR mode. However, the characteristics of the interpolation filter mode and the PLANAR mode are different, and the selection of the basic transform kernel should be more optimized.

[0261] In the reference software ECM, basic transforms can be divided into horizontal and vertical directions. The allowed transform modes in each direction include the following seven types: {'DCT2', 'DCT8', 'DST7', 'DCT5', 'DST4', 'DST1', 'IDTR'}.

[0262] Among them, DCT2, DCT8, and DCT5 are several subclasses of discrete cosine transform, DST7, DST4, and DST1 are several subclasses of discrete sine transform, and IDTR is Identity transform, which means no transformation.

[0263] Furthermore, in the reference software ECM, the most commonly used base transform mode is DCT2 in both the horizontal and vertical directions, here written as DCT2-DCT2. This is used as the primary transform before the inseparable secondary transform LFNST, and is also used when the multi-transform selection MTS technology is disabled. When MTS mode is selected, the transform process is a combination of the base transforms in the horizontal and vertical directions, rather than an inseparable transform.

[0264] In some embodiments, the method may further include: decoding a code stream to determine non-zero coefficient information of a current block; and determining at least one candidate transform kernel according to the non-zero coefficient information of the current block.

[0265] In the embodiment of the present application, the number of at least one candidate transform kernel is less than or equal to 6. That is, in the reference software ECM, based on the characteristics of the parsed non-zero coefficients in the current block, the current block can have at most 6 non-DCT2-DCT2 transform kernels to choose from.

[0266] Thus, in the embodiment of the present application, for a prediction block in an interpolation filtering mode, the residual MTS base transform kernel should be related to whether the current block has selected an interpolation filtering mode. More specifically, it can be related to which interpolation filtering mode is selected and / or the size and shape of the current block.

[0267] In a specific implementation, determining the target transform core of the current block may include: decoding a code stream to determine a transform core index value of the current block; and determining the target transform core of the current block from at least one candidate transform core according to the transform core index value.

[0268] It should be noted that the candidate base transform kernels for the current MTS mode of the ECM are related to whether the interpolation filter prediction mode is selected for the current block. If the interpolation filter prediction mode is used for the current block, the six selectable MTS transform kernels are as follows (transform kernel is: horizontal transform - vertical transform), as shown in Table 3.

[0269] Table 3

[0270] Here, when MTS is selected and the prediction mode of the current block is the interpolation prediction mode, a corresponding target transform core is selected from the six transform cores according to the parsed MTS transform core index value for inverse transformation.

[0271] In another specific implementation, determining a target transform core of a current block may include: decoding a code stream to determine a transform core index value of the current block; and determining a target transform core of the current block from at least one candidate transform core based on the transform core index value and a size parameter of the current block.

[0272] It should also be noted that for the candidates for the base transform kernel used in the current MTS mode of the ECM, the selectable base transform kernels for the MTS are related to whether the interpolation filter mode is selected for the current block, as well as the size and shape of the current block. The shape of the current block is: height × width. In one embodiment, this can be as shown in Table 4.

[0273] Table 4

[0274] Here, when MTS is selected and the prediction mode of the current block is interpolation prediction mode, the corresponding target transform kernel is selected for inverse transform based on the parsed MTS transform kernel index value and the shape and size of the current block. In this embodiment, the interpolation filter prediction mode can be applied to 4×4 to 32×32 luma blocks.

[0275] It should also be noted that the method for obtaining the candidate MTS transformation core may include:

[0276] Step 1: Encode an image set or a video set using an encoder including an interpolation filtering prediction mode;

[0277] Step 2: The residual values ​​of the blocks in the selected interpolation filter mode are then sorted by category (e.g., block shape and size, interpolation filter mode, etc.) to select possible horizontal and vertical transform kernels. The kernel selection criteria can be the SAD value, the SSE value, or another metric, such as transform coding gain, which is not specifically limited here. The transform coding gain is defined as the arithmetic mean of the transform coefficient variances divided by the geometric mean of the transform coefficient variances.

[0278] This embodiment provides a decoding method that decodes a bitstream and determines a target filtering mode for a current block; determines a reference area for the current block based on the size parameters of the current block and the target filtering mode; then determines a filter coefficient for the current block based on the reference area of ​​the current block; and then performs intra-frame prediction on the current block based on the filter coefficient to determine a predicted value for the current block. Thus, the intra-frame prediction technology based on interpolation filtering, when determining the reference area for calculating the filter coefficient, is not only related to the target filtering mode but also to the size parameters of the current block. For example, a large reference area can be used when the size of the current block is large, and a small reference area can be used when the size of the current block is small. This can reduce computational complexity and encoding time, while also improving intra-frame prediction accuracy and thus encoding and decoding performance.

[0279] In another embodiment of the present application, referring to FIG17 , a schematic flow chart of an encoding method provided by an embodiment of the present application is shown. As shown in FIG17 , the method may include:

[0280] S1801: Determine the target filtering mode of the current block.

[0281] It should be noted that the encoding method of the embodiment of the present application can be an intra-frame prediction method, specifically an improvement of the intra-frame prediction mode based on interpolation filtering to improve the cost-effectiveness of performance and complexity.

[0282] It should also be noted that in this embodiment of the present application, the current block includes at least a first color component and a second color component. For the first color component of the current block, the block can be simply referred to as a first color component block; and when the first color component is a luminance component, the first color component block can also be referred to as a luminance block. Similarly, for the second color component of the current block, the block can be simply referred to as a second color component block; and when the second color component is a chrominance component, the second color component block can also be referred to as a chrominance block.

[0283] It should also be noted that, in the embodiment of the present application, the target filtering mode may refer to a mode for performing intra-frame prediction on the current block using a target filter, wherein the target filter may refer to an interpolation filter.

[0284] In some embodiments, determining a target filtering mode for a current block may include:

[0285] Determine at least one candidate filtering mode; perform cost calculation on at least one candidate filtering mode to determine a cost result of at least one candidate filtering mode; determine a minimum cost result from the cost results of at least one candidate filtering mode, and determine the candidate filtering mode corresponding to the minimum cost result as the target filtering mode of the current block.

[0286] In the embodiment of the present application, the number of at least one candidate filtering mode may be determined based on the number of reference region categories and the number of target filter shapes of the current block.

[0287] In the embodiment of the present application, the cost result can be determined by using the distortion value method, specifically the rate-distortion cost method; however, it can also be the size of SAD, the size of MSE, the size of SSE or other criteria for judging the cost, which is not specifically limited here.

[0288] In some embodiments, the reference area category of the current block may include a first category, a second category, and a third category. The method may further include:

[0289] If the reference area category of the current block is the first category, determining that the reference area of ​​the current block includes an upper adjacent area and a left adjacent area;

[0290] If the reference area category of the current block is the second category, determining that the reference area of ​​the current block includes the upper adjacent area;

[0291] If the reference region category of the current block is the third category, it is determined that the reference region of the current block includes the left adjacent region.

[0292] It should be noted that, in the embodiment of the present application, the reference area of ​​the current block refers to the reconstructed area surrounding the current block. The upper adjacent area may refer to the reconstructed area adjacent to the upper side of the current block, and the left adjacent area may refer to the reconstructed area adjacent to the left side of the current block.

[0293] Exemplarily, the reference area category shown in FIG. 2A is the first category, the reference area category shown in FIG. 2B is the second category, and the reference area category shown in FIG. 2C is the third category.

[0294] In some embodiments, the shape of the target filter may include a first shape, a second shape, and a third shape, wherein the first shape may be a 4×4 square, the second shape may be a 2×8 rectangle, and the third shape may be an 8×2 rectangle; however, this is not specifically limited here.

[0295] Exemplarily, the target filter shown in FIG. 3A is of a first shape, the target filter shown in FIG. 3B is of a second shape, and the target filter shown in FIG. 3C is of a third shape.

[0296] In this way, the candidate filter modes for the current block can be obtained by combining the three reference region categories and the three target filter shapes. For example, a total of nine candidate filter modes can be combined here, and the target filter mode is one of the nine candidate filter modes.

[0297] S1802: Determine a reference area of ​​the current block according to a size parameter of the current block and a target filtering mode.

[0298] It should be noted that in the embodiment of the present application, after determining the target filtering mode of the current block, the reference area of ​​the current block can be determined in combination with the size parameter of the current block, where the size parameter of the current block may include the height and width of the current block.

[0299] In some embodiments, the size parameters of the current block include the height and width of the current block; determining the reference area of ​​the current block based on the size parameters of the current block and the target filtering mode may include: determining a minimum parameter from the height and width of the current block; and determining the reference area of ​​the current block based on the minimum parameter and the target filtering mode.

[0300] In an embodiment of the present application, the size of the reference area of ​​the current block is associated with the shape and minimum parameters of the target filter. Simply put, if the size of the current block is large, a large reference area can be used; if the size of the current block is small, a small reference area can be used. The number of rows and columns of the reference area can be derived based on the size of the current block. For example, as shown in Figures 12A, 12B, and 12C, the area within the dotted box is the reference area of ​​the current block, which depends on the shape of the target filter used by the current block and the size of the variable tplSize. The size of the variable tplSize is equal to the smaller of the width and height of the current block. For example, for a 4×8 current block, the value of the variable tplSize is 4; for a 16×16 current block, the value of the variable tplSize is 16.

[0301] It can be understood that in the embodiment of the present application, the enabling of the filtering mode can also be limited according to the size parameters of the current block. Specifically, taking Figure 12C as an example, for a current block with a width of 16 and a height of 4, the value of the variable tplSize is 4, which means that there are 4×16=64 pixels to be predicted in the current block; and there are tplSize×(tplSize+4×2)=48 samples used to obtain the filter coefficient. In other words, if the left adjacent area is used to obtain the filter coefficient, there are many pixels to be predicted, and there are few samples in the reference area used to obtain the filter coefficient. The filter coefficient obtained by obtaining too few samples often results in poor prediction effect. Therefore, in some embodiments, the method may also include:

[0302] If the multiple of the width of the current block and the first factor is smaller than the height of the current block, prohibiting the reference region category of the current block from being the second category, and determining the number of reference region categories of the current block is determined based on reference region categories other than the second category;

[0303] If the multiple of the height of the current block and the first factor is smaller than the width of the current block, the reference area category of the current block is prohibited from being the third category, and the number of reference area categories of the current block is determined based on other reference area categories except the third category.

[0304] In the embodiment of the present application, the value of the first factor may be a first preset constant. For example, the value of the first factor may be set to 2, but this is not specifically limited.

[0305] In an embodiment of the present application, it is possible to determine whether certain filtering modes are disabled based on the ratio of the width to the height of the current block. Specifically, if the ratio of the width to the height of the current block is less than the inverse of the first factor, that is, the width of the current block and the multiple of the first factor are less than the height of the current block, then the reference area category of the current block can be disabled as the second category, that is, it is prohibited to use the upper adjacent area of ​​the current block for calculating the filter coefficient. At this time, the reference area category in the target prediction mode may only be the first category or the third category; if the ratio of the width to the height of the current block is greater than the first factor, that is, the height of the current block and the multiple of the first factor are less than the width of the current block, then the reference area category of the current block can be disabled as the third category, that is, it is prohibited to use the left adjacent area of ​​the current block for calculating the filter coefficient. At this time, the reference area category in the target prediction mode may only be the first category or the second category.

[0306] Thus, assuming that the target filter still has three shapes, the number of candidate filter modes will be reduced accordingly because some reference region categories are disabled. For example, if the reference region category of the current block is disabled as the second category (i.e., the use of the upper adjacent region of the current block for filter coefficient calculation is prohibited), the number of candidate filter modes will be reduced to six. In other words, since some interpolation filter modes are restricted according to the ratio of the width to the height of the current block (referred to as the "aspect ratio"), the number of candidate filter modes allowed for use under different aspect ratios is different; therefore, encoding of the target filter mode can be performed based on the context model.

[0307] In some embodiments, after determining the target filtering mode of the current block, the method may further include: encoding the target filtering mode of the current block, and writing the obtained encoded bits into the bitstream.

[0308] In a specific implementation, encoding the target filtering mode of the current block and writing the obtained coded bits into the bitstream may include: determining a context model of the current block; encoding the target filtering mode of the current block based on the context model, and writing the obtained coded bits into the bitstream.

[0309] In the embodiment of the present application, the determination of the context model is associated with at least one of the following parameters:

[0310] The shape of the current block;

[0311] The ratio of the width to the height of the current block.

[0312] That is to say, in an embodiment of the present application, the selection of a context model may be related to factors such as the shape and aspect ratio of the current block. Specifically, there are multiple context models in the decoding end, and which context model to use for decoding may be determined based on factors such as the shape and aspect ratio of the current block. The reason is that for a current block with a narrow shape, since there may be fewer interpolation filter modes to select, the length of the codeword required to indicate a selected interpolation filter mode is short; while current blocks of other shapes allow different numbers of interpolation filter modes to be selected, and the length of the codeword required to indicate a certain interpolation filter mode is also long, which makes the probability of selecting an interpolation filter mode under different shapes different. In this way, different probabilities require the selection of different context models, and here different context model indexes can be used to determine which specific context model is used.

[0313] It should also be noted that in the embodiment of the present application, after the corresponding context model is selected, the target prediction mode can be encoded according to the context model. This allows the decoder to subsequently decode the bitstream based on the context model selected for the shape of the current block to obtain the target prediction mode for the current block.

[0314] It should also be noted that in embodiments of the present application, the target filtering mode may include the reference region category and target filter shape of the current block. Here, the target filtering mode may be written into the bitstream via the first syntax element identifier. That is, in some embodiments, the value of the first syntax element identifier is determined; the value of the first syntax element identifier is encoded based on a context model, and the resulting encoded bits are written into the bitstream.

[0315] In some embodiments, determining the value of the first syntax element identification information may include: if the current block uses the target filtering mode for prediction encoding, determining the value of the first syntax element identification information to be a first value; if the current block uses the non-target filtering mode for prediction encoding, determining the value of the first syntax element identification information to be a second value.

[0316] In the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the first syntax element identification information can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0317] For example, for the first value and the second value, the first value can be set to 1 and the second value can be set to 0; or the first value can be set to 0 and the second value can be set to 1; or the first value can be set to true and the second value can be set to false; or the first value can be set to false and the second value can be set to true. In the embodiment of the present application, the first value is set to 1 and the second value is set to 0, but this is not a specific limitation.

[0318] In this way, after the value of the first syntax element identification information is written into the bitstream, the decoder can subsequently determine whether the prediction mode of the current block is the target prediction mode by parsing the value of the first syntax element identification information. For example, if the parsed value of the first syntax element identification information is 1, then the prediction mode of the current block can be determined to be the target prediction mode. This not only improves prediction accuracy but also reduces computational complexity.

[0319] S1803: Determine a filter coefficient of the current block according to the reference area of ​​the current block.

[0320] It should be noted that in the embodiment of the present application, the reference area of ​​the current block can be classified and divided to construct an autocorrelation coefficient matrix and a cross-correlation coefficient vector that do not contain repeated areas, so that the encoder can derive the filter coefficients under each combination mode. In some embodiments, the method may further include:

[0321] Determine multiple candidate sub-reference regions for the current block, wherein the multiple candidate sub-reference regions do not overlap with each other; determine the autocorrelation coefficient matrix and the cross-correlation coefficient vector of each of the multiple candidate sub-reference regions based on the multiple candidate sub-reference regions and the shape of the target filter; and store the autocorrelation coefficient matrix and the cross-correlation coefficient vector of each of the multiple candidate sub-reference regions in a preset buffer area.

[0322] In a specific embodiment, the first candidate sub-reference region is any one of a plurality of candidate sub-reference regions. Taking the first candidate sub-reference region as an example, determining the autocorrelation coefficient matrix and the cross-correlation coefficient vector of the first candidate sub-reference region may specifically include: determining the input value of the target filter and the output value of the target filter corresponding to at least one reference pixel in the first candidate sub-reference region according to the shape of the first candidate sub-reference region and the target filter; determining the autocorrelation coefficient matrix of the first candidate sub-reference region according to the input value of the target filter corresponding to at least one reference pixel; and determining the cross-correlation coefficient vector of the first candidate sub-reference region according to the input value of the target filter and the output value of the target filter corresponding to at least one reference pixel. In this way, the autocorrelation coefficient matrix and the cross-correlation coefficient vector of each of the plurality of candidate sub-reference regions can be determined.

[0323] It should also be noted that in this embodiment of the application, the encoder needs to select from nine combinations of three reference regions and three filter shapes. When a combination is selected, the corresponding syntax element identification information is written into the bitstream. Then, if the decoder parses and determines that a certain combination has been selected, it only needs to derive the filter coefficients once. This makes the complexity of this technology much higher on the encoder side than on the decoder side.

[0324] However, for the three reference regions of Figures 2A, 2B, and 2C, they are all composed of three parts, R0, R1, and R2, as shown in Figures 18A, 18B, and 18C. Here, as shown in Figure 18A, the reference region of the current block can be divided into R0, R1, and R2. As shown in Figure 18B, the reference region of the current block can be divided into R0 and R1. As shown in Figure 18C, the reference region of the current block can be divided into R0 and R2.

[0325] Furthermore, f0, f1, and f2 are used to represent the shapes of three filters, R all ,R top ,R left Represents three reference regions. The autocorrelation coefficient matrix and cross-correlation coefficient vector under the nine combinations can be written as follows:

[0326] Where A represents the autocorrelation coefficient matrix and Y represents the cross-correlation coefficient vector. It should be noted that the construction of the autocorrelation coefficient matrix and the cross-correlation coefficient vector is similar to that of equations (7) and (8) at the decoding end and will not be described in detail here.

[0327] Furthermore, it can be observed that due to R all ,R top ,R left Can be composed of R0, R1, R2, so the above 9 combinations can be further decomposed into:

[0328] Furthermore, the decomposition by matrix and vector addition can be further expressed as:

[0329] Therefore, after simplification of Equations (15), (16), and (17), when constructing the autocorrelation coefficient matrix and the cross-correlation coefficient vector, only the following 9 groups are needed to construct the matrices and vectors required for all filter coefficients, as follows:

[0330] In this way, the encoder can select the filter coefficients of the current block from these nine groups of matrices and vectors to determine the required autocorrelation coefficient matrix and cross-correlation coefficient vector. Therefore, in some embodiments, determining the filter coefficients of the current block based on the reference area of ​​the current block may include: dividing the reference area of ​​the current block to determine at least one sub-reference area; obtaining the autocorrelation coefficient matrix and cross-correlation coefficient vector of each of the at least one sub-reference area from a preset buffer; determining the coefficients of the target filter based on the autocorrelation coefficient matrix and cross-correlation coefficient vector of each of the at least one sub-reference area; and determining the coefficients of the target filter as the filter coefficients of the current block.

[0331] That is to say, in an embodiment of the present application, when the encoding end derives the filter coefficients based on the current combination and performs rate-distortion optimization, each time an unconstructed autocorrelation coefficient matrix and cross-correlation coefficient vector are encountered, they need to be cached for use in subsequent other combinations; thereby reducing the computational complexity of the encoding end.

[0332] S1804: Perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0333] It should be noted that, in an embodiment of the present application, intra-frame prediction is performed on the pixels in the current block based on the filter coefficient to determine the predicted values ​​of the pixels in the current block, which may include: determining the reference sample value corresponding to the pixel to be predicted in the current block; and determining the predicted value of the pixel to be predicted in the current block based on the reference sample value and the filter coefficient corresponding to the pixel to be predicted in the current block.

[0334] In some embodiments, determining the reference sample value corresponding to the pixel to be predicted in the current block may include: based on the shape of the target filter, if the reference sample value is located in the reference area of ​​the current block, determining the reconstructed value at the corresponding position in the reference area as the reference sample value; if the reference sample value is located inside the current block, determining the predicted value at the corresponding position in the current block as the reference sample value.

[0335] It should also be noted that, in an embodiment of the present application, for the input of the target filter, that is, the reference sample value corresponding to the pixel to be predicted in the current block, if the corresponding position is within the reference area, then the reconstructed value is used as the input of the target filter; or, if the corresponding position is within the current block, then the predicted value that has been predicted is used as the input of the target filter.

[0336] It should also be noted that, in the embodiment of the present application, for the target filter, the interpolation filtering is predicted in the diagonal direction; and the pixels to be predicted on the same diagonal line can be predicted in parallel, as shown in FIG5 .

[0337] In some embodiments, determining the predicted value of the pixel to be predicted in the current block according to the reference sample value and the filter coefficient corresponding to the pixel to be predicted in the current block may include:

[0338] Determining a first input value of a target filter based on a reference sample value corresponding to a pixel to be predicted in a current block;

[0339] Determining a first output value of a target filter based on the first input value and the filter coefficient;

[0340] Determine a predicted value of a pixel to be predicted in the current block according to the first output value.

[0341] It should be noted that, in embodiments of the present application, the input to the target filter is obtained by subtracting a certain value from the reference sample value, which is then multiplied by the filter coefficient and summed. Therefore, in some embodiments, determining the first input value of the target filter based on the reference sample value corresponding to the pixel to be predicted in the current block may include: determining a second factor; and performing a subtraction operation on the reference sample value and the second factor to obtain the first input value of the target filter.

[0342] It should also be noted that, in an embodiment of the present application, determining the first output value of the target filter based on the first input value and the filter coefficient may include: determining the second output value of the target filter based on the first input value and the filter coefficient; and performing a first processing on the second output value to determine the first output value of the target filter.

[0343] In some embodiments, determining the second output value of the target filter based on the first input value and the filter coefficient may include: calculating the product of the first input value and the corresponding filter coefficient; setting the second output value of the target filter to be equal to the sum of n products; wherein n represents the number of input items corresponding to the target filter, and n is a positive integer.

[0344] For example, assuming that the reference sample value corresponding to the pixel r to be predicted in the current block can be The second factor can be represented by m, c i represents the i-th filter coefficient; i=0,1,2,…n-1. Then the second output value of the target filter is expressed as P out1 It is expressed as follows:

[0345] In a specific implementation, performing the first processing on the second output value to determine the first output value of the target filter may include: performing an addition operation on the second output value and a second factor to obtain the first output value of the target filter.

[0346] It should be noted that, in the embodiment of the present application, if the input of the target filter is subtracted by a certain value, the output of the target filter also needs to increase by the value. Therefore, the first output value of the target filter can be expressed as P out2 Indicates that,

[0347] In some embodiments, the value of the second factor may be a second preset constant. Alternatively, in some embodiments, the method may further include: determining a reconstructed value of at least one reference pixel in the reference area; performing mean calculation on the reconstructed value of the at least one reference pixel to obtain a first mean value; and setting the value of the second factor to be equal to the first mean value.

[0348] That is, the second factor can be calculated by averaging the reconstructed values ​​in the reference area, or it can be a preset constant, or even a specific value, such as the reconstructed value of the upper left corner of the current block, without specific limitation. For example, if the second factor is the mean of the reference area, then the mean is subtracted from the input of the target filter, and correspondingly, the mean is added to the output of the target filter to obtain the final prediction result.

[0349] In another specific implementation, performing a first processing on the second output value to determine the first output value of the target filter may include: determining the third output value of the target filter; determining the fourth output value of the target filter based on the second output value and the third output value; and performing an addition operation on the fourth output value and the second factor to obtain the first output value of the target filter.

[0350] It should be noted that in an embodiment of the present application, when calculating the output of the target filter, the number of input items includes not only the number of linear items, but also the number of nonlinear items and / or the number of bias items. Here, the third output value can be calculated based on the number of nonlinear items and / or the number of bias items, and the second output value can be calculated based on the number of linear items. In this case, for the second output value of the target filter, specifically: calculate the product of the first input value and the corresponding filter coefficient; set the second output value of the target filter to be equal to the sum of n products; where n represents the number of first-type input items corresponding to the target filter, and n is a positive integer.

[0351] In one specific implementation, the third output value is calculated based on the number of nonlinear terms. In some embodiments, determining the third output value of the target filter may include: determining the number of first-type input terms corresponding to the target filter based on a shape of the target filter; if the number of first-type input terms corresponding to the target filter is p, determining p+q filter coefficients for the target filter, where p and q are both positive integers; and determining the third output value of the target filter based on q filter coefficients and the number of q second-type input terms among the p+q filter coefficients.

[0352] In another specific implementation, the third output value is calculated based on the number of bias terms. In some embodiments, determining the third output value of the target filter may include: determining the number of first-type input terms corresponding to the target filter based on the shape of the target filter; if the number of first-type input terms corresponding to the target filter is p, determining p+m filter coefficients of the target filter, where p and m are both positive integers; and determining the third output value of the target filter based on m filter coefficients of the p+m filter coefficients and the m number of third-type input terms.

[0353] In another specific implementation, the third output value is calculated based on the number of nonlinear terms and the number of bias terms. In some embodiments, determining the third output value of the target filter may include: determining the number of first-type input terms corresponding to the target filter based on the shape of the target filter; if the number of first-type input terms corresponding to the target filter is p, determining p+k filter coefficients of the target filter, where p and k are both positive integers; determining the third output value of the target filter based on i filter coefficients and i second-type input terms among the p+k filter coefficients, and j filter coefficients and j third-type input terms among the p+k filter coefficients; wherein i and j are both positive integers, and k=i+j.

[0354] In the embodiment of the present application, the first type of input item number has a linear relationship with the reference sample value, the second type of input item number has a nonlinear relationship with the reference sample value, and the third type of input item number is preset bias information. In other words, the first type of input item number is a linear number of items, the second type of input item number is a nonlinear number of items, and the third type of input item number is a biased number of items.

[0355] Exemplarily, taking FIG. 14A , FIG. 14B , and FIG. 14C as examples, it is assumed that the linear terms of 15 taps are grid-filled positions, the nonlinear terms of 3 taps are dot-filled positions, and the black-filled positions represent the current positions to be predicted.

[0356] Here, the interpolation input p of 15 linear terms i =t i -m, i takes values ​​from 0 to 14, corresponding to the 14 grid filling positions around the current predicted position, ti is the reconstructed value or predicted value at the grid filling position (depending on whether the input required for the current predicted position is located in the current block or in the reference area), and m is a subtracted value, which can be the reconstructed value of the upper left corner of the current block, or the average value of the reference area, which is not specifically limited here.

[0357] Here, the interpolation input p of the three nonlinear terms i =((t i -m)×(t i -m)+midVal)>>bitDepth, i is the three positions filled with points, p i is the value of the nonlinear term, midVal and bitDepth are equal to 512 and 10 in the case of 10 bits. Thus, when the nonlinear term is added, for the current prediction position, the calculation formula of the first output value of the current position is:

[0358] It should be noted that in obtaining the filter coefficients, the corresponding nonlinear term values ​​should also be added when constructing the autocorrelation coefficient matrix and the cross-correlation coefficient vector; in addition, when there is a bias term, the bias term value should also be further increased; the setting here is based on actual conditions and is not specifically limited here.

[0359] It is also understandable that the three tap nonlinear terms added here can also be shown as Figures 15A, 15B, and 15C. Although Figures 15A, 15B, and 15C all add three nonlinear terms compared to Figures 14A, 14B, and 14C, since different filter shapes all use the same nonlinear terms, the calculation is simpler, further reducing complexity.

[0360] It is also understandable that, in addition to using three nonlinear terms, the present embodiment may also use more nonlinear terms. For example, Figures 16A, 16B, and 16C use five nonlinear terms, with the positions of the five nonlinear terms specifically being five dot-filled positions. Thus, in the present embodiment, the number of nonlinear terms should be a positive integer, with no specific limit, and may be designed differently based on performance and complexity requirements.

[0361] It should also be noted that in the embodiment of the present application, the predicted value of the pixel to be predicted in the current block can be determined based on the first output value, which can specifically include: performing a second processing on the first output value to obtain the predicted value of the pixel to be predicted in the current block.

[0362] In a specific implementation, the second processing may be setting the predicted value of the pixel to be predicted in the current block to be equal to the first output value.

[0363] In another specific implementation, the second processing may be to limit the first output value to a preset value range, which may also be referred to as a "clamping operation." The lower limit of the preset value range is the minimum reconstructed value (min) in the reference region, and the upper limit of the preset value range is the maximum reconstructed value (max) in the reference region.

[0364] That is to say, in the embodiment of the present application, the preset numerical range is between min and max. If the first output value is within the preset numerical range, then the first output value can be used as the predicted value of the pixel to be predicted in the current block; if the first output value is greater than max, then max can be used as the predicted value of the pixel to be predicted in the current block; if the first output value is less than min, then min can be used as the predicted value of the pixel to be predicted in the current block. Specifically, it can be expressed using the following formula: pred=Clip(min,max,P out2 ) (twenty one)

[0365] In this way, after the correction operation is performed on the first output value, it can be ensured that the predicted values ​​of all pixels in the current block are between min and max.

[0366] Furthermore, in some embodiments, the method may further include:

[0367] If the luminance component of the current block uses intra prediction based on the filter coefficients, determining a derived intra prediction mode for the luminance component of the current block;

[0368] If the chrominance component of the current block uses direct mode intra prediction, the direct mode is set as the derived intra prediction mode to determine the prediction value of the chrominance component of the current block.

[0369] It should be noted that, in the embodiment of the present application, the derived intra-frame prediction mode may be a traditional PLANAR mode, a DC mode, or an angle mode, etc., which may be specifically determined according to the aforementioned method of constructing a gradient histogram.

[0370] Here, for the DM mode (i.e. "direct mode" or "derived mode"), it is an efficient intra-frame chrominance prediction mode used in many standards when performing intra-frame prediction. When the chrominance block selects to use the DM mode, the chrominance block will obtain the mode selected by the luminance block at the corresponding position for intra-frame prediction.

[0371] Specifically, the interpolation filtering technique described in the previous embodiments only applies to intra-frame prediction of luma blocks. A straightforward approach is to extend this mode to chroma, but this requires deriving filter coefficients for chroma as well, which introduces significant computational complexity. In related art, there is no intra-frame prediction mode for chroma based on interpolation filtering. When the DM mode is selected for a chroma block, the DM mode is set to the PLANAR mode for prediction.

[0372] However, in an embodiment of the present application, for a luminance block using an interpolation filtering mode, a traditional prediction mode can be derived by constructing a gradient histogram. This traditional mode can be used when the chrominance mode selects the DM mode and the interpolation filtering mode is selected for the luminance block at the corresponding position.

[0373] Furthermore, in some embodiments, the method may further include:

[0374] When the current block meets a preset condition, determining a reference block of the current block;

[0375] If the reference block uses intra prediction based on filter coefficients, determining a derived intra prediction mode for the reference block;

[0376] The derived intra prediction mode is added to the intra prediction mode candidate list for the current block.

[0377] In this embodiment of the present application, the current block satisfies a preset condition, including at least one of the following:

[0378] The current block is an inter-frame prediction block;

[0379] The current block is an IBC block.

[0380] In the embodiment of the present application, for IBC blocks and inter-frame blocks, they are not intra-coded blocks, so they do not have intra-frame prediction modes, and the initial reference blocks of IBC blocks and inter-frame blocks are intra-frame prediction blocks. In the related art, when the reference block is obtained through the inter-frame block and IBC block, the intra-frame prediction mode of the reference block is also transferred to the current block. These intra-frame prediction modes are traditional intra-frame prediction modes (PLANAR, DC, angle mode). These transferred traditional intra-frame prediction modes will be used when the surrounding blocks are IBC blocks or inter-frame blocks when constructing the intra-frame prediction mode candidate list for the current block. In this way, when the position of the IBC block or inter-frame block reference is the interpolation filter mode, the traditional intra-frame prediction mode corresponding to the interpolation filter mode is used for transmission.

[0381] Furthermore, in some embodiments, referring to FIG. 19 , the method may further include:

[0382] S2001: Determine the residual value of the current block.

[0383] S2002: Perform transformation processing on the residual value to obtain the transformation coefficient of the current block.

[0384] S2003: Quantize the transform coefficients to obtain quantization coefficients of the current block.

[0385] S2004: Encode the quantization coefficients of the current block and write the obtained coded bits into the bitstream.

[0386] It should be noted that, in the embodiment of the present application, determining the residual value of the current block may include: determining the original value of the current block; determining the residual value of the current block based on the original value of the current block and the predicted value of the current block; and then encoding the residual value of the current block and writing the obtained coded bits into the bitstream.

[0387] It should also be noted that in the embodiment of the present application, a residual value of the current block can be determined by performing a subtraction operation based on the original value of the current block and the predicted value of the current block. When encoding the residual value of the current block, the residual value must first be transformed and quantized, and the resulting quantization coefficients are written into the bitstream, which is then transmitted to the decoding end via the bitstream.

[0388] Further, for step S2002, in some embodiments, transforming the residual value to obtain the transform coefficient of the current block may include: when the current block uses a multi-transform selection mode and the target filtering mode is an interpolation filtering mode, determining the target transform kernel of the current block; transforming the residual value according to the target transform kernel to obtain the transform coefficient of the current block.

[0389] In the embodiment of the present application, the determination of the target transformation kernel may be associated with at least one of the following parameters:

[0390] The target filtering mode of the current block;

[0391] The size parameters of the current block;

[0392] The shape of the current block.

[0393] It should also be noted that in this embodiment of the present application, the prediction results of the interpolation filter prediction are derived into a gradient histogram and matched to the traditional prediction mode, further selecting the method of the non-separable transform kernel. For other basic transform kernels other than the non-separable transform kernel, the transform kernel selection method is the same as that of the PLANAR mode. However, the characteristics of the interpolation filter mode and the PLANAR mode are different, and the selection of the basic transform kernel should be more optimized.

[0394] In the reference software ECM, basic transforms can be divided into horizontal and vertical directions. The allowed transform modes in each direction include the following seven types: {'DCT2', 'DCT8', 'DST7', 'DCT5', 'DST4', 'DST1', 'IDTR'}.

[0395] Among them, DCT2, DCT8, and DCT5 are several subclasses of discrete cosine transform, DST7, DST4, and DST1 are several subclasses of discrete sine transform, and IDTR is Identity transform, which means no transformation.

[0396] Furthermore, in the reference software ECM, the most commonly used base transform mode is DCT2 in both the horizontal and vertical directions, here written as DCT2-DCT2. This is used as the primary transform before the inseparable secondary transform LFNST, and is also used when the multi-transform selection MTS technology is disabled. When MTS mode is selected, the transform process is a combination of the base transforms in the horizontal and vertical directions, rather than an inseparable transform.

[0397] In some embodiments, the method may further include: determining non-zero coefficient information of the current block; and determining at least one candidate transform kernel according to the non-zero coefficient information of the current block.

[0398] In the embodiment of the present application, the number of at least one candidate transform kernel is less than or equal to 6. That is, in the reference software ECM, based on the characteristics of the non-zero coefficients in the current block determined after transformation and quantization, the current block can have at most 6 non-DCT2-DCT2 transform kernels to choose from.

[0399] Thus, in the embodiment of the present application, for a prediction block in an interpolation filtering mode, the residual MTS base transform kernel should be related to whether the current block has selected an interpolation filtering mode. More specifically, it can be related to which interpolation filtering mode is selected and / or the size and shape of the current block.

[0400] In a specific implementation, determining the target transformation core of the current block may include: determining at least one candidate transformation core; performing cost calculation on the at least one candidate transformation core to determine a cost result of the at least one candidate transformation core; determining a minimum cost result from the cost results of the at least one candidate transformation core, and determining the candidate transformation core corresponding to the minimum cost result as the target transformation core of the current block.

[0401] It should be noted that in the embodiment of the present application, the cost result can be determined by using the distortion value method, specifically the rate-distortion cost method; however, it can also be the size of the SAD, the size of the MSE, the size of the SSE or other judgment criteria, such as the transform coding gain, which is not specifically limited here.

[0402] In some embodiments, the method may further include: determining a transform core index value of the current block, wherein the transform core index value is used to indicate an index number of a target transform core in at least one candidate transform core; encoding the transform core index value of the current block, and writing the obtained encoded bits into a bitstream.

[0403] It should be noted that in this embodiment of the present application, the candidates for the base transform kernel used in the current MTS mode of the ECM are related to whether the interpolation filter prediction mode is selected for the current block. If the interpolation filter prediction mode is used for the current block, the six optional MTS transform kernels are as follows (transform kernel is: horizontal transform - vertical transform), as shown in Table 3.

[0404] Here, when MTS is selected and the prediction mode of the current block is the interpolation prediction mode, the MTS transform core index value can be determined and written into the bitstream; so that the subsequent decoding end can select the corresponding target transform core from the six transform cores according to the parsed MTS transform core index value for inverse transformation.

[0405] In other embodiments, the method may further include: determining a transform core index value of the current block, wherein the transform core index value is used to indicate an index number of a target transform core in at least one candidate transform core, and at least one candidate transform core has an associated relationship with a size parameter of the current block; encoding the transform core index value of the current block, and writing the obtained encoded bits into a bitstream.

[0406] It should also be noted that, in the embodiment of the present application, for the candidates of the basic transform kernel used in the current MTS mode of the ECM, the selectable basic transform kernel of the MTS is related to whether the interpolation filter mode is selected for the current block and the size and shape of the current block; as shown in Table 4. The shape size of the current block is: height × width.

[0407] Here, when MTS is selected and the prediction mode for the current block is interpolation prediction mode, the MTS transform kernel index value can be determined based on the size parameters of the current block and written into the bitstream; this allows the subsequent decoder to select the corresponding target transform kernel for inverse transformation based on the parsed MTS transform kernel index value and the shape and size of the current block. In this embodiment, the interpolation filter prediction mode can be applied to 4×4 to 32×32 luma blocks.

[0408] It should also be noted that the method for obtaining the candidate MTS transformation core may include:

[0409] Step 1: Encode an image set or a video set using an encoder including an interpolation filtering prediction mode;

[0410] Step 2: The residual values ​​of the blocks in the selected interpolation filter mode are then sorted by category (e.g., block shape and size, interpolation filter mode, etc.) to select possible horizontal and vertical transform kernels. The kernel selection criteria can be the SAD value, the SSE value, or another metric, such as transform coding gain, which is not specifically limited here. The transform coding gain is defined as the arithmetic mean of the transform coefficient variances divided by the geometric mean of the transform coefficient variances.

[0411] Furthermore, an embodiment of the present application further provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:

[0412] The target filtering mode of the current block, the residual value of the current block, and the transform kernel index value of the current block.

[0413] It should be noted that in this embodiment of the present application, when writing the bitstream, the target filter mode for the current block can also be written into the bitstream using the value of the first syntax element identification information. Furthermore, the residual value of the current block can also be written into the bitstream after transforming and quantizing the residual value. To facilitate the decoder to quickly determine the target transform kernel to use, the encoder also needs to write the transform kernel index value for the current block into the bitstream; this can improve encoding and decoding efficiency.

[0414] This embodiment provides an encoding method that determines a target filtering mode for a current block; determines a reference area for the current block based on the size parameters of the current block and the target filtering mode; then determines a filter coefficient for the current block based on the reference area of ​​the current block; and then performs intra-frame prediction on the current block based on the filter coefficient to determine a predicted value for the current block. Thus, the intra-frame prediction technology based on interpolation filtering, when determining the reference area for calculating the filter coefficient, is not only related to the target filtering mode but also to the size parameters of the current block. For example, a large reference area can be used when the size of the current block is large, and a small reference area can be used when the size of the current block is small. This can reduce computational complexity and encoding time, while also improving intra-frame prediction accuracy and, consequently, encoding and decoding performance.

[0415] In another embodiment of the present application, based on the encoding and decoding method described in the aforementioned embodiment, the intra-frame prediction mode based on interpolation filtering is improved, and the improvements are described in detail from several aspects below.

[0416] (a) Reference area of ​​the current block.

[0417] In the embodiments of the present application, the reference region always uses a reconstructed region composed of reconstructed pixel values of 13 rows and / or 13 columns, which results in a much higher computational complexity on small blocks than on large blocks. At the encoding end, the encoder needs to make decisions on block partitioning, and increasing the computational amount of small blocks is more likely to lead to an increase in encoding time. Based on this, the embodiments of the present application propose using a large reference region for large blocks and a small reference region for small blocks, and the number of rows and columns of the reference region can be derived according to the size of the block. For specific details, refer to the foregoing FIGS. 12A, 12B, and 12C.

[0418] (2) Restrict the enabling of the interpolation filtering mode according to the shape of the block.

[0419] For a current block with a width of 16 and a height of 4, if the left reconstructed region is used to obtain the coefficients of the interpolation filter, there are many pixels to be predicted, while there are few pixels in the region for obtaining the interpolation filter parameters, as specifically shown in FIG. 12C. Here, for a current block with a width of 16 and a height of 4, tplSize is 4, which means there are a total of 4×16 = 64 pixels to be predicted, and there are a total of tplSize×(tplSize + 4×2) = 48 samples for obtaining the filter coefficients. The interpolation filter coefficients obtained from too few samples often result in poor prediction effects.

[0420] In the embodiments of the present application, it is also proposed here that when the ratio of the width to the height of the current block: width×N < height, the upper-side reconstructed region is prohibited from being used to derive the interpolation filter coefficients, and when height×N < width, the left reconstructed region is prohibited from being used to derive the interpolation filter coefficients. For example, N = 2.

[0421] In this way, when encoding and decoding the interpolation filtering mode, since some interpolation filtering sub-modes are restricted according to the aspect ratio, the number of interpolation filtering sub-modes allowed under different aspect ratios is different. Therefore, when parsing the syntax element identifier of the interpolation filter, the selection of its context model should be related to the shape of the block and the aspect ratio factor. It should be noted that assuming that three types of reference regions and three shapes of filters can form nine interpolation filtering modes, each of which can be regarded as an interpolation filtering sub-mode; in other words, the interpolation filtering mode can include nine interpolation filtering sub-modes.

[0422] (3) Optimize the process of obtaining the autocorrelation coefficient matrix at the encoding end.

[0423] In the embodiments of the present application, by classifying and partitioning the reference regions, an autocorrelation coefficient matrix and a cross-correlation coefficient vector without overlapping regions can be constructed for the encoding end to derive the filter coefficients of each combination.

[0424] As in the aforementioned embodiment, the interpolation filter prediction technology determines the shape of the interpolation filter selected by the current block and the reference area category by parsing relevant syntax elements at the decoding end, traverses each position on the reference area to construct the autocorrelation coefficient matrix and the cross-correlation coefficient vector and solves the equation group to obtain the filter coefficient.

[0425] Among them, the constructed autocorrelation coefficient matrix and mutual correlation coefficient vector, the linear equation system is as follows,

[0426] in, represents the selected reference area, t represents the reconstructed pixel value, r represents the coordinate position of the reference area, p0…p N-1 Represents the coordinate relationship relative to position r, and the relative coordinates they refer to are the relative coordinate relationship between the input position and output position of the interpolation filter. c0…c N-1 is the filter coefficient to be solved, and m is the value subtracted from the input of the interpolation filter (a value added to the output at this time).

[0427] In an embodiment of the present application, the encoding end needs to screen from a total of 9 combinations of 3 reference areas and 3 filter shapes. When a certain combination is selected, the corresponding syntax element is encoded into the bitstream. If the decoding end parses and finds that a certain combination is selected, only the filter coefficient needs to be derived once; this makes the complexity of the technology on the encoding end much higher than that on the decoding end.

[0428] However, for the three reference regions of Figures 2A, 2B, and 2C, they are all composed of three parts, R0, R1, and R2, as shown in Figures 18A, 18B, and 18C. Here, f0, f1, and f2 are used to represent the three filter shapes. all ,R top ,R left Represents three reference regions. The autocorrelation coefficient matrix and cross-correlation coefficient vector under the nine combinations can be written as follows:

[0429] Where A represents the autocorrelation coefficient matrix and Y represents the cross-correlation coefficient vector.

[0430] Furthermore, it can be observed that due to R all ,R top ,R left Can be composed of R0, R1, R2, so the above 9 combinations can be further decomposed into:

[0431] The decomposition of matrix and vector addition can be further expressed as:

[0432] Therefore, after simplification, when constructing the autocorrelation coefficient matrix and the cross-correlation coefficient vector, only the following 9 groups are needed to construct the matrices and vectors required to obtain all filter coefficients, as follows:

[0433] Furthermore, when the encoder derives filter coefficients based on the current combination and performs rate-distortion optimization, each time it encounters an unconstructed autocorrelation coefficient matrix and cross-correlation coefficient vector, it needs to cache them for use in subsequent combinations; thereby reducing the computational complexity of the encoding end.

[0434] (4) Extension of chroma intra prediction mode.

[0435] The DM mode is an efficient intra-frame chroma prediction mode used in many standards for prediction. When the chroma block selects the DM mode, the chroma block will obtain the mode selected by the luminance block at the corresponding position for intra-frame prediction. The interpolation filtering technology described in the above embodiment only works on the prediction of intra-frame blocks of luminance. A direct approach is to extend the mode to chroma, but this will result in the need to derive filtering parameters for chroma, which will bring high computational complexity. In related technologies, there is no interpolation filtering prediction mode for chroma. When the DM mode is selected for the chroma intra-frame block, the DM mode will be set to PLANAR mode.

[0436] However, for the luminance block using the interpolation filter mode, a traditional prediction mode can be derived by constructing a gradient histogram. This traditional mode can be used when the chrominance mode selects the DM mode and the interpolation filter mode is selected for the luminance block at the corresponding position.

[0437] (5) Traditional prediction mode corresponding to transfer interpolation filtering.

[0438] In IBC blocks and inter blocks, they are not intra-coded blocks and therefore do not have intra prediction modes. However, their initial reference blocks are all intra prediction blocks. In related technologies, when obtaining a reference block through inter blocks and IBC blocks, the intra prediction mode of the reference block is also transferred to the current block. These intra prediction modes are traditional intra prediction modes (PLANAR, DC, and angular modes). These transferred traditional intra prediction modes are used when constructing the intra prediction mode candidate list for the current block when the surrounding blocks are IBC blocks or inter blocks.

[0439] In one possible implementation, the technology required to construct an intra prediction candidate list may include the following:

[0440] (i) Intra-coded blocks: These blocks can use a range of intra prediction techniques, such as spatial geometric partitioning mode (SGPM), template-based multiple reference line intra prediction (TMRL), most probable mode (MPM), and template-based intra mode derivation (TIMD);

[0441] (ii) Inter-frame blocks, which can use GPM (geometric partitioning mode);

[0442] (iii) Intra block copy (IBC) blocks, which can use the intra prediction mode and the copied blocks to obtain the prediction results.

[0443] That is, in the embodiment of the present application, when the position of the IBC block or inter-frame block reference is the interpolation filtering mode, the traditional mode corresponding to the interpolation filtering mode is used for transmission.

[0444] (6) Selection of the basic transform kernel for the prediction block in the interpolation filter mode.

[0445] After the current block is predicted, the encoder calculates the residual value between the predicted value and the original value. The residual value will be further transformed and quantized. At the decoder, the quantized coefficients parsed from the bitstream will be inverse quantized and inverse transformed to obtain the reconstructed residual value. The reconstructed residual value is added to the predicted value to obtain the reconstructed value.

[0446] The aforementioned embodiment introduces a method for deriving a gradient histogram from the interpolation filter prediction results and matching it to a traditional prediction mode, further selecting an inseparable transform kernel. For base transform kernels other than the inseparable transform kernel, the transform kernel selection method is the same as that for the PLANAR mode. However, the interpolation filter mode differs from the PLANAR mode in its characteristics, and the selection of the base transform kernel should be more optimized.

[0447] In the reference software ECM, basic transforms are divided into horizontal and vertical directions. The following seven transforms are allowed in each direction: {'DCT2', 'DCT8', 'DST7', 'DCT5', 'DST4', 'DST1', 'IDTR'}. DCT2, DCT8, and DCT5 are subclasses of the discrete cosine transform; DST7, DST4, and DST1 are subclasses of the discrete sine transform; and IDTR stands for identity transform, indicating no transformation.

[0448] In the reference software ECM, the most commonly used base transform mode is DCT2 in both the horizontal and vertical directions, written as DCT2-DCT2. This is used as the primary transform before the inseparable secondary transform (LFNST) and is also used when the Multiple Transform Select (MTS) technology is disabled. When MTS mode is selected, the transform process is a combination of the base transforms in the horizontal and vertical directions, rather than an inseparable transform. In ECM, up to six non-DCT2-DCT2 transform kernels can be selected for the current block, depending on the characteristics of the non-zero coefficients in the current block.

[0449] In an embodiment of the present application, for the prediction block of the interpolation filtering mode, the residual MTS basic transform kernel should be related to whether the interpolation filtering mode is selected for the current block. More specifically, it may be related to which sub-mode of the interpolation filtering mode is selected and / or the size and shape of the current block.

[0450] For example, the embodiments of the present application provide two implementations of basic change core candidates that can be used under the current MTS design of ECM.

[0451] In one possible implementation, the MTS selectable base transform kernel is related to whether the interpolation filter prediction mode is selected for the current block. If the interpolation filter prediction mode is used for the current block, the six MTS transform kernels are as follows (the transform kernel is: horizontal transform - vertical transform). See Table 3 for details. When MTS is selected and the prediction mode for the current block is interpolation prediction mode, the corresponding target transform kernel is selected from the six transform kernels based on the parsed MTS transform kernel index value for inverse transform.

[0452] In another possible implementation, the base transform kernel selectable by MTS depends on whether the interpolation filter mode is selected for the current block, as well as the size and shape of the current block. The block shape is: height × width. See Table 4 for details. When MTS is selected and the prediction mode for the current block is interpolation prediction mode, the corresponding target transform kernel is selected for inverse transform based on the parsed MTS transform kernel index value and the block shape and size. In this embodiment, the interpolation filter prediction mode is applicable to 4×4 to 32×32 luma blocks.

[0453] It should also be noted that the method for obtaining the candidate MTS transformation core may include:

[0454] Step 1: Encode the image set or video set using an encoder that includes an interpolation filter prediction mode

[0455] Step 2: The residuals of the blocks with the selected interpolation filter mode are sorted by category (e.g., block shape and size, interpolation filter mode) to select possible horizontal and vertical transform kernels. The kernel selection criteria can be the SAD size, SSE size, or other metrics such as transform coding gain. Transform coding gain is defined as the arithmetic mean of the transform coefficient variances divided by the geometric mean of the transform coefficient variances.

[0456] (7) Nonlinear terms in interpolation filtering.

[0457] In the interpolation filtering described in the aforementioned embodiment, the prediction of the interpolation filtering does not contain nonlinear terms or bias terms. In order to improve the coding performance gain brought by the nonlinear terms or bias terms, nonlinear terms or bias terms can also be added to the interpolation filtering. Among them, the 15 linear terms used in this implementation process are the three cases shown in Figures 3A, 3B and 3C. The linear terms of the 15 taps of the interpolation filter are grid filling positions, and the black filling position is the current position to be predicted. On this basis, 3 tap nonlinear terms can be added. The reconstructed pixel positions used by the nonlinear terms are shown in Figures 14A, 14B and 14C, specifically three dot filling positions.

[0458] Here, the interpolation input p of 15 linear terms i =t i -m, i takes values ​​from 0 to 14, corresponding to the 14 grid filling positions around the current predicted position, t i is the reconstructed value or predicted value at the grid filling position (depending on whether the input required for the current predicted position is located in the current block or in the reference area), and m is a subtracted value, which can be the reconstructed value of the upper left corner of the current block, or the average value of the reference area, which is not specifically limited here.

[0459] Here, the interpolation input p of the three nonlinear terms i =((t i -m)×(t i -m)+midVal)>>bitDepth, i is the three positions filled with points, p i is the value of the nonlinear term, midVal and bitDepth are equal to 512 and 10 in the case of 10 bits. Thus, when the nonlinear term is added, for the current prediction position, the calculation formula of the prediction value is as follows:

[0460] It should also be noted that in obtaining the interpolation filter coefficients, the corresponding nonlinear term value should also be increased when constructing the autocorrelation coefficient matrix and the cross-correlation coefficient vector; and / or, when there is a bias term, the bias term value should also be further increased when constructing the autocorrelation coefficient matrix and the cross-correlation coefficient vector.

[0461] In addition to the above embodiment, three tap nonlinear terms are added to the 15 tap linear terms of the target filter, as shown in Figures 15A, 15B, and 15C. The nonlinear terms are specifically three dot-filled positions, and the black-filled position represents the current position to be predicted. Although Figures 15A, 15B, and 15C all add three nonlinear terms compared to Figures 14A, 14B, and 14C, since different filter shapes all use the same nonlinear term, the calculation is simpler, further reducing complexity.

[0462] Furthermore, with respect to the number of nonlinear terms, in addition to using three nonlinear terms, the embodiment of the present application may also use more nonlinear terms, for example, five nonlinear terms are used in Figures 16A, 16B, and 16C. As shown in Figures 16A, 16B, and 16C, on the basis of the 15-tap linear terms of the interpolation filter, five-tap nonlinear terms are added (specifically, the positions are filled with dots). That is, in the embodiment of the present application, the number of nonlinear terms should be a positive integer, the specific number is not limited, and different designs can be performed according to performance complexity requirements.

[0463] In the embodiments of the present application, the specific implementation of the aforementioned embodiments is elaborated in detail through the above embodiments. It can be seen that according to the technical solutions of the aforementioned embodiments, while ensuring the encoding and decoding performance, the computational complexity can be reduced and the encoding time can be reduced, so that the cost-effectiveness of the encoding and decoding performance and the encoding complexity can be improved. At the same time, the intra-frame prediction accuracy can be improved, thereby improving the encoding and decoding efficiency.

[0464] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, see FIG20 , which shows a schematic diagram of the composition structure of an encoder provided by an embodiment of the present application. As shown in FIG20 , the encoder 220 may include a first determination unit 2201 and a first prediction unit 2202, wherein:

[0465] The first determining unit 2201 is configured to determine a target filtering mode of the current block; and determine a reference area of ​​the current block according to a size parameter of the current block and the target filtering mode;

[0466] The first prediction unit 2202 is configured to determine a filter coefficient of the current block according to a reference area of ​​the current block; and perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0467] In some embodiments, the first determination unit 2201 is further configured to determine at least one candidate filtering mode; perform cost calculation on at least one candidate filtering mode to determine the cost result of at least one candidate filtering mode; and determine the minimum cost result from the cost results of at least one candidate filtering mode, and determine the candidate filtering mode corresponding to the minimum cost result as the target filtering mode of the current block.

[0468] In some embodiments, the number of the at least one candidate filtering mode is determined based on the number of reference region categories and the number of shapes of the target filter of the current block.

[0469] In some embodiments, the target filtering mode includes a reference region category of the current block and a shape of a target filter.

[0470] In some embodiments, the first determination unit 2201 is further configured to, if the reference area category of the current block is the first category, determine that the reference area of ​​the current block includes the upper adjacent area and the left adjacent area; if the reference area category of the current block is the second category, determine that the reference area of ​​the current block includes the upper adjacent area; if the reference area category of the current block is the third category, determine that the reference area of ​​the current block includes the left adjacent area; wherein the upper adjacent area refers to the reconstructed area adjacent to the upper side of the current block, and the left adjacent area refers to the reconstructed area adjacent to the left side of the current block.

[0471] In some embodiments, the size parameters of the current block include the height and width of the current block; the first determination unit 2201 is further configured to determine a minimum parameter from the height and width of the current block; and determine a reference area of ​​the current block based on the minimum parameter and the target filtering mode.

[0472] In some embodiments, the size of the reference area of ​​the current block is associated with the shape and minimum parameter of the target filter.

[0473] In some embodiments, the first determination unit 2201 is further configured to, if the width of the current block and the multiple of the first factor are smaller than the height of the current block, prohibit the reference area category of the current block from being the second category, and determine that the number of reference area categories of the current block is determined based on reference area categories other than the second category; if the height of the current block and the multiple of the first factor are smaller than the width of the current block, prohibit the reference area category of the current block from being the third category, and determine that the number of reference area categories of the current block is determined based on reference area categories other than the third category.

[0474] In some embodiments, the value of the first factor is a first preset constant.

[0475] In some embodiments, referring to FIG. 20 , the encoder 220 may further include an encoding unit 2203 configured to encode a target filtering mode of a current block and write the obtained encoding bits into a bitstream.

[0476] In some embodiments, the first determining unit 2201 is further configured to determine a context model of the current block;

[0477] The encoding unit 2203 is further configured to encode the target filtering mode of the current block based on the context model, and write the obtained encoding bits into the bitstream.

[0478] In some embodiments, the determination of the context model is associated with at least one of the following parameters:

[0479] The shape of the current block;

[0480] The ratio of the width to the height of the current block.

[0481] In some embodiments, the first determination unit 2201 is further configured to determine multiple candidate sub-reference areas of the current block, and the multiple candidate sub-reference areas do not overlap with each other; determine the autocorrelation coefficient matrix and the mutual correlation coefficient vector of the multiple candidate sub-reference areas according to the shapes of the multiple candidate sub-reference areas and the target filter; and store the autocorrelation coefficient matrix and the mutual correlation coefficient vector of the multiple candidate sub-reference areas in a preset cache area.

[0482] In some embodiments, the first determination unit 2201 is further configured to determine the input value of the target filter and the output value of the target filter corresponding to at least one reference pixel in the first candidate sub-reference area based on the shape of the first candidate sub-reference area and the target filter; determine the autocorrelation coefficient matrix of the first candidate sub-reference area based on the input value of the target filter corresponding to at least one reference pixel; and determine the mutual correlation coefficient vector of the first candidate sub-reference area based on the input value of the target filter and the output value of the target filter corresponding to at least one reference pixel; wherein the first candidate sub-reference area is any one of the multiple candidate sub-reference areas.

[0483] In some embodiments, the first determination unit 2201 is further configured to divide the reference area of ​​the current block to determine at least one sub-reference area; obtain the autocorrelation coefficient matrix and cross-correlation coefficient vector of each of the at least one sub-reference area from a preset buffer area; determine the coefficients of the target filter based on the autocorrelation coefficient matrix and cross-correlation coefficient vector of each of the at least one sub-reference area; and determine the coefficients of the target filter as the filter coefficients of the current block.

[0484] In some embodiments, the first determining unit 2201 is further configured to determine a reference sample value corresponding to a pixel to be predicted in the current block;

[0485] The first prediction unit 2202 is further configured to determine a prediction value of the pixel to be predicted in the current block according to the reference sample value and the filter coefficient corresponding to the pixel to be predicted in the current block.

[0486] In some embodiments, the first determination unit 2201 is further configured to, based on the shape of the target filter, determine the reconstructed value at the corresponding position in the reference area as the reference sample value if the reference sample value is located in the reference area of ​​the current block; and determine the predicted value at the corresponding position in the current block as the reference sample value if the reference sample value is located inside the current block.

[0487] In some embodiments, the first prediction unit 2202 is further configured to determine a first input value of the target filter based on a reference sample value corresponding to the pixel to be predicted in the current block; determine a first output value of the target filter based on the first input value and the filter coefficient; and determine a predicted value of the pixel to be predicted in the current block based on the first output value.

[0488] In some embodiments, the first determining unit 2201 is further configured to determine a second factor; and perform a subtraction operation on the reference sample value and the second factor to obtain a first input value of the target filter.

[0489] In some embodiments, the first determining unit 2201 is further configured to determine a second output value of the target filter based on the first input value and the filter coefficient; and perform a first process on the second output value to determine a first output value of the target filter.

[0490] In some embodiments, the first determination unit 2201 is further configured to calculate the product of the first input value and the corresponding filter coefficient; and set the second output value of the target filter to be equal to the sum of n products; wherein n represents the number of input items corresponding to the target filter, and n is a positive integer.

[0491] In some embodiments, the first determining unit 2201 is further configured to perform an addition operation on the second output value and the second factor to obtain a first output value of the target filter.

[0492] In some embodiments, the first determination unit 2201 is further configured to determine the third output value of the target filter; determine the fourth output value of the target filter based on the second output value and the third output value; and perform an addition operation on the fourth output value and the second factor to obtain the first output value of the target filter.

[0493] In some embodiments, the first determination unit 2201 is further configured to determine the number of first type input items corresponding to the target filter based on the shape of the target filter; if the number of first type input items corresponding to the target filter is p, then determine p+q filter coefficients of the target filter, where p and q are both positive integers; and determine the third output value of the target filter based on q filter coefficients and q second type input items among the p+q filter coefficients.

[0494] In some embodiments, the first determination unit 2201 is further configured to determine the number of first type input items corresponding to the target filter based on the shape of the target filter; if the number of first type input items corresponding to the target filter is p, then determine p+m filter coefficients of the target filter, where p and m are both positive integers; and determine the third output value of the target filter based on m filter coefficients and m third type input items among the p+m filter coefficients.

[0495] In some embodiments, the first determination unit 2201 is further configured to determine the number of first type input items corresponding to the target filter based on the shape of the target filter; if the number of first type input items corresponding to the target filter is p, then determine p+k filter coefficients of the target filter, where p and k are both positive integers; and determine the third output value of the target filter based on i filter coefficients and i second type input items among the p+k filter coefficients and j filter coefficients and j third type input items among the p+k filter coefficients; wherein i and j are both positive integers, and k=i+j.

[0496] In some embodiments, there is a linear relationship between the number of first type input items and the reference sample value, there is a nonlinear relationship between the number of second type input items and the reference sample value, and the number of third type input items is preset bias information.

[0497] In some embodiments, the value of the second factor is a second preset constant.

[0498] In some embodiments, the first determination unit 2201 is further configured to determine a reconstructed value of at least one reference pixel in the reference area; perform mean calculation on the reconstructed value of at least one reference pixel to obtain a first mean; and set the value of the second factor to be equal to the first mean.

[0499] In some embodiments, the first prediction unit 2202 is further configured to perform a second process on the first output value to obtain a predicted value of a pixel to be predicted in the current block.

[0500] In some embodiments, the first prediction unit 2202 is further configured to set the predicted value of the to-be-predicted pixel in the current block to be equal to the first output value in the second processing.

[0501] In some embodiments, the first prediction unit 2202 is further configured so that the second processing is to limit the first output value to a preset numerical range; wherein the lower limit value of the preset numerical range is the minimum reconstructed value in the reference area, and the upper limit value of the preset numerical range is the maximum reconstructed value in the reference area.

[0502] In some embodiments, the first determination unit 2201 is further configured to determine the derived intra-frame prediction mode of the luminance component of the current block if the luminance component of the current block uses intra-frame prediction based on the filter coefficient; if the chrominance component of the current block uses intra-frame prediction of the direct mode, the direct mode is set to the derived intra-frame prediction mode to determine the predicted value of the chrominance component of the current block.

[0503] In some embodiments, the first determination unit 2201 is further configured to determine a reference block of the current block when the current block meets a preset condition; if the reference block uses intra-frame prediction based on the filter coefficient, determine the derived intra-frame prediction mode of the reference block; and add the derived intra-frame prediction mode to the intra-frame prediction mode candidate list of the current block.

[0504] In some embodiments, the current block satisfies a preset condition, including at least one of the following:

[0505] The current block is an inter-frame prediction block;

[0506] Current block IBC block.

[0507] In some embodiments, the first determining unit 2201 is further configured to determine an original value of the current block; and determine a residual value of the current block based on the original value of the current block and the predicted value of the current block;

[0508] The encoding unit 2203 is further configured to encode the residual value of the current block and write the obtained coded bits into the bitstream.

[0509] In some embodiments, the encoding unit 2203 is further configured to transform the residual value to obtain the transform coefficient of the current block; quantize the transform coefficient to obtain the quantization coefficient of the current block; and encode the quantization coefficient of the current block and write the obtained coded bits into the bitstream.

[0510] In some embodiments, the encoding unit 2203 is further configured to determine the target transform kernel of the current block when the current block uses a multi-transform selection mode and the target filtering mode is an interpolation filtering mode; and perform transform processing on the residual value according to the target transform kernel to obtain the transform coefficient of the current block.

[0511] In some embodiments, the determination of the target transform kernel is associated with at least one of the following parameters:

[0512] The target filtering mode of the current block;

[0513] The size parameters of the current block;

[0514] The shape of the current block.

[0515] In some embodiments, the first determination unit 2201 is further configured to determine at least one candidate transformation core; perform cost calculation on the at least one candidate transformation core to determine the cost result of the at least one candidate transformation core; and determine the minimum cost result from the cost results of the at least one candidate transformation core, and determine the candidate transformation core corresponding to the minimum cost result as the target transformation core of the current block.

[0516] In some embodiments, the first determining unit 2201 is further configured to determine non-zero coefficient information of the current block; and determine at least one candidate transform kernel according to the non-zero coefficient information of the current block.

[0517] In some embodiments, the number of the at least one candidate transform kernel is less than or equal to 6.

[0518] In some embodiments, the first determining unit 2201 is further configured to determine a transform core index value of the current block, wherein the transform core index value is used to indicate an index number of a target transform core in at least one candidate transform core;

[0519] The encoding unit 2203 is further configured to encode the transform kernel index value of the current block and write the obtained encoding bits into the bitstream.

[0520] In some embodiments, the first determining unit 2201 is further configured to determine a transform core index value of the current block, wherein the transform core index value is used to indicate an index number of a target transform core in at least one candidate transform core, and the at least one candidate transform core is associated with a size parameter of the current block;

[0521] The encoding unit 2203 is further configured to encode the transform kernel index value of the current block and write the obtained encoding bits into the bitstream.

[0522] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0523] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0524] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 220. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0525] Based on the composition of the encoder 220 and the computer-readable storage medium, refer to Figure 21, which shows a specific hardware structure diagram of the encoder 220 provided in an embodiment of the present application. As shown in Figure 21, the encoder 220 may include: a first communication interface 2301, a first memory 2302 and a first processor 2303; each component is coupled together through a first bus system 2304. It can be understood that the first bus system 2304 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 2304 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 2304 in Figure 21. Among them,

[0526] The first communication interface 2301 is used to receive and send signals when sending and receiving information with other external network elements;

[0527] A first memory 2302 is used to store computer programs that can be run on the first processor 2303;

[0528] The first processor 2303 is configured to, when running the computer program, execute:

[0529] Determine the target filtering mode for the current block;

[0530] Determine a reference area of ​​the current block according to a size parameter of the current block and a target filtering mode;

[0531] Determining a filter coefficient of the current block according to a reference area of ​​the current block;

[0532] Perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0533] It is understood that the first memory 2302 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 2302 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0534] The first processor 2303 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 2303. The above-mentioned first processor 2303 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented as a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 2302 , and the first processor 2303 reads the information in the first memory 2302 and completes the steps of the above method in combination with its hardware.

[0535] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0536] Optionally, as another embodiment, the first processor 2303 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0537] This embodiment provides an encoder. For this encoder, the intra-frame prediction technology based on interpolation filtering is not only related to the target filtering mode when determining the reference area for calculating the filter coefficient, but also related to the size parameters of the current block. For example, when the size of the current block is large, a large reference area can be used, and when the size of the current block is small, a small reference area can be used. In this way, the computational complexity and encoding time can be reduced; at the same time, the intra-frame prediction accuracy can be improved, thereby improving the encoding and decoding performance.

[0538] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, see FIG22 , which shows a schematic diagram of the structure of a decoder provided by an embodiment of the present application. As shown in FIG22 , the decoder 240 may include a decoding unit 2401, a second determination unit 2402, and a second prediction unit 2403, wherein:

[0539] The decoding unit 2401 is configured to decode the code stream and determine a target filtering mode for the current block;

[0540] The second determining unit 2402 is configured to determine a reference area of ​​the current block according to a size parameter of the current block and a target filtering mode;

[0541] The second prediction unit 2403 is configured to determine a filter coefficient of the current block according to a reference area of ​​the current block; and perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0542] In some embodiments, the target filtering mode includes a reference region category of the current block and a shape of a target filter.

[0543] In some embodiments, the second determination unit 2402 is further configured to, if the reference area category of the current block is the first category, determine that the reference area of ​​the current block includes the upper adjacent area and the left adjacent area; if the reference area category of the current block is the second category, determine that the reference area of ​​the current block includes the upper adjacent area; if the reference area category of the current block is the third category, determine that the reference area of ​​the current block includes the left adjacent area; wherein, the upper adjacent area refers to the reconstructed area adjacent to the upper side of the current block, and the left adjacent area refers to the reconstructed area adjacent to the left side of the current block.

[0544] In some embodiments, the size parameters of the current block include the height and width of the current block; the second determination unit 2402 is further configured to determine a minimum parameter from the height and width of the current block; and determine a reference area of ​​the current block based on the minimum parameter and the target filtering mode.

[0545] In some embodiments, the size of the reference area of ​​the current block is associated with the shape and minimum parameter of the target filter.

[0546] In some embodiments, the second determination unit 2402 is further configured to determine that the reference area category in the target prediction mode is any one item except the second category if the width of the current block and the multiple of the first factor are smaller than the height of the current block; and to determine that the reference area category in the target prediction mode is any one item except the third category if the height of the current block and the multiple of the first factor are smaller than the width of the current block.

[0547] In some embodiments, the value of the first factor is a first preset constant.

[0548] In some embodiments, the second determining unit 2402 is further configured to determine a context model of the current block;

[0549] The decoding unit 2401 is further configured to decode the code stream based on the context model and determine a target filtering mode for the current block.

[0550] In some embodiments, the determination of the context model is associated with at least one of the following parameters:

[0551] The shape of the current block;

[0552] The ratio of the width to the height of the current block.

[0553] In some embodiments, the second determination unit 2402 is further configured to determine the input value of the target filter corresponding to at least one reference pixel in the reference area and the output value of the target filter based on the reference area of ​​the current block and the shape of the target filter; determine the autocorrelation coefficient matrix based on the input value of the target filter corresponding to at least one reference pixel; determine the cross-correlation coefficient vector based on the input value of the target filter corresponding to at least one reference pixel and the output value of the target filter; determine the coefficients of the target filter based on the autocorrelation coefficient matrix and the cross-correlation coefficient vector; and determine the coefficients of the target filter as the filter coefficients of the current block.

[0554] In some embodiments, the second prediction unit 2403 is further configured to determine a reference sample value corresponding to the pixel to be predicted in the current block; and determine a predicted value of the pixel to be predicted in the current block based on the reference sample value and filter coefficient corresponding to the pixel to be predicted in the current block.

[0555] In some embodiments, the second determination unit 2402 is further configured to, based on the shape of the target filter, determine the reconstructed value at the corresponding position in the reference area as the reference sample value if the reference sample value is located in the reference area of ​​the current block; and determine the predicted value at the corresponding position in the current block as the reference sample value if the reference sample value is located inside the current block.

[0556] In some embodiments, the second prediction unit 2403 is further configured to determine a first input value of the target filter based on a reference sample value corresponding to the pixel to be predicted in the current block; determine a first output value of the target filter based on the first input value and the filter coefficient; and determine a predicted value of the pixel to be predicted in the current block based on the first output value.

[0557] In some embodiments, the second determining unit 2402 is further configured to determine a second factor; and perform a subtraction operation on the reference sample value and the second factor to obtain a first input value of the target filter.

[0558] In some embodiments, the second determining unit 2402 is further configured to determine a second output value of the target filter based on the first input value and the filter coefficient; and perform a first process on the second output value to determine a first output value of the target filter.

[0559] In some embodiments, the second determination unit 2402 is further configured to calculate the product of the first input value and the corresponding filter coefficient; and set the second output value of the target filter to be equal to the sum of n products; wherein n represents the number of input items corresponding to the target filter, and n is a positive integer.

[0560] In some embodiments, the second determining unit 2402 is further configured to perform an addition operation on the second output value and the second factor to obtain a first output value of the target filter.

[0561] In some embodiments, the second determination unit 2402 is further configured to determine the third output value of the target filter; determine the fourth output value of the target filter based on the second output value and the third output value; and perform an addition operation on the fourth output value and the second factor to obtain the first output value of the target filter.

[0562] In some embodiments, the second determination unit 2402 is further configured to determine the number of first type input items corresponding to the target filter based on the shape of the target filter; if the number of first type input items corresponding to the target filter is p, then determine p+q filter coefficients of the target filter, where p and q are both positive integers; and determine the third output value of the target filter based on q filter coefficients and q second type input items among the p+q filter coefficients.

[0563] In some embodiments, the second determination unit 2402 is further configured to determine the number of first type input items corresponding to the target filter based on the shape of the target filter; if the number of first type input items corresponding to the target filter is p, then determine p+m filter coefficients of the target filter, where p and m are both positive integers; and determine the third output value of the target filter based on m filter coefficients and m third type input items among the p+m filter coefficients.

[0564] In some embodiments, the second determination unit 2402 is further configured to determine the number of first type input items corresponding to the target filter based on the shape of the target filter; if the number of first type input items corresponding to the target filter is p, then determine p+k filter coefficients of the target filter, where p and k are both positive integers; and determine the third output value of the target filter based on i filter coefficients and i second type input items among the p+k filter coefficients and j filter coefficients and j third type input items among the p+k filter coefficients; wherein i and j are both positive integers, and k=i+j.

[0565] In some embodiments, there is a linear relationship between the number of first type input items and the reference sample value, there is a nonlinear relationship between the number of second type input items and the reference sample value, and the number of third type input items is preset bias information.

[0566] In some embodiments, the value of the second factor is a second preset constant.

[0567] In some embodiments, the second determination unit 2402 is further configured to determine a reconstructed value of at least one reference pixel in the reference area; perform mean calculation on the reconstructed value of the at least one reference pixel to obtain a first mean; and set the value of the second factor to be equal to the first mean.

[0568] In some embodiments, the second prediction unit 2403 is further configured to perform a second process on the first output value to obtain a predicted value of a pixel to be predicted in the current block.

[0569] In some embodiments, the second prediction unit 2403 is further configured to set the predicted value of the to-be-predicted pixel in the current block to be equal to the first output value in the second processing.

[0570] In some embodiments, the second prediction unit 2403 is further configured so that the second processing is to limit the first output value to a preset numerical range; wherein the lower limit value of the preset numerical range is the minimum reconstructed value in the reference area, and the upper limit value of the preset numerical range is the maximum reconstructed value in the reference area.

[0571] In some embodiments, the second determination unit 2402 is further configured to determine the derived intra-frame prediction mode of the luminance component of the current block if the luminance component of the current block uses intra-frame prediction based on the filter coefficient; if the chrominance component of the current block uses intra-frame prediction of the direct mode, the direct mode is set to the derived intra-frame prediction mode to determine the predicted value of the chrominance component of the current block.

[0572] In some embodiments, the second determination unit 2402 is further configured to determine a reference block of the current block when the current block meets a preset condition; if the reference block uses intra-frame prediction based on the filter coefficient, determine the derived intra-frame prediction mode of the reference block; and add the derived intra-frame prediction mode to the intra-frame prediction mode candidate list of the current block.

[0573] In some embodiments, the current block satisfies a preset condition, including at least one of the following:

[0574] The current block is an inter-frame prediction block;

[0575] The current block is an intra block copy (IBC) block.

[0576] In some embodiments, the decoding unit 2401 is further configured to decode the code stream and determine a residual value of the current block;

[0577] The second determining unit 2402 is further configured to determine a reconstructed value of the current block according to the predicted value of the current block and the residual value of the current block.

[0578] In some embodiments, the decoding unit 2401 is further configured to decode the code stream to determine the quantization coefficient of the current block; perform inverse quantization processing on the quantization coefficient to obtain the transformation coefficient of the current block; and perform inverse transformation processing on the transformation coefficient to obtain the residual value of the current block.

[0579] In some embodiments, the decoding unit 2401 is further configured to determine the target transform kernel of the current block when the current block uses a multi-transform selection mode and the target filtering mode is an interpolation filtering mode; and perform inverse transform processing on the transform coefficients according to the target transform kernel to obtain the residual value of the current block.

[0580] In some embodiments, the determination of the target transform kernel is associated with at least one of the following parameters:

[0581] The target filtering mode of the current block;

[0582] The size parameters of the current block;

[0583] The shape of the current block.

[0584] In some embodiments, the decoding unit 2401 is further configured to decode the code stream and determine a transform kernel index value of the current block;

[0585] The second determining unit 2402 is further configured to determine a target transform core for the current block from at least one candidate transform core according to the transform core index value.

[0586] In some embodiments, the decoding unit 2401 is further configured to decode the code stream and determine a transform kernel index value of the current block;

[0587] The second determining unit 2402 is further configured to determine a target transform core of the current block from at least one candidate transform core according to the transform core index value and the size parameter of the current block.

[0588] In some embodiments, the second determining unit 2402 is further configured to decode the code stream and determine non-zero coefficient information of the current block;

[0589] The second determining unit 2402 is further configured to determine at least one candidate transform kernel according to the non-zero coefficient information of the current block.

[0590] In some embodiments, the number of the at least one candidate transform kernel is less than or equal to 6.

[0591] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0592] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium, which is applied to the decoder 240 and stores a computer program. When the computer program is executed by the second processor, it implements any of the methods in the aforementioned embodiments.

[0593] Based on the composition of the decoder 240 and the computer-readable storage medium, refer to Figure 23, which shows a specific hardware structure diagram of the decoder 240 provided in an embodiment of the present application. As shown in Figure 23, the decoder 240 may include: a second communication interface 2501, a second memory 2502 and a second processor 2503; each component is coupled together through a second bus system 2504. It can be understood that the second bus system 2504 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 2504 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 2504 in Figure 23. Among them,

[0594] The second communication interface 2501 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0595] The second memory 2502 is used to store computer programs that can be run on the second processor 2503;

[0596] The second processor 2503 is configured to, when running the computer program, execute:

[0597] Decode the code stream and determine the target filtering mode of the current block;

[0598] Determine a reference area of ​​the current block according to a size parameter of the current block and a target filtering mode;

[0599] Determining a filter coefficient of the current block according to a reference area of ​​the current block;

[0600] Perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

[0601] Optionally, as another embodiment, the second processor 2503 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0602] It can be understood that the hardware functions of the second memory 2502 are similar to those of the first memory 2302, and the hardware functions of the second processor 2503 are similar to those of the first processor 2303; they will not be described in detail here.

[0603] This embodiment provides a decoder. For this decoder, the intra-frame prediction technology based on interpolation filtering is not only related to the target filtering mode when determining the reference area for calculating the filter coefficient, but also related to the size parameters of the current block. For example, when the size of the current block is large, a large reference area can be used, and when the size of the current block is small, a small reference area can be used. In this way, the computational complexity can be reduced, and the accuracy of intra-frame prediction can be improved, thereby improving the encoding and decoding performance.

[0604] In yet another embodiment of the present application, referring to FIG24 , a schematic diagram of the structure of a coding and decoding system provided by an embodiment of the present application is shown. As shown in FIG24 , the coding and decoding system 260 may include an encoder 2601 and a decoder 2602 .

[0605] In the embodiment of the present application, the encoder 2601 may be the encoder described in any one of the aforementioned embodiments, and the decoder 2602 may be the decoder described in any one of the aforementioned embodiments.

[0606] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0607] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0608] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0609] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0610] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0611] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0612] In an embodiment of the present application, whether at the encoding end or the decoding end, after determining the target filtering mode of the current block, the reference area of ​​the current block is determined according to the size parameters of the current block and the target filtering mode; then, based on the reference area of ​​the current block, the filter coefficient of the current block is determined; and then, based on the filter coefficient, intra-frame prediction is performed on the current block to determine the predicted value of the current block. In this way, the intra-frame prediction technology based on interpolation filtering, when determining the reference area for calculating the filter coefficient, is not only related to the target filtering mode, but also to the size parameters of the current block. For example, a large reference area can be used when the size of the current block is large, and a small reference area can be used when the size of the current block is small. In this way, while ensuring the encoding and decoding performance, the computational complexity can be reduced, the encoding time can be reduced, and the cost-effectiveness of the encoding and decoding performance and encoding complexity can be improved. At the same time, the accuracy of intra-frame prediction can be improved, thereby improving the encoding and decoding efficiency.

Claims

1. A decoding method, applied to a decoder, the method comprising: Decode the bitstream and determine the target filtering mode of the current block; Determining a reference area of ​​the current block according to a size parameter of the current block and the target filtering mode; Determining a filter coefficient of the current block according to a reference area of ​​the current block; Perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

2. The method according to claim 1, wherein: The target filtering mode includes a reference region category of the current block and a shape of a target filter.

3. The method according to claim 2, wherein: The method further comprises: If the reference area category of the current block is the first category, determining that the reference area of ​​the current block includes an upper adjacent area and a left adjacent area; If the reference area category of the current block is the second category, determining that the reference area of ​​the current block includes an upper adjacent area; If the reference area category of the current block is the third category, determining that the reference area of ​​the current block includes a left adjacent area; The upper adjacent region refers to a reconstructed region adjacent to the upper side of the current block, and the left adjacent region refers to a reconstructed region adjacent to the left side of the current block.

4. The method according to claim 3, wherein: The size parameter of the current block includes a height and a width of the current block; and determining a reference area of ​​the current block based on the size parameter of the current block and the target filtering mode includes: Determining a minimum parameter from the height and width of the current block; A reference area of ​​the current block is determined according to the minimum parameter and the target filtering mode.

5. The method according to claim 4, wherein: The size of the reference area of ​​the current block is associated with the shape of the target filter and the minimum parameter.

6. The method according to claim 3, wherein: The method further comprises: If the multiple of the width of the current block and the first factor is smaller than the height of the current block, determining that the reference area category in the target prediction mode is any one item except the second category; If the multiple of the height of the current block and the first factor is smaller than the width of the current block, the reference area category in the target prediction mode is determined to be any one item except the third category.

7. The method according to claim 6, wherein: The value of the first factor is a first preset constant.

8. The method according to claim 1, wherein: The decoding bitstream determines the target filtering mode of the current block, including: Determining a context model for the current block; The target filtering mode of the current block is determined based on the context model decoding bitstream.

9. The method according to claim 8, wherein: The determination of the context model is associated with at least one of the following parameters: The shape of the current block; The ratio of the width to the height of the current block.

10. The method according to claim 2, wherein: The step of determining the filter coefficient of the current block according to the reference area of ​​the current block comprises: Determine, according to the reference area of ​​the current block and the shape of the target filter, an input value of the target filter corresponding to at least one reference pixel in the reference area and an output value of the target filter; Determining an autocorrelation coefficient matrix according to an input value of the target filter corresponding to the at least one reference pixel; Determine a cross-correlation coefficient vector according to an input value of the target filter and an output value of the target filter corresponding to the at least one reference pixel; Determining the coefficients of the target filter according to the autocorrelation coefficient matrix and the cross-correlation coefficient vector; The coefficients of the target filter are determined as the filter coefficients of the current block.

11. The method according to any one of claims 2 to 10, wherein: The performing intra-frame prediction on the pixel points in the current block according to the filter coefficient to determine the predicted value of the pixel points in the current block includes: Determine a reference sample value corresponding to a pixel to be predicted in the current block; Determine a prediction value of the pixel to be predicted in the current block according to the reference sample value corresponding to the pixel to be predicted in the current block and the filter coefficient.

12. The method according to claim 11, wherein: The determining of the reference sample value corresponding to the pixel to be predicted in the current block includes: Based on the shape of the target filter, if the reference sample value is located in a reference area of ​​the current block, determining a reconstructed value at a corresponding position in the reference area as the reference sample value; If the reference sample value is located inside the current block, the prediction value at the corresponding position in the current block is determined as the reference sample value.

13. The method according to claim 11, wherein: The step of determining the predicted value of the pixel to be predicted in the current block according to the reference sample value corresponding to the pixel to be predicted in the current block and the filter coefficient comprises: Determining a first input value of the target filter based on a reference sample value corresponding to a pixel to be predicted in the current block; Determining a first output value of the target filter based on the first input value and the filter coefficient; Determine a prediction value of a pixel to be predicted in the current block according to the first output value.

14. The method according to claim 13, wherein: The determining the first input value of the target filter based on the reference sample value corresponding to the pixel to be predicted in the current block includes: Determine the second factor; A subtraction operation is performed on the reference sample value and the second factor to obtain a first input value of the target filter.

15. The method according to claim 14, wherein: The step of determining a first output value of the target filter based on the first input value and the filter coefficient comprises: Determining a second output value of the target filter based on the first input value and the filter coefficient; The second output value is subjected to a first processing to determine a first output value of the target filter.

16. The method according to claim 15, wherein: The determining, based on the first input value and the filter coefficient, a second output value of the target filter comprises: Calculating the product of the first input value and the corresponding filter coefficient; The second output value of the target filter is set to be equal to the sum of n products; wherein n represents the number of input items corresponding to the target filter, and n is a positive integer.

17. The method according to claim 15, wherein: The performing a first processing on the second output value to determine a first output value of the target filter includes: An addition operation is performed on the second output value and the second factor to obtain a first output value of the target filter.

18. The method according to claim 15, wherein: The performing a first processing on the second output value to determine a first output value of the target filter includes: determining a third output value of the target filter; Determining a fourth output value of the target filter according to the second output value and the third output value; An addition operation is performed on the fourth output value and the second factor to obtain a first output value of the target filter.

19. The method according to claim 18, wherein: The determining of the third output value of the target filter comprises: Based on the shape of the target filter, determining the number of first type input items corresponding to the target filter; If the number of first-type input items corresponding to the target filter is p, then p+q filter coefficients of the target filter are determined, where p and q are both positive integers; A third output value of the target filter is determined according to q filter coefficients among the p+q filter coefficients and q second type input items.

20. The method according to claim 18, wherein: The determining of the third output value of the target filter comprises: Based on the shape of the target filter, determining the number of first type input items corresponding to the target filter; If the number of first-type input items corresponding to the target filter is p, then p+m filter coefficients of the target filter are determined, where p and m are both positive integers; The third output value of the target filter is determined according to m filter coefficients among the p+m filter coefficients and m third-type input items.

21. The method according to claim 18, wherein: The determining of the third output value of the target filter comprises: Based on the shape of the target filter, determining the number of first type input items corresponding to the target filter; If the number of first-type input items corresponding to the target filter is p, then p+k filter coefficients of the target filter are determined, where p and k are both positive integers; The third output value of the target filter is determined based on i filter coefficients and i second type input items among the p+k filter coefficients and j filter coefficients and j third type input items among the p+k filter coefficients; wherein i and j are both positive integers and k=i+j.

22. The method according to claim 21, wherein: There is a linear relationship between the number of the first type input items and the reference sample value, there is a non-linear relationship between the number of the second type input items and the reference sample value, and the number of the third type input items is preset bias information.

23. The method of claim 14, 17 or 18, wherein: The value of the second factor is a second preset constant.

24. The method of claim 14, 17 or 18, wherein: The determining of the second factor comprises: determining a reconstructed value of at least one reference pixel in the reference area; Performing mean calculation on the reconstructed value of the at least one reference pixel to obtain a first mean value; The value of the second factor is set equal to the first mean.

25. The method of claim 13, wherein: The step of determining the predicted value of the pixel to be predicted in the current block according to the first output value includes: The first output value is subjected to a second process to obtain a predicted value of a pixel to be predicted in the current block.

26. The method according to claim 25, wherein: The method further comprises: The second process is to set the prediction value of the to-be-predicted pixel in the current block to be equal to the first output value.

27. The method according to claim 25, wherein: The method further comprises: The second processing is to limit the first output value to a preset value range; The lower limit value of the preset numerical range is the minimum reconstruction value in the reference area, and the upper limit value of the preset numerical range is the maximum reconstruction value in the reference area.

28. The method according to any one of claims 1 to 27, wherein: The method further comprises: If the luminance component of the current block uses intra prediction based on the filter coefficient, determining a derived intra prediction mode of the luminance component of the current block; If the chrominance component of the current block uses direct mode intra prediction, the direct mode is set as the derived intra prediction mode to determine the prediction value of the chrominance component of the current block.

29. The method according to any one of claims 1 to 27, wherein: The method further comprises: When the current block meets a preset condition, determining a reference block of the current block; If the reference block uses intra prediction based on the filter coefficient, determining a derived intra prediction mode of the reference block; The derived intra prediction mode is added to an intra prediction mode candidate list of the current block.

30. The method of claim 29, wherein: The current block meets the preset conditions, including at least one of the following: The current block is an inter-frame prediction block; The current block is an intra block copy IBC block.

31. The method according to any one of claims 1 to 30, wherein: The method further comprises: Decoding a bitstream to determine a residual value of the current block; A reconstructed value of the current block is determined according to the predicted value of the current block and the residual value of the current block.

32. The method according to claim 31, wherein: The decoding bitstream determines the residual value of the current block, including: Decoding a bitstream to determine a quantization coefficient of the current block; Performing inverse quantization processing on the quantized coefficients to obtain transform coefficients of the current block; An inverse transform is performed on the transform coefficient to obtain a residual value of the current block.

33. The method of claim 32, wherein: The performing inverse transformation processing on the transformation coefficient to obtain the residual value of the current block includes: When the current block uses a multi-transform selection mode and the target filtering mode is an interpolation filtering mode, determining a target transform kernel of the current block; The transform coefficients are inversely transformed according to the target transform kernel to obtain a residual value of the current block.

34. The method of claim 33, wherein: The determination of the target transformation kernel is associated with at least one of the following parameters: The target filtering mode of the current block; The size parameter of the current block; The shape of the current block.

35. The method of claim 33, wherein: The determining of the target transformation kernel of the current block comprises: Decoding a bitstream to determine a transform kernel index value of the current block; According to the transform core index value, a target transform core of the current block is determined from at least one candidate transform core.

36. The method of claim 33, wherein: The determining of the target transformation kernel of the current block comprises: Decoding a bitstream to determine a transform kernel index value of the current block; A target transform core of the current block is determined from at least one candidate transform core according to the transform core index value and a size parameter of the current block.

37. The method according to claim 35 or 36, wherein: The method further comprises: Decoding the bitstream to determine non-zero coefficient information of the current block; The at least one candidate transform kernel is determined according to the non-zero coefficient information of the current block.

38. The method of claim 37, wherein: The number of the at least one candidate transformation kernel is less than or equal to 6.

39. A coding method, applied to an encoder, the method comprising: Determine a target filtering mode for the current block; Determining a reference area of ​​the current block according to a size parameter of the current block and the target filtering mode; Determining a filter coefficient of the current block according to a reference area of ​​the current block; Perform intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

40. The method of claim 39, wherein: The determining of the target filtering mode of the current block includes: determining at least one candidate filtering mode; Performing cost calculation on the at least one candidate filtering mode to determine a cost result of the at least one candidate filtering mode; A minimum cost result is determined from the cost results of the at least one candidate filtering mode, and the candidate filtering mode corresponding to the minimum cost result is determined as the target filtering mode of the current block.

41. The method of claim 40, wherein: The number of the at least one candidate filtering mode is determined based on the number of reference region categories of the current block and the number of shapes of the target filter.

42. The method of claim 40, wherein: The target filtering mode includes a reference region category of the current block and a shape of a target filter.

43. The method of claim 42, wherein: The method further comprises: If the reference area category of the current block is the first category, determining that the reference area of ​​the current block includes an upper adjacent area and a left adjacent area; If the reference area category of the current block is the second category, determining that the reference area of ​​the current block includes an upper adjacent area; If the reference area category of the current block is the third category, determining that the reference area of ​​the current block includes a left adjacent area; The upper adjacent region refers to a reconstructed region adjacent to the upper side of the current block, and the left adjacent region refers to a reconstructed region adjacent to the left side of the current block.

44. The method of claim 43, wherein: The size parameter of the current block includes a height and a width of the current block; and determining a reference area of ​​the current block based on the size parameter of the current block and the target filtering mode includes: Determining a minimum parameter from the height and width of the current block; A reference area of ​​the current block is determined according to the minimum parameter and the target filtering mode.

45. The method of claim 44, wherein: The size of the reference area of ​​the current block is associated with the shape of the target filter and the minimum parameter.

46. ​​The method of claim 43, wherein: The method further comprises: If the multiple of the width of the current block and the first factor is smaller than the height of the current block, prohibiting the reference area category of the current block from being the second category, and determining that the number of reference area categories of the current block is determined based on other reference area categories except the second category; If the height of the current block and the multiple of the first factor are smaller than the width of the current block, the reference area category of the current block is prohibited from being the third category, and the number of reference area categories of the current block is determined based on other reference area categories except the third category.

47. The method of claim 46, wherein: The value of the first factor is a first preset constant.

48. The method of claim 39, wherein: The method further comprises: The target filtering mode of the current block is encoded, and the obtained encoding bits are written into a bitstream.

49. The method of claim 48, wherein: The encoding of the target filtering mode of the current block and writing the obtained encoding bits into a bitstream comprises: Determining a context model for the current block; The target filtering mode of the current block is encoded based on the context model, and the obtained encoding bits are written into a bitstream.

50. The method of claim 49, wherein: The determination of the context model is associated with at least one of the following parameters: The shape of the current block; The ratio of the width to the height of the current block.

51. The method of claim 42, wherein: The method further comprises: Determine a plurality of candidate sub-reference regions of the current block, wherein the plurality of candidate sub-reference regions do not overlap with each other; Determining, according to the multiple candidate sub-reference regions and the shapes of the target filter, respective autocorrelation coefficient matrices and mutual correlation coefficient vectors of the multiple candidate sub-reference regions; The autocorrelation coefficient matrices and the mutual correlation coefficient vectors of the plurality of candidate sub-reference regions are stored in a preset buffer area.

52. The method of claim 51, wherein: The step of determining, according to the plurality of candidate sub-reference regions and the shape of the target filter, the autocorrelation coefficient matrix and the mutual correlation coefficient vector of each of the plurality of candidate sub-reference regions comprises: Determine, according to the first candidate sub-reference area and the shape of the target filter, an input value of the target filter and an output value of the target filter corresponding to at least one reference pixel in the first candidate sub-reference area; Determining an autocorrelation coefficient matrix of the first candidate sub-reference region according to an input value of the target filter corresponding to the at least one reference pixel; Determine a cross-correlation coefficient vector of the first candidate sub-reference region according to an input value of the target filter and an output value of the target filter corresponding to the at least one reference pixel; The first candidate sub-reference region is any one of the multiple candidate sub-reference regions.

53. The method of claim 51, wherein: The step of determining the filter coefficient of the current block according to the reference area of ​​the current block comprises: Dividing the reference area of ​​the current block to determine at least one sub-reference area; Acquire the autocorrelation coefficient matrix and the mutual correlation coefficient vector of each of the at least one sub-reference regions from the preset buffer area; Determining coefficients of the target filter according to the autocorrelation coefficient matrix and the cross-correlation coefficient vector of each of the at least one sub-reference regions; The coefficients of the target filter are determined as the filter coefficients of the current block.

54. The method according to any one of claims 42 to 53, wherein: The performing intra-frame prediction on the pixel points in the current block according to the filter coefficient to determine the predicted value of the pixel points in the current block includes: Determine a reference sample value corresponding to a pixel to be predicted in the current block; Determine a prediction value of the pixel to be predicted in the current block according to the reference sample value corresponding to the pixel to be predicted in the current block and the filter coefficient.

55. The method of claim 54, wherein: The determining of the reference sample value corresponding to the pixel to be predicted in the current block includes: Based on the shape of the target filter, if the reference sample value is located in a reference area of ​​the current block, determining a reconstructed value at a corresponding position in the reference area as the reference sample value; If the reference sample value is located inside the current block, the prediction value at the corresponding position in the current block is determined as the reference sample value.

56. The method of claim 54, wherein: The step of determining the predicted value of the pixel to be predicted in the current block according to the reference sample value corresponding to the pixel to be predicted in the current block and the filter coefficient comprises: Determining a first input value of the target filter based on a reference sample value corresponding to a pixel to be predicted in the current block; Determining a first output value of the target filter based on the first input value and the filter coefficient; Determine a prediction value of a pixel to be predicted in the current block according to the first output value.

57. The method of claim 56, wherein: The determining the first input value of the target filter based on the reference sample value corresponding to the pixel to be predicted in the current block includes: Determine the second factor; A subtraction operation is performed on the reference sample value and the second factor to obtain a first input value of the target filter.

58. The method of claim 57, wherein: The step of determining a first output value of the target filter based on the first input value and the filter coefficient comprises: Determining a second output value of the target filter based on the first input value and the filter coefficient; The second output value is subjected to a first processing to determine a first output value of the target filter.

59. The method of claim 58, wherein: The determining, based on the first input value and the filter coefficient, a second output value of the target filter comprises: Calculating the product of the first input value and the corresponding filter coefficient; The second output value of the target filter is set to be equal to the sum of n products; wherein n represents the number of input items corresponding to the target filter, and n is a positive integer.

60. The method of claim 58, wherein: The performing a first processing on the second output value to determine a first output value of the target filter includes: An addition operation is performed on the second output value and the second factor to obtain a first output value of the target filter.

61. The method of claim 58, wherein: The performing a first processing on the second output value to determine a first output value of the target filter includes: determining a third output value of the target filter; Determining a fourth output value of the target filter according to the second output value and the third output value; An addition operation is performed on the fourth output value and the second factor to obtain a first output value of the target filter.

62. The method of claim 61, wherein: The determining of the third output value of the target filter comprises: Based on the shape of the target filter, determining the number of first type input items corresponding to the target filter; If the number of first-type input items corresponding to the target filter is p, then p+q filter coefficients of the target filter are determined, where p and q are both positive integers; A third output value of the target filter is determined according to q filter coefficients among the p+q filter coefficients and q second type input items.

63. The method of claim 61, wherein: The determining of the third output value of the target filter comprises: Based on the shape of the target filter, determining the number of first type input items corresponding to the target filter; If the number of first-type input items corresponding to the target filter is p, then p+m filter coefficients of the target filter are determined, where p and m are both positive integers; The third output value of the target filter is determined according to m filter coefficients among the p+m filter coefficients and m third-type input items.

64. The method of claim 61, wherein: The determining of the third output value of the target filter comprises: Based on the shape of the target filter, determining the number of first type input items corresponding to the target filter; If the number of first-type input items corresponding to the target filter is p, then p+k filter coefficients of the target filter are determined, where p and k are both positive integers; The third output value of the target filter is determined based on i filter coefficients and i second type input items among the p+k filter coefficients and j filter coefficients and j third type input items among the p+k filter coefficients; wherein i and j are both positive integers and k=i+j.

65. The method of claim 64, wherein: There is a linear relationship between the number of the first type input items and the reference sample value, there is a non-linear relationship between the number of the second type input items and the reference sample value, and the number of the third type input items is preset bias information.

66. The method of claim 57, 60 or 61, wherein: The value of the second factor is a second preset constant.

67. The method of claim 57, 60 or 61, wherein: The determining of the second factor comprises: determining a reconstructed value of at least one reference pixel in the reference area; Performing mean calculation on the reconstructed value of the at least one reference pixel to obtain a first mean value; The value of the second factor is set equal to the first mean.

68. The method of claim 56, wherein: The step of determining the predicted value of the pixel to be predicted in the current block according to the first output value includes: The first output value is subjected to a second process to obtain a predicted value of a pixel to be predicted in the current block.

69. The method of claim 68, wherein: The method further comprises: The second process is to set the prediction value of the to-be-predicted pixel in the current block to be equal to the first output value.

70. The method of claim 68, wherein: The method further comprises: The second processing is to limit the first output value to a preset value range; The lower limit value of the preset numerical range is the minimum reconstruction value in the reference area, and the upper limit value of the preset numerical range is the maximum reconstruction value in the reference area.

71. The method according to any one of claims 39 to 70, wherein: The method further comprises: If the luminance component of the current block uses intra prediction based on the filter coefficient, determining a derived intra prediction mode of the luminance component of the current block; If the chrominance component of the current block uses direct mode intra prediction, the direct mode is set as the derived intra prediction mode to determine the prediction value of the chrominance component of the current block.

72. The method of any one of claims 39 to 70, wherein: The method further comprises: When the current block meets a preset condition, determining a reference block of the current block; If the reference block uses intra prediction based on the filter coefficient, determining a derived intra prediction mode of the reference block; The derived intra prediction mode is added to an intra prediction mode candidate list of the current block.

73. The method of claim 72, wherein: The current block meets the preset conditions, including at least one of the following: The current block is an inter-frame prediction block; The current block is an intra block copy IBC block.

74. The method of any one of claims 39 to 73, wherein: The method further comprises: Determining an original value of the current block; Determining a residual value of the current block according to an original value of the current block and a predicted value of the current block; The residual value of the current block is encoded, and the obtained encoded bits are written into a bit stream.

75. The method of claim 74, wherein: The step of encoding the residual value of the current block and writing the obtained coded bits into a bitstream includes: Performing transformation processing on the residual value to obtain a transformation coefficient of the current block; quantizing the transform coefficients to obtain quantized coefficients of the current block; The quantized coefficients of the current block are encoded, and the obtained encoded bits are written into a bit stream.

76. The method of claim 75, wherein: The performing transformation processing on the residual value to obtain the transformation coefficient of the current block includes: When the current block uses a multi-transform selection mode and the target filtering mode is an interpolation filtering mode, determining a target transform kernel of the current block; The residual value is transformed according to the target transformation kernel to obtain the transformation coefficient of the current block.

77. The method of claim 76, wherein: The determination of the target transformation kernel is associated with at least one of the following parameters: The target filtering mode of the current block; The size parameter of the current block; The shape of the current block.

78. The method of claim 76, wherein: The determining of the target transformation kernel of the current block comprises: determining at least one candidate transformation kernel; Performing cost calculation on the at least one candidate transformation core to determine a cost result of the at least one candidate transformation core; A minimum cost result is determined from the cost results of the at least one candidate transformation core, and the candidate transformation core corresponding to the minimum cost result is determined as the target transformation core of the current block.

79. The method of claim 78, wherein: The method further comprises: Determining non-zero coefficient information of the current block; The at least one candidate transform kernel is determined according to the non-zero coefficient information of the current block.

80. The method of claim 79, wherein: The number of the at least one candidate transformation kernel is less than or equal to 6.

81. The method of claim 78, wherein: The method further comprises: Determine a transform core index value of the current block, wherein the transform core index value is used to indicate an index number of the target transform core in the at least one candidate transform core; The transform core index value of the current block is encoded, and the obtained encoding bits are written into a bitstream.

82. The method of claim 78, wherein: The method further comprises: Determine a transform core index value of the current block, wherein the transform core index value is used to indicate an index number of the target transform core in the at least one candidate transform core, and the at least one candidate transform core is associated with a size parameter of the current block; The transform core index value of the current block is encoded, and the obtained encoding bits are written into a bitstream.

83. A code stream, the code stream is generated by bit encoding according to information to be encoded; wherein, The information to be encoded includes at least one of the following: A target filtering mode of a current block, a residual value of the current block, and a transform kernel index value of the current block.

84. An encoder, comprising a first determining unit and a first predicting unit, wherein: The first determining unit is configured to determine a target filtering mode of the current block; and determine a reference area of ​​the current block according to a size parameter of the current block and the target filtering mode; The first prediction unit is configured to determine a filter coefficient of the current block according to a reference area of ​​the current block; And performing intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

85. An encoder, comprising a first memory and a first processor, wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 39 to 82 when running the computer program.

86. A decoder, comprising a decoding unit, a second determining unit and a second predicting unit, wherein: The decoding unit is configured to decode the bitstream and determine a target filtering mode for a current block; The second determining unit is configured to determine a reference area of ​​the current block according to a size parameter of the current block and the target filtering mode; The second prediction unit is configured to determine a filter coefficient of the current block according to a reference area of ​​the current block; And performing intra-frame prediction on the current block according to the filter coefficient to determine a prediction value of the current block.

87. A decoder, comprising a second memory and a second processor, wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 38 when running the computer program.

88. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 38 is implemented, or the method according to any one of claims 39 to 82 is implemented.

Citation Information

Patent Citations

  • Filtering method and apparatus for improving prediction in image coding system

    CN108293111A

  • Image encoding / decoding method and device, and recording medium storing bitstream

    CN111247796A

  • Image encoding / decoding method and apparatus using sample filtering

    CN111837388A

  • Intra-frame prediction-based video coding method and device

    CN112425161A

  • Method of decoding moving pictures in intra prediction

    KR1020130105114A

Cited By

  • Parameter debugging method and system for identification code decoding

    CN120996060A